By Ariel Kaminer
Looking out over the quadrangle before him as students dashed from one class to the next, James Muyskens was feeling proud one recent afternoon, and why not?
The college he had led for the past 11 years had just been awarded second place in a new ranking of American higher education — ahead of flagship state universities, ahead of elite liberal arts colleges, even ahead of all eight Ivy League universities.
The college is Queens College, a part of the City University of New York with an annual tuition of $5,730, and a view of the Long Island Expressway.
Catering to working-class students, more than half of whom were born in other countries, Queens does not typically find itself at the top of national rankings. Then again, this was not a typical ranking. It was a list of colleges that offer the “best bang for the buck.”
“Elation,” said Muyskens, recalling his delight when he learned of the honor. “Thrilled!”
Purists might regard such bottom-line calculations as an insult to the intellectual, social and civic value of education. But dollars-and-cents tabulations like that one (which was compiled by Washington Monthly), are the fastest-growing sector of the college rankings industry, with ever more analyses vying for the attention of high school students and their parents who are anxious about finances.
(UC Davis ranked 46th on Washington Monthly’s value scale.)
President Obama sharply raised the ante in August with a plan to rate colleges on their value and affordability and to tie those ratings to the $150 billion in financial aid that the federal government supplies each year. Should Obama’s plan come to pass, value would not just be a selling point for colleges, it would be a matter of life and death.
But there is no agreement on how to measure the value of a college, and there is no agreement, or anything even close, on what value is in the first place.
“It’s a quest for the Holy Grail,” said Judith Scott-Clayton, a professor of economics and education at Columbia University’s Teachers College. “It sounds good, it sounds like something we’d love to know, to be able to rank the value of these institutions, but when it comes down to practicalities, it’s very, very difficult.”
U.S. News and World Report, whose academic rankings have long been derided — and obsessively followed — by college presidents, now publishes “best value” lists as well.
Princeton Review, which has advised decades of prospective students on the best party schools, more recently began listing the best value schools, too.
Forbes Magazine got in the is-it-worth-the-money game too, as did, among others, The Wall Street Journal, The Alumni Factor, Kiplinger’s Personal Finance and Payscale, a company that gathers data about the job market.
Some of these analyses approach value as largely a function of cost: How much is tuition? What subsidies are available? Others define it as return on investment: How much do graduates earn? Some factor in student satisfaction or academic ranking or graduation rates or economic diversity, all in varying quantities.
These widely divergent definitions produce wildly divergent results. Queens College did splendidly in a list that emphasizes social mobility and civic virtue. Another New York City public college, Baruch, took third place; No. 1 was Amherst. (UC Davis ranked 46th.)
But on a ranking that emphasizes alumni salaries, like Payscale’s list of Colleges Worth Your Investment, Queens comes in 341st. The top spot there goes to Harvey Mudd College, in Claremont, which has a much higher tuition, but whose graduates disproportionately enter lucrative fields like science and engineering. The top tier of U.S. News’ list somehow features Harvard, Princeton, Williams — and Soka University of America, a tiny Buddhist college in Southern California that admitted its first undergraduates in 2001.
Forbes’ most recent list starts out with the five national service academies, which charge no tuition. And when the Education Trust, an advocacy group, set out to list the colleges that do right by low-income students, it found only five entries, including Queens.
As for the federal government, no one yet knows how it will perform its evaluation. Arne Duncan, the education secretary, has said that his goal is a ratings system, not a single first-place-to-last-place ranking, and that the ratings will compare only schools that are similar in their mission, their student population and so on. Harvard and Yale, that is; not Harvard and Soka University of America.
“We’ll be looking at access,” Duncan said in a recent speech, “such as the percentage of students receiving Pell grants. We’ll be looking at affordability, like average tuition, scholarships and loan debt. And we’ll be looking at outcomes, such as graduation and transfer rates, alumni satisfaction surveys, graduate earnings and the advanced degrees of college graduates.”
In search of the best way to measure those categories, and of other kinds of information that ought to be included, Education Department officials will soon fan out across the country for months of town hall meetings, school forums and interviews. The plan is to compile the ratings by the start of the 2015-16 school year and to link those ratings to federal aid by 2018.
But Carolyn Hoxby, a Stanford professor and an author of an influential study about the failures of college as an engine for social mobility, said the effort is doomed to fail.
“I do not believe the federal government currently has the capacity to generate a ratings system that will even be neutral,” she said. “I think it’s more likely that it will be harmful to students.”
A truly useful analysis, she suggested, would have two main parts. “One, a lot of information about the outcomes for students — sometimes obvious things, like earning and employment, but also whether they serve in the public sector, whether they give back to society, do people have stable families, how do their kids turn out, do they end up dependent on social programs.”
The second requirement, she said, would be the ability to control for variations in the student body. “Let’s say you looked at Harvard, Yale, Stanford,” she explained. “You’d say they have all these great outcomes. But that doesn’t necessarily mean that’s the value added by those colleges, because their students were terrific” to begin with.
None of the current rankings undertake that kind of 360-degree analysis. According to Hoxby, none of them could: the necessary data is just not available at this time.
Even seemingly straightforward calculations, upon inspection, can be anything but. Take a question like how much alumni earn. Prospective students want to know how easily graduates can land good first jobs. So someone devising an algorithm to measure the value of a college’s degree might want to factor in first-year salaries.
Salaries 20 years after graduation might give a fuller picture of career arcs, but a college can change a great deal in 20 years. And should a college that graduates a lot of third-tier business majors necessarily trump one that produces top candidates in fields that pay less well, like teaching or nursing?
The rankings approach is working just fine for Queens College, which will prominently feature the Washington Monthly honor in its new recruitment materials, but even its president has his misgivings.
“I happen also to be a philosopher,” Muyskens said, “and I know that if we go too far on the value added and cost, we’re going to have people too focused on the practical to explore.”
As he spoke, students in the next room were participating in a study group about the Middle East, learning how to engage with opponents without getting bogged down in accusation and retribution. Face-to-face interactions like that can be the most enriching part of college, Muyskens said, but they never make it into any algorithm of value.
He is proud of the school’s high score, but he said, “If it isn’t balanced by other perspectives, it’s dangerous.”