U.S. News's corrupt college rankings
by Robert L. Woodbury
THE ANNUAL "America's Best Colleges" issue of U.S. News & World Report is to the newsmagazine what the annual "Swimsuit Issue" is to Sports Illustrated. Both are best sellers that make big money for their publishers. And both succeed because they are sexy, glamorous, superficial and largely without redeeming social value. But "America's Best Colleges" has evolved into something else, too: a universally recognized barometer and instigator of major higher-education trends, many of them perverse.
At colleges across the nation, presidents and their staffs, trustees and special university task forces analyze one year's U.S. News charts and immediately begin plotting how they might raise their college's standing in the next issue. This is not surprising; the results of a rise in rank are significant, and the consequences of a dramatic fall can be severe.
U.S. News bases it rankings on multiple statistical measuring sticks, each with a different weighting, arrayed across seven major categories. These include: academic reputation, student selectivity, student retention, faculty resources, financial resources, alumni giving and graduation rates.
Following are some strategies that colleges across America could adopt, if they haven't already, to improve their scores on one or more of these measuring sticks -- and, thereby, raise their rankings. These strategies should carry warning labels, however, because most contribute to bad public policy and undermine the integrity of the institution itself. Any college or university adopting these strategies should not be naive about the questionable role they are playing in a bigger game.
Produce an application deluge. An important measure of success is selectivity; and selectivity begins with lots of applications for admissions. So a college should market itself to as many audiences as possible, encourage applications from one and all (whether or not the applicant can get in or is interested in the place), make it as easy as possible to complete and pay for an application, and promote the notion that the college is the ideal "back-up" or "alternative" institution. Admissions staff should not discourage any students from applying, even if their chances of getting in are slim or their interests better pursued elsewhere.
Reject as many as possible. The second half of the selectivity equation is to accept the lowest percentage of those who apply, which means disappointing as many applicants as possible. The trick is to not accept any applicant who won't actually enroll (called "yield" by professionals) or, as at least one institution has tried, turn down those who are most able and most likely to go elsewhere. To minimize turndowns from students, colleges can: use as many early-decision dates as possible; pay careful attention to clues from interviews or prior applicant profiles; use financial or other inducements with wavering students; recognize that amenities like classy dormitories are more persuasive than numbers of periodicals in the library; and actively recruit the "chosen."
Spend money. This is not a game for those who would conserve resources, encourage efficiency, preserve capital for the long run or control the escalation of tuition rates. The gross amount of resources per student, largely irrespective of how it is spent, is a critical measure in the ratings game. So raising tuition, increasing the amount and variety of fees, seeking a quick jump in the annual fund, and taking a larger percentage yield from endowment can be productive strategies, at least in the short run. If the university's reputation rises, then the long run will take care of itself.
Let SATs reign. The currency of the day, whatever the controversies and protestations, boils down to test scores and high-school grades. If a college's average SAT or ACT scores (as well as high-school-class ranks) are going up, its ratings will rise; if its average scores are in decline, its rank will probably decline as well. One easy strategy is to make SAT or ACT submissions voluntary, which, quite logically, tends to raise the average of those submitted. A more comprehensive strategy is simply to "buy" students with higher scores. Through liberal use of merit awards, tuition discounts and other manipulations of financial-aid systems, one can target those applicants with higher scores who tend to be wealthier, whiter and less eligible for need-based financial aid.
Avoid nontraditional students. Whether the measuring stick used by U.S. News is test scores or persistence of first-year students or average time-to-degree or percentage of alumni giving, the system is stacked against colleges that enroll part-time, commuter, older, at-risk or more ethnically and racially diverse student bodies. Any institution that raises the proportion of full-time, higher-income, so-called traditional, residential students will be on its way to raising its ranking. This goal can be advanced through recruitment strategies, program changes, "profiling" of applicants, increased use of financial aid based on merit as opposed to need and sophisticated marketing.
Create reputation. The key factor in the ratings game is not the objective quality of an institution's educational standing, nor is it a determination of the qualitative difference a college has made in the life and learning of a student (what professionals call "value added"). The most heavily weighted factor is an institution's reputation for quality as voted by a very particular audience of peer presidents, provosts and admissions officers. And reputation carries a momentum that perpetuates itself without any necessary correlation with quality. Key strategies then would include the following: Practice careful target marketing on a regular basis to the primary voters, namely other presidents, provosts and admission officers; hire a consulting firm that specializes in placing a college's name in key national or regional media outlets, such as Time magazine or The Today Show; attract extraordinary national media attention leading to the "Flutie effect" (Boston College quarterback Doug Flutie's gridiron exploits, mostly in one game, led to a dramatic rise in applications to Boston College); and seek a "halo" effect by changing the name or apparent standing of the institution (say, by changing one's title from "college" to "university").
Change the rules or change the league. U.S. News tinkers with its formulae every year and listens carefully to suggestions from the stream of college presidents who visit or contact its offices annually. Over the years, presidents have argued for -- and won -- changes in the system that, not surprisingly, enhance the standings of their institutions. A far bolder scheme, which has worked for some institutions, is to change the peer group in which a college is placed. A common one in the past was to engineer a switch from the category of Public Comprehensive to Public Liberal Arts; today, one might move from Public Bachelor's to Public Liberal Arts. Several colleges, voila!, have suddenly risen from a mediocre standing in a former classification to a top ranking, often on a "regional" basis, in their new categorical home.
Exploit statistical keys to the illusion of quality. U.S. News attempts to find proxies for quality that might suggest that quality really exists. For example, a comparison of the average salary of professors or the percentage of faculty with Ph.D.'s or faculty/student ratios or the percentage of classes with fewer than 20 students all affect ratings. Some of the measures, of course, are dubious indicators of whether students are learning, and all the measures can be misleading. (A "good" faculty/student ratio doesn't necessarily mean that faculty are spending time with students.) In any case, each of these measures can be recalibrated to the college's benefit. For example, the belief that small classes benefit learning is commonly accepted. An effective strategy then might be to create a course schedule where all classes enroll 10 to 19 students or more than 100, rather than a distribution pattern that scheduled most classes with 20 to 50 students. With some imagination and statistical brainstorming, most of the gauges of quality can be reformulated to achieve a better score.
Pursue a comprehensive strategy of quick fixes. The goal of the game is to achieve a noticeable jump in ranking in a short time. Baby steps don't work. A strategy might be to create a one- or two-year surge in the ratings game by some combination of the following: a major allocation of funds and marketing for merit scholars; the dropping of programs such as football or elementary education that may attract students scoring lower on SATs; accepting a smaller than usual entering class; reducing the number of nontraditional students; implementing a one-year program to manipulate quality indicators such as class size; employing an expensive political-style marketing firm to woo the small group of peer academic administrators who do the voting; launching a special campaign to dramatically increase the percentage, not necessarily the amount, of alumni giving; and pursuing a variety of other tactics that directly match the yardsticks used by U.S. News.
Get with the program. Ultimately, what's more important than any particular strategy is understanding the assumptions, approach and methodology that governs the ratings game. When Consumer Reports rates and compares cars, it measures them on the basis of categories such as performance, safety, reliability and value. It tries to measure "outputs" -- in short, what the car does. U.S. News mostly looks at "inputs" (money spent, class size, test scores of students, degrees held by faculty) rather than assessing what the college or university actually accomplishes for students over the life of their enrollment. If Consumer Reports functioned like U.S. News, it would rank cars on the amount of steel and plastic used in their construction, the opinions of competing car dealers, the driving skills of customers, the percentage of managers and sales people with MBAs and the sticker price on the vehicle (the higher, the better).
Perverse effects. There is a problem with all this, of course. If all colleges and universities adopt these strategies, as an increasing number are, then the success of any single institution in raising its rank will be frustrated. If everyone does it, no one will get ahead and everyone will be diminished by the pettiness of the game and the illusion that minute changes in ranking mean something.
But the U.S. News rating system and higher education's response to it wield a far more pernicious impact. Values that we used to claim were important to the integrity and social value of our colleges and universities are being eroded.
First, the tyranny of the ratings in a very competitive business tends to distract us from what we, whether professionals or interested citizens, truly believe makes for a quality education for students in general and the differing needs of individual students in particular.
Second, the deeply held commitment to educational opportunity for lower-income students, people of color, nontraditional and part-time students, is frustrated when ratings depend so heavily on attracting the high test scorers with merit scholarships and other schemes that not only disadvantage the poorer and less experienced, but undervalue such assets as commitment, character, perseverance, leadership and creativity.
Third, the rankings denigrate the enormous value-added achievements of "lesser" colleges and universities that make an enormous difference in students' lives while giving preference to far wealthier institutions which may have only a marginal impact on the lives of students who already have great advantages in life.
Fourth, there seems to be no place in the ranking criteria for a college's role in civic education and the contribution of the institution to local, regional, national and global well-being.
The ranking of colleges and universities by neat formulae and dubious statistical measures is distorting, illusory and, ultimately, harmful to democratic values we all share. The real losers are, once again, the less advantaged among us. And when they lose, we all lose.
Robert L. Woodbury is former chancellor of the University of Maine System. This originally appeared in the current issue of Connection, the journal of the New England Board of Higher Education.
© 2004 College Advisor of New England