One minor change throws all the rankings into question. See Theodore P. Seto, Understanding the U.S. News Law School Rankings
, 60 SMU L. Rev. 493 (2007), available at http://papers.ssrn.com/sol3/papers.cfm?abstract_id=937017
Consider this excerpt from Professor Seto's article:
I begin with my conclusions. First, U.S. News’ law school “ranks” are unreliable – that is, they are subject to significant random error. . . .
The first conclusion can be illustrated by a simple example involving a change in the numbers of U.S. News's lowest-ranked school--which I will call the “bottom anchor” but otherwise leave unnamed. Assume that the reported nine-month employment rate for graduates of the bottom anchor falls by just one percentage point and nothing else changes at any school in the country
. . . .
As one might expect, nothing happens to the bottom anchor's overall score (by definition, zero) or rank (180th). But this tiny change wreaks havoc on the relative ranking of the top one hundred law schools.
Seattle and San Francisco jump six ranks, Fordham jumps from 32nd to 27th, and Rutgers Camden, San Diego, and Indiana Indianapolis each jump four. Houston, Kansas, Nebraska, and Oregon, by contrast, each drop three ranks. Overall, forty-one of the top one hundred schools change rank.
Fordham's dean gets a bonus. Fingers are pointed and voices raised at Houston. All because of a trivial change in the employment statistics of a single school far away in the spreadsheet. Stranger still, if the bottom anchor's nine-month employment rate falls an additional four percentage points (that is, a total of five percentage points)--and nothing else changes at any school in the country--most of these effects disappear, but the reordering moves into the Top Ten. University of California (UC) Berkeley and Virginia both drop from 8th to 9th place. At the other schools named above, it is as if nothing had ever happened.
Prospective students, employers, and faculty members, reading that UC Berkeley and Virginia have dropped to 9th place, may decide to go elsewhere. Regents, trustees, and university presidents, reading that Seattle, San Francisco, and Fordham have advanced dramatically in the rankings, may record this accomplishment in the apparently responsible deans' performance evaluations. What the foregoing example suggests, however, is that basing decisions on this kind of difference or change in U.S. News ranks is unwarranted.Id.
at 509-10 (citations omitted) (emphasis added).
And keep in mind that there were several other schools this year with inaccurate data (Nebraska, and one other I can't remember).