Law School Discussion

Nine Years of Discussion
;

Author Topic: Suggested Changes to US News Ranking System  (Read 3358 times)

SouthSide

  • Sr. Citizen
  • ****
  • Posts: 1563
  • Full speed ahead.
    • View Profile
Re: Suggested Changes to US News Ranking System
« Reply #10 on: April 13, 2006, 02:13:49 AM »
Here are my ideas:

1) Decrease the weight given to entering GPA, and increase both the entering LSAT and the acceptance rate weightings. GPA is just too random to be very useful.

The LSAT may be standard, but it's pretty useless on its own.

Besides, there's a fairly easy solution to this that some UG admissions offices use: you keep track of students' grades in law school, and see if any institutional trends pop out.  You then alter your index formula to account for it.

It was a pain in the ass before computers were in wide use, but now all you need is a work-study gopher and a copy of Excel.


I agree that law schools can track student's grades and adjust for difficult schools or majors. My point was that because the rankings emphasize undergrad GPA, law schools are encouraged NOT to make appropriate adjustments. Rather, if a law school wants to do better in the rankings, it can just select for students who have high GPAs, even if its own internal data indicates this is an unreliable measure of success in law school. This has a systematic distortionary effect. I can't think of any prominent schools in Palo Alto that are particularly notorious for this, can you?

Quote
2) Only measure employment ratio nine months after graduation. Give people a little bit more time. Not everyone wants to get a job right after law school.

That's not quite how legal employment works: generally, you wind up working for one of the two firms you worked at your 2L summer.  If you get a job 9 months after graduation, you're at the bottom of your class.  Having both gives you an idea about how many firms are really willing to hire a school's students, and if so, how many students have a fairly easy time getting hired.

[

Fair enough. I take this criticism. I still have a problem with this aspect of the rankings. Probably because it is the easiest for schools to falsify. However, my proposal doesn't solve that.

Quote
3) Do not measure financial aid at all. This should be considered separately in terms of which law schools offer the most value, but should not affect the consideration of a school's inherent quality.

Why not?

Because how much aid or LRAP a school provides says nothing about the quality of the school itself. Including this in the rankings actively misinforms prospective students. Say I am torn between School A, which is offering me a half-scholarship, and School B, which is offering me nothing. I turn to the rankings to see what I am giving up by taking the money at School A. However, the rankings are falsely inflating School A by including a factor that I am already considering. In other words, the financial aid counts twice for School A. First, it offers me a financial incentive to go there over a potentially better school. Second, it makes School A look better in the rankings than it otherwise would.

Or, to put it another way. Let's say that I am considering between School A and School B, both of which I have gotten an equal good feeling from, and neither of which has offered me any money. I look at the rankings and discover they are tied. Unbeknownst to me, School A generally offers a lot more financial aid than School B, which is factored into the rankings. The fact that the two schools are tied indicates that School B is better in all other ways. Thus, assuming the money offers for me are the same, I should go to School B. But the rankings don't tell me that.

Quote
These are starters for a better process. Any others?

Stop using Yale as the benchmark and tinkering the formula every year to ensure that it comes out at #1.  Pick the criteria and their relative importance first, and then see where the schools fall.

I don't think they use Yale as a benchmark. Rather, I think the things the ranking happens to value are things Yale is good at.
Columbia 2L.

Groundhog

  • Global Moderator
  • Sr. Citizen
  • ****
  • Posts: 4009
    • View Profile
Re: Suggested Changes to US News Ranking System
« Reply #11 on: April 13, 2006, 03:27:03 AM »
They do sort of use Yale as a benchmark, but it's not by assigning Yale #1 and figuring out how to do that. They assign the highest overall school a score of 100, and all other schools get rough percentages compared to the top law school, which has always been Yale.

Lily Jaye

  • Sr. Citizen
  • ****
  • Posts: 3095
  • A Schizophrenic Veronica
    • View Profile
Re: Suggested Changes to US News Ranking System
« Reply #12 on: April 13, 2006, 10:26:41 AM »
Excellent points, Lily, although they have analyzed UGPA and found it to be a pretty insignificant factor of predicting success in law school, compared to the LSAT solely.[ My feeling is that if we are going to have representative rankings, perhaps we should act the way the schools do-LSAT above all. Even alone, it's the most predictive factor in law school success.

It's the most predictive, but it's a very bad predictor. Oddly enough, it's far more predictive when you take GPA into account with it.

They do sort of use Yale as a benchmark, but it's not by assigning Yale #1 and figuring out how to do that. They assign the highest overall school a score of 100, and all other schools get rough percentages compared to the top law school, which has always been Yale.

Look at how they tweak their formula every couple of years.  Now take a formula from a few years ago, apply today's numbers to it, and see what you get.  (I'll give you a hint: Yale won't be #1.)

These links refer to UG rankings, but given the drama that broke out when CalTech hit #1, I wouldn't be surprised if the same cloud hangs over the law division:

http://www.washingtonmonthly.com/features/2000/0009.thompson.html
http://www.slate.com/id/34027/
Random 2L who does not spend nearly as much time here as she should.

Lily Jaye

  • Sr. Citizen
  • ****
  • Posts: 3095
  • A Schizophrenic Veronica
    • View Profile
Re: Suggested Changes to US News Ranking System
« Reply #13 on: April 13, 2006, 10:40:43 AM »
Quote
3) Do not measure financial aid at all. This should be considered separately in terms of which law schools offer the most value, but should not affect the consideration of a school's inherent quality.

Why not?

Because how much aid or LRAP a school provides says nothing about the quality of the school itself.[/quote]

But LSAT does?

IF the only thing that matters is the quality of the school itself, then only prof cites would count and you'd have Leiter's system

I have more to say but I just got a dean's scholarship at Georgetown.  I'm going to go jump around for a while.
Random 2L who does not spend nearly as much time here as she should.

prestigesq

  • Sr. Citizen
  • ****
  • Posts: 115
    • View Profile
Re: Suggested Changes to US News Ranking System
« Reply #14 on: April 13, 2006, 11:12:01 AM »
Stop ranking schools outside the top 30 on a national scope, these schools woiuld be more accurately ranked regionally.

Happy_Weasel

  • LSD Obsessed
  • *****
  • Posts: 5547
  • Me and Gir, the happy weasel.
    • MSN Messenger - ominusdemon2@msn.com
    • View Profile
    • Email
Re: Suggested Changes to US News Ranking System
« Reply #15 on: April 13, 2006, 12:40:43 PM »
I would go with top 20, actually. Then rank the rest regionally. There should also be an-unranked local schools category for schools under the reputation of 2.0.

Groundhog

  • Global Moderator
  • Sr. Citizen
  • ****
  • Posts: 4009
    • View Profile
Re: Suggested Changes to US News Ranking System
« Reply #16 on: April 13, 2006, 05:08:33 PM »
Interesting points, Lily. Could you provide me with some specifics? Sorry, I'm not much of an Excel genius so it would help to be hand-held through mucking around the create your own rankings.

SouthSide

  • Sr. Citizen
  • ****
  • Posts: 1563
  • Full speed ahead.
    • View Profile
Re: Suggested Changes to US News Ranking System
« Reply #17 on: April 13, 2006, 06:31:38 PM »
Congratulations Lily!! That's awesome!

The reason incoming LSATs say something about a school is because they speak to the quality of the student body, which most people would say is a huge factor in the quality of a school. Financial aid just doesn't do this.

I actually think US News would love to dislodge Yale from the top spot, but it would be hard to devise a system that would do that without screwing up the overall rankings too much. The reason the CalTech ranking was so controversial was because the public wouldn't accept that CalTech is the best college in the country. In the case of law schools, I think the public would definitely accept Harvard or Stanford in the top spot, and a shakeup at the top would sell a lot more magazines. They tinker with the college rankings all the time to shift around the top schools, just to keep it interesting.

The problem is that any ranking just has to give weight to admit rate and student/faculty ratio. These will always privilege a smaller school like Yale over a larger one like Harvard. The fact is that these two schools are offering slightly different products. I don't think one is better than the other, I think it depends on what you want in a school.
Columbia 2L.

Lily Jaye

  • Sr. Citizen
  • ****
  • Posts: 3095
  • A Schizophrenic Veronica
    • View Profile
Re: Suggested Changes to US News Ranking System
« Reply #18 on: April 14, 2006, 02:15:18 PM »
Congratulations Lily!! That's awesome!

Thanks. :)

Quote
The reason incoming LSATs say something about a school is because they speak to the quality of the student body, which most people would say is a huge factor in the quality of a school.

Doesn't this rely on the assumption that LSATs reflect intelligence?  As far as I can tell, they're just reflective of how much disposable income and time (i.e., SES) a student has.

Quote
Financial aid just doesn't do this.

First of all, I should probably clarify that expenditures per student =/= financial aid.  Unfortunately.

It seems to me that the best schools are the ones with the most financial resources -- and from the student's perspective, how much of those resources are used for their benefit.  IIRC, library size is meant to be a proxy for schools' resources in terms of how well they aid their professors, but expenditures per students is a proxy for how much of those resources are actually spent on students.

Quote
I actually think US News would love to dislodge Yale from the top spot, but it would be hard to devise a system that would do that without screwing up the overall rankings too much. The reason the CalTech ranking was so controversial was because the public wouldn't accept that CalTech is the best college in the country.

I suggest you re-read the links I posted.  The issue isn't what the public would accept, the issue is what elite reporters and the editor -- who's a Harvard grad -- would accept.

And remember, by "accept" I really mean "not fire you":

"When Elfin was first charged with creating a ranking system, he seems to have known that the only believable methodology would be one that confirmed the prejudices of the meritocracy: The schools that the most prestigious journalists and their friends had gone to would have to come out on top. The first time that the staff had drafted up a numerical ranking system to test internally--a formula that, most controversially, awarded points for diversity--a college that Elfin cannot even remember the name of came out on top...

Elfin subsequently removed the first statistician who had created the algorithm and brought in Morse, a statistician with very limited educational reporting experience....

To Elfin, however, who has a Harvard master's diploma on his wall, there's a kind of circular logic to it all" (http://www.washingtonmonthly.com/features/2000/0009.thompson.html)

When CalTech came out on top, it was only because another editor and statistician who decided to take USNWR's "thumb [off] of the scale" were forced out.  Clearly they didn't hear the office scuttlebut about their 1980s predecessor.

Quote
In the case of law schools, I think the public would definitely accept Harvard or Stanford in the top spot, and a shakeup at the top would sell a lot more magazines. They tinker with the college rankings all the time to shift around the top schools, just to keep it interesting.

Then why not routinely let someone else get the top?

Shifting it around is part of it, but you'd get more shifts with the same formula year to year.  That suggests the real purpose is to keep HYP at the top.

Quote
The problem is that any ranking just has to give weight to admit rate and student/faculty ratio. These will always privilege a smaller school like Yale over a larger one like Harvard. The fact is that these two schools are offering slightly different products. I don't think one is better than the other, I think it depends on what you want in a school.

I agree.  But how people actually choose schools =/= rankings methodology.

Interesting points, Lily. Could you provide me with some specifics? Sorry, I'm not much of an Excel genius so it would help to be hand-held through mucking around the create your own rankings.

Actually, the hard part is finding the law rankings methdology from previous years. :-\
Random 2L who does not spend nearly as much time here as she should.