Law School Discussion

Nine Years of Discussion
;

Author Topic: Something wrong with Ciolli's rankings?  (Read 650 times)

V00Jeff

  • Sr. Citizen
  • ****
  • Posts: 466
    • View Profile
Something wrong with Ciolli's rankings?
« on: March 13, 2005, 11:18:33 AM »
OK, someone that's good with numbers...why does Ciolli's new study ( http://www.autoadmit.com/studies/ciolli/ ) have so many counterintuitive results?

Like UNC being better than Yale and Penn in the NY/NJ/PA region? Or Vandy being better than Chicago and Northwestern in the Midwest?  Or Columbia and Chicago being no. 1 and no. 2 in the Pacific, while Stanford is no. 7 and Berkeley is no. 16?  Why is Texas no. 9 in the TX/OK/AR/LA region?

Is Ciolli's study really flawed, or is that our perceptions of regional prestige are so wrong?

My best guess is that the results seem so weird because Ciolli's study lumps in the big markets that are what most people are interested in with a whole bunch of other markets in a given region. I mean, the region with DC in it also has Florida, NC, SC, and Virginia.  The NY region also has NJ, which might be the same as NY for all practical purposes, but it also has PA.  And the worst is the Pacific region, which has SF and LA mixed in with Hawaii, Alaska, Oregon, and Washington.  Somehow I think Ciolli would have been better off if he studied city-wide placement instead of regional placement.  I mean, this study is clearly directed at people who are trying to compare the top 15 schools.  How many top-15 students are really trying to get a job in South Carolina, Alaska, or Oregon?
Attending: Columbia

IndyEntrepreneur

  • Full Member
  • ***
  • Posts: 80
  • 3.94 / 168
    • Yahoo Instant Messenger - indy_entrepreneur
    • View Profile
Re: Something wrong with Ciolli's rankings?
« Reply #1 on: March 13, 2005, 11:33:51 AM »
My best guess is that the results seem so weird because Ciolli's study lumps in the big markets that are what most people are interested in with a whole bunch of other markets in a given region.

Agreed.  Though interesting in its own right, I'm not sure that this format has as much utilitarian value for someone like me as would one with state-by-state rankings.  Other than a few money-whores who will move ANYWHERE for a big-$ job (and for whom national placement rankings will be sufficient), I would guess that most people will have fairly specific geographic targets based on factors like climate, cost of living, taxation, average length of run-on sentences, etc.  One of my post-school targets is Florida, for example.  I would guess that this new study is of limited use to me because of the inclusion of DC in the same region as FL.  Strictly from an intuitive level, I would think that the DC numbers will tend to skew the data for that region, making them less relevant when considering only lesser markets within that region.

Seems like it could be fun/useful if Ciolli would release his state-by-state raw data so that we could "regionalize" it, if you will, for our own particular set of preferences.  I suppose I could re-gather it myself, but I don't have that much time on my hands at this point.   :)
Accepted:  Boalt, Duke, IU Bloomington (in-state, 19.5k/3 yr), Miami, U Illinois (full tuition), U Texas(in-state, 5k/yr)
Rejected:  ---
Waitlisted:  Northwestern, U Michigan

Pending:  Harvard, Notre Dame, USC

http://www.lawschoolnumbers.com/display.php?user=IndyEntrepreneur

V00Jeff

  • Sr. Citizen
  • ****
  • Posts: 466
    • View Profile
Re: Something wrong with Ciolli's rankings?
« Reply #2 on: March 13, 2005, 11:46:26 AM »
Yeah.  I think state-by-state regions would be better.  But even then, I think you'd run into some problems.  People that want to go to SF might want specific info on that area...or at least, the greater Bay area...and not have that info be tainted by LA or central california stuff.  Same thing with most of the big markets...people that want to work in Boston probably don't want data on the rest of Mass, and people that want to work in the Raleigh/Durham area of NC probably don't want info on Winston-Salem or Charolotte, etc. etc.  I guess that's why I think city-by-city stuff would be the best-- maybe with some minor adjustments to account for places like Orange County or Northern Virginia that most people would probably think of as being within the larger markets of LA and DC.  Maybe the ABSOLUTE best would to poll students at the top 15 schools to see what kind of markets people look for.  That way you could sort out questions like, "Do people look to work in the greater bay area, or San Francisco?"  Or, "Do people include Orange County and San Diego firms if they want to work in LA?"
Attending: Columbia

DannyJunior

  • Guest
Re: Something wrong with Ciolli's rankings?
« Reply #3 on: March 13, 2005, 11:48:12 AM »
Anthony addresses this in the study. The rankings only show elite firm placement. Many students from Yale intend to go into government work and prestigious PI positions, and thus don't work for a Vault 50 firm. Regional rankings are also somewhat problematic due to issues of self-selection. Many grads of top schools decide to go to a particular region over another on their own. The fact that they do not go to a specific region in large proportions does not suggest that they cannot go there.

These issues explain some of the variance you take issue with. For what the study is intended to tackle (placement at elite firms, with focuses on specific areas of the country) I think this study goes above and beyond anything done in the rankings literature so far. The strengths and advantages of Anthony's methodology are myriad, and the possible misrepresentations are all noted and satisfactorily addressed in the paper - of which the actual rankings lists are but a small part..

DannyJunior

  • Guest
Re: Something wrong with Ciolli's rankings?
« Reply #4 on: March 13, 2005, 11:51:34 AM »
Seems like it could be fun/useful if Ciolli would release his state-by-state raw data so that we could "regionalize" it, if you will, for our own particular set of preferences.  I suppose I could re-gather it myself, but I don't have that much time on my hands at this point.   :)

Actually I THINK that the raw data, including state-by state data, will be available sometime soon, if its not already.

IndyEntrepreneur

  • Full Member
  • ***
  • Posts: 80
  • 3.94 / 168
    • Yahoo Instant Messenger - indy_entrepreneur
    • View Profile
Re: Something wrong with Ciolli's rankings?
« Reply #5 on: March 13, 2005, 12:59:10 PM »
Seems like it could be fun/useful if Ciolli would release his state-by-state raw data so that we could "regionalize" it, if you will, for our own particular set of preferences.  I suppose I could re-gather it myself, but I don't have that much time on my hands at this point.   :)

Actually I THINK that the raw data, including state-by state data, will be available sometime soon, if its not already.

Do you know the granularity of the data?  I.e., does it go down to a city level, or is it aggregated by state?  If it is at the city level, it could even be fun to add a web front-end to it and allow people to select cities, states, or regions of interest, and create their own customized report.  This, of course, does nothing for overall prestige analysis for a given school, but could be a nice tool for someone who just wants to find out which school will be best for them based on their own personal factors.  To me, that's the exciting thing about this study, and, in particular, the data behind this study.  It seems to open the door to a level of personalized analysis rather than the "one size fits all" rankings of prior systems.
Accepted:  Boalt, Duke, IU Bloomington (in-state, 19.5k/3 yr), Miami, U Illinois (full tuition), U Texas(in-state, 5k/yr)
Rejected:  ---
Waitlisted:  Northwestern, U Michigan

Pending:  Harvard, Notre Dame, USC

http://www.lawschoolnumbers.com/display.php?user=IndyEntrepreneur

uponcripplecreek

  • Sr. Citizen
  • ****
  • Posts: 285
    • View Profile
Re: Something wrong with Ciolli's rankings?
« Reply #6 on: March 14, 2005, 12:49:34 PM »
I don't believe there is any individual state data, because one of the key factors in the study is the USN&WR data on regional placement percentages which they unfortunately do not break down any further.

Check out some of the threads on xoxo for more detailed responses and theories to the questions you posed.