THE NEW YORK TIMES
March 11, 2009Study Offers a New Test of Potential Lawyers
By JONATHAN D. GLATERJust what makes a good lawyer?
In trying to answer that question, professors at the University of California, Berkeley, have come up with a test that they say is better at predicting success in the field than the widely used Law School Admission Test.
The LSAT, as the half-day exam is known, does not claim to predict much beyond a student’s performance in law school. But critics contend that it does not evaluate how good a lawyer someone will be and tests for the wrong things. They also say it keeps many black and Hispanic students — who tend to have lower scores — out of the legal profession.
Marjorie M. Shultz, a law professor who retired last year from Berkeley and is one of the study’s authors, said she began to examine the issue after California voters approved Proposition 209, which banned consideration of race in admissions.
“Proposition 209 and the reduced numbers of minority admits prompted me to think hard about what constitutes merit for purposes of law school admission, and to decide LSAT was much too narrow, as well as having big adverse impact,” Professor Shultz said.
The Law School Admissions Council, which administers the LSAT, helped finance Professor Shultz’s research, which has not appeared in any scholarly journals. Nonetheless, Wendy Margolis, a council spokeswoman, defended the LSAT, saying that how a student does in law school “has a great deal to do with ultimate success as a lawyer.”
Ms. Margolis added, “We think it would be difficult to predict success as a lawyer prior to law school.”
But that is exactly what Professor Shultz and Prof. Sheldon Zedeck, a colleague in the university’s psychology department, wanted to do.
To find out what applicant traits should figure in admissions decisions at law schools, they coordinated individual interviews, focus groups and ultimately a survey of judges, law school professors, law firm clients and hundreds of graduates of Berkeley’s law school.
They asked, among other things, “If you were looking for a lawyer for an important matter for yourself, what qualities would you most look for? What kind of lawyer do you want to teach or be?”
The survey produced a list of 26 characteristics, or “effectiveness factors,” like the ability to write, manage stress, listen, research the law and solve problems. The professors then collected examples from the Berkeley alumni of specific behavior by lawyers that were considered more or less effective.
Using the examples, Professor Shultz and Professor Zedeck developed a test that could be administered to law school applicants to measure their raw lawyerly talent.
Instead of focusing on analytic ability, the new test includes questions about how to respond to hypothetical situations. For example, it might describe a company with a policy requiring immediate firing of any employee who lied on an application, then ask what a test taker would do upon discovering that a top-performing employee had omitted something on an application.
More than 1,100 lawyers took the test and agreed to let the researchers see their original LSAT scores, as well as grades from college and law school.
The study concluded that while LSAT scores, for example, “were not particularly useful” in predicting lawyer effectiveness, the new, alternative test results were — although the new test was no better at predicting how well participants would do in law school. Unlike the LSAT, the new test did not produce a gap in scores among different racial or ethnic groups.
But participants might have performed differently on it, had they taken the test when they were applying to law school. Professor Shultz said this was one reason the next step in the research should include tracking test takers over time, from when they apply to law school through their careers.
David E. Van Zandt, dean of the law school at Northwestern, said he would welcome a supplement to the LSAT to evaluate applicants, a sentiment echoed by other law school deans.
John H. Garvey, dean of Boston College Law School and past president of the Association of American Law Schools, said, “It would be good for us and for other schools to have other measures that complement the LSAT and that would help us identify promising candidates.”
While his school’s admissions decisions involved much more than just LSAT scores — grades, work experience, recommendations and the like — Mr. Garvey said that more and possibly better predictive information would be helpful.
“Everybody would be happy for that,” he said. “There is not that much magic in the LSAT that we wouldn’t be willing to add to it to accomplish our more important goals.”
Both Professor Shultz and Ms. Margolis, the Law School Admission Council spokeswoman, said the next step was to survey lawyers nationwide, not just alumni of Berkeley, to test the measures of lawyer quality in a bigger pool. http://www.nytimes.com/2009/03/11/education/11lsat.html?_r=1&ref=education