LSAT 163 = IQ 132

[ssasiflow]

A noted test expert, Mr. Rosner has discussed his research on the process by which SAT items are pretested. In brief, Mr. Rosner has discovered that pretest questions answered correctly more by whites than students of color are the ones then chosen for inclusion on the SAT exam.



Short Bio: Jay Rosner

Jay Rosner, the Executive Director of The Princeton Review Foundation, is a lawyer and an admission test expert, with a specialization in assisting historically excluded students on tests such as the SAT, GRE, MCAT and LSAT. In 2001 he testified as an expert witness on testing in the University of Michigan Law School affirmative action trial in Detroit. He is a consultant to KIPP schools and to Bob Moses' Algebra Project. He is based in the San Francisco bay area, but his work is national in scope. The Foundation he heads currently runs programs jointly with such national organizations as the Thurgood Marshall Scholarship Fund, the NAACP, and the Tavis Smiley Foundation and local organizations like the Asian Pacific Fund and Aspira of New York City. Jay holds a BA from the University of Pennsylvania, and a JD from Widener University. Prior to attending law school, Jay was a public high school math teacher for two years. He is the father of two daughters: one a recent college graduate, the other a college senior.


An excerpt from one of his latest articles is pasted below.

Preferences? On the SAT, They Happen to be White
_____________________________ ___________________

I've been a critic of the SAT for over 20 years, particularly regarding its impact on African American, Latino and Native American students. Recently, the president of a well-known northeastern university e-mailed me this question: "What do you say to people when they ask about the quantitative section of the SAT I? Surely there is no racial preference when it comes to this subject." In response, I just show folks two SAT questions and a bit of additional data. I don't need to do anything overly technical. The only concepts I use here are percentages and subtraction.

Educational Testing Service (ETS) is the company that puts together the SAT (and the GRE, GMAT, etc.). Below are two SAT math questions that ETS pretested. ETS pretests every potential SAT question by putting it into an unscored section of the SAT so that ETS can try it out. In other words, ETS wants to see ahead of time how tens of thousands of students do on it before the question is considered for later use on a scored SAT section. Thus, ETS finds out in pretesting the overall percentage of students answering the question correctly, along with the percentage of whites, blacks, Latinos, men, women, etc., answering correctly.

---------------------------------------------------------
Question #1:

If v2x is an integer, which of the following must be an integer?

a) vx
b) x
c) 4x
d) x
e) 2x

Is there a racial preference in this question? It happens that 7% more BLACKS that whites answered this question correctly. I'm an SAT expert, and I don't know why.

----------------------------------------------------------
Question #2:

If the area of a square is 4x, what is the length of a side?

a) x
b) 2x
c) 4x
d) x
e) 2x

Is there a racial preference in this question? It happens that 11% more WHITES than blacks answered this question correctly. Again, I don't know why.
------------------------------------------------------------

If you had the above statistics from pretesting (which ETS had) and needed to choose one of these questions to use on the SAT, which onewould you choose? ETS selected and used the second question, and rejected the first question. My testimony in the University of Michigan Law School case documented how ETS systematically rejects for use virtually every pretested question that favors blacks, and selects for use many pretested SAT questions that favor whites by 20% or more. Question #2 above, with an 11% difference, is slightly less than the average white/black difference of 15% on the SAT math questions that I researched. And, note that 4 of the 5 answer choices are identical. The white/black test score gap is a cumulative result of the individual questions that are chosen. I have data on 240 math questions used on 4 real SATs - exactly 1 question "favored" blacks (by 3%), and all the other 239 "favored" whites (by an average of 15%). Does this sound fair to you? Someone saying, "Surely, there is no racial preference (on the math section)," is certainly expressing what is commonly accepted wisdom. It's just wrong. There is no obvious bias, no bias that I can show you in the content of these or virtually any other SAT question. But there is a massive, unconscionable white preference created through question selection. I believe that this is not done by ETS with a racist motive, but it is a consistent, reliable, predictable and foreseeable pattern on the SAT.

------------------------------------------------------------------------
The answer to Question #1 is c. To get the correct answer, you must substitute 1/2 for x. The answer to Question #2 is b. Squaring a side of length 2x produces the correct answer.
------------------------------------------------------------------------

[ssasiflow]

The Educational Testing Service is located on a 376-acre estate outside of Princeton, NJ. One reporter has described it as "part corporate headquarters, part college campus, and part state park." The names on the buildings suggest something like Ivy League Inc.: Conant Hall, Brigham Library, and the Chauncey Conference Center pay homage to the Harvard and Princeton men who helped build the standardized testing movement. Even more memorable than ETS's buildings is its financial portfolio. Contrary to popular belief, ETS is a non-profit corporation - although it has exhibited behavior more like a company traded on Wall Street. Since its inception 51 years ago it has grown into an organization with a $456 million budget. According to the IRS it has real estate valued at $133.4 million and holds $34.8 million in cash and $132 million in stock. More than 2,100 people work for ETS. The current president of ETS earned $467,481 plus $49,664 in deferred compensation in 1998.

After a portfolio like this, one has to be reminded that "ETS World" is supposed to be about education. More accurately, it is about the business of manufacturing standardized tests, nearly 13 million of them every year. It administers the Graduate Record Examination (GRE), the Law School Aptitude Test (LSAT) [not any more] the Medical College Aptitude Test (MCAT), and the Test of English as a Foreign Language (TOEFL). But the jewel in its crown is the SAT. This exam, along with the PSAT, is given to more than 5 million teenagers every year. The story of how ETS and its progeny, the SAT, came to play such a dominating role in American educational life is the subject of Nicholas Lemann's sprawling narrative The Big Test: The Secret History of the American Meritocracy. Lemann has written an important book. It makes a contributi onto our understanding of the history of standardized testing in the United States. He has uncovered new information on the origins of ETS that forces us to seriously question why we allow a private testing industry to wield such an enormous influence in our public life. Furthermore, Lemann skillfully brings the reader into this subject matter through personal stories that illustrate larger social and educational issues.

[...]

HENRY CHAUNCEY'S ROLE
Much of the first part of the book is built around the life and times of Henry Chauncey, the first president of ETS from 1948-1972. Lemann draws upon ETS's archival papers and personal interviews with Chauncey to give an insightful and textured treatment of a man who wanted to test just about anything that moved. Lemann chronicles Chauncey's life from his Puritan family lineage, through his modest upbringing as the son of an Episcopal minister, to his student days at the Groton School, then on to Harvard, and eventually to what looked like the pinnacle of his career within the Yankee establishment, assistant dean of Harvard College. Lemann paints a picture of a man at one with the clubby rituals and tweed-suit world of prep schools and Ivy League colleges. However, even then Chauncey thought this rarefied world smacked a bit too much of privilege. While many Harvard students today come from wealthy backgrounds, back in the 1920s it was truly a closed world. Harvard was almost exclusively a place for rich, young, white men hailing from established families and well-heeled boarding schools. Academic merit was secondary to social standing. When Chauncey became an assistant dean in 1933, he was much taken by James Conant, the new president of Harvard College, and his desire to "reform" this system of privilege. Conant wanted to make Harvard and the other Ivy League schools places for a deservinge lite, where academic merit counted more than bloodline or checkbook.

As Lemann writes, it was a vision that hearkened back to the Jeffersonian ideal of "natural aristocrats," worthy scholars plucked from the masses to lead American democracy to greatness. True, no women or poor immigrants, much less African Americans, Asians, or Latinos fit into this notion of equal opportunity. Nonetheless, Chauncey convinced Conant that he had found the right selective tool to ensure academic merit. Enter the IQ test. Standardized testing was in full swing long before Chauncey fell in love with tests. By the 1930s, IQ tests had become a mainstay in American schools. They were being used to rank, sort, and track millions of students based on single number scores. Men like Lewis Terman, Edward Thorndike, and Carl Brigham had made careers out of claiming these tests proved the eugenic superiority and inferiority of different groups of people.

To learn from one of the masters, in 1933 Chauncey visited Carl Brigham, then a professor of psychology at Princeton. Brigham had already authored the influential A Study in American Intelligence (1923), where he had written, "The decline of American intelligence will be more rapid ... owing to the presence here of the Negro." Brigham still believed these ideas when he founded the original SAT in 1926. However, as Lemann notes, by the time Chauncey visited him in the 1930s Brigham had moved away from his racist ideology. In fact, he was doubtful that Chauncey and Conant could really construct a bias-free intelligence test. And he was not keen on the idea of creating a national agency that would administer all sorts of standardized tests that purported to measure natural ability. In 1938 Brigham wrote a letter to Conant stating,

"The very creation of powerful machinery to do more widely those things that are now being done badly will stifle research, discourage new developments, and establish existing methods, and even existing tests as the correct ones."

"If the unhappy day ever comes when teachers point their students toward these newer examinations, and the present weak and restricted procedures get a grip on education, then we may look for the inevitable distortion of education in terms of tests."

This is from the man who did so much to justify racial immigration quotas and tracking based on standardized test scores. It is a wonderful piece of research by Lemann and surely one of the most ironic and prophetic messages in the history of educational testing. (One wonders which Brigham ETS honored when they named their library after him.) Despite Brigham's misgivings there was nothing stopping Henry Chauncey. He was a man on a mission. Lemann describes how Chauncey was able to leverage a contract with the College Board to administer the SAT in 1948. We read about how Chauncey led the charge to move the SAT out of the Ivy League and help make it become the entrance test of choice for most colleges. This all came about at a time when American public education was expanding at all levels in the wake of the post-war baby boom.

[ssasiflow]

TESTS AND INTELLIGENCE
Lemann gives us plenty of detail about the private world of lunch deals and endless meetings, where the ideas for a national testing industry get hatched. This is valuable up to a point. What is not put forward is an examination of any of the theories of intelligence in the 1930s and '40s. At this time intelligence was still seen by most educators as something fixed, finite, and easily measurable by single number scores. However, there were more progressive theories about student learning. These ideas would cast doubt on the efficacy of testing for natural aptitude, suggesting instead that understanding intelligence is a complicated endeavor and scores on standardized IQ tests have a lot to do with economic class and social caste. Lemann chooses not to discuss the work of Horace Mann Bond, Otto Kleinberg, Margaret Mead or others who questioned the dominant paradigms about intelligence.

Lemann also does not analyze the effect tests were having on schools. What was life like for students who were streamed out of the college bound classes? How did the tracking by tests in schools reflect the larger social and economic inequality in society? These questions are not explored. It is too bad because taking time to explain how ETS has been in part an outgrowth of the dominant views of intelligence and schooling would have led him down a different path in the second half of the book. The last two parts of The Big Test are also structured around personal lives. Lemann attempts to chart the rise of ETS through several central and minor characters. This results in some of the same strengths and shortcomings evident in the first part. We learn about the hopes and dreams of some students who score well and go on to Harvard and Yale. Their backgrounds vary but they all have in common high scores on the SAT. While telling their stories, Lemann provides an interesting account of how the California higher education system is formed and the role Clark Kerr plays in using SAT scores in the early 1960s to make Berkeley the designated, elite public university in the country.

Of particular interest in this section are a few examples that are extremely relevant to high-stakes testing today. Lemann rightly points out that despite ETS claims to the contrary, its packaged tests are coachable and measure something else besides "aptitude." His review of the Stanley Kaplan and Princeton Review test prep companies is fascinating. Since the 1950s and '60s these companies have charged fees to help boost test scores for those clients who could afford them. John Katzman, the founder of Princeton Review and a person who has made his fortune prepping students for the SAT, has his own brutal assessment of the test. Lemann writes that Katzman believed the "SATs were pernicious, meaningless bull foisted upon America's youth by a greedy corporation." More than a few people would agree with Katzman.

Lemann also discusses the criticisms that began to mount in the 1970s. These complaints focused on the class and racial bias reflected in SATs scores. One of ETS's own researchers became a would-be whistleblower. Winston Manning tried to persuade his bosses at ETS that they needed to revise the SAT. Manning drew upon statistical evidence that correlated parental income and education to the actual SAT scores of their children. He wanted to have these factors counted so SAT scores could be revised upward for many students who did not have some of the privileges higher scoring students had. He even questioned, God forbid, the notion that there was one primary kind of intelligence that could be measured on a sit-down, timed exam. As Lemann points out, ETS would have none of this. They had too much of a good thing going. Manning's critique ended up in ETS's dustbin, and Manning soon retired.

Theseexamples could have been an opportunity for Lemann to examine what he only touches upon when he writes,

The SAT and the other ETS tests had worked their way deeply into the fabric not just of higher education but of the whole life of the upper middle class, which was substantially oriented around trying to ensure that its children got high SAT scores and therefore berths in better colleges ... Much of the curriculum in American elementary and secondary education had been reverse-engineered to raise SAT scores.

GOING OFF TRACK
Lemann whets our appetite. We want to read how these tests reinforce class-stratification and distort curriculum and instruction. We want to see how the SAT contributes to tracking in American schools. These discussions would have been a natural segue into how most states today are using high-stakes tests to evaluate student mastery in academic subject areas. Unfortunately, Lemann does not do this.

Instead, Lemann chooses to venture into areas that take the book off track. He cites the origins of affirmative action and the dilemma around using test scores to achieve racial diversity in school and workplace. He then spends the last third of the book describing Proposition 209, the California ballot initiative to roll back affirmative action in the state. This is principally covered through the life of Molly Unger, one of the people Lemann designated as an example of the modern meritocratic elite. Affirmative action and Proposition 209 are all worthy topics to examine, but Lemann should have made them into magazine essays or another book. These issues get lost in the almost-a-soap-opera melodrama of Unger's unsuccessful efforts to stop Proposition 209. The episode just dangles out there, shedding no light on what is happening in schools. This time Lemann does not succeed in using personal portrait and local story to unify a larger social narrative.

This is all too bad, because Lemann does succeed in prying open some of the secrets of the testing industry, an industry that has only gotten bigger since he first began writing this book. He does not go the next step. Without some review of alternative assessments and best practices that work in schools today, his recommendations for change seem naïve and disjointed. On the one hand Lemann believes that the "chief aim of school should be not to sort out but to teach as many people as well as possible. ..." Yet, the way he thinks we can get there is by establishing "greater national authority over education. High schools should prepare their students for admission to college by teaching them a nationally agreed-upon curriculum. Tests for admission to college should be on the mastery of this curriculum."

This is no antidote to the SAT. If Lemann had spent less time describing the life stories of the elite and more time interviewing teachers and students in classrooms, he might have discovered that externally prescribed curricula and tests are wreaking havoc in our schools. Top-down, packaged tests and curricula have never ushered in an era of substantive improvement for most students. Lemann avoids the more difficult question of how our educational system can achieve excellence and equity for all students in away that makes schools an interesting place to learn. Nicholas Lemann shows us how Henry Chauncey and his successor sat ETS made a lot of money creating a testing juggernaut that served a meritocratic elite. He just never shows us how we could change "ETS World" into an educational system that would work for all students.


http://www.rethinkingschools.org/archive/14_03/sat143.shtml

[wolfman1977]

I haven't done a math problem in ages, so bear with me if I sound like an idiot.  Shouldn't the answer to #2 (the one about the sqare whose area is 4x) be "2 times the square root of x"?  Squaring that number would give you an area of 4x.

[dirtyviolin]

This has been my impression of the LSAT, and other standardized tests, since I was first introduced to them. 
Historically upper and middle class white males were the majority (faculty and students) in institutions of higher education (undergraduate as well as graduate and professional programs).  This created an academic and social culture that catered to individuals well versed in the customs, mores, values of this social group.  When standardized tests were first developed for the academic community, primarily by white upper and middle class males, the questions were inherently biased in favor of this community.  As a result, individuals from this community have historically performed well on these tests.  As the tests have evolved, questions that a majority of the high performing test takers get wrong have been eliminated.  These, oftentimes, are the same questions that URMs get right.  It may represent a concerted effort, on the part of the test makers, to ensure adequate numbers of members from their community are able to matriculate into the most prestigious institutions (high scores serve as justification).  I believe, however, that it is a little more complex.  The test makers are trying to ensure that those with the skills they have determined best display the propensity for success in institutions of higher education score the highest.  Since the questions have always been culturally biased, due to the history of higher education and standardized tests, members of the dominant culture (and those well versed in the mores, values, etc. of the dominant culture) are at a distinct advantage since their "yardstick" is that by which all others are measured.

Then, what we are doing via these tests is perpetrating the "white class of professionals" while denying the minorities the opportunity to measure up even when the latter do have the ability to do so. Most people, though, find all this so "natural," that they may not even understand why one would scream so high about it -- that's the way it has always been for them and it's quite natural that minorities have been and continue to be discriminated against! After all, is it not the bias embedded in the test development process and built into the structure of these tests?! And while some of us may find this outrageous, they may simply add, "Well, that's why we have in place affirmative action programs!" Excuuuuuuuse meeeee, why don't you abolish these unconstitutional, racist and sexist tests altogether and let minorities compete freely with their fellow students, so that the former would not have to rely at all on piecemeal shitluck "affirmative action admissions"?

[panashi]

The point is that the LSAT is learnable...(i know some will disagree...but if its not learnable then why are u practicing?)

So those with access to classes and the ability to take off from work to study are at an advantage.

The LSAT is learnable like reading is learnable.

[mellissa]

DEATH TO THE MOTHERFUCKING NAZI AMERICA!

[M2]

The point is that the LSAT is learnable...(i know some will disagree...but if its not learnable then why are u practicing?)

So those with access to classes and the ability to take off from work to study are at an advantage.

The LSAT is learnable like reading is learnable.

good analogy. I really like that.

[mellissa2]

DEATH TO THE MOTHERFUCKING NAZI AMERICA!

[Ginatio]

I didn't realize it was so easy to get into mensa... i always thought they were the 1%ile, uptight intellectual elitists... they seem like a joke to me now.