Will Barratt, Ph.D.
Coffman Distinguished Professor Emeritus
A Little About Testing
Standardized testing has a long and dodgy history. There are several critical questions to ask in exploring this issue. The first, and maybe most critical one is: Standardized on which people? In the testing industry this is a validity issue. In the social sciences this is a question of bias. The second question is: Does this test ask the best questions given the intended use of the test. This too is a validity question. This too is a question about bias. A third question is: Do the scores on this test correlate with things they should NOT correlate with (discriminant validity) and do the scores on this test correlate with things they should correlate with (concurrent validity). For example tests of academic achievement, like the SAT, should correlate with other tests of academic achievement, and should NOT correlate with things like gender, ethnicity, social class, religion, ability, geographic location, and so forth.
Lets use the example of an oral thermometer as a standardized test. On which people was it standardized? Who decided that 98.6 F / 36.1 C is the standard, or normal, temperature? Is that number an average of all people in all climates? If so, what is the variation? Is the temperature the same for men and women? (Hint - look up basal thermometer.) You can apply these types of questions to standard tests and measures.
Standardized US college admissions tests were first developed to help admissions officers compare students from different high schools. The underlying assumption was that some high schools were much better than others, meaning that a C from Boston Latin School may be the same as an A from a rural New England public school. Note that people still refuse to question this unfounded assumption.
To accomplish this intended winnowing of students for acceptance into upscale campuses using a standardized test the College Entrance Examination Board (CEEB) was created. Note that the founding schools were upscale, private, and wealthy. Not a single Land Grant university was included. Yes, there is a hierarchy of prestige among colleges. Even the name of the first test in 1926 was the Scholastic Aptitude Test. As though scholastic achievement came from an aptitude, rather than accomplishments. I leave it to the reader to unpack the social class bias inherent in the assumption about individual aptitude The SAT is now simply called the SAT.
In 1959 American College Testing brought forward the ACT as a competitor to the SAT. Coming from The University of Iowa (My Ph.D. school) the test was more comprehensive, testing a wider range of subject matter than the SAT. I leave it to the reader to unpack the rivalry between Ivy League Private Schools (SAT) and Land Grant Public Colleges (ACT).
Please note that both the SAT and the ACT are among the most sophisticated assessment tools on the planet today. I should also note that the US, perhaps because of this rivalry, is way ahead of nearly every other nation in college admissions assessment. The SAT and ACT are very good tests. But are they the right tests?
Challenges to the SAT
Coffman Distinguished Professor Emeritus
A Little About Testing
Standardized testing has a long and dodgy history. There are several critical questions to ask in exploring this issue. The first, and maybe most critical one is: Standardized on which people? In the testing industry this is a validity issue. In the social sciences this is a question of bias. The second question is: Does this test ask the best questions given the intended use of the test. This too is a validity question. This too is a question about bias. A third question is: Do the scores on this test correlate with things they should NOT correlate with (discriminant validity) and do the scores on this test correlate with things they should correlate with (concurrent validity). For example tests of academic achievement, like the SAT, should correlate with other tests of academic achievement, and should NOT correlate with things like gender, ethnicity, social class, religion, ability, geographic location, and so forth.
Lets use the example of an oral thermometer as a standardized test. On which people was it standardized? Who decided that 98.6 F / 36.1 C is the standard, or normal, temperature? Is that number an average of all people in all climates? If so, what is the variation? Is the temperature the same for men and women? (Hint - look up basal thermometer.) You can apply these types of questions to standard tests and measures.
A Little History
Standardized US college admissions tests were first developed to help admissions officers compare students from different high schools. The underlying assumption was that some high schools were much better than others, meaning that a C from Boston Latin School may be the same as an A from a rural New England public school. Note that people still refuse to question this unfounded assumption.
To accomplish this intended winnowing of students for acceptance into upscale campuses using a standardized test the College Entrance Examination Board (CEEB) was created. Note that the founding schools were upscale, private, and wealthy. Not a single Land Grant university was included. Yes, there is a hierarchy of prestige among colleges. Even the name of the first test in 1926 was the Scholastic Aptitude Test. As though scholastic achievement came from an aptitude, rather than accomplishments. I leave it to the reader to unpack the social class bias inherent in the assumption about individual aptitude The SAT is now simply called the SAT.
In 1959 American College Testing brought forward the ACT as a competitor to the SAT. Coming from The University of Iowa (My Ph.D. school) the test was more comprehensive, testing a wider range of subject matter than the SAT. I leave it to the reader to unpack the rivalry between Ivy League Private Schools (SAT) and Land Grant Public Colleges (ACT).
Please note that both the SAT and the ACT are among the most sophisticated assessment tools on the planet today. I should also note that the US, perhaps because of this rivalry, is way ahead of nearly every other nation in college admissions assessment. The SAT and ACT are very good tests. But are they the right tests?
Challenges to the SAT
"Testing the Testers: The Nader-Nairn Report on the ETS" 1982, Change Magazine was an exploration of the limitations and uses of the SAT. ETS quickly responded with "Test Use and Validity: A Response to Charges in the Nader/Nairn Report on ETS". The intricacies of these documents is worth your time if you have the interest.
The Nader-Nairn report, among other things, claimed that the SAT was racially biased, since ethnic minority students got, on average, lower scores that ethnic majority students. The ETS writers noted in their response that the SAT was not racially biased, but biased toward students whose families had higher incomes that could be used to provide experiences that resulted in higher test scores.
According the the Educational Testing Service today: "Groups of students (such as male, female, Black, Hispanic, etc.) may have different average scores on the same test. This does not necessarily mean that the test is biased. If the groups actually have different knowledge and skills because of different educational backgrounds and opportunities, the scores will reflect those differences." https://www.ets.org/about/faq
So, "different educational backgrounds and opportunities" is a primary reason for group score differences. Yeah, that is pretty much social class. And the reality is more complicated. Many high school students with academically strong educational backgrounds and opportunities do not take advantage of those experiences. As an example think of the parents who allegedly bribed college admissions people to admit their children. One would assume those sons and daughters had loads of opportunities, and just didn't learn anything that could be applied to the SAT or ACT. On the other hand there are the people who grew up with weak educational backgrounds and opportunities, who took advantage of what was offered and did well on the SAT: the First Generation students on college campuses.
Standardized testing is standardized on everyone who took the test that year (from different educational backgrounds), and who are thinking about going to college. In the USA this is about half of all high school students. So, that is the comparison group. Is that the right group on which to standardize?
Questions on the SAT and ETS are geared toward knowledge and skills needed to be successful on campus. Some cultures use and value this knowledge base and skill set, some do not, at least according to ETS authors. So, by admission, these assessment tools are biased toward a culture that values the specific knowledge base and skill set being tested.
Is the SAT, or the ETS, asking the right questions? Maybe, except the prediction of success in college during the first year based on standardized test score is weak, at best. So these tests do not correlate very highly with academic success on campus.
So What
As the ETS writers note: Educational backgrounds and opportunities rule. These are related to family culture, wealth, opportunities, motivations, and a myriad other things.
tl;dr
Yes, the ETS and SAT are social class biased. But, how much bias is there?
