Social Class and Standardized Testing

Will Barratt, Ph.D.
Coffman Distinguished Professor Emeritus

A Little About Testing

Standardized testing has a long and dodgy history.  There are several critical questions to ask in exploring this issue. The first, and maybe most critical one is: Standardized on which people? In the testing industry this is a validity issue. In the social sciences this is a question of bias. The second question is: Does this test ask the best questions given the intended use of the test. This too is a validity question. This too is a question about bias. A third question is: Do the scores on this test correlate with things they should NOT correlate with (discriminant validity) and do the scores on this test correlate with things they should correlate with (concurrent validity).  For example tests of academic achievement, like the SAT, should correlate with other tests of academic achievement, and should NOT correlate with things like gender, ethnicity, social class, religion, ability, geographic location, and so forth.

Lets use the example of an oral thermometer as a standardized test.  On which people was it standardized? Who decided that 98.6 F / 36.1 C is the standard, or normal, temperature? Is that number an average of all people in all climates? If so, what is the variation? Is the temperature the same for men and women? (Hint - look up basal thermometer.) You can apply these types of questions to standard tests and measures.

A Little History

Standardized US college admissions tests were first developed to help admissions officers compare students from different high schools. The underlying assumption was that some high schools were much better than others, meaning that a C from Boston Latin School may be the same as an A from a rural New England public school. Note that people still refuse to question this unfounded assumption.

To accomplish this intended winnowing of students for acceptance into upscale campuses using a standardized test the College Entrance Examination Board (CEEB) was created. Note that the founding schools were upscale, private, and wealthy.  Not a single Land Grant university was included. Yes, there is a hierarchy of prestige among colleges.  Even the name of the first test in 1926 was the Scholastic Aptitude Test.  As though scholastic achievement came from an aptitude, rather than accomplishments. I leave it to the reader to unpack the social class bias inherent in the assumption about individual aptitude  The SAT is now simply called the SAT.

In 1959 American College Testing brought forward the ACT as a competitor to the SAT. Coming from The University of Iowa (My Ph.D. school) the test was more comprehensive, testing a wider range of subject matter than the SAT.  I leave it to the reader to unpack the rivalry between Ivy League Private Schools (SAT) and Land Grant Public Colleges (ACT).

Please note that both the SAT and the ACT are among the most sophisticated assessment tools on the planet today. I should also note that the US, perhaps because of this rivalry, is way ahead of nearly every other nation in college admissions assessment. The SAT and ACT are very good tests. But are they the right tests?

Challenges to the SAT

"Testing the Testers: The Nader-Nairn Report on the ETS" 1982, Change Magazine was an exploration of the limitations and uses of the SAT.  ETS quickly responded with "Test Use and Validity: A Response to Charges in the Nader/Nairn Report on ETS".  The intricacies of these documents is worth your time if you have the interest.  

The Nader-Nairn report, among other things, claimed that the SAT was racially biased, since ethnic minority students got, on average, lower scores that ethnic majority students. The ETS writers noted in their response that the SAT was not racially biased, but biased toward students whose families had higher incomes that could be used to provide experiences that resulted in higher test scores. 

According the the Educational Testing Service today: "Groups of students (such as male, female, Black, Hispanic, etc.) may have different average scores on the same test. This does not necessarily mean that the test is biased. If the groups actually have different knowledge and skills because of different educational backgrounds and opportunities, the scores will reflect those differences."  https://www.ets.org/about/faq

So, "different educational backgrounds and opportunities" is a primary reason for group score differences. Yeah, that is pretty much social class. And the reality is more complicated.  Many high school students with academically strong educational backgrounds and opportunities do not take advantage of those experiences.  As an example think of the parents who allegedly bribed college admissions people to admit their children.  One would assume those sons and daughters had loads of opportunities, and just didn't learn anything that could be applied to the SAT or ACT.  On the other hand there are the people who grew up with weak educational backgrounds and opportunities, who took advantage of what was offered and did well on the SAT: the First Generation students on college campuses.

Standardized testing is standardized on everyone who took the test that year (from different educational backgrounds), and who are thinking about going to college.  In the USA this is about half of all high school students. So, that is the comparison group. Is that the right group on which to standardize? 

Questions on the SAT and ETS are geared toward knowledge and skills needed to be successful on campus. Some cultures use and value this knowledge base and skill set, some do not, at least according to ETS authors.  So, by admission, these assessment tools are biased toward a culture that values the specific knowledge base and skill set being tested.

Is the SAT, or the ETS, asking the right questions? Maybe, except the prediction of success in college during the first year based on standardized test score is weak, at best. So these tests do not correlate very highly with academic success on campus.

So What

As the ETS writers note: Educational backgrounds and opportunities rule. These are related to family culture, wealth, opportunities, motivations, and a myriad other things.

tl;dr

Yes, the ETS and SAT are social class biased.  But, how much bias is there? 

The Success Trap - The Social Class Con Game

Will Barratt, PhD

The world of college and success is a rigged system. Once you have been seduced by the upward mobility career success trap, you will actively resist any criticism of the success and upward mobility system.

This is basic cognitive psychology. We all resist challenges to our ego identities. And if one of your ego identity is wrapped up in success, then, well, any criticism of that success system will feel like a personal criticism.

There is a giant conspiracy created and managed by all the people in the system. This is a collaborative cooperative, the ultimate in social control because the inhabitants run the system that imprisons them. No one person or agency is in charge, the success system is self sustaining by people invested in that system.

The College Confidence Game

Edward Smith, in Confessions of a Confidence Man, has six stages to the con, but he notes some stages can be omitted. David Mauer in The Big Con has nine stages. Here, I will use stages used in campus recruiting, retention, and graduation programs.

Setting the Stage. This is the well established role of college in the 'successful life' narrative. This con game is old and well used. "The bigger the trick, the older the trick, the easier it is to pull. You believe it can't be that old, and it can't be that big for so many people to have fallen for it." (Ritchie, 2005). The narrative intertwines college, success, work, and consumption. The stage is not set evenly for all people in all social classes. The appeal of college, the college=success narrative is middle and upper-middle class targeted. 

Recruitment. In the con this is roping the mark, on campus it is recruiting, pre-admissions, school counseling, vocational tracking, and many other things. The mark, the pre-freshman, is guided into the game. Tales of research on college graduate earnings, on career success, and visits from successful graduates feature here. These are all real, not illusory, benefits. The best propaganda is mostly true. The reality is that not everyone receives college benefits in the same amount. The key to recruiting, and retention, is to show the winners.

Retention. Retention programs are an example of keeping people in the game with success and the promise of more success. Colleges need students to keep playing, to stay enrolled, to keep payi8ng tuition. Like in computer games, colleges will celebrate the little wins and help students level up. Success in college, during classes, during out-of-class experiences is the key driver to keep people in the game. How many certificates did you get as a student? (See Bourdieu on Cultural Capital.)

The small payout used to keep the player in the game, the student in college, usually comes not as money, but as other forms of capital. In the con game the 'coin of the realm' is cash. Using Bourdieu's ideas of capital, campus uses cultural and social capital as sources of pay out. Presidential scholars on campus get to meet and chat with high prestige people, for example. The illusion of building social networks of successful people is useful. One interpretation of the success of students from high prestige colleges is that success is not about the education, but about the network of friends, or
Social Capital.

Many cons have a manufactured crisis, a limited time offer, to have the mark invest money. College does not have a manufactured crisis because education for success is the long con. Coin of the realm on campus is not money, but time, cultural capital, and social capital. However, there are exceptions, like limited time offers of scholarships and graduate assistantships. Scarcity is a widely used recruitment and membership tactic.

The Pay Out. Graduation and jobs. All too often students are not guided well into the world of work, or have been guided into majors with limited job prospects, or with job prospects with low pay. The barista with an Art History degree is a common counter-narrative in the success in college discussion. There should be a long discussion about college as vocational training - study for a job, or college as gaining generalized knowledge and learning how to learn - study for life. (I lean toward the learning how to learn side of this complex problem, given the rate of change in the world.)

Oh, and the college has your tuition. The student, or government, continues to pay to play. College has been commodified. While US students may owe money, they don't owe it to college. Campus leaders have already cashed the check.

The Challenge. When the con game is challenged, when problems arise, the system rises to respond. Often shifting the narrative. After all, the members of the media reporting on the system are themselves embedded in the success system, seeking the next big news story to bolster personal success. Yes, that is a critical problem. While there are counter narrative voices, like Mike Rowe (The Dirty Jobs person), these voices are drowned out by the volume of college=success choruses.

Challenges often occur in any con game, as they do in college. The campus debt crisis currently clouds the idea of economic payout for college. At least until you look closely at payout. There is differential payout for college graduates, there are differences in family financial literacy about saving for college, there is a dynamic employment market (Occupational Outlook Handbook).

However, these challenges are a vocational/income view of college that is seen in most of the world. Students, in most of the world, enroll in a specific major and cannot change. Enrollment is often determined by standardized test scores so the highest scorers go into medicine, less high go into engineering. This tactic is just another incarnation of the college=success con game.

Coda
This analysis is interesting, in part because all marketing uses features of a con game. Alternatively the primary form of power in use today is education, it is not reward or punishment. The reality is that college is, for the most part for most people, a good thing. 

tl;dr college is a con game, and is a good benefit to students

References

Smith, E. H. (1923). Confessions of a confidence man: A handbook for suckers. Scientific American Publishing Company.

Mauer, D. (1940). The Big Con: The Story of the Confidence Man.

Ritchie, G (2005). Revolver.