Monday, January 17, 2005

Scoring the new SAT essays

The Sunday Washington Post featured an article about grading the SAT essays. After being strictly a multiple choice test for so many years, the SAT is now adding an essay.

I don't think adding an essay makes any sense. This doesn't mean I don't think writing is important. I just don't think there's a practical way to accurately judge a person's writing ability based on only a single essay.

The wonderful thing about the SAT is how accurately it measures a person's ability. So many people retake the SAT only to get a new score extremely close to their last score. Maybe it went up by a few tens of points if they prepped really hard for the retest. But that's about it.

Why can't colleges judge applicants' writing ability by looking at their grades in high school English classes? Don't high school grades mean anything? The SAT is supposed to add to high school grades by creating a score that's standardized across all high schools. It's not supposed to replace your entire high school record.

I suspect that unlike the multiple choice sections of the SAT, the essay section will be strongly influenced by test preparation. Students with rich parents will take the expensive test prep courses and learn the proper way to write for the SAT essay.

And what about this statement that the Washington Post managed to slip in:

Bremen, an English professor at the University of Texas at Austin, notes that the writer provides only one real example. Nevertheless, he says, the writer displays "a clear chain of thought" and should be rewarded, "despite his Republican tendencies."

That should raise people's alarm bells! Of course, proper test preparation will teach students to never write anything political, but the English professor was commenting on an essay about squirrels hording food supplies; I don't understand how in the world it could be construed as "Republican."

Somehow, the change to the SAT seems like change for the sake of political correctness. Liberals hate the SAT because it demonstrates quantitatively that people differ in intelligence, and they also hate it because the average black score is significantly lower than the average white score; the difference [possibly] exceeds a standard deviation. (I took statistics in college, I knew it would be useful someday!) [See comments for additional statistics.]

Also in the blogosphere, Betsy complains about the lack of feedback students will receive regarding their essay scores.

18 comments:

R said...

"Liberals hate the SAT because it demonstrates quantitatively that people differ in intelligence, and they also hate it because the average black score is significantly lower than the average white score;"

Please expound upon this more. It's quite a statement to make without following it up with more critical thought. I'd like to see where you go with this.

mikeca said...

There are many people that have trouble with the concept that your score on this one test will decide so much about a person’s future. My concern is how will this essay be graded? What will they do to make sure the grading is fair and uniform?

I have seen research that shows that on average high school grade point average (HSGPA) is as good a predictor of college GPA as the combined SAT score, and that the combination of SAT + HSGPA is a slightly better predictor of college GPA. As I recall, this held true for minority groups as well. The only interesting thing, is that the SAT score appears to be a better predictor of college GPA for women than it is for men, although the difference is small, where as HSGPA was an equally good predictor of college GPA for both men and women.

On what basis should colleges admit people? Should they admit the most intelligent people? Should they admit the best prepared? Should they admit the mostly likely to succeed in college? Should they admit the people mostly likely to contribute to the community after graduation? Should they admit the people mostly like to bring large endowments to the college?

These are not all the same thing. The most intelligent people are not always to mostly like to graduate. The smartest student I knew in college flunked out because he could not remember to come to class and do his homework, but he could get 100 on tests with 1/10 the work of any other student.

John said...

"Liberals hate the SAT because it demonstrates quantitatively that people differ in intelligence, and they also hate it because the average black score is significantly lower than the average white score; the difference exceeds a standard deviation."

Pathetic.

Really don't know what else to say.

However, I agree with [R]. I would like to see some support to back up your claim. Where are the statistics to show that the "difference exceeds a standard deviation?" I really would like to see them. Thanks.

Libertarian Racist

Jacqueline Mackie Paisley Passey said...

I thought the racial differences were common knowledge. Anyhow, it only took me 3 minutes of Googling to find this:

From the College Board’s own website: SAT Scores Vary by Race/Ethnicity: http://tinyurl.com/3pk45
White: 531 math, 528 verbal, 1059 total
Black: 427 math, 430 verbal, 857 total
Difference: 104 math, 98 verbal, 202 total

Standard Deviation (bottom of page 12): http://tinyurl.com/67p4f
112 math, 114 verbal, so 226 combined

So, close to, but not quite a full standard deviation in difference between the two groups.

LG, maybe you’re thinking of the difference between Asians (the highest scoring group) and blacks (the lowest scoring group)? Asians score 577 math, 507 verbal, 1084 total, with a total score 227 points higher than blacks’ total score.

Jacqueline Mackie Paisley Passey said...

I have mixed feelings about the addition of essays. Having read my college classmates' papers, there are MANY MANY MANY people graduating from high school and getting admitted to college with no clue how to write. Obviously their English class grades are not a good signal of their writing ability.

On the other hand, grading of essays is a very subjective task. But, there are ways to control for this. They now have an essay on the GRE, and my understanding of how its graded is two different people score it, and if the score is off by a point or more (on a 5 point scale) it is looked at more carefully and scored by a third person. I assume the SATs have a similar system in place.

Standardized tests are necessary because there is so much variation from school to school, class to class, teacher to teacher. Schools need some sort of objective criteria for comparing applicants. Now, what should be on those standardized tests is certainly an issue...

Libertarian Girl said...

The white standard deviation is 100 verbal and 102 math, and I think the standard deviation for the combined score would be a little lower than the sum of the individual standard deviations.

dadahead said...

Whoa, I cannot believe the ignorance / naivety on display in this post.

You say:

The wonderful thing about the SAT is how accurately it measures a person's ability. And what ability, pray tell, does it measure? If you mean anything besides the ability to take the SAT, your claim is completely unfounded.

There simply is no evidence that there is any such trait as "intelligence" which can be measured by any test at all, be it the SAT or the IQ test or whatever.

Try again!

Jacqueline Mackie Paisley Passey said...

There's actually tons of evidence that such a thing as general intelligence exists. Look up "g theory".

Dadahead, unless you've got a PhD in Cognitive Psychology and can back your claims up with substantive evidence, you probably shouldn't post that there’s no such trait as intelligence that can be measured, unless you want to display your own ignorance and naivety.

dadahead said...

How dare you talk back to dadahead!

I'll have you know that g theory is now all but completely discredited. No one has ever been able to specify what exactly the supposed "g factor" actually is, and have just asserted that it is whatever is measured by IQ tests. Sorry, no, begs the question.

Not that it's any of your business, but I happen to know quite a bit about Cog. Psych., and thus I know that no respectable cognitive psychologist makes the claim that he has discovered some trait called "intelligence" in the human mind-brain that can be measured via standardized test.

The search for the g factor is in fact hopeless, because the word/concept "intelligence" is not a scientific concept or term, but an everyday / ordinary-language one. There is no more reason to invoke the concept of intelligence in science than there is to invoke the concept of love, or communication, or desire.

Try again!!

R said...

"Dadahead, unless you've got a PhD in Cognitive Psychology and can back your claims up with substantive evidence, you probably shouldn't post that there’s no such trait as intelligence that can be measured, unless you want to display your own ignorance and naivety."

This is awesome! I didn't know we got to place ridiculous restrictions on specific posters! Okay, okay, let me try!

Jacqueline, unless you can calculate pi to 97 decimal places, in your head, you can't post because you've thus displayed a lack of basic intelligence.

This is fun! It's like I'm back in middle school blowing shit out of my ass that makes me sound much more intelligent than I really am!

Phoenician in a time of Romans said...

Liberals hate the SAT because it demonstrates quantitatively that people differ in intelligence, and they also hate it because the average black score is significantly lower than the average white score; The teeny tiny minor flaw in your gloating is that "black" and "white" in America are cultural constructs, not genetically seperated pools. For all meaningful biological purposes, you're the same people.

Which means that the differences in SAT scores would have to be due to, um, social differentation. They're a demonstration not of any biological differences, but of the biases of your country. The poor [blacks] don't do so well because they get poorer education, because their culture militates against intellectual performance, and because they're told they're dumber than whites.

Proving "liberals" correct. Oops.

Trippyshoe said...

Okay, I'm not even going to read the other comments or even your entire post. I don't have time. I just want to address one point.

"The wonderful thing about the SAT is how accurately it measures a person's ability. So many people retake the SAT only to get a new score extremely close to their last score. Maybe it went up by a few tens of points if they prepped really hard for the retest. But that's about it."I'm not sure about anyone else, but that was far from true for me. I'm a high school senior. The first time I took the SAT, I got 1330. The second time, I got 1470. The third and final time, I got 1550. I would contend an increase of 140 and 80 are rather substantial. Many of my friends have similar stories.

Jacqueline Mackie Paisley Passey said...

Please be explaining how my culture determined the pigment of my skin, or any of a number of other physical, genetic factors that differentiate between the races.

Phoenician in a time of Romans said...

Please be explaining how my culture determined the pigment of my skin, or any of a number of other physical, genetic factors that differentiate between the races.Feel free to provide examples of these differences in terms of allele frequencies. Don't forget to provide statistical ranges for said frequencies. And don't forget we're not talking about differences between, say, Norwegians and Zulu; we're talking about the differences between "white" Americans and "black", as in those descended from former slaveowners and former slaves. Please don't try to include more recent immigrants from Africa or the Carribean.

You may want to consider the implications of the "one drop" rule. You may also find this interesting. Or, at a slightly higher level, this.

"This point requires further attention. There is no doubt that there are some important biologic differences among populations, and molecular techniques can help to define what those differences are. Some traits, such as skin color, vary in a strikingly systematic pattern. The inference does not follow, however, that genetic variation among human populations falls into racial categories or that race, as we currently define it, provides an effective system for summarizing that variation. The confused nature of this debate is apparent when we recognize that although everyone, from geneticists to laypersons, tends to use "race" as if it were a scientific category; with rare exceptions,15 no one offers a quantifiable definition of what a race is in genetic terms."

Steph said...

>I just don't think there's a practical way to accurately judge a person's writing ability based on only a single essay.

I think it's a great idea. To apply for Ivy Leagues or other nice schools, the colleges usually require you to take the SAT II in a few subjects, one of them usually being an English test that includes an essay.

>So many people retake the SAT only to get a new score extremely close to their last score.

Not necessarily true. I had a 120 point difference the two times I took it, and my teachers believed I could do much better a third time. Another person I know had a larger point difference, her score going from a 1420 to a 12-something.

>Why can't colleges judge applicants' writing ability by looking at their grades in high school English classes?

Because many people don't learn how to write in their English classes and still make As. You could ask the same question about the SAT as it was... why not look at students' high school math grades instead of making them take some standardized test?

>Students with rich parents will take the expensive test prep courses and learn the proper way to write for the SAT essay.

Just like rich parents prep their kids to give them an unfair advantage getting into Ivy League schools.

>Liberals hate the SAT because it demonstrates quantitatively that people differ in intelligence, and they also hate it because the average black score is significantly lower than the average white score

Which is a load of crap. As I pointed out in another post, tests aren't the only way to measure... well... anything. One could be a genius but be pressured into pretending they're not.

Also, does that mean you believe that white people are quantitatively smarter/more intelligent than black people? If so, do you believe there is a biological/genetic factor and that, if there is, should we test for it and eliminate those that don't measure up?

paige said...

Your blog is great! It's hard to find blogs with good content and people talking about test preparation these days! I have a secret test preparation exposed if you want to come check it out

emily from blog articles said...

Hi Libertarian Girl,

As a result of reading your blog "Scoring the new SAT essays", I think you will discover my site on college entrance tests a great help.

To illustrate our diverse range of tests, here are some of the latest search terms that located our site ... reading tests for college age students, act/sat college prep tests, interest and aptitude tests for college students, act college tests, college assessment tests, college career tests, attitude and groundwork for college entrance and college entrance exam study sheet.

We have over 200 study prep guides and aids to help you sail through your exams without weeks and months of endless studying.

Best Regards
Emily

J Bartholomew said...

Managing Time requiries skills %%desc%% also mean that you %%desc%% learn new skills.