Harvard psychology Professor Steven Pinker makes a number of highly contentious assertions in his big The New Republic piece on Ivy League admissions, and I am not going to try to challenge or refute any of them. All I want to talk about is his idea that it would be a better admissions system if the Ivies just relied almost exclusively on standardized tests.
I think it’s a terrible idea, and not just because I think standardized tests discover a too narrow band of aptitude. Prof. Pinker is clearly bitter that a lot of Harvard students skip his lecture, in his opinion, with the blessing of the Harvard administration. They do so to work on the student paper, to row on the crew team, to sing in ensembles, and to goof off or otherwise satisfy some higher priority. But, here’s the thing. Most college students learn as much or more from interacting with other students than they do from their instruction in classes. This is just as true at Harvard as it is at Ohio State. Kids are learning when they play a role in a play or play racquetball in the rec center. So, it’s not only important that colleges have organized activities outside of the classroom, but it’s important that the student body is diverse enough to have a starting quarterback for the football team, a cellist for the orchestra, and an editor for the newspaper. A college body made up purely of the kids who scored the best on standardized tests would be unlikely to mesh with the requirements of the school.
As a society, we have a tendency to look at college admissions as something that should done purely on merit, which puts the entire enterprise squarely on the individual applicant and not on the culture of the school. It’s strikes many people as unfair that a student with lower grades and aptitude scores would be selected over one with higher ones. That’s understandable, but it is the wrong way of looking at things. When I was growing up in Princeton, I knew two students at the university, one from Oklahoma and one from Alaska, who never would have been accepted if they had grown up in New Jersey. They got in specifically because they came from states that had only a handful of applicants. And they added something to the culture of the school that one more kid from Princeton High School would not have. That admissions process discriminated against me, but it wasn’t all about me. It was about having a student body that was itself educational.
I went to boarding school in New England for one year, and the school had a program that brought in black children on scholarship from Far Rockaway, New York. Without question, I learned more from interacting with those kids, who had both an urban sensibility and the experience of discrimination, than I learned in any of my classes. Most of those kids would not have aced a standardized test, but they benefited from the quality education they received and they gave back by educating everyone around them.
Prof. Pinker seems to think that Harvard and the Ivies should have a different standard than the Ohio States of the world, primarily because they can select only the brightest students who score the best on standardized tests. If you want to sing in a choir or star in a play, you can do that anywhere. So, maybe Harvard should just shut down all its non-classroom activities. Those things are a distraction from academics.
I suppose we could create schools like that. They would admit only the best test-takers and they would provide nothing but classes and coursework and labs. But, in my book, those kids would be getting only half an education. Actually, I think they’d be getting somewhat less than half.
Right on target, yet again.
Example: I went to Penn, coming from Wisconsin, as a recruit to the rowing team. I had an English and Middle East Studies dual major, and after I graduated I went to Egypt as a fellow at the Center for Arabic Studies abroad. While most of my classmates were fairly shut inside American University (where our program was located), interacting with upper-class English-speaking Egyptians in whatever spare time they had from the crushing workload of the intensive Arabic language studies. I joined a rowing club, and rowed every morning and afternoon with teammates from all walks of life, very few of whom spoke English.
I had some trouble keeping up with schoolwork, and took some grief from my instructors, but they could tell what I was getting out of my extra-curricular activities. By the end of the year I had some of the best colloquial Egyptian arabic skills in the program, in addition to friendships outside of academe, some of which have lasted to this day.
Apropos the recent blog about how %70 of Harvard seniors sent their resumes to Wall Street, how is skewing selection towards standardized testing going to improve THAT travesty?
%70 of Harvard seniors sent their resumes to Wall Street, how is skewing selection towards standardized testing going to improve THAT travesty?
It’s all a matter of perspective. Frames of reference.
The children of our emerging Wall St. aristocrats will have even more of an advantage because test prep courses that can costs thousands of dollars can make almost any standardized test much easier by anyone who could conceivably “pass” the test otherwise.
Capitalists seek to be oligarchs and oligarchs seek to be aristocrats. The draw to be the founder of a dynasty, with the immortality that accompanies the myth is very appealing to sociopaths.
Standardized testing + the destruction of public education gets us back on track to what they want.
There’s a school just down the street from Harvard that Dr. Pinker that places much more emphasis on quantitative skills. Perhaps he should move his operation there.
Ugh. Need an editor.
Pinker is a Skinner-box psychologist when it comes to education. And edges close to Murray bell-curve views on the origins of talent. “Evolutionary biology” is very easy to manipulate.
The New Republic is another one of the cesspools you could safely avoid except for pinyata diaries, which I have no doubt that this is.
Of course, the Ivies should rush to leave no child behind. And they should evaluate their faculty using the same objective means. Including Pinker and most of the current inhabitants of the Kennedy School of Public Policy.
But then we know the politics of establishing the “standards” for the standardized tests, don’t we? Let Texas lead the way.
Actually, he is as far from Skinner as any psychologist today. Of course, behaviorism and Skinnerism is totally moribund and discredited. And the thing that killed it WAS evolutionary psychology.
It might be slightly more complicated than that.
Of course it is more complicated. From 1950 – 1980, the behaviorist notions were gradually chipped away with a series of studies. It took many hundreds, if not thousands. Prior to 1980, there was no such thing as “cognitive psychology”. People talked about “language behavior” and “problem-solving behavior”, and the notion of the “mind” was not considered a useful construct. Specific studies of taste aversion, language acquisition, and others of this type led to the conclusion that some kinds of learning were easy to do, while others were not. Taste aversion, in particular, is a very fast one. Give a rat a new taste with a substance that causes upset tummy, and that rat will never eat anything with that flavor again. That was a total refutation to all learning theories, which emphasized increase in probability of emitting a specific behavior in response to a specific stimulus, etc.
So what is it that education is about that standardized testing enables?
Or is Pinker really just interested in some sort of eugenic aristocratic sorting mechanism?
My experience of Pinker’s books, coming from an information science point of view, is that they depend too much on mechanistic analogies from the construction of theory. That was the fundamental flaw that Watson introduced into his method of research a hundred years ago.
As for behaviorism being discredited, it has not been discredited enough and lives on in the “learned helplessness” theory that led the US to think that torture was a good thing.
Ideas have consequences.
There was cognitive psychology available in the late 70s if you were doing linguistics (as I was) or computer science. In linguistics most people were following Pinker’s Professor Chomsky into a purely model-theoretical world though, in which communication or human behavior in general could not really be imagined. If anything killed behaviorism, I think, it wasn’t evolutionary psych, which is entirely compatible with behaviorism, though many exponents like Pinker reject behaviorism, but cognitive science, in which behaviorist determinism was scrapped.
Earlier gestalt psychologists demonstrated experimentally that behaviorism was incorrect. They just didn’t have an alternative theory to offer. One of the first to begin breaking through the behaviorism model was Ernest Schachtel in Metamorphosis.
Throughtout my school years I was a very very good test taker. I loved taking tests, particularly standardized, multiple-choice tests. The message I received (which I was perhaps very eager to hear) was that this meant I was really really smart and so destined for all kinds of success.
Well it doesn’t work that way. It’s basically a very juvenile perspective.
So I think I have some grounds for saying that being able to do well on standardized tests is a tiny little capacity that is almost entirely unrelated to success in life, by almost any metric. Including success in supposedly academic or professional fields. I mean, it’s nice to be able to learn and retain a lot of information. It helps. But if you haven’t learned a variety of other (mostly more important) life skills, it doesn’t really matter how “smart” you are.
Very true. And it’s why the whole system around admissions to elite schools is bunk. Harvard is a great place but it’s not worth close to what one pays to attend unless one’s lucky enough to get substantial assistance. Even expenses at state schools have gotten out of hand. Higher education is in trouble.
Every test is a gatekeeper for a level. The test is given to provide information about who should go on to obtain training, get education, get a degree.
All tests are, for that reason, flawed. Those who did very well in the previous environment, and had the high test scores, may not do well in the next level. It’s simple to find examples of this. College football is the easiest. The pro draft attempts to use the test (college record) to predict who will do well, as well as a second test (combine performance). My WAG (wild-asses guess) is that these two account for 30-40% of the variance. The rest is stuff that cannot be found at the college level – performance against larger, faster, more skilled/selected athletes, heart, desire, and coachability. Look at Tim Tebow. He was a good, if unconventional, college quarterback. He won the Heisman, surely a mark of excellence. At the pro level, his lack of discipline ruled him out of the QB role. His lack of coachability led him to obstinately refuse to consider the wide-out position. So, he’s gone, never to return.
So it is in many disciplines. We look at test scores. But those with the highest test scores are sometimes the least good, because they were so successful at the lower level. Their level of success perversely cuts against them at the next level. They sometimes come in expecting to perform as well, but the competition and demands of the next level make it hard to do this. How many cases are there of high school academic stars who did not succeed in college? Many many. Especially engineering is an area in which this happens. Kids from HS go and expect the straight A grades. In college, in engineering programs, the straight-A performance is very hard to pull off. Classes are 2x as fast, the depth is greater, the coddling and help is sometimes non-existant.
So, relying on test scores alone? I think that is a bad idea. You need other measures. How persistent are they? Will they ask for help? Will they admit that they need assistance? How willing are they to find the answer? Do they seek knowledge for it’s own sake, or do they study to the test? Do they work well with others? Do they seek study partners?
Part of this is also a statistical artifact as well. It is termed “truncated correlation”. Take a good predictive relationship:
| | xxxxx
| |xxxxx
| |xxxx
| xxx|
| xxx |
| xxx |
| xxx |
+ ———————————–
Std Test
The Y-axis is college performance. This relationship is very strong. Std test level tells you A LARGE AMOUNT about CP. However, those to the left of the middle line do not go to college. Thus, we only observe those to the right of the middle line. If you look at the close-to-circular xs left after you remove the cases to the left of the line, you see a much weaker relationship.
This is the “truncated correlation”.
Need an editor. That middle line should be straight up and down.
There are lots of hidden assumptions in testing that are in fact denials of what might be randomly assigned results. The first is that the relationship between X and Y are in fact a cause-and-effect relationship.
Also there is an old variation of Murphy’s law that says that the data you have available for processing never matches the process that it is modeling.
Archaeologists fighting over whether C-14 counts of pollen grains show a site to be pre-Clovis are more methodologically rigorous than the tests used to sort out which humans succeed and which fail. And knowledge of the range of test scores often set the expectations for students and those who deal with them in a way that is self-fulfilling of success or failure.
Other than way over-valuing the SAT as a measure of merit (iirc high school grades are more predictive of college grades and graduation than the SAT), Pinker raised many valid questions.
What gets lost is that a meritocracy (should that be what we collectively desire and the evidence for that isn’t strong) can’t begin at age eighteen. The distribution of innate intelligence (assuming good pre and post natal nutrition, healthcare, and secure home) may not differ by SES status. But from then on out the higher the quality of education and nurturance, the easier it is for children develop cognitively in accordance to whatever innate abilities they were born with. Not going to see any “average” kid like GWB who is from a poor or lower middle-class family go to an Ivie.
Didn’t Columbia try this in 1964 or so only to end up having admitted one of their least successful classes ever?
Evaluating people accurately is hard! Ranking people by scores, or height, or whatever, is easy!
Most of you have not had a graduate course in testing, which I have, and most of you do not know persons who work at ETS (SAT) or the ACT (ACT), which I do. I probably know a bit more about testing than do you.
Testing in the modern sense began in WWII, with the Army Alpha. They had a large number of men, mostly young, mostly with little or no military experience. They determined that a test was needed, and this Army Alpha was constructed to grade and categorize men for placement in services. Who would do well as an infantryman? As a signals person? As a medic? As a driver? All important questions, because the flow of men was very large, and something had to be done to sort them out for maximum efficiency.
Following the end of the war, the number of persons attending college exploded, with the GI Bill and other forces of educational expansion. The issue there lay in the categorization of ability. If you have a 4.0 GPA at high school 1, and a 4.0 at High School 2, are you of equal ability? Clearly not, so the standardized test provides a common metric. There is a huge statistical and test-management process underlying the tests, so that the scores are consistent from year to year, and that an increase in score has a meaning – test equating, item equating, etc.
Along the way, a variety of issues arose. Items originally had considerable cultural baggage – items about polo are not likely to work with kids from Back of the Yards in Chicago. So, cultural neutrality was needed. Additionally, various kinds of abilities were gradually teased out – Howard Gardner DID NOT invent this idea, as it originated in U Chicago under LL Thurstone in 1927 (my graduate program was the LL Thurstone Psychometric Lab at UNC-CH). But the notion that persons can have ability in various directions is important.
Today, we are also hearing about new but obvious ideas. What about persistence? What about low vs high levels of frustration tolerance? What about ADHD? All of these underlie other approaches, such as the portfolio method, etc.
I doubt that standardized tests will go away, as the need to compare students from different places persists. I think that the tests provide useful information, but do not believe that they should be the SOLE determinant of admission. That would be a bad idea, and I expect to see the essay, the need for a diverse portfolio of experiences to continue.
Most difficult full year of courses I ever took. Only about a third of us limped through to the finish. By comparison, statistics was easy.
I come at this from having jobs in statistical quality control in the 1970s and during the IT recession being a seasonal test reader for a standardized testing company that was scoring a lot of state tests. The practice, like most businesses, is shaped to the needs of the client. And readers get staff session nudges and retraining so that the clients’ scores do not dramatically depart from national norms (even though the most likely should).
There’s very much a Lake Wobegone effect built into the client-test scoring company relationship. But I’m not sure that the bias is toward “above average” although it could be.
It was a very interesting and seemingly corrupt relationship being acted out in the way things were done. (And the most insistent client was from a New England state).
I don’t want to be the bomb thrower here, but I’m wondering who cares about these schools. If Pinker wants undergrads to pay attention to his lectures he should get a job at U. Mass. or Stony Brook or someplace. (Professor Krugman has just left your evil Princeton for the Graduate Center of the City University of New York, bwahahaha!)
The elite Harvard is training isn’t much of a public elite, with grads heading largely to finance and consulting and business and Silicon Valley or Alley, although a goodly number end up serving the School Rephorm movement for a couple years in Teach For America before they embark on their careers as entrepreneurs. For the government-service elite, schools like Liberty and Regent with no academic character at all are as significant as Yale and Harvard.
Pinker needs to ask himself what kids are going to Harvard for. but I can tell him in advance it’s not going to be in the hopes of being on the cutting edge of an academic discipline, whatever their test scores, except for a minority who are there essentially by chance.
This says it best:
Speaking of Princeton and standardized testing reminds me of a book I read back in 1974, I think, The Tyranny of Testing by Dr. Banesh Hoffmann, a mathematician at Princeton who had worked with Einstein. In the 1960s, Princeton put him in charge of a committee to find a better way of screening applicants because SATs were a very poor predictor of success at Princeton. I forget what they came up with, probably interviews and such, but he did explore the problems with standardized tests in his book.
As I point out above, tests are a good predictor of success in college. This notion that they are a bad predictor is incorrect. It is the fact that they are used as a selector which creates the false impression that they are a bad predictor.
“I suppose we could create schools like that. They would admit only the best test-takers and they would provide nothing but classes and coursework and labs.”
I think that’s already been done; the description fits MIT and CalTech pretty well. And they seem to produce a pretty well qualified set of graduates.
Sorta, with some exceptions. The big caveat being tuition for those private institutions, which at 41K for Caltech and 45K for MIT is more expensive than even Harvard’s notoriously high dues (38K).
In other words, a large part of college is a much more expensive version of what other countries practice for free (the gap year). Until recently, the Swedish compulsory (military) service was pretty much the same as a gap year, since most kids opted for social services instead of military ones. And since military training is expensive, and Sweden hasn’t been at war for over a hundred years, it worked out pretty well. I believe they abolished the compulsory service a few years back, though.