I've been thinking a lot lately about standardized state tests. This fall, I spent about six weeks observing a classroom of third graders in Grand Rapids as they got ready to take their MEAP tests for the first time.
I was interested in this because the MEAP has a big impact beyond the walls of a school. Standardized test scores have been shown to affect housing prices. And housing prices affect all kinds of things, from consumer spending,to municipal tax revenues to, well, school funding.
So, test scores matter.
But I've also been thinking about something Howard University professor Ivory Toldsonsaid to me earlier this year. Toldson also edits the Journal of Negro Education, and he's spent a lot of time researching the test-score gaps between black boys and all other groups. Which means he's also spent a lot of time researching test scores in general, and why they're important.
His conclusion: test scores are only important to adults, not kids.
"Colleges don't look at state tests," Toldson told me. "They don't use that as a criteria to evaluate students. And so when we are telling kids they have to do well on this test, they're not doing well for themself, they're doing well for an adult's job. And there's nothing that we've presented them concretely that tells them that you doing well on this test means you will do well in life. We haven't given them that message."
"...when we are telling kids they have to do well on this test, they're not doing well for themself, they're doing well for an adult's job. And there's nothing that we've presented them concretely that tells them that you doing well on this test means you will do well in life." -Ivory Toldson
Those words were in my mind a lot as I talked to the third graders at Congress Elementary in Grand Rapids. The teachers and administrators at the school get kids to care about the state test by telling kids it's a chance to "show off" what they know. It's a way to make their teachers proud.
When I asked the kids why the test was important, many of them said the same thing. They wanted to show off. For them, doing well on the test was like putting on a good school play. It matters how the play is received.
But other kids were less sure. Some couldn't tell me why they had to take the test. A few kids told me the test was to help them learn. I'm sure test-taking does teach some things. It certainly helps kids practice how to sit still and focus on a task - something third graders rarely do. But I don't think any adult believes a math test is actually teaching a kid math. And, when pressed, the kids couldn't tell me what they've learned from taking tests. They just thought that was the point.
So, how else can you get kids to care about taking tests?
One idea I came across recently is to pay them.
The idea of paying kids to do well on standardized test had kind of a moment a few years ago, though it's not something you hear much about today.
One of the reasons the idea gained steam was because of a program run from 2004-2007 in the small Appalachian town of Coshocton, Ohio. There, local leaders came together to run a rigorous experiment on whether paying kids for test scores could really work. A local philanthropist donated $100,000 to the Coshocton City Schools to be used as an incentive program for the city's elementary schools. Kids were randomly selected to be able to participate in the incentive program, so there would be an effective "control group" of kids who did not receive any incentives.
The program was evaluated by researcher Eric P. Bettinger. At the time the program ran, Bettinger was at Case Western University in Ohio. He has since moved to Stanford. You can read Bettinger's full peer-reviewed research paper here.
In the paper, Bettinger lays out the details of the Coshocton experiment:
Each year, students in grades 3 through 6 take five different achievement tests in math, reading, writing, science, and social studies. Eligible students receive $15 for each test on which they score proficient or better. On any test for which a student scores in the "advanced" or "accelerated" designation under Ohio's state testing program, the student receives $20 instead of just $15. Thus, an eligible student who scores proficient on all five tests would receive $100.
Rather than pay the incentive in cash, which parents could have just used to spend on themselves, local leaders in Coshocton created something called "Coshocton children's bucks." These were essentially gift certificates that could only be redeemed for children's items at the store. Bettinger writes that even the cashiers at the local Walmart were instructed to ask children and their parents if the item they were buying with their Coshocton children's bucks was really for the child.
So, did it work?
Kinda, yeah.
Bettinger lays out the results in terms of standard deviations, so it's a bit difficult to parse the particulars, but the basic conclusion out of Coshocton is that there was a significant difference in math test scores for kids who got money for good test results and kids who didn't. There was less of an impact for other test subjects, but math was a definite winner. And the program had especially strong benefits for the kids on either end of the achievement spectrum. The kids at the top and the bottom both improved much more than the kids in the middle.
Bettinger evaluates the cost-effectiveness of the program by asking whether the same improvement could have been gained if Coshocton had simply spent the money on hiring a new teacher so that class sizes could be reduced. Based on his research, the answer is no. Paying students to do well on the test was more effective than hiring another teacher.
Coshocton isn't the only place in the world where this technique has been attempted. And, as the New York Times reported in 2008, the results for other pay-for-performance test schemes have been mixed.
For me, the money quote in that Times piece comes from Kati Haycock, of the D.C.-based Education Trust, which focuses on the racial achievement gap. She told the paper:
“There’s some part of all of us that gets a little queasy at this sort of buying stuff,” she said. “That said, the problem of underperformance, especially among poor and minority kids, is so serious and has been with us for so long that I’m not begrudging anybody who has good will here from trying something so we can hopefully learn something from it.” “Frankly, rich kids get paid for high grades all the time and for high test scores by their parents,” Ms.Haycock added. “So this isn’t so different.”
The results from the Coshocton experiment provide an interesting case for why it might be a good idea to pay students to do well on standardized tests. That doesn't mean we definitely should pay students.
What I take away from the results is that, when you give kids a reason to care about the tests, they stand a better chance of doing well. Money might make them care. But so might other things, like the chance to "show off."
The point is that the tests, as important as they may be to adults, are not important for kids at all. If you want kids to care, you have to find a reason for them to care.
