Last week, the House Education committee at the state capitol passed a piece of legislation that would force schools to flunk any third grader who failed to get a score of "proficient" on the state's standardized reading test.
The legislation still has to pass both the full House and Senate and get a signature from the governor before it could become law. And, late in the week, there was news that state leaders may be putting the idea on hold while they gather input from teachers and school administrators (most of whom oppose the idea of flunking third graders based on a single test score).
So while everyone pauses to gather their thoughts about a proposal that could force nearly 40 percent of Michigan's third graders to repeat a grade, I thought it's worth taking a dive into the research to see how this plan has worked out in other places where it's been tried. It turns out, the idea has been studied quite a lot. So here are four main takeaways from the research.
1. Holding third graders back based on reading test scores can be part of a successful strategy to improve future test scores.
Much of the research about holding kids back, for any reason, has shown that it has a negative impact. The primary takeaway from that body of research is that forcing a child to repeat a grade makes it more likely the child will later drop out of school. But that research leaves us with a chicken-and-egg question: Did repeating a grade actually cause the kids to drop out later in life, or was repeating a grade just an early indication of a lack of interest in school work - which would have eventually ended in dropping out anyway?
The largest body of research on the specific policy of holding third graders back based on test scores comes from Florida, where the so-called third grade "reading guarantee" law has been in effect for a decade. To try to evaluate the laws' effect, and to answer the chicken-and-egg question, researchers at the Manhattan Institute for Policy Research (a free-market leaning think tank) looked at outcomes for kids who were right at the edge of the pass-fail line for reading scores. As they wrote in their report:
By studying the long-term performance of children who just barely passed the test, as well as those who were just barely left behind, it was possible to compare two essentially identical populations: one set of students who moved forward despite only borderline understanding of the material; and another set who stayed behind a year and received tutoring, mentoring, and other remedial interventions.
So what did they find? From the report's conclusion:
The results of our analyses are very encouraging for the use of Florida's test-based promotion policy. We find evidence that students remediated under the policy make large academic gains relative to their socially promoted peers-gains that are meaningful and sustained at least through middle school.
It's important to note, though, that these "academic gains" are measured purely based on the students' future performance on standardized tests. So you could ask: Did repeating a grade make the students better learners, or did it just make them better test takers?
A recent working paper published by the Harvard Kennedy School takes a stab at answering those questions, by looking at whether students who were held back in grade three were more likely to be held back again in later grades. Turns out, they were less likely to be held back in later years. The paper also found no effect on whether the retained third graders would later be placed in special education, or whether they would be chronically absent in future grades.
It's still too early to say whether Florida's law had an effect on drop outs, since the first students subjected to the law are only now reaching graduation age.
2. You can't just flunk a bunch of third graders and expect them to become better readers. Intensive (and expensive) support programs are key.
If improving standardized test scores are your goal, it's hard to argue with Florida's overall strategy. From 1992-2011, Florida's test scores in reading, math and science outpaced all but one other state. As the Mackinac Center for Public Policy has shown, Florida's scores rose much more quickly than Michigan's, even though Florida had a higher percentage of students who qualify for free or reduced lunch.
But no one believes that holding back third graders was the sole reason for Florida's test score improvements. For one thing, as the Mackinac paper above shows, Florida's biggest gains came before the third grade retention law went into effect. For another, the Florida law also came with some serious investments in helping kids learn how to read.
As the Brookings Institution noted in its analysis of Florida's law:
First, retained students must be given the opportunity to participate in their district's summer reading program. Schools must also develop an academic improvement plan for each retained student and assign them to a "high-performing teacher" in the retention year. Finally, retained students must receive intensive reading interventions, including ninety uninterrupted minutes daily of research-based reading instruction (a requirement that has since been extended to all students in grades K-5).
This raises another important cause and effect question for policymakers: Is it the grade retention that has helped Florida students improve test scores, or the intensive (and expensive) support programs for students who are held back?
Last year, Ohio passed a "read or flunk" law similar to the one in Florida, and one Republican state legislator offered her answer to the question. In a segment aired on the PBS Newshour, Ohio Sen. Peggy Lehner said she believed the state would have to spend an extra $50 - $60 million to make sure its new law is a success.
The "read or flunk" proposal being discussed in Michigan so far has no provisions for added spending.
3. Failing students based on test scores will disproportionately affect minority students.
At State of Opportunity, we've spent a considerable amount of time exploring the racial gaps in student test scores. By now, you probably know that black and Latino students don't perform as well on standardized tests as white students.
Some organizations, such as The National Center for Fair and Open Testing, even argue that racial test score gap exists because the tests themselves are biased against minorities.
"... the federal Department of Education's Office of Civil Rights reports that black children in Florida were nearly three times as likely as white children to be held back in third grade. Hispanic children were nearly twice as likely as white children to be held back."
So if it becomes enshrined in state law that no eight year old will pass the third grade unless they pass the test, it stands to reason that minority students will be held back at higher rates.
And, in fact, according to the Brooking's research cited above, the federal Department of Education's Office of Civil Rights reports that black children in Florida were nearly three times as likely as white children to be held back in third grade. Hispanic children were nearly twice as likely as white children to be held back.
What effect that has on the kids depends a little on whose research you believe. The team behind the Manhattan Institute study has concluded that the minority students who are held back do eventually catch up. But the paper from the Harvard Kennedy School reaches a different conclusion, finding that black students showed less improvements than white or Hispanic students after being held back in third grade.
4. If your system is built on test scores, teachers and administrators will find a way to game the system.
This is an issue that will come up whenever any accountability measure is put in place for schools. A working paper published by the National Bureau of Economic Research all the way back in 2002 showed how some Florida school districts re-classified students as disabled in order to keep those students' low test scores from hurting the district under new accountability rules.
So, this is nothing new.
In the first year of Florida's third grade retention policy, about 13.5 percent of all third graders were held back. Five years later, that number had dropped to 5.6 percent. The Harvard Kennedy School paper says most of the reduction was achieved because more 3rd graders passed the reading test. Most.
Florida's law, like the proposal under consideration in Michigan, has a provision to exempt certain students from being held back based on test scores. Researchers at the University of Arkansas found that those exemptions were unequally distributed among Florida's third graders. Specifically, black and Hispanic students were less likely to get an exemption, even if they had the same low score as a white student who did get an exemption. From the paper:
Controlling for other factors, African-American and Hispanic students with scores under the retention threshold are significantly more likely to be retained under the policy than white students with similarly low scores. African-American students are about 4% more likely to be retained under the policy than white students, and Hispanic students are about 9% more likely to be retained under the policy than white students.
The paper also concluded that students from low-income families were more likely to be granted an exemption that would allow them to continue on to the fourth grade, but that, ultimately, the students who were held back performed better on future standardized tests.
A growing number of states are considering these "read or flunk" retention laws. Stateline, a reporting project from the Pew Charitable Trusts, reports that 15 states plus the District of Columbia now have such third grade retention laws on the books. But the research on these programs is complex, and it's still not clear exactly what it is about the laws that helps students: holding them back a grade, or the intensive reading help they get when they're held back.
With all of these factors to consider, maybe its wise Michigan legislators have hit the pause button on the proposals here.
