Value of test-driven education questioned by UB researcher

Lee's research was part of a recently released report by the Board of Testing and Assessment, a branch of the National Research Council titled, "Incentives and Test-Based Accountability in Education." His article presents a meta-analysis of 14 studies on the effects of test-driven external accountability policies on reading and math achievement.

A copy of Lee's study is available here.

The other selected studies in the report examine whether or not high school exit exams, teacher performance pay, direct student rewards and test-based incentive programs, such as 2001's No Child Left Behind Act (NCLB), are effective at achieving high academic standards.

The report concludes that these different approaches have not consistently generated positive effects on student achievement. Lee's research, specifically, was instrumental for the report's conclusion and was identified as the "preferred estimate" for test-based school incentive program effects.

"This report is very timely and critical for ongoing debates about NCLB and the future directions of national education policy," says Lee. "Previous studies often were long on description but short on prescription."

Lee reviewed study results that used cross-state causal-comparative or correlational studies of high stakes testing and accountability policy. His review was restricted to studies that used independent national test measures such as data from the National Assessment of Educational Progress (NAEP) and the National Educational Longitudinal Study (NELS). Lee wanted studies that were free of the potential risk of biased or contaminated results.

His review examined variations among the 14 studies and asked questions like: how were state accountability policies defined and measured, how were students, schools and states chosen for analysis and what statistical methods were used.

Some of Lee's findings:

• Past research on the impact of test-driven accountability policy on achievement falls short of meeting rigorous scientific research standards

• Evidence on the effects of accountability policy on academic achievement must be weighed carefully with more evidence on potential harms and risks

• The meta-analysis of 76 effect sizes from 14 selected studies showed weak effect on improving average achievement and no effect on narrowing the racial achievement gap

School-level incentives -- like those of NCLB -- produce some of the larger effects among the programs studied, but the gains are concentrated primarily in elementary grade mathematics and countered by lowered high school gains, and are small in comparison with the improvements the nation hopes to achieve, Lee says.

Evidence also suggests that high school exit exam programs, as implemented in many states, actually decrease the rate of high school graduation without increasing student achievement.

"Unfortunately, recent economic recession and budget cuts led the government to seek cheaper and faster programmatic solutions, while generating many underfunded mandates and cutting investments in capacity-building and infrastructure," says Lee.

The report, which was sponsored by the Carnegie Corporation of New York and the Flora Hewlett Foundation, urges policymakers to support the development of promising new models that use test-based incentives in more sophisticated ways, as one aspect of a richer accountability and improvement process.

"We need balance between short-term incentives and long-term capacity building to improve education" says Lee. "I suggest stronger national and state support for school capacity-building, including smaller classes, better trained teachers and evidence-based practices."

Provided by University at Buffalo