State VAMS don't reflect content, quality of teachers' instruction

New research published online today in Educational Evaluation and Policy Analysis (EEPA), a peer-reviewed journal of the American Educational Research Association, finds weak to nonexistent relationships between state-administered value-added model (VAM) measures of teacher performance and the content or quality of teachers' instruction. Based on their results, the authors question whether VAM data will be useful in evaluating teacher performance and shaping classroom instruction.

"Instructional Alignment as a Measure of Teaching Quality," by Morgan S. Polikoff at the University of Southern California and Andrew C. Porter at the University of Pennsylvania, also finds no association between multiple-measure teaching effectiveness ratings—which combine value-added measures with survey and observational ratings of teacher quality—and the content of teachers' instruction in the classroom. www.aera.net/Newsroom/RecentAE … d/15510/Default.aspx

"While value-added measures do provide some useful information, our findings show that they are not picking up the things we think of as being good teaching," said Polikoff. "Given the growing extent to which states are using these measures for a wide array of decisions, our findings are troubling."

Polikoff and Porter's study is the first large-scale analysis of the relationship of instructional alignment (i.e., the degree to which the content of classroom instruction is aligned with state standards and assessments, such as the Common Core standards) and teaching effectiveness with teacher VAM scores and a composite measure of teacher effectiveness.

The authors investigated these relations based on a subsample of 327 fourth- and eighth-grade mathematics and English language arts teachers across all six school districts included in the Measures of Effective Teaching (MET) study, including New York City, Dallas, Denver, Charlotte-Mecklenburg, Memphis, and Hillsborough County, Florida.

"We expected to see that instruction aligned with state standards and assessments would strongly predict value-added teaching performance measures on those assessments, but that wasn't the case," said Polikoff.

Recent policy initiatives such as the Race to the Top Fund and No Child Left Behind waivers have created a push to improve the methods by which teachers are evaluated. Many of these methods focus on teachers' contributions to student learning, often measured by value-added models and multiple-measure evaluation systems.

With the rollout of Common Core State Standards, according to Polikoff, the development of a deeper understanding of the ways effective teachers implement the standards in the classroom is imperative.

"Our results suggest that it's going to be difficult to use these systems to improve teacher performance," said Polikoff.

Their findings, the authors noted, lead to a disconcerting question: "If VAMs are not meaningfully associated with either the content or quality of instruction, what are they measuring?"