'Vital signs' of teaching captured by quick, reliable in-class evaluation

A 20-minute classroom assessment that is less subjective than traditional in-class evaluations by principals can reliably measure classroom instruction and predict student standardized test scores, a team of researchers researchers reported.

The assessment also provides immediate and meaningful feedback making it an important new tool for understanding and improving instructional quality, according to psychologists from the University of Rochester and the University of North Carolina at Chapel Hill.

"Education researchers broadly agree that teachers matter," explained co-author Edward Deci, the Gowen Professor in the Social Sciences at the University of Rochester. "But there is less consensus about precisely what defines effective instruction and how to measure it. This assessment is able to capture the vital signs of teaching. It's a bit like a doctor taking your blood pressure and pulse for a quick picture of your health," said Deci.

Deci, Diane Early, a scientist at the University of North Carolina at Chapel Hill, and Ronald Rogge, an associate professor of psychology at the University of Rochester, reported their results in the Summer 2014 issue of the High School Journal.

In this study, the researchers asked trained observers to rate the classroom instruction of 58 math and English teachers in four high schools in Arizona using a tool developed by the Institute for Research and Reform in Education, a non-profit that uses evidence-based practices to help struggling schools. The 15-item tool focuses on three aspects of instruction: the engagement of students, how closely schoolwork aligns with state and local standards, and whether coursework is appropriately challenging. Called the EAR Protocol—short for Engagement, Alignment, and Rigor—the assessment already has been used in more than 100 schools, but this current study is the first to test the its objectivity and ability to predict student learning as measured by standardized tests.

The protocol is based on educational research showing that when students' basic psychological needs are met, learning outcomes improve, explained Deci. For example, when teachers are excited about their subjects and supportive, students are more likely to be engaged. When instructors present challenging schoolwork along with structured supports for mastering those assignments, students build a genuine sense of competence and confidence. "It's like learning how to play tennis. You improve when you play with someone who is just a bit better than you are," he said.

The researchers found that higher classroom ratings for engagement, alignment, and rigor were correlated with better student outcomes on standardized tests, after controlling for prior year test scores.

"The assessment captures surprisingly complex and fundamental qualities of teaching," said Early. "It's easy to use and 20 minutes is short enough for administrators to fit into the confines of their busy workday. And it's adaptable for all grades and subjects, from math and English to art and physical education."

The study also showed that observers can use the tool reliably. "Different observers of the same classroom came to the same conclusions," explained Early.

By highlighting areas where teachers need improvement, the assessments can help identify what kinds of professional development may be most helpful, the authors wrote. Follow-up assessments can then test to see if additional training enhances classroom instruction. "It's hard to know whether you are improving if there is no objective feedback measure," explained Early.

The assessment also helps teachers and administrators focus on the same key indicators of teaching quality: engagement, alignment, and rigor. "If adopted widely, the evaluations could provide a common language for talking about the vital signs of high-quality teaching," said Early.