Study examines how 'data in the wild' is used in diverse school district

February 12, 2019 by Mike Krings, University of Kansas
Credit: CC0 Public Domain

For several decades, federal policymakers have pushed for education in America to be a more scientific endeavor. Researchers have sought to improve all aspects of education, but little attention has been paid to the experiences of educators putting research into practice. By studying one school district's adoption of a comprehensive reform initiative, a recent study from the University of Kansas found that teachers became "data-deferent" rather than "data-driven" as ideally intended. The authors argue districts and expert consultants can do more to facilitate practitioner access and understanding of educational research. They should also expand what counts as "data" and consider how reform models can both maintain fidelity while remaining flexible enough to fit local circumstances.

KU researchers spent five months at Garden City Public Schools as part of an ethnographic study into how the district and its 18 schools addressed the prevalent challenges of a culturally diverse, multilingual student population. During the study, they saw teachers, administrators and others working to implement Multi-Tiered System of Supports, known as MTSS, a research-backed reform method designed to identify and address student needs before they fall behind. That presented a unique opportunity to study how research is put into practice and how data is used in schools to inform educational change.

"With any sort of reform model, you try to make something abstract happen in real life. However, in real life, lots of things can happen," said Jennifer Ng, associate professor of educational leadership & policy studies at KU and a co-author. "Our analysis illustrates how reform occurred. For all of the talk about how education should be data-driven, there has not been much close study of what another scholar calls 'data use in the wild.'"

Ng co-authored the study with Don Stull, professor emeritus of anthropology at KU, and Rebecca Martinez of Indiana University. It is forthcoming in the journal Teachers College Record.

During the five-month study, researchers interviewed teachers, observed classrooms, attended school board meetings and interacted with school personnel across the district as Garden City Public Schools worked to implement MTSS. They found teachers were forced to rely on data generated by a prescribed set of standardized assessments to determine if students needed additional supports or were performing at an appropriate grade level. There was no room for teachers to offer additional observations of students or draw upon their own judgment to inform appropriate courses of action.

"We paid attention to how certain concepts became meaningful," Ng said. "Data, for example, came to be viewed as a number derived from a single standardized test. Everything that was not a number was very carefully distinguished as an anecdote or mere feeling."

Sole reliance on numeric data had the effect of suggesting other factors, such as teachers' direct observations or parental input were not valid ways of knowing about students and their needs. Outside consultants who worked with district personnel to implement the reform assured teachers the reform model was research-based and thus infallible. There was also a focus on strict adherence to instructional treatments.

The dominant logic of scientifically based research is that its outcomes will lead to certain and generalizable recommendations. However, that does not consider the particular contexts of individual schools. In Garden City, the community and district context includes large numbers of students who are English language learners, may have suffered significant trauma as refugees or migrants, and had varied prior educational experiences.

"Part of what we observed in the resistance, or discontent, of practitioners was not only their professional expertise being diminished, there were contextual and cultural variables that they felt unable to factor in," Ng said.

The researchers argue that educational practitioners should be given access to data and helped to understand why and how it can actually inform improvement. They should also be granted the flexibility to add their own expertise and professional experience to research-based reform. Educational interventions can have great success in improving educational practices, but a one-size-fits-all approach to implementing reform is not only unlikely to be effective, but it can also be counterproductive.

"With any school reform, you don't just adopt an idea and have everything automatically fall into place," Ng said. "You're depending on a lot of people across different roles to do their part. There can be misunderstandings or even active resistance that will play a role in how it's carried out."

The study provided a unique opportunity to see how research and data are handled in the field and adds a greater understanding of how reform can be modeled for the greatest success in widely varying schools across the nation.

"This was a unique opportunity," Ng said. "Research on data use has tended to focus on very specific uses or simply reinforce the mantra that 'data is good.' We're grateful to the Garden City district for giving us the chance to directly examine the implementation of research to practice."

Explore further: White math teachers treat students differently in predominantly black schools

Related Stories

Professor's book outlines how communities can reform education

December 17, 2013

In the nation's public schools, some students are marginalized based on disabilities or differences, real or perceived. A University of Kansas professor has co-edited and contributed to a new book that explores how students, ...

Teachers and Trump

November 5, 2018

Teachers felt immense pressure from school leaders and families to respond in a certain way—or not at all—in their classrooms following the 2016 presidential election, according to new research from Michigan State University.

Study explores what makes strong science teachers

June 29, 2018

A new study shows that eighth-grade science teachers without an educational background in science are less likely to practice inquiry-oriented science instruction, a pedagogical approach that develops students' understanding ...

Small high school reform boosts districtwide outcomes

April 28, 2015

Creating small high schools improves outcomes for students in the overall school district - both in new small schools and existing larger schools - according to a study of New York City schools by researchers at New York ...

Stop teaching kids how to be happy, says education expert

October 5, 2018

A leading educational psychologist is urging schools to stop thinking of wellbeing as another subject to be taught. Instead she is urging them to create healthier schools where students naturally develop wellbeing and a love ...

Recommended for you

Physicists reveal why matter dominates universe

March 21, 2019

Physicists in the College of Arts and Sciences at Syracuse University have confirmed that matter and antimatter decay differently for elementary particles containing charmed quarks.


Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.