Report offers framework for evaluating D.C. school reform efforts, along with first impressions

Mar 04, 2011

A new report from the National Research Council offers a framework for evaluating the effects of a 2007 reform law on the District of Columbia's public schools. The evaluation program must include systematic yearly public reporting of key data as well as in-depth studies of high-priority issues, such as teacher recruitment and retention, the report says. As part of the evaluation program, the Mayor's Office should produce an annual report to the city on the status of the public schools, including an analysis of trends and all the underlying data.

The evaluation program should be structured to provide ongoing, independent monitoring and feedback on a number of key aspects of D.C.'s public education system, which are described by the committee that wrote the report. "Although the immediate goal for the District is to evaluate the changes enacted in 2007, we also see an opportunity to build a continuing program of analyses that will be useful regardless of future political or personnel changes," said Christopher Edley, committee co-chair and dean of the School of Law at the University of California, Berkeley.

In 2007 the Public Education Reform Amendment Act (PERAA) shifted control of D.C. Public Schools (DCPS) from an elected school board to the mayor and created the position of chancellor, among other changes. The act also mandated an independent five-year evaluation to determine whether enough progress has been achieved to warrant continuation of the new structure. The D.C. City Council asked the Research Council to provide guidance on how to structure the evaluation and to offer initial feedback about implementation of PERAA, to the extent possible.

Report Offers First Impressions, With Cautions

The report offers some first impressions of school reform under PERAA and notes that the city and DCPS have made a good-faith effort to implement the required changes, but the committee cautions that it is premature to draw general conclusions about the reforms' effectiveness at this time.

Data suggest that a modest improvement in student test scores has continued since PERAA was enacted. However, the report stresses that scores on their own cannot identify the reasons for that improvement. Further examination is necessary to determine whether they can be linked to the reforms or are the result of other factors, such as demographic changes in D.C.'s highly mobile student population. Also, because the rising scores are averages, further analysis and monitoring are needed to determine how various subgroups of students -- such as particular grade levels or students in particular areas -- are faring; some could be making gains while others are not.

The report emphasizes that student test scores should not be the only factor used to judge improvement in the school system. A comprehensive evaluation should address the full range of district responsibilities: quality of teachers, principals, and other personnel; quality of classroom teaching and learning; capacity to serve vulnerable children and youth; promotion of family and community engagement; and quality and equity of operations, management, and facilities.

Although the committee found that the administrative and governance changes required by PERAA have generally been carried out, the report notes that the Office of the Ombudsman was defunded three years after its creation. Since the office was intended to be the primary channel for parents to communicate with school officials and seek redress for complaints, its absence is significant, said the committee.

Evaluation Program Should Be Long-Term, Independent

The report offers a framework for evaluating key elements of reform under PERAA that affect student outcomes -- including management structures and roles and conditions for learning -- as well as the student outcomes themselves.

A set of specific, measurable indicators should be established as soon as possible, building on suitable indicators already collected in the District, the report says. These would track how well the city's public schools are doing, how well the key strategies the city has chosen to improve education are working, the conditions for student learning, and the capacity of the system to produce valued outcomes.

In addition, in-depth studies should be designed to provide deeper analysis of specific questions about high-priority issues, including both the targets of PERAA and long-standing challenges in District schools, such as special education. These studies can answer questions about the causes of trends seen in indicator data, explore potential reasons for disappointing outcomes, provide information to help alter existing strategies, or help explain why some strategies appear to be working better in some schools than in others. They are key, the report notes, because they are the means by which evaluators can answer policymakers' questions about the effects of particular policies and practices.

To develop and sustain an infrastructure for ongoing research and evaluation of its , D.C. will need to engage local universities, philanthropic organizations, and other institutions, the report says. The program must be independent of school and city leaders and responsive to the needs of all stakeholders, and its research should meet the highest standards for technical quality. There is no one model for the city to follow, but examples of research and evaluation programs that have been created in other cities -- such as Chicago, New York City, and Baltimore -- could be useful as D.C. develops its own program, the report notes.

"An independent evaluation program that can be an ongoing source of objective information and analysis will be an invaluable resource for the city," said committee co-chair Robert Hauser, executive director of the National Research Council's Division of Behavioral and Social Sciences and Education. "There are no quick answers. Education reform itself is a long-term process, and the evaluation of its outcomes also has to be seen in the long term."

The committee will discuss the report at a public briefing to be held at 2 p.m. on Sunday, March 6, at the National Academies' Keck Center, 500 Fifth St., N.W., Washington, D.C. A live audio webcast of the briefing will be available at http://national-academies.org.

Explore further: Why aren't consumers buying remanufactured products?

add to favorites email to friend print save as pdf

Related Stories

Study: Privatized Philly schools did not keep pace

Apr 09, 2009

Public middle-grades schools placed under private management in 2002 as part of a state-run overhaul of the Philadelphia School District did not keep pace with the rest of the city's public schools, according to a study published ...

Poor student outcomes linked to aging schools

Dec 03, 2010

When assessing education, much attention goes to the administrative control of the school district, teaching and testing. But little goes to the growing evidence that where learning occurs matters. American school buildings ...

K-12 education should include engineering

Sep 08, 2009

The introduction of K-12 engineering education has the potential to improve student learning and achievement in science and mathematics, increase awareness about what engineers do and of engineering as a potential career, ...

See something? Tell the teacher

Nov 23, 2010

(PhysOrg.com) -- Many school districts are pushing principals to spend more time in classrooms observing and evaluating teachers but few are using the information they gather to improve education.

Recommended for you

Why aren't consumers buying remanufactured products?

1 hour ago

Firms looking to increase market share of remanufactured consumer products will have to overcome a big barrier to do so, according to a recent study from the Penn State Smeal College of Business. Findings from faculty members ...

Expecting to teach enhances learning, recall

1 hour ago

People learn better and recall more when given the impression that they will soon have to teach newly acquired material to someone else, suggests new research from Washington University in St. Louis.

Understanding the economics of human trafficking

Jul 28, 2014

Although Europe is one of the strictest regions in the world when it comes to guaranteeing the respect of human rights, the number of people trafficked to or within the EU still amounts to several hundred ...

User comments : 0