Credit: CC0 Public Domain

A payment on results approach to delivering education aid, which is championed by international institutions including the World Bank, is in danger of backfiring in some of the countries it aims to help, researchers believe.

The concerns are raised in a new study, by academics at the Universities of Cambridge and Addis Ababa, which examines results-based financing in education and heavily critiques one such program in Ethiopia. It urges donors not to treat the approach as a "magic bullet" for , echoing other studies which have flagged similar doubts.

Results-based financing is a funding model that has been widely adopted by Western governments and institutions to provide education aid to lower-income countries. Rather than handing out grants up front, the approach requires recipient governments to meet a set of target conditions which are agreed with donors in advance. The money is released as these conditions are met.

The targets vary, but typically involve improvements to attainment and enrolment in schools. According to the World Bank, results-based financing "could have a substantial impact in terms of achieving results that matter" in support of the UN's Sustainable Development Goals.

The new study examined the 'Program for Results' (PforR) scheme: a results-based financing package underpinning the latest phase of the Ethiopian government's education reforms. This draws on a pooled fund, supported by a consortium of donors led by the World Bank.

Although the research is broadly supportive of the principle of linking funding to results, it found that several aspects of the financing project were unfit for purpose from the start. Many of the targets set through PforR, for example, fell short of those of the education reforms themselves. The researchers also argue that key groups of children, such as those with disabilities, were overlooked in the target-setting; inadequate systems were put in place to measure results, and some were unaware of the new system months after it began.

Professor Pauline Rose, Director of the Research for Equitable Access and Learning (REAL) Center, University of Cambridge, said: "The shortcomings we identified suggest that the potential for this results-based financing program to improve education and learning is limited. In the , it could end up undermining the very reforms it is meant to support."

The study is not the first to question how results-based financing packages are being structured and implemented. Similar problems have been highlighted in several previous assessments, including an evaluation of a pilot program in Ethiopia in 2015, and an assessment of funding programs in Mozambique, Nepal and Tanzania, in 2021.

The PforR initiative began in 2018 and is expected to run until 2023. Researchers examined the original program appraisal document, and interviewed 72 of the donors and responsible for its creation and delivery.

They found that many targets set through the scheme failed to match the ambition of the Ethiopian government's reforms. Just 40% were linked to improving academic results, which is the principal aim of the government's initiative. The PforR plan also specified that attainment should be measured at 2,000 schools which had been earmarked as requiring improvement. The bar set for the attainment targets that would unlock further funding was therefore often low; one donor described them as "a bit soft".

While some of these targets took gender parity into account, researchers found that they overlooked other equity issues, such as how far education reforms were supporting marginalized groups including children with disabilities and those from the poorest backgrounds.

In the few cases where the PforR plan did specify targets for these groups, they were often widely considered to be inadequate. For example, education officials told the researchers that they had raised concerns at the plan's draft stage about a target for expanding the number of Inclusive Education Resource Centers in Ethiopia. The researchers calculate that this target, if achieved, would affect just 10% of schools and fail to reach the majority of children with disabilities. The feedback raising this concern was never taken into account.

The paper criticizes what appears to be a back-to-front approach to data-gathering. Several interviewees observed that systems were not in place to measure whether the PforR targets were being met before the program started. Instead, improving data collection was itself set as a goal. In some cases, the study finds, this may mean that inaccurate information produced under the old, faulty system is likely to be contradicted mid-program, creating the false impression that some targets are being missed.

The analysis also found a "significant gap in knowledge" about the program's introduction among regional and woreda (district) officials in the local education authorities charged with delivering results.

Months after it commenced, one official told researchers that he had "no clear understanding" of what 'Program for Results' meant or involved. Another said that they had only heard "a rumor that the school grant is to be changed". "These interviews were carried out during the first year of the implementation," co-author, Dr. Belay Hagos from Addis Ababa University, said. "We didn't expect everyone to have a comprehensive knowledge of what it involved, but we did expect they would at least be aware of it."

The authors suggest that these findings add further weight to existing evidence that some results-based financing packages are being implemented without adequate, contextualized planning, and without necessary preconditions—such as data-gathering measures—in place.

Rose added that there were doubts about how more recent developments in Ethiopia—notably the double shock of COVID-19 and conflict—would affect the arrangements. "Some of the reforms to which the funding is tied have inevitably ground to a halt since 2018," she said. "It is not entirely clear who will be responsible when results aren't achieved in this context, and what sort of funding the government might eventually receive."

The research is published in Third World Quarterly.

More information: Is the Programme for Results approach fit for purpose? Evidence from a large-scale education reform in Ethiopia, Third World Quarterly (2022). DOI: 10.1080/01436597.2022.2047920