Billions of dollars on AIDS prevention: Did any of it work?

July 25, 2018 by Trevor Butterworth, American Statistical Association

In January 2003, President George W. Bush asked Congress to back the President's Emergency Plan for AIDS Relief (PEPFAR) with a $15 billion commitment "to turn the tide against AIDS." Congress agreed, and the program—which was continued by the Obama and Trump administrations—has, as noted in The New England Journal of Medicine, "had an unprecedented impact on the pandemic of HIV and AIDS."

But how do we really know that? Yes, HIV rates have—with the exception of a few countries—declined, but can we say the programs PEPFAR funded in countries with the highest prevalence of HIV and AIDS were responsible for those declines? After all, childhood mortality rates from AIDS dropped in sub-Saharan countries that did not receive PEPFAR funding.

Unfortunately, PEPFAR didn't create a statistical plan at the outset that could easily answer these questions. A statistical plan is a bit like asking your accountant what data you should record before you start your business—so you'll know whether you're making or losing money. The question facing PEPFAR was whether they had collected enough data over the years to evaluate their program interventions.

These kinds of after-the-fact analyses are fraught with difficulty. "Planned evaluations are preferable, of course, but we can still learn a lot from post-hoc evaluations," says Donna Spiegelman, the newly appointed Susan Dwight Bliss Professor of Biostatistics at the Yale School of Public Health. "I would never let the perfect be the enemy of the good."

Few were as qualified as Spiegelman to take on the challenge of figuring out whether PEPFAR had worked. With a joint doctorate from Harvard in biostatistics and epidemiology and rigorous training in mathematical statistics, Spiegelman had become an expert in developing methods to evaluate prevention science. In 2014, she was the first (and as yet only) biostatistician to receive the National Institutes of Health Director's Pioneer Award, a sort of MacArthur genius grant for biomedical researchers that frees and funds them—$500,000 per year for five years—to pursue bold, risky, innovative research.

At around the same time as Spiegelman received the Pioneer Award, she met the newly appointed U.S. Global AIDS Coordinator and Ambassador-at-Large Deborah Birx. Birx, a physician and former colonel in the U.S. Army, had been director of the Centers for Disease Control and Prevention's Division of Global HIV/AIDS and worked as a physician in Kenya. As Kenya was one of the original recipients of PEPFAR funding, Birx thought it would be a good candidate for Spiegelman to look at; it would be the most data-rich environment among PEPFAR's recipients.

With doctoral student Dale Barnhart, Spiegelman focused on 10,000 clinics running what they call PMTCT—prevention of mother-to-child transmission—of AIDS. PEPFAR had spent $248 million supporting PMTCT in Kenya between 2004 and 2014, and for those aged under five years had halved during this period. "The basic idea," says Spiegelman, "was to see if child mortality went down as funding went up."

The rationale behind the intervention is that, in order to prevent the transmission of HIV to children, pregnant mothers have to be tested. "If they are tested," says Spiegelman, "they can be treated—and that treatment will drastically lower the transmission rate from 30 or 40 percent to below 1 percent."

That was the basic idea, but teasing out causal inferences was anything but simple. Thankfully, as PEPFAR is a U.S. government program, it has to report to Congress how much it spends each quarter. From this data, Barnhart extracted a decade's worth of spending on PMTCT. They then obtained neonatal and data from the Kenyan government and other agencies.

By doing a "difference in difference" analysis, they were able to look at the effects of varying levels of funding for PMTCT among the Kenyan provinces over time, along with trends in local infant mortality, and then compare them to other provinces with similar demographics that did not have PMTCT programs—or had them but with less financial support. The team did a wide range of sensitivity analyses to model missing data, but these had little impact on the outcome.

"The results were quite dramatic," says Spiegelman. Provinces in the top quartile of PEPFAR spending in any given year over the study period experienced more than 30 percent lower infant mortality than those in the bottom quartile. "Cumulative PMTCT spending significantly reduced infant mortality, saving the lives of many Kenyan children," she says. "We've shown, and rigorously, that the U.S. investment in PEPFAR delivered on its promise."

All of this research was carried out while Spiegelman was professor of epidemiologic methods in the departments of epidemiology, biostatistics, nutrition and global health at the Harvard T.H. Chan School of Public Health. Since July 1, she has been director of the new Center for Methods in Implementation and Prevention Science (CMIPS) at Yale University's School of Public Health. "We're hoping to be the go-to place for implementation and prevention science methods in the world. It is my dream job."

Explore further: 15 years later, PEPFAR continues to save lives

More information: … fm?abstractid=329908

Related Stories

15 years later, PEPFAR continues to save lives

January 24, 2018

Experts from the National Institute of Allergy and Infectious Diseases (NIAID), part of the National Institutes of Health, have penned a New England Journal of Medicine perspective recognizing the United States President's ...

Health groups urge Congress not to allow AIDS fight to wane

November 29, 2017

A coalition of nearly 40 advocacy groups said Wednesday they're concerned about the Trump administration's commitment to the global fight against AIDS so they're urging senior members of Congress to make sure money for key ...

New IOM report highlights PEPFAR's successes

February 20, 2013

The President's Emergency Plan for AIDS Relief (PEPFAR) has saved and improved millions of lives worldwide and offered proof that HIV/AIDS services can be effectively delivered on a large scale even in countries with high ...

Recommended for you

Physicists 'flash-freeze' crystal of 150 ions

February 20, 2019

Physicists at the National Institute of Standards and Technology (NIST) have "flash-frozen" a flat crystal of 150 beryllium ions (electrically charged atoms), opening new possibilities for simulating magnetism at the quantum ...

The holy grail of nanowire production

February 20, 2019

Nanowires have the potential to revolutionize the technology around us. Measuring just 5-100 nanometers in diameter (a nanometer is a millionth of a millimeter), these tiny, needle-shaped crystalline structures can alter ...


Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.