Researchers develop 'personalized advantage index,' a new decision-making tool

January 8, 2014

One of the primary social motivations for scientific research is the ability to make better decisions based on the results. But whether it is deciding what material to use in making a solar panel, what antibiotic to use on an infection or when to launch a satellite, most decisions involve weighing multiple factors, all of which interact with one another in determining the best course of action.

Now, researchers at the University of Pennsylvania and the University of Pittsburgh have developed a decision-making model that compares and weights multiple variables in order to predict the optimal choice.

They tested their model on data from a study of patients seeking treatment for depression, who received either or medication. By using the model to generate a score for each patient that indicated which treatment was likely to be more effective for him or her, researchers showed an advantage equivalent to that of an effective treatment relative to a placebo.

Called the "predictive advantage index," this analytic tool could be used not just in but in any decision-making scenario with complex, and potentially conflicting, variables.

"If you pay attention to only one variable, you're going to make a decision that is only true with all else being equal," said Robert DeRubeis, professor and chair of the Department of Psychology in Penn's School of Arts and Sciences. "But we know that all else is not equal. We need to take all of those inequalities into account at once to find out what is likely to work the best."

The study was led by DeRubeis and Zachary Cohen, a doctoral candidate in Psychology. Nicholas Forand, Lois Gelfand and Lorenzo Lorenzo-Luaces, also of Psychology, contributed to the research. The Penn team collaborated with Jay C. Fournier, assistant professor of psychiatry at the University of Pittsburgh.

It was published in the journal PLOS ONE.

In developing the predictive advantage index, the researchers chose a decision-making case with which they were familiar as psychologists: which treatment would be more effective for a depressed patient, cognitive behavioral therapy or medication? Both types of treatments have been shown to be effective in combating depression, but some patients respond better to one type than the other.

The researchers drew upon a longitudinal study of 154 patients that received one of those two treatments. The study collected data on the success of the treatment each patient received, as well as personal information that might play a role in which of the treatments each patient was best suited for, such as marital status, number of previous exposures to antidepressant medications or number of negative life events experienced in the past year. Psychologists have long used this type of information to make clinical recommendations but have never been able to look at all of the relevant data in a comprehensive way.

"The status quo for many decades," Cohen said, "has been for clinicians to either use clinical judgment, intuition based on what they have done before and the results they've seen, or to use a single variable to push the decision in one direction or the other."

Many randomized control trials have tackled relevant single variables, such as marital status or negative life events, to determine whether they are indicative of one type of treatment's effectiveness over another. While such studies have occasionally provided strong evidence of certain traits' correlation with the effectiveness of a given treatment, they have a serious limitation when it comes to applying them to clinical decision-making.

"These studies always look at those variables in isolation," Cohen said. "That leaves clinicians with little ability to go beyond those single variables. What happens if you have a patient come in with conflicting variables? Which one do you trust?"

In building the predictive advantage index, the researchers' goal was not to determine which variable to trust above all others but to see the degree each variable played in producing the outcome. They used a "leave-one-out" validation method, where they cycled though each of the patients, using the relationships between the variables and outcomes for the other 153 to predict the results for the 154th. This method was important to avoid "overfitting," or an over-use of variables that leads to an accurate prediction for those whose data are used to develop the predictive algorithm but fails to be fully generalizable to those outside the data set.

The result of this statistical technique is an algorithm that maximizes the predictive value of the group of variables by assigning weights to each one. This provides useful context for the variables that both intuition and isolated studies suggest may play a role in a given decision's outcome, as well as a new lens to look at the efficacy of those individual variables.

"Personalized medicine approaches might involve genetic assays or neuroimaging or other kinds of diagnostics," Cohen said. "The cost of all of this data gathering needs to be weighted against the value it provides to the ultimate predication of efficacy, which is something [the predictive advantage index] can help us investigate."

While the researchers cannot yet say how accurate the personalized advantage index's predictions will be outside the context of their test case, the early results suggest it would have clinical value.

"If this treatment selection approach had been used when the patients in our data set were actually being treated," DeRubeis said, "it would likely have produced an average benefit that's equivalent to what you see between groups given medication and groups given a placebo."

Machine learning and other artificial intelligence techniques could further improve the index by comparing variables in complex, non-linear ways, opening the door to a new way of doing analytics for many types of applications, including ones with more complicated variables and outcomes.

"This is a way to begin to close the chasm between the wealth of information on how to improve outcomes and how that information is actually applied," DeRubeis said.

Explore further: Inflammatory biomarkers improve the clinical prediction of mortality in COPD

Related Stories

Recommended for you

Android's Nougat update isn't flashy, but still pretty handy

September 28, 2016

Nougat, Google's latest update of its Android smartphone software, isn't particularly flashy; you might not even notice what's different about it at first. But it offers a number of practical time-saving features, plus a ...

Disabled man gets license, shows driverless tech's potential

September 28, 2016

Former Indy Racing League driver Sam Schmidt has done a lot in the 16 years since an accident left him paralyzed from the neck down. He runs a racing team and a foundation. He's raced a sailboat using his chin. But the man ...

Microsoft teams with Bank of America on 'blockchain'

September 27, 2016

Microsoft and Bank of America Merrill Lynch on Tuesday announced they are working together to make financial transactions more efficient with blockchain technology—the foundation of bitcoin digital currency.

Pilots, air traffic controllers shifting to text messaging

September 27, 2016

Airline pilots and air traffic controllers are on schedule to switch to text communications at most of the nation's busiest airports by the end of the year, a milestone that holds the potential to reduce delays, prevent errors ...

Hyperloop pushes dream of low-cost futuristic transport

September 23, 2016

Is it a plane, is it a train? No, say supporters of Hyperloop, a futuristic mode of transport floated by Silicon Valley billionaire Elon Musk that promises high-tech, high-speed and cheap travel over long distances.

MIT's flea market specializes in rare, obscure electronics

September 25, 2016

Once a month in the summer, a small parking lot on the Massachusetts Institute of Technology's campus transforms into a high-tech flea market known for its outlandish offerings. Tables overflow with antique radio equipment, ...


Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.