Scientists devise new method to solve significant variables conundrum

October 26, 2015, Columbia University

Scientists at Columbia University, the University of California, San Diego (UCSD) and Harvard University have presented an alternative method to address the challenge of using significant variables to make useful predictions in areas such as complex disease.

Shaw-Hwa Lo and Tian Zheng of Columbia, Adeline Lo of UCSD and Herman Chernoff of Harvard present findings in a paper to appear in Proceedings of the National Academy of Sciences on Monday, October 26, that demonstrates that statistically significant variables are not necessarily predictive. In addition, very predictive variables do not necessarily have to appear significant and thereby evade a researcher using as a criterion to evaluate variables for prediction.

Statistical significance is a traditional, long-standing measure in any researcher's toolbox but thus far, scientists have been puzzled by the inability to use results of statistically significant variants in complex diseases to make predictions useful for personalized medicine. Why aren't significant variables leading to good prediction of outcomes? This conundrum affects both simple and complex data in a broad range of science and social science fields.

In their findings, the authors demonstrate that what makes variables good for prediction versus significance depends on different properties of the underlying distributions. They suggest that progress in prediction requires efforts toward a new research agenda of searching for a novel criterion to retrieve highly predictive variables rather than highly significant variables.

They also present an alternative approach, the Partition Retention , which displays strong power in prediction. The researchers applied the method to a well-known dataset, the van't Veer dataset, and reduced the prediction error rate from 30% to 8%, finding breast cancer genes that are highly predictive - and not significant.

Their results show that using their method to examine the top five interacting breast cancer genes they were able to find predicted breast cancer relapse effectively, when the outcome would not have appeared using significance measures. Previous methods were only 70% correct in predicting something as significant as breast cancer relapse. Using the new method and avoiding significance as a criterion, scientists correctly predicted such an outcome with 92% accuracy.

"What we're saying here is that using the previously very well-known methods might not be appropriate when we care about predicting important outcomes," says Professor Lo. "Our alternative approach seems to do very well in , and is relevant for many scientific fields."

Explore further: New algorithm aimed at combating science's reproducibility problem

More information: Why significant variables aren't automatically good predictors, www.pnas.org/cgi/doi/10.1073/pnas.1518285112

Related Stories

Drug sensitivity predicted computationally

June 3, 2014

With modern high-throughput technologies researchers can measure a multitude of molecular properties from cancer cells. A big question of precision medicine is how computational modelling can be used to predict sensitivity ...

Team develops prognostic test for E2F4 in breast cancer

January 7, 2015

By looking at the expression levels of downstream genes of the regulators in breast cancer, investigators at Dartmouth Hitchcock's Norris Cotton Cancer Center (NCCC), led by Chao Cheng, PhD, have identified a gene signature ...

Recommended for you

Japan to make crater on asteroid to get underground samples

March 18, 2019

Japan's space agency said Monday that its Hayabusa2 spacecraft will follow up last month's touchdown on a distant asteroid with another risky mission—dropping an explosive on the asteroid to make a crater and then collect ...

0 comments

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.