American Statistical Association releases statement on statistical significance and p-values

March 7, 2016

The American Statistical Association (ASA) has released a "Statement on Statistical Significance and P-Values" with six principles underlying the proper use and interpretation of the p-value. The ASA releases this guidance on p-values to improve the conduct and interpretation of quantitative science and inform the growing emphasis on reproducibility of science research. The statement also notes that the increased quantification of scientific research and a proliferation of large, complex data sets has expanded the scope for statistics and the importance of appropriately chosen techniques, properly conducted analyses, and correct interpretation.

Good statistical practice is an essential component of good scientific practice, the statement observes, and such practice "emphasizes principles of good study design and conduct, a variety of numerical and graphical summaries of data, understanding of the phenomenon under study, of results in context, complete reporting and proper logical and quantitative understanding of what data summaries mean."

"The p-value was never intended to be a substitute for scientific reasoning," said Ron Wasserstein, the ASA's executive director. "Well-reasoned statistical arguments contain much more than the value of a single number and whether that number exceeds an arbitrary threshold. The ASA statement is intended to steer research into a 'post p<0.05 era.'"

"Over time it appears the p-value has become a gatekeeper for whether work is publishable, at least in some fields," said Jessica Utts, ASA president. "This apparent editorial bias leads to the 'file-drawer effect,' in which research with statistically significant outcomes are much more likely to get published, while other work that might well be just as important scientifically is never seen in print. It also leads to practices called by such names as 'p-hacking' and 'data dredging' that emphasize the search for small p-values over other statistical and scientific reasoning."

The statement's six principles, many of which address misconceptions and misuse of the p-value, are the following:

  1. P-values can indicate how incompatible the data are with a specified statistical model.
  2. P-values do not measure the probability that the studied hypothesis is true, or the probability that the data were produced by random chance alone.
  3. Scientific conclusions and business or policy decisions should not be based only on whether a p-value passes a specific threshold.
  4. Proper inference requires full reporting and transparency.
  5. A p-value, or , does not measure the size of an effect or the importance of a result.
  6. By itself, a p-value does not provide a good measure of evidence regarding a model or hypothesis.

The statement has short paragraphs elaborating on each principle.

In light of misuses of and misconceptions concerning p-values, the statement notes that statisticians often supplement or even replace p-values with other approaches. These include methods "that emphasize estimation over testing such as confidence, credibility, or prediction intervals; Bayesian methods; alternative measures of evidence such as likelihood ratios or Bayes factors; and other approaches such as decision-theoretic modeling and false discovery rates."

"The contents of the ASA statement and the reasoning behind it are not new—statisticians and other scientists have been writing on the topic for decades," Utts said. "But this is the first time that the community of statisticians, as represented by the ASA Board of Directors, has issued a statement to address these issues."

"The issues involved in statistical inference are difficult because inference itself is challenging," Wasserstein said. He noted that more than a dozen discussion papers are being published in the ASA journal The American Statistician with the statement to provide more perspective on this broad and complex topic. "What we hope will follow is a broad discussion across the scientific community that leads to a more nuanced approach to interpreting, communicating, and using the results of statistical methods in research."

Explore further: ASA issues top five 'Choosing wisely' recommendations

More information: dx.doi.org/10.1080/00031305.2016.1154108

Related Stories

ASA issues top five 'Choosing wisely' recommendations

January 29, 2014

(HealthDay)—The top five anesthesiology-related pain medicine issues that physicians and patients should question have been released by the American Society of Anesthesiologists (ASA) as part of the Choosing Wisely campaign.

New guide to data interpretation

June 5, 2014

Western blotting is a widely used technique to detect specific proteins. Although considered a semi-quantitative method, the results are often interpreted quantitatively. Scientific articles often do not specify how researchers ...

Four-times daily ASA more effective in post-CABG patients

January 16, 2015

(HealthDay)—For patients undergoing coronary artery bypass graft (CABG) surgery, four-times daily acetyl-salicylic acid (ASA) seems more effective than once-daily 81 mg or 325 mg ASA, according to a study published online ...

Statistics is the fastest-growing undergraduate STEM degree

February 3, 2015

Statistics—the science of learning from data—is the fastest-growing science, technology, engineering and math (STEM) undergraduate degree in the United States over the last four years, an analysis of federal government ...

ASA issues statement on role of statistics in data science

October 1, 2015

In a policy statement issued today, the American Statistical Association (ASA) stated statistics is "foundational to data science"—along with database management and distributed and parallel systems—and its use in this ...

Recommended for you

0 comments

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.