This article has been reviewed according to Science X's editorial process and policies. Editors have highlighted the following attributes while ensuring the content's credibility:


peer-reviewed publication

trusted source


Q&A: Using social science to forecast societal change

Using social science to forecast societal change
Social scientists’ average forecasting errors, compared against different benchmarks. We ranked the domains from least to most error in Tournament 1, assessing forecasting errors via the MASE. The estimated means for the scientists and the naive crowd indicate the fixed-effect coefficients of a linear mixed model with domain (k = 12) and group (in Tournament 1: Nscientists = 86, Nnaive crowd = 802; only scientists in Tournament 2: N = 120) as predictors of forecasting error (MASE) scores nested in teams (Tournament 1 observations: Nscientists = 359, Nnaive crowd = 1,467; Tournament 2 observations: N = 546), using restricted maximum likelihood estimation. To correct for right skew, we used log-transformed MASE scores, which were subsequently back-transformed when calculating estimated means and 95% CIs. In each tournament, the CIs were adjusted for simultaneous inference of estimates for 12 domains in each tournament by simulating a multivariate t distribution. The benchmarks represent the naive crowd and the best-performing naive statistical benchmark (historical mean, average random walk with an autoregressive lag of one or linear regression). Statistical benchmarks were obtained via simulations (k = 10,000) with resampling (Supplementary Information). Scores to the left of the dotted vertical line show better performance than a naive in-sample random walk. Scores to the left of the dashed vertical line show better performance than the median performance in M4 tournaments. Credit: Nature Human Behaviour (2023). DOI: 10.1038/s41562-022-01517-1

Pandemic impacts on people everywhere brought a unique opportunity for social scientists to study and forecast changes in society. The Forecasting Collaborative was founded in 2020 by Dr. Igor Grossmann, associate professor of psychology at the University of Waterloo, to evaluate the accuracy of social and data scientists' approaches in predicting social change.

What was the major finding of your study?

The study revealed that social scientists were overall no better at predicting how major social issues—well-being, prejudice toward minorities, or political polarization—would unfold over the first year of the pandemic than the or statistics such as the historical average. We did find that consideration of prior data and multidisciplinary research teams were somewhat more accurate.

Since no one can predict the future, how should social scientists engage in social commentary?

Scientists should be mindful of their limitations. Along with consideration of base-rate information and an appreciation of temporal dynamics in their studies, social scientists would benefit from more intellectual humility and consider if their lab studies scale up to what happens "in the wild."

In addition, they should avoid predictions without necessary caveats. For instance, a researcher said at a major scientific conference that their theory "predicted everything in the pandemic." It is statements like this that damage the credibility of social scientists.

Is there a risk of eroding public trust in social science research?

Yes, especially if one makes predictions about things one has little knowledge about –one may know how to explain a particular phenomenon but not know how to predict the next steps in this phenomenon. Social scientists should be aware of the limits of their knowledge, and junior scientists should receive better training in prediction methods to mitigate this risk.

"Insights into accuracy of ' forecasts of societal change," by Grossmann et al, is published in the journal Nature Human Behaviour.

More information: Igor Grossmann et al, Insights into the accuracy of social scientists' forecasts of societal change, Nature Human Behaviour (2023). DOI: 10.1038/s41562-022-01517-1

Journal information: Nature Human Behaviour

Citation: Q&A: Using social science to forecast societal change (2023, February 10) retrieved 10 June 2023 from
This document is subject to copyright. Apart from any fair dealing for the purpose of private study or research, no part may be reproduced without the written permission. The content is provided for information purposes only.

Explore further

Researchers find that certain social factors can predict early death in older adults


Feedback to editors