This article has been reviewed according to Science X's editorial process and policies. Editors have highlighted the following attributes while ensuring the content's credibility:

fact-checked

peer-reviewed publication

proofread

Experiment reveals why pundits can be unreliable

Experiment reveals why pundits can be unreliable
Credit: Weiwei Chen

Decisions by policymakers and managers often rely on predictions and forecasts provided by experts. For instance, policymakers may increase public investments if they anticipate economic stagnation, and companies may introduce new products if they anticipate sufficient demand. The predictions of future trends by experts play a crucial role in informing these decisions. Thus, the accuracy of experts is closely monitored, and forecasters with a proven track record of accurate predictions may have significant career opportunities.

The need for a good reputation, though, can lead to misreport information that might reflect negatively on their reputation for being well informed, according to a series of experiments reported by Marco Ottaviani and Salvatore Nunnari (Bocconi Department of Economics) along with Debrah Meloso (Toulouse Business School) in a paper published in Management Science.

The authors designed an urn and balls scheme. Each ball has an outer shell and an inner core. The inner core itself is either blue or orange. The outer shell is opaque (it does not allow to see the core's color) and is also either blue or orange. There are two 10-ball urns corresponding to the quality of information of the forecaster. The inner core perfectly matches the color of the outer shell for all the balls contained in the informative urn, corresponding to a forecaster who perfectly knows the future. The uninformative urn captures a forecaster who has no ability to predict the future: the color of the shell is independent of the color of the core.

The game proceeds as follows: First, a ball is drawn from either the informative or the uninformative urn with equal probability, but neither the forecaster nor the evaluator know which urn the ball is coming from. Second, the forecaster sees the color of the (but not the color of the inner core) of the drawn ball and reports it to the evaluator. The evaluator then observes the color of the and assesses the probability that the forecaster has observed a ball drawn from an informative urn. The forecaster is compensated according to the evaluator's assessment that the ball was drawn from an informative urn.

Both the forecaster and the evaluator are shown the panel displayed above, informing them of the real distribution of the blue and orange cores, which is the same in the two urns. Sometimes it is 6/4 (simulating a situation of high uncertainty), sometimes 8/2 (simulating a less uncertain situation). The uninformative urn always contains five blue shells and five orange ones, as displayed in the figure.

Intuitively, in the less uncertain condition (8/2), even after seeing an orange shell the forecaster thinks that a blue core is more likely than an orange core. In fact, four of the seven orange shells must have a blue core. So, by misreporting, the forecaster can increase the probability of correctly guessing the core.

The experiment broadly confirms that the forecasters have an incentive to misreport to safeguard their reputation and that there is more misreporting the more certain is the situation. In the more uncertain condition (6/4) misreporting is at 51%, while in the more certain condition (8/2) misreporting is at 63%.

"Our results have implications for the use of expert advice as input of managerial decision-making and the design of markets for professional forecasting," Prof. Ottaviani says. "Our experimental evidence suggests that firms should trust experts' advice when the phenomenon to forecast is more uncertain. On the other hand, when the firm already has and the relevant variables are less uncertain, expert advice is not only less valuable but also less trustworthy."

"Furthermore," adds Prof. Nunnari, "the evaluators' approach plays a role. We found that experts have a strong incentive to reveal their truthfully when their reputation is strongly affected by the ex-post accuracy of their statements. Evaluators should link their evaluation to expert's ex-post accuracy, rather than to experts' advice, thus reducing the ' incentives to misreport information."

More information: Debrah Meloso et al, Looking into Crystal Balls: A Laboratory Experiment on Reputational Cheap Talk, Management Science (2023). DOI: 10.1287/mnsc.2022.4629

Journal information: Management Science

Provided by Bocconi University

Citation: Experiment reveals why pundits can be unreliable (2023, February 8) retrieved 28 March 2024 from https://phys.org/news/2023-02-reveals-pundits-unreliable.html
This document is subject to copyright. Apart from any fair dealing for the purpose of private study or research, no part may be reproduced without the written permission. The content is provided for information purposes only.

Explore further

Quantum dot ring lasers emit colored light

2 shares

Feedback to editors