March 23, 2016 report
Statistics suggests that unanimous agreement in witnessed events may be sign of an error
(Phys.org)—A team of researchers with The University of Adelaide and one with University of Angers has found that the probability of a unanimous agreement in witnessed events is low enough that instances of such are likely a sign of an error. In their paper published in Proceedings of the Royal Society A, the researchers suggest their findings could have an impact on fields as diverse as legal proceedings, archaeological assessments and even cryptographic testing.
When a jury is given testimony by many witnesses to a crime, all fingering the same person, the consensus is generally that the police have caught the right person. But, a statistical assessment of such instances by the research team suggests that may not always be the case. They suggest that the opposite may in fact be true, that the more witnesses fingering the same person, the greater likelihood there is that the wrong person has been caught.
The reasoning by the team goes along the lines of logic; if 100 people observe an apple sitting on an otherwise bare table and all confirm it was an apple, than there is a strong likelihood that it was an apple sitting there. But, what happens when the observation is not so simple—for example, what if 100 people see a man carrying a bag of money out of a bank after a robbery, and all 100 agree that it was the man police have identified as the robber. That might be a problem because prior research has shown that when asked to identify a person that witnesses have seen for just a few seconds, especially if that person is running away, can be as low as 50 percent correct. When performing Bayesian analysis on such scenarios, the team reports, the numbers grow worse as the number of people unanimously agrees on something they believe they have seen. Put another way, statistically speaking, it is nearly impossible for 100 people to all correctly identify a person in such a situation—thus, if they do, it calls into question the validity of the results.
The researchers note that their findings apply to other areas as well—if 100 archeologists agree on the source of a find, for example, the odds are great that there is an error somewhere, because statistics suggests there should be at least some differences in the results.
More information: Lachlan J. Gunn et al. Too good to be true: when overwhelming evidence fails to convince, Proceedings of the Royal Society A: Mathematical, Physical and Engineering Science (2016). DOI: 10.1098/rspa.2015.0748
Abstract
Is it possible for a large sequence of measurements or observations, which support a hypothesis, to counterintuitively decrease our confidence? Can unanimous support be too good to be true? The assumption of independence is often made in good faith; however, rarely is consideration given to whether a systemic failure has occurred. Taking this into account can cause certainty in a hypothesis to decrease as the evidence for it becomes apparently stronger. We perform a probabilistic Bayesian analysis of this effect with examples based on (i) archaeological evidence, (ii) weighing of legal evidence and (iii) cryptographic primality testing. In this paper, we investigate the effects of small error rates in a set of measurements or observations. We find that even with very low systemic failure rates, high confidence is surprisingly difficult to achieve; in particular, we find that certain analyses of cryptographically important numerical tests are highly optimistic, underestimating their false-negative rate by as much as a factor of 280.
Journal information: Proceedings of the Royal Society A
© 2016 Phys.org
