How statistical thinking should shape the courtroom

April 25, 2017 by Daniel J. Denis, The Conversation
Courtroom decisions are more like a game of chance than you may think. Credit: aerust/flickr, CC BY

The probabilistic revolution first kicked off in the 1600s, when gamblers realized that estimating the likelihood of an event could give them an edge in games of chance.

Today, statistics has become the dominant way to communicate scientific findings. But courts can be hesitant to incorporate statistical into decisions. Indeed, they have historically been antagonistic toward probabilities and are loath to be swindled by slippery statistics.

However, as an educator of statistics who has consulted in a variety of contexts and has served as expert witness to the U.S. District Court in Montana, I find that both my experience and my review of the evidence suggest that courts increasingly feature statistical thinking – whether or not it is identified as such.

Society needs to prioritize educating juries in the language of statistics. Otherwise, juries will be forever at the mercy of convincing, yet potentially invalid, testimony. Courtroom decisions should be based on facts and probabilities, not manipulation by a skilled prosecutor or defense attorney.

Thinking statistically

Probabilities changed the way human beings thought about outcomes. They are a useful tool for expressing our uncertainty about events in the world.

Will it rain today? It will or it will not, that much is certain. But probability allows us to express our ignorance about whether it will rain and quantify the degree to which we are uncertain. Stating "it will probably rain today" constituted a very innovative and different way of thinking.

Probabilities play a role in our daily lives, in decisions from whether to take an umbrella to work to whether to purchase flood insurance. We can consider "statistical thinking" to be any situation where probabilities are involved.

To some extent, humans are intuitive statisticians. For instance, research suggests we can revise a belief in the light of new evidence as prescribed by a statistical theorem, if the probabilities are given in a relatively intuitive rather than abstract fashion.

Statistical reasoning pervades many of the conclusions we draw regarding scientific phenomena. Even physics has had to acknowledge the reality of probabilities. So, if the courts use scientific findings as evidence, probabilities should naturally make their way into courtroom decisions.

Evaluating the evidence

If juries do not understand the nature of statistical conclusions, then they will be tempted to believe that scientific evidence is conclusive and deterministic, rather than probabilistic. For example, probabilities show us that cigarette smoking does not necessarily lead to cancer. Rather, extensive nicotine addiction likely leads to cancer.

Evidence can only fit a theory probabilistically. If we flip a coin 10 times and get 10 heads in a row, that suggests the coin may not be fair, but does not "prove" that it is biased.

Consider the analysis of DNA found at the . Is the DNA that of the accused? Maybe. Not definitively. A statistician might say, "The probability of this degree of DNA match occurring by chance is extremely small. The match may be due to chance, but since this probability is so small, we may conclude that it likely did not occur by chance, and use it as evidence against the accused."

Of course, human judgment is fickle. Until jurors are trained to make rational decisions based on facts and probabilities, they will continue to be easily swayed by convincing litigators.

In the 1995 trial of OJ Simpson, for example, the bloody gloves found at the crime scene constituted powerful evidence against the accused. The samples obtained were extremely likely to belong to the defendant.

A statistically educated jury would not fall for Johnnie Cochran's classic defense: "If it does not fit, you must acquit." They would know in advance that no evidence, whatever the kind, fits a theory perfectly.

Cochran's statement was, statistically speaking, utter nonsense. Of course no model fits perfectly, but which is the more probable model? That's the task jurors ultimately face, even if they often perceive it as a "guilt" versus "no guilt" .

Whenever courts work with DNA matches, they must incorporate acceptable risk and error. But if such uncertainty can be quantified accurately, then it can serve as an aid in decision-making.

Statistical thinking indeed plays a role in the decision between guilt and innocence in a . When a renders a "guilty" verdict, there is always the chance that the accused is not guilty, but that the many circumstances of the case simply lined up against him or her to lead the jury to a guilty verdict. In other words, the probability of the observed evidence under the assumption of innocence is so low that the evidence likely occurred under a more probable "narrative" – that of guilt.

But, when we make such a decision, we do so with a risk of error. This could be quite devastating to a defendant falsely put to death when all along he or she was innocent. For example, when researchers applied DNA testing to death row inmates in Illinois, they found that the results exonerated several inmates.

Errors in probability-based decisions can indeed be costly. Without a grasp of how virtually all decisions are based on probabilistic thinking, no jury can be expected to adequately assess any evidence in a rational way.

Base rates

Courts also struggle with whether and how to use base rates, another type of statistical tool.

A base rate is the probability of some characteristic being present in the population. For instance, say an individual takes a diagnostic test for a disease, such as HIV. The probability that she has the disease would be higher if she were sampled from a high-risk group – for example, if she shares needles to support a drug addiction, or engages in promiscuous sex with risky partners.

Courts often ignore base rate information. In Stephens v. State in 1989, the Wyoming Supreme Court heard testimony that "80 to 85 percent of is committed by a close relative of the child." They ultimately dismissed this, concluding that it was difficult to understand how statistical information would help reach a decision in an individual case.

In another case, a justice of the Minnesota Supreme Court proclaimed that she was "at a loss to understand" how base rates could help predict whether a particular person posed a danger to the public.

Part of the problem is that this information can appear biased against the accused. For instance, consider again the defendant accused of child sexual abuse. The that he is guilty might be evaluated in light of the fact that most perpetrators of abuse are relatives of or closely related to the family. This could be interpreted as biasing the evaluation against the accused. However, the courts have considered base rates in employment discrimination cases, an area where perhaps this information seems more naturally relevant (for example, Hazelwood School Districts v. United States).

If the courts are willing to use base rate information in discrimination cases, they should be encouraged to consider them in other cases as well, even if they seem less intuitive.

Learning to think statistically

Courts should make it a priority to instruct juries on how to interpret probabilistic evidence, so that they are not at the mercy of a convincing, yet potentially misleading, prosecutor or expert witness.

For example, juries might learn elementary statistics through coin-flipping lessons. This could help them, at minimum, find a way to think about the usual "beyond a reasonable doubt" instruction in a criminal trial.

When the assumption of innocence is rejected in favor of guilt, one does so with a risk of being wrong. How much risk is a jury willing to tolerate? Five percent? One percent? Surely such risk must also depend on the severity of the proposed punishment. Every decision is an exercise in risk and cost benefit analysis.

Until juries learn elements of statistical thinking, they are likely to continue making verdict decisions without the appropriate framework in mind. Probabilities have taken over the world, and this fact needs to be recognized by the courts.

Explore further: The mathematics of jury size: Statistical model shows several interesting properties of US jury configurations

Related Stories

Can science lead to justice?

December 8, 2004

Science and the English legal system are fundamentally incompatible, Chris Pamplin, editor of the UK Register of Expert Witnesses, writes in Chemistry & Industry magazine. ‘The courts want certainty; science cannot provide ...

Rain or Shine? Computer Models How Brain Cells Reach a Decision

December 14, 2009

(PhysOrg.com) -- Yale University researchers have devised a computer model to explain how the brain makes decisions based on statistical probabilities-as, for instance, when a doctor makes a diagnosis based on several conflicting ...

Study measures bias in how we learn and make decisions

April 26, 2016

Thinking about drawing to an inside straight or playing another longshot? Just remember that while human decision-making is biased by potential rewards, what we know about individual cues that help us to make those decisions ...

Recommended for you

Excavators find tombs buried in Bolivia 500 years ago

November 17, 2018

Archaeologists say they found tombs at a Bolivian quarry containing remains from more than 500 years ago that give an insight into the interaction of various peoples with the expanding Inca empire.

Preventing chemical weapons as sciences converge

November 15, 2018

Alarming examples of the dangers from chemical weapons have been seen recently in the use of industrial chemicals and the nerve agent sarin against civilians in Syria, and in the targeted assassination operations using VX ...

1 comment

Adjust slider to filter visible comments by rank

Display comments: newest first

manfredparticleboard
not rated yet May 08, 2017
"... and are loath to be swindled by slippery statistics."

And yet, are perfectly happy using weasel words, and highly prescribed interpretations that ensure guilt can be interpreted in the widest possible terms and exculpatory evidence is dismissed or diminished. The old trick of forcing yes or no answers to a state that exists in between is probably the most well known one for trapping someone into admitting a certainty exists when the truth can easily be otherwise.

'This court finds you guilty....well, because we made sure of it!'

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.