Researchers apply Benford's law to physics exams to see if they can do better than chance

November 29, 2013 by Bob Yirka, report

Researchers apply Benford’s law to physics exams to see if they can do better than chance
The distribution of leading digits in end-of-chapter excercise answers from two popular introductory physics text- books (Knight, Young & Freedman) and an analytical chem- istry textbook (Skoog). The dashed horizontal line indi- cates uniform distribution of first digits. Statistical analysis confirms conformation to Benford’s Law, overlayed as black squares. Credit: arXiv:1311.4787 []
( —A team of scientists who specialize in multiple-choice test assessment at Brock and Trent University's in Canada has conducted a study to find out if students could use Benford's law to help them pass multiple-choice physics tests. They have posted a paper describing their results on the preprint server arXiv.

Benford's law, named after Frank Benford, a physicist working with probability, says that the numbers 1, 2, and 3 are much more likely to appear at the beginning of multiple digit numbers that are based on real-world measurements of such things as physical constants, the addresses of random people, etc. More specifically, the number 1, the law says, will appear first in a series of digits 30 percent of the time. The number 2 will appear first 18 percent of the time and the number 3 will appear first 13 percent of the time, and so on, with higher numbers appearing less and less often.

This got the team in Canada wondering if students taking a multiple-choice test might be able to do better than random guessing if they followed Benford's law instead—theory suggests that if they used Benford's law to answer all of the questions, they should be able to score at least 51 percent correct—a passing mark. To find out, they created their own 500 question multiple-choice (each with three choices) exam along with a dataset of real answers from a real test. Incorrect answers were represented randomly.

To run the test, the researchers chose the lowest possible number for each answer. In analyzing the results, they found that they could indeed score 51 percent on the test, without any knowledge of underlying physics concepts.

But, there was a catch, when the researchers applied the same science to a real physics , they found they were able to do no better than chance. That was because, they discovered, the wrong answers followed Benford's law as well.

The research team still can't explain why the incorrect answers followed Benford's law, but can report that clever students seeking to apply the law to their multiple choice tests, likely won't fare any better than chance, which would mean they would fail miserably.

Explore further: Number one rules in nature: study

More information: Benford's Law: Textbook Exercises and Multiple-choice Testbanks, arXiv:1311.4787 []

Benford's Law describes the finding that the distribution of leading (or leftmost) digits of innumerable datasets follows a well-defined logarithmic trend, rather than an intuitive uniformity. In practice this means that the most common leading digit is 1, with an expected frequency of 30.1%, and the least common is 9, with an expected frequency of 4.6%. The history and development of Benford's Law is inexorably linked to physics, yet there has been a dearth of physics-related Benford datasets reported in the literature. Currently, the most common application of Benford's Law is in detecting number invention and tampering such as found in accounting-, tax-, and voter-fraud. We demonstrate that answers to end-of-chapter exercises in physics and chemistry textbooks conform to Benford's Law. Subsequently, we investigate whether this fact can be used to gain advantage over random guessing in multiple-choice tests, and find that while testbank answers in introductory physics closely conform to Benford's Law, the testbank is nonetheless secure against such a Benford's attack for banal reasons.

Related Stories

Number one rules in nature: study

February 17, 2011

( -- Researchers from The Australian National University have used a long-forgotten mathematical rule to reveal that in nature the number one dominates, as well as detect natural events like earthquakes for the ...

New Pattern Found in Prime Numbers

May 8, 2009

( -- Prime numbers have intrigued curious thinkers for centuries. On one hand, prime numbers seem to be randomly distributed among the natural numbers with no other law than that of chance. But on the other hand, ...

Recommended for you

New insights into magnetic quantum effects in solids

January 23, 2019

Using a new computational method, an international collaboration has succeeded for the first time in systematically investigating magnetic quantum effects in the well-known 3-D pyrochlore Heisenberg model. The surprising ...

Rapid and continuous 3-D printing with light

January 22, 2019

Three-dimensional (3-D) printing, also known as additive manufacturing (AM), can transform a material layer by layer to build an object of interest. 3-D printing is not a new concept, since stereolithography printers have ...

Scientists discover new quantum spin liquid

January 22, 2019

An international research team led by the University of Liverpool and McMaster University has made a significant breakthrough in the search for new states of matter.

Researchers capture an image of negative capacitance in action

January 21, 2019

For the first time ever, an international team of researchers imaged the microscopic state of negative capacitance. This novel result provides researchers with fundamental, atomistic insight into the physics of negative capacitance, ...

Toward ultrafast spintronics

January 21, 2019

Electronics have advanced through continuous improvements in microprocessor technology since the 1960s. However, this process of refinement is projected to stall in the near future due to constraints imposed by the laws of ...


Adjust slider to filter visible comments by rank

Display comments: newest first

3.7 / 5 (3) Nov 29, 2013
Nothing as useless as a multiple choice test. Especially in the natural science - no matter how you structure it - it fails to test the most important part: whether you have understood the subject AND the implications.

Regurgitating based on a "that looks about right", "I can exclude the other three choices as dumb" or "That seems familiar" is the worst way of deluding yourself into thinking that you have some aptitude in these subjects.
not rated yet Nov 29, 2013
Yup, I have repeatedly scored well on multiple choice tests while having only cursory knowledge of the subject matter. They fail miserably at testing students, but are very easy for the teachers compared to, you know, *real* tests.
not rated yet Nov 29, 2013
antialias_physorg: I accidentally rated your comment one rather than 5. Sorry:(.
5 / 5 (1) Nov 30, 2013
It happens. Don't worry. I'm not here for the ratings.

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.