Study shows 'readability' scores are largely inaccurate

Jan 08, 2014 by Matt Shipman

(Phys.org) —Teachers, parents and textbook companies use technical "readability" formulas to determine how difficult reading materials are and to set reading levels by age group. But new research from North Carolina State University shows that the readability formulas are usually inaccurate and offer little insight into which age groups will be able to read and understand a text.

"Teachers often use readability levels when giving reading assignments to ," says Dr. John Begeny, an associate professor of psychology at NC State and lead author of a paper describing the work. "We wanted to know if the readability formulas are valid, or if teachers who think they're assigning a simpler book to struggling readers, for example, may actually be assigning a more difficult one."

For the study, researchers had 360 students – ranging from second to fifth grade – read six written passages out loud. The researchers assessed the students' performance, giving each student an "oral reading fluency" score, which is considered a good metric for measuring reading ability.

The researchers then used eight different readability formulas to see which level each formula gave to the six written passages. Results varied widely, with one passage being rated from first grade to fifth grade level.

The levels assigned by the readability formulas were then compared with researchers' assessments of each student's actual ability to read the material. Seven of the eight readability formulas were less than 49 percent accurate, with the worst formula scoring only 17 percent accuracy. The highest-rated formula was accurate 79 percent of the time.

"Overall, this work shows that and parents should be very cautious about using readability levels when giving reading assignments to students," Begeny says.

Explore further: Interview blues—anxious, slow talkers often do not get the job

More information: The paper, "Can Readability Formulas Be Used to Successfully Gauge Difficulty of Reading Materials?" is published in the January issue of the journal Psychology in the Schools. onlinelibrary.wiley.com/doi/10… /pits.21740/abstract

Abstract: A grade level of reading material is commonly estimated using one or more readability formulas, which purport to measure text difficulty based on specified text characteristics. However, there is limited direction for teachers and publishers regarding which readability formulas (if any) are appropriate indicators of actual text difficulty. Because oral reading fluency (ORF) is considered one primary indicator of an elementary aged student's overall reading ability, the purpose of this study was to assess the link between leveled reading passages and students' actual ORF rates. ORF rates of 360 elementary-aged students were used to determine whether reading passages at varying grade levels are, as would be predicted by readability levels, more or less difficult for students to read. Results showed that a small number of readability formulas were fairly good indicators of text, but this was only true at particular grade levels. Additionally, most of the readability formulas were more accurate for higher ability readers. One implication of the findings suggests that teachers should be cautious when making instructional decisions based on purported "grade-leveled" text, and educational researchers and practitioners should strive to assess difficulty of text materials beyond simply using a readability formula.

Related Stories

Twitteracy: Using Twitter to promote academic literacy

Jan 06, 2014

By using Twitter—the social network whose users communicate in "tweets" no more than 140 characters long—a Buffalo teacher engaged a group of high school students in a spirited discussion of a novel.

Recommended for you

Accountancy stereotypes add up to stable profession

12 hours ago

The media is littered with celebrity trainers, bakers, nutritionists, even gardeners. But, one profession is always missing from the roster - the celebrity accountant. The reason is most likely due to the negative stereotypes ...

Volunteers not safe from workplace bullying

13 hours ago

Despite the charitable nature of volunteering, new research from Murdoch University and Edith Cowan University has found one in three volunteers can experience workplace bullying.

The problem with solitary confinement

13 hours ago

It's a practice that has been in the news since the Ashley Smith case first made headlines and, last week, the Ontario government announced it had launched a review of its solitary confinement policies.

User comments : 0

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.