Study finds limited highlighting boosts reading comprehension

If you scroll through the average student's digital textbook or reading, you will probably see multi-colored streaks scattered everywhere. However, new research reveals that excessive highlighting may do more harm than good.

Researchers at Waterloo excel at creating new technologies, investigating human-technology interactions, and exploring how to mitigate harm. That's why it's no surprise that two computer scientists from Waterloo investigated whether technology controlling the number of words a user can highlight could affect their reading comprehension.

The work earned the duo the Best Paper Award at the Conference on Human Factors in Computing Systems (CHI 2024) one of the top-ranked conferences in computer science and the leading international HCI conference.

The research, "Constrained Highlighting in a Document Reader Can Improve Reading Comprehension," was published in the Proceedings of the CHI Conference on Human Factors in Computing Systems.

"There are lots of theories in psychology that show having constraints is really beneficial, especially for encouraging creativity," says Nikhita Joshi (MMath '20), a Ph.D. student specializing in human-computer interaction (HCI) research at the David R. Cheriton School of Computer Science.

"Traditionally, software constraints were mainly used for error-proofing. However, my research focuses on using constraints to influence positive outcomes for users, which I call 'bounded interactions.'"

These theories inspired Joshi and her supervisor Dr. Daniel Vogel to recruit 127 participants to read a short story. After 24 hours, they completed a reading comprehension test, answering 20 multiple-choice questions within five minutes. The participants were divided into three groups: no highlights, limited highlights of 150 words and unlimited highlights.

The duo chose this restraint after running a similar experiment where participants were assigned different conditions such as 50 words, 150 words and 250 words. The group capped at 150 words had the highest test results, leading to promising research directions.

For this study, Joshi designed a web-based document reader using React JS in JavaScript, which hosted the reading and test interfaces. This tool can notify how many words a user highlights and if they are exceeding the limit.

Notably, the group with limited highlights scored the highest on the reading comprehension test, with scores 11% higher than the unlimited highlights, and 19% higher than the no highlights groups. This difference is equivalent to one to two letter grades. There were also no noticeable differences in each group's reading time, showing that highlighting restraints does not impede a user's mental demand, effort or frustration.

This research was the first to prove that restricted highlighting can boost reading comprehension. It is also the first to explore user interface constraints for text marking. Excessive highlighting has been a long-standing problem in pedagogy. Although many researchers have proposed solutions such as in-person self-regulation training, it can be time-consuming and strenuous. As a result, this innovative technology may be a faster and easier solution for better study habits.

In a follow-up questionnaire, most of the participants stated that the cap prompted them to concentrate on the most important parts of the story. Their highlights were shorter and focused on keywords like nouns, which are strategies recommended by some university learning centers.

What surprised the researchers the most was that some participants were reluctant to delete highlights despite having the option. Instead, they were more mindful, adopting different tactics like "highlighting a noun at the beginning of the sentence and using this as a bookmark to refer to later in the document," Joshi shares.

This research comes at a time when literacy scores are declining across the globe. "There are concerns that the development of students' cognitive skills may be declining, especially with the development of tools like ChatGPT," Joshi says. "Reading is core to other cognitive skills like self-reflection, self-regulation, and critical thinking. All of that is super important to human intelligence."

In the future, the group may launch a prototype of their current work. Joshi also wants to conduct a similar experiment but with non-fiction documents like academic papers or news articles. However, she would investigate and impose a different condition since 150 words may be too restrictive for long and complex reading.

One idea is to have a limit relative to the document structure. "Instead of having this one encompassing limit for the entire research paper, we would split the limit into different sections," she says. "Perhaps the related work section can have a lower highlight limit, but the result sections could be higher since that's the most important part of the paper."