Automated feedback: The new science of grading STEM papers

September 13th, 2017
Wake Forest biology professors Dan Johnson and Sabrina Setaro talk with graduate students Noah Bressman and Jenny Howard about new methods for grading science writing, in a lab in Winston Hall. Credit: WFU / Ken Bennett
Science educators at Wake Forest University are testing how automated feedback combined with new one-on-one teaching methods can improve scientific writing from STEM undergraduates - and result in better explanation of research to the public.

The National Science Foundation (NSF) has awarded the research team nearly $280,000 to conduct the writing experiment. It comes at a pivotal time, with claims of junk science and fake news making headlines almost daily.

"Students don't see that writing is important in the sciences," said Dan Johnson, who leads the research team and is a professor in the Biology Department. "They don't see much value in it. They don't realize we actually live and die by our writing."

The STEM (science, technology, engineering and math) writing study will build on an automated lab report feedback system called SAWHET (pronounced saw-it). Undergraduate science students have been using SAWHET to submit and revise lab reports online for two years.

SAWHET (or STEM Automated Writing Help Tool) guides students to break down their lab reports into the proper sections of a scientific paper - title, extract and citations, for example. Help buttons explain those sections, and the software can ask some basic editing questions: Do you need a citation to back up this statement? Do you state your hypothesis? Once the writer addresses those questions, the report is re-submitted.

With the NSF grant, the researchers will teach SAWHET to ask additional content-focused questions, allowing instructors and teaching assistants to concentrate on providing deeper, one-on-one feedback.

Time has been the primary challenge to teaching writing more aggressively to STEM students. One college course can include 150 students. If the instructor gives a writing assignment and edits it twice for about 15 minutes each, that can add up to 70 hours of work.

But writing and communication skills are increasingly important to employers. A report by the Association of American Colleges & Universities found that 80 percent of employers surveyed wanted colleges to emphasize written and oral communication skills more.

"Employers are often complaining that writing skills are poor," said Sabrina Setaro, program director of the STEM writing project. "Writing helps students be concise, order thinking, create a story and build an argument."

That's why the Wake Forest study also will introduce new methods that instructors will use for teaching and evaluating students' scientific writing. Instructors will be encouraged to let SAWHET do the low-level proofing and format checking, so they can do a deeper dive into the logic, arguments and thinking behind the written words.

"We're seeing places where people are using less effective practices because it's familiar - but we've got evidence that shows that doesn't help a student progress," Setaro said.

This isn't the first time Johnson and Setaro have flipped the undergraduate science curriculum. They both work on BioBook, a collaborative, interactive online biology resource that replaces traditional written textbooks and encourages inquiry-based learning. The basic edition is free; a customizable version is available, for which students pay $30 per year for access. About 750 students and professors at two universities are using it right now.

SAWHET is an offshoot of BioBook.

"Students were using the assessments in BioBook, and we realized that writing was a major barrier for them," Johnson said. "We wanted to give students a chance to assess their writing, too."

Provided by Wake Forest University

This Phys.org Science News Wire page contains a press release issued by an organization mentioned above and is provided to you “as is” with little or no review from Phys.Org staff.

More news stories

Two teams independently test Tomonaga–Luttinger theory

(Phys.org)—Two teams of researchers working independently of one another have found ways to test aspects of the Tomonaga–Luttinger theory that describes interacting quantum particles in 1-D ensembles in a Tomonaga–Luttinger ...

Dawn mission extended at Ceres

NASA has authorized a second extension of the Dawn mission at Ceres, the largest object in the asteroid belt between Mars and Jupiter. During this extension, the spacecraft will descend to lower altitudes than ever before ...

Close up view of growing polymer chain show jump steps

(Phys.org)—A team of researchers at Cornell University has devised a means for watching as a polymer chain grows after application of a catalyst. In their paper published in the journal Science, the team explains how they ...

Breakthrough in ultra-fast data processing at nanoscale

A research team from the National University of Singapore has recently invented a novel "converter" that can harness the speed and small size of plasmons for high frequency data processing and transmission in nanoelectronics.