Automated feedback: The new science of grading STEM papers

September 13th, 2017
Wake Forest biology professors Dan Johnson and Sabrina Setaro talk with graduate students Noah Bressman and Jenny Howard about new methods for grading science writing, in a lab in Winston Hall. Credit: WFU / Ken Bennett
Science educators at Wake Forest University are testing how automated feedback combined with new one-on-one teaching methods can improve scientific writing from STEM undergraduates - and result in better explanation of research to the public.

The National Science Foundation (NSF) has awarded the research team nearly $280,000 to conduct the writing experiment. It comes at a pivotal time, with claims of junk science and fake news making headlines almost daily.

"Students don't see that writing is important in the sciences," said Dan Johnson, who leads the research team and is a professor in the Biology Department. "They don't see much value in it. They don't realize we actually live and die by our writing."

The STEM (science, technology, engineering and math) writing study will build on an automated lab report feedback system called SAWHET (pronounced saw-it). Undergraduate science students have been using SAWHET to submit and revise lab reports online for two years.

SAWHET (or STEM Automated Writing Help Tool) guides students to break down their lab reports into the proper sections of a scientific paper - title, extract and citations, for example. Help buttons explain those sections, and the software can ask some basic editing questions: Do you need a citation to back up this statement? Do you state your hypothesis? Once the writer addresses those questions, the report is re-submitted.

With the NSF grant, the researchers will teach SAWHET to ask additional content-focused questions, allowing instructors and teaching assistants to concentrate on providing deeper, one-on-one feedback.

Time has been the primary challenge to teaching writing more aggressively to STEM students. One college course can include 150 students. If the instructor gives a writing assignment and edits it twice for about 15 minutes each, that can add up to 70 hours of work.

But writing and communication skills are increasingly important to employers. A report by the Association of American Colleges & Universities found that 80 percent of employers surveyed wanted colleges to emphasize written and oral communication skills more.

"Employers are often complaining that writing skills are poor," said Sabrina Setaro, program director of the STEM writing project. "Writing helps students be concise, order thinking, create a story and build an argument."

That's why the Wake Forest study also will introduce new methods that instructors will use for teaching and evaluating students' scientific writing. Instructors will be encouraged to let SAWHET do the low-level proofing and format checking, so they can do a deeper dive into the logic, arguments and thinking behind the written words.

"We're seeing places where people are using less effective practices because it's familiar - but we've got evidence that shows that doesn't help a student progress," Setaro said.

This isn't the first time Johnson and Setaro have flipped the undergraduate science curriculum. They both work on BioBook, a collaborative, interactive online biology resource that replaces traditional written textbooks and encourages inquiry-based learning. The basic edition is free; a customizable version is available, for which students pay $30 per year for access. About 750 students and professors at two universities are using it right now.

SAWHET is an offshoot of BioBook.

"Students were using the assessments in BioBook, and we realized that writing was a major barrier for them," Johnson said. "We wanted to give students a chance to assess their writing, too."

Provided by Wake Forest University

This Science News Wire page contains a press release issued by an organization mentioned above and is provided to you “as is” with little or no review from Phys.Org staff.

More news stories

After a reset, Сuriosity is operating normally

NASA's Curiosity rover is busy making new discoveries on Mars. The rover has been climbing Mount Sharp since 2014 and recently reached a clay region that may offer new clues about the ancient Martian environment's potential ...

Study: With Twitter, race of the messenger matters

When NFL player Colin Kaepernick took a knee during the national anthem to protest police brutality and racial injustice, the ensuing debate took traditional and social media by storm. University of Kansas researchers have ...

Researchers engineer a tougher fiber

North Carolina State University researchers have developed a fiber that combines the elasticity of rubber with the strength of a metal, resulting in a tougher material that could be incorporated into soft robotics, packaging ...