Automated feedback: The new science of grading STEM papers

September 13th, 2017
Wake Forest biology professors Dan Johnson and Sabrina Setaro talk with graduate students Noah Bressman and Jenny Howard about new methods for grading science writing, in a lab in Winston Hall. Credit: WFU / Ken Bennett
Science educators at Wake Forest University are testing how automated feedback combined with new one-on-one teaching methods can improve scientific writing from STEM undergraduates - and result in better explanation of research to the public.

The National Science Foundation (NSF) has awarded the research team nearly $280,000 to conduct the writing experiment. It comes at a pivotal time, with claims of junk science and fake news making headlines almost daily.

"Students don't see that writing is important in the sciences," said Dan Johnson, who leads the research team and is a professor in the Biology Department. "They don't see much value in it. They don't realize we actually live and die by our writing."

The STEM (science, technology, engineering and math) writing study will build on an automated lab report feedback system called SAWHET (pronounced saw-it). Undergraduate science students have been using SAWHET to submit and revise lab reports online for two years.

SAWHET (or STEM Automated Writing Help Tool) guides students to break down their lab reports into the proper sections of a scientific paper - title, extract and citations, for example. Help buttons explain those sections, and the software can ask some basic editing questions: Do you need a citation to back up this statement? Do you state your hypothesis? Once the writer addresses those questions, the report is re-submitted.

With the NSF grant, the researchers will teach SAWHET to ask additional content-focused questions, allowing instructors and teaching assistants to concentrate on providing deeper, one-on-one feedback.

Time has been the primary challenge to teaching writing more aggressively to STEM students. One college course can include 150 students. If the instructor gives a writing assignment and edits it twice for about 15 minutes each, that can add up to 70 hours of work.

But writing and communication skills are increasingly important to employers. A report by the Association of American Colleges & Universities found that 80 percent of employers surveyed wanted colleges to emphasize written and oral communication skills more.

"Employers are often complaining that writing skills are poor," said Sabrina Setaro, program director of the STEM writing project. "Writing helps students be concise, order thinking, create a story and build an argument."

That's why the Wake Forest study also will introduce new methods that instructors will use for teaching and evaluating students' scientific writing. Instructors will be encouraged to let SAWHET do the low-level proofing and format checking, so they can do a deeper dive into the logic, arguments and thinking behind the written words.

"We're seeing places where people are using less effective practices because it's familiar - but we've got evidence that shows that doesn't help a student progress," Setaro said.

This isn't the first time Johnson and Setaro have flipped the undergraduate science curriculum. They both work on BioBook, a collaborative, interactive online biology resource that replaces traditional written textbooks and encourages inquiry-based learning. The basic edition is free; a customizable version is available, for which students pay $30 per year for access. About 750 students and professors at two universities are using it right now.

SAWHET is an offshoot of BioBook.

"Students were using the assessments in BioBook, and we realized that writing was a major barrier for them," Johnson said. "We wanted to give students a chance to assess their writing, too."

Provided by Wake Forest University

This Science News Wire page contains a press release issued by an organization mentioned above and is provided to you “as is” with little or no review from Phys.Org staff.

More news stories

Genome of American cockroach sequenced for the first time

A team of researchers with South China Normal University and the Chinese Academy of Sciences has for the first time sequenced the genome of the American cockroach. In their paper published in the journal Nature Communications, ...

Diagnosing breast cancer using red light

Optical Mammography, or OM, which uses harmless red or infrared light, has been developed for use in conjunction with X-rays for diagnosis or monitoring in cases demanding repeated imaging where high amounts of ionizing radiation ...

Researchers track Chinese space station as it falls

A defunct Chinese space station, Tiangong-1, is expected to fall to Earth any day now—on March 31, give or take a few days. When it does, it will be the largest manmade object to re-enter Earth's atmosphere in a decade.

New innovations in cell-free biotechnology

A Northwestern University-led team has developed a new way to manufacture proteins outside of a cell that could have important implications in therapeutics and biomaterials.

Mars Curiosity celebrates sol 2,000

NASA's Mars Curiosity rover just hit a new milestone: its two-thousandth Martian day, or sol, on the Red Planet. An image mosaic taken by the rover in January offers a preview of what comes next.