Automated-grading skeptic uses Babel to expose nonsense essay

Apr 30, 2014 by Nancy Owano weblog
Credit: Wikipedia

( —The good news: A former MIT instructor and students have come up with software that can write an entire essay in less than one second; just feed it up to three keywords. The bad news: The essay is gibberish. Oh, wait, more news: The nonsense essay was fed through an online writing product using essay-scoring technology. Perelman pasted the essay into the answer field, clicked "submit," and the paper got a score of 5.4 out of 6. The essay, after all, had good grammar and impressive vocabulary words. The end result was nonsense, nonetheless.

For the curious, the essay had sentences such as "Privateness has not been and undoubtedly never will be lauded, precarious, and decent.". The program was fed one keyword: "privacy." The key players behind this , called Babel, are Les Perelman, former director of undergraduate writing at the Massachusetts Institute of Technology, together with Harvard and MIT students. Perelman is concerned over the idea of using software to grade essays. The Babel program delivers essays that are intentionally gibberish to prove the weaknesses of automated essay-grading software. The Babel Generator stands for "Basic Automatic B.S. Essay Language" Generator.

A detailed report on Perelman's work appeared in Monday's The Chronicle of Higher Education, where Steve Kolowich wrote that Perelman's fundamental problem with essay-grading automatons is that they are not measuring any of the real constructs that have to do with writing. Kolowich said Babel Generator is turning "the concept of automation into a farce: machines fooling machines for the amusement of human skeptics."

Still, not everyone falls into Perelman's camp. With further development, machines giving student feedback may become increasingly useful. Kolowich observed that interesting work in automated essay grading is also taking place at MIT, where computer scientists at edX, co-founded by the university, is developing an automated essay-scoring system. This system is called the Enhanced AI Scoring Engine, or EASE. The engine is defined as a library that allows for machine learning-based classification of textual content, for tasks such as scoring student essays.

"Essentially," wrote Kolowich, "the edX software tries to make its machine graders more human. Rather than simply scoring essays according to a standard rubric, the EASE software can mimic the grading styles of particular professors."

At the same time, the report noted, Perelman used to teach a course with Anant Agarwal, chief executive of edX, back in the 1990s, and said "they have been talking about running some experiments to see if the Babel Generator can be used to inoculate EASE against some of the weaknesses the generator was designed to expose."

Explore further: Study: Interventions help women's reluctance to discuss accomplishments

add to favorites email to friend print save as pdf

Related Stories

NASA selects essay competition winners

May 10, 2007

The National Aeronautics and Space Administration has identified the winners of its high school essay competition to describe "Air Transportation in 2057."

Recommended for you

Taxi app Uber defies ban in Spain

4 hours ago

Smartphone car-ride service Uber vowed Friday to keep operating in Spain despite a ban following a lawsuit by taxi drivers and said it would fight the case in court.

Sony film mess reflects gadgets-entertainment gap

12 hours ago

Sony's iconic gadgetry and the star appeal of Hollywood may have appeared to be a perfect match when the electronics giant bought Columbia Pictures in 1989. A quarter century later, it's apparent that Sony ...

Streaming release of 'Interview' test for industry

Dec 25, 2014

Sony's "The Interview" has been a hacking target, a punchline and a political lightning rod. Now, with its release online at the same time it debuts in theaters, it has a new role: a test for a new kind of ...

User comments : 0

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.