Experiment finds AI-based intervention helps undergrads pass STEM course

U.S. college students majoring in STEM fields currently graduate about 20% less often than their non-STEM peers, a resounding clarion call for better assisting those students, especially in their first few semesters. Though systemic, long-term shifts—away from lecturing, toward the sharing of evidence-based teaching practices—should help, the inertia of academia can sometimes slow their adoption.

Some educators and researchers have looked to supplement those grander shifts with a few nudges in the right direction. One promising candidate: periodic interventions designed to help struggling students find their way.

Nebraska's Mohammad Hasan and Bilal Khan recently investigated whether machine learning—a form of AI that can identify patterns in data, then use that learned recognition to forecast outcomes—might contribute to the cause. To start, Hasan and Khan trained a model on the homework, quiz and midterm scores, along with the final grades, of 537 students who took a computer science course between 2015 and 2018.

Later, they applied the model to a class of 65 undergrads enrolled in that same course. At three points in the semester—six weeks, nine weeks and 12 weeks in—32 of the students received an automated message via the university's course management system. That message relayed the model's projection of a student passing the course: "Good," "Fair," "Prone-to-Risk" or "At-Risk." The remaining 33 students, those in the control group, always received an "Unable to Make a Prediction" message.

Among the control group, 24 of 33 students—roughly 73%—passed the course. Those who received actual forecasts based on their trajectories, meanwhile, fared substantially better: 29 of the 32 students passed, good for a rate of nearly 91%. And of the surveyed students who reported actively checking their status, 86% said they increased their effort after seeing the forecasts.

The research is published in the journal PLOS ONE.

Hasan and Khan said the findings, though early, point to the value of integrating AI-based interventions into STEM courses. The duo plans to conduct a larger, longer-term study that could help determine whether variables beyond scores—course-related behaviors, perceptions of science, demographics—might generalize and expand the use of the interventions beyond a single uniform course.