How good is your therapist? This machine will tell you

July 20, 2018 by Allie Nicodemo, Northeastern University
Tad Hirsch. Credit: Northeastern University

For people struggling to overcome addiction, good therapy can be hugely beneficial. But bad therapy is worse than no therapy at all.

The challenge is being able to tell the difference. Practicing counselors rarely receive feedback, so it's difficult for them to know if they are providing optimal treatment to their patients.

Northeastern professor Tad Hirsch is working to change that. He has developed a system that records counseling sessions and rates them, generating a report card for therapists.

The technology is already in use at a university training clinic and a network of opioid addiction treatment facilities. Once scaled up, the system could create a new standard of care for therapy and lead to improved mental health for patients all over the world.

Here's how it works: Hirsch and his collaborators started by recording 356 counseling sessions. From those sessions, they pulled out 300,000 statements and compiled them into a database. Then a team of psychology experts combed through the statements and coded them.

Coding is an analytical process in which data are categorized. In this case, the experts coded statements based on how closely they aligned with techniques used in motivational interviewing, a type of psychotherapy.

Next, researchers used the coded dataset to train a machine learning model. This is a form of artificial intelligence. Once the model is trained, the result is a system that can record a counseling session, create a transcript of what was said, analyze the transcript, grade the session, and provide specific feedback to clinicians.

In other words, a computer is evaluating therapists and telling them how they can improve.

How do therapists react to being assessed by a machine? Hirsch and his collaborators recently completed a study to find out, testing the technology with 21 counselors. The results were overwhelmingly positive.

"We found that across the board, they all saw value in what we were doing," said Hirsch, a professor of design at Northeastern whose research focuses on the intersection of design, engineering, and social justice.

"Clinicians described the technology as accurate, insightful, and useful. They also thought the tool was particularly valuable for training, as a way of providing feedback to counselors as they were getting certified," Hirsch said.

Therapists also expressed a high degree of trust in the machine learning model. Hirsch said this is likely because advances in have been covered extensively in the news, and the public generally has a positive view of the technology. Plus, as Hirsch said one therapist put it, "It's hard to argue with a computer."

And while the machine learning evaluates sessions with about 90 percent accuracy compared to a human expert, Hirsch cautions that the system is not infallible. And the stakes are high.

"This is different than when Netflix makes a movie recommendation that you don't happen to like," Hirsch said. He encourages therapists to speak up if they disagree with the machine's assessment and to use the as a springboard for honing their practice.

"As designers, we want to ensure that the predictions our models are making are contextualized, such that people can understand how these systems are working well enough to interpret findings and results they might be seeing, rather than just take them at face value," Hirsch said.

Explore further: Scientists unravel brain networks of cardiac arrest survivors

Related Stories

Personalized 'deep learning' equips robots for autism therapy

June 27, 2018

Children with autism spectrum conditions often have trouble recognizing the emotional states of people around them—distinguishing a happy face from a fearful face, for instance. To remedy this, some therapists use a kid-friendly ...

Training artificial intelligence with artificial X-rays

July 6, 2018

Artificial intelligence (AI) holds real potential for improving both the speed and accuracy of medical diagnostics. But before clinicians can harness the power of AI to identify conditions in images such as X-rays, they have ...

Recommended for you

What happened before the Big Bang?

March 26, 2019

A team of scientists has proposed a powerful new test for inflation, the theory that the universe dramatically expanded in size in a fleeting fraction of a second right after the Big Bang. Their goal is to give insight into ...

Cellular microRNA detection with miRacles

March 26, 2019

MicroRNAs (miRNAs) are short noncoding regulatory RNAs that can repress gene expression post-transcriptionally and are therefore increasingly used as biomarkers of disease. Detecting miRNAs can be arduous and expensive as ...

Can China keep it's climate promises?

March 26, 2019

China can easily meet its Paris climate pledge to peak its greenhouse gas emissions by 2030, but sourcing 20 percent of its energy needs from renewables and nuclear power by that date may be considerably harder, researchers ...

In the Tree of Life, youth has its advantages

March 26, 2019

It's a question that has captivated naturalists for centuries: Why have some groups of organisms enjoyed incredibly diversity—like fish, birds, insects—while others have contained only a few species—like humans.


Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.