Worse results and more drop-outs recorded when teaching is in English, Swedish study finds

Using English as the language of instruction in higher education has a marked negative impact on learning outcomes when it is not the students' first language, according to a new study from KTH Royal Institute of Technology and Chalmers University of Technology in Sweden.

When 2,000 Swedish students were divided up into English-language and Swedish-language versions of an introductory course in programming, those students who were taught in English obtained much worse results, and more dropped out of the course prematurely.

English is increasingly used as a global language of instruction in higher education, known as English Medium Instruction or EMI, according to a report from the Language Council of Sweden (Språkrådet) in 2022. An important starting point for EMI is that the language of instruction must not affect students' learning quality. The aim is that a student taking a course in English should learn just as well as a student studying in their first language. But a new study now casts doubt on this assumption.

Swedish students taking a course in which the language of instruction was English performed significantly worse, and they also dropped out to a much higher degree compared with students who took the same course in Swedish.

"Our study is unusual in this field since the groups were assigned on a completely random basis. This means that the only influencing factor was the language of instruction, and we were surprised by the results," says Hans Malmström, Professor at the Department of Communication and Learning in Science at Chalmers University of Technology, and one of the researchers behind the study.

More test questions answered correctly in the Swedish course

When 2,263 students registered for an introductory course in programming, they were randomly divided into either an English or a Swedish version of the course. The course was entirely digital and self-paced. Students' performance was measured based on the number of correctly answered test questions and on how many left the course without completing it.

When the researchers compared the number of questions answered correctly in the two versions of the course it emerged that those who studied in Swedish gave the correct answers to 73 percent more questions in the test on average.

"It's important to remember that the only difference here is the language of instruction. The fact that the students on the Swedish-language course performed significantly better indicates that the use of English as the language of instruction can have a negative impact on learning under certain circumstances," says Olle Bälter, Associate Professor in Human-Computer Interaction and one of the researchers from KTH Royal Institute of Technology.

More students dropped out prematurely from the English course

When the researchers measured the student completion rate on the course, the results were confirmed; 25 percent more students dropped out of the English-language course and here too the difference between the two versions of the course was statistically significant.

"Someone who drops out is not likely to have learnt as much as someone who completes the course. So, in this respect too we see that English as the language of instruction can lead to poorer learning outcomes," says Viggo Kann, Professor of Computer Science at KTH Royal Institute of Technology.

The study now raises issues about the advantages and disadvantages of having such a large proportion of teaching at universities and higher education institutions in English, something that is common in Sweden as well as in other countries where English is not the first language for the majority of the population.

"It is important to remember that a single study should not be used as the basis for a radical overhaul of the language or teaching policy in higher education, neither at local nor national level. However, we believe the results from this research can contribute to a more informed discussion about the consequences of using English as the language of instruction," says Malmström.

The study in brief

• 2,263 students took part in a study which investigated how teaching in English affects the academic performance of students.
• The participants were randomly divided into an English and Swedish version of a programming course.
• Academic performance was measured in two ways: the number of test questions answered correctly and the completion/drop-out rate.
• The design of the study—a randomized control study—made it possible to control for confounding variables; the only influencing factor was the language of instruction.
• Students who took the Swedish version of the course gave correct answers to 73 percent more test questions on average.
• Students who took the English version of the course dropped out of the course at a 25 percent greater rate.
• The conclusion of the study is that teaching in English had a negative impact on the academic performance of the course participants.

The study English-medium instruction and impact on academic performance: a randomized control study has been published in Applied Linguistics Review and was carried out by Olle Bälter, Viggo Kann, Chantal Mutimukwe and Hans Malmström. The researchers work at Chalmers University of Technology and KTH Royal Institute of Technology.