Those who fail productively are all the wiser

Researchers from ETH Zurich have demonstrated the positive effects of productive failure on learning outcomes. The success rate for one of ETH's largest courses was increased by 20 percent.

For a long time, the dominant paradigm in teaching has been that we learn new things best when someone explains them to us. First instruction, then practice: this is the educational formula still applied in countless classrooms and lecture halls today.

Researchers from the Professorship for Learning Sciences at ETH Zurich have now demonstrated that exactly the opposite is the case. "If you want to achieve ideal learning outcomes, it's better to first puzzle over a problem that is specifically relevant to a topic before then exploring the underlying principles," explains ETH professor Manu Kapur, who authored the study together with postdoctoral scientist Tanmay Sinha. The key to this approach is the experience of productive failure—a theory conceptualized and developed by Kapur.

15 years of educational research

Sinha's and Kapur's study is a meta-analysis of educational research from the past 15 years. The authors looked at 53 studies with 166 comparative analyses, all dealing with the question of which learning strategy is more effective: instruction before practice or vice versa. The primary topical focus was on how well school-age and university students comprehended concepts in the disciplines of mathematics, physics, chemistry, biology and medicine or were able to successfully apply them. The study did not include general skills, such as sensemaking when reading and writing proficiency, or problems from humanities and social science disciplines.

Almost half (45 percent) of the students tested were in grades 6 to 10 (at secondary school) at the time of the study, meaning they were between the ages of 12 and 18. Over a third (37 percent) were currently undergraduates, and one in six (15 percent) were still in primary school. Almost half (43 percent) of students came from North America, over a quarter each from Europe (26 percent) and Asia (28 percent).

Three times as efficient as a good instructor

The results have turned the last several decades of educational research upside-down: all of the students achieved much better learning success when they had to solve exercises and problems before the concepts required were explained to them. However, this held true more for secondary school students and undergraduates than for students at primary school. According to the authors, this can be explained by a combination of factors: primary school students often have too little knowledge in an area to solve problems effectively. In addition, their analytical reasoning and problem-solving abilities maybe less mature.

What is particularly astonishing is how starkly this affects learning outcomes: "Practice before learning the theory is nearly twice as efficient as receiving a year of instruction from an outstanding teacher," explains Kapur. Moreover, if students fail "productively" during the practice stage, their learning outcomes are up to three times better than what a very good teacher can achieve in a year.

Why productive failure pays off

But what exactly is happening when students fail productively? Sinha and Kapur say that there are four mechanisms at work here, corresponding to four "As": first, a problem should activate as much relevant knowledge as possible. "Productive failure," says Kapur, "requires a certain amount of prior knowledge. If a person wants to solve a statistical problem like finding the standard deviation productively, for example, they should at least be familiar with the most fundamental concepts such as the mean." Second, students should recognize the deficit between what they do and do not know already; this gives them awareness. Third, this makes them more receptive to new concepts and sparks their interest in solving the problem, i.e. it changes their affect, or psychological state.

The fourth and final stage is for the instructor or instructional material to provide an explanation that applies the new concept to solve the problem and demonstrates why the students' solutions missed the target. This can be described as knowledge assembly. "Learning outcomes depend on teaching in such a way that these four mechanisms all play a key role," explains Kapur. This is particularly true when students tackle problems that can be grasped intuitively but for which they are still lacking the knowledge required to solve the problem unless they are taught the new concepts.

20 percent higher success rates at ETH Zurich

But ETH Professor Kapur's team went beyond a meta-analysis and tested their theory directly in one of the largest year-long courses taught at ETH, Linear Algebra, which enrols around 650 students from the Department of Mechanical and Process Engineering. The course structure follows the traditional approach: concepts are introduced in lectures and then applied and explored in exercises.

Led by doctoral student Vera Baumgartner and in collaboration with ETH mathematics Professor Norbert Hungerbühler, Kapur's team created a set of tasks that students could voluntarily attempt to solve before five key lectures each semester. The goal of the exercises was productive failure. Roughly, sixty percent of students took advantage of the opportunity and completed the extra work. The results were impressive: historically, just over half of students (55 percent) on average pass the course. The success rate among those students who productively failed ahead of the lectures was 20 percent higher, and their marks were considerably better. For the authors, this clearly shows that those who engage in productive failure more often learn more.