After school matters

“Our study of selected After School Matters apprenticeships found that youth in the program engaged in fewer problem behaviors, particularly gang activity and selling drugs,” said Barton Hirsch, the Northwestern University professor of education and social policy who led the evaluation.

In addition, the study found youth in After School Matters demonstrated more of what psychologists call “self-regulation,” considered a key component of positive youth development. Self-regulation involves the ability to stay focused on achieving goals despite emotional and other distractions, said Hirsch.

The large-scale study followed 535 Chicago high school students from 10 Chicago public high schools. All of them had applied to participate in After School Matters internships although only half won a lottery allowing their participation. Ninety-one percent of the control group -- those students who lost the After School Matters lottery and did not participate in After School Matters apprenticeships -- participated in organized after-school activities, community-based programs or paid work.

The study is the first randomized controlled study -- the gold standard in evaluation research -- of a high school after-school program since the 1980s. Hirsch conducted it with Northwestern statistics professor and Institute for Policy Research fellow Larry Hedges and University of Wisconsin-Extension professor JulieAnn Stawicki.

The researchers found no statistically significant difference between students in After School Matters and the control group in the areas of job skills and academic performance. Ninety-two percent of all the students studied came from low-income households and almost all were minority students.

“It’s a ‘yes but’ message,” said Hirsch, who nonetheless called the results “promising.” “The After School Matters apprenticeships were more oriented toward skill development, creating a product and looking toward the future -- and this seems to have paid off,” he added.

At the same time, Hirsch said the findings make the researchers cautious. “Because the 13 apprenticeships that we studied were among the best After School Matters offered, we still don’t know how the average After School Matters apprenticeship compares with the average alternative after-school program,” he noted.

The researchers purposefully chose to study After School Matters’ better apprenticeship programs to determine what happens when a program is implemented well. The apprenticeships in graphic design, computer repair, culinary arts, songwriting and producing, and other areas were chosen on the basis of recommendations and preliminary evaluations.

After School Matters executive director David Sinski emphasized the value of the study, saying “It’s important for us to have this validation of what we do well and ensure we apply the findings to all After School Matters programs.” He noted that After School Matters recently formed a Program Quality Division focusing on continual improvement of the program model.

“That division includes training instructors on evidence-based program quality methods,” said Sinski. He anticipates that better preparing instructors and other enhancements “will result in even greater outcomes for teens in our programs.”

After School Matters, which has programs throughout the City of Chicago, seeks to provide out-of-school opportunities that expose teens to career areas and help them develop skills. Ten-week sessions in fall and spring meet for three hours three times weekly. Two instructors with expertise in the area direct each apprenticeship.

At the time of the study, students were paid the equivalent of $5 per hour.

Based on their findings, the researchers have made suggestions for strengthening After School Matters programs. “They need to do better in getting across to teens that the skills they learn in After School Matters will help them get jobs,” said Northwestern’s Hirsch. “And apprenticeship instructors need to insist on high-quality work and keep young people focused on the task at hand.”

An innovative outgrowth of the study is a curriculum for teaching job interview skills. With the help of human resources professionals, the researchers designed a mock job interview to assess marketable job skills.

Before the teens were given interview training, human resources experts found many teens had experiences and skills that employers value but in interviews the youth often failed to convey those experiences or communicate their credentials.

“That changed when they received our brief interview curriculum,” Hirsch said. “In several Chicago Public Schools classrooms where students went through interview training, the mock interviews nearly tripled the would-be hiring rate.”

After School Matters executive director Sinski expects the study to have repercussions beyond Chicago.

“Organizations in New York, Boston and other cities have replicated our apprenticeship model,” Sinski said. “This study helps us all recognize the key components that make the apprenticeship experience so valuable for young people, as we work collectively to provide important out-of-school time opportunities for teens nationwide.”

More information: Copies of the complete After School Matters evaluation are available online at http://www.sesp.no … artonHirsch/

Provided by Northwestern University