What works for K-6 students at risk for failure in math and reading

What works for K-6 students at risk for failure in math and reading

Marta Pellegrini, University of Florence, Italy

The Campbell Collaboration has recently published a meta-analysis to evaluate the effectiveness of interventions for K-6 students at risk for failure in mathematics and reading. To be included in the review, studies had to use randomized or quasi-experimental designs and evaluate interventions conducted during the regular school day. A total of 205 studies were included in the review. Of these, 93% were randomized experiments and 86% took place in the United States. The included studies were placed in the following categories based on the intervention characteristics: coaching of personnel; computer-assisted instruction; incentives; peer-assisted instruction; progress monitoring; small-group instruction.

Overall, results showed significant positive effect for outcomes measured immediately after the intervention (ES = +0.30) as well as for follow-up outcomes (up to 2 years; ES = +0.27). Consistent with findings of previous reviews on students at risk for failure, peer-assisted instruction and small-group instruction by an adult showed the largest effect sizes (ES between +0.35 and +0.45). Regarding factors that may have influenced the effectiveness of interventions, the study found evidence that effect sizes were larger for early grades compared to older grades.

The authors concluded that implementing targeted interventions that involve the use of peer-assisted approach or small-group instruction by adults can reduce the gap in reading and math achievement between struggling students and their classmates.

Mathematics interventions for adolescents with mathematics difficulties: A meta-analysis

Mathematics interventions for adolescents with mathematics difficulties: A meta-analysis

By José L. Arco-Tirado, Faculty of Education, University of Granada (Spain)

A recent meta-analysis published in the Learning Disabilities Journal has synthesized the findings of interventions designed to improve the mathematics achievement of secondary school students with mathematics difficulties (MD).

A group of nine researchers performed a systematic literature review focusing on mathematics interventions targeting two categories of students having a MD: students with a documented disability under the Individuals with Disabilities Education Act (IDEA) (i.e. math LD) and students with persistent low mathematics achievement at the secondary level.

The authors identified 45 studies that met their inclusion criteria reporting findings for 49 interventions from which 139 ESs were extracted and used for the meta-analysis. They obtained a positive, statistically significant mean effect of +0.52 (p < .001), suggesting that the math interventions were generally effective.

In terms of moderators of treatment effects, the authors found significant effects for two intervention characteristics (i.e., including content domain and intervention duration), and two study design characteristics (i.e., type of measure and fidelity of implementation), and with non-significant effects on the other intervention and study design features tested.

The study identified three intervention models, Cognitive Based Instruction, Technology-based Interventions, and Visual Representation, that produced a statistically significant mean ES, while the other intervention models did not. Finally, in relation to content domains, four content domains, including fractions, numbers and operations, ratios and proportions, and multiple content domains, produced statistically significant mean ES.

These results have important implications for teachers practice since they extend the evidence-base of those interventions reviewed as effective to improve math performance across multiple math domains in secondary students with MD.

The importance of monitoring the success of an intervention program

The importance of monitoring the success of an intervention program

By Winnie Tam, Centre for University and School Partnership, The Chinese University of Hong Kong

Although integrating technology into education has been found to be effective means in helping students, it may fail to produce impacts because of poor implementation. A recent cluster-randomized controlled trial study conducted by Mo and colleagues investigated the effect of a computer-assisted learning (CAL) program in China on English achievement, and compared the differences in implementation of the program between institutions. The CAL program was designed to provide remedial tutoring for learning English during computer class sessions organized by a computer teacher. A balanced cohort of 120 primary schools in one of China’s poorest provinces was randomized into three groups: CAL implemented by a government agency, the same program implemented by a non-government organization (NGO), and no CAL program (control schools). A total of 5,253 Grade 4 students completed the program.

Both treatment groups adopted the same software packages for the tutoring, and both the government agency and the NGO conducted implementation, training, and monitoring of the program’s progress. The NGO, a university-based education-oriented entity, designed the implementation protocol and trained designated computer teachers to organize CAL sessions. The government agency assigned local education officers, county program managers, to carry out the program in addition to their other duties. The effectiveness of the program was assessed by standardized English language test scores after one academic year (2013-14), while controlling for baseline test scores and students’ characteristics. The results indicated:

•            Students in the NGO CAL program performed significantly better than those in the control group (E.S. = +0.16)

•            The government CAL program did not improve students’ English achievement compared to the control group.

The authors conducted mechanism analysis to explore potential causes for two identical CAL protocols producing different outcomes. The findings showed the following:

•            In 38% of the government CAL schools, regular English classes were replaced with CAL classes in comparison to 18% of the NGO CAL schools. Breaking the protocol may diminish regular English time.

•            Instead of computer teachers, 48% of government CAL schools assigned English teachers to run the CAL sessions while 18% NGO schools did the same. If protocol is not followed, the workload of English teachers may increase.

•            NGO staff were more likely to call or visit their program schools and monitor the progress than government officials (different by 50% points).

•            Correlation analysis indicated that replacement of class time was negatively associated with English test scores with a small effect size.

•            Direct monitoring was positively correlated with student scores.

In view of the finding that violation of protocol may diminish program impact, effective monitoring is one of the important factors for successful intervention.

The Medium-Term Effects of Tutoring

The Medium-Term Effects of Tutoring

By Justin Hill, Johns Hopkins University

Verónica Cabezas and colleagues recently concluded eight years of data collection from a randomized controlled trial designed to explore the short-term and medium-term effects of a tutoring intervention in Chile.  The program was administered by the Minister of Education and was directed toward fourth grade students from low socioeconomic backgrounds attending low-performing schools.  Students in the treatment group received 15 weekly 90-minute tutoring sessions with a focus on shared reading.  The program previously demonstrated small to moderate short-term effects that are similar or slightly smaller than previous research on tutoring programs.  Overall reading and literature test scores showed a small effect (ES = + 0.059), as did reading comprehension scores (ES = + 0.107), but use of language, texts production, and attitude towards reading did not show significant effects. 

The medium-term effects, which extend through the end of high school, are the focus of the study.  The treatment shows a significant beneficial effect on dropping out prior to the end of high school (ES = – 0.018) and a significant positive effect on completing high school on time (ES = + 0.032).  Positive effects are noted on attendance in both primary school (ES = + 0.814) and secondary school (ES = + 1.139), as is a positive effect on primary school grades (ES = + 0.088).  While the full sample did not show effects on 8th grade language or math test scores, there was a positive significant effect observed for students at a high risk of dropping out (ES = + 0.083 for language; ES = + 0.170 for math).  Finally, there was also a positive significant effect noted on 10th grade math scores for the full sample (ES = + 0.085). The authors conclude with an attempt to understand the mechanism responsible for the observed improvements.  They suggest that short-term effects result from both the academic content of the tutoring and the human connection made during tutoring sessions, but that medium-term effects are primarily driven by the human connection rather than academic content.  The effects observed in this study provide evidence that tutoring may not only help to improve short-term improvements in student understanding, but also help students from disadvantaged backgrounds improve in a range of longer-term outcomes.