Impacts of an American teacher professional development program on student achievement

By Carmen Pannone, University of Cagliari, Italy

Recently, the What Works Clearinghouse (WWC) produced a systematic review on the effectiveness of Reading Apprenticeship, a professional development program that can be delivered online or in person. The program leads teachers to help their students develop reading comprehension and acquire interest, engagement, and confidence in reading. It also aims to enhance social-emotional learning, as well as academic achievement in math, reading, science and social sciences for middle and high school students. This review focuses on the effects on academic achievement.

Five out of the nine studies located from a literature search met the WWC inclusion criteria. The selected studies were randomized control trials involving a total of 22,176 American students in grade 7-9. The most assessed outcomes were reading comprehension (n=5) and literacy achievement (n=3); mathematics achievement was assessed by two studies; science, general academic achievement, social sciences, and vocabulary were assessed only by one study, and life sciences by another one.

The results from one study showed that Reading Apprenticeship had a statistically significant positive effect on science achievement (ES=+0.11) and on general academic achievement (ES=+0.07, p=0.02). In the other domains, the overall effects of the program were not statistically significant (p>0.05), but the program seemed promising for life sciences (ES=+0.16), social studies (ES=+0.15), literacy achievement (ES=+0.04), and reading comprehension (ES=+0.04), with no effects on vocabulary nor on mathematics.

Are self-management interventions effective to improve student behaviors?

Marta Pellegrini, University of Cagliari (Italy)

One commonly employed approach for managing difficult and disruptive conduct in school environments is through the implementation of self-management interventions. It refers to a set of techniques that students are taught to use in order to assess and monitor their own behaviors.

A recent Campbell Systematic review looked at the impact of self-management interventions delivered at school to improve K-12 student behavior in the classroom. The review included experimental studies that either assigned groups or single students to the intervention or the control group, yielding 75 single-case studies and 4 group studies.

The average effect size for single-case studies was +0.69, with higher effects for students in special education and for African American students. The average effect size for group studies was +0.63, but the effect could be influenced by the small number of studies.

Excellence gaps and equity in education

Marta Pellegrini, University of Cagliari (Italy)

Differences in achievement levels among subgroups of students are referred to as “excellence gaps,” which primarily concern the highest-performing students. While certain students may struggle to attain the minimum expected levels of academic achievement, there are others who are already performing at levels beyond their current grade before the beginning of the school year.

Excellence gaps are connected to equitable school systems. Often students from disadvantaged socio-economic backgrounds have less access to advanced learning opportunities than their peers. A recent systematic review included qualitative and quantitative studies to determine what kind of research has been conducted on strategies to reduce excellence gaps in K-12. The review included 80 studies categorized by strategy, such as school accountability system support, teacher professional learning, and universal screening with local norms. The review highlighted the role of each strategy in reducing excellence gaps. Overall, results suggested considering four key points: Prepare, Place, Evaluate, and Adjust – students should be assessed for advanced learning programs before enrollment and during implementation, participating in the appropriate advanced programs, regularly evaluated through formative assessments, and have placements modified as necessary to optimize learning outcomes.

The effectiveness of digital monitoring tools on student academic achievement

By Marta Pellegrini, University of Cagliari (Italy)

Digital monitoring tools are instruments that support teachers in obtaining, organizing, and analyzing student data from test assessments. Through these tools teachers are also provided with feedback on the data they receive.

A recent review studied the effects of digital monitoring tools on student academic achievement. Studies included had to compare an experimental group in which teachers used digital monitoring tools with a control group. Each group had to include at least 20 teachers and the intervention had to last a minimum of 12 weeks. Only independent measures, such as standardized tests, were included in the meta-analysis.

A total of 14 studies were included in the review. Most of the studies evaluated the effects of the intervention on mathematics or reading. Studies were carried out more frequently in primary school than secondary school. Overall the results showed positive effects of digital monitoring tools on student academic achievement (ES =+ 0.12). The effects were larger in primary school (ES = +0.14) than secondary school (ES = +0.04), as well as in reading (ES = +0.17) and math (ES = +0.10) compared to language (ES = +0.02).

The authors categorized the interventions in three groups:

A.           programs with a low feedback intensity (1-2 times a year) targeting teachers and principals. The tools provided class-level feedback and predictive feedback as well as quite intensive activities were implemented through the program. Content included mainly technical information.

B.           programs with a low feedback intensity (1-2 times a year) targeting teachers and principals. The tools provided class-level feedback and predictive feedback. Intervention activity varies in its intensity. Content included class or school support.

C.           programs with a high feedback intensity (1+ times a month) targeting only teachers. The tools provided class-level feedback, not predictive feedback. The intervention activities were not very intensive. Content included technical information, class or school support and data translation into instruction. Among the three types of interventions, A and C were the most effective with an effect size of +0.25 and +0.13, respectively.