Teach for America: Analyzing effects across various timeframes

By Carmen Pannone, University of Cagliari (Italy)

Teach for America (TFA) recruits and trains teachers to address hard-to-staff vacancies in public schools, providing support throughout their 2-year commitment as corps members. Studies, including randomized controlled trials and analyses of administrative records, have shown that TFA has consistently yielded positive effects on student outcomes, particularly in math, with more modest results  in other subjects such as English Language Arts (ELA).

A recent study aimed to further explore the impact of Teach for America (TFA) by assessing its short- and long-term effects. The authors conducted this study by analyzing data from Miami-Dade County Public Schools (M-DCPS) spanning from 2010 to 2021. Using a fixed effects model to compare outcomes of TFA teachers with a control group of other teachers in the same schools, the authors found statistically significant positive effects of the TFA program on same-year math scores (ES = +0.09) and ELA scores (ES = +0.03). Additionally, they observed a slight decrease in chronic absenteeism (ES = -0.01, non-significant) and suspensions (ES = -0.01, p < .10), along with a slight increase in grade repetition (ES = +0.02). When considering these non-test factors collectively, students in TFA classrooms for a given year exhibited a non-significant overall increase of about +0.01 standard deviations.

Additional analyses exploring the impacts in the year following being in a Teach For America classroom revealed students exposed to TFA tended to demonstrate improved non-test outcomes in their subsequent year of school (ES = +0.03). On the contrary, the positive impact of TFA on math test scores did not appear to be persistent. Recognizing the high turnover rates of TFA corps members in high-need schools, the authors acknowledge that TFA’s presence in such contexts highlights the challenge of securing talented teachers for these schools. In the absence of viable alternative hiring programs, TFA seems to provide a vetted option with demonstrated benefits for students across various outcomes.

Student Success Skills’ effects on Hispanic students

By Susan Davis, Johns Hopkins University

Student Success Skills (SSS) is a program designed to help students reflect and develop purposeful academic, social, and self-management skills that ultimately lead to improved performance in school. A guidance counselor delivers weekly classroom lessons in academic goal-setting and in non-academic areas such as tracking health, wellness, and anxiety, which when addressed, have been shown in research to positively affect academic achievement.

Because the Hispanic student population is the fastest-growing of all student subsets, to ensure adequate cultural responsiveness, researchers examined the impact of SSS on attendance, self-regulation, and test anxiety on the subset of 681 Hispanic fifth grade students within a 2019 randomized controlled trial of SSS (383E, 298 C) from 30 schools in a single district. Following a one-day training, school counselors in the experimental group were randomly assigned to deliver SSS weekly for 45 minutes over five weeks, with a monthly booster session January-March. Control counselors continued with regular counseling practices. . All students were tested two weeks before the experimental group began receiving treatment, two weeks after the fifth lesson and thirty weeks after the fifth lesson. Students were evaluated using three years’ attendance data and two student surveys: the Student Engagement in School Success Skills (SESSS) Self-Regulation of Arousal subscale and the Motivated Strategies for Learning Questionnaire (MSLQ) test anxiety subscale.

Results showed that at 30 weeks post intervention, students who had been in the SSS group had better attendance and less test anxiety than controls. These improvements were not evident in at the first post-test, but were evident at the second, implying that time is needed both for students to create better habits and for schools to nurture the environments needed to encourage attendance and promote a safe environment. These findings mirror the findings of the overall RCT from which this data was drawn. Self-regulation showed no statistically significant difference between the two groups at either pre- or post-test—while experimental students’ self-regulation scores were indeed higher than controls’, the difference was not statistically significant.

The relation between syntactic skills and reading comprehension

By Winnie Tam, Centre for University and School Partnership, The Chinese University of Hong Kong

Theories of reading development emphasize the importance of syntactic skills, which involve understanding and manipulating sentence structures, in relation to reading comprehension. In fact, some researchers regard syntactic skills to be universal predictors of reading comprehension. A recent meta-analysis by Tong and colleagues investigated the relation between syntactic skills and reading comprehension for two languages with distinctive syntactic structures: Chinese and English.

The meta-analysis included a total of 59 articles, comprised of 75 independent samples and 235 correlations. Among these, 37 studies had participants whose first language was English, while 22 studies had participants whose first language was Chinese.  The results of the meta-analysis are shown below:

•            Overall, a moderate correlation was found between syntactic skills and reading comprehension in both English and Chinese (r=+0.54 in both languages).

•            The correlations between reading comprehension and various syntactic tasks were as follows: recalling sentences (r=+0.59), error detection/correction (r=+0.55), word order (r=+0.53), oral cloze (r=+0.49), and sentence-picture matching (r = +0.49).

•            The correlation between oral cloze tasks and reading comprehension was significantly stronger in Chinese than in English. However, two tasks exhibited similarity between the two languages: error detection / correction (r=+0.57 English; r=+0.53 Chinese) and a word order task (r=+0.52 English; r=+0.53 Chinese).

•            No significant difference was found between modality of syntactic task: oral (r=+0.55) and written (r=+0.53). Moreover, correlations in the oral mode were similar across the two languages (r=+0.54 English; r=+0.58 Chinese).

The authors noted that the striking similarity in effect sizes suggests a universal role of syntactic skills in reading comprehension across English and Chinese. Therefore, comprehensive theories of reading comprehension should include syntactic skills as an integral component.

Preliminary results from the Personalized Learning Initiative

By Cynthia Lake, Johns Hopkins University

Despite substantial federal investment through the Elementary and Secondary School Emergency Relief (ESSER) Fund, the education sector still grapples with challenges related to pandemic-induced learning loss. High dosage tutoring has been a leading strategy in accelerating academic catch-up efforts. The Personalized Learning Initiative (PLI) led by the University of Chicago Education Lab, in collaboration with MDRC and researchers from the University of Toronto and Stanford University, focuses on whether and how high dosage tutoring can be scaled to benefit more students. Their preliminary report finds that incorporating tutoring during the school day is more effective than after-school or virtual tutoring and underscores the importance of structured, consistent approaches.

The initiative worked with Chicago Public Schools (CPS), Illinois; Fulton County Schools, Georgia; the New Mexico Public Education Department; and a mid-sized urban school district in California. The New Mexico and California partners focused on out-of-school tutoring, virtual tutoring at home and after-school respectively, and both struggled with low student participation that precluded rigorous evaluations. In contrast, CPS and Fulton County Schools, who used their ESSER funding to incorporate tutoring during the school day, had more robust participation and were able to participate in randomized controlled trials during the 2022-2023 school year.  

Across sites, over 2,000 K-11 students were randomized into high dosage tutoring in math or reading or to business-as-usual control groups. In Fulton County, most of the schools began tutoring in January and 82% of students in the high dosage tutoring group participated in at least one session. Also notable was that 35% of those assigned to the business-as-usual group also received at least one tutoring session. The mean dosage for all assigned students was 9.0 sessions in the treatment group compared to 3.48 sessions in the control group. In Chicago, the mean dosage for the treatment group was 18.40 sessions for all assigned students, with 67% of them attending at least one session. The control group had a crossover rate of 18%, with a mean dosage of 4.61 sessions. Across both subjects and sites, the intent-to-treat impact was +0.04 and the treatment-on-the-treated effect size was +0.14. Despite challenges in maintaining data quality and delivery consistency, the combined outcomes from Fulton County and Chicago indicate a generally positive impact, particularly in math, where the treatment-on-the-treated (TOT) effect was 0.27 SD.

The broader lesson learned from the first year of the Personalized Learning Initiative is that high dosage tutoring can generate meaningful learning gains, provided it is structured and well-supported. The strategy that appears to work best involves a combination of smaller student-tutor ratios, structured curricula, and the use of trained tutors during the school day.