Were online supports equally effective during Covid-19 lockdown in China?

By Winnie Tam, Centre for University and School Partnership, The Chinese University of Hong Kong

Universally, students suffered loss of learning during the pandemic lockdown when schools were closed all over the world. Clark and colleagues published a paper that  evaluated the impact of lockdown during the Covid-19 outbreak on students’ academic outcomes in China using administrative data on 9th grade students from 3 Chinese Middle schools in the same county in Baise City. During the Covid-19 lockdown period (mid-February to early April 2020), three schools made different arrangements for students:

•            School A did not provide any online education support.

•            School B used an online learning platform provided by the local government to support students. School B provided online lessons which were recorded by their own teachers.

•            School C used the same online platform as School B over the same period which was managed in the same fashion as in School B. The only difference between School B and C was that School C obtained on-line lessons recorded by the highest-quality teachers in the city, instead of their own teachers.

The final sample consisted of 20,185 examination results from 1,835 students who took all 11 exams in the five compulsory subjects (Chinese, Math, English, Politics, and History). Exams 1-10 were conducted before lockdown (Nov 2017 to Jan 2020), and students took Exam 11 immediately after the county reopened (Apr 2020). The last semester of 9th grade was used for revision and to prepare for the high-school entrance exam for which all materials had already been taught during middle school. Using a difference-in-difference framework, the impact of online education support is shown below.

•            Overall, using an online learning platform improved students’ total exam scores significantly relative to the scores of students without learning support (ES = +0.22, around 26 exam points).

•            The quality of recorded lessons mattered. Exam results of School C, whose lessons were recorded by external better-quality teachers, achieved better results than School B, whose lessons were recorded by their own teachers (ES = +0.06).

•            Not all online education was equal. The performance of students who used a computer for online education was better than that of those who used a smartphone (School B: ES=-0.14; School C: ES=-0.15).

•            Results of quantile DID analysis showed that low-achieving students benefited the most from online learning support, while there was no impact on top academic performers.

Since the data used in the study were obtained from only three middle schools in one Chinese county, further investigation is needed for generalizability. Along the same lines, using students exposed to the same environment would improve internal validity of the study. Researchers also stated that they investigated the effect of an online learning platform used to review materials already taught rather than reviewing platforms concerned with learning new knowledge.

Online Tutoring for struggling students during COVID-19 in Italy

By Marta Pellegrini, University of Florence, Italy

During the first Covid-19 lockdown in Italy in Spring 2020, researchers from Harvard Kennedy School and Bocconi University (Milan) piloted a project to give middle school struggling students the support needed due to the school closure. The program, TOP (Tutoring Online Program), was designed and offered to all schools in Italy in grades 6 to 8. TOP consists of online tutoring activities in mathematics, Italian, and English without a specific curriculum but mainly focused on helping students with homework. Tutors were volunteer university students trained to support students individually three to six hours a week.

Seventy-six middle schools participated in the study, with 1,059 students randomly assigned to either receive TOP for 6 weeks (n=530) or to be in an untreated control group (n=529). Of those students, 88% of the TOP group completed the final test, compared with only 46% of the control group.  Measures of academic achievement in math, Italian, and English were developed for this project based on the Italian National Test. Students’ aspirations, socio-emotional skills and psychological well-being were also assessed using a survey.

Statistically significant positive results on the academic achievement test were found for students in the treatment group, with a mean effect size of +0.26. Results were not significant on students’ aspirations nor for socio-emotional skills. A significant decrease of depression symptoms reported by the students was found for the treatment group (ES = -0.16) as well as an increase in happiness (ES = +0.16) as reported by parents.  However, these results must be interpreted cautiously, due to the differences in outcome test completion between the TOP and control group.

This project proposed a concrete cost-effective answer to the school closure in Italy by designing and evaluating a tool to help vulnerable students and prevent inequalities during the COVID-19 pandemic.

How did absenteeism during the COVID-19 pandemic affect student outcomes?

By Chenchen Shi, School of Education, Renmin University of China

In March 2020, most schools in the United States transitioned to distance learning in an effort to contain COVID-19. During the transition, a significant number of students did not fully engage in remote learning opportunities due to resource or other constraints. An urgent question for schools around the nation is how much the pandemic impacted student outcomes.

In a recent article published by Educational Researcher, the authors used administrative panel data from six large CORE Districts in California to approximate the impact of the pandemic by analyzing how absenteeism affected student outcomes.

The results showed wide variation in absenteeism impacts on academic and social-emotional outcomes by grade and subgroup, as well as the cumulative effect of different degrees of absence. Student outcomes generally suffer more from absenteeism in mathematics than in English language arts. Negative effects are larger in middle school. Absences also negatively affect social-emotional development, particularly in middle school, which can affect other student outcomes down the line.

The impact of closing failing schools

Stanford University’s Center for Research on Education Outcomes (CREDO) has released findings from a new study on school closures. The report, authored by Chunping Han and colleagues, systematically examines closure of low-performing charter and traditional public schools (TPS). A main goal of the study was to see whether children whose schools had been closed for poor performance do better or worse in their new schools.

The authors used existing longitudinally linked data that CREDO had developed in partnership with 26 state education agencies. They identified low-performing, full-time, regular (non-alternative) schools and closures in those 26 states from academic year 2006-07 to 2012-13. A total of 1,522 low-performing schools, including 1,204 TPS and 318 charters, were closed in the 26 states during the study period. To measure academic performance across the low-performing schools, the authors used scores from state standardized achievement tests.

Key findings of the study included:

• A little less than half of displaced closure students landed in better schools.
• In both the charter and traditional public school sectors, low-performing schools with a larger share of black and Hispanic students were more likely to be closed than similarly performing schools with a smaller share of disadvantaged minority students.
• The quality of the receiving school made a significant difference in post-closure student outcomes. Closure students who attended better schools post-closure tended to make greater academic gains than did their peers from not-closed low-performing schools in the same sector, while those ending up in worse or equivalent schools had weaker academic growth than their peers in comparable low-performing settings.