Custodes Futurifor the advancement of humanity

Educational factor, barrier 3 of 6

Disrupted schooling

Betthäuser, Bach-Mortensen, and Engzell (2023) pooled 42 studies from 15 countries on COVID-19 school disruptions.

Evidence

  • Learning lost. Students lost about 35% of a normal school year's worth of learning, and the losses remained stable over time rather than being recovered.
  • Who lost more. Children from lower socioeconomic backgrounds and children in poorer countries lost more, widening inequality. Losses were larger in math than in reading.
  • Persistence. Historical evidence from teacher strikes, shortened school years, and wartime disruption suggests learning deficits tend to persist long-term.
  • What helps. Keeping schools open and targeted catch-up. The evidence is moderate and still being studied.

Note: A later paper, not yet peer reviewed, has reanalyzed the dataset for publication bias (Luskova et al., 2026), so the exact size is still being examined.

Proposed and experimental methods

Methods that are proposed, under trial, approved in some places, or tried and then failed. Each shows a stage label and an evidence rating. A stage label shows how far a method has progressed, not whether it works. The stage labels are explained on the educational factor page.

  • Phone-call tutoring during school closures (Large trial, B). During COVID-19 closures in Botswana, weekly SMS messages plus phone calls to parents and children raised learning by 0.12 standard deviations, or 0.89 standard deviations per $100 (Angrist et al., 2022). Randomized trials in India, Kenya, Nepal, the Philippines, and Uganda found that phone tutorials raised learning by 0.30 to 0.35 standard deviations on average, government teachers achieved effects similar to NGO instructors, and SMS alone gave mixed results (Angrist et al., 2023). Unlike many replications, implementation quality improved as the program spread (Angrist et al., 2023b). The five-country paper is not yet peer reviewed.
  • High-dosage tutoring, and the fade of its effects at scale (In wide use, contested, B). A meta-analysis of 89 randomized trials found a pooled effect of 0.288 standard deviations, largest for programs using teachers or paraprofessionals, held at least three days a week during school (Nickow et al., 2024). In Chicago, daily two-to-one math tutoring raised scores by 0.16 standard deviations in one trial and 0.37 in a replication, at $3,500 to $4,300 per student a year (Guryan et al., 2023). An expanded meta-analysis of 265 trials found effects of 0.55 standard deviations in programs with under 100 students but 0.14 in programs with 1,000 or more (Kraft et al., 2024), and several post-pandemic district programs reported small or no gains, partly because students attended only about a third of sessions (Schwartz, 2025).
  • Large unconditional recovery grants to schools (In wide use, B). Using differences in U.S. federal pandemic relief created by the funding formula, a study of more than 5,000 districts in 29 states found that each $1,000 per student improved 2022-23 test scores by 0.0086 standard deviations in math and 0.0049 in reading (Dewey et al., 2024). The data cannot show which uses of the money worked, and the study is a working paper.
  • Reversed: replacing canceled exams with a statistical grading model (Failed or reversed, B). When England canceled its 2020 exams, the regulator Ofqual adjusted teachers' predicted grades with a model based on schools' past results, which lowered 39.1% of A-level teacher grades, 35.6% by one grade (Whittaker, 2020). After public protest, on August 17, 2020, Ofqual withdrew the model and gave students their teacher-assessed grade or the model grade, whichever was higher (Ofqual, 2020).

Sources cited on this page

  1. Betthäuser, B. A., Bach-Mortensen, A. M., & Engzell, P. (2023). A systematic review and meta-analysis of the evidence on learning during the COVID-19 pandemic. Nature Human Behaviour, 7(3), 375-385. DOI A Strong
  2. Angrist, N., Evans, D. K., Filmer, D., Glennerster, R., Rogers, H., & Sabarwal, S. (2025). How to improve education outcomes most efficiently? A review of the evidence using a unified metric. Journal of Development Economics, 172, 103382. link B Moderate
  3. Angrist, N., Ainomugisha, M., Bathena, S. P., Bergman, P., Crossley, C., Cullen, C., et al. (2023). Building resilient education systems: Evidence from large-scale randomized trials in five countries (NBER Working Paper No. 31208, revised June 2025). National Bureau of Economic Research. link B Moderate
  4. Angrist, N., Bergman, P., & Matsheng, M. (2022). Experimental evidence on learning using low-tech when school is out. Nature Human Behaviour, 6(7), 941-950. link B Moderate
  5. Dewey, D. C., Fahle, E. M., Kane, T. J., Reardon, S. F., & Staiger, D. O. (2024). Federal pandemic relief and academic recovery (NBER Working Paper No. 32897). National Bureau of Economic Research. link B Moderate
  6. Guryan, J., Ludwig, J., Bhatt, M. P., Cook, P. J., Davis, J. M., Dodge, K., et al. (2023). Not too late: Improving academic outcomes among adolescents. American Economic Review, 113(3), 738-765. link B Moderate
  7. Kraft, M. A., Schueler, B. E., & Falken, G. (2024). What impacts should we expect from tutoring at scale? Exploring meta-analytic generalizability (EdWorkingPaper No. 24-1031). Annenberg Institute at Brown University. link B Moderate
  8. Nickow, A., Oreopoulos, P., & Quan, V. (2024). The promise of tutoring for PreK-12 learning: A systematic review and meta-analysis of the experimental evidence. American Educational Research Journal, 61(1), 74-107. link B Moderate: (A for smaller programs)
  9. Schwartz, S. (2025, November 18). Why hasn't tutoring been more effective? Education Week. link B Moderate
  10. Whittaker, F. (2020, August 13). A-level results 2020: Top grades up by 2.4 percentage points. Schools Week. link B Moderate

Every source for this factor is listed on the educational factor page.