When the COVID-19 pandemic reached the United States in early 2020, public school officials were suddenly and unexpectedly forced to make life-and-death decisions. As the American death toll mounted (more than 1 million at the time of this writing), states across America closed their schools. This proved controversial, with many liberals joining conservatives in worrying that doing so would set American children back in their education. Over time, research emerged suggesting those fears were not unfounded.
Now, many critics of school closures are claiming vindication after the release on Monday of the Nation's Report Card by the National Center for Education Statistics (NCES). It revealed, among other things, that between 2019 and 2022 American fourth-graders and eighth-graders suffered the sharpest drop in math skills ever recorded since the National Assessment of Educational Progress (NAEP) began these studies in 1969. Reading scores also fell in most states during those years, reaching levels not seen since 1992 and with both grades seeing drops of three points in their average scores (220 to 217 for fourth grade and 263 to 260 for eighth grade). In math, by contrast, the average scores fell from 241 to 236 for fourth-graders and 282 to 274 for eighth-graders. Researchers believe that a drop of 10 points is equivalent to roughly one year of learning.
Does this mean that opponents of school closings and advocates of early re-openings were correct? That certainly is the verdict of many online commentators, from conservative pundit Ben Domenech and education journalist Alexander Russo to podcast host John Ziegler.
Experts, on the other hand, say the story — and the study results — are more complicated and nuanced than that. Even the people behind the Nation's Report Card are skeptical that you can draw direct conclusions.
"Using our data, especially as it is now, to say that school closures led to the declines and scores: we wouldn't endorse that," Grady Wilburn of the NCES told Salon. "We are hoping to dig into the data a little bit later. This is just our initial release."
Wilburn added that the NCES plans on doing further research on the subject of how school closures impacted student performance, but "right now making a strict a straight-line comparison between school closures and the decline that we're seeing is not endorsed by us."
"Using our data, especially as it is now, to say that school closures led to the declines and scores: we wouldn't endorse that."
Dr. Jack Schneider, a professor at the University of Massachusetts–Lowell School of Education who wrote a 2018 paper for the peer-reviewed journal Teachers College Record about the history and effectiveness of standardized testing, noted to Salon by email that the NAEP was not designed to tease out what leads to specific test outcomes. That said, he also felt that it "isn't surprising" that fourth-and-eighth graders posted worse scores after the pandemic.
"After all, schooling was massively disrupted, as was life in general," Schneider observed. "Only the most privileged students would have had stable conditions across the past two and a half years." He also added that the NAEP scores do not show, as some may believe, that individual students are doing worse now than before the pandemic. "That's one of the most problematic aspects of the 'learning loss' narrative — it suggests that students know less today than they did two and half years ago," Schneider wrote to Salon. "That isn't the case. Instead, it's the case that students on average did not progress as much as we might have expected them to progress in ordinary times."
Want more health and science stories in your inbox? Subscribe to Salon's weekly newsletter The Vulgar Scientist.
Dr. Ethan Hutt, a professor of education at the University of North Carolina–Chapel Hill who co-authored the aforementioned Teachers College Record article with Schneider, agreed that it is not possible to "reasonably attribute" the decline solely to school closings. He instead cited the principle of theory construction known as Occam's Razor, which holds that other things being equal the least complicated explanation for a question is the likely answer — in this case, that the pandemic writ large led to declining school outcomes, with the school closings serving as a single variable among many.
"What in the last few years could have caused a sudden and unprecedented drop in NAEP scores?" Hutt rhetorically asked Salon. "But it is foolish to go from the general observation that the pandemic disrupted schooling to the much more specific observation that a specific aspect of the pandemic response (like school closings) is responsible for a certain portion of the decline. We just don't have evidence for that at the moment."
Indeed, Hutt pointed out that even if there is a measurable statistical effect around school closures, it would likely be "amplified or mitigated by a host of other factors" such as students' socio-economic status, the quality of their online instruction, their age and other variables.
"We see declines across almost all race ethnicities; however, you do see a larger drop when you compare the drops of black students and Hispanic students to white students."
"The fact that no state saw an increase in NAEP scores should be an indicator that the pandemic was challenging for everyone and resulted in wholesale disruption of daily lives and routines (inside school and out)," Hutt concluded. "Even in places where schools were open it is fair to wonder what instruction in those settings was like and not hard to imagine it might have been less effective than usual."
Schneider also underscored the unprecedented nature of the COVID-19 outbreak.
"These students have lived through a once-in-a-century global pandemic," Schneider wrote to Salon.
Dr. Dan Goldhaber of the American Institutes for Research, who wrote a 2021 article for the peer-reviewed journal Phi Delta Kappan on the pandemic's impact on education, confirmed that the NAEP results are not on their own sufficient to determine whether school closings and remote school caused lowered test scores. At the same time, he added that "I do think that remote schooling explains a portion of the test score decline," referring Salon to a May study run by Goldhaber's Calder Center that used testing data from more than 2.1 million students.
"Students in schools that were remote more often in 2020-21 did considerably worse than those that were in person in that school year, but test scores were also down for in-person schools," Goldhaber explained. He added that, in the same report, they found that "schools being remote had a disproportionately bad impact on schools serving economically disadvantaged students."
In addition to the unique stresses that students were forced to bear during the pandemic, education quality likely declined because of the fact that teachers also faced unusual pressures — particularly stress.
"Another potential factor is trauma and stress for school staff, who were experiencing many of the same things as their students," Goldhaber's Phi Delta Kappan co-author Dr. Paul Bruno, a professor of education at the University of Illinois at Urbana-Champaign, told Salon by email. "Certainly there is evidence that educator stress increased during the pandemic. So it's not hard to imagine that even when instruction was being provided to students, either in person or online, both students and school staff may have been dealing with issues that meant there was less of a focus on conventional academics, or that instruction may have been less effective in those areas."
"The fact that no state saw an increase in NAEP scores should be an indicator that the pandemic was challenging for everyone and resulted in wholesale disruption of daily lives and routines."
Having established that school closures were only part of a much larger cluster of pandemic-related factors that harmed education, the inevitable question becomes what structural problems in American society led to these outcomes. According to Dr. Richard Wolff, professor emeritus of economics at the University of Massachusetts–Amherst, the core issue is that capitalist societies like America's focused more on helping out capitalists during the pandemic than they did helping the vast majority of struggling citizens. As Wolff put it, "the values contest arose over saving life and health versus saving profit and business."
"Lockdowns were widely opposed by business for that reason," Wolff told Salon by email. Wolff also noted that private schools tended to have more resources to try to keep learning processes going, including having more capacity for parent volunteers. By contrast, children who did not come from privileged background were more likely to simply suffer with inadequate education. Wilburn confirmed this when describing the racial disparities picked up by the NAEP report.
"We see declines across almost all race ethnicities; however, you do see a larger drop when you compare the drops of black students and Hispanic students to white students," Wilburn explained. Yet even there, the results could be complicated: "For eighth grade reading, white students were the only group to show a decline. All the other race ethnicities were flat. And so that actually narrowed the achievement gap between white, black, and white and Hispanic students."
Dr. Jahneille Cunningham, a postdoctoral scholar at the University of California–Los Angeles School of Education, wrote to Salon that both previous research and anecdotal observations confirm that there have been racial and economic disparities in how COVID-19 has impacted the education system.
"We know that students from low-income households were disproportionately impacted by the closure of schools," Cunningham explained. "Additionally there is evidence to suggest that school closures were more common in schools with higher proportions of students who identify as Black, Indigenous and People of color (BIPOC), students experiencing homelessness, and students who are English language learners."
Cunningham also noted that the tests themselves are not necessarily reliable.
"Education researchers have long argued that standardized test scores do not fully capture learning, but rather a student's ability to take a test on a specific day in a particular setting," Cunningham wrote. "The term 'teaching to the test' captures this sentiment, as educators have been held accountable for their student's test performance and consequently focus their teaching on what students will be tested on. That being said, we must critically examine what is being measured on these tests in the first place."
For his part, Wilburn concluded that parents who wish to learn more about education policy and their children should focus less on the national picture and more on the details which emerge when you look at individual states and districts.
"We have 26 districts when we get representative representational data for, and in eighth grade reading, the large cities are actually stable," in contrast with national figures. "There are some places that I would ask your readers to dig into and try to see what patterns that they may find in the data."
Shares