Agenda Item
b. Approval of the Renewal of Northwest Evaluation Association (NWEA) Measures of Academic Progress (MAP) Assessment Systems ( in an amount not to exceed $1,300,000.00) - (updated 07.08.2022)
Summary: Presented by: Mr. Oliver J. Lewis, III, Associate Superintendent, Office of Continuous Improvement (Accountability)
Request: In support of Board Policy II, it is requested that the Board of Education approve the renewal of the Northwest Evaluation Association (NWEA) Measures of Academic Progress (MAP) Assessment for the 2022-2023 school year in an amount not to exceed $1,300,000.00.
The requested purchase is an assessment expenditure that does not require bid per the District’s Purchasing Board Policy (DJE).
Why: The DeKalb County School District currently utilizes Measures of Academic Progress (MAP) as a formative, interim assessment in grades K-10 in Reading, Language Usage, Mathematics, and Science. MAP also serves as the District’s universal screener to identify gifted and magnet students, as well as provides supplemental student achievement data to guide Multi-Tiered System of Supports (MTSS) at the local school level.
Additionally, the District requires a tool that:
Is Diagnostic (measures student growth and identify students in need of additional support)
Provides students with an online learning path/continuum-based students’ identified need
Allows the District to compare student performance against other students to determine growth and mastery percentile gaps (this is particularly important for our at-risk subgroups)
Allows the District to determine students level of mastery based on state standards and learning criteria (i.e., success criteria and learning targets) through the use of RIT scores
Can be used as a predictor towards the GA Milestones, SAT and ACT
Provides easy to understand data district, school and individual student reports (at the demographic level)
Provides Family Reports
In preparation for the 2022-2023 school year, the requested renewal is to permit the DeKalb County School District to enter into a continued agreement with NWEA to offer district-wide licenses for MAP assessments and to provide professional learning services to ensure all staff build efficiency in the instructional application of assessment data.
The renewal of MAP reduces the number of norm-referenced assessments offered in a single school year as it replaces the following assessments: Renaissance’s STAR Early Literacy, STAR Reading, STAR Math, the Iowa Test of Basic Skills (ITBS).
Details: This proposed budget item is requesting funds not to exceed $1,300,000.00 to cover the costs associated with the District renewal of the NWEA MAP assessment system and professional learning services.
NWEA Measures of Academic Progress (MAP) is a suite of assessments that deliver data to make a difference in student progress. MAP Growth measures areas related to the Georgia Standards of Excellence in RIT (Rausch Unit) scores. As a result, educators can easily measure growth in learning through the school year, and from year-to-year in Reading, Math, Language Usage, and Science, which also affords educators the capability to differentiate instruction based on the tool’s functionalities. The tool is built on one of the most robust normative scales in the United States.
MAP assessments are computer-adaptive tests that increase in rigor as students answer items, correctly. When a student incorrectly answers a question, the test becomes less difficult as a means to match the next test item with the student’s academic aptitude. Therefore, a student takes an assessment created explicitly for his or her learning level. In addition, MAP Skills provides each student with his/her own specific learning path that identifies skill gaps and provides activities to help students master specific foundational skills.
The GA Department of Education is also closely determining the possibility of the NWEA MAP Assessment as a viable replacement of the year end summative assessment, currently known as the Georgia Milestones. A comprehensive pilot through the GA MAP Partnership has been underway and a stated decision is expected based on the pilot.
Financial impact: The financial impact is an amount, not to exceed, $1,300,000.00 for FY23.
The purchase amount will be paid using general funds budgeted to the Office of Continuous Improvement (Accountability), Department of Assessment Administration, charge code:
100.1000.530010.00011.7540.9990.8010.030.0000 (Other Purchased Services)
Contact: Mr. Oliver Lewis, Associate Superintendent, Office of Continuous Improvement (Accountability), 678.676.0300
Ms. Allison Q. Scott, Director of Assessment Administration, Office of Continuous Improvement (Accountability), 678.676.0300
Effective: July 2022
Status: Contract approved by General Counsel.
Student Achievement Growth
During the COVID-19 Pandemic
Insights from Metro-Atlanta School Districts
Tim Sass & Thomas Goldring
Metro Atlanta Policy Lab for Education
May 2021
Student Achievement Growth During COVID-19 1
Highlights
• The COVID-19 pandemic has led to substantial reductions in student achievement
growth, and these impacts have grown over time.
• The impacts of the pandemic on student achievement growth vary considerably by
subject, grade level, and school district.
• Students eligible for free or reduced-price meals—a crude measure of poverty—
generally experienced lower achievement growth (relative to similar students prior
to the pandemic) than did students who were not eligible for subsidized meals, but
the magnitude of the differences varied considerably across grades, subjects, and
districts.
• On average, historically marginalized groups, such as Black students, Hispanic
students, and English learners, tended to experience greater reductions in
achievement growth (relative to similar students prior to the pandemic) than did
White and English-proficient students, but these differences varied substantially by
grade, subject, and district.
• Students who returned to in-person instruction in fall 2020 experienced greater
achievement growth per instructional day than did students who continued to
learn remotely, but their growth was still less than that of in-person learners prior
to the pandemic.
Policy Recommendations
In the wake of the COVID-19 pandemic, public school districts in Georgia face crucial
decisions on how to use federal funds to get students back on track academically. Based
on our research and the existing evidence, we recommend:
1. Differential supports and interventions based on each student’s achievement
growth;
2. High-dosage, small-group tutoring for students who have incurred the most
significant impacts;
3. Learning opportunities during the summer and other breaks that are targeted
toward students whose learning growth has been significantly diminished, are tied
to classroom content, and which provide strong incentives for participation;
4. Extended learning time during the school day that is tailored toward student needs;
and
5. Robust data systems that track who is offered additional learning opportunities, the
extent of participation, and student achievement to determine the efficacy of the
interventions and to make any necessary midcourse adjustments.
Student Achievement Growth During COVID-19 2
Introduction
The COVID-19 pandemic disrupted formal education on an
unprecedented scale.
By order of the governor, public schools in Georgia closed in mid-March 2020 and
students transitioned to remote learning for the remainder of school year (SY) 2019–20.
Many school districts offered virtual schooling, but the degree to which students could
engage in online learning varied considerably, depending in part on access to digital
devices and the availability of a reliable internet connection.1 The pandemic-induced
combination of family disruptions, closures of schools, and the swift transition to remote
learning reduced achievement growth for many students throughout the United States
heading into SY 2020–21.2
Most districts in the Atlanta metro area began SY 2020–21 with fully remote instruction.
However, as the year progressed, many schools began to offer parents a choice of in-
person instruction for their child, either in a hybrid approach with a mix of in-person
and remote instructional days or fully face-to-face instruction. Parents could also opt for
their students to continue with fully remote instruction.
Initial projections, using national data on the length of school closures and pre-pandemic
evidence of summer learning loss, indicated that students would likely start SY 2020–21
with only about two-thirds of the learning gains in reading and less than half the learning
gains in math, relative to a typical year.3 Fortunately, reports from four of the largest
formative assessment vendors found that actual reductions in student achievement
growth from the start of the pandemic to the fall of SY 2020–21, while significant,
were not as large as had been initially feared.4 More recently, however, a report from
the makers of the i-Ready assessment indicated that by the middle of SY 2020–21, the
proportion of students who were not prepared for grade-level work was much larger
than in prior years, particularly in schools in which a majority of students are Black or
Hispanic.5
Determining the extent to which student achievement growth has
decreased is an essential component of many forthcoming policy
decisions.
1
Herold (2020); Tagami (2020); Walker (2020).
2
Kuhfeld et al. (2020a); vonHippel (2020).
3
Kuhfeld et al. (2020b).
4
Curriculum Associates (2020); Bielinski, Brown, & Wagner (2020); Kuhfeld et al. (2020b); Renaissance
Learning (2020).
5
Curriculum Associates (2021).
Student Achievement Growth During COVID-19 3
Understanding the magnitude of declines in achievement growth will help districts
determine what sort of intervention strategies will be sufficient to counteract the losses
and what resource levels will be required to meet the challenge.
Evidence on differences in achievement growth across grades
and student groups can be used by policymakers to target
interventions to students with the greatest need.
Similarly, knowing how achievement growth varies by instructional mode (remote, hybrid,
and face-to-face) will inform decisions about the use of remote instruction, both for
the remainder of the pandemic and beyond. There is a pressing need for such evidence.
Georgia schools will receive $4.25 billion in funding from the American Rescue Plan
Act of 20216 and districts must soon make decisions about how to use those funds to
remediate the impact of the pandemic on students and support their paths for long-run
success.
Methodology
Research Questions
To respond to the urgent need for information to guide evidence-based decisions, we
address the following key research questions:
1. What has been the effect of the COVID-19 pandemic on students’ achievement
growth?
2. How have changes to student achievement growth varied by grade level, subject,
and student demographic characteristics?
3. Are students who return to in-person learning likely to improve their achievement
growth over and above their rate of growth while learning remotely?
4. What strategies will be most effective in helping students recover from the
educational impacts of the pandemic?
Sample Description and Empirical Methods
To analyze the effects of the pandemic on student learning, we use administrative
data from three metro-Atlanta school districts for the period from SY 2017–18 to SY
2020–21. We combine information on students’ enrollment, grade, and demographic
characteristics (including their race, ethnicity, gender, free or reduced price-meal status,
English learner status, and disability status) with scores on formative assessments.
Formative assessments are low-stakes exams that districts typically administer two or
three times per year; they provide a measure of student achievement at multiple points
during the school year and have been administered both before and during the pandemic.
6
Tagami (2021).
Student Achievement Growth During COVID-19 4
Our empirical approach compares each student’s actual test scores with an estimate of
their expected achievement level had the pandemic not happened. This process includes
three steps.
1. We estimate the determinants of students’ fall and winter test scores in years
prior to SY 2019–20.
2. We use these historical relationships and actual test scores in the fall and winter of
SY 2019–20 (just prior to the pandemic) to forecast what assessment scores in the
fall and winter of SY 2020–21 would have been in the absence of the pandemic.
3. We estimate the pandemic-induced difference in achievement growth by comparing
actual scores of students taking formative assessments in the fall and winter of SY
2020–21 to projected achievement levels, absent the pandemic.
Because we account for prior test-score histories and observable student characteristics
when projecting student achievement, a student’s achievement growth is not being
compared to that of the average student, but rather to what students with similar
characteristics and test-score histories in the same district would be expected to
obtain in a “normal” year. Thus, to the extent that pandemic-induced reductions
in achievement growth are larger for students who started at lower achievement
levels, they will exacerbate achievement gaps that existed in the absence of the
pandemic.
Limitations of the Analysis
Our use of formative assessments presents certain challenges. First, not all the districts
in this study used the same formative assessment over time. This limited the grade levels
we could analyze in one district; in another district, it necessitated converting scores
on one exam to the scale of another exam. Second, fewer students took formative
assessments in SY 2020–21 compared to earlier years. This means that our findings
are representative of the effects of the pandemic on students who took the formative
assessments during SY 2020–21, but may not be representative of the impacts on all
students who were enrolled in the district when the pandemic began. Third, the precise
timing of formative assessments varies across school years, which could distort year-to-
year comparison of scores. Fourth, many students took formative assessments at home
during the pandemic rather than at school, which was the standard practice before the
pandemic. It is possible that parents or others could have assisted students during test-
taking at home, which would inflate students’ scores.
For ease of interpretation, we often convert changes in test scores over time to the
equivalent number of months of learning. This conversion introduces additional fuzziness
to the calculation of achievement growth because it relies on definitions for an average
month of learning in scale score points, as well as the number of instructional days
between testing dates.
The Appendix provides details on the data and methods used in our analyses, discusses
the limitations in more depth, and provides robustness checks of the methodology and
student samples we use.
Student Achievement Growth During COVID-19 5
Finding 1: Average Impacts on
Achievement
By winter of SY 2020–21, average achievement in elementary and
middle schools was often months behind where students likely
would have been had the pandemic not occurred. Students often
fell further behind between the fall and winter of SY 2020–21,
sometimes dramatically so.
Estimates of average achievement are based on each student’s actual achievement level
on a math or reading formative assessment in the fall or winter of SY 2020–21. Growth
in achievement is measured relative to the student’s achievement level in the winter of
SY 2019–20. We compare each student’s actual achievement level to their projected
achievement level (based on historical relationships and each student’s demographic
characteristics and test scores in the pre-pandemic part of SY 2019–20). These
individual-level differences between actual and projected achievement are then averaged
within grades in each district.
A negative value for achievement growth indicates that a student’s actual achievement
level is less than one would expect in a normal year. For example, a value of –3 months
indicates that a student is nearly one-third of a 9.5-month school year behind where
they would normally be—had the pandemic not occurred.
A negative value does not imply a reduction in knowledge; rather, it signals that
achievement growth has slowed. Moreover, the change in achievement captures
everything that happened to a student during this period that could have affected
their learning, much of which is beyond the control of schools.
Figure 1 reports average math achievement on assessments taken in the fall and winter
of SY 2020–21 by district and grade level. The key takeaways are as follows: First, average
math achievement was often behind where students likely would have been had the
pandemic not occurred. Second, students in most upper elementary and middle school
grades, with some notable exceptions, fell further behind during the first half of SY
2020–21. Third, the slowdown in achievement growth during the first half of SY 2020–21
was often significantly larger than the slowdown between the start of learning-related
disruptions in mid-March 2020 and the fall assessment at the beginning of SY 2020–21.
Figure 2 is analogous to Figure 1, but it reports average reading achievement by district
and grade level. A similar pattern of results holds. Average reading achievement was
lower than what one would expect had the pandemic not occurred in almost every
grade level in the fall and in every grade level in the winter. Additionally, comparing
the results in Figure 1 for math and Figure 2 for reading, the slowdown in achievement
growth was often larger for reading than for math, although this pattern varied across
the districts.
Student Achievement Growth During COVID-19 6
Figure 1. Average math achievement estimates from winter of SY 2019–20 to fall of
SY 2020–21, and from winter of SY 2019–20 to winter of SY 2020–21, by district and
grade level
District A
0.7 1.0
0
-0.1 -0.4 -0.3
-0.8 -0.4
-0.9
Months
-1.8 -2.2
-4
-8
Grade 4 Grade 5 Grade 6 Grade 7 Grade 8
Fall Winter
District B
0
-0.9
Months
-2.0 -1.9
-2.4
-4 -3.5 -3.5 -3.2
-4.2
-5.0
-6.9
-8
Grade 4 Grade 5 Grade 6 Grade 7 Grade 8
Fall Winter
District C
0.6
0
-0.3 -0.1
-0.5
-1.5
Months
-2.0 -2.2
-3.2 -2.9
-4 -3.5
-8
Grade 4 Grade 5 Grade 6 Grade 7 Grade 8
Fall Winter
Student Achievement Growth During COVID-19 7
Figure 2. Average reading achievement estimates from winter of SY 2019–20 to fall of
SY 2020–21, and from winter of SY 2019–20 to winter of SY 2020–21, by district and
grade level
District A
0.2
0
-0.5
-1.4 -1.7
Months
-2.1
-2.8 -2.5
-4 -3.8
-4.2 -4.2
-8
Grade 4 Grade 5 Grade 6 Grade 7 Grade 8
Fall Winter
District B
0
-0.2
-1.1 -1.0 -1.0
Months
-2.0 -2.1 -1.9
-2.3
-2.7
-4 -3.1
-8
Grade 4 Grade 5 Grade 6 Grade 7 Grade 8
Fall Winter
District C
0
-1.1
Months
-2.7 -2.5
-4 -3.9
-3.4
-5.4
-6.2
-6.9
-8 -7.5 -7.3
Grade 4 Grade 5 Grade 6 Grade 7 Grade 8
Fall Winter
Student Achievement Growth During COVID-19 8
An important takeaway from figures 1 and 2 is that estimates of slower achievement
growth using assessment data from the fall of SY 2020–21 do not capture the full extent
of learning-related disruptions due to COVID-19. Many students fell substantially further
behind prior cohorts during the first half of SY 2020–21.
Finding 2: Impacts by Student
Demographic Groups
Students eligible for free or reduced-price meals generally
experienced slower achievement growth (compared to pre-
pandemic trends) than students who were ineligible, but the
magnitude of the differences varied considerably across grades,
subjects, and districts. Similarly, traditionally marginalized student
groups, including Black students, Hispanic students, and English
learners, generally experienced larger reductions in achievement
growth, but relative impacts differed substantially across grades,
subjects, and districts.
Within a given district, subject, and grade, students have been affected unequally by
COVID-19. Figure 3 displays average math achievement by eligibility for free or reduced-
price meals (FRPM), a crude measure of poverty, for two districts.7 For both districts,
achievement growth from winter SY 2019–20 to winter SY 2020–21 in upper elementary
grades (grades 4 and 5) was reduced by about one month more for FRPM-eligible
students than for students from more affluent families. Some disparities also exist in
middle grades (grades 6 to 8), though the differences are generally much more modest
than in the upper elementary grades.
7
Well over 90 percent of students in District C are eligible to receive free or reduced-price meals and
thus comparisons of FRPM-eligible and ineligible students are not very meaningful.
Student Achievement Growth During COVID-19 9
Figure 3. Average math achievement estimates for winter of SY 2019–20 to winter of
SY 2020–21, by grade level and free or reduced-price meal eligibility status
District A
1.3
0.4 0.6 0.5
0
-1.5 -1.6
Months
-1.7
-2.4
-3.1 -2.8
-4
-8
Grade 4 Grade 5 Grade 6 Grade 7 Grade 8
FRPM Not FRPM
District B
0
Months
-2.8
-4 -3.3 -3.5 -3.6
-3.9
-4.7 -4.5
-5.3
-6.9 -7.1
-8
Grade 4 Grade 5 Grade 6 Grade 7 Grade 8
FRPM Not FRPM
As illustrated in Figure 4, reading achievement growth among FRPM-eligible students
in District A was substantially lower than for ineligible students, with a disparity of up
to five-and-a-half months in sixth grade. By contrast, the achievement-growth gap in
District B was much smaller across all grade levels. This does not necessarily mean than
District B was more effective in educating students experiencing low-income during
the pandemic. A number of factors could have been at play, including differences in the
demographic characteristics of students, differences in the formative test employed,
and the fact that the proportion of students tested in District B fell substantially in the
winter of SY 2020–21. These issues are analyzed in detail in the Appendix.
In addition to comparisons by FRPM-eligibility status, we also analyzed variation in
changes to student achievement growth by students’ race and ethnicity, gender, disability
status, and English learner status. Graphical depictions of these comparisons by grade,
subject, and district are provided in the Appendix.
Student Achievement Growth During COVID-19 10
Figure 4. Average reading achievement estimates for winter of SY 2019–20 to winter of
SY 2020–21, by grade level and free or reduced-price meal eligibility status
District A
0
-0.7
Months
-2.2 -1.9 -2.0
-2.7 -2.9
-4
-4.0
-6.1
-8 -7.5 -7.3
Grade 4 Grade 5 Grade 6 Grade 7 Grade 8
FRPM Not FRPM
District B
0
-0.7 -0.8
-1.1
-1.7 -1.6
Months
-2.4 -2.0 -2.4
-2.7 -2.8
-4
-8
Grade 4 Grade 5 Grade 6 Grade 7 Grade 8
FRPM Not FRPM
For racial and ethnic comparisons, our findings generally mirror those from the
breakdowns by FRPM-eligibility status. Overall, Black students and Hispanic students
tended to experience larger reductions in achievement growth (relative to pre-pandemic
trends) than did White students. However, observed differences vary substantially by
subject, grade, and district. Thus, one should avoid drawing broad conclusions about
variations in pandemic impacts by student race or ethnicity.
For comparisons of English learners to English-proficient students, the findings are more
consistent across grades, subjects, and districts than for differences by family income
or race and ethnicity. For math, in each of grades 4 to 7 across all three districts,
average changes to achievement growth are larger for English learners than
for English-proficient students. The specific magnitudes do vary, but differences are
generally in the range of zero to two months of learning. For reading, average reductions
in achievement growth are larger for English learners in all three districts in grades
6 and 7. Likewise, losses are greater for English learners in grades 4 and 5 in both
districts B and C; in District A there is essentially no difference. As with other sorts of
Student Achievement Growth During COVID-19 11
comparisons, observed cross-district differences in pandemic impacts do not necessarily
imply differences in the efficacy of English learner programs during the pandemic.
The composition of English learners (e.g., country of birth and refugee status) varies
considerably across districts.
Interestingly, female students generally experienced smaller reductions to student
achievement growth than did their male peers. These differences were more
pronounced in middle school than in elementary school. One plausible explanation
is that peer interactions differ in an online environment, leading girls to feel more
comfortable. Alternatively, girls may be more mature than boys in the middle grades
and possess more of the self-discipline required for learning in a remote environment.
However, understanding the actual mechanisms behind these observed gender
differences will require further research.
Finally, we expected we would find that, on average, students experiencing disability
would have fared much worse (relative to prior trends) during the pandemic than their
peers who do not experience disability. In general, this is not the case. We do not find
a consistent pattern of larger average losses for students experiencing disability. Rather,
differences are frequently small or even favor students who experience disability in some
cases. One possible explanation for these unexpected results is that parents or other
caregivers of children experiencing disability may have been more likely to help their
children during test-taking at home. However, we cannot rule out the possibility that
some students with disabilities may have actually benefitted from learning in an online
environment.
Finding 3: Effect of Returning to
In-Person Learning
Students who returned to in-person instruction in the fall
of SY 2020–21 experienced greater achievement growth per
instructional day than students who continued to learn remotely,
but their growth was still less than that of in-person learners prior
to the pandemic.
Only one of the three participating districts, District A, offered in-person learning
opportunities for students during the fall semester of SY 2020–21. In that district, all
students started the semester with virtual instruction. This was followed by a phase-
in of hybrid models in September and early October. It was not until mid-October
that parents could choose full-time in-person instruction for their child. Most students
who participated in fully in-person instruction ended up spending 30–50 percent of
instructional days between their fall and winter assessments learning remotely (and
50–70 percent of instructional days learning in-person).
Student Achievement Growth During COVID-19 12
To analyze differences in student achievement growth across instructional modes,
we rely on the original exam scale scores, rather than convert scores into months of
learning. Because we are interested in relative achievement growth across instructional
modes within a single district, the months-of-learning measure is unnecessary, and
we avoid any additional uncertainty associated with the conversion from one metric
to another. We also measure changes in test scores on a per-day basis, to account
for variability in the timing of in-person instruction and differences in the dates when
individual students took their fall and winter assessments. Finally, as a point of reference,
we include the average achievement growth per instructional day from fall to winter
testing for the most recent pre-pandemic year, SY 2019–20, when essentially all
instruction was in-person and there were no pandemic-related disruptions in students’
lives.
Figure 5 shows average achievement growth per instructional day for students whose
mode of instruction in the fall of SY 2020–21 was between 30 percent and 50 percent
remote learning or between 90 percent and 100 percent remote learning. Average
achievement growth per instructional day was generally higher for students who
returned to in-person instruction during the fall semester of SY 2020–21, especially
among elementary grades. The difference was highest for grade 5 reading, in which
average achievement growth for students who returned to in-person instruction was
double that of students who were 90–100 percent remote. Middle school grades tended
to have a smaller difference in achievement growth across instructional modes, with
slightly better average achievement gains for entirely remote learners in grades 7 and 8.
Students who received in-person instruction for 50–70 percent of their instructional
days generally experienced greater achievement gains in both math and reading than
did students who learned remotely all, or nearly all, of the time. However, it is still
likely that the achievement growth of students participating in in-person reading
instruction during the pandemic was less than for students receiving in-person
reading instruction prior to the pandemic. Even if we extrapolate the gains from
in-person learning at 50–70 percent of instructional days to a hypothetical 100 percent
in-person learning (i.e., in-person instruction for the entire fall semester), fall-to winter
student achievement growth in reading would have been less than in the pre-pandemic
period, ranging from 36 percent of achievement growth per day in SY 2019–20 for grade
8 to 95 percent of pre-pandemic achievement growth per day in grade 6.
For math, the evidence suggests that in-person instruction was at least as
effective in the first half of SY 2020–21 as it was in the first half of SY 2019–20
in most grades, although the rate of learning was substantially lower for in-person
math instruction in grade 7 in the first half of SY 2020–21 than during the first half
of SY 2019–20, pre-pandemic. This finding implies that enhanced in-person learning
opportunities in the spring of SY 2020–21 will lead to improvements in achievement
growth. However, even students who return to campus will likely have greater cumulative
achievement losses in reading than they did at the end of the fall term.
Student Achievement Growth During COVID-19 13
Figure 5. Average achievement growth per instructional day in SY 2020–21, by grade level
and mode of instruction
District A: Math
0.3
Scale score
0.2 0.18
0.13 0.15
0.13 0.12 0.13
0.12 0.11
0.10 0.10
0.1 0.09 0.08 0.09 0.09 0.09
0.0
Grade 4 Grade 5 Grade 6 Grade 7 Grade 8
0% remote (pre-pandemic) 30-50% remote 90-100% remote
District A: Reading
0.3
0.23 0.23
Scale score
0.2
0.16 0.15
0.11 0.11
0.10
0.1 0.08 0.08 0.08
0.07 0.07 0.07
0.06 0.06
0.0
Grade 4 Grade 5 Grade 6 Grade 7 Grade 8
0% remote (pre-pandemic) 30-50% remote 90-100% remote
Discussion of Findings
It has been generally assumed that the COVID-19 pandemic and subsequent school
closures slowed student learning in metro-Atlanta public schools, but to date there
has been only speculation about the impact on student achievement growth. In this
report, we provide the first hard evidence on the effects of the pandemic on student
achievement growth. We find that up to the start of SY 2020–21, losses were moderate,
generally no more than two to three months of learning in most cases. However, these
losses mounted during the first half of SY 2020–21: Across most measures, students are
three to six months or more behind where they would have been had the pandemic not
occurred. Further, the reductions in achievement growth have been very uneven. There is
substantial variation in the effects of the pandemic across subject areas, grade levels, and
student demographic characteristics.
Student Achievement Growth During COVID-19 14
The return to in-person learning in the second half of SY 2020–21 will help stem the
dramatic slowdown in achievement growth. However, our analysis of learning growth
differences across instructional modes suggests that relatively slower growth in reading
achievement—especially for middle-school students—is likely to continue during the
second half of the year. This means that by the end of the school year, large numbers of
public school students in the metro-Atlanta area will likely have experienced pandemic-
related losses in achievement growth equivalent to the learning that normally occurs
over half a school year or more.
School districts now face the challenge of how best to mitigate the reductions in student
achievement that have occurred. The American Rescue Plan Act of 2021 and other
federal relief efforts will provide districts with significant financial resources to introduce
programs to help students return to their pre-pandemic learning trajectories.Yet money
alone will not solve the problem. Now more than ever, it is crucial to utilize evidence to
guide programmatic and policy decisions.
The evidence in this report and the accompanying Appendix
can be used to inform decisions about what grades and subject
areas to target and which students will need the greatest level of
support.
Prior research can also guide the selection of remediation strategies and the matching
of strategies to the level of need. Last summer, we published a curated summary of
the evidence on the efficacy and cost effectiveness of commonly used remediation
strategies.8 Reviews have also been conducted by other groups of researchers and
reached similar conclusions.9
The strategy that has yielded by far the greatest impact on student achievement and
has the clearest evidence for its efficacy is high-intensity small-group tutoring aligned
with classroom content. However, this strategy comes with the highest price-tag.
Consequently, it needs to be selectively targeted toward students who have incurred
the greatest reductions in achievement growth. There is also good evidence that
extending the length of the school day during the regular academic year can accelerate
achievement growth broadly, especially in reading.
A third promising strategy is the provision of learning opportunities during summer
or other breaks in the typical academic calendar. As with tutoring, alignment of
content with student needs and regular class curricula is key. Moreover, lack of student
participation in these programs can be a significant impediment to the efficacy of this
remediation strategy. Consequently, providing strong incentives for enrollment and
continued engagement in these programs is important.
8
Pan and Sass (2020).
9
See, for example, Allensworth and Schwartz (2020).
Student Achievement Growth During COVID-19 15
Districts should use federal funding to invest in three evidence-
based policies: supporting high-intensity, small group tutoring,
extending the school day, and creating significant incentives for
students to attend sumer school.
No matter what remediation strategies are selected, they will not work perfectly
for all students immediately, and there will be opportunities to make improvements
along the way. In order to make such mid-course corrections, it is essential to roll out
programs in such a way that targeted students and a reasonable comparison group
of non-participants can be clearly identified. It will also be necessary to keep track of
participation and continue to measure student outcomes along the way. These planning
and data collection efforts will yield more efficient use of scarce resources and bolster
the ability of districts to ensure an academic recovery over the coming years for the
students most gravely affected by the COVID-19 pandemic.
Acknowledgments
This report would not have been possible without the MAPLE research-practice
partnership that we have developed over the past four years. As always, we thank
our metro-Atlanta school district partners for their commitment to evidence-based
decision-making, guidance throughout the preparation of this report, and ongoing
partnership. Henry Woodyard, Alexa Prettyman, Sungmee Kim, Sarah Barry, and Aarthi
Arcot provided superb research assistance. We thank our colleagues at the Georgia
Policy Labs for their tireless efforts towards the preparation of this report: Chris Thayer,
Tyler Rogers, Maggie Reeves, Ketisha Kinnebrew, Dan Kreisman, Jon Smith, and Robert
McMillan. Any remaining errors are our own.
Student Achievement Growth During COVID-19 16
References
Allensworth, E. and Schwartz, N. 2020. School Practices to Address Learning Loss.
EdResearch for Recovery Brief No. 1. Retrieved from annenberg.brown.edu/sites/
default/files/EdResearch_for_Recovery_Brief_1.pdf
Bielinski, J., Brown, R., & Wagner, K. 2020. Findings from Fall Screenings: Data on COVID
Learning Loss and Updated Recommendations for Instruction. [white paper]. Irvine,
CA: Illuminate Education. Retrieved from illuminateed.com/download/findings-from-fall-
screenings-data-on-covid-learning-loss-and-updated-recommendations-for-instruction/
Curriculum Associates. 2020. Understanding Student Needs Early Results from Fall
Assessments. Retrieved from curriculumassociates.com/-/media/mainsite/files/i-ready/
iready-diagnostic-results-understanding-student-needs-paper-2020.pdf
Curriculum Associates. 2021. What We’ve Learned about Unfinished Learning: Insights
from Midyear Diagnostic Assessments. Retrieved from curriculumassociates.com/-/
media/mainsite/files/i-ready/iready-understanding-student-needs-paper-winter-
results-2021.pdf
Herold, B. 2020. The Disparities in Remote Learning Under Coronavirus (In Charts).
Education Week, April 10, 2020.
Kuhfeld, M., Soland, J., Tarasawa, B., Johnson, A., Ruzek, E., Liu, J. 2020a. Projecting the
potential impacts of COVID-19 school closures on academic achievement. Educational
Researcher, 49(8).
Kuhfeld, M., Tarasawa, B., Johnson, A., Ruzek, E., Lewis, K. 2020b. Learning during
COVID-19: Initial findings on students’ reading and math achievement and growth.
NWEA. Retrieved from ewa.org/sites/main/files/file-attachments/learning_during_
covid-19_brief_nwea_nov2020_final.pdf?1606835922
Pan, W. and Sass, T. 2020. Potential Remediation Strategies in the Wake of COVID-19
School Closures: A Review of the Literature. Retrieved from gpl.gsu.edu/publications/
remediation-covid-19/
Renaissance Learning. 2020. How Kids Are Performing: Tracking the Impact of COVID-19
on Reading and Mathematics Achievement. Renaissance Special Report Series, Fall 2020
Edition. Retrieved from renaissance.com/how-kids-are-performing/
Tagami, T. 2020. “Time Will Tell the Toll on Schools Closed for Virus,” Atlanta Journal-
Constitution, April 12, 2020, p.1.
Tagami, T. 2021. Schools Get $4.25 Billion Shot in Arm. Atlanta Journal-Constitution,
March 15, 2021, p. 1.
vonHippel, P. 2020. How will the Coronavirus Crisis Affect Children’s Learning? Unequally.
Education Next - News. Retrieved from educationnext.org/how-will-coronavirus-crisis-
affect-childrens-learning-unequally-covid-19/
Walker, M. 2020. “Some Students Locked out of Virtual Classes,” Atlanta Journal-
Constitution, April 26, 2020, p.1.
Student Achievement Growth During COVID-19 17
About the Authors
Tim R. Sass is a Distinguished University Professor in the
department of economics at Georgia State University and the
W.J. Usery Chair of the American Workplace in the Andrew
Young School of Policy Studies. He is also the faculty director of
the Metro Atlanta Policy Lab for Education (MAPLE). His research
interests include the teacher labor supply, the measurement of
teacher quality, and school choice.
Thomas Goldring is the director of research at the Georgia
Policy Labs. He supports the faculty directors in managing research
projects and providing analytical and technical support across
GPL’s three components. He has researched issues in K-12
education, including educational accountability, school finance, and
graduation rates; early childhood education; career and technical
education; post-secondary education; and education and mortality.
About the Georgia Policy Labs
The Georgia Policy Labs (GPL) is a collaboration between Georgia State University and
a variety of government agencies and school districts to promote evidence-based policy
development and implementation. Housed in the Andrew Young School of Policy Studies,
GPL works to create an environment where policymakers have the information and
tools available to improve the effectiveness of existing policies and programs, try out
new ideas for addressing pressing issues, and decide what new initiatives to scale. The
goal is to help government entities more effectively use scarce resources and make a
positive difference in people’s lives. GPL has three components: The Metro Atlanta Policy
Lab for Education works to improve K-12 educational outcomes; the Career & Technical
Education Policy Exchange focuses on high-school-based career and technical education
in multiple U.S. states; and the Child & Family Policy Lab examines how Georgia’s
state agencies support the whole child and the whole family. In addition to conducting
evidence-based policy research, GPL serves as a teaching and learning resource for state
officials and policymakers, students, and other constituents. See more at gpl.gsu.edu