Impact of Virginia Preschool Initiative on 8th Grade Outcomes
Author: Deborah L. Jonas, Ph.D. Principal Research Scientist at SRI International
March 10, 2016
Deborah L. Jonas, Ph.D.[i]
Principal Research Scientist at SRI International
For two years, I was part of a strong and diverse research team[ii] that leveraged data from VLDS to evaluate the long-term impact of Virginia’s Preschool Initiative (VPI). VPI was established in Virginia in the 1990s, and provides communities with funds to provide more than 17,000 at-risk children with high quality preschool. The program is designed to help students arrive in Kindergarten with the skills they need to be successful in school.
This study, sponsored by the Virginia Early Childhood Foundation, was the first to assess the impact of VPI participation on middle school outcomes. The study results provided important insights about the VPI program and about the use of VLDS for research:
- Children who participated in VPI in 2005/06 Kindergarten cohort showed early literacy benefits in the first year of school, compared to similar peers who did not participate in VPI.
- Students maintained early learning gains by being promoted to 8th grade on-time more often than their peers.
- There were no detectable differences in 8th grade scores or proficiency on Standard of Learning (SOL) literacy tests for students who were promoted to 8th grade on-time and took the traditional SOL test. This may reflect a true pattern, or it may be an artifact of the methods or the data available to assess literacy skills 10 years after participating in public preK programs.
- Using VLDS to evaluate educational programs provides important and useful information to help policymakers, educators, and parents make decisions, but there are many ways to strengthen statewide data systems that would result in even more useful results.
This particular study resulted in a number of specific questions from diverse stakeholders. For example, people wondered, do early benefits of VPI on students’ reading skills demonstrated in other studies decay over time? Are some communities better able to sustain gains that VPI participants make in the early years compared to their peers, and if so, how? Are there long-term benefits of VPI participation on other important academic outcomes, such as mathematics skills or graduation rates? What about the impact of VPI on students’ social and emotional skills, which are important for school and life success?
The Virginia Early Childhood Foundation provided answers to some common questions in a Frequently Answered Questions document. However, for the most part, we don’t know the answers to these or other questions that stakeholders asked when the study was released. Some of these questions are answerable with existing data from VLDS. For example, if we wanted to know what the impact of VPI is on mathematics achievement, data are available, and researchers could use methods similar to those in the 2015 study, and estimate the impact on math outcomes. Similarly, using similar methods and waiting several years, researchers can study the impact of VPI participation on high school graduation rates.
However, there remain a large number of questions that are unanswerable with current data. For example, the study demonstrated limits of available data on children’s participation in VPI. The research team found that records from children who attended VPI in 2005/06 were incomplete—we were missing at least 20 percent of children enrolled in VPI, and perhaps more. This is because the study team had no reliable way to determine whether children were served in VPI or other public preschool programs (for more details, I’ll refer interested readers to the report). Other data limitations led the research team to spend considerable time determining the most rigorous approach to the research itself, and also developing recommendations for the state to consider to strengthen data collections to support long-term evaluation of VPI. For example, available data did not provide information on children’s preschool attendance, or whether the programs they were in were part-time or full-time, both of which are permitted with state funds. These types of limitations impact the amount or “dosage” of preschool children receive, which could influence long-term impact. Other data limitations are described in the report.
While this study offers guidance for improving data collection, the most effective approach would be to include research and evaluation experts when setting up collections.
[i] Dr. Jonas is currently a Principal Research Scientist at SRI International. Her role on the VPI evaluation was carried out as part of her role as the Tom Chewning Research Fellow, Virginia Early Childhood Foundation. All content of this blog reflects the views of Dr. Jonas. Content does not necessarily reflect the views or policies of SRI International or the Virginia Early Childhood Foundation.
[ii] The research project was funded in part by 4-VA: Universities Collaborating to Achieve Virginia's Goals for Higher Education. Researchers on the project were: John Almarode and Elissa N. Edwards (James Madison University); Isabel Bradburn (Virginia Tech); Jason Downer, Erik Ruzek, and Jamie DeCoster (University of Virginia); Deborah Jonas (Virginia Early Childhood Foundation); and Sarika Gupta (George Mason University).