Strengthening the Role of VLDS for Program and Policy Evaluation
Author: Deborah L. Jonas, Ph.D. Principal Research Scientist at SRI International
April 14, 2016
In a previous blog post, (click here ) I described results from a 2015 study of the middle school outcomes for children who participated in the Virginia Preschool Initiative (VPI).
The research team working on that study experienced a host of challenges in figuring out how best to use existing data to assess whether participation in the VPI impacts children’s middle school outcomes. The team encountered a host of challenges in the data. For example, the data did not identify all children who were enrolled in public preschool programs, nor was there a reliable way to distinguish VPI participation from participation in other types of public preschool.
These challenges lead me to pose question relevant to a wide range of education issues in Virginia:
- How can Virginia leverage VLDS to ensure that new policies and programs have efficient and sufficient data to support cost-effective, long-term, rigorous evaluation?
To ensure that children are educated with effective programs and to ensure that taxpayers are getting their money’s worth, it is important to rigorously evaluate major state-funded programs—to ensure that they are meeting policy objectives, and to improve or replace programs when needed. The urgency of strengthening VLDS and other data systems recently increased. The new federal accountability law, the Every Student Succeeds Act, requires schools to pay closer attention to the evidence base for programs that use federal funds, and to evaluate funded programs for which evidence isn’t sufficient to meet federal evidentiary standards. Further, under the new law, federal funds that support Virginia’s schools that are lowest performing can only be used for programs for which there is a strong evidence-base—further raising the bar.
While evaluation can be expensive, it is more cost-effective to plan evaluations up front, and to the extent possible, use statewide data that are collected in ways that align with rigorous evaluation methods and other accountability needs. Virginia can help more students benefit from state-funded programs by improving approaches to program implementation—in particular, by implementing and scaling programs and policies in ways that are amenable to rigorous evaluation. By maximizing the value of VLDS, Virginia has great potential to carry out cost-effective evaluation while maintaining student and citizen privacy.
VLDS has already provided insights on diverse topics including workforce development, high school students’ preparation for college, and the long-term outcomes of preschool programs. As is, VLDS is providing important and useful information about the relationship between student participation in education programs and their school, college, and employment outcomes. We also know that if state agencies partnered with evaluation experts to develop program implementation and data collection strategies that enable future rigorous program evaluation, leaders would have even better information to guide policy and funding decisions.