ISR Awards

June 2014 Supplement, Statistic back end for Gates Evaluation Engine

Since 2012, a University of Michigan-based team led by Ben Hansen, Research Associate of the SRC and Associate Professor of Statistics, has contributed statistical programming to the Evaluation Engine, a broader software project of RTI and the Bill and Melinda Gates Foundation (BMGF). As described in this contract’s original Project Description:
Many educational interventions can informatively be evaluated using data available from administrative databases, by comparing outcomes of intervention-group students to those of comparison students matched to them on pre-intervention administrative variables. Forgoing experimental control of who receives the intervention and using available outcome variables rather than arranging data collection specifically to suit the intervention, such studies should in principle be cheap and fast; in current practice, however, gaining access to the needed databases can be costly and time consuming, and may require masking of some of the data. The Gates Foundation’s Evaluation Engine project aims to drastically streamline the process of performing matched comparisons using administrative education data by presenting a web-interface, database and statistical analysis program enabling researchers to conduct statistical analyses on FERPA-protected data that they have no direct access to.
After two years of effort, the Evaluation Engine is a working prototype, assisting analysis of data from two participating states (Tennessee and New Mexico). Statistical programming support is now provided by contractors to BMFG rather than University of Michigan employees, although the PI of this project’s effort is needed to plan, coordinate and oversee remaining adjustments to the code base. His effort is also required to produce documentation and to plan and add new statistical features. The proposed contract amendment would support this effort and associated travel costs.