Producing Unbiased Estimates of Propensity Models During Data Collection

There is an increasing interest in responsive designs (Groves, R. M., and Heeringa, S. G. [2006], “Responsive Design for Household Surveys: Tools for Actively Controlling Survey Errors and Costs,” Journal of the Royal Statistical Society: Series A (Statistics in Society), 169(3), 439–457). These designs make use of incoming data from the field to make decisions about design changes. Estimates from propensity models are a potential input to these designs. For example, estimated propensities may be used to predict outcomes from additional effort or to assign different data collection protocols to cases. However, if these models are estimated during the data collection, they may be vulnerable to biased estimates of coefficients and associated propensities. When these biases may occur is an empirical question. This paper looks at several empirical examples and considers alternative solutions to the problem of producing unbiased estimates.