Comparing Post-Stratification and Propensity Score Nonresponse Adjustment: Bias Correction and Precision Loss – a Case Study with the Swiss ESS 2012 Data

Nonresponse bias is a well-studied issue. Some techniques to reduce this source of errors are applied during data collection (e.g., targeted fieldwork) and some post-survey. Depending on the available paradata, nonresponse adjustments can be calculated to hopefully correct for bias.  The problem, especially with low response rates, is the loss in precision that it causes. If adjustment weights vary highly, the confidence intervals become larger. The increase in standard errors can in some cases counter-balance the decrease in bias. Moreover, a good nonresponse adjustment is based on variables that highly correlate with the response propensities; such variables are rarely available. For this reason the choice of a nonresponse adjustment technique and variables used have to be thought of carefully.

We will study two nonresponse adjustments for the ESS 2012 survey in Switzerland. The first will be based on socio-demographical variables from the population register from which the sample is drawn. Such paradata are commonly used in post-survey adjustment, as they are often the only data available. But they are known to have low correlations with response propensities and with many key variables. In a second step, data from the nonresponse survey that was conducted shortly after the main ESS 2012 will also be used to construct post-survey adjustment.  The nonresponse survey is designed to collect information that correlates highly with the propensity to answer and should lead to an efficient nonresponse adjustment. The expected decrease in bias could sadly be neutralized by the possibly substantial effect on precision of such a weighting scheme. A second shortcoming is that the core of the nonrespondents that did not participate to either survey cannot be corrected for. Our aim is to compare these two methods, assessing the effect on estimates, nonresponse bias and on the precision of these estimates by applying a bootstrapping.