Comparing the Quality of Personal-Register and Non-Personal-Register Samples in Cross-National Interviewer-Administrated Surveys Using Internal Criteria of Representativeness - Comparative Analysis Based on European Social Survey Data

Wednesday, 18 July 2018: 18:15
Oral Presentation
Piotr JABKOWSKI, Institute of Sociology, University of Poznan, Poland
Piotr CICHOCKI, Adam Mickiewicz University, Poland
This presentation examines demographic representativeness of different types of probabilistic samples based on the results of 130 different surveys form 7 rounds of European Social Survey. Focusing on the distinction between personal-register (i.e., individual name) and non-personal-register (i.e., household or address) samples, it demonstrates that the latter exhibit systematically larger gender- and age-biases. Expanding upon a ‘gold standard’ evaluation (Groves 2006), an analysis based on internal criteria (Sodeur 1996, Kohler 2007) leads to the conclusion that the inferior quality of surveys involving interviewer-driven within-household selection of target respondents results from the impact of illegitimate substitutions. Such protocol-defying selection of individuals with higher levels of readiness and availability only superficially improves survey outcome rates while yielding samples of actually inferior quality. The internal-criteria approach provides a straightforward and undemanding way of monitoring representativeness of samples, and proves especially handy when it comes to large cross-country projects as it requires no data external to the survey results, and allows for comparing surveys regardless of possible differences in sampling frames, sampling design and fieldwork execution procedures.

[1] Groves, R. M. (2006). Nonresponse rates and nonresponse bias in household surveys. Public Opinion Quarterly, 70(5), 646-675.

[2] Kohler, U. (2007). Surveys from inside: An assessment of unit nonresponse bias with internal criteria. Survey Research Methods, 1(2), 55-67.

[3] Sodeur, W. (1997). Interne Kriterien zur Beurteilung von Wahrscheinlichkeitsauswahlen. ZA-Information/Zentralarchiv für Empirische Sozialforschung, 41, 58-82.