Using Paradata to Monitor Interviewers’ Instrument Navigation Behavior and Inform Instrument Technical Design: Case Studies from a National Household Surveys in Ghana and Thailand

Many computer-assisted personal interview software captures paradata (i.e., empirical measurements about the process of creating survey data themselves), computer user actions, including times spent on questions and in sections of a survey (i.e., timestamps) and interviewer or respondent actions while proceeding through a survey. In these cases, the paradata file contains a record of keystrokes and function keys pressed, as well as mouse actions. These paradata files can be used for quality assurance checks and reporting, particularly when interviews are not audio recorded.

This presentation uses data from (1) the Ghana Socioeconomic Panel Study a collaboration between the by Economic Growth Center at Yale University, the Institute for Statistical, Social and Economic Research at University of Ghana, and the Survey Research Center at University of Michigan; and (2) the Evolution of Health, Aging, and Retirement in Thailand in collaboration with the National Institute of Development Administration and the Survey Research Center at University of Michigan. Both studies utilize unique team management and travel structures, and have a complex instrument design. In addition, interviewers are allowed to interview respondents within the same sample unit without any particular order and to switch among varied interviewing components in a flexible fashion. Paradata is heavily relied upon to monitor interviewers’ behaviors.

We first categorize interviewer navigation patterns such as mid-section break-offs through varied interviewing components. These navigation patterns are then inspected for predictive power against data quality indicators such as response changes and non-response. Subsequently, we analyze interviewer, household, and geographic characteristics and identify quality control metrics (e.g., interview length) to determine if interviewer behaviors and interview efficiency can be predicted by interviewer’s team behavior or household characteristics, among all other information available. Finally, we will present how analyses can be practically applied to improve interview efficiency and data quality of interviewer administered surveys.