“Last Year Your Answer Was ...”: The Impact of Cognitive Effort, Life History, and Dependent Interviewing on Measures of Change

Longitudinal surveys allow for studying response change within respondents not possible in cross-sectional studies. Prior studies, however, suggest reports of change in longitudinal studies are potentially error-prone. Proactive dependent interviewing (DI) reminds respondents of previous answers, asking if there has been any change since the last survey, and is a possible method to reduce errors by assisting recall and reducing cognitive burden.  However, DI also may lead to satisficing, allowing acquiescence by stating that the situation is the same when it is not, leading to underreporting of change.

The Innovation Panel (IP) survey in the UK has conducted experiments on wording of several DI questions (both subjective/objective and categorical/continuous) across 5 waves.  The first two waves asked questions independently (no DI); waves 3 and 4 asked two versions, reminding respondents of previous answers and asking either “Is this still the same?” or “Has this changed?”; Wave 5 (mixed-mode face-to-face and web) includes both wordings as well as two forced choice versions including same/changed options.

Results indicate that first, DI does not always produce lower levels of change compared to independently asking questions. Second, asking the unbalanced “still the same” version leads to less change in general than other versions, suggesting possible acquiescence in DI.  Third, the web survey elicited significantly less change, again suggesting that reductions in change using DI may be in part due to reductions in cognitive effort. Fourth, people who change responses in any given wave were significantly more likely to also change in the next wave, suggesting a respondent component, either in response style or actual volatility in status.

We explore these findings further by including measures of cognitive ability and other respondent characteristics as well as volatility in status, captured through life course histories.  A discussion about the impact of DI and possible improvement concludes.