An Evaluation of the Effect of Mode-Switching in Panel Surveys Using Recall Data

A key concern about the web survey data quality is difficulty garnering a probability sample because there are no good frames of email addresses for a general population. In a panel survey, it is possible to switch respondents to web mode after initial recruitment via face to face methods (F2F), thus mitigating the problem of sample selection and allowing the collection of rich information at lower marginal cost. However, web respondents generally seem more likely to take shortcuts than respondents in interviewer-administered modes (e.g., Heerwegh and Loosevelt, 2008). This may even be exacerbated by switching from F2F to web: by contrast to an interview, self-administration feels particularly “unsupervised” and, without an interviewer to motivate them to be conscientious, web respondents may take shortcuts and minimize their effort compared to their style of participation in previous F2F interviews . This raises the more general issue of whether it is possible to maintain the integrity of time-series in which there is a midstream mode switch (FTF to web). In this paper, we report results of an experiment in a panel survey (the UK Household Panel Survey Innovation Panel (UKHLSIP)) that compared the accuracy of past event recall, validated by responses at previous interview, comparing respondents who were switched to web mode with those that continued to be interviewed F2F. We also assess how asking for pre-commitment to careful answering might mitigate any potential loss of data quality resulting from switching to web. More generally, our results allow us to understand a little better how mode-switching interacts with cognitive processes underlying survey response to produce data of varying quality.