Ways to Uncover Cognitive Processes: A Comparison of Cognitive Interviewing and Online-Probing

Cognitive processes shape the way respondents answer survey questions: how they understand a question, retrieve the relevant information, form a judgment and how they communicate it. At the different steps of this process respondents can encounter problems.

Cognitive interviewing is a technique which aims to uncover the cognitive processes of respondents when answering a question and to detect problematic items in this regard.

In the last years the number of international survey programs increased. Cognitive interviewing is a valuable tool to reveal differences in the item interpretation across countries. The application of cognitive interviewing in cross-national surveys has so far been restricted, due to, e.g. restricted availability of cognitive interviewers across countries (Miller et al. 2011).

One additional inexpensive solution is online-probing which implements probing techniques from cognitive interviewing within online surveys. Although online-probing has been developed only recently, both techniques, traditional cognitive interviewing and online probing, have been applied successfully and could shed light on comparability issues of items in cross-national research.

However, there is still a remaining research gap. So far, both techniques have not been compared. Within the project “Optimizing Probing Procedures for Cross-National Web Surveys” data has been collected which enables such a comparison. The presentation will shed light on the differences and similarities between cognitive interviewing and online-probing. In particular, it will show what type of cognitive problems the different techniques are able to detect and in which context which of the two techniques proves itself more useful to improve items for future use.