Identifying Sensitive Questions with Non-Reactive Methods

Sensitive questions result in misrepresented responses and identifying those questions is a crucial task to get straight answers to sensitive topics. Using paradata as non-reactive data should reflect the uncertainty and the perceived social desirability while answering the questionnaire. By tracking response change and page change in addition to item non-response and response latencies, the researcher has a powerful tool to reveal exceptional response patterns in web surveys. In pretests these methods can also help to cut costs of further surveys that are usually used to identify sensitive questions and they can also show problems with wording and too high cognitive effort in earlier stages.

Non-reactive methods such as measuring response latencies and event tracking adds information to be analyzed without adding load to the respondents. Such events while filling in the questionnaire can be, for example, changing the page and selecting or changing an answer option. Tracking these events not only by time, but with multiple answers over time per item, view the process of completing the questionnaire instead of leaving the black box untouched, which is between handing out the form and getting the (final) responses. In this way it is possible to reveal answers of first choice, response change and page change as event types besides response latencies and the order of completion.

The paradata from a web survey is compared to data from a questionnaire about the perceived social desirability and threat of disclosure of the questions used in the former one. The response patterns uncovered by the event tracking should correlate with these sensitivity measurements. Sensitive questions should have, in addition to the occurrence of non-response, a) higher response latencies, b) multiple page views, c) more response changes and d) lower intra-item correlation between the first and final answer.