How Much Does the Mode of Response Matter? a Comparison of Web-Based and Mail-Based Response When Examining Sensitive Issues in Social Surveys

It is argued that traditional ways of collecting social surveys are threatened by the rising data-collection costs and the declining response rates. In an attempt to solve this problem, researchers have started to utilize cheaper and easier data collection methods, especially those focusing on various types of online data. Current research on survey methodology has criticized sample-to-population representativeness of many online surveys. At the same time, however, research on how the mode of data collection affects to responses is almost completely lacking. Our paper takes both of these issues into account by analyzing the differences between the two modes of data collection used in nationally representative studies: web-survey response and conventional mail-response. We examine the representativeness of the samples by comparing web-based to mail-based survey responses. In addition, we analyze whether the responses using Web-questionnaire are different from the mail-questionnaire responses when examining respondents’ attitudes towards sensitive issues such as immigrants and patriotism. Our data are derived from the International Social Survey Program (ISSP) 2013. We selected two countries for the analysis, Finland (n=1, 243) and Norway (n=1, 585), both of which applied similar methods of data collection technique (self-conducted mail survey and web survey). The results indicate that although the popularity of the Web-surveys has increased during recent years, the mode of response is still associated with socio-demographic background. When analyzing attitudes towards sensitive issues, our results suggest that the mixed-mode survey is a reliable method of data collection especially after controlling for background variables, and correcting selectivity bias by weighting the data.