Comparing Mobile Phone, PC and Paper-and-Pencil Survey Experience: A Lab Experiment on Mode Effects

This exploratory paper investigates the modality between data collected using an online questionnaire filled in on a mobile phone device and on laptop computer and a paper-and-pencil questionnaire. An experiment was conducted on 5th of July 2013 in thermal comfort conditions of a lab at the University of Ljubljana, excluding external effects. A total of 75 students were randomly assigned to three experimental conditions. A short questionnaire included questions on knowledge, attitudes and behaviour towards organic food products. The two online questionnaire versions were designed in a way that the visual design was most similar to the paper-and-pencil version. In order to explore the participants’ willingness-to-buy and willingness-to-pay for organic certified food, the paper-and-pencil version involved the actual presentation of a glass bottle of organic apple juice of 1 L and a package of organic spaghetti of 500 gr, while the online surveys involved the video projection of both organic foods. For the online versions, paradata on time and changes of answers were collected. The results and experiences from the three data collection modes will be explored in order to answer questions on the time differences between the modes, on the data quality, and on respondents’ performance. This experiment is one of the first studies on mode effect for mobile phone online questionnaires as the lab conditions guarantee the exclusion of confounding effects of self-selection and non-response which typically limit the measurement of the pure mode effect.