A Replication of the Experiment on Separating Scale Points from Non-Substantive Responses of Tourangeau, Couper, and Conrad 2004

Web surveys are commonly based on self-administered modes using written language to convey information. This kind of language is usually accompanied by visual cues. Research has shown the visual placement of response options can affect how respondents answer questions because they sometimes use interpretive heuristics. One such heuristic is called the “middle means typical” heuristic. It implies that the middle option serves as an anchor since it is seen as the most typical one. We replicate the experiment on “separating scale points from non-substantive responses” by Tourangeau et al. (2004) and extend it by using eye-tracking methodology. Specifically, we investigate respondents’ response behavior when the conceptual midpoint of a response scale falls on one side of the visual midpoint – i.e., the conceptual midpoint is unequal to the visual one. By recording respondents’ eye movements, we are able to observe how they process the questions and options to draw conclusions about their processing. We conducted a lab experiment with three groups: In the first group (n=45) the non-substantive options were presented as additional radio buttons. In the second group (n=46) the non-substantive options were separated by a space. In the third group (n=41) the non-substantive options were separated by a line. Altogether, we expect a higher fixation count and time on the first half of the response scale and on the conceptual midpoint in the conditions with separation because it is assumable that there is no shift in respondents’ perception of the response scale due to the consistent midpoints. In these conditions, we also expect a higher fixation count and time on the non-substantive options and more gaze-switches between substantive and non-substantive options. Finally, we expect a higher amount of non-substantive responses in the conditions with separation and a mean shift toward the visual midpoint in the condition without separation.