Effects of the Visual Presentation of Don't Know Options in Rating Scales on Responses. Comparing Results of a Papi and an Online Panel Experiment

Several studies show that in self-administered settings respondents use both, verbal information as well as nonverbal, visual features of rating scales within the question response process. Nonverbal, visual scale features (VSF) are for example font, size, color, or graphical display of the rating scale. However, the findings of the existing studies are not consistent and thus, there is still much to learn about the specifics of personal and situational settings that make respondents susceptible for VSF.

This research examines attitude strength and demographical characteristics as potential moderators of the effects of VSF. Two randomized experiments were conducted varying the visual layout of a five-point agreement-disagreement rating scale. Three different visual layouts were tested in each experiment: a. the don’t know option is separated from the other scale categories by a clearly visible vertical divider line, b. the don’t know option is simply added to the right hand side of the scale, and c. no don’t knowoption is offered. The first experiment included 307 German students using PAPI mode; the second experiment included 450 respondents of the GESIS online access panel.

The assumption is that respondents are influenced in their selection of the middle, extreme and don’t know response category by the VSF. By means of multilevel analysis, the likelihood of selecting these categories are estimated for both studies separately. For the paper-and-pencil experiment significant effects of the VSF on the likelihood of response category selection was found; attitude strength shows to be a strong moderator of the VSF effects. However, in the online setting these findings could not be replicated; demographic variables have no or just little moderating effects.