Mixed-Mode Design and Quality of Response in Panel Surveys

Mixed modes are commonly used in large-scale panel surveys, and the mode effects they evoke have been widely discussed. The mode effects associated with mode sequences in mixed-mode surveys, particularly in the concurrent or sequential designs, have attracted considerable attention. Concurrent designs offer participants a choice of modes at the first contact. Sequential designs offer one mode first, followed by another mode for those who did not participate in the first mode. Both designs are subject to selection effects, as participants can choose the mode they like. Concerning cost, response rates, or data quality, previous mixed-mode surveys have explored alternative designs by tailoring mode sequences or initial contact strategies for subgroup, based on the cost of modes, previous response behaviors, respondent characteristics, and incentives. Selection effects, however, remain to bias the estimates of mode effects.

This study examines mode effects across the alternative designs different from traditional concurrent and sequential mixed-mode approaches, using four-wave panel data from the Panel Study of Family Dynamics surveys collected in 2018, 2020, 2022, and 2024. A common feature of the four waves is that the targeted samples were pre-assigned to web mode or face-to-face mode, primarily based on their level of cooperation in previous waves and the availability of email addresses. In the 2018 and 2020 surveys, web mode and face-to-face interviews were conducted simultaneously. In the 2022 and 2024 surveys, web mode was conducted first, followed by face-to-face interviews. We use propensity score matching to control for selection effects, enabling us to estimate the mode effects on response behaviors. The outcome variables include item nonresponse (e.g., “refusal” and “don’t know” answers), response styles on attitude scales, and socially desirable responses. This study helps to understand the difference in mode effects on response behaviors between two types of mixed-mode surveys activated simultaneously and sequentially.