The Predictive Validity of Factorial Survey Experiments – a Systematic Review

Factorial survey experiments (FSEs) have become a widely used method in social science research for capturing behavioral intentions through hypothetical scenarios. Despite their extensive use, the predictive validity of FSEs—whether and to what extend reported behavioral intentions align with actual behavior—remains a topic of debate. Rare systematic validation studies show ambiguous findings across the exemplary fields of application and provide mixed evidence.

The given talk presents a systematic review of existing research, examining the conditions under which FSEs successfully predict real-world behavior. The review aims to identify key factors that influence the alignment between behavioral intentions and actions. Particular attention is given to the realism and complexity of vignettes, social desirability bias, level equivalence, context-factors and surveyed population. In addition, the review will take into consideration theoretical frameworks often employed in FSEs, such as social action theories and theories on decision making, to understand how these models may also help to explain observed gaps between intentions and behavior.

By synthesizing the findings, the review seeks to clarify the social settings and methodological conditions under which FSEs provide valid behavioral predictions in order to contribute to the ongoing methodological debate on factorial surveys. Some guiding considerations will be outlined for researchers, aiming to enhance the predictive validity of their survey instruments. In sum, the presentation contributes to the development of FSEs as a robust tool for studying complex social behaviours.