Replicability in the Social Sciences: Extent, Reasons and Consequences

Thursday, 19 July 2018: 08:30-10:20
RC33 Logic and Methodology in Sociology (host committee)

Language: English

The replicability crisis in the social sciences threatens the credibility of research results and the basis of our work. However, more studies are needed to determine the exact share of empirical results that cannot be replicated, and whether this problem differs on the basis of topical areas (e.g., educational-research), research designs (e.g., experimental-research) and data-collection methods (e.g., online-surveys).

There are many reasons why previous results might not be replicated. Firs, cases of scientific misconduct have been identified, where results have been intentionally fabricated. Second, publication bias causes confirming studies to be published, whereas the majority of null results remain invisible. Third, different, but equally plausible methods of data collection and analysis often lead to different conclusions.

Replication as the most important research validation institution may take different forms: reanalyzing (a) the same or (b) alternative data with (c) the same or (d) different methods. One particular interesting form of replication methodology represents the so called ‘crowd sourcing data analyses’, where different teams conduct parallel analyses and later compare the methodology and results.

The aim of this session is (1) to stimulate the debate about the need of more replication in the social sciences, (2) to illustrate the ways  to efficiently organize this institution and (3) to gain knowledge about the most important reason for the failure to replicate results. We invite researchers to submit papers discussing all these topics. In particular we welcome examples of replications of existing results and studies comparing substantive conclusions when using different methods and data.

Session Organizers:
Jochen MAYERL, University of Kaiserslautern, Germany, Elmar SCHLUETER, Justus-Liebig-University Giessen, Germany and Volker STOCKE, University of Kassel, Germany
Oral Presentations
Reason for the Failure to Replicate Results
Andrea BREITENBACH, University Marburg, Germany
Replicating Findings Regarding Attitudes Towards the International Security Assistance Force (ISAF) in Afghanistan in Five European Countries
Henrik ANDERSEN, University of Kaiserslautern, Germany; Christoph GIEHL, TU Kaiserslautern, Germany
A Replication of the Experiment on Separating Scale Points from Non-Substantive Responses of Tourangeau, Couper, and Conrad 2004
Cornelia NEUERT, GESIS - Leibniz Institute for the Social Sciences, Germany; Jan HÖHNE, Georg-August-Universität Göttingen, Germany; Timo LENZNER, GESIS - Leibniz Institute for the Social Sciences, Germany; Ting YAN, Westat, USA