Disclosing Hypotheses for a Badge: An Agent-Based Model of Preregistration

Open science aims at increasing the reliability of scientific findings as well as the trust between scientists through the promotion of transparency. These measures are designed to reduce informational asymmetry, which lies at the basis of mistrust. However, both theoretical understanding and empirical evidence on the efficacy of transparency in restoring trust remain incomplete.

This paper focuses on preregistration, the practice of depositing hypotheses, experimental design, and statistical plans before data collection. Preregistration aims to prevent Hypothesizing After the Results are Known (HARKing), which can reduce the reliability of statistical analyses.

We argue that, besides acting as digital repositories, infrastructures like the Open Science Framework (OSF) provide researchers with a signaling system that may alter the incentives to disclose private information. According to information economics, when producers can emit verifiable signals and consumers exhibit mistrust or skepticism toward those who do not reveal what they know, a full unraveling occurs so that types are correctly identified in equilibrium (Full Disclosure Principle). We hypothesize that a similar mechanism should support widespread preregistration. Scientists who have clear hypotheses before running their experiments should choose to preregister them (voluntarily disclose) to avoid being mistaken for those who may hypothesize ex-post. Skeptical consumers of science (reviewers) observe these decisions and should infer whether a paper is HARKed.

Using an agent-based model, we explore the conditions for widespread preregistration without top-down intervention. While preregistration is optimal at the population level, barriers such as signal costs, reviewer polarization, and uncertainty about signal meaning hinder compliance. In light of these results, we propose policy interventions that could incentivize scientists to preregister. Using our ABM, we show that non-monetary incentives like badges significantly increase preregistration by raising transparency salience and reducing uncertainty. These incentives also lower the number of early adopters needed for the norm to unravel.