Bias Induced By a Researcher’s Political Ideology in the Production of Research Findings

The political ideology of a researcher may introduce bias at various stages of the research process. When researching policy-relevant topics, researchers’ priors and policy preferences may shape the design, execution, analysis, and interpretation of results. Detection of such bias is challenging because research itself is not normally part of a controlled experiment. Our analysis exploits a rare opportunity where 161 researchers working independently in 71 research teams took part in an experiment. They were surveyed about their position regarding immigration policy prior to learning that their task was to test an immigration-policy-related hypothesis: that immigration reduces public support for social welfare policies. We measure the impact of prior ideological positions on empirical estimates and peer rating of each other’s model specifications. We found that research teams composed of pro-immigration researchers estimated a more positive impact of immigration on public support for social programs. For example, pro-immigration teams were 11.4 percentage points less likely to find large, statistically significant positive effects of immigration, while anti-immigration teams were 14.6 percentage points less likely to find large, statistically significant positive effects. Moreover, the methods used by teams with strong pro- or anti-immigration priors received lower peer ratings. Their lower-rated models are the mechanism causing observably biased outcomes. These results expand our understanding about the myriad sources of bias that enter the research process at different stages. The analysis makes a case for including political ideology as an important factor in the debate over making science more reliable and reproducible.