What We Need to Know about Cross-Country Equivalence When Studying Gender Differences in Labour Market Outcomes

Since early 1960s comparative sociology has been relying on survey data from nationally representative samples. The 1980s saw a rapid growth of cross-national survey collections which continues in the 21^st century. While much progress has been made in ensuring international comparability of attitudinal indicators, non-equivalence problems are rarely considered in the aftermath of data collection. Instead researchers often assume that if items have been carefully developed, tested and well translated, their meaning is approximately the same in heterogeneous populations. This pervasive methodological oversight has led to flaws in accepted conclusions of some influential social research (see  ANNU. REV. SOCIOL. 2014 or SOC SCI RES 2015).

One of the key challenges facing the research on labor market experiences of men and women is to distinguish universal processes from country-specific legal, economic and social conditions. Yet, the research on work experiences that compares men and women’s perceptions of their work autonomy, discretionary time, promotion prospects, satisfaction and career development opportunities has employed cross-national survey data without due concern to the implications of potential distortions to cross-country measurement equivalence.

To alert researchers to these problems we present a comprehensive analysis of a number of frequently used scales developed to measure labour market experiences in the European Social Survey, the World Values Survey, the International Social Survey Programme, and Survey of Adult Skills. We assess these scales with four strategies: 1) multi-group confirmatory factor model with several sets of fit indices, 2) regression approach, 3) multilevel confirmatory factor analysis and 4) alignment optimization. Nearly all scales display a substantial degree of international non-equivalence and lack of cross-gender comparability. We present substantive examples with minimal technical detail and propose practical solutions for researchers in comparative sociology and related fields.