The Consequences of Ignoring Measurement and Aggregation Error in Random-Effects Age, Period, and Cohort Analysis

Quantitative research in the social sciences making use of individual-level data to form contextual level indices stands at risk of falling prey to both measurement error and aggregation error (Marsh et al. 2009). Measurement error emerges when observed indicators measuring the same construct(s) are summed up to one or more indices without specifying the underlying latent variable, the factor scores that link the latent variable(s) to the observed indicators, and related disturbance terms. Complementary, aggregation error emerges when a manifest or latent individual-level variable is aggregated on the contextual level instead of estimating it immediately on the contextual level.

We elaborate on the consequences of neglecting both measurement and aggregation error in multilevel age, period, and cohort analysis using German Media Analysis (MA) data 1978-2009 as well as German Politbarometer (PB) data. Regarding the MA, we first analyze the association between individuals' leftist and rightist party preference on the one hand, and usage of leftist and rightist quality papers on the other hand (cognitive consonance effect; cf. Festinger 1957). We use cross-classified random effects models to disentangle age, period and cohort effects (Yang & Land 2006). Next, we use additional PB data to estimate doubly-latent measurement models for political involvement by period and cohort controlling for both measurement and aggregation error. We store period- and cohort-specific latent means of these between-level models and merge it to the MA data in order to account for the variance of the cognitive consonance effect over time. Finally, results are compared to more 'naïve' accounts of contextual-level index building.