Big Data, Big Theory: Moving Beyond New Empiricism to Generate Powerful Explanations

Big Data is said to be ushering in a new paradigm of empiricism in which volume allows data to speak for themselves without theory. Commentators proclaim the end of theory is nigh. In this paper we address this challenge by exploring how Big Data can work with sociological theory to generate more powerful explanations. We do so through discussing a major study that explores the viability of digital inclusion initiatives in Australian schooling. We present a combined methodological framework of user behaviour and network analysis with conceptual tools from the sociological approach Legitimation Code Theory (‘LCT’).

Our data comprises students and teachers’ real-time behaviours of digital device usage (from 50,000 Android table devices over two years) at a high level of fine granularity. This dataset allows for an inductive analysis of patterns of usage and the network. However, these do not by themselves explain social practices – the patterns of behaviour themselves need explaining. This raises questions of the kind of theory capable of making sense of the results of Big Data analysis. We argue that the relational and realist nature of LCT enables it to enhance Big Data by exploring the organizing principles underlying diverse practices, contexts, beliefs and attitudes. LCT is able to take up and meet the epistemological and methodological opportunities and challenges of Big Data because it offers a flexible, multi-dimensional and responsive toolkit rather than fixed hypotheses or overarching theory into which data are fitted. Moreover, its relational nature grasps how components are related to changes in other components in a system and its sociological framework enables links between the inductively generated patterns of Big Data and the social actors involved. Bringing them together thus goes beyond the identification of patterns and frequency to reveal the often hidden organising principles helping to shape practice.