Healthy disagreement among scientists drives the creation of new knowledge and is a necessary precursor to consensus upon which technologies, policies, and new knowledge can be built. Yet, in spite of its prominence in popular and theoretical models of scientific progress, disagreement has received little empirical attention, with progress stymied by a lack of appropriate data and widely-accepted quantitative indicators. In this talk, we outline progress in overcoming these challenges, illustrating how increasingly-available full-text data and new approaches to measuring disagreement are paving the way for a more comprehensives, empirical, and quantitative understanding of the salience and features of disagreement in science at multiple levels of analysis. Using a rigorously-validated cue-word based approach, instances of disagreement are identified from the citation sentences of millions of publications, and incorporated into a singular indicator of disagreement. Using this indicator, we simultaneously reveal the structure of disagreement between macro-level fields and the enormous heterogeneity across meso-level subfields. At the micro-level, we complement these data with published comments—the most unambiguous instance of criticism in science—in order to better understand the sociological drivers of disagreement, including author gender, seniority, prestige, and more. This project establishes a firm methodological and empirical foundation for a science of scientific disagreement, which will prove essential for validating theories of scientific progress, building tools for scholarly search and discovery, designing consensus-aware science policy, and for effectively communicating epistemic uncertainty and consensus to the public.

Dakota Murray is a Postdoctoral research associate at the Center for Complex Network Research at Northeastern University, and has demonstrated experience studying sources of bias in scientific careers, the mobility of scientists, and disagreement in science. He holds a doctorate in Informatics from Indiana University Bloomington. His current research focuses on leveraging full-text data and computational tools for developing an empirical and quantitative approach for studying scientific debates and consensus.