The Miller Prize for is awarded for the best work appearing in Political Analysis the preceding year.
2021 Winner | |
Recipients |
Reagan Mozer (Bentley University)
Luke Miratrix (Harvard)
Aaron Russell Kaufman (NYU Abu Dhabi)
L. Jason Anastasopoulos (University of Georgia)
|
Work | "Matching with Text Data: An Experimental Evaluation of Methods for Matching Documents and of Measuring Match Quality" |
Citation |
On behalf of this year's Miller Prize committee (myself, Alexander Theodoridis, Patrick Brandt, and Jeff Gill), I’m delighted to announce the winner of the Society for Political Methodology’s 2021 Miller Prize for the best paper published in Political Analysis. This year the prize goes to the article "Matching with Text Data: An Experimental Evaluation of Methods for Matching Documents and of Measuring Match Quality," by Reagan Mozer, Luke Miratrix, Aaron Russell Kaufman, and L. Jason Anastasopoulos. The paper represents a significant advance in the important area of incorporating text data into a causal-inference framework. Please join us in congratulating the authors for this excellent piece of scholarship. The abstract is pasted below. Matching for causal inference is a well-studied problem, but standard methods fail when the units to match are text documents: the high-dimensional and rich nature of the data renders exact matching infeasible, causes propensity scores to produce incomparable matches, and makes assessing match quality difficult. In this paper, we characterize a framework for matching text documents that decomposes existing methods into (1) the choice of text representation and (2) the choice of distance metric. We investigate how different choices within this framework affect both the quantity and quality of matches identified through a systematic multifactor evaluation experiment using human subjects. Altogether, we evaluate over 100 unique text-matching methods along with 5 comparison methods taken from the literature. Our experimental results identify methods that generate matches with higher subjective match quality than current state-of-the-art techniques. We enhance the precision of these results by developing a predictive model to estimate the match quality of pairs of text documents as a function of our various distance scores. This model, which we find successfully mimics human judgment, also allows for approximate and unsupervised evaluation of new procedures in our context. We then employ the identified best method to illustrate the utility of text matching in two applications. First, we engage with a substantive debate in the study of media bias by using text matching to control for topic selection when comparing news articles from thirteen news sources. We then show how conditioning on text data leads to more precise causal inferences in an observational study examining the effects of a medical intervention. |
Selection committee | Bear Braumoeller (Ohio State), Alexandar Theodoridis (UC, Merced), Patrick Brandt (UT, Dallas), and Jeff Gill (ex officio, American) |
Past Recipients
Past Selection Committees
Year | Committee |
2020 | Bear Braumoeller (Ohio State), Alexandar Theodoridis (UC, Merced), Patrick Brandt (UT, Dallas), and Jeff Gill (ex officio, American) |
2019 | Pablo Babera (LSE), Jennifer Pan (Stanford), and Jeff Gill (American University) |
2018 | Jennifer Pan (Stanford), Pablo Barberá (LSE), and Jonathan Katz (CalTech) |
2017 | Patrick Brandt (UT Dallas, chair), Devin Caughey (MIT), Sunshine Hillygus (Duke) and Michael Alvarez (Cal Tech, ex officio) |
2016 | Neil Malhotra (Stanford, chair), Megan Shannon (Colorado), Arthur Spirling (NYU) and Thad Dunning (UC Berkeley) |
2015 | Neil Malhotra (Chair), Thad Dunning, Meg Shannon, Arthur Spirling |
2014 | David Nickerson (Chair), Devin Caughey, Justin Grimmer, Brad Jones |
2013 | David Nickerson (Chair), Devin Caughey, Justin Grimmer, Brad Jones |
2012 | Burt Monroe (Chair), Justin Grimmer, David Nickerson, Greg Wawro |
2011 | Dan Wood (Chair), Kosuke Imai, Greg Wawro, Burt Monroe |
2010 | Dan Wood (Chair), Kosuke Imai, Greg Wawro, Burt Monroe |
2009 | Dan Wood (Chair), Kosuke Imai, Greg Wawro, Burt Monroe |
2008 | Tobin Grant (Chair), David Darmofal (winner from previous year), Michael Hanmer, Orit Kedar, Drew Linzer |
2007 | Brian Pollins (Chair), Robert Franzese (winner from previous year), William Berry |
2006 | Brian Pollins (Chair), David Nickerson (winner from previous year), Stanley Feldman |