Randomization inference with natural experiments: An analysis of ballot effects in the 2003 California recall election

Since the 2000 U.S. Presidential election, social scientists have rediscovered a long tradition of research examining the effects of ballot format on voting. Using a new dataset collected by The New York Times, we investigate the causal effect of being listed on the first ballot page in the 2003 California gubernatorial recall election. California law mandates a unique randomization procedure of ballot order that, when appropriately modeled, can be used to approximate a classical randomized experiment in a real world setting. We apply randomization inference based on Fisher's exact test, which directly incorporates the exact randomization procedure and yields accurate nonparametric confidence intervals. Our results suggest that being listed on the first ballot page causes a statistically significant increase in vote shares for more than 40% of the minor candidates, whereas there is no significant effect for the top two candidates. We also investigate how randomization inference differs from conventional estimators that do not fully incorporate California's complex treatment assignment mechanism. The results indicate appreciable differences between the two approaches.