Summary

Indigo, Daisychain, and Rural Groundgame conducted an RCT in Virginia’s 2023 general election to test the effectiveness of a last-minute SMS turnout message. We randomly assigned 355,123 voters to either receive one of ten SMS turnout messages or no SMS message on election day. We found little evidence that any treatment condition increased turnout.

Research design

We first removed any voters who were Republicans, did not have a cell phone number, did not have an identifiable polling place, or had already voted. We then removed any cell phones who were identified as “do not text” by Rural Groundgame. We then randomly assigned the remaining 355,123 voters to the following conditions:

The following treatment variations are defined as follows:

  • Gentle SP: Imply that we will check whether they voted.
  • Hard SP: Information about the number of times they’ve voted in the last three elections and saying their neighborhood typically votes.
  • Polling place info: Provide the voter polling place information and a link to find more details about their polling location.
  • Polling place info Google maps link: Provide the voter polling place information, a link to find more details about their polling location, and a Google maps URL to their polling location.
  • Text with Daisychain charm: Add Daisychain’s animated “charms” to make the treatment feature (e.g., social pressure or polling place info) more engaging. 

We regressed whether voters voted (100) or not (0) on experimental condition dummy variables using ordinary least squares. We also regressed vote on dummy variables for whether a voter was assigned to a Daisychain, polling information, and social pressure treatment with all of the available treatment interactions to back out the main effects.* We use robust standard errors in all models. We also show models without and with covariates. We control for party, gender, ethnicity, age, 2022 general turnout, 2021, general turnout, and general 2020 turnout.

Results

In brief, neither a text in general nor any specific message increased turnout to a statistically significant degree, regardless of whether we added covariates. In the first table, the first two columns show that being assigned to any text condition did not clearly increase voter turnout. The last two columns show that none of the variations clearly increased voter turnout. 

We then backed out the main effects of the treatment conditions. None of the conditions are statistically significant. However, in column 1, the Daisychain feature may have increased turnout by 0.31pp (p = 0.12), but the estimate dropped to 0.09pp (p = 0.56) when we included covariates.

Discussion

These estimates are in line with the Analyst Institute's most recent estimates for cold and warm SMS which suggests texts may simply have very small turnout effects. More research on Daisychain’s charms, ideally with a larger sample size, could show more confident estimates of their distinct turnout effect.

*We mean center all of the treatment conditions so we can interpret the main effects.

Appendix

By experimental group turnout averages:

Control: 34.8% (SE = 0.27%)

Gentle social pressure: 35.1% (SE = 0.27%)

Gentle social pressure and Daisychain charm: 35.1% (SE = 0.27%)

Gentle social pressure and poll info: 34.9% (SE = 0.27%)

Gentle social pressure, poll info, and Daisychain charm: 35.2% (SE = 0.27%)

Hard social pressure: 34.8% (SE = 0.27%)

Hard social pressure and Daisychain charm: 35.2% (SE = 0.27%)

Hard social pressure and poll info: 35.1% (SE = 0.27%)

Hard social pressure, poll info, and Daisychain charm:  35.5% (SE = 0.27%)

Poll info: 34.7% (SE = 0.27%)

Poll info link: 34.4% (SE = 0.27%)