on
Blog Post 4: Expert Prediction and Incumbency
## ── Attaching packages ─────────────────────────────────────── tidyverse 1.3.1 ──## ✔ ggplot2 3.3.5 ✔ purrr 0.3.4
## ✔ tibble 3.1.6 ✔ dplyr 1.0.7
## ✔ tidyr 1.2.0 ✔ stringr 1.4.0
## ✔ readr 2.1.2 ✔ forcats 0.5.1## ── Conflicts ────────────────────────────────────────── tidyverse_conflicts() ──
## ✖ dplyr::filter() masks stats::filter()
## ✖ dplyr::lag() masks stats::lag()##
## Please cite as:## Hlavac, Marek (2022). stargazer: Well-Formatted Regression and Summary Statistics Tables.## R package version 5.2.3. https://CRAN.R-project.org/package=stargazer##
## Attaching package: 'janitor'## The following objects are masked from 'package:stats':
##
## chisq.test, fisher.test## Registered S3 method overwritten by 'geojsonlint':
## method from
## print.location dplyr## Linking to GEOS 3.9.1, GDAL 3.4.0, PROJ 8.1.1; sf_use_s2() is TRUE## To enable caching of data, set `options(tigris_use_cache = TRUE)`
## in your R script or .Rprofile.Hi everyone! We’re back this week with another set of models this time exploring the important concepts of expert prediction and incumbency. This week through building models with expert predictions and incumbency, I’ve set out to investigate: How well do experts do at explaining election outcomes on average and does incumbency have a redundant effect or add something profound to our own predictions?
To first begin investigating this question, I explored how well expert predictions have historically predicted elections in the past. I did this by comparing the true Democratic vote Share in the 2018 midterms to the expert predictions of party victory in each district. To do this, I first started by aggregating the true democratic vote share for each district and mapping them. The states with the most vote share (above 50%) are the bluest and the states with the least democratic vote share (under 50%) are the reddest, while the tossup states are the whitest. The map can be viewed below.
##
|
| | 0%
|
| | 1%
|
|= | 1%
|
|= | 2%
|
|== | 2%
|
|== | 3%
|
|== | 4%
|
|=== | 4%
|
|=== | 5%
|
|==== | 5%
|
|==== | 6%
|
|===== | 6%
|
|===== | 7%
|
|===== | 8%
|
|====== | 8%
|
|====== | 9%
|
|======= | 9%
|
|======= | 10%
|
|======= | 11%
|
|======== | 11%
|
|======== | 12%
|
|========= | 12%
|
|========= | 13%
|
|========= | 14%
|
|========== | 14%
|
|========== | 15%
|
|=========== | 15%
|
|=========== | 16%
|
|============ | 17%
|
|============ | 18%
|
|============= | 18%
|
|============= | 19%
|
|============== | 19%
|
|============== | 20%
|
|============== | 21%
|
|=============== | 21%
|
|=============== | 22%
|
|================ | 22%
|
|================ | 23%
|
|================ | 24%
|
|================= | 24%
|
|================= | 25%
|
|================== | 25%
|
|================== | 26%
|
|=================== | 26%
|
|=================== | 27%
|
|=================== | 28%
|
|==================== | 28%
|
|==================== | 29%
|
|===================== | 29%
|
|===================== | 30%
|
|===================== | 31%
|
|====================== | 31%
|
|====================== | 32%
|
|======================= | 32%
|
|======================= | 33%
|
|======================= | 34%
|
|======================== | 34%
|
|======================== | 35%
|
|========================= | 35%
|
|========================= | 36%
|
|========================== | 36%
|
|========================== | 37%
|
|========================== | 38%
|
|=========================== | 38%
|
|============================= | 41%
|
|============================= | 42%
|
|============================== | 42%
|
|============================== | 43%
|
|============================== | 44%
|
|=============================== | 44%
|
|=============================== | 45%
|
|================================ | 45%
|
|================================ | 46%
|
|================================= | 46%
|
|================================= | 47%
|
|================================= | 48%
|
|================================== | 48%
|
|================================== | 49%
|
|=================================== | 49%
|
|=================================== | 50%
|
|=================================== | 51%
|
|==================================== | 51%
|
|==================================== | 52%
|
|===================================== | 52%
|
|===================================== | 53%
|
|====================================== | 54%
|
|====================================== | 55%
|
|======================================= | 55%
|
|======================================= | 56%
|
|======================================== | 56%
|
|======================================== | 57%
|
|======================================== | 58%
|
|========================================= | 58%
|
|========================================= | 59%
|
|========================================== | 59%
|
|========================================== | 60%
|
|========================================== | 61%
|
|=========================================== | 61%
|
|=========================================== | 62%
|
|============================================ | 62%
|
|============================================ | 63%
|
|============================================ | 64%
|
|============================================= | 64%
|
|============================================= | 65%
|
|============================================== | 65%
|
|============================================== | 66%
|
|=============================================== | 66%
|
|=============================================== | 67%
|
|=============================================== | 68%
|
|================================================ | 68%
|
|================================================ | 69%
|
|================================================= | 69%
|
|================================================= | 70%
|
|================================================= | 71%
|
|================================================== | 71%
|
|================================================== | 72%
|
|=================================================== | 72%
|
|=================================================== | 73%
|
|=================================================== | 74%
|
|==================================================== | 74%
|
|==================================================== | 75%
|
|===================================================== | 75%
|
|===================================================== | 76%
|
|====================================================== | 76%
|
|====================================================== | 77%
|
|====================================================== | 78%
|
|======================================================= | 78%
|
|======================================================= | 79%
|
|======================================================== | 79%
|
|======================================================== | 80%
|
|======================================================== | 81%
|
|========================================================= | 81%
|
|========================================================= | 82%
|
|========================================================== | 82%
|
|========================================================== | 83%
|
|========================================================== | 84%
|
|=========================================================== | 84%
|
|=========================================================== | 85%
|
|============================================================ | 85%
|
|============================================================ | 86%
|
|============================================================= | 86%
|
|============================================================= | 87%
|
|============================================================= | 88%
|
|============================================================== | 88%
|
|============================================================== | 89%
|
|=============================================================== | 89%
|
|=============================================================== | 90%
|
|=============================================================== | 91%
|
|================================================================ | 91%
|
|================================================================ | 92%
|
|================================================================= | 92%
|
|================================================================= | 93%
|
|================================================================= | 94%
|
|================================================================== | 94%
|
|================================================================== | 95%
|
|=================================================================== | 95%
|
|=================================================================== | 96%
|
|==================================================================== | 96%
|
|==================================================================== | 97%
|
|==================================================================== | 98%
|
|===================================================================== | 98%
|
|===================================================================== | 99%
|
|======================================================================| 99%
|
|======================================================================| 100%
In order to compare efficacy of expert predictions, I then created a map of how likely experts predicted each district was to go to each party. To do so, I combined numerical ratings that expert predictors gave each district based on how likely they were to lean towards a certain partisanship. The scaled range from 1 which mean safely leaning democrat to 7 which meant it was safely lean republican.In the map below, blue is associated with lower numbers and democrats, white(4.0) with the most uncertain swing districts, and red with the highest numbers and republican victories. The image can be seen below.
Comparing the two images, it appears that experts were incredibly accurate at predicting which way districts were going to swing and event the degree to which how large the voting share would be. From the images, the degree of certainty to which experts predicted a state would lean one way or another directly correlated to how far the democratic vote share percentage veered from 50%. The only routine differences I see between the expert predictions and vote share percentage in actuality is that it appears in districts that vote share was more favorable to democrats in regions that experts thought would lean slightly more republican.
Now that we’ve established that expert predictions in aggregate are quite close to the actual vote share percentage exhibited in historical elections, it is time to see how strong of a predictor they really are. As well as, how might incumbency play a role in our predictions as well.
Before we get into the nitty gritty of the models, let’s recap that incumbency is one of the most fundamental and talked about variables that go into election predictions. But for those who don’t avidly read Nate Silver’s every tweet, you may be wondering why? So let’s first break this down.
Incumbency is typically seen as an advantage but the label of being an incumbent itself isn’t necessarily the key determinant of winning on its own, rather it is the meaning behind what having held office before. Being an incumbent can lead to an advantage due to name recognition alone, but also if a candidate is viewed to have done a good job whether it be ensuring economic success or passing favorable legislation for their district. However, poor performance or coincidental measures of performance like the economy may have profoundly negative effects. As such, incumbency itself may not represent the whole picture of an electoral outcome.
Building off of last week’s models, I recognized that the economic variables of performance used were not good at explaining the variation in two party vote share in an election. As such, I decided to drop this variable from my model altogether. In addition, since I knew I was bringing in expert predictions, and understood that these predictions incorporate polling heavily, I decided to throw out the general poll variable as well. As such, this week I decided to test two models: one with just average expert ratings and another with average expert ratings and incumbency. To start, (due to technical difficulties) I created one model that looks only at democrats. I’m trying to look at how democratic vote share is impacted by whether a democrat is an incumbent or not.
Model 1: \[ Democratic Vote Share = Average Expert Prediction \] Model 2: $$ Democratic Vote Share = Average Expert Prediction + IncumbentPresident
$$
The first model predicts democratic vote share based upon average expert ratings only.After training it on historical data for each district, the model predicted that among all districts nationally that democrats in 2022 would win approximately 50.0125 percent of the two party vote. On average the prediction for all districts had an average r^2 value of 0.7815. This says that combining expert predictions for an average yielded a promising way to explain the variation in election data. I believe this is likely because expert prediction is incorporating all variables that pundits have viewed to be effective in the past. Moving on to my second model, where I included both average expert prediction and a variable of demcorat incumbency, the model predicted that the Democrats would win approximately 48.28% of the two party vote, with an average r^2 value for all districts of 0.878481. This is the highest r^2 value I’ve seen in my election modeling endeavors thus far. This indicates that combining expert prediction with an incumbency variable is promising way of explaining variation in democratic vote share percentage in an election, since on average the model that includes incumbency has a higher predictive power than the one without it. As such, the model with expert prediction and the incumbency variable was the strongest. With presidential approval being such a significant portion of election theory and literature, I’m not surprised our incumbency variable improved our predictions even beyond the average of expert predictions variable.
All in all, experts appear to be doing their job quite well as aggregating expert prediction on the theory of the wisdom of crowds maintains strong predictive power in the modeling of election outcomes. Taking an average rating of all expert predictions captured a good picture of future election outcomes. My only fear with this is that since oftentimes expert prediction algorithms are like a black box with unknown predictors, adding many variables on top of this covariate would lead to overfitting the model or exhibit possible collinearity among predictors. Similarly, incorporating a variable for incumbency seems to also be a good explainer of election outcomes. I will continue to include this variable in various iterations next week to test the best mode of measuring incumbency and how to use it in my predictions both nationally and at the district level moving forward.
CITATIONS: Campbell, J. E., & Mann, T. E. (2016, July 28). Forecasting the presidential election: What can we learn from the models? Brookings. Retrieved October 3, 2022, from https://www.brookings.edu/articles/forecasting-the-presidential-election-what-can-we-learn-from-the-models/