Ok, the results are in. I finally gave up on Missouri, and called it for McCain, since the returns are leaning in that direction. As an interesting little side project, I analyzed the prediction accuracy of www.fivethirtyeight.com vs www.intrade.com by computing the probability of the observed outcome based on the predictions by the two websites. Intrade predicted a 0.19% probability of this election outcome, while 538 predicted a 2.4% probability of this outcome. Note that by "this outcome" I am referring to the state-by-state results. There are a very large number of possible outcomes, and some of them have infinitesimally small probabilities. For instance, all states voting for either candidate would have an extremely low probability. The small probability from Intrade is due to the cascaded probabilities of many estimates which were significantly smaller than 100% (lots of ~90% predictions). Things would look very different if a single one of 538's 100% predictions had gone the other way. The cascaded probabilities would then be 0%, while Intrade's would still be much greater than 0% (Intrade predicted no 0% or 100% outcomes, while 538 predicted many).
Another way to look at the prediction accuracy is simply as a function of the number of correct state outcome predictions. Intrade only missed one (Indiana), while 538 missed two (Indiana and Missouri).
I have not analyzed this from an electoral-vote-weighted perspective. I took the perspective that each state behaves like a weighted coin toss, and each website has its own method for estimating the odds of the result. I simply wanted to understand the accuracy of that prediction, not the accuracy of the composite prediction (the overall winner). The sites themselves gave their predictions for the winner: 538 predicted a 98% probability of an Obama win, while Intrade predicted a 90% probability. If I were to construct a Monte Carlo simulation based on the state-by-state predictions, I'd have very similar results. However, this wouldn't be sufficient to assess the accuracy of the predictions. We'd need many more trials.
In conclusion, although Intrade more accurately predicted the results, it appears that 538 is a more accurate estimator of outcome probabilities.
1 comment:
It leads to the interesting question of if these two sites were so accurate, why did we need the 100's of other polls being conducted. It seems like we must be nearing the point of 'peak polling'.
Post a Comment