COUPCAST: ACCURACY

Return to Main CoupCast page

We can assess the accuracy of the forecast is by seeing how well the model would have predicted past coup attempts. To do this, we rank each of the roughly 130,000 monthly predictions of coup risk (January 1975 to present) into one of fifty intervals, with the lowest interval containing months below the 2nd percentile of coup risk, the second-lowest interval including months between the 2nd and 4th percentiles, and so on. If the model is accurate, more coups should occur in the intervals containing the months with the highest forecasted coup risk.

The world has witnessed 240 coup attempts since January 1975, so if we were randomly guessing then we would expect 2% of these coups (roughly 5 coups) in each of these 2-percentile intervals. We illustrate this baseline as a dashed line in the graph above. The gold columns in the graph show that CoupCast provides a much more accurate estimate, with substantially more coups at higher levels of predicted coup risk. In fact, half of all coup attempts occurred where the model predicted coup risk to be above the 90th percentile. Conversely, less roughly 5% of coup attempts occurred where CoupCast forecasted coup risk to be lower than average.

The monthly rankings are also highly predictive, though less so during periods when many more countries suffered coups regularly. Leaders ranked in the top-4 for predicted coup risk during a given month experienced nearly a fifth of all coup attempts. The majority of all coup attempts occurred against leaders ranked among the top-25 in predicted coup risk.

For more information about how these predictions are generated, please visit the methodology page.