Support The Moscow Times!

Why Russian Pollsters So Poorly Predict

Those who didn't follow the U.S. presidential election closely might be surprised to learn exactly who won one of the most exciting and memorable victories of all time, defeating a powerful, experienced opponent who is accustomed to holding the advantage.

But this election was actually won by analysts who use modern methods of data analysis to predict the results of elections. It was also won by political consultants who use those findings to fashion their strategies. The losers in this election were other analysts who spoke on television and wrote newspaper columns and who recklessly based their predictions on their unreliable "sixth senses."

Predicting an election outcome is more than just correctly guessing the winner or even guessing the correct point spread between the various candidates. For a prediction method to be successful, its methodology must be understandable, it must be open to independent verification, and it should systematically produce accurate predictions. For several months prior to the U.S. election, a handful of experts analyzed voter data almost as soon as it came in, making a special effort to avoid the errors typically made when people unfamiliar with mathematical statistics interpret the results.

It is interesting that Republican candidate Mitt Romney's team of analysts fared far worse at this task than those working for the general public. The analysts employed by U.S. President Barack Obama not only disclosed more information about their strategies over the course of the campaign and more actively shared the details of their predictions, but they also were ultimately more accurate than Romney's as well.

This should be a lesson for Russian politicians, too. They pay a lot of attention to closed surveys and studies, including those conducted by security services and private firms. But the quality of closed research is significantly lower than that of open surveys, even when both types are conducted by the same firm.

In general, Russian polling firms produce research of very low quality. In December, both Kremlin-linked and independent pollsters were correct in the sense that they predicted the official tally in advance. Embarrassingly, hard statistical analysis in two papers already accepted for publication by the Proceedings of the National Academy of Sciences demonstrated that the official tally was way off the real returns.

It is difficult to imagine just how thoroughly U.S. specialists will analyze data for economic or other purposes. Similarly, highly competitive election campaigns, which need every possible vote, use the most modern models and techniques available. There are many independent firms that work to achieve high-quality predictions in political contests to later win lucrative commercial orders from business clients.

This isn't done in Russia, but not because domestic businesspeople are ignorant. In fact, experience shows that Russians are quick to take techniques and methodologies developed abroad and adapt them for domestic use whenever the need arises. The reason is simply that Russian businesses have higher profit margins because of low competition and rarely need to rely on ultra-precise data analysis. But that is only for the time being. When competition increases in Russia, domestic businesses will be thankful that modern methods of statistical analysis were perfected during the 2012 U.S. presidential election.

Konstantin Sonin is a professor at the New Economic School in Moscow and a columnist for Vedomosti.

Related articles:

The views expressed in opinion pieces do not necessarily reflect the position of The Moscow Times.

… we have a small favor to ask.

As you may have heard, The Moscow Times, an independent news source for over 30 years, has been unjustly branded as a "foreign agent" by the Russian government. This blatant attempt to silence our voice is a direct assault on the integrity of journalism and the values we hold dear.

We, the journalists of The Moscow Times, refuse to be silenced. Our commitment to providing accurate and unbiased reporting on Russia remains unshaken. But we need your help to continue our critical mission.

Your support, no matter how small, makes a world of difference. If you can, please support us monthly starting from just 2. It's quick to set up, and you can be confident that you're making a significant impact every month by supporting open, independent journalism. Thank you.

Continue

Read more