Myths and the reality of election forecasts

By, Atanu Biswas is a Professor of Statistics at the Indian Statistical Institute, Kolkata

Global examples of mismatch of results from the exit polls

• Take the example of the Venezuelan recall referendum of 2004 to determine whether Hugo Chávez, then President, should be recalled from office. A huge discrepancy with the exit poll created a massive uproar worldwide.
• The “Orange Revolution” in Ukraine during late November 2004 to January 2005 was also in the immediate aftermath of the presidential election.
  • The allegations of electoral fraud were strengthened by several exit polls exhibiting a substantial lead for Viktor Yushchenko.
• Similarly, widespread protests over the disputed parliamentary elections of 2003 triggered the “Rose Revolution” in Georgia, and culminated in the ousting of President Eduard Shevardnadze.
• One wonders if polls predictions are still so trustworthy. Certainly not.

The Bihar mandate

• Pollsters performed miserably in Bihar in 2015 in one of their worst performances in Indian elections.
• Although the opinion polls mostly predicted a victory by the National Democratic Alliance this time, very few of them could guess the quantum.
• The exit polls, on the other hand, mostly predicted a Rashtriya Janata Dal+ victory.
• A funny thing is that the minimum and maximum predictions for the RJD+ were 76 seats (a range of 71-81) and 180 seats (a range of 169-191), out of 243 seats.
• One wonders whether the RJD+ securing anything less than 71 or more than 191 was at all practicable. Thus, there will be some pollster or other to claim a perfect or near-perfect prediction.

Surveys and lessons learnt

• Yes, poll predictions have failed miserably on many historical occasions including some which were in the developing process.
• For example, before the 1936 U.S. presidential elections, a reputed magazine, the Literary Digest, conducted an opinion poll survey with a massive 24 lakh samples, and predicted 57% vote share for Republican Alfred Landon, and 43% for President Franklin Roosevelt. In reality, Roosevelt got 62% votes against 38% favouring Landon.
• This episode was a lesson about the importance of ‘selection bias, as the individuals under study were affluent people having a telephone, club memberships and magazine subscriptions in that era; Roosevelt had less support among such people.
• Also, people understood the importance of ‘non-response bias’ as only 24% of the people who were approached actually responded to the survey.
• Similarly, the severe failure of Gallup’s poll prediction of the 1948 U.S. election taught pollsters about the importance of ‘random’ sampling.
• In 2016, Mr. Trump supporters were under-counted. Later, there was an attempt to label that “shy Trump factor” or “hidden Trump vote”.
• Certainly, the winner of opinion/exit polls is not necessarily the winner in the election.
• Hillary Clinton, Ed Miliband or Atal Bihari Vajpayee could vouch for that.
• A small State like Delhi has caused misery for pollsters, be it in 2015 or 2020.
• In the U.K., for example, no major survey could predict the victory of Conservatives in 2015.