Nov 27, 2011

Bad polling

I posted this in the comments section of another post, however I think it deserves a wider audience so I've copied and pasted here. These are the poll results from the election period Te Karere Digipolls compared with the actual results. Like I've said earlier, it shows that the polls were well out. I've rounded the numbers and have only included the main candidates.


Te Tai Tokerau:

Digipoll – Hone 42
               Kelvin 35
               Waihoroi 20

Actual – Hone 43
              Kelvin 38
              Waihoroi 16


Fairly close resemblance between the poll and the actual results in this electorate. Certainly within the margin of error of 4.9% so we’ll give them this one. However...


Tamaki Makaurau: 

Digipoll – Pita 58
               Shane 23
               Kereama 14

Actual -  Pita 40
              Shane 36
              Kereama 16


This poll grossly overstated Pita Sharples’ support. Shit, it was well out. This poll did seem to confirm what many Maori political commentators were thinking though. I haven’t seen anyone come up with a satisfactory answer as to why Shane did so well. Maybe low turnout hurt Pita. With 16% of the vote Kereama Pene certainly did I guess.


Hauraki-Waikato:

No poll from Te Karere on this electorate but the Marae Digipoll from September put it like this.


Digipoll – Nanaia 59
               Angeline 18
               Tau Bruce 13

Actual – Nanaia 59
              Angeline 22
              Tau Bruce 17


A decent result from the Marae Digipoll. They correctly stated Nanaia’s support and were thereabouts with Angeline and Bruce Mataki of the Maori Party. Remember this poll did not factor in undecideds.


Waiariki:

Digipoll – Te Ururoa 56
                Annette 22
                Louis 22

Actual –  Te Ururoa 41
               Annette 30
               Louis 23


Again, this poll was well out. Well beyond the margin of error. The poll overstated Te Ururoa’s support by a country mile. This race was tight – not the cakewalk the poll was predicting. 

Correction: I had the wrong figures before - all corrected now. 

Ikaroa Rawhiti:

There was no Te Karere poll here too, but we’ll have a look at the Marae Digipoll results.


Digipoll – Parekura 40
               Na 49
               Tawhai 1

Actual – Parekura 62
              Na 24
              Tawhai 14


Wow. I don’t think any poll in the history of mankind has ever been this out of whack. However, it should be remembered that this poll was conducted in September and they phrased the questions oddly. Something like “which party will you give your candidate vote to” instead of saying “who will you give your candidate vote to” or something like that.


Te Tai Hauauru: 
No Te Karere poll here either. However, Marae Digipoll from September comes in handy.


Digipoll – Tariana 49
                Soraya 40
                Fred Timitimu 1
                Jack (didn’t register I don’t think).

Actual –  Tariana 48
               Soraya 30
               Fred 8
               Jack 11


Tariana’s support was about right, but the poll exaggerated Soraya’s support and barely even factored Fred Timutimu. Jack McDonald didn’t even feature in the Marae poll but polled 11% and he wasn’t even going for the electorate vote.


Te Tai Tonga:

Digipoll – Rahui 46
                Rino 35
                Dora 10
                Clinton 9

Actual –  Rahui 31
               Rino 42
               Dora 15
               Clinton 8


Again, way off. Well outside the margin of error. The poll got Clinton about right, but failed miserably in predicting the correct outcome and share of the vote.

The polls in the Maori electorates have failed. Theses poll failures follow the Native Affairs Baseline poll which failed to accurately predict the share of the vote each candidate would receive in the Tai Tokerau byelection. I don’t think I can ever take a poll on the Maori electorates seriously again. To be fair, the polls were conducted late October to mid November and things change at the last minute. Polls are, of course, snapshots of a moment in time. The moment these polls were picturing was spread across a significant period of time.

Lastly, I think the other criticisms I have made of polls in the Maori electorates still stand.

6 comments:

  1. Thanks for the analysis. And I concur with your criticism.

    ReplyDelete
  2. Sorry, but this post is silly, as you admit yourself at the end. Polling is not perfect, but there were several factors that you mostly ignore:

    MARGIN OF ERROR
    - every poll has a margin of error. The poll is really saying we are - for example - 95% confident that the true result is that Annette has 40% plus or minus say 4.9% margin of error. Which means she could have anywhere between 35 and 45% support. This doesn't fully explain the results differing from the polls as she jumped up from 22%, but there are other factors that influence that below.

    TIME PERIOD
    - the polls you cite were taken over a large time period, which affects the results. As you note, some polls were taken in September, 2 months before the actual vote! Voters can and do change their mind between the last polls and the election, especially if those polls were months before. Think of what happens if a candidate has a brainfart moment on TV - they can lose a massive amount of support overnight. That doesn't mean the old poll showing their higher level of support was wrong, just that their *true* support level has shifted.

    - Ironically, you noted this in earlier posts, where you excused Annette's low polling in Waiariki as she had just declared. That is what clearly happened in Te Tai Hauauru - the poll did not overstate Soraya's support so much as the poll was so early it didn't properly register Fred and Jack's support.

    SWINGS
    - when one candidate gains or loses support, those votes come from other candidates, so the effect is magnified. For example, in Te Tai Tonga, Rahui's drop of 15% from poll to election is almost exactly matched by the combined 7% and 5% gains for Rino and Dora. Add in the margin of error, and it's not so dramatic a swing. Rahui *could* have been on 41% and Rino on 40% in the poll and that would have been within the typically large margin of error (often around 5%) in the Maori electorates.

    WORDING
    - The Ikaroa poll should have had wording like "which party will you give your party vote to?" Your wording was just as loaded as the original question. ;)

    Anyhow, I hope you can see the science in stats does not guarantee a result in the election. Frustrating as it is, voters can and do change their minds. Especially over the long time periods you are talking about - the election campaign hadn't even started in September, and Annette only officially stood 4 weeks before the election!

    Only reason I bang on about this is 'cos some on the left are starting to sound like Flat Earthers (ie Martyn Bradbury) talking about the landline and undecided non-issues.

    Cheers.

    ReplyDelete
  3. Hi anon, thanks for that.

    I did, I think, mention that the margin of error was 4.9%, but in many cases the results fell well outside the margin of error.

    I also mentioned that the polling period would have an effect. On that point, I'm a little annoyed that the Maori media were holding these polls up as an accurate reflection when they knew, given the long polling period, that the poll probably would not have reflected opinion at the time of release.

    No doubt about swings - but so much? I can't see any reason why Pita would be subject to an 18 point swing when he ran the best campaign, attracted the most media (by far) etc etc.

    Thanks for correcting the wording. Don't know what I was thinking there ha.

    Thanks for your points

    ReplyDelete
  4. Waiariki was 41 / 30 / 23

    ReplyDelete
  5. Cheers, I had the results from an earlier point in the night.

    ReplyDelete
  6. Yeah too right, the polls are just shit especially the maori ones. The polls are only manufactured to give the media crap to talk about. They are not to be taken seriously, although many do.

    ReplyDelete

Rules:

1. Anonymous comments will be rejected. Please use your real name or a pseudonym/moniker/etc...
2. No personal abuse. Defamatory comments will be rejected.
3. I'll reject any comment that isn't in good taste.