That poll which shows Colin Craig ahead in Rodney, revealed

Colin Craig’s Conservative Party has been claiming they will win Rodney, on the basis of this poll report, which is on their website. The key extract is:

47.2% of those who had decided who they were likely to vote for as an electorate candidate would vote for Craig.

This would position Craig in first place in the electorate, ahead of ‘The National Party Candidate/Mark Mitchell/Lockwood Smith (36.3%).

The polling was done by Research First. Whale recently revealed their director is a candidate for the Conservative Party. I commented at the time:

This does not mean that the poll results are or are not valid. As I said, validity is based on knowing the question asked and the methodology. You can poll for an organisation you are involved with. For example, Curia does an annual poll for the Republican Movement on whether people want NZ to become a republic when the Queen dies. Now I am on the Council of the Movement, but this doesn’t influence the results. The key thing is I have publicly disclosed my involvement.

Now in response to a request from Whale, Research First have released details of the questions they asked. It is good they have done so, because as I said the exact questions asked are often vital to interpreting a poll’s results. Their response says:

 Relevant questions included the following, in order of being asked:

1 For your party vote, have you decided who you will be voting for in the election?

2 Which party do you currently intend to vote for?

3 Have you heard of the Conservative Party?

4 Have you heard of Colin Craig?

Participants were read a brief preamble to provide context: ‘Colin Craig is the leader of the Conservative Party of New Zealand. In the Auckland mayoral election, Colin came third with over 40,000 votes’. Then asked…

5 If Colin were to stand in as a candidate in your electorate, what would be the likelihood that you would vote for Colin to be your member of parliament (on a scale of 1 = definitely; 2 = likely; 3 = neutral; 4 = unlikely and 5 = very unlikely)?

Those who identified they were neutral or unlikely to vote for Craig were asked:

6 Who do you intend to vote for?

Okay, let’s take this step by step. The first two questions are pretty standard. Then there are two specific question asking awareness of the Conservative Party and Colin Craig. Then a statement was read out which puts Craig in a positive light (mentioning his votes in the Auckland Mayoral election), and then they ask people how likely it is they will vote for Craig, and only if they say they are neutral or unlikely to they even ask you who else you will vote for.

The results are no surprise, once you realise this is the questions that were asked, and in what order. You have a number of factors here influencing the responses, namely:

• The mention of the Conservative Party and Colin Craig first
• The description of Colin Craig provided to respondents
• The question only asked about voting for Colin Craig, with no mention of anyone else
• Only if you say you are neutral or unlikely to vote Craig, do you even get asked whom else you might vote for
• The other candidates are unprompted, so you are comparing unprompted results vs a prompted result.

I am surprised that Research First did not insist on these questions being included in their report, as in my opinion they are quite vital to it. I also think it is unwise to compare answers to a prompted question to answers to an unprompted question.

If I was wanting to poll that seat, and get a result which was fairly trying to ascertain support, the questions I would use are either:

Which candidate, or party’s candidate are you likely to vote for with your electorate vote?

or

The candidates for Rodney are Colin Craig, Conservative; Beth Houlbrooke, ACT; tracey Martin, New Zealand First; Mark Mitchell, National; Terea Moore, Green and Christine Rose, Labour. Which candidate are you likely to vote for with your electorate vote?

One might also have a follow up lean question for those undecided. I very strongly suspect that the results to the questions above would be vastly different to the results of the poll commissioned by the Conservative Party.

There is never any 100% correct version of a question, and rarely 100% incorrect version. In terms of ascertaining potential support for Colin Craig, those questions may be legitimate if commissioned for internal use only. But what I think was wrong was to have them publicly reported as Craig being “in first place”. The questions should have been reported.

The lesson for media here, is to always ask for the questions. Those media who reported the poll, should be wary of doing so in future without checking.