September 4, 2012 11:00am by David Farrar

The Helensville poll

The Conservatives hired Research First to poll Helensville voters on the issue of gay marriage and gay adoption.

About 89 per cent said their MP should consult their electorate over conscience votes and vote the way voters wanted them to, it found.

While the majority did sat y they want their MP to vote as they say (no surprise), there are a few things to note about the poll. If you look at the small print on the pamphlet, it is actually a poll of both Helensville and Rodney electorates.

The full poll results are online on the Conservative Party website. It was done by Research First.

I blogged last year my criticisms of the poll done by Research First for the Conservatives in Rodney. It purported to show Craig ahead of the National candidate and had serious flaws such as mixing results from an unprompted question with a prompted question. It is a matter of record that Craig lost the seat by over 12,000 votes.

I’m pleased to say that this poll is, in my opinion, much sounder than last year’s Rodney one. In fact in some areas it is more transparent than almost any other poll report. There are also some areas you can be critical, and I’ll get into those also, but first I’ll highlight the really good stuff that Research First have done with this poll.

The full report has been made available online.
They disclose their call-back policy – a laudable six times.
They disclose their response rate, and detail the reasons for non-response. This is an A+ level of transparency. I don’t think I have seen another NZ poll report that does this.
They disclose their demographic breakdown, and what variables they weighted on.
They include the full questionnaire as an appendix to the report.
The question order appears to be sound, with the question on marriage vs civil unions at the beginning of the poll

So full credit to Research First for their report. In some ways, it is a model of transparency.

It is only by being so transparent, can one have full scrutiny of results of a poll. Now with that in mind, I am going to comment on a few areas, where I would have done things differently. But bear in mind that almost all polling reports will have areas where you can argue about question wording and the like. This would not be unique to Research First. I do think the Rodney poll last year was a long way short of best practice, but this poll is a huge improvement, and in some areas excellent.

But I would make the following points, in interpreting the poll.

The poll is of both Helensville and Rodney residents, but does not disclose the actual number of respondents in each electorate. The 355 responses have a margin of error of 5.2% but if half came from each electorate the margin of error for each is 7.5%. As it was used for a pamphlet targeting Helensville, it would be nice to have the numbers for that electorate only, and even the results for each of the two electorates if they are statistically significant.
The question asking if they agree or disagree “Environment, upbringing, life experiences and personal relationships all influence personal choice when it comes to sex, sexual orientation, and gender identity” is not a particularly helpful one as it mixes in sex and sexual orientation. Of course upbringing, life experiences and relationships influence your personal choices when it comes to sex. But for most people it does not influence their sexual orientation. It is best to avoid questions that list several things, with the possibility they agree with some and disagree with some. I don’t think the results to that question can be interpreted to mean anything. I personally could not answer agree or disagree to that question as it is both.
The question “It is ideal that adopted children be raised by a mum and a dad (ie a heterosexual couple) rather than two mums or two dads (ie a homosexual couple)” is a perfectly valid question. However I would note that this does not mean someone agreeing is against same sex couples being able to adopt. I would agree with that statement as an ideal, yet I also think same sex couples should be able to adopt if it is in the best interests of the child. So nothing wrong with that question, but be careful that it is not interpreted beyond what it said.
The statement “Are you aware that the proposed change will enable gay couples to adopt children as a couple under the Adoption Act 1955?” makes an assertion that is not necessarily the case. It is unclear whether the changes to the Marriage Act will enable a married same sex couple to adopt. One part of the Adoption Act refers to merely spouses, while another refers to “husband and wife”. There is a diversity of opinion on what the impact will be, and I suspect a Judge would eventually decide. So it would have been better to say “may” rather than “will”.
The Conservative Party pamphlet claims 89% agree that your local MP should consult their electorate and vote the way you want them to. This is wrong. The findings are 69% say they should consult and vote as the electorate says. 20% say they should consult and make up their own mind. Note that this error is not anything to do with Research First, but instead the Conservative Party.
I’d also make the general point that while people were asked whether they agreed or disagreed with a number of statements, respondents were never asked a direct question on the Wall bill such as “Do you think your MP should vote for or against Louisa Wall’s bill which would allow same sex couples to marry”. This is not a criticism. There is no obligation to ask questions that you may not like the results of. I’m just pointing out that agreement with an assertion over civil unions and marriage is not the same thing as a more specific question on the bill.
This is debatable, but you could make a case for using the term “same sex marriage bill” rather than the “gay marriage” bill. Likewise the statement “If the definition of marriage is changed, as proposed by the bill, it will enable gays to adopt children as a couple under the Adoption Act 1955” is less than ideal, talking about “gays” rather than “same sex couples” or even “gay and lesbian couples”.

None of the above are show-stoppers. As I said, I’m pleased with the level of transparency, which allows a critique to be done. Alison McCulloch did a critique of polls done by Curia earlier this year.

UPDATE: Forgot to mention one other issue which occurred to me. Respondents were told up-front that the poll was on gay marriage, and this led to around 100 people saying they didn’t want to take part. This may have impacted the results a bit also, as it is more likely those with strong views on the issue will want to take part. Ideally, in my opinion, you want to tell respondents generally what the poll is about (such as saying “on a current political issue) but need to be careful not to be too specific as it may then become a bit self-selecting.