Grumpollie on polling cellphones

Grumpollie (a senior experienced NZ pollster) blogs on the issue of calling cellphones. He quotes some data to note:

In fact, you can see in the next chart that nearly half (47%) of non-landline households receive more then $40,000 per year, and a quarter (25%) receive more than $70,000 per year. So sure, absolutely, there is a skew toward lower income (and likely younger) households, but don’t assume that cell-only households all contain lower income people and young people.

Andrew breaks down the spread by income being:

• 38% of households are under $40,000 income of which 8% do not have landlines
• 23% of households are under $40,000 income of which 3% do not have landlines
• 39% of households are under $40,000 income of which 4% do not have landlines

He then points out:

Most polling companies will weight their data to (EDIT: try to) correct for things like non-coverage of age and socioeconomic groups. The problem with weighting is it assumes the people you have surveyed (and are weighting) are similar to the people who you haven’t surveyed.

But here’s what you need to think about if you’re wondering how big the cellphone non-coverage issue really is:

• Do voters in the under $40k income band who are not covered differ in party support from voters in the under $40k income band that are covered?
• Do voters in the $40-70k income band who are not covered differ in party supportfrom voters in the $40-70k income band that are covered?
• Do voters in the $70k+ income band who are not covered differ in party support from voters in the $70k+ income band that are covered?

When I say differ, I’m not talking differ ‘just a little’. When you factor in the size of each non-covered income group (8%, 3%, and 4%), they would have to differ massively from each covered income group, when it comes to party support, for them to make much difference at all to a total poll result.

I think it is inevitable that polling in NZ will start to include mobile phones and/or Internet panels in their sampling. But as Andrew points out,  the impact at this stage of non-inclusion is likely to be minor. One company (Roy Morgan) does include cell phones, so one interesting thing will be to see how they go in terms of accuracy at the election. In the US the company with the highest proportion of cellphones called (Gallup) had basically the worst result.