Cell-phone polling

Dim-Post highlights a comment left on his blog an an anonymous pollster:

The problem with calling cell phones doesn’t really lie in the cost of calls. For a polling company, calling a cell phone doesn’t cost that much more than calling a landline. The problem is the complexity and cost of employing dual sampling frames when the proportion of cell phone users without a landline is still very low. If the purpose of calling cell phones is to reduce non-coverage of likely voters, then you may actually need to ‘screen out’ those you call on cell phones who also have a landline (because they are already covered by the landline sample frame).

If we assume 6% of eligible voters have cell phones and no landline, that means that 94% of the people you call on a cell phone will not be eligible to take part (again, because they are already covered by the landline sample frame). This is where the cost would really begin to build up – all those interviewer hours required just to screen people out (eek!).

This is not the only way to reduce non-coverage – but it’s actually one of the more straight forward and ‘statistically pure’ ways (ie, you can develop some sort of weighting scheme, but the more you weight, the greater the design effect (which increases the margin or error, and decreased the accuracy of a poll).

To make things more complex:

– Some people have more than one cellphone, meaning that the probability of them being called is higher, so additional weighting would need to be applied to adjust for the probability of selection (you may notice that some polls weight by household size and the number of landlines connected to a house – this is adjusting for the probability of section)

– There are a lot of cell phone numbers that are out of use, but when they are called they still go through to a voice mail. Unlike landlines (which you can ‘ping’ to test the connection), it is very difficult (ie, near impossible) to determine if there is actually an eligible person at the end of a number, so you’ve got no measure of the success rate of your sampling approach (ie, refusal rates, response rates, qualifier rates etc).

– At the moment such a small proportion of New Zealanders have a cell phone with no landline that party support would need to be DRAMATICALLY different among those people for this particular type of non-coverage to influence the poll results for party vote (eg, support for Labour among cell phone only voters may need to be TWICE what it is among landline voters for the party vote result to shift by more than, say, the margin of error).

When the proportion of people with cell phones and no landline is considerably larger than it is today (like it is in some other countries), then it will definitely make sense to employ a dual sampling frame approach. In NZ though (at least in 2011) most pollsters got things pretty close to the election day result so this would suggest non-coverage of cell phone only voters isn’t a big issue just yet. If cell phone plans get cheaper, then polling approaches will probably need to change to keep up.

All comments I agree with. As time goes on it will become more of an issue in NZ, but at this stage the impact is limited. Also polling companies compensate for those without landlines by having quotas or weighting by age and income.