Say you want to reach a representative sample of the U.S. electorate for a presidential poll. The Obama-McCain race is relatively close these days, with the Democrat's lead hovering around 5 to 6 points in most surveys. Someone tells you that he's selected a sample that's predominantly under 40 years of age (oops, that one favors Obama); disproportionately renters rather than homeowners (Obama-leaning again); full of college students (sounds like a Starbucks Obama thing to me) -- and, for good measure, includes a higher proportion of blacks and Hispanics than the national population does.
At this point you throw up your hands and exclaim: "Why are we concentrating on such a pro-Obama universe? He could be leading by 20 points or more among those people!"
He could. He probably is. But in actuality, the sample I've described is either not being included at all in many national polls or is being undercounted. Why? Because I'm talking about the growing number of American cellphone users who have no other type of phone or who choose to go wireless for the vast majority of their interactive needs. And this election cycle -- for the first, and perhaps only, time -- this group has the chance to render presidential polls "wrong from the start": potentially disguising at least 2 to 3 percentage points of Obama support and maybe more.
Heretofore my industry has dismissed the cellphone-only population with a troika of "yes, buts." Yes, they're undercounted, but 1) they don't vote anyway; 2) their numbers are still small; and 3) we can find acceptable substitutes in the land-line population.
And to be honest, there is a fourth, still more powerful rationale that remains unstated: "Yes, they're undercounted, but it's too damn difficult and expensive to reach them."
Each of those "yes, buts" except the last one is being overwhelmed by the facts on the ground. The most recent estimate places the number of "wireless" adult Americans at around 30 percent -- with slightly more than half of those only using cellphones and the rest possessing both land lines and cellphones but using the latter far more often. Looking at the data over time, it is clear those numbers are getting higher each month. The 2004 presidential election exit poll conducted by the National Election Pool found that 7.1 percent of all voters in that election were cellphone-only. Recent data indicates that the percentage could be twice as high in 2008.
As this trend continues, the differences between cellphone and land-line respondents will dissipate simply because of heft -- the larger the cellphone universe, the more it will contribute to national opinion, and the more it will resemble the general population. Cellphone-only and land-line users will come to resemble each other. But for now, the growing use of cellphones presents pollsters with a tremendous challenge.
The 2008 election has featured a sizable increase in under-40 voting, no doubt inspired partly by Barack Obama's message, but also by new and more refined methods of communicating that message -- and by a campaign that has the resources to use those methods. As a result, it is obvious that the cellphone-only universe could be a statistical outlier: one of the most, if not the most, dependably pro-Obama constituency other than African-Americans.
By law, cellphone users cannot be called by an automatic dialing system (to prevent obnoxious telemarketing), and cellphone numbers are not part of the normal random-digit-dialing residential-exchange universe. Survey companies prefer to conduct polls using automatic dialing, but to find cellphone-only voters, they must employ the less-efficient hand-dialing method. Cellphone users must be sampled separately and at greater cost in time and money. This means that polls utilizing the cheaper and more efficient means of making survey calls do not include cellphone interviews.
And as survey respondents, these voters are less cooperative anyway. Even if they are contacted, they are less likely to take a call, or to arrange a call-back, than land-line households -- further increasing the cost of reaching them.
Many survey companies have looked at these impediments and decided that it is simply not worth the extra effort and cost to track down cellphone users. (I am not going to name names, but one should assume that polls conducted by robo-calls undercount cellphone-only voters.) The pollsters' rationalizations hinge on the theory that a sampling of similarly profiled land-line voters -- younger, better-educated, more transient -- will yield similar results. But polling has certainly missed trends, and segments of the population, before. The Literary Digest never stopped to think in 1936 that its readership might not reflect the views of the entire country when its poll predicted a victory for Republican challenger Alf Landon over incumbent Franklin Roosevelt.
This year, the increasingly inexcusable failure to count a growing pool of voters could prove mathematically embarrassing. Let's say that with the campaigns' increased focus on the Web, Facebook, phone-texting and other targeted ways to communicate to younger Americans, voter turnout rises and this cellphone-only universe climbs from under 10 percent of the electorate to something closer to 20 percent. If these voters' preference is 60-40 for Obama, they alone will increase his national total by 2 percentage points. And those could easily be conservative projections. In fact, Gallup Poll results from earlier this year (prior to Obama's designation as the presumptive Democratic nominee) had a 4-point swing in favor of Obama once cellphone-only respondents were folded into the overall sample.
After 2000 can any public or private polling organization willingly use a sampling methodology that understates a candidate's support by 4 points, or more than 3 million voters?
The old ways of compensating for the cellphone-only population are clearly inadequate. Land-line results are no longer a reasonable substitute. Those land-line users -- even in the same younger demographic groups -- have different attitudes from those of their wireless counterparts. They are much more likely to own a home, to oppose gay marriage, to have health insurance, to be married with children. Surely pollsters cannot claim that the two groups reflect the same opinions in the midst of a national election campaign.
While cellphone interviewing is complicated for any purpose, in the presidential election there is one additional elephant in the room. The lack of relationship between a cellphone's area code and the physical location of the phone complicates things greatly. Consider the number of college students at the large state universities in battleground states who have out-of-state cellphone numbers, but are going to vote in Madison, Wis.; Columbus, Ohio; Ann Arbor, Mich.; or Gainesville, Fla. Consider also the numbers of students in big cities with several large universities, like Philadelphia. Currently, Gallup's screening methodology does not account for this discrepancy. In a national survey, it makes less difference, but in individual states, this issue will require additional consideration.
Consciously limiting the respondent universe in a presidential election survey, resulting in a wildly incorrect prediction, did tremendous damage to the reputation of the Literary Digest 72 years ago. In 2008, we have the audacity to hope that the polling industry will not make the same mistake. In 1948, the failure of the whole polling industry to predict Harry Truman's victory over Thomas Dewey was due to many factors, the most important of which was missing the size of the late surge toward Truman -- but there was also a little undersampling that seems ironic given our phone habits 60 years later. Pollsters may have missed some Democratic voters in 1948 because they were technologically behind Republican voters. They were less likely than Republicans to have land lines.
Shares