Last week, Gallup and USA Today announced an end to their 20-year partnership. The reasons for the divorce were vague. In a statement, Gallup said "the worlds of journalism and survey research have been changing and evolving," and that "Gallup.com has become the major outlet for Gallup’s work," implying that the organization no longer needs a media outlet to help spread its poll results (and its brand). USA Today, meanwhile, reaffirmed its commitment to polling, stating that the company "is in the final stages of negotiating an arrangement with another polling organization," implying that Gallup's results weren't up to the paper's standards. Amicable or not, this split is good news for nearly everyone—USA Today, other media organizations, poll aggregators, election watchers—but Gallup.
If USA Today pulled the plug because it was dissatisfied with Gallup's polling, then the paper made a wise move. Even though Obama comfortably won reelection by 4 points, Gallup’s final poll showed Romney ahead by 1 point and consistently found Romney winning by a wider margin—as much as 7 points—over the final few weeks of the campaign. The final USA Today/Gallup swing state poll was off by a similar margin, finding a tied race even though Obama easily won 11 of Gallup’s 12 swing states by a combined 4 points. It is worth noting that no other live interview survey contacting cell phones found Romney ahead heading into Election Day. And Gallup’s 2012 failures were not anomalous. Heading into the 2010 midterms, Gallup showed Republicans leading the generic congressional ballot by a staggering 15-point margin, far greater than their eventual 6-point victory. In 2008, Gallup found Obama winning by 11 points, again missing Obama’s 7-point victory by 4 points. In comparison, Pew Research survey struck within one percentage point of the final outcome in the same three elections.
It's no surprise, then, that some have questioned Gallup’s practices and methodology. Last year, the Huffington Post’s Mark Blumenthal found that Gallup was not properly weighting its sample of American adults to the census' targets for race, consequently understating the strength of Democratic candidates. Months later, Gallup introduced a series of methodological changes that appeared intended to correct for the flaws identified by Blumenthal, yet Gallup's surveys continued to underestimate the non-white share of likely voters, and consequently underestimated support for the president. Part of the problem was Gallup’s likely-voter screen, which excluded too many new or transient voters by asking questions like, "Do you know where people in your neighborhood vote?" Yet Gallup also found Obama doing worse among registered voters than other surveys, suggesting that their issues ran deeper than their likely-voter screen. And, perhaps most seriously, the Justice Department has sued Gallup, accusing the pollster of bilking federal agencies by inflating prices for contracts.
The good news, meanwhile, is that USA Today isn't quitting the polling business. For all the derision of “media polls” last year, national media organizations tend to employ pollsters that adhere to the industry’s best practices, including conducting live interviews with voters who only use a cell phone. Those surveys are increasingly expensive as response rates declined with the advent of call screening, and the media’s financial difficulties have combined with rising costs to reduce the number of media polls. At the same time, the demonstrable accuracy of polling aggregators like FiveThirtyEight, Pollster, and RealClearPolitics has diminished the news value of individual surveys. But the media’s commitment to high-quality, expensive surveys remains indispensible to the accuracy of polling aggregators, and USA Today’s decision to continue their national polling operation, despite the economics and their experience with Gallup, is excellent news for anyone invested in improving the accuracy of national polling averages.
Although the polling averages were generally right about November's election, the declining number of high-quality surveys was already taking a toll. The averages tended to understate the president’s eventual margin of victory, in part because of the proliferation of cheaper, low-quality polls that under-sampled young or non-white voters. Although a well-funded pollster like Gallup shouldn’t have fallen into that category, their flawed daily tracking poll exerted constant weight on the national averages. Unfortunately, the break-up with USA Today does not mean Gallup is done with political polling. Unlike many public pollsters, which are contracted by media organizations to conduct political surveys, Gallup’s public political polls are conducted at a loss, funded by revenue from Gallup’s other private ventures like consulting for large corporations.
So why does Gallup do it? The publicity these polls earn for the Gallup brand is priceless. But that brand has taken a big hit in the divorce from USA Today, the Justice Department's lawsuit, Blumenthal's expose, and years of poor polling results. Meanwhile, another firm is poised to take Gallup's place in the national polling mix—not that it'll be much of challenge for USA Today's new partner to improve on Gallup's wide margin of error.