I get the impression that a lot of people are getting tired of debating the details of PPP’s methodology, but I think it’s worth flagging a few recent polls in Georgia, which nicely illustrate the questions about their methodology.
Georgia, you may recall, is where this discussion began. Back in August, I argued that a PPP survey of Georgia was demonstrably too white (71 percent white, 24 percent black). On Twitter, PPP’s director, Tom Jensen, vigorously defended a relatively white electorate, arguing that 1) there was a black candidate on the ballot in 2010, therefore black turnout would be lower; 2) blacks were 24 percent of the electorate in 2006, the last time there wasn’t a black candidate.
But over the last two weeks, PPP released two new surveys of Georgia voters. One, conducted by PPP for Better Georgia, shows Republican Governor Nathan Deal’s lead falling into the single digits. PPP also conducted a post-shutdown survey of Georgia’s Senate race, finding Michelle Nunn locked in a dead-heat with an unnamed Republican challenger.
What’s striking is the abrupt transformation of the racial composition of the electorate. The white share of the electorate plummeted from 71 percent in August to 63 and 62 percent in the two October polls. The black share of the electorate was at 29 percent in the Better Georgia poll—nearly at 2012 levels—although just 25 percent in PPP’s newest public survey.
I just can’t fathom the justification for these types of fluctuations. PPP said, explicitly, that it did not believe the 2014 electorate would be this diverse.
And it has a real effect on their polls: If PPP had used the August weights for its October poll, Nunn would have trailed by 6 points. Deal would have led by 10 points if PPP used the August weights for its October poll. And obviously, it was in PPP's interest to avoid both of those outcomes. Rather than show either Nunn or her opponent clearly ahead in the Georgia Senate race, as they would have needed to if they had held the racial composition of the electorate constant at either October or August levels, PPP showed two dead-heats. In the gubernatorial contest, PPP reported that Deal’s standing had faltered after bad news coverage, but much of the change was just due to changes in the racial composition of their sample.
As far as I'm aware, the only good explanation is PPP's willingness to use the self-reported vote from the 2012 election, which Jensen says informs how PPP determines the racial composition of its samples. But Jensen said they only use the 2012 vote in one direction: to correct for overly liberal samples. That explanation is not consistent with the changing racial composition of PPP’s Georgia polls: If PPP only used the 2012 vote to correct liberal samples, and PPP believed that the racial composition of the electorate would be less diverse than it was in 2010, as they said on Twitter, then they should have published a poll with the racial composition of the August survey and a larger Romney advantage in the self-reported 2012 vote.
On the other hand, PPP’s Senate release doesn’t even include a question about the 2012 election. Now, as we’ve learned, that doesn’t mean they didn’t ask the question and use it—since they deleted nearly all of the 2008 election questions from their 2012 surveys. Referring to those 2012 surveys, Jensen said that they deleted the question to avoid Republican criticism. If that were true in this instance, it would mean that their survey showed Obama doing even better than he did in 2012. That would be highly inconsistent with PPP’s stated use of the self-reported 2012 vote; it would be far more consistent with the possibility that PPP was weighting toward a desired result or party ID.