Dales' Electoral College Breakdown goes into excellent detail about polling methods in his recent update. Specifically, he talks about how pollsters determine who are likely voters. Some, like Gallup, use extensive questions to get at the issue. Zogby predicts turnout based on estimates by party. So, each poll is only as good as its methodology, and if its assumptions and predictions hold up, then the poll looks good, like Zogby predicting high turnout in 2000. Get it wrong, and you look like an idiot. He sums up the issues:
When examining poll results, it is important to keep in mind that not every likely voter poll is alike. Sometimes, comparing the results produced by one organization to that of another organization is like comparing apples to oranges.As you know, the Federal Review Composite Poll™ compares apples and oranges, by using a weighted average of polls. This means that if all of the polls have bad assumptions, are wrong in their predictions about turnout and who is a "likely voter", then the Composite Poll will be wrong, too. Unless, the polls are randomly wrong (not generally favoring one candidate or the other), in which case the hope is that the Composite Poll will even out the swings caused by the bad methodology. In addition to this hope that varying "likely voter" methodology will be evened out by the average, we also weight polls less heavily in the average if they purport to be registered voter polls or merely adult polls, and we weight polls more heavily if they have a larger sample of voters. But I don't weight the Zogby poll more heavily than, say, the Gallup Poll, if they all use "likely voters" and have the same sized sample. (Rasmussen, a special object of my disdain, is further reduced in weight just because I don't trust it. The reason I use it anyway is described on the methodology page).
The main pitfall in this weighted average approach is that one poll used in the Composite may have the perfect methodology and assumptions and be dead-on with it's prediction, but the Composite is going to dirty that up by averaging with other polls. But until November 3, we won't know who's assumptions were correct. It would be interesting to see Gerry take the raw numbers from Gallup or Zogby (if he can get them) and then apply his own assumptions regarding likely voters and party turnout. That's a number I'd weight heavily in the Composite.
