Why are polls sometimes so wrong?

MERRIMACK, N.H. — When politicians see polls they don't like, they recite a cliche: The only poll that counts is the one on Election Day.

Tuesday, the voters of New Hampshire proved the cliche right.

For days, poll after poll showed Illinois Sen. Barack Obama opening a big lead heading into the New Hampshire Democratic primary. But when the votes were counted, New York Sen. Hillary Clinton won. Even she seemed surprised.

Were the polls all wrong? Did the pollsters misjudge how many women would vote? Did voters lie when pollsters called? Or were the polls right about Obama leading, proving that debates and campaigning the last weekend really do matter and can sway voters at the last hour?

Regardless of the answers, many analysts urged a postmortem to figure out what the heck happened in New Hampshire.

"It is simply unprecedented for so many polls to have been so wrong," said Gary Langer, the polling director for ABC News, in a memo posted at his Web site. "We need to know why."

Every poll done for the news media in New Hampshire after the Iowa caucuses showed Obama gaining and opening a lead on Clinton.

A McClatchy-MSNBC poll conducted immediately before and after Iowa showed Obama with the support of 33 percent in New Hampshire, Clinton with 31.

Polls conducted after Obama's Iowa win showed him with a bigger lead. One survey for C-SPAN and Reuters showed Obama up 42-29 percent over Clinton. Six public polls for news media and universities showed him with an average lead of 8.3 percentage points.

None showed Clinton close, let alone ahead. Yet she beat Obama by 39-36 percent.

So what happened?

One possibility widely mentioned Wednesday was that white New Hampshire voters might have lied to pollsters, expressing support for black Obama, then voting against him once they were in the privacy of the polling booth.

That's happened before, and it's noteworthy that there was no big discrepancy on the Republican side, where all top candidates were white.

"There will be a lot of claims about what happened, about respondents who reputedly lied, about alleged difficulties polling in biracial contests," Langer said. "That may be so. It also may be a smokescreen, a convenient foil for pollsters who'd rather fault their respondents than own up to other possibilities — such as their own failings in sampling and 'likely voter' modeling."

Lee Miringoff, a political scientist and the director of the Marist Institute for Public Opinion at Marist College in New York, noted that most public polls accurately measured Obama's support.

His own Marist Poll, for example, found Obama with 36 percent heading into the primary. An average of five polls measured Obama's pre-primary support at 38.4 percent, very close to the eventual vote total of 37 percent.

What the polls missed was the turnout for Clinton in New Hampshire.

The polling average before the primary showed her with the support of 31 percent of likely voters. She ended up with 39 percent.

One possible reason the polls were so far off was that pollsters miscalculated when they screened those who answered their phones to find "likely voters."

Another is that the timing of the polls missed a late surge of support for Clinton, particularly among women, influenced by a debate Saturday night, Sunday talk shows, round-the-clock campaigning and an emotional response from Clinton on Monday to the stress of the campaign.

"What the weekend polls found was an Obama lead as primary day approached," Miringoff said in an article on his Web site, www.maristpoll.marist.edu. "What they do not reflect is what was apparent here in New Hampshire. The context of the campaign was changing. The last hours of the campaign were a media feeding frenzy over Clinton's show of emotion when responding to a voter's question on Monday morning."

Indeed, such last-minute volatility may help explain why some Republican polls in Iowa were very different from the final results.

A poll of Iowans for McClatchy and MSNBC a week before the caucuses, for example, showed Republican Mitt Romney gaining and pulling ahead of rival Mike Huckabee, a former Arkansas governor, by 27-23 percent.

A poll three days later for the Des Moines Register found Huckabee ahead, 32-26 percent.

The final vote days later: Huckabee won, 35-24 percent.

So, the McClatchy-MSNBC poll was wrong and the Register poll was right — right? Not necessarily.

First, polls reflect public sentiment only at the time they're taken; they aren't predictions of a vote, even a few days later. Most news stories say that, or should.

Second, look at when the polls were conducted. Elections are dynamic processes. Candidates say things, campaigns air commercials, people talk, read, argue, change their minds. Polls on different dates can yield different results.

"Timing is half of everything," said Brad Coker, the managing partner of Mason-Dixon Polling & Research, which conducts polls for McClatchy and MSNBC.

In Iowa, a McClatchy-MSNBC poll conducted Dec. 26-28 found Romney ahead of Huckabee, but the final voting results last Thursday were the reverse.

Coker noted that two other polls taken at the same time produced similar results.

A survey by the American Research Group conducted Dec. 26-28 found Romney with 32 percent and Huckabee at 23 percent. A Strategic Vision poll Dec. 26-27 showed Huckabee with 29 percent, Romney with 27 percent. Both showed Romney with more support than he'd had before Christmas and Huckabee with less.

The Des Moines Register poll took a different snapshot over a slightly different period, Dec. 27-30. It had a larger sample, 800 likely voters in each party versus 400, which cut the error margin from 5 percentage points to 3.5.

The Register poll also assumed a greater turnout by first-time caucus attendees. Coker said he based his assumed turnout on historic averages.

He said he thought that the McClatchy-MSNBC poll was accurate at the time, and that circumstances changed to benefit Huckabee.

The key, he said, was Huckabee's decision Dec. 31 to pull a negative ad he'd planned against Romney.

"Romney had been closing the gap on Huckabee. I think our numbers were accurate," Coker said. "What happened that we can't account for is what happened in the days before the caucus was held. Huckabee turned the whole thing around with the decision not to go negative."

Huckabee also thinks his support swelled after that.

"I really believe the decision I made was part of the reason that we won and won decisively," he said.

While the news media scoffed at the gesture, Coker said, evangelical Christian voters liked it.

The McClatchy-MSNBC poll found that 48 percent of respondents were evangelical Christians, about the historic average.

"We got what we got," Coker said, explaining that the poll didn't apply a weighting formula to increase or decrease the Christian sample.

Yet on the night of the caucuses, an entrance poll found that 60 percent of the Republican caucus attendees were evangelical Christians.

All polls, of course, are just numbers. How they're read depends in large part on how the news media portray them. That's one reason that the National Council on Public Polls recommends that news media be cautious in interpreting polls.

Another thing to look for in polls is the margin of error.

That means that 95 percent of the time, any number in a poll could be higher or lower by as much as the margin of error. It's a matter of statistical probability.

The other 5 percent of the time? They're flat-out wrong.