This installment of Headlamp’s examination into best polling practices will explore the common polling practice of weighting. Similar to sampling, weighting can make or break if a poll is an accurate representation of public opinion. Essentially, weighting is used to account for potential differences between a poll’s sample and the universe the poll is trying to represent. A pollster can weight by party affiliation, region, or like below, education level. Using fairly simple math a “weight” for each education level can be created for a population.
In political polling, weighting is often used to “weight” the sample to better reflect what the pollster believes the Election Day universe will resemble. That’s a different beast than a poll that is trying to reflect the mood of, say, the Alaskan general public. Who turns out on Election Day is a different universe from who receives a Permanent Fund Dividend check each year. Moreover, the breakdown of Alaskans who turn out for a primary election is vastly different than usually seen in a general election. In Alaska, where Republican Party registration outweighs those of Democrats dramatically, turnout can vary greatly between election types. The bottom line with regard to different weighting types is that readers should be cautious when applying a poll’s results to the general public. Often what a poll is testing, and how it is weighted, may mean it has no real connection to the public’s overall sentiments.
More often than not, incorrect weighting can be seen in the results in electoral polling. While Headlamp is the first to admit that accurately polling Alaska is no easy task, that is no excuse to be irresponsible when claiming to represent an Alaskan population. As former political blogger Amanda Coyne and elections forecasting expert Nate Silver have both pointed out in the past, past Alaskan public opinion and electoral polls haven’t exactly been representative of the real results. National pollsters tend to get Alaska wrong. PPP in particular has an extremely spotty track record in the state.
So why does this matter?
As Headlamp has said, polls are a necessary tool for testing public opinion and to gauge potential election outcomes. But public polls that are used as instruments to move legislators or push certain policy (e.g. recent polls commissioned by the Rasmuson Foundation and the Alaska Dispatch News) should do a better job of making their “weights” public. Using flawed “weights” can cause a dramatic swing in poll results, thus, Headlamp would encourage readers to be curious about the weighting of a public poll—or why they were not revealed.