Can You Trust Polling Results: Representative Sampling

Headlamp returns to polling best practices with our second installment: representative sampling. An accurate sample is essential to a successful poll, and conversely, a sample that inaccurately represents a population can skew the data and subsequent conclusions. Based on the Association for Public Opinion Research’s (AAPOR) 2012 presentation, “A Journalist’s Guide to Survey Research on Election Polls”, Headlamp offers some insight on how to wade through survey sampling—the good and the bad.

Let’s begin with types of samples. A good sample is a probability sample. As seen in the slide below, probability samples employ a scientific method and involve a randomly selected sample. Polling people who you pass on the street may seem random, but you would just be polling the clientele of whatever coffee shop you happen to be in front of—not the real and entire population you’re looking for.


In addition, a sample has to encompass enough of the right people to be considered an accurate survey of a population. For example, this article in today’s Alaska Dispatch, points out the folly of a poll that over-sampled one segment of Alaska’s population. As that article makes clear; demographics are extremely important when it comes to determining the accuracy of a poll. For a poll to be considered a representative sample it must duly reflect the different demographic variances of a population. For example, a poll could not claim to be representative of a state’s population if its sample was 75 percent young, female Hispanics, while in reality young, female Hispanics only made up 5 percent of that state’s overall population.

Getting the sample right is absolutely vital to ensuring that a poll’s results can be trusted. Getting the question right is just as important. Up next: Who supports the governor’s plan to fill the budget gap?   What are the results when that same question is asked differently in several polls?