Public opinion polls may not be as straightforward as you think

Public opinion polls are often considered "the will of the people" but a new study on the role of polls in South Korea shows that they may not always be that transparent.

"Using polls to gauge what people think about politics is not as simple as it sounds, as there are multiple mediating factors between what people think and how their views may be represented in the media," says co-author Sunmin Kim, an assistant professor of sociology at Dartmouth. "Our research shows more broadly, how in a democracy, measuring what people think or want can be a highly haphazard and unpredictable process based on the ways public opinion poll data are collected and reported."

The findings are published in Politics & Society

To examine the relationship between public opinion and polling, the research team chose to focus on South Korea because in the span of one generation, the country transitioned between the 1950s to the late 1980s from an authoritarian regime to a democratic society where polling has become an integral part of the political fabric.

In addition to forecasting elections, polls are sometimes used to select the candidates for national primary elections. For example, during the 2002 presidential campaign, following a televised debate between two opposition candidates, Roh Moo-hyun of the Millennium Democratic Party and Chung Mong-joon, who ran as an independent, formed a coalition, with Roh as the candidate on the basis of winning a poll. He wound up defeating another candidate and winning the presidency.

In spring 2014, Kim and co-author Taeku Lee, the Bae Family Professor of Government at Harvard University, traveled twice to South Korea to interview 23 pollsters, journalists, academics, and campaign experts. They asked about the public perception of polls, the quality of the data, media coverage of poll results, and the relationship between polls and democracy in the country.

"There's a contradiction between the ideal of polling and its practice," says Kim. "The reality is that there's all kinds of noise, including bad behavior, and opportunism that comes into play."

Through this qualitative study in South Korea, the results demonstrate how the process of polling has its weaknesses, which pertain to technology, methodology, intentions, and reporting of data.

First, technology may affect a polling sample. Although South Korea boasts a 95% to 98% cell phone coverage rate, the interviewees caution that conducting polls by cell phone may result in overrepresenting young people.

Second, the way a poll is administered and whether it is automated or conducted in person can affect the response rate. Respondents may be more inclined to express their opinions depending on which poll method is used.

In South Korea, some political candidates hire polling firms to conduct polls, but in reality, they are hired to help with campaign messaging through the poll questions, such as by asking negative questions about a rival.