Can Social Media Be Used for Political Polling?
This blog was originally authored by me and published on the Radian6 website on July 18, 2012. I’m posting it here for posterity.
Every month, there are one or two blogs that look at the topic of using social media for opinion polling.
It made me curious as to whether social media could be used for opinion and tracking polls in politics. Could political and campaign staff at the national level produce their own opinion polls on those issues that don’t justify the expenditure but surely haunt their sleep because they can’t commission a poll?
My conclusion? No, but it can give political operatives some early warning signals as to what issues are on a slow boil.
All Hail: The Glorious and Fundamental Principle of ‘Equal Probability of Selection’
The principle of equal probability is the fundamental core of polling. It states that every member of a population has an equal – or in some instances a known – probability of being selected in a sample,allowing pollsters to limit the number of calls they have to make; typically, 1,000 individuals is enough. Because everyone has an equal opportunity to be called, the opinions of these individuals are representative of the general population.
This is important. It is the first fundamental reason why social media cannot currently be used to replace polling in politics. This principle cannot be applied to social media monitoring today.
The Rules of Engagement: Comprehensive Sampling Frame
The comprehensive sample frame determines how polling agencies identify their random sample, ensure its randomness, and ensure its accuracy. Since 1986, the telephone has been the preferred method of polling. It has been the preferred method because upwards of 95% of US and Canadian homes, according to the most recent American and Canadian census, have a landline-based telephone, although that is beginning to change in significant ways.
When pollsters call this randomly selected list of individuals, they ask to speak to the person in the household that is 18 years of age or older who had their birthday most recently. This ensures that the individuals polled will be random, protecting the fundamental principle of equal probability of selection.
After the data has been collected and data processed, each respondent is assigned a weighted value so that the total weighted sample represents the demographic characteristics of the latest US or Canadian census. Age, gender, race, educational attainment, and region are weighted accordingly, and the result is a representation of opinions held by all Americans or Canadians.
It’s Cumulative but Not Representative
Data recovered from social media is cumulative in nature. It begins by pulling in large volume on specific keywords chosen by the analyst over a set date range. The data is then collected to show how many conversations are taking place on social media channels related to those keywords. Currently there is no way to impose the scientific principles discussed above.
In its current incarnation, social media monitoring does not allow for the selection of random individuals using the probability of equal selection principle.
- While upwards of 95% of Americans and Canadians can be reached by telephone, the percentage of individuals with access to the Internet in their homes in 60% in the United States and 79% in Canada. As a frame of reference, social media has not reached the level of universality achieved by the telephone.
- Unless someone volunteers to put their demographic data in their social profiles, it is not possible to know all of the important demographic information that is needed to properly weight the results. Location data, age, gender, race, and educational attainment are not always listed, and many people use profile images that are not of themselves.
Having no way to ensure that the information gathered via social media monitoring can uphold the rigorous scientific standards of traditional polling methods there is no way that the data can be made representative of anything, let alone the opinions of a nation.
The Political Variables
In the political arena, there is further reason why social media cannot take the place of traditional polling.
The majority of political opinions expressed on social media have a tendency to adhere to the meta-narratives of both political campaigns and the mainstream media. People may be inclined to include their personal opinions when retweeting a news article, but there is substantial room for a type of response bias, a phenomenon in which people do not share their own personal opinions because they may be socially unacceptable. Since social media is a public forum, the inhibition to state one’s true beliefs can be a significant factor in what they write.
Social media monitoring acts as a window into these conversations, but it is not possible to ask questions for deeper analysis on these topics or even on topics unrelated to those running in the established meta-narrative.
Finally, because the reliability of demographic data is subject to what users of social media voluntarily provide, it is impossible to account for regional and local issues that shape races district-to-district, riding-to-riding. It is difficult to drill down to understand how government policies are impacting local economies, or account for the popularity of an entrenched incumbent.
What Is It Good For?
The strengths of social media data in the political arena lie elsewhere and I’ll demonstrate that Friday using the last Canadian federal election, held in the spring of 2011, as my case study.