In the wake of the presidential elections, the market research industry is in a period of deep reflection. And for good reason: most major polls got the election result quite wrong. As Wonks everywhere examine the meaning and implications, one thing has become quite clear: advanced social listening and research isn’t now just a “nice to have” but a “must have.”

As reported by the Wall Street Journal, BBC, and others, in many cases social media intelligence helped forecast the correct election result through signals and analysis simply not available through traditional survey methods. It clearly reminded us that people are people; and not just numbers. Language that was used, the “intensity” of the opinion expressed, volume, emotion, engagement, actions, the unprompted nature of the analysis and other factors gleaned from this vast, unstructured, human conversation were shown in many cases to predict outcomes many others missed. And given that the president-elect cited social media as the critical component for his success, perhaps this is also not entirely surprising. As a relatively straightforward exercise, we tracked expressed “voter intent” (voters who explicitly posted who they were planning to vote for). These results, as below, predicted a Trump win. And this analysis was before applying more advanced analysis such as sentiment, emotion and intensity.

aaeaaqaaaaaaaaizaaaajduzmdblnwrjltzmmzytndjhzi05zwu3ltrindjlnda0zdixmw

What does this mean for social media and insight professionals? The next generation of social media listening and research is now no longer a “nice to do” strategy but must become an intrinsic input into any meaningful insights and analytics works, ranging from brand tracking, to predictive analytics, market mixed modeling and more. Not including social data translates into blind spots that can go missing through more traditional approaches.

As the leader in social insights/research for over a decade, the results of the election simply confirm our own experiences: rigorous and accurate social media data using the latest semantic technologies that goes beyond the “obvious” and analyze not just sentiment, but also emotion, intensity and more, is not just qualitative, but also quantitative, predictive and rapidly driving impact at some of the largest brands in the world. It is the effective and essential bridge between “data” and “human.”

As Joel Rubinson, former Chief Research Officer at the Advertising Research Foundation (and Converseon advisor), says, “Once again we see conclusive evidence that social media data are, in fact, data with predictive power regarding brand preference including political candidates as brands.”

New Machine Learning Approaches Yielding Essential Benefits

Much of this breakthrough analysis is due to new approaches to social intelligence, specifically the use of machine learning to mimic human analysis at a large scale and with high precision. Until recently the conundrum was how to balance accuracy with scale. It was simply not feasible to analyze millions of conversations through human coding and automated sentiment was simply not good enough.

No more: while most basic social listening techniques continue to take “keyword spotting” approaches, new technologies, like Converseon’s ConveyAPI technology, winner of Dataweek’s Top Innovator in Social Data Mining, use active, semi-supervised machine learning techniques that continually evolve and refine. The result is near human level precision and rigor at the speed and scale that only software can provide.

Key benefits to this approach include:

  • Analyze Conversation Like Humans at Massive Scale: These new technologies are now able to be applied to capture the nuances of human expression. For example, if “trust” is a key attribute, few consumers will explicitly say “I trust (x brand or candidate).” Instead they may say, “I slept better tonight knowing that my child is using (x brand).”    This is clearly an expression of “trust” that traditional “Boolean” or keyword phrases would miss. This ability to categorize and analyze nuanced language as humans understand it opens the door to a broad range of analysis previously not available, such as brand tracking. And with facet level analysis (the most granular level possible), we can capture not just an expressed sentiment towards a candidate, for example, but what topics/issues are driving that opinion. With millions of relevant conversations a day, these technologies are required to separate signals from the noise and understand what’s truly happening (and what might happen next) quickly.
  • Discover New Topics and Attributes: Researchers often feel like they know what topics and attributes are most important for a candidate or a brand. But what if they don’t? Many surveys are designed “top-down.” Organic topic discovery technology takes a “bottoms up” approach by analyzing large “conversation” data sets to unearth key that naturally bubble up from the data – without bias –and allow us to discover new areas of analysis that otherwise would go missed. Let’s face it, in the world of social media, opinions, attributes and topics change quickly and it’s critical for brands to understand what is emerging naturally to stay ahead of the curve. Were the topics being discussed really the ones that were most important to target groups? While brand attributes may be consistent, the values of your audiences are often changing constantly. Organic topic discovery is essential to align brand (and candidate) attributes to key segments and make sure nothing is missed.
  • Go Beyond “Just Data” to Human Expression: There is deep meaning in human expression that simply cannot be captured on a spreadsheet. ConveyAPI goes beyond sentiment to intensity, emotion and more. It bridges the qualitative and quantitative.  Understanding how strongly someone expresses an opinion, for example, is often a predictor of action (sales, votes, etc.). Advocacy allows the understanding of potential turn out and enthusiasm (versus just general sentiment). Combining the rigorous quantitative scoring with the ability to drill down to the actual discussions provides a level of context, texture and “humanness” to the data that is often missing in traditional insights and analytics (and was missed in some of the analytics used in the campaigns). In the wake of the elections, there was a clear admission by some of too much reliance on just faceless data and analytics. And in a situation where some voters were clearly reticent to share views in prompted surveys and polls, passive, unprompted “listening” represented a less intrusive approach.
  • Get the Lingo: Understanding the actual language of social media conversation is essential for insights, content marketing and even search engine optimization. Far too often candidates, and brands, talk past their targets. Social media conversation often precedes (and complements) typical keyword analysis. And as search engines delve even more deeply into semantic connections and topics, this social listening data is increasingly critical. What language is resonating? Which candidate shared the language of voters and who talked “passed them?” Memes like #crookedhillary were shown to take root, fairly nor not, where other memes did not. This type is what we call “Linguistic Analysis.”
  • Avoid Sampling: Sample theory clearly has its limitations. Black swans are often missed. With advanced social insight technology, entire data sets can be annotated and understood at vast scale. No sampling is necessary. There clearly was a large set of voters who flew under the radar of the survey samples. Today, the entire public social conversation can be analyzed quickly.
  • Time Travel: With data archived for several years, researchers can go also back to any point in time to delve deeply into the data for meaningful, accurate and unbiased results. Being able to go back to real data from the early stage of the Trump phenomena, before he became a serious candidate, for example, likely would have provided interesting insights for the Clinton campaign that they might have missed otherwise. This analysis can’t be surveyed retroactively. With social media, it can.
  • Know the “Who”: We also now know “who” is talking. Interests, political associations, psychographics, demographics, segment; our ability to understand not just about the opinions expressed but “who” is expressing those opinions allows for a higher level of analysis than previously available.

The results of the election simply confirm what many leading brands are already recognizing: that social data and insights are now essential to meaningful insights work. And clearly, there are deep lessons learned for brands and insights professionals from these election results.

This is why we at Converseon are seeing a significant upsurge in the use of our advanced social in many areas including social brand tracking, predictive analytics, influencer analysis, customer care, content marketing and much more. Social data is rapidly being integrated into brand trackers to both augment and at times replace traditional approaches, showing that “better, faster, cheaper” does actually exist. And as many learned, without it, you may well be flying blind.