Cold Coffee? Sentiment Analysis Examined

Social Media is often used for market research: using tools such as automatic, software-based sentiment analysis make it apparently easy to gage how consumers feel about a product … or do they? 

What does the article examine?
The article examines how reliable sentiment analysis software really is. Given that this type of software is used increasingly in marketing research, this article examines the fundamental question if automated research performed by programmes and algorithms can be trusted.

Which concepts are discussed in the article?
The article does not draw largely on previous theoretical frameworks (apart from coding of sentiments). Rather, the article compares outcomes of manual sentiment analysis and two software products providing sentiment analysis.

Where is the data from and what methodology is being used?
The dataset is derived from Twitter conversations about coffee, a common item talked about on social media … and previously researched. The comparison between of sentiment analysis is based on 200 random tweets, coded by humans and two different software packages frequently used to generate this type of analysis.
What are the main outcomes?
In short: software-performed sentiment analysis has some very serious problems. The article shows how two software programmes are not delivering the same results when compared, and how both software programmes fail to be convincingly similar to how humans would have coded the tweets.
discussing two concepts, implied meanings and irony all cause problems with #sentimentanalysis Click To Tweet

A number of potential reasons are discussed, which include tweets discussing two concepts, implied meanings and irony being used amongst others.

What are the implications and why should you read it?

The article seriously puts into question the often unquestioned reliance on software-generated sentiment analysis of social media data. In the article the authors give plenty of examples of tweets where one software “disagreed” with the other software they tried – or indeed often both software tools “disagreeing” with the human coders.
Humans and/or one/both of two software tools disagreed in over 2/3 of the cases. #sentimentanalysis Click To Tweet

The article clearly shows the dangers of over-reliance on software tools… and also questions how far such tools can ever replace human coding.

Also check out:
Chapter 10:”Measurement, Metrics and Analytics” in my book: Social Media Marketing

Full Reference: 

Canhoto, A. I., & Padmanabhanb, Y. (2015). ‘We (don’t) know how you feel’ – a comparative study of automated vs. manual analysis of social media conversations. Journal of Marketing Management, 31(9–10), 1141–1157. http://doi.org/10.1080/0267257X.2015.1047466


Also published on Medium.

You may also like...