Wherever there are sensations, ideas, emotions, there must be words.Swami Vivekananda

Sentiment AnalysisWhat, Where and How?

By: Raksha Sharma, PhD Student, IITBUnder Supervision of: Prof. Pushpak Bhattacharyya

Smile of Mona Lisa

Is she actually smiling?

Is she actually happy?

What is she happy about?

Mona Lisa16th CenturyArtist: Leonardo da Vinci

2

Sentiment Analysis (SA)

Task of tagging text with orientation of opinion.

This is a good movie.

This is a bad movie.

This movie is set in Australia.

Subjective

Objective

3

Applications of SA- In-domain SA

- Cross-domain SA- Star-rating Prediction

- Cross-lingual SA

SA Systems- Rule Based SA- ML-based SA

Lexical Resources for SA- SentiWordNet

- LIWC- Lexicon by Wiebe, Bing Liu etc.

Introduction of SA- Definition

- Challenges- Source of SA- Resources

Outline

4

Terminology/JargonSentiment Analysis

Opinion Mining

Emotion Analysis

Affective Computing

Affect Analysis

Positive or Negative

Happy or Sad or Angry or Surprised or Afraid etc.

5

Definition by Bing Liu“Sentiment analysis aims to determine the attitude of a speaker or a writer with respect to some topic or the overall contextual polarity of a document. The attitude may be his or her judgement or evaluation, affective state, or the intended emotional communication”

6

Definition: Sentiment as a Quintuple

< oj , fjk , sijkl , hi , tl >

Where oj is a target object, fjk is a feature of the object oj , sijkl is the sentiment value of the opinion holder hi on feature fjk of object oj at time tl.

7

Definition: Type of Candidate data for SA

a. Document based

The sentiment in the entire document is summarized as positive, negative or objective

b. Sentence based

Individual sentences bearing sentiments in the text are identified.

c. Phrase based

Phrases bearing sentiments in a sentence are identified.

8

Challenges of SA

9

Challenges of SA• Domain dependent

• Sarcasm

• Thwarted expressions

• Negation

• Implicit polarity

• Time-bounded

10

Domain DependenceFrequency of Usage:

“The main characters were miscast”

“The storyline was well-written”

“It’s a blockbuster”

• Can we guess domain here? If yes, then why?

Chameleon Nature:

“ The car’s steering wheel is unpredictable”

“The story line of UdtaPunjab was unpredictable”

11

Sarcasm in Speaking

Sarcasm uses words ofa polarity to represent

another polarity.

Example: “The perfume is soamazing that I suggest you wear it

with your windows shut”

12

ThwartingThe sentences/words that

contradict the overall sentiment

of the set are in majority

Example: “The actors are good,

the music is brilliant and appealing.

Yet, the movie fails to strike a chord.”

13

Negation“I did not like the movie.”

“Not only is the movie boring, it is also the biggest waste of producer’s money.”

“Not withstanding the pressure of the public, let me admit that I have loved the movie.”

14

Implicit Polarity“This phone allows me to send SMS.”

“This phone has a touch-screen.”

“It’s bigger in size than normal smartphones”

15

Time Bound“The camera of the mobile phone is less than one mega-pixel – quite uncommon for a

phone of today.”

“Nobody is using windows based phone now a days”

16

Flavours of SA• Subjective/Objective• Emotion analysis• SA with magnitude• Entity-specific SA• Aspect-specific SA• Perspectivation

“The movie is good.”

“People say that the movie is good.”

“This movie is awesome.”

“dude.. just get lost.”

“Whoa! Super!!”

“Taj Mahal was constructed by Shah Jahan in the memory of his

wife Mumtaz.”

“Taj Mahal is a masterpieceof an architecture and

symbolizes unparalleled beauty.”

“India defeated England in the cricket match badly.”

“The camera is the bestin its price range. However,a pathetically slow interfaceruins it for this cell phone.”

“The Leftists were arrestedyesterday by the police.”

17

Source of Data for SA

18

Source: Opinion on the Web• Does web really contain sentiment-related information?• Where?• How much?• What?

19

User-generated content• Web 2.0 empowers the user of the internet

• They are most likely to express their opinion there

• Temporal nature of UGC: ‘Live Web’

• Can SA tap it?

20

Where?• Blogs• Review websites• Social networks• User conversations

A website, usually maintained

by an individual with regular

entries of commentary,

descriptions of events.

Some SPs: Blogger, LiveJournal,

Wordpress

• Blogs• Review websites• Social networks• User conversations

Multiple review websites offering specific to general-topic

reviews

Some SPs: mouthshut, burrrp,bollywoodhungama

• Blogs• Review websites• Social networks• User conversations

Websitesthat allow people to

connect with one anotherand exchange thoughts

• Blogs• Review websites• Social networks• User conversations

Conversations betweenusers on one of the above

21

How much?• Size of blogosphere

– Through the ‘eyes’ of the blog trackers

• Technorati : 112.8 million blogs (excluding 72.82 million blogs in Chinese as counted by a corresponding Chinese Center)

• A blog crawler could extract 88 million blog URLs from blogger.com alone• 12,000 new weblogs daily

Reference : www.technorati.com/state-of-the-blogosphere/

22

How much?• 12,22,20,617 unique visitors to facebook in December 2009

• Twitter: 2,35,79,044

Reference : http://www.ebizmba.com/articles/social-networking-

websites

23

What? Reviews• www.burrrp.com• www.mouthshut.com• www.justdial.com• www.yelp.com• www.zagat.com• www.bollywoodhungama.com• www.indya.com

Restaurant reviews (now, for a variety of ‘lifestyle’ products/services)

A wide variety of reviews

Movie reviews by professional critics, users. Links to external reviews also present

Professionals: Well-formed

User: More mistakes

24

A typical Review website

Snapshot: www.mouthshut.com25

Sample Review 1(This, that and this)

FLY E300 is a good mobile which i purchased recently with lots of hesitation. Since this Brand is not familiar in Market as well known as Sony Ericsson. But i found that E300 was cheap with almost all the features for a good mobile. Any other brand with the same set of features would come around 19k Indian Ruppees.. But this one is only 9k.

Touch Screen, good resolution, good talk time, 3.2Mega Pixel camera, A2DP, IRDA and so on...

BUT BEWARE THAT THE CAMERA IS NOT THAT GOOD, THOUGH IT FEATURES 3.2 MEGA PIXEL, ITS NOT AS GOOD AS MY PREVIOUS MOBILE SONY ERICSSION K750i which is just 2Mega Pixel.

Sony ericsson was excellent with the feature of camera. So if anyone is thinking for Camera, please excuse. This model of FLY is not apt for you.. Am fooled in this regard..

Audio is not bad, infact better than Sony Ericsson K750i.

FLY is not user friendly probably since we have just started to use this Brand.

‘Touch screen’ today signifiesa positive feature.

Will it be the same in the future?

Comparing old products

The confused conclusion

From: www.mouthshut.com26

Sample Review 2(Good old sarcasm)

“I’ve seen movies where there was practically no plot besides explosion, explosion, catchphrase, explosion. I’ve even seen a movie where nothing happens. But White on Rice was new on me: a collection of really wonderful and appealing characters doing completely baffling and uncharacteristic things.”

Review from: www.pajiba.com27

What? Comments• Two types of comments:

– Comments about the article/ blogpost:• Very well-written indeed…

– Comments about the topic of the article:• I agree with you.. I used to love **’s movies at a point of time but these

days all he comes out with is trash. <Often leads to a conversation>

• Comments about the blogger:• If you think Shahid Kapoor is ugly, go buy glasses. While you are at it, buy

yourself a brain too

28

Lexical Resources for SA

- SentiWordNet- LIWC- Trends

- Lexicon by Wiebe, Bing Liu etc.

29

Lexicons• SentiWordnet (SWN)• Linguistic Inquiry and Word Count

(LIWC)

• Lexicon by Bing Liu

• Lexicon by Wiebe

excellent

pathetic

poorilleg

al functional

worthwhile

fabulous

blunder

disaster

extravagance

Over-the-top

30

SentiWordnet (SWN)

• Maximum of triple score (for labeling)• Difference of polarity score (for semantic

orientation)

Max(s) = .625 → Negative

pestering P = 0, N = 0.625, O = 0.375

Diff(P,N) = - 0.625Negative

31

LpLn

synonymy

antonymy

Construction of SWN

The sets at the end of kth step are called Tr(k,p) and Tr(k,n)

Tr(k,o) is the set that is not present in Tr(k,p) and Tr(k,n)

Seed words

32

Linguistic Inquiry &Word Count (LIWC)

Core dictionary of 4500 words and word stems (e.g. happ*) organized in 4 categories

PronounsPrepositionsConjunctions

Linguistic processes

InterjectionsFillers (“hmm”, “oh”)

Speaking processes

Words related to work, home, etc.

Personal concerns

Words dealing with affect and opinion

Psychological processes

Tentative (possible)Certainty (definitely)Inhibition (prevented)

....

Cognitive processes

Positive emotion Negative emotion Anxiety

Anger Sadness

Affective processes

915 words

713 words

33

Creation of LIWC

34

•Define Category Scales•Determine categories•Determine how they can be grouped into a hierarchy

•Populate Manually•Manual evaluation by three judges•For each word, decide whether or not a word should be placed in this category or moved higher up in the hierarchy

Trends of LexiconsApproach

Labels

Key takeaway

LIWC

Manual

Hierarchy of categories

Decide hierarchy of categories; have judges interacting with each other

ANEW

Manual

Valence, Arousal, Dominance

have a set of annotators annotating in parallel

EmoLexi

Manual

Five emotions

Use crowd-sourcing. Attention to quality control.

WordnetAffect

Semi-supervised

Affective labels

Annotate a seed set. Expand using Wordnet relations.

Chinese emotion lexicon

Semi-supervised

Five emotions

Annotate a seed set. Expand using similarity matrices

35

Some Popular Lexicons• Lexicon by Bing Liu: A list of positive and negative opinion

words or sentiment words for English (around 6800 words). – Link: https://www.cs.uic.edu/~liub/FBS/sentiment-analysis.html#lexicon

• Lexicon by Wiebe: A list of positive and negative opinion words or sentiment words for English (around 1900 words).

– Link: http://mpqa.cs.pitt.edu/lexicons/

36

SA Systems- Rule Based SA- ML-based SA

37

Approaches for SASentiment Analysis

StatisticalRule Based

Rule based classifierBy using Sentiment Lexicon

Unsupervised classifierLearning subjectivity

from an initial set of clues

Semi Supervised classifierUsing Active Learning

Supervised classifier(Need annotated corpus

completely)Bayes classifier, SVM, Logistic Regression etc

38

ClassificationInput: d-dimension feature vector: X = [x1, x2, …… , xd]

Output: Y: scalar, discrete

- Y belongs to { 2,3, …., k}

Goal: Learn a function (a classifier or a model): F: X->Y

Training of classifier (Learning of function F): statistical or rule-based

39

X: Feature - Set Identification of right feature-set (collection of words or their combinations) is very important for any classification task.Example:

1. Sentiment Analsysis: Task: Whether a given review is positive or negative

X= {All adjectives presernt in corpus as features or All adverb and adjective as features or all unique words as features etc.}

Class: k=2 (positive or negative)

2. Email Classification: Task: Whether a given mail content is spam or not X = {Sender-name, IP-address, make, money, lottery} K=2 (spam or not-spam)

3. Customer Risk Assesment: Task: Whether a customer registered on FlipKart is fraud (defaulter) or notX = {............................................................................................}K = 2 (Defaulter or not defaulter) 40

One very Naive SA systemAssign +1 weight to all positive words and -1 weight to all negative words.

f(X) = w1 * #positive-words + w2 * #negative-words

Example: #UdtaPunjab is a very well scripted and awesomely directed movie. Hat's off to the producer for producing such a non commercial film. All four actors were up-to-the-mark!!

- Polarity orientation of words can be obtained from any reliable lexicon.

X = Features = [well, awesomely, Hat's off, up-to-the-mark]

f(X) = (+1 * 4) + (-1*0) = +4

//Assigning a fixed weight is not a good idea, participation of word in review sentiment classification may vary accoridng to the corpus. Hence, learning of weight from the courpus is more accurate way for sentiment classification.

41

A Rule-based SA engine

Turney, Peter D. "Thumbs up or thumbs down?: semantic orientation applied to unsupervised classification of reviews." Proceedings of the 40th annual meeting on association for computational linguistics. Association for Computational Linguistics, 2002.

42

A Rule-based SA engine (2)

43

A Rule-based SA engine (3)

44

A Rule-based SA engine (4)

45

Probability-based SA system (Statistical)

Task: Determine class (weight) of a word from corpus.Compute: Pr(tag|word), given labelled Corpus

Pr(tag|word) = Pr(tag, word) / Pr(word) = Count(tag,word) / Count(word)

Example: It was an awesome movie.

Pr(positive|awesome) = 150 / 200 = 0.75Pr(negative|awesome) = 50 / 200 = 0.25

Pr(positive|awesome) > Pr(negative|awesome)

Positive Negative

Awesome 150 50

Statistics from corpus (Assumption)

46

A ML-based SA engine

Pang, Bo, Lillian Lee, and Shivakumar Vaithyanathan. "Thumbs up?: sentiment classification using machine learning techniques." Proceedings of the ACL-02 conference on Empirical methods in natural language processing-Volume 10. Association for Computational Linguistics, 2002.

47

• Predicting reviews as positive or negative on the document level

• Simple ML-based classifiers– Term presence/Term frequency– Unigram/bigram– Adjectives

A ML-based SA engine (2)

48

A ML-based SA engine (3)

49

Applications of SA- In-domain SA

- Cross-domain SA- Star-rating

Prediction- Cross-lingual SA

50

In-domain SA

• Statistically significant words represent the usable information for sentiment classification in a domain.

• Identification and Exclusion of irrelevant information for classification leads to improvement in accuracy of in-domain SA.

51

Test for significance of “unpredictable” in the Movie domain

Example: Unpredictable

Observed (O): Actual count of the word in the corpus.Expected (E): Half of the total count of the word in the corpus.

52

➢Presents sentiment analysis in a domain as a two-stage process.

Inclusion of DDPW in Sentiment Analysis

Data division: - 70% for training & DDPW, - 15% cross validation, - 15% for testing.

53

Inclusion of DDPW in Sentiment Analysis (2)

54

Star-rating Prediction

55

Star-rating Prediction (2)

56

Star-rating Prediction (3)

57

Sentiment Analysis at IIT Bombay

58

●Balamurali, A. R., Aditya Joshi, and Pushpak Bhattacharyya. "Harnessing Word-Net senses for supervised sentiment classification." Proceedings of the Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 2011.

●The authors proposed a novel approach of using word sense as features instead of the traditional lexical features.

●The motivation behind this was: “A very good 8GB pen drive at such a cheap rate ” (positive) or “Video quality of my new MP4 player is very cheap” (negative)

●They observed that the accuracies improved when the senses were used as opposed to simple word features.

SA using WSD system

59

SA using Sentiment Lexicon

60

●Joshi, Aditya, et al. "C-Feel-It: A Sentiment Analyzer for Micro-blogs." ACL (System Demonstrations). 2011.

●C-Feel-It: sentiment classifiers based on bag-of-words features.

●Bag-of-words features are derived from the following lexicons:

SentiWordNet (Esuli and Sebastiani, 2006)

Subjectivity lexicon (Wiebe et al., 2004)

Inquirer (Stone et al., 1966)

Taboada (Taboada and Grieve, 2004)

●The system achieved an accuracy of 52.58% .

C-Feel-It

61

TwiSent●Mukherjee, Subhabrata, et al. "TwiSent: a multistage system for analyzing sentiment in twitter." Proceedings of the 21st ACM international conference on Information and knowledge management. ACM, 2012.

●Twisent : A sense based system using various heuristics (modules) and linguistic resources (lexicons).

●It incorporates the following modules:

spam filters, spell checker and normalization Pragmatics, entity specificity

The system achieved an accuracy of 66.69% which was 14.11% higher than C-Feel-It system.

62

DisSent●Mukherjee, Subhabrata, and Pushpak Bhattacharyya. "Sentiment Analysis in Twitter with Lightweight Discourse Analysis." COLING. 2012.

●DisSent : based on discourse features.

●It makes use of discourse relations like the connectives and conditionals which can be used to incorporate discourse information in any bag-of-words model, to improve sentiment classification accuracy.

●Example: “Think i'll stay with the whole 'sci-fi' shit. but this time...a classic movie.” (Overall polarity of the sentence is positive).

●The system achieved an accuracy of 88.13% (Unigrams + IWSD Sense of Unigrams + Discourse Features)

63