Old ANEW: A sentiment about sentiment analysis and word lists

Posted on Updated on

Anew

Affective Norms for English Words (ANEW) is one affective word list among a number of others. ANEW seems to be regarded as a sort of reference in sentiment analysis research. It records valence, arousal and dominance on 1034 words on a continous scale between 1 and 9.

A downside with ANEW is the restricted license: You are not allowed to use it in for-profit projects. Another problem is that ANEW was not developed for modern sentiment analysis. Slang words, such as ‘shit’ and ‘wow’, do not occure in ANEW. ANEW also lacks the inflection variants: It has ‘annoy’, but not  ‘annoyed’, ‘annoys’, ‘annoying’ and ‘annoyingly’. You have to do, e.g., word stemming to match words against ANEW.

I began to construct my own word list when I started to do sentiment analysis of COP15 Twitter messages and “temporal sentiment analysis”. It presently lists 1480 words with their associated valence between -5 and +5. I have inflection variants and slang words.

In the past days I have looked into the discrepancies between my list and ANEW. The figure shows a scatterplot of the valences of mine and ANEW for the words that I can match. I stemmed both ANEW and my list and listed the words that differed in positive/negative valuence. These words were: affected, aggression, aggressions, aggressive, applause, alert, alienation, brave, hard, mischief, mischiefs, profiteer, silly.

‘Brave’ and ‘applause’ I record as negative: A clear sign error in my list, that I need to correct.

‘Affected’ I have as negative (it is usually used in a negative sense), while ANEW has ‘affection’ as very positive. My word stemming has a problem here. There is a similar problem with my ‘alienation’ and ‘profiteer’ compared to ANEW’s ‘alien’ and ‘profit’.

‘Aggression’ and similar words I have as negative while ANEW has ‘aggressive’ as slightly positive which I find strange. The same pattern occures for ‘silly’. That is negative to me. ‘Hard’ I have as slightly negative while ANEW has it slightly positive. In some connotations the word is used positively, but otherwise it seems to be used in mostly negative contexts. ‘Alert’ I too have as negative, while ANEW sets it as positive. On Twitter it seems mostly to be used on a negative sense, e.g.:

Lost Pet Lost Pet Alert, have you seen Jake (link)

or

Russian Armed Forces on High Alert Over North Korea (link)

‘Mischief(s)’ I have as negative while it is slightly positive in ANEW. WordNet has two senses of the word that is clearly negative: “reckless or malicious behavior that causes discomfort or annoyance in others” and “the quality or nature of being harmful or evil”. WordNik reports a lot of definitions with one somewhat positive: “An inclination or tendency to play pranks or cause embarrassment.” On Twitter it often seems to be used ironically with a basic positive sense.

I need to fix the errors in my word list and extend it, but I think it is a good alternative to ANEW.

By the way Sune Lehmann Jørgensen has humorously called my sentiment word list AFINN as a wordpun on my name and ANEW. :-)

Advertisements

3 thoughts on “Old ANEW: A sentiment about sentiment analysis and word lists

    Jeff Berg said:
    December 20, 2010 at 2:29 pm

    Is there any chance you plan to make your word list public? I’ve been playing around with the ANEW word set to analyze some data and, in looking at some of the data, wondering what he impact would be of examining slang words as well.

    Anonymous said:
    December 21, 2010 at 2:18 pm

    @Jeff BergThanks for your interest in the word list. It is now available for download from the DTU Informatics publication database: http://www2.imm.dtu.dk/pubdb/views/publication_details.php?id=5981

    Elanor Colleoni said:
    February 21, 2011 at 4:42 pm

    good job!!!!

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s