Wednesday, February 18, 2009

Sentiment analysis

Think small - Understanding macro picture in a discipline often involves aggregation of micro-data. For example - GDP of a country is the sum of all income and output (not strict though), average happiness level is simple average of happiness index (let's just assume there exists one) among many other. Aggregation helps us "simplify" our observation. You can use sum(), average(), range () type functions to define the aggregation mechanism of a data. Since we often don't have technologies to use telescopic and microscopic view using the same device, the aggregate function (especially the average) leads to the conflict between intuition and reality.

Sep's work on sentiment analysis ( is exciting because it provides a unique flexibility to decompose the data and to aggregate them at the same time. The difference between small and large has become smaller !! This is a very powerful application of technology. It's like the ability to observe a social phenomenon from both macro and micro perspective.

Imagine - If we could build a massive machine that will monitor all the "economic activity" of the world and help us get a microscopic view of the economic world. If we could Geo-track the outbreak of disease or other social disruptions to the detail. How about observing the effect of a new marketing campaign or promotional message ? Did today's in NYtimes create more positive feelings or negative feelings ?

Interestingly, there are some companies who offer sentiment analysis software (refer to Sep has also offered some APIs for others to build applications on his work.

No comments:

Post a Comment