As I have said many times previously, if 2009 was all about the hype of Real-time, the future is all about capturing Relevance in real-time. Datasift has partnered with Twitter to get the full Twitter firehose and is building a platform to enable curation and filtering in real-time.

An introductory video about Datasift was posted in their first blog post, which didn't reveal much about how the platform works. Now, uber-geek Robert Scoble has posted a video of an extensive discussion with Datasift's founder, Nick Halstead.

This post is a summary of Datasift as discussed above concluding with my own thoughts.

The Basics

Twitter's firehose at present has around 800 tweets/sec, or 70 million tweets/day. Datasift can filter this firehose using over 20 variables. Examples of these variables include:

• Profile information like name, location, bio, number of follows, followers, lists, etc.
• Text and language of tweets
• Geo-location of tweets
• Verified users
• Source of tweets web, Seesmic, TweetDeck, etc.
• Number of Retweets
• Whether tweet contains a hyperlink

Datasift is a rules-based engine that can filter this firehose using thousands of complex rules and provide a filtered stream in real-time within milliseconds. It is built using a Service Oriented Architecture and has an API.

The Rules

Rules can comprise of any combination of filters using the above variables. Rules can be combined and merged, or added and subtracted, into a single new rule. Stream outputs from Datasift using such rules can become columns in Twitter clients like TweetDeck.

Here are a few examples of how rules can be used:

• Show me tweets containing google from users who don't have social media in their bio, and who have more than 500 followers.
• Show me tweets from my curated Twitter list of tech brands that have more than 100 Retweets.
• Show me tweets originating from within a radius of 5 miles from the location of XYZ Conference that don't have swear words, irrespective of whether their tweets contain the hashtag for the conference.
• Show me tweets originating from Starbucks shops around the world, of users who are Verified Accounts, irrespective of what they're about.

Datasift's website is intended as a community website for curators and developers to collaboratively work on developing these rules. You can leverage rules created by others to avoid duplication of effort. Rules are classified with tags, and Datasift provides search, ranking and trending for easier discoverability of rules.

Partnerships for Influence Tracking and Sentiment Analysis

Datasift has partnered with PeerIndex and Klout to enable filtering using their influence and authority scores. It has also partnered with a firm for real-time sentiment analysis.

Thus, any of the above rules can be filtered further using such scores, and a stream of tweets with negative sentiment about a brand or product, combined with any other rules, can be monitored in real-time.

Alerts and Analytics

For esoteric rules that may provide a result infrequently, alerts can be set up. The example discussed is of any politicians from a Twitter list tweeting the word scandal. Developers can send these alerts as email, SMS, or notifications on smartphones.

The resulting streams from all rules applied by the engine are stored by Datasift. This data can be extracted, segmented, and analyzed later. For example, this can be used to track the performance of social media campaigns.

Relevance Filtering of Links

Datasift can use TweetMeme and other databases to check the links in tweets, and determine whether they are relevant to a specific topic. Not much details on how this is achieved, but apparently, Nick says that all sites are already classified into different subjects by Tweetmeme and other such databases.

Blekko-style Twitter Search

Datasift has developed a prototype of Twitter search along the lines of Blekko's slashtags. Thus, along with your query text, you can use filters such as /nolinks to get tweets without links, or /California to get tweets originating from CA.

RSS Feeds

Compared to the massive volume of the Twitter firehose, the volume of RSS is minimal. Datasift plans to have their own PubSubHubbub server. Developers and third-parties can plugin any RSS feeds and use Datasift's filtering rules to get an output feed.

Revenue Model

One option is free access to the stream with in-stream ads. Ads will be tailored and designed for the target form factor desktop/mobile/tablet/etc.

Second option is selling data B2B for developers and brand companies, charged by volume of data consumed.

Prospective Partners

Datasift is seeking to work with startups like Flipboard, who are creating new ways for curated content consumption. This can also include any of the startups focusing on Relevance, such as TwitterTimes or Paperli.

My Thoughts

When I compared approaches to filtering information for relevance, I had suggested that the service most likely to succeed would be the one that supports multiple approaches and platforms. We can easily see that Datasift supports all platforms and several approaches like crowdsourced filtering, influence filtering, location filtering, etc. It is easily the most powerful relevance filtering engine I have seen yet.

The market of end-users for curated real-time content is at present unknown. Startups involved in creating pleasant experiences for consuming content have yet to find a monetization strategy. The degree of Datasift's success from an end-user perspective is largely dependent on:

• The creativity of developers and curators to create compelling experiences, and
• How the monetization strategies of presentation apps fare and how Datasift is able to work with them

Nevertheless, with the amount of content being created online growing exponentially, curation and filtering will eventually become necessities for any social media client. It is just a matter of time.

I also see a bright future on the B2B front. By partnering with influence and authority tracking companies, combined with sentiment analysis, Datasift may already be a compelling choice for brand monitoring and social media reputation tracking.

Lastly, thanks to Robert Scoble and Nick Halstead for the interesting interview.

Any time there's a new movie hitting theaters, there's a good chance that some of your friends (and thousands of other people) are tweeting about it. Unfortunately, there's also a good chance that you're going to miss or forget about those tweets. fflick, a new site that was just launched by four ex-Digg employees, is looking to help fix that problem: it's built a sentiment engine to try to harness Twitter and turn it into a massive movie critic.

From a looks perspective, fflick is well done, with a colorful design that isn't overwhelming or cluttered. After logging in and hooking up the service to your Twitter account, you'll see a grid of films that are in theaters and opening soon, each with a percentage rating showing the overall sentiment of the film. Clicking on a film's logo will take you to the real meat of the site: the filtered tweets themselves. Initially you'll see a list of tweets from your friends that are relevant to the movie you're looking at this could prove very useful if you are following a lot of movie buffs on Twitter, but your mileage may vary. If your friends fail you, the second tab presents a list of tweets that the site has deemed interesting, which usually means they were tweeted by a celebrity or news outlet.

Finally, you'll find the Positive and Negative tweets, which display a list of tweets that the site has automatically categorized using its sentiment engine. After looking through a handful of movies, I found that the system was pretty good at identifying both negative and positive tweets, though there were occasionally some tweets that were only mentioning a film in passing or were miscategorized. My biggest gripe at this point is that tweets that are anticipating a movie (I can't wait to see Inception) and are actually reviewing a movie (Inception was great!) are categorized into the same bucket. In other words, just because a movie has a high positive score, it may not be any good it might just have a lot of buzz.

The fflick team says that's actually by design, as this gives studios a chance to tell at a glance if their film is generating a lot of positive or negative buzz. They also say that they're going to let you further slice the data in the future, allowing you to filter by criteria like age, gender, and intent.

fflick is planning to generate revenue by selling the data it collects to studios (they can also make money through affiliate fees by selling tickets, and through advertising). The team also says that fflick is only the first vertical they're going after they have plans to apply their sentiment engine in other areas down the line. The team of four is currently raising funding, and counts Digg founder Kevin Rose as an advisor.

fflick is far from being the first service to try to track sentiment across Twitter. In fact, we previously covered Flixup, which was looking to do something very similar with tweets and movie reviews (that company has now pivoted to launch Miso, a site to share what you're watching'). Mombo does this as well.

Other companies that have tackled sentiment on Twitter include Tweetfeel and Rankspeed. It's possible that Twitter itself may even offer sentiment tracking services at some point Summize was addressing this problem back in 2007, and was acquired by Twitter eight months later to power Twitter Search.

Analytic.ly facilitates the searching and filtering of online conversations in real time, over an extended period of time, across events, global social media stats, keywords and brands. The service is fed by a custom database with millions of social media conversations already stored and indexed.

After the jump what you need to know about PeopleBrowsr' s Analytic.ly.

The application in a sentence

Analytic.ly enables users at every level in industries affected by online dialogue to visualize, study and measure the conversations that define their markets.

Ways the product benefits social media campaigns

• Monitoring conversations: Through a friendly dashboard, keyword searches model conversations, revealing a sliding scale of results that can be examined at varying depths.
• Flexibility: Ability to get stats for any keyword or @name.
• Cross Platform: Analytic.ly gathers data from multiple social media platforms.
• Breadth: The Analytic.ly dashboard offers several different lenses to magnify what is happening around events, brands, keywords and global sentiment on any given subject.

Coolest feature

The DIY Analytics Platform allows searches on sets of keywords and @Names, shows historical and real time conversation data gathered from Twitter, Facebook, MySpace, blogs and more.

Feature rundown

• Events organizes and presents the total and most popular Tweets, Re-Tweets, links, word clouds and general social media data about TV shows, movies, concerts, elections, conferences all in real-time as well as historically. Data displays for mentions by volume and location, most popular links, word cloud and more.
• The Global dashboard unveils data on trends, number of users, gender and locations, as well as the state of global sentiment tied to keywords.
• The Brands module has been built for publishers and social media advertisers. Publishers can see the top 1000 brands that about which its audience is talking, as well as count and ranking of mentions for each brand. Brands can see a list of the top 10,000 influencers whose audience is talking about the brand, get qualified exposure and lift, plus identify hotspots and potential publishers.
• The Searches section grabs data about @Names, brand or other keywords, with data on those participating in the conversation, influential followers, the size of the audience, engagement levels, the number of views, keyword association, sentiment, demographics and location.

User profile

Analytic.ly is an introductory package for social media strategists and corporate social media team leaders who are looking for efficient ways to measure conversation. Available enterprise solutions offer deeper analytics.

Pricing

• $10/one keyword or @name
• $90/20 keywords or @names
• Enterprise DIY analytics platform pricing on request from PeopleBrowsr

Compared to competition

According to the company, Analytic.ly, Has the underlying data, is easier to understand, provides customization with its DIY analytics. platform and offers an end-to-end marketing solution including social media campaigns based on the analytics data.

With an ever-increasing number of social media marketing options, brands and agencies are searching for solutions to improve the effectiveness and efficiency of their campaigns. This series reports on significant product releases and updates to assist in selecting the most appropriate tools to meet individual objectives Recent posts covered the social media monitoring and management solutions HootSuite, Involver AMP, NetVibes.Com, Context Optional Social Marketing Suite and Scout Labs.

Analytic.ly facilitates the searching and filtering of online conversations in real time, over an extended period of time, across events, global social media stats, keywords and brands. The service is fed by a custom database with millions of social media conversations already stored and indexed.

After the jump what you need to know about PeopleBrowsr' s Analytic.ly.

The application in a sentence

Analytic.ly enables users at every level in industries affected by online dialogue to visualize, study and measure the conversations that define their markets.

Ways the product benefits social media campaigns

• Monitoring conversations: Through a friendly dashboard, keyword searches model conversations, revealing a sliding scale of results that can be examined at varying depths.
• Flexibility: Ability to get stats for any keyword or @name.
• Cross Platform: Analytic.ly gathers data from multiple social media platforms.
• Breadth: The Analytic.ly dashboard offers several different lenses to magnify what is happening around events, brands, keywords and global sentiment on any given subject.

Coolest feature

The DIY Analytics Platform allows searches on sets of keywords and @Names, shows historical and real time conversation data gathered from Twitter, Facebook, MySpace, blogs and more.

Feature rundown

• Events organizes and presents the total and most popular Tweets, Re-Tweets, links, word clouds and general social media data about TV shows, movies, concerts, elections, conferences all in real-time as well as historically. Data displays for mentions by volume and location, most popular links, word cloud and more.
• The Global dashboard unveils data on trends, number of users, gender and locations, as well as the state of global sentiment tied to keywords.
• The Brands module has been built for publishers and social media advertisers. Publishers can see the top 1000 brands that about which its audience is talking, as well as count and ranking of mentions for each brand. Brands can see a list of the top 10,000 influencers whose audience is talking about the brand, get qualified exposure and lift, plus identify hotspots and potential publishers.
• The Searches section grabs data about @Names, brand or other keywords, with data on those participating in the conversation, influential followers, the size of the audience, engagement levels, the number of views, keyword association, sentiment, demographics and location.

User profile

Analytic.ly is an introductory package for social media strategists and corporate social media team leaders who are looking for efficient ways to measure conversation. Available enterprise solutions offer deeper analytics.

Pricing

• $10/one keyword or @name
• $90/20 keywords or @names
• Enterprise DIY analytics platform pricing on request from PeopleBrowsr

Compared to competition

According to the company, Analytic.ly, Has the underlying data, is easier to understand, provides customization with its DIY analytics. platform and offers an end-to-end marketing solution including social media campaigns based on the analytics data.

With an ever-increasing number of social media marketing options, brands and agencies are searching for solutions to improve the effectiveness and efficiency of their campaigns. This series reports on significant product releases and updates to assist in selecting the most appropriate tools to meet individual objectives Recent posts covered the social media monitoring and management solutions HootSuite, Involver AMP, NetVibes.Com, Context Optional Social Marketing Suite and Scout Labs.

Analytic.ly facilitates the searching and filtering of online conversations in real time, over an extended period of time, across events, global social media stats, keywords and brands. The service is fed by a custom database with millions of social media conversations already stored and indexed.

After the jump what you need to know about PeopleBrowsr' s Analytic.ly.

The application in a sentence

Analytic.ly enables users at every level in industries affected by online dialogue to visualize, study and measure the conversations that define their markets.

Ways the product benefits social media campaigns

• Monitoring conversations: Through a friendly dashboard, keyword searches model conversations, revealing a sliding scale of results that can be examined at varying depths.
• Flexibility: Ability to get stats for any keyword or @name.
• Cross Platform: Analytic.ly gathers data from multiple social media platforms.
• Breadth: The Analytic.ly dashboard offers several different lenses to magnify what is happening around events, brands, keywords and global sentiment on any given subject.

Coolest feature

The DIY Analytics Platform allows searches on sets of keywords and @Names, shows historical and real time conversation data gathered from Twitter, Facebook, MySpace, blogs and more.

Feature rundown

• Events organizes and presents the total and most popular Tweets, Re-Tweets, links, word clouds and general social media data about TV shows, movies, concerts, elections, conferences all in real-time as well as historically. Data displays for mentions by volume and location, most popular links, word cloud and more.
• The Global dashboard unveils data on trends, number of users, gender and locations, as well as the state of global sentiment tied to keywords.
• The Brands module has been built for publishers and social media advertisers. Publishers can see the top 1000 brands that about which its audience is talking, as well as count and ranking of mentions for each brand. Brands can see a list of the top 10,000 influencers whose audience is talking about the brand, get qualified exposure and lift, plus identify hotspots and potential publishers.
• The Searches section grabs data about @Names, brand or other keywords, with data on those participating in the conversation, influential followers, the size of the audience, engagement levels, the number of views, keyword association, sentiment, demographics and location.

User profile

Analytic.ly is an introductory package for social media strategists and corporate social media team leaders who are looking for efficient ways to measure conversation. Available enterprise solutions offer deeper analytics.

Pricing

• $10/one keyword or @name
• $90/20 keywords or @names
• Enterprise DIY analytics platform pricing on request from PeopleBrowsr

Compared to competition

According to the company, Analytic.ly, Has the underlying data, is easier to understand, provides customization with its DIY analytics. platform and offers an end-to-end marketing solution including social media campaigns based on the analytics data.

With an ever-increasing number of social media marketing options, brands and agencies are searching for solutions to improve the effectiveness and efficiency of their campaigns. This series reports on significant product releases and updates to assist in selecting the most appropriate tools to meet individual objectives Recent posts covered the social media monitoring and management solutions HootSuite, Involver AMP, NetVibes.Com, Context Optional Social Marketing Suite and Scout Labs.

Analytic.ly facilitates the searching and filtering of online conversations in real time, over an extended period of time, across events, global social media stats, keywords and brands. The service is fed by a custom database with millions of social media conversations already stored and indexed.

After the jump what you need to know about PeopleBrowsr' s Analytic.ly.

The application in a sentence

Analytic.ly enables users at every level in industries affected by online dialogue to visualize, study and measure the conversations that define their markets.

Ways the product benefits social media campaigns

• Monitoring conversations: Through a friendly dashboard, keyword searches model conversations, revealing a sliding scale of results that can be examined at varying depths.
• Flexibility: Ability to get stats for any keyword or @name.
• Cross Platform: Analytic.ly gathers data from multiple social media platforms.
• Breadth: The Analytic.ly dashboard offers several different lenses to magnify what is happening around events, brands, keywords and global sentiment on any given subject.

Coolest feature

The DIY Analytics Platform allows searches on sets of keywords and @Names, shows historical and real time conversation data gathered from Twitter, Facebook, MySpace, blogs and more.

Feature rundown

• Events organizes and presents the total and most popular Tweets, Re-Tweets, links, word clouds and general social media data about TV shows, movies, concerts, elections, conferences all in real-time as well as historically. Data displays for mentions by volume and location, most popular links, word cloud and more.
• The Global dashboard unveils data on trends, number of users, gender and locations, as well as the state of global sentiment tied to keywords.
• The Brands module has been built for publishers and social media advertisers. Publishers can see the top 1000 brands that about which its audience is talking, as well as count and ranking of mentions for each brand. Brands can see a list of the top 10,000 influencers whose audience is talking about the brand, get qualified exposure and lift, plus identify hotspots and potential publishers.
• The Searches section grabs data about @Names, brand or other keywords, with data on those participating in the conversation, influential followers, the size of the audience, engagement levels, the number of views, keyword association, sentiment, demographics and location.

User profile

Analytic.ly is an introductory package for social media strategists and corporate social media team leaders who are looking for efficient ways to measure conversation. Available enterprise solutions offer deeper analytics.

Pricing

• $10/one keyword or @name
• $90/20 keywords or @names
• Enterprise DIY analytics platform pricing on request from PeopleBrowsr

Compared to competition

According to the company, Analytic.ly, Has the underlying data, is easier to understand, provides customization with its DIY analytics. platform and offers an end-to-end marketing solution including social media campaigns based on the analytics data.

With an ever-increasing number of social media marketing options, brands and agencies are searching for solutions to improve the effectiveness and efficiency of their campaigns. This series reports on significant product releases and updates to assist in selecting the most appropriate tools to meet individual objectives Recent posts covered the social media monitoring and management solutions HootSuite, Involver AMP, NetVibes.Com, Context Optional Social Marketing Suite and Scout Labs.

Frustrated by fridge thieves who continually ignore your polite (or not-so-polite) requests to keep their paws to themselves? Don't get mad get creative! You could end up with something so crazy it might actually workthat is, if it doesn't totally backfire.

You could go for the classic bluff, with the hopes that you'll inspire just enough self-doubt to encourage the thief to go for the next lunch bag over.

Or you could hold out hope that the thief in question is either very lazy, very stupid

or that he or she appreciates your twisted sense of humor enough to take mercy on your pathetic self.

Or you could just bring this for lunch. (Somehow, I think this would have been safe, even without the note.)

Thanks to Theresa in Birmingham, Alabama; Stacey in Manchester, New Hampshire; Alissa in Memphis, Tennessee; and Marianne in Dublin, Ireland for submitting!

related: That's breastmilk!

It's a beautiful thing to watch people tweet. Especially when you try to figure out what they're tweeting about, how they feel, and where they're tweeting from then put that all on a map and animate the changes over time.

Researchers from Northeastern University and Harvard College recently built a pretty and captivating visualization of U.S. Twitter users' moods, as determined by sentiment analysis. You can watch two days worth of tweeted moods in this video:

Watching the cartogram pulse as it indicates the proportional volume and sentiment of tweets is probably more interesting than the researchers' conclusions about their data: people are happier on weekends than weekdays; West Coast happiness trends are usually about three hours after East Coast happiness trends. Still, it's neat to learn that Thursday evening aka just about the time I'm posting this is the time of the week Twitter users tend to express the most negative sentiment.

Meanwhile, over at Betaworks, they've just posted a visualization of the record-setting sustained volume of tweets from around the world during the World Cup. There's no sentiment analysis here just counting of relevant hashtags but you can see the volume of tweets corresponding with the occurrence of games that people in a country are interested in.

People seem to tweet less about being frustrated than being triumphant; for the championship game, Betaworks noted an immediate lack of traffic in the Netherlands after losing while the attention in Spain persisted through the next morning. Or it could be that disappointed tweeters just left off the proper hashtag formulation when lamenting their team's loss.

These aren't the first Twitter geography-plus-time visualizations videos, of course. Here's one from last year that maps tweets with the words just landed and just arrived in:

Just Landed 36 Hours from blprnt on Vimeo.

Here's another of tweets from people who were pleased about Barack Obama's inauguration. And here's one from the New York Times depicting tweets during the Super Bowl. (Those two aren't embeddable, so you'll have to click through.)

Perhaps the data geeks who made these were too busy thinking about the visuals, but for some reason or another none of them include a soundtrack. So it's up to you to put some trippy music on, chill out, and watch the Twitter visualizations.

Calling Wolfram Alpha a computational knowledge engine surely puts off a lot of people who are mathematically and statistically challenged. I am one of them, so it took me a fair amount of time to get around to using Wolfram Alpha and the way it takes a query and spits out factual data.

If you are still drunk on Google and other traditional ways of doing search, take a sober pill and give Wolfram Alpha a look.

Yes, Wolfram Alpha still doesn't have a handle on the entire human knowledge, but it has its unique take on answering questions, comparing keywords, and projecting data.

After understanding the true power of Wolfram Alpha, you will appreciate it as one of the most powerful search engines out there. But is Wolfram Alpha too daunting for a layman? Looking at all the mathematical equations and statistical graphs that the search tool drums out, it seems so.

But even an Average Joe like me can sometimes throw everyday problems at it. And come out with answers easy enough to read. So here are ten of those search queries.

Which Website Is More Popular

Instead of individually looking up figures, you can do a quick comparison of websites to see which is more popular. Computational comparisons are a nifty feature of Wolfram Alpha. You can also lookup website data of any web address.

(Wolfram Alpha pulls in the data from Alexa.com's database. Some SEO experts do have a problem with the way Alexa draws its data however).

My IP Is?

Wolfram Alpha comes with location awareness. So it can used just like an IP tracker. Type in Where Am I?' and you get your exact geo-located co-ordinates.

Alternatively, you can find the souls behind an IP number or an URL by putting it as a search string.

As The Crow Flies

You can get the direct travel times and the distance between two cities. Average figures are calculated for ships, cars, and airplanes. Other information like local times at both locations is also attached.

Get To Know Your Kissing Cousins

Wolfram Alpha has a Genealogy engine that can trace your family relationships. To make it easier, you also get a tree' diagram. Just the thing to find out the right introductions at family dos.

What We Speak & Write

Wolfram Alpha has a host of tools or converters' which are of real use. For instance, you can figure the number of words needed to fill up the right number of pages (and the typing time to finish a book).

Then the search engine extends a cheating' hand with its puzzle solvers along with the usual dictionary help. Here's what the Words & Linguistics features can do:

The Time In Timbuktu

You can get the time information of any place, compare local times of several places, and also convert a time in any city to your time.

Following the Date & Time calculations, you can do a host of things like printing out calendars, find out how many days till Christmas, or get the dates of holidays for a given year. The results are as per your location.

Switch On The Weather Channel

Wolfram Alpha can be your personal weather search engine. It covers a host of ways to get specific weather information and forecasts for your city or one of your choice.

Watch What You Are Eating

Wolfram Alpha can be used as an online calorie counter. You can not only get nutritional information on a food, you can also get it for those that end up as packaged brands on our kitchen shelves. Yes, some of the information goes into the actual mineral and molecular composition, but you can pick out the calorie information for the serving and the percentage of daily intake. And export is all as a PDF.

Again, let's put a food (or beverage) through a comparison. It tells me that drinking vodka would be the quicker way to get the beer belly.

Hit The GymOr Climb The Stairs

You know how many calories you are picking up. So use the same search box to find out how many you can burn with a specific activity. There are a few ways you can compute the physical exercises needed to burn the fat.

When you do this calculation, also scroll down and check out the Equivalent Activities section. Mine tells me that I can repair an airplane for two hours or conduct an orchestra for three to burn the same amount of calories.

Color Swatch From A Search Box

We know bittersweet' as a feeling, but do you know what it looks like? Wolfram Alpha tells us that it's a color that's a shade closer to salmon and Indian red. Complementary colors are also given which tell us which colors can be matched. If you are looking for a three color scheme, pick up the swatch on the color triad. Designers can also quickly pick up the hexadecimal and RGB color codes.

From top to bottom, Wolfram Alpha is a search engine. Just like any other search engine, it depends on third party sources for the information and its accuracy. Also, results will hit the target or miss the spot depending on the search query you feed it. It is not a magic oracle but a work in progress. But the work done so far is impressive.

Let us know if and how you are thinking of using Wolfram Alpha every day.

Got Questions? Ask Them Now FREE on MakeUseOf Answers!

• The 5 Most Advanced Search Engines On The Web (29 comments)
• A Quick Guide To Google's New Search Features (23 comments)
• 6 Sites To Search For Images Shared On Twitter (13 comments)
• 5 Ways To Do A Site-Specific Search Using Search Engines (18 comments)
• 4 Awesome Visual Search Engines to Transform Your Search (12 comments)
• 3 Unique Bing Search Operators You Will Definitely Love (5 comments)
• 10 Web Tools To Try Out Sentiment Search & Feel the Pulse (12 comments)
• What Is A Meta Search Engine & How It Works? [Technology Explained] (5 comments)
• Two Easy Ways To Synchronize Zotero Data (11 comments)
• Top 7 Semantic Search Engines As An Alternative To Google (45 comments)

If you go by tweets, Americans are happiest in the early mornings and then gradually become more negative as the work day progresses, according to researchers at Northwestern and Harvard universities.

A team of five researchers at the two schools studied 300 million U.S. tweets from September 2006 to August 2009 and performed sentiment analysis on them. They found that the evenings (presumably when people get off work and are socializing with friends) and early mornings (before work) are the happiest times of the day. You can see in the map to the right that around 4 p.m., the map becomes bright red, indicating negative emotions. The trend is distinct on both the East and West Coasts.

Furthermore, sentiment creeps back up on weekends after a trough on Thursday evenings.

The researchers created the visualization of mood changes throughout a day below using density maps. Larger areas on the map reflect a greater number of tweets at that time. Green colors are more positive while reds are more negative. (You can tell that California never gets quite as unhappy as the East Coast too.)

The project is one of many ongoing studies into status updates and sentiment analysis. Facebook has its own in-house team led by a research scientist named Cameron Marlow. The social network tracks a metric called Gross National Happiness, inspired by the idea originally coined by the Bhutanese government in the early 1970s. There are constantly updated graphs on Gross National Happiness here for the U.S. and a number of other or Westernized countries where you can see similar troughs and peaks in sentiment around weekends and workdays.

Tags: sentiment analysis

Companies: Twitter

Exclusive SeatGeek, one of the finalists of TechCrunch50 2009, has raised $1 million in Series A funding led by Founder Collective and joined by NYC Seed.

The funding brings SeatGeek's total investment to over $1.5 million, and it intends to use the new capital to hire more engineers and business developers.

Founder Collective and NYC Seed thusly join SeatGeek's current investor line-up, which includes Stage One Capital, Trisiras Group, PKS Capital and angel investors Arie Abecassis, Sunil Hirani, Thomas Lehrman, Allen Levinson and Mark Wachen.

SeatGeek is a bit of an odd duck in the secondary ticket market. The startup essentially forecasts the price of sports and concert tickets on the secondary market, analogous to what Farecast (now Bing Travel) does for airline tickets.

This helps ticket buyers determine whether to buy a ticket immediately or wait for a price drop, while at the same time helping sellers identify the optimal time to unload tickets.

SeatGeek says its crawlers have helped compile a database of millions of ticket transactions as well as a bunch of other factors that influence prices, data it uses to algorithmically predict ticket prices as accurately as possible.

SeatGeek will henceforth focus on making its data and analytics more accessible to strategic partners, consumers, and the media. Recently, the company partnered with Nielsen Entertainment to bring its ticketing data and analytics to the record industry.

In addition, the New York-based company announced the launch of MLB Fan Sentiment pages, which show historical ticketing trends for each team and ranks each team on fan popularity.

For example, on the New York Yankees Fan Sentiment page, fans can see how recent events like George Steinbrenner's death and Alex Rodriguez' 600th home run chase affected fan interest in the team.

SeatGeek says it will roll out Fan Sentiment pages for all major sports teams and artists in the upcoming weeks.

So, how are these devices measuring up in the arena of public opinion? A few 140-character user reviews can say a lot about a phone's early success or failure, and we've got details on what social media junkies are saying about a few Android devices, including the Droid X, the HTC Evo, and the HTC Incredible.

In the past year or so, we've gone from smartphones to superphones slick, sexy pieces of hardware with huge screens, HD capabilities, powerful cameras, zippy processors, impressive browsing capabilities and more apps than you can shake a stick at. Technology has accelerated quickly over these months, and Android devices are competing nicely with Apple's iPhones. How they compete amongst themselves, however, is another story.

Many of these phones are selling faster than they can be built. And each one is making a significant media splash at its launch. In the public eye, is there one Android to rule them all?

Android Devices by Sentiment

In looking over data from sentiment analysis tool Chatterscope, which does some natural-language processing on tweets containing a given keyword, we see some interesting patterns emerging. Chatterscope even parses emoticons, and it can determine the difference between a moderately satisfied customer and a truly enthusiastic reviewer. Here's what we learned from tracking Android-related keywords for eleven days.

Between July 8, 2010 and today, Chatterscope collected 13, 303 tweets about the Droid X, 976 about HTC's Evo, and 427 tweets about the HTC Droid Incredible. Clearly, the Droid X which launched just last week saw the lion's share of buzz in terms of volume. We can attribute most of this noise to the fact that the device is brand-new and getting a lot of press coverage at the time, leading to more tweets and retweets of blog posts, consumer reviews and news articles.

When we dice up the tweets into positive and negative sentiment, a different picture emerges. While the Droid X gets more tweets overall, the Incredible shows by far the best positive-to-negative tweet ratio:

And when it comes to supergood and superbad tweets, i.e. the overwhelmingly positive and negative opinions and reviews, the Incredible was again the clear winner, with zero tweets of the I hate this phone and would rather die than use it variety.

The underperforming device in terms of consumer sentiment, then, is the Evo. Why would this device Sprint's first 4G phone, which had completely sold out by June 8, just 4 days after its debut be getting such a slew of negative Twitter reviews? We have seen some frustration expressed over this fact, which may account for a fair amount of negative sentiment; looks like the Evo's manufacturers need to step up their game while demand is still hot and before the buzz around other devices sweeps that demand aside.

Android Devices by Volume Over Time

Another app we used to parse Twitter data about these three phones is The Archivist, which looks at users, links, keywords and tweet volume over a period of time.

What we learned from a pure volume analysis is that the Incredible lags in terms of overall mentions possibly because the Incredible launched at the end of April; the Evo and Droid X, which debuted June 4 and July 15, respectively, haven't quite cooled yet in terms of media mentions and general public interest.

The Droid X shows a predictable peak around its launch date, but mentions have sharply declined, perhaps signaling a dropping-off in consumer interest. On the other hand, the Evo's sustained buzz signals good things for that device. Overall, during a 12-day period, the Archivist picked up 63,329 tweets about the Evo, 43,025 about the Droid X, and just 6,466 about the Incredible:

Android Devices Being Compared to the iPhone

Going into this analysis of Android versus Android, we wanted to leave any comparisons to the iPhone 4 completely out of the picture. However, the Archivist showed us that consumers themselves are making comparisons between these Androids and the iPhone in a huge number of their tweets.

We checked out the lists of words used in the same tweets as these Android brand names; only the Incredible didn't have the term iPhone in the top 5 associated keywords.

For the Droid X, iPhone was the third-most used word in tweets about the Droid X, outnumbered only by instances of the terms Motorola and Verizon. In these Droid X tweets, the word iPhone shows up in a full 10% of all updates.

When looking at tweets about the Evo, iPhone is the fifth most-used term, topped by HTC and 4G. The term iPhone appears in 9.8% of tweets about the Evo. And for the Incredible, iPhone is the tenth most-used term, showing up in 6.4% of Incredible-related tweets.

For qualitative data on why Twitter users would be putting Android phones in the same tweets as the iPhone, we turned to generic Twitter search. What we saw were frank, brief, gut reactions that compared one device to another; in a significant majority of the tweets we tallied, the Android devices were favorably compared and even considered preferable to the iPhone. Call reception, device speed, open-source software, device-specific features and even hardware aesthetics were touted as Android advantages.

Granted, these tweets all represent subjective user opinions; still, these are the kinds of intuitive, emotion-charged beliefs that move product and having these evaluations spread over a medium such as Twitter is definitely positive for the makers and retailers of these devices.

Let's just hope those manufacturers can keep up with demand.

As more and more Android superphones roll out with each passing month, we'd love to know which phones are catching your interest. Should we be tracking Samsung's Galaxy S, too? And what about the Charm from T Mobile?

Let us know which phones are most exciting to you; we'll add them to our list of devices to track, and we'll update you on their progress in a couple weeks.

More About: android, droid incredible, droid x, evo, htc evo, HTC Incredible, incredible, iphone, smartphone, social media, twitter

For more Mobile coverage:

• Follow Mashable Mobile on Twitter
• Become a Fan on Facebook
• Subscribe to the Mobile channel
• Download our free apps for iPhone and iPad

Shared by Mahendra
Challenges of real-time sentiment analysis

EVER noticed how opinionated the world has become? It's not just all those product reviews on websites like Amazon, Expedia or eBay. From Facebook to Twitter, personal opinions litter the blogospheresometimes in narratives apropos of nothing in particular; other times embedded in comments on the news; and frequently, sad to say, as flagrant plugs by shills masquerading as innocent bystanders.

Once we sought advice from friends and family about what, where and when to buy something. We read newspapers and magazines to form opinions about current events, movies to watch, new books worth reading, who or what to vote for, and why. We went to libraries to check out how to solve problems that concerned us. In business, we relied on surveys, focus groups and consultants.

That's all begun to change. Thanks to the internet, we are now inundated with advice from millions upon millions of opinionated folk we've never metand frankly haven't a clue how to assess. The fashionable term for trying to glean useful insights from it all is crowd sourcing. But read any selection of blogs (there are over 112m of them in the English language alone) and you quickly learn that meaningful information is in short supply.

Ironically, that may not matter much. As a fledgling investment banker, your correspondent learned many years ago that, as far as markets were concerned, emotions trump facts any day. It isn't events that move markets, but reactions to them, so long as they are shared by a big enough bunch of traders. What's the sentiment on the street? is the cry heard on trading floors around the world. Sure, in their Mr Spock mode traders scour the horizon for hard-nosed data, read their analysts' reports and digest breaking news. But then Captain Kirk kicks in and they interpret these through a prism of personal experience, with its predilections, prejudices, doubts, and fears.

That is equally true of politics and public opinion. At Oxford University, Sandra Gonzlez-Bailn and colleagues at Barcelona Media Innovation Centre have been using the emotional content of online discussions to predict how American presidents fare at election time. The technique provides an alternative to approval ratings, which gauge support based on a wide range of issues over the short-term, or opinion surveys, which collect responses to a narrow, pre-selected set of issues over the long-term. The attraction of the emotion-based approach is that it hones in on issues that people actually find important and want to discuss, rather than on topics predetermined by pollsters. It also offers clues to the psychological mechanisms that lie behind shifts in public moodas happened most noticeably in America after the terrorist attacks of September 11th, 2001.

But it is in the economic sphere that emotionally charged opinions matter most. They make the world go round by informing our purchasing decisions about houses, motor cars, mobile phones and many other bits of merchandise. So, we need to take opinionswhether level-headed or misguidedvery seriously indeed. Above all, we need to find better ways of tapping the wisdom of the jabbering online masses while dispensing with the drivel.

Your correspondent isn't the first to ponder how to exploit the wonders of word-of-mouth. Social scientists have been asking themselves similar questions for years. More recently, academics in natural-language processing have embraced the topic. Now, entrepreneurs are getting in on the act. Over the past few years, 60-odd companies have set up shop to develop tools for clients needing a better grip on what, deep down, their customers or constituents really think.

Most of the work to date has used semantic search engines to parse text retrieved from the web for meaning, disambiguating words with similar spellings by taking their context into account. So far, however, such natural-language processing has favoured narrow fields like medicine or law where the terminology is limited. The computational burden would be too much if used with conventional search engines like Google or Bing, which continuously index the entire web rather than merely a slice of it.

A better understanding of what's actually being said on the web has come from an approach called deep content analysis. This goes way beyond the realm of simple semantic search, allowing computers to understand the complete and unambiguous meaning of sentences. Still, it doesn't help distinguish the relatively clear, objective statements of fact from the invariably subjective and shifting opinions that give voice to a person's inner feelings, sentiments and attitudes to various things.

It is precisely the difficulty of extracting this emotionally charged content from the detached, hard-boiled sort that makes sentiment analysis such a tough nut to crack. Often, the relevant sentences or clauses are buried in long forum posts, blogs, or open-ended replies in stacks of questionnaires. Just finding them can take armies of analysts equipped with marker pens and printouts weeks on end. And then one has to decide whether the sentiment concerned is positive, neutral or negative, assigning some numerical ranking to it (say, +5 , 0 or -2), so the overall results can be digitised and processed as raw data.

The problem doesn't end there. In mathematical terms, an opinion is what Bing Liu, a computer scientist at the University of Illinois in Chicago, calls a quintuple or five-variable expression. The variables include the object being evaluated, its various features, the strength of the sentiment (in numerical terms), the person who expressed it, and when. The analysts' job is to mine the text for all five pieces of information in order to identify distinct quintuples. Next, any pseudo-sentiments (spam) must be filtered out. Only then can the data be processedand an averaged-out quintuple generated.

All of which sounds like a lot of hard work. No surprise, then, that so many start-ups have rushed to fill the need. Providing smart software that takes the grunt work out of mining text for opinions has helped Clarabridge of Reston, Virginia, grow at over 50% annually for the past few years. The company's automated sentiment tools are used by AOL, Marriott, Nissan, Wal-Mart, Wendy's, United Airlines and a dozen other Fortune 1,000 firms.

Clarabridge's software lets firms process all the customer feedback that normally goes to waste (typically 80%) because it's trapped in some unstructured form. Equally important, such software allows sentiment analysis, which would take weeks to do manually, to be carried out in real-timeand on an 11-point scale instead of the basic three-value sort (positive, neutral or negative). This gives firms a deeper understanding of their customers' needs, and helps them respond more rapidly to changes in demand.

Lately, your correspondent has seen a sentiment engine based on ideas derived from decoding the human genome that spits out real-time opinions about the stockmarket's behaviour almost as quickly as the index can react. He wouldn't be at all surprised if in a year or two such an opinion-harvester were bundled with a program-trading system to create a money-spinning killer app. If only he had got his hands on one before bidding farewell to the trading floor to become an impoverished inky-fingered wretch instead. An opinionated one, mind you.