Stinky Teddy is a meta-search engine that utilizes search feeds from multiple search providers. The glue that brings it all together are the so-called real-time search engines. Using data from these real-time search engines I can figure out how people are using words and phrases in their current online conversations.
Twitter is an invaluable resources for this real-time data as it is the largest available source for real-time user generated content. I highlight the word available because Facebook does not provide access to their users' real-time status updates. The one drawback of Twitter is that the contributers (Tweeters) are a highly biased sample of humanity, mostly techies and narcissists. Normal people don't tweet -- most average Joes and Janes on Twitter don't have enough real followers to justify the effort it takes to send out a Tweet (do you talk out loud when nobody is around to hear you?). Twitter by itself generally does not allow me to figure out what is important to "real" people.
So it was with great sadness that I read this week about the discontinuation of the OneRiot real-time search API. Technically they are launching their new Advertising Network, but hidden somewhere in the announcement is the demise of the search engine and search API.
I've been using the OneRiot search API for over a year now, but today deactivated it on Stinky Teddy (the search API will be turned off in a couple of days). This is a big blow -- OneRiot did a superb job of surfacing fresh content that normal people care about. I used the OneRiot API with the "rest of the world besides Twitter" option. This is the world where U2 is more important than Justin Bieber. I'm not sure of the exact details of OneRiot's algorithm, but it worked very well from both algorithmic perspective (great content) and a technical level (fast response, no outages, stable API). Goodbye old friend.
All is not lost. Collecta still helps me to capture the pulse of the normal world. Hopefully Bing opens up their social API (they are indexing Facebook public updates now). Also, I'm starting to index select parts of the web myself.