Web 2.0

This evening, I was adding new links to my Google Profile so as to improve my social search experience. As I added MyBlogLog profile, I got totally thrilled to see that all my social profiles were automatically identified by Google. These were the sites and services which I added to MyBlogLog earlier.

Now, this is really a great use of such open data and results in a wow experience. Lijit, a social blog search tool, and some other services have been using this data since a long time. If you are a developer, you can also grab this data using our APIs or microformats.

But, one way open is not enough!

While Google lets you import data in, you cannot export the same data out. Looking forward, I hope Google, Facebook, Yahoo and others will become more two way open and will drive creation and adoption of such open standards.

Exactly one year ago today, I launched Retweet Rank. Just a weekend hack, but turned out to be a lot more useful service than what I expected. Retweet Rank was the first service to track retweets and measure influence of twitter users based on how frequently they are retweeted and continues to be a favorite place on the web.

The traffic numbers are not bad. To date, retweet ranks of 760,000 users have been looked up. Every month, thousands of users tweet their rank and over 50,000 unique users visit the site.

Things which delighted me the most were tiny mentions of the service from folks like Evan Williams, Robert Scoble, Tim O’Reilly, Guy Kawasaki, Mashable, ReadWriteWeb and many more. It also got highlighted in five books, including a bestseller, Web Analytics 2.0, by Avinash Kaushik.

Retweet Rank is a playground for me. I have been experimenting with SEO, placements, messaging, advertising, related users, etc. I will continue to do small changes to make the retweet tracking and quality measurement on twitter easier. It would be interesting to know what new you would like to see on Retweet Rank. If you have any thoughts, do leave your comments below.

Thank you all for the support and wish you a very Happy New year!

I read an interesting post from Chris Heilmann this evening on how you can be a better developer by being cleverly lazy. I especially liked this paragraph from his post:

Librarians are great people. Instead of knowing everything that is in the library they know exactly where to find the information to solve a problem. As a cleverly lazy developer you should do the same.

As a developer, you and I should know where to find the best solutions and how to use them. Be up-to-date with what’s happening in our domains and smartly solve our problems.

If you are developing web applications, you should read chris’s blog, you will find it interesting.

Rediff.com (a leading Indian web portal) has launched the new design with minimal stuff on it. Lot more clean and tidy. The focus is essentially on search, as Ashish writes at pluggd.in:

The bet is very simple – content game hasn’t worked (the good news is that it’s not working for others as well). Search has higher CTR (clickthrough rate) and that’s where the monetization might work for Rediff (though the search intent falls under Google’s purview!)

Another thing to note is ads are gone from homepage now. This means better user experience but a drop in short term revenue for rediff. Given they are struggling to retain users, this move looks good.

I appreciate the new design much more. But, like earlier I won’t use it. Also, looking at initial reactions, I doubt if this design will appeal to their existing users. Their focus on search can actually prove fatal.

Did they forget Next button?

Search results do not show a next button, though previous button is present.

Rediff mobile is still the same heavy site. I hope its redesign will follow soon.

Twitter has become a powerful medium for viral marketing and bringing traffic to a website. My hack: retweetrank, went viral all due to twitter. Tweets by Robert Scoble, Tim O’Reilly, Guy Kawasaki and tons of others (600+) made it easy for retweetrank to cross 30,000 users within 10 days.

One of the notable thing was the spike in site traffic after Scobleizer tweeted the link. Below is the graph showing requests/sec.

Here I coin (:P) a term: “Scobleizer effect“:

“Scobleizer effect is the phenomenon of a popular twitter user tweeting a link of a smaller site, causing a sudden enormous spike in the site traffic. The link spreads rapidly with several users retweeting the same and results in more small traffic spikes.”

Thanks to appengine, I need not worry about scaling.

Webaroo has raised $10 million in a recent round of funding for its product SMSGupshup. Webaroo was founded by IIT Bombay alumnus and serial entrepreneurs Rakesh Mathur (founder of Armedia, Junglee, Stratify) and Beerud Sheth (founder of Elance) in 2004. Webaroo is one of the incubatees of SINE, IIT Bombay.

I was an intern at Webaroo, contributed to the personalization logic of its products, when they were under development.

SMS GupShup is mobile group SMS service that allows users to create mobile communities and broadcast messages to them, similar to twitter, but more like yahoo/google groups for mobile. Currently, GupShup sends SMS messages only to Indian mobile phone numbers. It is being used by over 300,000 publishers sending around 10M messages per day. SMS to mobile devices is one of the major costs for Webaroo.

Webaroo started with their product for offline browsing, which didn’t strike the market well, even after a couple of years of the launch. With Google gears competition, its life ahead, is even tougher. I think one of the reasons why webaroo couldn’t pick up is due to the focus on caching static content, while google gears, powered easy caching and sync of dynamic content. Remember The Milk and Google reader are my favorite apps which I use over google gears.

Earlier, Rakesh Mathur with other founders, sold Junglee to Amazon in 1998 for about $250 million. Junglee had Stanford professor Jeff Ullman on their board, who was also advising Google folks that time. Interestingly, after making a good money, Rakesh Mathur & team approached their friends Larry Page and Sergey Brin to acquire Google. But the deal could not happen as Sergey was looking for a ten digit price.

I have been damn lazy to write this post, but recently after reading a RWW post, 11 Search Trends That May Disrupt Google, I decided to gather my thoughts here.

Adding to the RWW post, let me try to bring up some minus and plus of todays search era ruled by Google & partly Yahoo/MS.

Things which are still not touched efficiently by the popular search engines:

• Natural language Processing:
  • We would like all questions like “Which is the world’s tallest mountain peak?” to be answered on search: Google/Yahoo could not answer it, but to my surprise ask.com did it! Still, we have to wait for a breakthrough.
  • Ignoring stop words, doing word stemming, etc. can really change the meaning significantly. For eg, searching Apples on google, returns results mainly for Apple Inc.
• Multi-lingual search: Web being driven with focus on US market, problems of the rest of the world (especially eastern world) do not really get sufficient attention. Today’s web search experience does not have multi-lingual features!
  I spent couple of years during my masters at Media Lab Asia, IIT Bombay, under Prof. Krithi, with people working on Multi-lingual search for project aaqua.org. Multi-lingual search works pretty good here. Try searching “onion” or “कांदा“, you get identical results . Such a search experience on the whole web, will be awesome!
• Treatment of Symantec data: Lot of standard formats have emerged like RDFs, microformat, RSS, etc, but still they are treated in almost same way as other web pages.
• Personalization & Data mining: There are a few signs of google personalizing the results. But, nothing significant yet!
• Multimedia search: None of the search engines is doing a great job here, which is attributed to complex and computationally expensive image processing. But, pretty significant research is up for the same in Google, Yahoo and Microsoft. A recent publication in WWW08 from googlers suggested a concept ImageRank, similar to PageRank which can actually work well.

Some of the cool innovations in todays search:

• Improved UI/visualizations: UI innovations are the most prominent amongst all. Here are a few set of examples:
  • Yahoo Search: “Get Smart“

    

    
    

  • Google: “Get Smart, Bangalore“
    
    

  

  
  

  • Get Smart, Bangalore on Guruji.com, India focussed search engine

    
    

  

  
  

  • Searching a name on Yahoo with linkedin enhancement (searchmonkey)

  

  
  

  • MS Dhoni on Yahoo India
    

    

    
    

• Openness/APIs: Google/Yahoo have been pretty open in terms of providing search APIs, applications, etc. Want to experience google search in a terminal: Try out www.goosh.org
• Specialized searches like: local/maps: Local/maps and other focussed searches like publication search, patent search, etc. are doing pretty good. Directions are now available in India also with Yahoo Maps the only provider

What else can be tried on search?

There are couple of things things which I think can work for search, but we need to overcome spam problems for these:

• WikiSearch: Allow users to tag/rank search results. Something like digg/delicious for keywords…
• Push based update notification model: Search results are not uptodate. Even for popular pages they lag by few days. Introduce a push based model, something like blog.gs, it can help?

Update: Nov 20: Google has released SearchWiki, my first suggestion/prediction comes true

Disclaimer: All opinions are solely mine and and do not necessarily reflect the opinions of my employer.

Yesterday, a friend of mine was looking for a way to download original photos from one of his sets on flickr. Since we could not find any flickr downloader which can run on Linux, (for windows you can try : http://flickrslideshow.fateback.com/), I quickly wrote a script using flickr APIS. The sed and awk power made it v. easy

Below is the script:

SET=”<Set – id >” # Enter SET ID here from which photos have to be downloaded, for eg. SET=”72157604130281022″
APPKEY=”" #Your APP key here, get one from http://www.flickr.com/services/api/keys/
curl “http://api.flickr.com/services/rest/?method=flickr.photosets.getPhotos&
api_key=$APPKEY&photoset_id=$SET&extras=original_format&per_page=500″ | sed ‘~s/title=”[a-zA-Z0-9_ ?(.]*//g’ | awk ‘/id=/ {print “http://farm3.static.flickr.com/”$4″/”$2″_”$8″_o.jpg” }’ | sed ‘~s/\(server=\|originalsecret=\|id=\|”\)//g’ > p
wget -i p
rm p

You can also download this script by clicking here

Update: Script does not work for video download.

Early this week, Yahoo acquired IndexTools, an analytics company. IndexTools offers tools for monitoring and analyzing websites.

The interesting thing, which happened the very next day (10th April) after the acquisition, was an email send by Google Analytics team to all (or many) analytics users notifying about the benchmarking feature, which was launched on March 5th!

The email stated:

“We are writing to let you know about a change in our service offerings. If you have logged into your account recently, you may have noticed that you can now choose to share your Google Analytics data. … We’re also happy to announce industry benchmarking as the first new feature available …. Benchmarking lets you compare your metrics against industry verticals….”

I feel the email, on the next day of acquisition was not a coincidence, rather Google wanted to try its best to avoid any chunk of users migrating to Yahoo/IndexTools. If Google just had to notify the analytics users about the service change, they must have done this a month back.