Natural Language Image Search with Yahoo Boss and Google App Engine

Natural language processing is partly used in text search today, but its use in image search is mostly unexplored. I did a quick hack: askBoss, which retrieves images to questions posed in natural language. askBoss attempts to enhance image results for queries around factual question answering. It uses Yahoo Boss (Search) APIs through Boss Mashup framework and is deployed on Google App Engine.

This hack is an extension of Vik Singh’s qna service, which finds answer using the popular phrases in the top search results for a query. I do image search for the best answers and blend them with the regular image search results. The hack is a basic prototype and natural language image search gets triggered only for questions (queries including who/what/which).

Try askBoss:

Below is a quick comparison of search results obtained by askBoss, Google image search and Y! image search for query: who is batman in the dark knight?

askBoss results: who is batman in the dark knight?

askboss: who is batman in the dark knight

Google image search results: who is batman in the dark knight?

google: who is batman in the dark knight

Yahoo Image search results: who is batman in the dark knight?

yahoo: who is batman in the dark knight

Try askBoss:

With Yahoo Boss APIs and a deployment platform like Google App Engine, building a decent search service is pretty easy. I could finish this hack within a few hours by using Boss Mashup Framework and App Engine. Apart form the qna service, other popular Boss API/app engine integrations include 4hoursearch aka YUIL.

Update 1: askBOSS got covered in TechCrunch and Yahoo Search Blog.

Update 2: Now running on BOSS v2 and code open sourced on github

Share and Enjoy:
  • Digg
  • Facebook
  • StumbleUpon
  • email
  • Twitter
  • Yahoo! Buzz
  • Its really cool, I came to know this really worth visiting, just bookmarked your site.
    The place where fun never ends

  • Thanks for share this information. Great post... i will come next time

  • This is very useful, thanks ;)

  • James88

    Hi, I found your blog on this new directory of WordPress Blogs at I dont know how your blog came up, must have been a typo, i duno. Anyways, I just clicked it and here I am. Your blog looks good. Have a nice day. James.

  • Hi
    Where did you get words for this paper? Is it from your head???

  • Steve

    Very nice Saurabh!

    I tried "who is saurabh?". Page 3 :)

  • vik

    you got techcrunch'ed :)

  • Thanks Ian & Vik.

  • vik

    very nice boss example! we'll try to link promote this in our examples.

  • Very cool. I like the results for "who's the fairest of them all"

blog comments powered by Disqus