../[ dariusz grabka ]\..

If you’re interested in the state of the art of large domain (internet) image search, then undoubtedly Google comes up over and over again.

Google Image Search, with its simple interface and reasonable results, is the de-facto consumer-grade image search engine. Offerings from competitors are actually a little more feature rich, especially the MSN Live Image Search, but don’t resonate as loudly in academia or popular usage.

As an example, compare search results for “red corvette” from the big three: Google, MSN, Yahoo. MSN nails the exploratory task: no-refresh scrolling, quick access to filters such as “photos”, “black and white”, and image size options that feel a little more usable and natural than Googles. Yahoo! attempts some categorising and support for ontologies in their interface; while not perfect, it’s a direction highly praised in cutting-edge research.

Focus on a subset

Despite the simple Image Search interface, Google has been very active in image search research, particularly in hopes of making their results more relevant.

First off, there’s the neat little trick that allows you to search for faces.

In general, the goal is to reach a higher level of result-relevancy. In order to do that, a service need sto analyse images more deeply than by just reading the text that surrounds pictures. Image analysis that involves a computer “seeing” an image and extracting features and content is very expensive in terms of computation. So in the interest of progress, despite the heavy computing cost, the New York Times reports how Google plans to roll out some of their image analysis features on a subset of internet images, rather than try to analyse all the images on the whole of the internet. Google is focusing on the top several thousand topics of interest based on query popularity, and popularity in online shopping (iPods, Wiis, Air Force Ones, etc.)

This concept of focusing on a subset of internet images related to shopping isn’t new: like.com has been in the process of launching and relaunching (that’s a link to the CEO’s blog) its visual search service since early 2007. Their service helps you shop the way you do in real life - visually, with visual feedback and comparison of how things actually look compared to similar items in similar price brackets.

Community Tagging

While image analysis and retrieval based on image features is the future, in the mean time we have tags. Tags are human-contributed textual descriptions of images. There are a variety of problems with tags (inconsitencies, incompleteness, subjectivity, etc.), but their presence reduces the image search problem to a far more manageable text search problem.

With this in mind, Google has quietly launched the Google Image Labeler, a game in which two randomly matched people tag an image co-operatively, and score points. Using a games to entice people to contribute annotation information is cool, but it appears to suffer from over-simplification: a picture of a migrating albatross is far more likely to be tagged as “bird” rather than as “transatlantic migration” or “albatross.” Though I’ve read some promising research that focuses on developing better tools that aid in annotation of images, which will hopefully lead to tags that are more complete, categorised, and community edited.

Leveraging Recommendations

Google has done quite a bit of development and research on the topic of recommendations, especially recommendations related to their Google News service.

I can’t help but think of the possibilities of using recommendation algorithms enhance image search. A service can push new image content to the forefront during queries, inter-mixed with the most relevant results, and allow people searching to annotate the recommended images. This would quickly collect information on how relevant that new image is to that query. Users could dismiss the image as irrelevant by never clicking on it, despite how often it appears in the first row of the result set.

That kind of information would provide a feedback loop for the developers of the relevancy algorithms, and potentially also allow for very personal image results: if those recommendations and your responses to them were kept around and associated with your Google account, future results could take advantage of your previous feedback.

There is much progress to be made in image search, and based on how often I come across interesting and relevant Google research in the field, it’s obvious that Google will continue to be a leader as image search matures.