Tuesday, October 17, 2006

Search Flickr by Sketch with Retrievr

Retrievr is probably one of the most unique search engines out there. I think almost all other search engines rely on text input to conduct searches, but Retrievr allows you to search Flickr images by drawing a sketch. You can choose the thickness of your brush and color. Based on some tests, color seems to influence the search results more than shape. Drawing the shape of a car probably won't return pictures of cars, for example. The programmer Christian Langreiter describes how it works:

Retrievr creates and stores a compact representation of each photo in its database. The system pulls only the most important features -- broad shapes, blocks of color and spatial relationships between different colored areas -- out of detailed images to create shorthand approximations of every photo. (The storage mechanism extracts the 120 "strongest" features from an image to create something called a "wavelet transform," which contains much less data than the photo itself and facilitates lightning-fast searches.)
These photographic abbreviations are compared with the user's sketches, and Retrievr suggests matches. Mr. Langreiter advises Retrievr artists "to capture the grand scheme of things" to achieve the best results.

To get the best results, Langreiter has this recommendation

"You have to get the colors right and the location," he says. "The details are not that important in the algorithm." Mr. Langreiter urges users to remove as much detail as possible from drawings and to avoid working in a line-art style all together; the key is well placed blobs of color.