Reverse image search – or searching not by words but by visual content – has just become available to users of Yandex.com. This feature works on computer vision technology created by developers at Yandex. We’ve named it “Sibir” (Russian for “Siberia”), from the acronym CBIR, which stands for “content-based image retrieval”.
The technology transforms pictures, submitted by users in the form of search queries, into sets of “visual phrases” – that is, numerical representations of images’ key features. Next, from the billions of pictures available on the internet, the search engine selects images containing the same visual phrases as those in the submitted picture, sifting out all the rest.
This is an important step, because speeding up the process depends on reducing the scope of the search. The computer then compares the position of key features on the images and ranks them according to their similarity to the submitted picture, so that the closest resemblances will be at the top of the list. This entire process – from uploading or selecting an image to search results – takes less than a second.
Last year we announced Sibir in Russia, Kazakhstan, Ukraine and Belarus. Since then, we have been studying how this feature is used and in which situations. In a year we have come to understand that reverse image search is something like a toothache. While we don’t have it, we don’t think about it. But as soon as a tooth starts to ache, we’d sell our soul for a dentist. So while reverse image search is far from being the most in-demand service on the internet, when the need arises it’s acute.
Here are the five most frequent situations in which users need reverse image search:
1. “Are they lying to me with a false image?”
Is that a genuine portrait of the person trying to make friends with you on a social network or dating website? Is it the actual car/flat/bike/kitchenware that a private seller is trying to flog on an online forum or classified advertisements website? Is this really the beach and hotel that the travel agency is sending you to?
2. “What the hell is that?”
Submit an unidentified image as your search query and find one just like it, but this time with a caption describing what is shown. So that you won’t mistake, say, an African purple frog for a mole or a shrew (heaven forbid!).
3. “I need the same image, but in higher resolution or without cropping.”
Picture editors at all sorts of web-based or offline publications, including newspapers, magazines, books and catalogues, or anyone working on a presentation or report, be it professionally or as a student, will appreciate a possibility to find a desired image to exact specifications.
4. “I’m a photographer and I’d like to check whether my copyright has been infringed online.”
Professional photographers or artists can use this search feature to make sure they aren’t missing on royalties.
5. “I saw a cool watch on the internet but the model wasn’t named. I want to use that image to search internet stores and find one selling exactly that model.”
Reverse image search is a godsend to shopacholics. Or to anyone on the lookout for a birthday gift or a wedding present for their favourite blogger posting pictures of objects of their desire on their blog.
Computer vision is a very complex, research- and resource-intense technology. While many companies and scientific centres are researching it and have developed technologies capable of working with small collections of pictures, very few have developed products based on computer vision that can work with tens of billions of images. Before the industry mastered the method of image search using computer vision, search engines found pictures by using the accompanying text that describes the visual content. This had its drawbacks, for example the query “ants” could easily turn up pictures of caterpillars from a nature article on the subject of “ants against caterpillars”.
Our image search is just the first step on what will be a rather difficult path. Yandex today can offer its users pictures that match an uploaded image or have identical fragments, but the day when we’ll be able to tell our users what it is that they have photographed isn’t far away. Even now Yandex’s reverse image search can find not only that exact same picture, but also other images containing the same object.
