Earlier this week Yandex, the Russian search giant, released a new search algorithm inspired by neural networks to better understand users’ intent and handle long-tail queries. The new algorithm is named after Korolyov, the lead Soviet rocket engineer and spacecraft designer in the 1950s and 1960s.
‘Korolyov’ significantly improves upon its predecessor, ‘Palekh,’ the neural network-inspired algorithm Yandex released in late 2016. ‘Korolyov’ matches the meaning of a search query to all of the content of a web page, whereas ‘Palekh’ only looked at headlines. Yandex also applies ‘Korolyov’ to a far greater number of the top relevant pages than ‘Palekh’ – 200,000 vs 150 per search query.
Like all modern AI-based systems, ‘Korolyov’ improves itself with the more data it gets from queries. As the largest search engine in Russia, Yandex users are helping the algorithms to continuously provide a high-quality search experience.
How does Korolyov work?
- To better understand users’ intent, the Yandex search team trained neural networks with information from billions of search queries and crawled pages which it reduced into numbers.
- Korolyov creates a semantic map: it assessesthe proximity of the numbers that represent the meanings of words on web pages in its index and then matches those to the numbers that represent the search queries.
- This algorithm then feeds into MatrixNet, Yandex’s proprietary machine learning ranking algorithm, which considers results from ‘Korolyov’ and a number of other ranking factors before search results are returned to the user.
How do Yandex users benefit from this new algorithm?
The algorithm is meant to better understand user intent and handle long-tail queries. The search engine will have an improved understanding of what users mean, allowing it to process queries with missing information better. Here are a few examples: a few ingredients to a recipe the user doesn’t know the name of or a question on who proved that the Earth is round. The more searches the engine receives, the better it gets, as it learns every second from millions of searches.
This article is a modified version of a story which first appeared on Russian Search Marketing, a syndication partner of East-West Digital News.
