Russian-speaking engineers seem to have some special skills, or secret sauce, to develop face recognition and related AR technologies, if judging by the number of startups with roots in Eastern Europe that have made the news over the past few years. Emerging from Russia, Belarus and Ukraine in particular, they have topped international contests, received sizable investments, struck up partnerships or even been acquired by international tech giants.
In this article which is part of a comprehensive report on startup innovation in Central and Eastern Europe, EWDN chief editor Adrien Henni and writer Vladimir Kozlov review some of the most important cases, and offer possible explanations for this local technological trend.
In September 2015, the Ukrainian startup community and technology media went into a frenzy after the announcement of Snapchat acquiring Looksery, an Odessa-based image processing startup. The deal, which reportedly cost the US company $150 million, marked the most spectacular startup success story in the country so far.
Looksery had created an app to change your face in real time, while having a video chat with someone (1). The company was founded by Victor Shaburov, a serial entrepreneur and active supporter of all kinds of programming contests for talented youth.
In 2013, Shaburov met Yuri Monastyrshyn, a fourth-year student in Odessa, Southern Ukraine, who became the COO of his new project, Looksery. The product team was hired in Sochi, Russia, as the two countries still had peaceful relations. After the acquisition by Snapchat, the Looksery team relocated to Silicon Valley to work together with their new colleagues.
Eastern European tech for Silicon Valley giants
Looksery was just the first of a series of success stories involving face recognition startups from Eastern Europe.
Six months later, in early 2016, Facebook bought Masquerade, a Minsk-based developer of another video filter app, MSQRD, which at one point was one of the most popular apps in the App Store. The US giant characterised it as “a fantastic app with world-class imaging technology for video.”
In the summer of 2017, another Belarusian company, AIMatter, was acquired by Google, just months after raising $2 million from local investors. Its SDK for real-time photo and video editing on mobile, dubbed Fabby, was claimed to operate faster than other available solutions. Fabby allows users, for instance, to separate an object from its original background. It was built upon a neural network-based AI platform.
Russian teams are in the running, too. Last year in an unexpected surge, an app made by Alexey Moiseenkov, an employee of the Mail.Ru Group, became a hit in the regional and then global rankings. Dubbed ‘Prisma,’ the app – also based on neural networks – invites its users to add stunning artistic effects to their photos by combining them with works of art. A photo of your cat can look like a Picasso painting, with the cat as the source for content and Picasso artwork as the source for style.
The approach was not new (see DeepArt.io, based in Germany, and Ostagram, a Russian project created by Sergey Morugin), but Prisma’s distinctive advantage was in its speed and availability in mobile format.
In 2017, Prisma launched Sticky, a new app that turns selfies into stickers for sharing in your social feeds. The company is still independent, but recent rumors had it that Facebook and Snapchat had it in their sights for a potential acquisition.
Prisma can transform your photos into artworks modelled on iconic paintings such as Edvard Munch’s’ “The Scream”
The roots of excellence
Mobile computer vision problems, which developers face when developing such technologies, are not just complicated mathematics, it is also a matter of performance as the computing power of most current mobile devices is limited.
“Nobody needs an AR app which will work only on iPhone X. There are billions of devices in the world and very small percentage of them are expensive devices. If you want to make a mass market mobile application it should work on an iPhone 5s and give 30 frames per second, which is a challenge in its own right,” notes Viktor Prokopenya, a serial entrepreneur with Belarusian roots who recently launched, jointly with a Russian billionaire, a $100 million investment programme for AI startups.
The talent of East European engineers in solving such an array of technological challenges has something to do with the education traditions that go back to the Soviet past. After all, for all its moral and economic failures, communism did demonstrate a rare capacity to raise generations of brilliant scientists and engineers in several domains.
“In the field of machine intelligence, the Markov chains, Markov random field and Markov models, are the very basis of deep learning, while the Hidden Markov Model (by Ruslan Stratonovich) is the main model for speech recognition. Alexey Ivakhnenko, who was a prominent a Soviet and Ukrainian mathematician, is still referred to as the father of Deep Learning,” notes Alexander (Sasha) Galitsky, a prominent Russo-Ukrainian investor who invests globally through the Almaz Capital fund.
While the Soviet regime collapsed, the local mathematics education system was preserved to a large extent. “The transformation went gradually, in an evolutionary way,” says Alexander Kurbatsky, professor of the computer programming department of the Belarusian State University in Minsk.
“Thus, good math education created a nurturing environment for today’s digital transformation, which goes beyond the territory of the former Soviet Union,” he adds, in reference to the dozens of thousands IT engineers and entrepreneurs who have moved to Silicon Valley, Berlin, Israel or even Asia over the past 20 years.
Next-gen startups with “empathic cameras”
A considerable number of talents still live and work in the region, however, with new players still springing up in the fields of face recognition and related AR technologies.
After the acquisition by Facebook, two former MSQRD team members, Andrey Yanchurevich and Dmitriy Dorin, launched AR Squad, a startup dedicated to “next-gen AR content.” “Funny masks were a part of MSQRD’s viral strategy. Today we are developing a new level of interaction between users and augmented reality,” they say. The young company has hired MSQRD’s whole art team as well as additional strong experts from this field. It serves clients from the USA, Russia and Belarus.
Banuba, a startup born in Belarus but operating worldwide from its Hong Kong and Cyprus offices, recently developed an AR mobile software development kit for app developers and publishers worldwide. To a large extent, this technology relies on AI algorithms to recognise people’s faces and bodies, understand their emotions, facial expressions, postures and gestures, and estimate race, age and gender.
The solution can be used in a variety of apps – from content suggestions to improve your mood, to very practical recommendations, for example, which hairstyle suits you best, or how you will look like in future. According to its CEO Vadim Nekhai, Banuba’s advantages lie in its “capability of mixing technologies on the same device, and optimising them for ground-breaking performance results.”
In early 2017, Banuba secured $5 million from Prokopenya and his investment partners. Following this, the startup launched a “technology-for-equity” program to enroll app developers and publishers across the world.
The first such agreement was signed with Inventain, another startup from Belarus, to develop AR-based mobile games. A special purpose vehicle (SPV), dubbed ‘Camera First,’ was created with a €1 million capital injection from Banuba to develop the games.
How Banuba technology maps and analyses a smiling face
The implications of mobile AI and AR technologies are much deeper, though, than making entertaining picture and video features. “Such technologies can be leveraged by virtually every kind of app,” believes Prokopenya, who invested in Banuba.
Prokopenya envisions a world where “any app can characterise its user through the camera: male or female, age, ethnicity, level of stress, etc. Then content can be optimised for maximum performance, based on such parameters.”
Thus, fitness apps can see how much you’ve gained or lost by your face, while games can adapt their pricing models to the type of user. Rather than an intrusion into privacy, the investor sees something “empathic” in such applications: “They will understand the kind of emotions your face is showing, and adjust their content correspondingly.”
In the commercial field, a face recognition software has been developed by Russian startup VisionLabs. The solution, dubbed ‘Face_IS,’ allows retailers to make personalised offers to customers whose faces have been recognised. The technology may also be applied to ad personalisation, user identification in Internet-of-Things projects, medicine, transportation, virtual and augmented reality.
Another VisionLabs solution, ‘Luna’, allows businesses to “verify and identify customers instantly” based on photo or video images, thanks to a “unique quality and performance pattern recognition technology.”
This award-winning technology has attracted the attention of Google and Facebook. These companies have supported VisionLabs’ efforts to develop professional communities in the fields of computer vision and neural networks.
The good and the bad
Another promising, but controversial technology from the region is FindFace. It was launched in 2015 by Moscow startup NTechLab, which presented it as “the world’s most accurate facial recognition technology for face detection, verification and identification.”
FindFace is built on deep learning and a neural network-based architecture. In November 2015, NTechLab won the MegaFace Benchmark, a world championship in face recognition held in the US. The challenge was to recognise the largest number of people in a database of more than a million photos. The Russian startup bypassed more than 100 competitors, including Google with its programme FaceNet.
The app has made the news in Russia, where social media users use it to put names to faces using only a picture. FindFace will match the picture with people’s pages on Vkontakte, the leading local social network. The app is mainly being used for dating purposes, but it has also allowed police to identify criminals, and ill-intentioned users to harass young women. Nevertheless, the startup recently raised $1.5 million in a round led by Impulse VC, a venture fund that is reportedly affiliated with Russian billionaire Roman Abramovich.
Grigory Bakunov, director of technology distribution at Yandex, thought of an antidote to such intrusive technologies. Taking some time out from his day job, he developed an “anti-facial recognition algorithm” to conceal people’s identities with the help of makeup.
“The service was able to offer futuristic makeup that could trick smart cameras with just a few facial lines,” he said in a media interview. The project proved short lived, however, because Bakunov realised that it would now be possible to deceive banks and police.
“That’s why we decided we shouldn’t launch it on the market. The chance that someone might use it for nefarious purposes was too high,” Bakunov said.
Grigory Bakunov, the Yandex engineer who developed an “anti-facial recognition” solution
(1) Looksery’s app allows users to video chat with other people using a face filter that could make them look, for instance, like a panda or a grumpy cat. Alternatively, users can use the app to perfect their faces, removing blemishes, smoothing their skin tone or changing the colour of their eyes. The idea of “selfie-editing” wasn’t radically new, but Looksery was the first to offer an editing function for live videos.
This article was republished in VentureBeat, a syndication partner of EWDN.
