Tinkoff, the Russian digital banking leader, has integrated its AI-powered voice assistant ‘Oleg’ into Clubhouse — claiming to be the first one in the world to do so.
The capabilities of the voice assistant as a “a full-fledged Clubhouse user” are described as follows:
- ‘Oleg’ can use its text-to-speech and speech-to-text capabilities (Tinkoff VoiceKit) in real time to “help room creators communicate and moderate discussions in Clubhouse;”
- He can enter rooms, transcribe speech in real time, and stream the text in his own Clubhouse Telegram channel;
- He can also moderate Clubhouse rooms, voice questions to speakers, remind users about time limits, regulations, etc. (1)
Making his debut in Clubhouse on March 11, ‘Oleg’ converted speech-to-text and streamed a conference-call held in the Tinkoff Investments room. CEO Oliver Hughes and other senior executives discussed Tinkoff Group’s financial performance and record net profit in 2020.
“Our voice assistant team is currently experimenting with various user scenarios in Clubhouse to determine how room creators or listeners can benefit from our technologies,” said Pavel Kalaidin, Director of Artificial Intelligence at Tinkoff.
“The assistant can also come in handy when listeners are unable to voice a question to speakers,” he added, citing the examples of a noisy environment or unwillingness to interrupt the speakers: “For such cases, we are designing an interface through which users can forward their questions to Oleg’s Telegram chat. ‘Oleg’ will then voice the question with perfect pronunciation, keeping the user anonymous, if necessary.”
A challenge in group speech recognition is the summarization of information, Kaladin conceded. “Interjections, fillers and incoherent speech make reading even a good transcript difficult. For that reason, we are looking into ways of processing the text and capturing the gist of what is said to create a shorter and more readable transcript.”
Kalaidin said his company is “open to working with Clubhouse communities” to have ‘Oleg’ serve content makers and listeners.
‘Oleg’ was announced by Tinkoff founder Oleg Tinkov in late 2018. In January 2021, Tinkoff Mobile launched a 2.0 version of the voice assistant, enabling customers to create their own mobile concierge with custom features.
Another Russian-made, cutting-edge voice assistance is Alice, launched by Yandex in 2017. The search leader also offers a professional version of its service, ‘SpeechKit Pro,’ to recognize or voice texts in several languages.
(1) Tinkoff VoiceKit can also create robots to automate call centers, speed up the production of audiobooks and voice-overs as well as video editing, create applications for people with disabilities, facilitate SEO activities, and much more. This kit relies on deep neural network models which were developed by Tinkoff in recent years for speech recognition and synthesis. These developments came as part of the company’s ‘AI First’ strategy.
