The service recognizes spoken language and translates it into text so that a person with a hearing impairment can hear what is being said to him. Then the user can type the response text, and the program will voice it, that is, turn it into oral speech for the interlocutor. This allows you to calmly lead with others, without using tension, for example, paper with a pen.
Video conference with a volunteer
At any time, a hearing impaired person has the opportunity to call a Russian sign language interpreter to solve everyday problems, consult on important issues, as well as in emergency situations.
Virtual sign language interpretation
The virtual sign language interpreter recognizes Russian sign language and translates it into text or speech. It works with the help of artificial intelligence, which was trained on the basis of the collected dataset of more than 50 thousand gestures.
Virtual sign-language interpreter
We create our own character who will become an assistant to people with disabilities
This is what a virtual translation of text into sign language will look like
How is the virtual sign language interpretation performed?
A team of sign language interpreters records thousands of gestures in the studio of our partners Soyuzmultfilm.
With the help of "motion capture" technology, these gestures become the basis for training artificial intelligence (that is, a virtual character).
How does "motion capture" work?
A sign language interpreter in a special suit demonstrates gestures in RSL. They are transferred to the 3D model and subsequently assigned to the virtual character.
Sergey Lobankov, director and actor of computer animation in motion capture technology, adjusts the helmet of a sign language interpreter to assign articulation to the virtual character of the "SURDO-HELP"
"SURDO-HELP" will help people with hearing disabilities to communicate in the modern world
The first platform for the deaf in the Russian Federation, which will be presented on the web portal and available in the mobile application
For sign language speakers, there are two sign language options to choose from - a real or virtual sign language interpreter
For the deaf, who do not speak sign language well, there is an opportunity to use the service of translating printed text into speech and vice versa. The text is entered on any device and voiced. Or the speech is translated into text on the screen of the device.
Wide range of applications
Personal communication in any environment and any situation
In public places without obstacles: multifunctional centers, clinics, hospitals, pharmacies, shopping centers, libraries, exhibitions, museums
Current stage of development — designing
We are currently in the process of collecting the dataset. Sign language interpreters, deaf BMSTU students and the hard of hearing involved are already filming gesture videos for the project.
With the help of "motion capture" we record gestures and transfer them to our virtual sign language interpreter
Website, Web App
«SURDO-HELP» will be available on all platforms
As part of the project, we are developing a library of video books for people with hearing impairments.
An example can be seen in the video on the right. A professional sign language interpreter translated the story of N. N. Nosov into Russian sign language.
In the future, we are planning to make a whole collection of videos with works of Russian and foreign literature from different directions.