"Sensor-Tech" with "Soyuzmultfilm" completed filming of gestures of a virtual assistant for deaf people

"Sensor-Tech" Lab together with "Soyuzmultfilm" studio are creating a unique 3D character animated on the basis of gestures of a real specialist. He will become a virtual sign language interpreter in the digital service for the deaf and hearing impaired "SURDO-HELP". Working on such a project is the first experience for both the Laboratory and the film studio. We developed algorithms of actions and selected the necessary technologies from scratch.

Currently, the Lab team has already started processing the recorded gestures, cleaning them from unnecessary noise, improving articulation and bringing each gesture to the standard. Next, we plan to test gesture recognition with deaf users.

"Our goal is to make the sign language interpreter as realistic as possible. To do this, we record and add more and more gestures to the database every month. In addition to filming in the studio, a huge amount of work is also done inside the team: every gesture is processed by programmers and 3D designers", – said Denis Kuleshov, director of "Sensor-Tech" Lab.

"SURDO-HELP" is a unique online assistant for the deaf and hearing impaired. The main features of the service are speech-to-text translation, video communication with a sign language interpreter and virtual sign language interpreting. The platform will be available in 2024 and will become one of the most convenient and large-scale Russian services for hearing-impaired users.

SURDO-help is being developed with the support of RFRIT.