Meet Priyanjali Gupta, the student behind AI sign language translation

Priyanjali Gupta used her knowledge to create a deep learning-based AI model trained to recognize six American Sign Language signs and translate them into English in real time

Priyanjali Gupta from Vellore Institute of Technology is on the last year of his undergraduate education in computer science data science but has already scored significant AI advances with all the great work he was doing: inclusionary innovations.

As one of such, most of his break-through innovated works include development of translating ASL with an accuracy to English while still making a real time translation: thus bridging communication for those deaf or hearing challenged and an unknown user of their sign.

She was challenged by her mother to do something practical and impactful using the engineering skills. That is when she thought of how her expertise can be used to help the deaf community.

Trying to train the model on both images and videos

After a year of intense work, Gupta developed her deep learning-based model by using the object detection API from TensorFlow and transferring the learning with a pre-trained SSD MobileNet model.

Six simple ASL basic signs- “Hello“, “I love you“, “Thank you“, “Please, YES and NO“-can all be interpreted to this system through just holding the webcam attached on to the computer, at what makes the process real in seconds.

Despite that the project picked up significantly with above 58,000 LinkedIn likes, Gupta believes that it has also been an arduous assignment to be dealing with.

In the future, Gupta wants to enhance the model by training the system not on single frames but on videos, using Long-Short Term Memory networks to increase the accuracy and allow the recognition of sign sequences while in motion. This could be a major leap in real-time detection of sign languages.

Gupta’s device instantly gained widespread publicity throughout the tech world, as it epitomizes how artificial intelligence can open doors toward inclusivity. Since AI is often damaging because it eliminates jobs, this example—and many more—actually gives proof that AI can eliminate barriers and have quite an impact on society.

Source: GitHub