Eshara, an AI-powered Arabic sign language translator using avatars, has won the 2024 UAE James Dyson Award.
Developed by three talented engineering students from American University of Sharjah (AUS) – Salma Shaarawi, Mohamed Eid and Aley Eshra – Eshara addresses a critical gap in the Middle East: the lack of Arabic sign language translation services.
In line with the UAE's vision to improve integration for people of determination, Eshara bridges the divide between hard-of-hearing individuals and the public in Arabic-speaking societies. By focusing on Arabic sign language, the platform offers a unique solution, facilitating communication where traditional translation services fall short.
For the past seven years, the James Dyson Award has invited engineering and design undergraduates and recent graduates in the UAE to "design something that solves a problem”. The instructions are intentionally broad and open-ended, encouraging students to use innovative technology to address major global issues.
The Problem
The winning team of three students recognised the significant communication barrier hard-of-hearing individuals face, and identified a crucial gap: Traditional translation services often lack sign language support in Arabic, despite it being one of the most commonly spoken languages locally. Driven by the desire to create an inclusive world where everyone can communicate effortlessly, they developed the innovative Arabic sign language translation system Eshara.
Team Eshara’s inspiration came from a pivotal conversation with the African Sign Languages Resource Center (ASLRC). Observing the everyday struggles of hard-of-hearing individuals in interacting with society, along with the scarcity of accessible solutions, drove them to harness technology to bridge the communication divide. The trio aims to foster an inclusive world where effortless interaction is possible for everyone regardless of hearing ability.
The Invention
The students began conducting vast amounts of research and development to bring the project to life, creating multiple prototypes to improve accuracy and reliability. They filmed and collated a dataset of 62,000 videos of themselves signing to train their deep learning model effectively. Through this, they were able to develop an innovative sign language translation system integrating both sign-to-text and text-to-sign capabilities.
After testing, the students created a web application that allows users to sign gestures via their phone camera, which are then translated into text in real time. This makes the technology widely accessible through standard web browsers without requiring specialised equipment. Likewise, the team focused on the reverse translation process, creating custom avatars for converting text messages back into sign language. These avatars, meticulously animated to resemble the students themselves, add a unique touch of personalisation to the application.
By combining real-time sign language recognition with animated avatar-based sign language output, this comprehensive system bridges the communication gap between sign language users and non-users, potentially revolutionising accessibility in digital communication.
Salma Shaarawi said: "Winning the 2024 UAE James Dyson Award has been amazing, as it not only spotlights our innovation but also sheds light on the critical problem. Whilst working on the innovation, we were faced with a lot of challenges and doubts, but winning the award has reassured us that there is demand for and interest in such an invention. This has further fuelled our passion to refine Eshara and explore new technologies that break down communication barriers.”
Eshara takes home £5,000 as the James Dyson Award national prize to support its next steps in development and commercialisation. The team plans to use the funds to create a mobile app, expanding their system to support additional Arabic dialects and sign languages beyond Arabic to benefit the global hard-of-hearing community. This expansion will involve collaborating with sign language experts and gathering new datasets for various sign languages.
Dr. Eng. Suaad Al Shamsi, who has been a judge of James Dyson Award in the UAE for several years, said: “Eshara team's innovative approach to enhancing communication for the hard-of-hearing community exemplifies the kind of inventions we are excited to see when judging the James Dyson Award. They tackled a real-world problem that others have ignored, and used advanced technology to create an inclusive solution that is not only impressive but essential in today's world. I look forward to observing how they will progress and witnessing their innovation in action.”
Eshara will advance to the next stage of the James Dyson Award, with the international Top 20 shortlist, selected by Dyson engineers, to be announced on 16th October, and the global winners, chosen by James Dyson, on 13th November.
Along with crowning Eshara the winners of the 2024 James Dyson Award in the UAE, the judges announced DewHopper and OcularSky as the runners-up.
UAE Runners-up
Problem: The MENA region is significantly impacted by water scarcity, which adversely affects its agricultural productivity, food security and livelihoods.
Solution: The DewHopper technology operates based on the principle of condensing air humidity through enhanced geothermal cooling. In arid regions, the soil mainly consists of sands, which provides the advantage of maintaining lower temperatures at deeper levels.
How the system works: DewHopper passes the air through a series of ducts leading underground. Through an electronic cooling effect, it conducts heat into the ground. Placed within the ducts are adsorbents to collect water. This harvested water can be used for irrigation or other purposes. The system not only offers a local solution to water scarcity but also provides a sustainable method for water regeneration in arid lands near coastal areas.
Problem: Lack of accessibility and accurate eye disease diagnosis.
Solution: OcularSky operates as an application on any smartphone, connected to the cloud, where pre-trained AI models diagnose the captured images of the eye. To use it, the healthcare practitioner places the smartphone on a 3D-printed gadget equipped with a stabilising tube to focus on the ocular region. The smartphone's camera and flashlight are aligned with the tube. Once properly positioned, the device is placed about five centimetres away from the patient’s eye. The healthcare practitioner then captures the image, and the application automatically crops it and uploads it to the cloud for instant diagnosis. For a clearer image, it is recommended that the healthcare practitioner uses pupil dilation eye drops as usual.
