Microsonar Technology Enables Real-Time Fingerspelling Tracking

A groundbreaking technology called SpellRing has been developed by a research team led by Cornell University. SpellRing combines a speaker, microphone, mini gyroscope, and clever machine learning to enable real-time American Sign Language (ASL) communication. This innovative device has the potential to revolutionize ASL translation by continuously tracking entire signed words and sentences, making it a game-changer in the field of assistive technology.

In its current form, SpellRing allows users to input text into computers or smartphones through fingerspelling, a technique used in ASL to spell out words that do not have corresponding signs, such as proper nouns, names, and technical terms. The device, believed to be the first of its kind, addresses the limitations of existing technologies that recognize fingerspelling in ASL, which are often bulky and impractical for everyday use.

Hyunchul Lim, a doctoral student in information science and a key member of the research team, highlighted the importance of developing a compact and efficient solution for capturing the intricate finger movements in ASL. The SpellRing is worn on the thumb and utilizes a combination of inaudible sound waves, a mini gyroscope, and deep-learning algorithms to accurately interpret ASL fingerspelled letters in real time.

The device, which is housed in a 3D-printed ring and casing no larger than a standard U.S. quarter, represents a significant advancement in assistive technology for the deaf and hard-of-hearing community. By leveraging machine learning and innovative sensor technology, SpellRing offers a practical and user-friendly solution for enhancing communication and accessibility for ASL users.

SpellRing builds upon the success of a previous project called Ring-a-Pose developed by the Smart Computer Interfaces for Future Interactions (SciFi) Lab at Cornell. François Guimbretière, a professor of information science and co-author of the research, emphasized the transformative potential of machine learning in enabling new ways of interacting with the world, as demonstrated by projects like SpellRing.

Lim, who underwent ASL training as part of the SpellRing research, highlighted the complexity of ASL as a visual language that encompasses not only hand movements but also facial expressions, body gestures, and head movements. This holistic approach to ASL communication underscores the importance of developing inclusive and versatile technologies like SpellRing to support diverse communication needs within the deaf and hard-of-hearing community.

The research behind SpellRing was made possible through funding from the National Science Foundation, underscoring the significance of this innovative project in advancing assistive technology and accessibility for individuals who rely on ASL for communication.