Smart Glasses Detect Eye Gaze and Expressions with Ultrasound

Researchers in the US have made groundbreaking advancements in the field of wearable technology by developing two innovative devices that utilize ultrasound sensing to track a person's gaze and facial expressions. This cutting-edge technology, created at Cornell University, is compact enough to be integrated into commercial smart glasses, virtual reality (VR), or augmented reality (AR) headsets, while consuming significantly less power compared to traditional camera-based tools.

Both devices, named GazeTrak and EyeEcho, employ speakers and microphones mounted on an eyeglass frame to emit ultrasound waves towards the face and capture the reflected signals resulting from facial and eye movements. GazeTrak stands out as the first eye-tracking system that relies on acoustic signals, while EyeEcho is the pioneering system capable of continuously and accurately detecting facial expressions and replicating them through an avatar in real time.

The remarkable efficiency of these devices is evident in their extended battery life, lasting several hours on smart glasses and over a day on a VR headset. Beyond their applications in consumer technology, these technologies hold promise in the medical field, particularly in the diagnosis and monitoring of neurodegenerative diseases such as Alzheimer's and Parkinson's. Patients with these conditions often exhibit abnormal eye movements and reduced facial expressions, making this technology invaluable for tracking disease progression remotely from a patient's home.

"It's small, it's cheap, and super low-powered, so you can wear it on smart glasses every day without draining your battery," remarked Cheng Zhang, the director of the Smart Computer Interfaces for Future Interactions (SciFi) Lab responsible for developing these groundbreaking devices. In a VR setting, researcher Ke Li emphasized the importance of accurately recreating facial expressions and gaze movements to enhance user interactions.

By utilizing ultrasound technology instead of video-based tracking, the devices address privacy concerns associated with camera systems. Ke Li highlighted the growing importance of privacy as VR/AR headsets become more compact and ubiquitous, emphasizing the compatibility of these technologies with lightweight, smart AR glasses. François Guimbretière, a professor of information science at Cornell, underscored the potential of these devices in mitigating privacy risks associated with wearable technology.