A person who is blind can detect a bowl is in front of them while wearing a pair of smart glasses
LIL DEVERELL/UNIVERSITY OF TECHNOLOGY SYDNEY, AUSTRALIA, CC-BY 4.0
Smart glasses that generate sound when an object enters their field of view could be a valuable tool for individuals who are blind, assisting them in locating specific items.
Some blind individuals have the ability to navigate using echolocation, which involves emitting a sound that bounces off objects in their surroundings, allowing them to gather information about their environment.
Taking inspiration from this ability, Howe Zhu and his colleagues at the University of Technology Sydney in Australia set out to develop glasses that provide information about nearby objects to blind individuals. “We wanted to enhance the navigation experience for blind people,” says Zhu.
The team started by using augmented-reality glasses with two front-facing cameras and built-in speakers. They then created a smartphone app that utilized deep learning to process the visual information captured by the glasses and identify four objects: a bowl, cup, book, and bottle.
A unique sound was assigned to each object, which played through the glasses’ speakers when the object came into the wearer’s field of view as they looked around a room. For instance, when a book was in view, the wearer heard the sound of a page turning.
To assess the functionality of the glasses, the researchers recruited seven individuals with varying degrees of vision impairment and seven blindfolded individuals with no sight issues. All participants were seated at a table with the four objects placed on top.
When asked to pick up one of the objects, the blind or visually impaired participants were able to do so correctly 81% of the time, compared to a 73% success rate for the blindfolded participants. Zhu explains that “taking away a sense” from a blindfolded individual would naturally lead to a longer adjustment period when using the glasses.
Additionally, the participants who were blind or visually impaired reported no additional cognitive strain while wearing the glasses, according to a questionnaire. This indicates that the glasses were relatively easy to adapt to.
The team’s future plans involve expanding the technology to recognize a wider range of objects and enabling individuals to use the glasses while moving around.
