A pair of glasses can turn a bottle into a sound you can point your head toward

Turning objects into sounds that sit around the head

The idea behind acoustic touch is simple enough to explain in one breath. Instead of saying that a bottle sits at two o’clock, the glasses play a short sound and place it where the bottle is. That sound becomes a point in space around the head. A book might make the noise of pages turning, while a cup makes the soft knock of being set down on wood.

The listener does not need to decode words; they can turn their head until the sound feels centered. This turns a visual search into an auditory one, with direction carried by where the sound appears. The prototype used augmented-reality glasses that watched a small area in front of the wearer. Its active listening zone stretched about 36.6 degrees from side to side and 20 degrees from top to bottom.

That shape was chosen after four blind or low-vision pilot testers helped define how much of the room should be heard. Inside that zone, each detected object produced its own repeated icon. The repetition was fastest when the object sat near the center of view and slower toward the edges. The zone had ten layers, with a twenty-millisecond delay added as an object moved outward.

At the outer edge, repeated sounds were spaced by about one hundred milliseconds. This rhythm gave the ear a target: when the sound felt centered, the wearer was close to its direction. Distance came from the glasses’ two front cameras, which compared their views and placed each item at a matching depth in the sound scene.

A phone running the object-recognition model handled books, bottles, bowls, and cups in about 66 milliseconds per frame while drawing about 4.5 watts. The sounds were everyday noises rather than abstract beeps, so an object could be recognized by its character as well as its position. Short icons may ask less of attention than a spoken sentence, which could matter for people with hearing loss.

The device was developed with mobility specialists and blind or low-vision testers, not only engineers. Its goal was to make nearby objects feel present in the room around the head, rather than announced as text in the air.

What blind and sighted users found in two short tasks

In a trial with 14 participants, seven blind or low-vision people and seven sighted volunteers wearing blindfolds searched for objects in two short tasks. The seated task put two or three household items on a table, then asked each person to find one target without help. With the glasses, blind or low-vision participants picked the right object in about 81.7 percent of trials. They did slightly worse than with clock-face speech, which reached 90.7 percent, and a little better than external speakers at 85 percent. That closeness matters because the idealized conditions knew where every object was before the search began. The glasses had to find unknown placements on their own, without perfect advance knowledge. Blindfolded sighted participants found the glasses harder, reaching only 73.0 percent success. Clock-face speech gave them 93.8 percent, and external speakers gave 89.0 percent. The gap was statistically significant, so the difference did not look like chance alone. Even three sighted volunteers with mobility expertise scored no better than five without that background, at 64.8 percent versus 79.2 percent. Prior blindfold experience did not clearly make the sound-based search easier. In the standing task, all participants searched for one target among several distractor bottles while using full-body movement. On average they needed 1.67 reach attempts, with blind or low-vision people at 1.76 and sighted participants at 1.52. That means most people found the right bottle after one or two reaches, though some still mixed up similar containers. Search time in the seated task stayed near 40 seconds for both groups, about 38.25 seconds and 39.63 seconds. Success or failure did not change that time much, nor did adding a third object to the table. Hand motion told another story for the sighted group, who reached faster with glasses than with speakers but took less direct paths. Compared with clock-face speech, their reaches were slower overall and less clean. For blind or low-vision participants, hand speed, path efficiency, and reaching time did not differ meaningfully across conditions. The standing clutter test also showed that a book, bowl, or cup could be picked out among many bottles. Taken together, the two tasks suggest the glasses can support object finding in real movement, but they do not replace clear verbal labels for everyone.

Why the glasses did not feel easier for everyone

For the blindfolded sighted group, the glasses added a layer of work that the other conditions did not require. Clock-face speech told them where an object was in plain directions, while external speakers placed sounds next to known objects. The glasses, by contrast, asked each person to turn their head and build a mental map from moving sound cues. That active scanning was not the same as listening passively. In the seated search with fourteen people, blindfolded sighted participants succeeded on 73.0 percent of trials with the glasses. The same group reached 93.8 percent when an external voice gave clock directions and 89.0 percent when speakers beside objects played sounds. The gap was statistically significant, which means it was unlikely to be a small random wobble. The workload ratings pointed in the same direction, with blindfolded sighted participants reporting higher mental effort and total effort than the blind or low-vision group. They were not simply making mistakes because they had never used the device before. Three of them were orientation-and-mobility specialists, and their overall score did not clearly beat that of the other sighted participants. That pattern suggests novelty mattered more than prior blindfold experience. The glasses also had to work from unknown object positions, while the comparison setups already knew where everything was. That difference made the wearable device a harder task, not because it gave no information but because it required more interpretation. Moving from seated to standing added another strain for the blindfolded sighted group, whose heart rate, breathing rate, and skin temperature rose significantly. The rise likely reflected the stress of navigating an unfamiliar space without vision or a mobility aid. Repeated practice could reduce that response, but on this test day it made the glasses feel less effortless. So the device did not fail for everyone; it simply placed a different burden on people who had not yet learned to trust moving sounds.

How ambiguous sounds and imperfect detection shaped the results

A seated search with 14 people showed that ambiguous sounds and imperfect detection shaped the results. The four objects used different everyday sounds: page turning for a book, scraping for a bottle, a lid on a bowl, and a cup set down on wood. Thirteen of fourteen participants said they mixed up the bottle, bowl, and cup sounds. Those three icons were all container related, so their edges blurred together in memory. The sighted group detected the bowl sound less often than the cup sound. The gap was 12.14 percent and statistically significant. They also identified bottles and bowls better when an outside voice gave clock directions than when they relied on the glasses.

The comparison setups knew where every object sat before each trial. The glasses had to find unknown objects with a camera model that could miss or misplace them. Blind or low-vision participants picked the right object in about 81.7 percent of trials with the glasses. Clock-face speech reached 90.7 percent and external speakers 85 percent. For blindfolded sighted people, the glasses gave 73.0 percent success. Clock-face speech reached 93.8 percent and external speakers 89.0 percent, so the gap was real.

Head-motion data from the same seated task pointed to recognition errors, not poor listening. Successful trials had significantly less head rotation than failed ones, while scanning speed stayed similar. This pattern suggests that wider sweeps exposed the camera to more uncertain objects. In a standing search, all participants found a target among several bottles. They needed an average of 1.67 reach attempts to do it. Still, some had trouble telling a bowl from a bottle.

When only one glass item and a book appeared, people could often infer the confused sound. That shows deduction can help when icons are not unique enough. The ambiguous sounds did not make the glasses useless, but they made them less crisp than perfect verbal labels. Imperfect detection added another layer, because the device had to guess what was there before it could sound it. Their similar success was harder to earn, because they scanned actively and relied on fallible recognition.

Where the sound stops being a toy

The glasses did not beat perfectly informed helpers in every test. They held their own with people who live without sight. In a two-task trial on 14 participants, blind or low-vision users reached for the right object at about the same rate as with clock directions or speakers beside objects. That is a strong result. The glasses had to find unknown items through imperfect camera vision while the wearer turned their head and built a mental map. The comparison conditions knew where everything was.

Everyday objects can be made present without speech. A sound placed in space can carry direction, distance, and identity at once. This may suit busy environments better than spoken labels. For people with hearing loss or limited attention, short icons could reduce the load of decoding words while still preserving enough meaning to act on. The obstacle is that the current system only knows four objects well. Its camera work can mislead when the head sweeps too wide. It also runs hot and drains power on a phone strong enough for the task.

If recognition becomes reliable, if icons are easy to tell apart, and if the hardware stays cool and light, acoustic touch could move from lab table to kitchen counter. The harder question is whether a device that turns the world into sound will feel like help or like another thing to manage.