New Tools to Help Patients Reclaim Damaged Senses
November 23, 2004
By SANDRA BLAKESLEE

Cheryl Schiltz vividly recalls the morning she became a wobbler.
Seven years ago, recovering from an infection after surgery with
the aid of a common antibiotic, she climbed out of bed feeling
pretty good.

"Then I literally fell to the floor," she said recently. "The
whole world started wobbling. When I turned my head, the room
tilted. My vision blurred. Even the air felt heavy."

The antibiotic, Ms. Schiltz learned, had damaged her vestibular
system, the part of the brain that provides visual and
gravitational stability. She was forced to quit her job and stay
home, clinging to the walls to keep from toppling over.

But three years ago, Ms. Schiltz volunteered for an experimental
treatment - a fat strip of tape, placed on her tongue, with an
array of 144 microelectrodes about the size of a
 postage stamp. The strip was wired to a kind of carpenter's
level, which was mounted on a hard hat that she placed on her
head. The level determined her spatial coordinates and sent the
information as tiny pulses to her tongue.

The apparatus, called a BrainPort, worked beautifully. By
"buzzing" her tongue once a day for 20 minutes, keeping the
pulses centered, she regained normal vestibular function and was
able to balance.

Ms. Schiltz and other patients like her are the beneficiaries of
an astonishing new technology that allows one set of sensory
information to substitute for another in the brain.

Using novel electronic aids, vision can be represented on the
skin, tongue or through the ears. If the sense of touch is gone
from one part of the body, it can be routed to an area where
touch sensations are intact. Pilots confused by foggy conditions,
in which the horizon disappears, can right their aircraft by
 monitoring sensations on the tongue or trunk. Surgeons can feel
on their tongues the tip of a probe inside a patient's body,
enabling precise movements.

Sensory substitution is not new. Touch substitutes for vision
when people read Braille. By tapping a cane, a blind person
perceives a step, a curb or a puddle of water but is not aware of
any sensation in the hand; feeling is experienced at the tip of
the cane.

But the technology for swapping sensory information is largely
the effort of Dr. Paul Bach-y-Rita, a neuroscientist in the
University of Wisconsin Medical School's orthopedics and
rehabilitation department. More than 30 years ago, Dr.
Bach-y-Rita developed the first sensory substitution device,
routing visual images, via a head-mounted camera, to electrodes
taped to the skin on people's backs. The subjects, he found,
could "see" large objects and flickering candles with their
backs. The tongue, sensitive and easy to
 reach, turned out to be an even better place to deliver
substitute senses, Dr. Bach-y-Rita said.

Until recently sensory substitution was confined to the
laboratory. But electronic miniaturization and more powerful
computer algorithms are making the technology less cumbersome.
Next month, the first fully portable device will be tested in Dr.
Bach-y-Rita's lab.

The BrainPort is nearing commercialization. Two years ago, the
University of Wisconsin patented the concept and exclusively
licensed it to Wicab Inc., a company formed by Dr. Bach-y-Rita to
develop and market BrainPort devices. Robert Beckman, the company
president, said units should be available a year from now.

Meanwhile, a handful of clinicians around the world who are using
the BrainPort on an experimental basis are effusive about its
promise.

"I have never seen any other device do what this one does," said
Dr. F. Owen Black, an expert on vestibular
 disorders at the Legacy Clinical Research and Technology Center
in Portland, Ore. "Our patients are begging us to continue using
the device."

Dr. Maurice Ptito, a neuroscientist at University of Montreal
School of Optometry, is conducting brain imaging experiments to
explore how BrainPort works.

Dr. Eliana Sampaio, a neuroscientist at the National Conservatory
of Arts and Metiers in Paris, is using the BrainPort to study
brain plasticity. Sensory substitution is based on the idea that
all sensory information entering the brain consists of patterns
carried by nerve fibers.

In vision, images of the world pass through the retina and are
converted into impulses that travel up the optic nerve into the
brain. In hearing, sounds pass through the ear and are converted
into patterns carried by the auditory nerve into the brain. In
touch, nerve endings on skin translate touch sensations into
patterns carried into the brain.

These patterns travel to special sensory regions where they are
interpreted, with the help of memory, into seeing, hearing and
touch. Patterns are also seamlessly combined so that one can see,
hear and feel things simultaneously.

"We see with the brain, not with the eyes," Dr. Bach-y-Rita said.
"You can lose your retina but you do not lose the ability to see
as long as your brain is intact."

Most important, the brain does not seem to care if patterns come
from the eye, ear or skin. Given the proper context, it will
interpret and understand them. "For me, it happened
automatically, within a few minutes," said Erik Weihenmayer, who
has been blind since he was 13.

Mr. Weihenmayer, a 35-year-old adventurer who climbed to the
summit of Mount Everest two years ago, recently tried another
version of the BrainPort, a hard hat carrying a small video
camera. Visual information from the camera was translated into
pulses
 that reached his tongue.

He found doorways, caught balls rolling toward him and with his
small daughter played a game of rock, paper and scissors for the
first time in more than 20 years. Mr. Weihenmayer said that, with
practice, the substituted sense gets better, "as if the brain
were rewiring itself."

Ms. Schiltz, too, whose vestibular system was damaged by
gentamicin, an inexpensive generic antibiotic used for
Gram-negative infections, said that the first few times she used
the BrainPort she felt tiny impulses on her tongue but still
could not maintain her balance. But one day, after a full
20-minute session with the BrainPort, Ms. Schiltz opened her eyes
and felt that something was different. She tilted her head back.
The room did not move. "I went running out the door," she
recalled. "I danced in the parking lot. I was completely normal.
For a whole hour." Then, she said, the problem returned.

She tried more
 sessions. Soon her balance was restored for three hours, then
half a day. Now working with the BrainPort team at the University
of Wisconsin, Ms. Schiltz wears the tongue unit each morning. Her
balance problems are gone as long as she keeps to the regimen.

How the device produces a lasting effect is being investigated.
The vestibular system instructs the brain about changes in head
movement with respect to the pull of gravity. Dr. Bach-y-Rita
speculated that in some patients, a tiny amount of vestibular
tissue might survive and be reactivated by the BrainPort.

Dr. Black said he had seen the same residual effect in his own
pilot study. "It decays in hours to days," he said, "but is very
encouraging."

Blind people who have used the device do not report lasting
effects. But they are amazed by what they can see. Mr.
Weihenmayer said the device at first felt like candy pop rocks on
his tongue. But that sensation quickly
 gave way to perceptions of size, movement and recognition.

Mr. Weihenmayer said that on several occasions he was able to
find his wife, who was standing still in an outdoor park, but he
admitted that he also once confused her with a tree. Another
time, he walked down a sidewalk and almost went off a bridge.

Nevertheless, he is enthusiastic about the future of the device.
Mr. Weihenmayer likes to paraglide, and he sees the BrainPort as
a way to deliver sonar information to his tongue about how far he
is from the ground.

Dr. Ptito is scanning the brains of congenitally blind people
who, wearing the BrainPort, have learned to make out the shapes,
learned from Braille, of capital letters like T, B or E. The
first few times they wore the device, he said, their visual areas
remained dark and inactive - not surprising since they had been
blind since birth. But after training, he said, their visual
areas lighted up when
 they used the tongue device. The study has been accepted for
publication in the journal Brain.

Dr. Ptito says he would like to see if he could teach his
subjects how to read drifting letters like those in advertising
displays. Not seeing motion is a big problem for the blind, he
said.

In another approach, Dr. Peter Meijer, a Dutch scientist working
independently, has developed a system for blind people to see
with their ears. A small device converts signals from a video
camera into sound patterns delivered by stereo headset to the
ears. Changes in frequency connote up or down. Changes in pixel
brightness are sensed as louder or softer sounds.

Dr. Yuri Danilov, a neuroscientist and engineer who works with
Dr. Bach-y-Rita, said the research team had thought of dozens of
applications for the BrainPort, which he called a "USB port to
the brain."

In one experiment, a leprosy patient who had lost the ability to
 experience touch with his fingers was outfitted with a glove
containing contact sensors. These were coupled to skin on his
forehead. Soon he experienced the data coming from the glove on
his forehead, as if the feelings originated in his fingertips. He
said he cried when he could touch and feel his wife's face.

The federal government has also shown interest in sensory
substitution technology. The Navy is exploring the use of a
tongue device to help divers find their way in dark waters at
night, said Dr. Anil Raj, director of the Institute for Human and
Machine Cognition at the University of West Florida in Pensacola.

The sensors detect water surges, informing Navy Seals if they are
following the correct course. The Army is thinking about sending
infrared signals from night goggles directly to the tongue, Dr.
Raj said.

In another application, student pilots have been fitted with body
sensors attached to aircraft
 instruments. When the airplane starts to pitch or change
altitude, they can feel the movements on their chests.

Sensory substitution technology may eventually help millions of
people overcome their sensory disabilities. But the devices may
also have more frivolous uses: in video games, for example.

Dr. Raj said the tongue unit had already been tried out in a game
that involved shooting villains. "In two minutes you stop feeling
the buzz on your tongue and get a visual representation of the
bad guy," he said. "You feel like you have X-ray vision.
Unfortunately it makes the game boring."

href="javascript:ol('http://www.nytimes.com/2004/11/23/science/23sens.html?e
x=1102194065&ei=1&en=fb1e2069ff7ef426');">http://www.nytimes.com/200
4/11/23/science/23sens.html


VICUG-L is the Visually Impaired Computer User Group List.
To join or leave the list, send a message to
[log in to unmask]  In the body of the message, simply type
"subscribe vicug-l" or "unsubscribe vicug-l" without the quotations.
 VICUG-L is archived on the World Wide Web at
http://maelstrom.stjohns.edu/archives/vicug-l.html