Thank you for your answers, Tom. They sound quite sensible to me.
I'm glad to see complementary work being done at Smith-Kettlewell.

Based on what I know of the current state-of-the-art in machine
vision, I tend to be pessimistic about the feasibility of a machine
that interprets and reads things except in nearly ideal and overly
restricted situations and environments. However, you are certainly
right that there are similar big open questions with respect to the
human brain learning to do these things through a general auditory
representation. The Chinese language sounds like gibberish to me,
and learning it would be very slow (and frustrating to an adult),
and yet over a billion people speak and understand it fluently.
So there seems to be no way that we can predict final results from
initial hardship. To what extent the foreign language analogy carries
over to The vOICe's seeing with sound approach I do not know. I do
not wish to be suggestive. Maybe there are basic reasons why humans
can learn a foreign language and not the interpretation of arbitrary
soundscapes of visual scenery, but for now such reasons are not known.
There is a serious need for scientific research in this area.

Tom writes

> There is wide belief that the brain can learn to be "plastic"
> such that it can learn to "see" with portions of its self which
> were not intended for that purpose, and that the audio perception
> capabilities of blind people are somehow magical and can be
> trained to process unfamiliar complex images in an entirely new
> way.
>
> Despite much misunderstood rumor on this subject I have seen no
> actual scientific evidence to support this claim, and my personal
> experience does not support it either.

I agree with you that there is no such scientific evidence.
I can only hope and pray that the human brain will turn out to
be sufficiently "plastic", but I do not know. The steep learning
curve is one of my biggest concerns, because I do not wish to
bother human volunteers with a major training effort if the end
results would turn out to be disappointing. Regretfully, I do
not have good answers here.

> I have listened to several images on Mr. Meijer's web site and
> find them entirely meaningless despite the fact that I use the
> Smith-Kettlewell oscilloscope regularly.

Yes, most examples are meant for showing sighted visitors the
close relation between sight and sound, and these soundscapes
are way too complicated in sound to make sense of without having
a sighted reference for comparison. However, a few examples try
to be mentally accessible with modest effort and without being
entirely trivial. For instance, recently I presented an example
image containing the single period of a sine wave curve going up,
down, and then up again, as well as ten little squares sounding
as little noise bursts at various positions within the soundscape.
Pitch always indicates vertical position, and stereo panning from
left to right indicates horizontal position of any visual items.

The one-second 22K WAV audio file for this image is at the URL

http://ourworld.compuserve.com/homepages/Peter_Meijer/voiscopebw.wav

and its slow-motion two-second 44K WAV counterpart at the URL

http://ourworld.compuserve.com/homepages/Peter_Meijer/voiscopebw2.wav

With all the unknowns that we are dealing with, I'm indeed glad
that Smith-Kettlewell is working on one of the complementary
approaches. Hopefully one or more of the various inroads will
turn out to be successful, and I will be most interested in what
Smith-Kettlewell will achieve in the camera and machine vision
area. It is good to have different opinions and different research
projects based on that.

Best wishes,

Peter Meijer


Seeing with Sound - The vOICe
http://ourworld.compuserve.com/homepages/Peter_Meijer/winvoice.htm


VICUG-L is the Visually Impaired Computer User Group List.
To join or leave the list, send a message to
[log in to unmask]  In the body of the message, simply type
"subscribe vicug-l" or "unsubscribe vicug-l" without the quotations.
 VICUG-L is archived on the World Wide Web at
http://maelstrom.stjohns.edu/archives/vicug-l.html