3D Virtual Sound Project: Project Proposal
ejhong@cs.stanford.edu
Goal
The following is an initial set of ideas for an implementation of a library to
support (near?) real time playback of preprocessed sound which has been
adjusted to take into account the relative position between listener and
source so that sound appears to be heard externally at a particular location.
This library could then be used with a tracking device and headphones to
enhance a virtual experience.
There are many factors which affect the transformation of sound
patterns between the source and our ears including the shape of the pinnae,
distance to source and room acoustics (reflection and refraction). This
implementation will make use of head related transfer function (HRTF) samples
(probably those obtained from the KEMAR dummy made publicly available from MIT)
which should capture the interaural differences which are due to the location
and shape of the ears. Hopefully, these "generic" samples will be adequate
to produce externalization of sound over standard headphones.
Algorithm
In order to produce the proper sound for an ear, the input sound signal must
be convolved with the HRTF for the relative position between the ear and the
source and additionally be filtered to compensate for other factors such as
the microphone used to record the HRTFs and the headphone setup used to
display the sound. I will focus on first filtering only by the HRTF and look
into other adjustments as needed and as time permits.
The convolution process is probably quite expensive and may be difficult to
do in real time without special hardware support. Thus my implementation
will preprocess sounds with the HRTF and create a sound file for specific
relative ear locations (as is done with the Focal Point Type 2 -another Type
2!). Upon playback, the implementation will then index into the appropriate
sound file for the given relative location between source and listener and
possibly incorporate distance adjustments as well. Althoughthis implementation
uses a lot of space, it will be much faster than trying to convolve on the fly.
Problems will include developing an algorithm for quickly indexing into the
correct sound file, finding the minimal number of locations to store and
perhaps doing some form of interpolating for positions between the sound file
positions.
Equipment Necessary
I'll probably try and do the project on an SGI with support for stereo
and headphones. I haven't yet had the chance to find what kind of support
the SGI's have for stereo sound and performing convolutions (perhaps
pre-processing won't be necessary). These are all I will need to see if the
sound localization in real-time work or not.
If time permits, I would like to try the implementation with some form of
tracking device and visual virtual reality system.
Back to overview
e-mail:
ejhong@cs.stanford.edu
Last modified: March 20, 1996 by Eugene Jhong