Synthesizing sounds from physically based motion.
This paper describes a technique for approximating sounds that are generated by the motions of solid objects. The technique builds on previous work in the field of physically based animation that uses deformable models to simulate the behavior of the solid objects. As the motions of the objects are computed, their surfaces are analyzed to determine how the motion will induce acoustic pressure waves in the surrounding medium. Our technique computes the propagation of those waves to the listener and then uses the results to generate sounds corresponding to the behavior of the simulated objects.
Figure

1: The top image shows a multi-exposure image from an
animation of a metal bowl falling onto a hard surface. The lower
image shows a spectrogram of the resulting audio for the first five
impacts.
Although the field of computer graphics traditionally focuses on
generating visuals, our perception of an environment encompasses
other modalities in addition to visual appearance. Because these
other modalities play an integral part in forming our impression of
real-world environments, the graphics goal of creating realistic synthetic
environments must also encompass techniques for modeling
the perception of an environment through our other senses. For ex-
ample, sound plays a large role in determining how we perceive
events, and it can be critical to giving the user/viewer a sense of
immersion.
The work presented in this paper addresses the problem of automatically
generating physically realistic sounds for synthetic en-
vironments. Rather than making use of heuristic methods that are
specific to particular objects, our approach is to employ the same
ACM SIGGRAPH 2001, Los Angeles, California, August 12-17, 2001
simulated motion that is already being used for generating animated
video to also generate audio. This task is accomplished by
analyzing the surface motions of objects that are animated using
a deformable body simulator, and isolating vibrational components
that correspond to audible frequencies. The system then determines
how these surface motions will generate acoustic pressure waves in
the surrounding medium and models the propagation of those waves
to the listener. For example, a finite element simulation of a bowl
dropping onto the floor was used to compute both the image shown
in

Figure

1 and the corresponding audio.
Assuming that the computational cost of physically based animation
is already justified for the production of visuals, the additional
cost of computing the audio with our technique is negligible. The
technique does not make use of specialized heuristics, assumptions
about the shape of the objects, or pre-recorded sounds. The audio is
generated automatically as the simulation runs and does not require
any user interaction. Although we feel that the results generated
with this technique are suitable for direct use in many applications,
nothing precludes subsequent modification by another process or a
Foley artist in situations where some particular effect is desired.
The remaining sections of this paper provide a detailed description
of the sound generation technique we have developed, a review
of related prior work, several examples of the results we have
obtained, and a discussion of potential areas for future work. Presenting
audio in a printed medium poses obvious difficulties. We
include plots that illustrate the salient features of our results, and
the proceedings video tape and DVD include animations with the
corresponding audio.
Background
Prior work in the graphics community on sound generation and
propagation has focussed on efficiently producing synchronized
soundtracks for animations [18, 30], and on correctly modeling re-
flections and transmissions within the sonic environment [14, 15,
21]. In their work on modeling tearing cloth, Terzopoulos and
Fleischer generated soundtracks by playing a pre-recorded sound
whenever a connection in a spring mesh failed [31]. The DIVA
project endeavored to create virtual musical performances in virtual
spaces using physically derived models of musical instruments
and acoustic ray-tracing for spatialization of the sound sources [27].
Funkhouser and his colleagues used beam tracing algorithms and
priority rules to efficiently compute the direct and reflected paths
from sound sources to receivers [15]. Van den Doel and Pai mapped
analytically computed vibrational modes onto object surfaces allowing
interactive sound generation for simple shapes [34]. Richmond
and Pai experimentally derived modal vibration responses using
robotic measurement systems, for interactive re-synthesis using
modal filters [26]. More recent work by van den Doel, Kry, and
Pai uses the output of a rigid body simulation to drive re-synthesis
from the recorded data obtained with their robotic measurement
system [33].
Past work outside of the graphics community on simulating the
acoustics of solids for the purpose of generating sound has centered
largely on the study of musical systems, such as strings, rigid bars,
membranes, piano sound boards, and violin and guitar bodies. The
techniques used include finite element and finite difference meth-
ods, lower dimensional simplifications, and modal/sinusoidal models
of the eigenmodes of sound-producing systems.
Numerical simulations of bars and membranes have used either
finite difference [3, 8, 10] or finite element methods [4, 5, 24]. Finite
differencing approaches have also been used to model the behavior
of strings [6, 7, 25].
Many current real-time techniques model the modes of acoustical
systems, using resonant filters[1, 9, 35, 37] or additive sinusoidal
synthesis [28]. In essence, modal modeling achieves ef-ficiency by removing the spatial dynamics, and by replacing the
actual physical system by an equivalent mass-spring system which
models the same spectral response. However, the dynamics (in particular
the propagation of disturbances) of the original system are
lost. If the modal shapes are known, the spatial information can be
maintained and spatial interactions can remain meaningful.
If certain assumptions are made, some systems can be modeled
in reduced dimensionality. For example, if it is assumed that a drum
head or the top-plate of a violin is very thin, a two-dimensional
mesh can be used to simulate the transverse vibration [36].
In many systems such as strings and narrow columns of air,
vibration can be considered one-dimensional, with the principal
modes oriented along only one axis. McIntyre, Schumacher and
Woodhouse's time-domain modeling technique has proven to be
very useful in simulating musical instruments with a resonant system
which is well approximated by the one-dimensional wave equation
[20]. Such systems exhibit the d'Alembert solution, which is
a decomposition of the one-dimensional wave equation into left-
going and a right-going traveling wave components. Smith introduced
extensions to the idea taken from scattering filter theory
and coined the term Waveguide Filters for simulations based on
this one-dimensional signal processing technique [29]. The waveguide
wave-equation formulation can be modified to account for frequency
dependent propagation speed due to stiffness, as described
in [11]. In this technique, the propagation speeds around the eigenmodes
of the system are modeled accurately, with errors introduced
in damping at frequencies other than the eigenmodes.
When considering that the various efficient techniques described
above are available, it should be noted that the approximations are
flawed from a number of perspectives. For example, except for
strings and simple bars, most shapes are not homogeneous. It is
important to observe that for even moderate inhomogeneity, re-
flections at the points of changing impedance have to be expected
that are not captured in a straightforward way. Further, the wave-equation
and Euler-Bernoulli equation derivations also assume that
the differential equations governing solid objects are linear. As a
result the above methods can produce good results but only under
very specific conditions. The method that we describe in the next
section requires more computation than most of the above tech-
niques, but it is much more general.
3 Sound Modeling
When solid objects move in a surrounding medium, they induce
disturbances in the medium that propagate outward. For most fluid
media, such as air or water, the significance of viscous effects decreases
rapidly with distance, and the propagating disturbance has
the characteristics of a pressure wave. If the magnitude of the
pressure wave is of moderate intensity so that shock waves do not
form and the relationship between pressure fluctuation and density
change is approximately linear, then the waves are acoustic waves
described by the equation
where t is time, p is the acoustic pressure defined as the difference
between the current pressure and the fluid's equilibrium pressure,
and c is the acoustic wave speed (speed of sound) in the fluid. Fi-
nally, if the waves reach a listener at a frequency between about
20 Hz and 20,000 Hz, they will be perceived as sound. (See Chapter
five of the text by Kinsler et al. [19] for a derivation of Equation
(1).)
The remainder of this section describes our technique for approximating
the sounds that are generated by the motions of solid
objects. The technique builds on previous work in the field of physically
based animation that uses deformable models to simulate the
Deformable
Object SimulatorExtract Surface Vibrations
Motion Data Compute Wave Propagation
Image Renderer Sound Renderer

Figure

2: A schematic overview of joint audio and visual rendering.
(a) (b)

Figure

3: Tetrahedral mesh for an F]3 vibraphone bar. In (a), only
the external faces of the tetrahedra are drawn; in (b) the internal
structure is shown. Mesh resolution is approximately 1 cm.
behavior of the objects. As the motion of the solid objects is com-
puted, their surfaces are analyzed to determine how the motion will
induce acoustic pressure waves in the surrounding media. The system
computes the propagation of those waves to the listener and
then uses the results to generate sounds corresponding to the simulated
behavior. (See Figure 2.)
3.1 Motions of Solid Objects
The first step in our technique requires computing the motions of
the animated objects that will be generating sounds. As these motions
are computed, they will be used to generate both the audio and
visual components of the animation.
Our system models the motions of the solid objects using a
non-linear finite element method similar to the one developed by
O'Brien and Hodgins [22, 23]. This method makes use of tetrahedral
elements with linear basis functions to compute the movement
and deformation of three-dimensional, solid objects. (See Figure 3.)
Green's non-linear finite strain metric is used so that the method can
accurately handle large magnitude deformations. A volume-based
penalty method computes collision forces that allow the objects to
interact with each other and with other environmental constraints.
For the sake of brevity, we omit the details of this method for modeling
deformable objects which are adequately described in [22].
We selected this particular method because it is reasonably fast,
reasonably accurate, easy to implement, and treats objects as solids
rather than shells. However, the sound generation process is largely
independent of the method used to generate the object motion. So
long as it fulfills a few basic criteria, another method for simulating
deformable objects could be selected instead. These criteria are
Temporal Resolution - Vibrations at frequencies as high
as about 20,000 Hz generate audible sounds. If the simulation
uses an integration time-step larger than approximately
105 s, then it will not be able to adequately model high frequency
vibrations.
Dynamic Deformation Modeling - Most of the sounds that
an object generates as it moves arise from vibrations driven by
elastic deformation. These vibrations will not be present with
techniques that do not model deformation (e.g. rigid body
simulators). Similarly, these vibrations will not be present
with inertia-less techniques.
Computer Graphics Proceedings, Annual Conference Series, 2001
Surface Representation - Because the surfaces of the objects
are where vibrations transition from the objects to the
surrounding medium, the simulation technique must contain
some explicit representation of the object surfaces.
Physical Realism - Simulation techniques used for physically
based animation must produce motion that is visually
acceptable for the intended application. Generating sounds
from the motion will reveal additional aspects of the motion
that may not have been visibly apparent, so a simulation
method used to generate audio must compute motion that is
accurate enough to sound acceptable in addition to looking
acceptable.
The tetrahedral finite element method we are using meets all of
these criteria, but so do many other methods that are commonly
used for physically based animation. For example, a mass and
spring system would be suitable provided the exterior faces of the
system were defined.
3.2 Surface Vibrations
Once we have a method for computing the motion of the objects,
the next step in the process requires analyzing the surface's motions
to determine how it will affect the pressure in the surrounding fluid.
Let be the surface of the moving object(s), and let ds be a differential
surface element in with unit normal ^ and velocity . If
we neglect viscous shear forces then the acoustic pressure, p, of the
fluid adjacent to ds is given by
c is the fluid's specific acoustic impedance. From [19],
the density of air at 20C under one atmosphere of pressure is
and the acoustic wave speed is
giving
Representing the pressure field over requires some form of
discretization. We will assume that a triangulated approximation of
exists (denoted ) and we will approximate the pressure field
as being constant over each of the triangles in .
Each triangle is defined by three nodes. The position, , and
velocity, _, of each node are computed by a physical simulation
method as discussed in Section 3.1. We will refer to the nodes of a
given triangle by indexing with square brackets. For example, [2]
is the position in world coordinates of the triangle's second node.
The surface area of each triangle is given by
and its unit normal by
2a
The average pressure over the triangle is computed by substituting
the triangle's normal and average velocity, , into Equation (2) so
that
The variable p tells us how the pressure over a given triangle
fluctuates, but we are only interested in fluctuations that correspond
to frequencies in the audible range. Frequencies above this range
will need to be removed or they will cause aliasing artifacts.1 Lower
1We assume that the simulation time-step is smaller than the audio sampling
period. This is the case for our examples which used an integration
time-step between 106 s and 107 s.
ACM SIGGRAPH 2001, Los Angeles, California, August 12-17, 2001
{
l l l
Current Time Pressure Impulse
Time Delay

Figure

4: One-dimensional accumulation buffer used to account
for travel time delay.
frequencies will not cause aliasing problems, but they will interact
with later computations to create other difficulties. For example,
an object moving at a constant rate will generate a large, constant
pressure in front of it. The corresponding constant term will show
up as an inconvenient offset in the final audio samples. More im-
portantly, it may interact with latter visibility computations to create
unpleasant artifacts. To remove undesirable frequency components,
we make use of two filters that are applied to the pressure variable
at each triangle in .
First, a low-pass filter is applied to pto remove high frequencies.
The low-pass filter is implemented using a normalized kernel, K,
built from a windowed sinc function given by
sin(2fmaxt)

where fmax is the highest frequency to be retained, t is the simulation
time-step, and w is the kernel's half-width. The low-pass
filtered pressure, g, is obtained by convolving p with K and sub-sampling
the result down to audio rate.
The second filter is a DC-blocking filter that will remove any
constant component and greatly attenuate low-frequency ones. It
works by differentiating a signal and then re-integrating the signal
using a lossy integrator. The final filtered pressure, p~, after application
of the DC-blocking filter is given by
where is a loss constant between zero and one, g is the low-pass
filtered pressure, and the subscripts index time at audio rate.
For the examples presented in this paper, fmax was 22,050 Hz
and we sub-sampled to an audio rate of 44,100 Hz. The low-pass
filter kernel's half-width was three times the wavelength of fmax
d3=(fmaxt)e). The value of was selected by trial and
error yielding good results.
3.3 Wave Radiation and Propagation
Once we know the pressure distribution over the surface of the
objects we must compute how the resulting wave propagates outward
towards the listener. The most direct way of accomplishing
this task would involve modeling the region surrounding the objects
with Equation (1), and using the pressure field over as prescribed
boundary conditions. This approach would lead to a coupled
solid/fluid simulation. Unfortunately, the additional cost of
the fluid simulation would not be trivial. Instead, we can make a
few simplifying assumptions and use a much more efficient solution
method.
Huygen's principle states that the behavior of a wavefront may
be modeled by treating every point on the wavefront as the origin
of a spherical wave, which is equivalent to stating that the behavior
of a complex wavefront can be separated into the behavior of a set
of simpler ones [17]. Using this principle, we can approximate theAmplitude (dB)40200
Predicted
Simulated
Frequency (Hz)
Figure

5: Spectrum generated by plucking the free end of a
clamped bar. Predicted values are taken from [19].
result of propagating a single pressure wave outward from by
summing the results of a many simpler waves, each propagating
from one of the triangles in .
If we assume that the environment is anechoic (no reflections)
and we ignore the effect of diffraction around obstacles, then a reasonable
approximation for the effect on a distant receiver of the
pressure wave generated by a triangle in is given by
p~ax!r
where is the location of the receiver, is the center of the triangle,
is the angle between the triangle's surface normal and the vector
, and x!r is a visibility term that is one if an unobstructed
ray can be traced from to and zero otherwise.2 The cos() is
a rough approximation to the first lobe of the frequency-dependent
beam function for a flat plate [19].
Equation (10) is nearly identical to similar equations that are
used in image rendering with local lighting models, and the decision
to ignore reflected and diffracted sound waves is equivalent
to ignoring secondary illumination. A minor difference is that the
falloff term is inversely proportional to distance, not to distance
squared. The sound intensity, measured in energy per unit time and
area, does falloff with distance squared, but eardrums and microphones
react to pressure which is proportional to the square-root of
intensity [32].
2The center of a triangle is computed by averaging the locations of its
vertices and low-pass filtering the result using the sinc kernel from Equation
(6). Likewise, the normal is obtained from low-pass filtered vertex lo-
cations. The filtering is necessary because the propagation computations are
performed at audio rate, not simulation rate, and incorrectly sub-sampling
the triangle centers or normals will result in audible aliasing artifacts.
Computer Graphics Proceedings, Annual Conference Series, 2001
A more significant difference arises because sound travels several
orders of magnitude slower than light, and we cannot assume
that sound propagation is instantaneous. In fluids such as air, sound
does travel rapidly enough that we may not directly
notice the delay except over large distances, but we do notice indirect
effects even at small distances. For example, very small delays
are responsible for both Doppler shifting and the generation of
interference patterns. Additionally, if we wish to compute stereo
sound by using multiple receiver locations, then delay differences
between a human listener's ears as small as 20 s provide important
cues for localizing sounds [2].
To account for propagation delay we make use of a one-dimensional
accumulation buffer that stores audio samples. All
entries in the buffer are initially set to zero. When we compute
s for each of the triangles in at a given time, we also compute a
corresponding time delay
c
The s values are then added to the entries of the buffer that correspond
to the current simulation time plus the appropriate delay.
(See

Figure

4.)
In general, d will not be a multiple of the audio sampling rate. If
we were to round to the nearest entry in the accumulation buffer, the
resulting audio would contain artifacts akin to the jaggies that occur
when scan-converting lines. These artifacts will manifest themselves
in the form of an unpleasant buzzing sound as if a saw-tooth
wave had been added to the result. To avoid this problem, we add
the s values into the buffer by convolving the contribution with a
narrow (two samples wide) Gaussian and splatting the result into
the accumulation buffer.
As the simulation advances forward in time, values are read from
the entry in the accumulation buffer that corresponds to the current
time. This value is treated as an audio sample that is sent to the
output sink. If stereo sound is desired we compute this propagation
step twice, once for each ear.
We have implemented the described technique for generating audio
and tested the system on several examples. For all of the examples,
two listener locations were used to produce stereo audio. The locations
are centered around the virtual viewpoint and separated by
along a horizontal axis that is perpendicular to the viewing
direction. The sound spectra shown in the following figures follow
the convention of plotting frequency amplitude using decibels, so
that the vertical axes are scaled logarithmically.

Figure

1 shows an image taken from an animation of a bowl
falling onto a hard surface and a spectrogram of the resulting audio.
In this example, only the surface of the bowl is used to generate
the audio, and the floor is modeled as rigid constraint. The spectrogram
reveals the bowl's vibrational modes as darker horizontal
lines. Variations in the degree to which each of the modes are excited
occur because different parts of the bowl hit the surface on
each bounce.
The bowl's shape is arbitrary and there is no known analytical
solution for its vibrational modes. While being able to model arbitrary
shapes is a strength of the proposed method, we would like
to verify its accuracy by comparing its results to known solutions.

Figure

5 shows the results computed for a rectangular bar that is
clamped at one end and excited by applying a vertical impulse to
the other. The accompanying plot shows the spectrum of the resulting
sound with vertical lines indicating the mode frequencies
predicted by a well known analytical solution [19]. Although the
(a)
(b)

Figure

A square plate being struck (a) on center and (b) off
center.
Amplitude (dB)6040200
Predicted
Simulated
Frequency (Hz)
Amplitude (dB)6040200
Predicted
Simulated
Frequency (Hz)

Figure

7: Spectrum generated by striking the square plate shown
in

Figure

6 in the center (top) and off from center (right). Predicted
values are taken from [12].
results correspond reasonably well, the simulated results are noticeably
flatter than the theoretical predictions. One possible explanation
for the difference is that the analytical solution assumes
the bar to be perfectly elastic, while our simulated bar experiences
internal damping.

Figure

6 shows two trials from a simulation of a square plate
(with finite thickness) that is held fixed along its border while being
struck by a weight. In the first trial the weight hits the plate
on-center, while in the second trial the weight lands off-center horizontally
by 25% of the plate's width and vertically by 17%. The
frequencies predicted by the analytical solution given by Flecher
and Rossing [12] are overlaid on the spectra of the two resulting
sounds. As with a real plate, the on-center strike does not sig-
nificantly excite vibrational modes that have nodes lines passing
ACM SIGGRAPH 2001, Los Angeles, California, August 12-17, 2001
Real Bar
Simulation
Simulated 1
Simulated 2
Measured
Tuning
Amplitude (dB)6040200
Frequency (Normalized)

Figure

8: The top image plots a comparison between the spectra of
a real vibraphone bar (Measured), and simulated results for a low-resolution
(Simulated 1) and high-resolution mesh (Simulated 2).
The vertical lines located at 1, 4, and 10 show the tuning ratios
reported in [12].
through the center of the plate (e.g. the modes indicated by the second
and third dashed, red lines). The off-center strike does excite
these modes and a distinct difference can be heard in the resulting
audio. The simulation's lower modes match the predicted ones
quite well, but for the higher modes the correlation becomes ques-
tionable. Poor resolution of the higher modes is to be expected as
they have a lower signal-to-noise ratio and are more easily affected
by discretization and other error sources.
The bars in a vibraphone are undercut so that the first three partials
are tuned according to a 1:4:10 ratio. While this modifica-
tion makes the instrument sound pleasant, the change in transverse
impedance between the thin and thick portions of the bar prevent
an analytical solution. We ran two simulations of a 36 cm long bar
with mesh resolutions of 1 cm and 2 cm, and compared them to a
recording of a real bar being struck. (The 1 cm mesh is shown in

Figure

3.) To facilitate the comparison, the simulated audio was
warped linearly in frequency to align the first partials to that of the
real bar at 187 Hz ( F]3), which is equivalent to adjusting the simulated
bar's density so that is matches the real bar. The results of this
comparison are shown in Figure 8. Although both the simulated and
real bars differ slightly from the ideal tuning, they are quite similar
to each other. All three sounds also contain a low frequency component
below the bar's first mode that is created by the interaction
with the real or simulated supports.
The result of striking a pendulum with a fast moving weight is
shown in Figure 9. Because of Doppler effects, the pendulum's periodic
swinging motion should modulate both the amplitude and the
frequency of the received sound. Because our technique accounts
for both distance attenuation and travel delay, it can model these
phenomena. The resulting modulation is clearly visible in the spectrogram
(particularly in the line near 500 Hz) and can be heard in
the corresponding audio.
Because this sound generation technique does not make additional
assumptions about how waves travel in the solid objects, it
can be used with non-linear simulation methods to generate sounds
for objects whose internal vibrations are not modeled by the linearwave equation. The finite element method we are using employs
a non-linear strain metric that is suitable for modeling large de-
formations. Figure 10 shows frames from two animation of a ball
dropping onto a sheet. In the first one, the sheet is nearly rigid and
the ball rolls off. In the second animation, the sheet is highly compliant
and it undergoes large deformations as it interacts with the
ball. Another example demonstrating large deformations is shown
in

Figure

11 where a slightly bowed sheet is being bent back and
forth to create crinkling sound. Animations containing the audio for
these, and other, examples have been included on the proceedings
video tape and DVD. Simulations times are listed in Table 1.
5 Conclusions and Future Work
In this paper, we have described a general technique for computing
physically realistic sounds that takes advantage of existing simulation
methods already used for physically based animation. We have
also presented a series of examples that demonstrate the results that
can be achieved when our technique is used with a particular, finite
element based simulation method.
One area for future work is to combine this sound generation
technique with other simulation methods. As discussed in Section
3.1, it should be possible to generate audio from most deformable
body simulation methods such as mass and spring systems
or dynamic cloth simulators. It may also be possible to generate
audio from some fluid simulation methods, such as the method
developed by Foster and Metaxas for simulating water [13].
Of the criteria we listed in Section 3.1, we believe that the required
temporal resolution is most likely to pose difficulties. If
the simulation integrator uses time-steps that are larger than about
105 s, the higher frequencies of the audible spectrum will not be
sampled adequately so that, at best, the resulting audio will sound
dull and soggy. Unfortunately, small time-steps result in slow sim-
ulations, and as a result a significant amount of research has focused
on finding ways to allow numerical integrators to take larger
steps while remaining stable. In general, the larger time-steps are
achieved by removing the high-frequency components from the
motions being computed. While removing these high-frequency
components will at some point create visually objectionable arti-
facts, it is likely that sound quality will be adversely affected first.
Rigid body simulations are also excluded by our criteria because
they do not model the deformations that drive most of the vibrations
that produce sound. This limitation is particularly unfortunate
because rigid body simulations are widely used, particularly in
interactive applications. Because they are so commonly used, developing
general methods for computing sounds for rigid body and
large time-step simulation methods is an important area for future
work.
Although our sound propagation model is relatively cheap to
compute, it is also quite limited. Reflected and diffracted sound
transport often play a significant role in determining what we hear
in an environment. To draw an analogy with image rendering, our
current method is roughly equivalent a local illumination model and
adding reflection and diffraction would be equivalent to stepping up
to global illumination. In some ways global sound computations
would actually be more complex than global illumination because
one cannot assume that waves propagate instantaneously. Other researchers
have investigated techniques for modeling acoustic reflec-
tions, for example [14], and combining our work with theirs would
probably be useful.
Our listener model could also be improved. As currently im-
plemented, a listener receives pressure waves equally well from all
directions. While this behavior is ideal for an omni-directional mi-
crophone, human ears and most real microphones behave quite dif-
ferently. One obvious effect is that the human head acts as a blocker
so that high frequency sounds from the side tend to be heard better
Computer Graphics Proceedings, Annual Conference Series, 20012500Frequency (Hz)Amplitude (dB)500400
Time

Figure

9: The spectrogram produced by a swinging bar after it is
struck by a weight.
by that ear while low frequencies diffract around the head and are
heard by both ears. Other, more subtle effects, such as the pattern
of reflections off of the outer ear, also contribute to allowing us to
localize the sounds we hear. Other researchers have taken extensive
measurements to determine how sounds are affected as they enter a
human ear, and used the compiled data to build head-related transfer
functions [2, 16]. Filters derived from these transfer functions have
been used successfully for generating three-dimensional spatialized
audio. We are currently working on adding this functionality to our
existing system.
Our primary goal in this work has been to generate a tool that is
useful for generating audio. However, we have also noticed that the
audio produced by a simulation makes an excellent debugging tool.
For example, we have observed that symptoms of a simulation going
unstable often can be heard before they become visible. Other
common simulation errors, such as incorrect collision response, are
also evidenced by distinctively odd sounds. As physically based
animation continues to becomes more commonly used, sound generation
could become useful not only for its own sake but also as
standard tool for working with and debugging simulations.

Acknowledgments

The images in this paper were generated using Alias-Wavefront's
Maya and Pixar Animation Studios' RenderMan software running
on Silicon Graphics hardware. This research was supported, in part,
by a research grant from the Okawa foundation, by NSF grant number
9984087, and by the State of New Jersey Commission on Science
and Technology Grant Number 01-2042-007-22.


--R


Numerical simulations of xylophones: II.
Measurements and efficient simulations of bowed bars.
The Physics of Musical Instruments.
Realistic animation of liquids.
A beam tracing approach to acoustic modeling for interactive virtual environments.

Figure 10: These figures show a round weight being dropped onto
The surface shown in (a) is rigid while the
one shown in (b) is more compliant.
Los Angeles
Example Figure Simulation t Nodes Elements Surface Elements Total Time Audio Time
Clamped Bar 5 1 107 s 125 265 246 240:4 min 1:26 min 0:5


Vibraphone Bar 8 1 107 s 539 1484 994 1309:7 min 5:31 min 0:4
Swinging Bar 9 3 107 s 130 281 254 88:4 min 1:42 min 1:6
Rigid Sheet 10.
Compliant Sheet 10.
Bent Sheet 11 1 107 s 678
Table 1: The total times indicate the total number of minutes required to compute one second of simulated data

listed indicate the time spent generating audio as a percentage of the total simulation time.
using one 350 MHz MIPS R12K processor while unrelated processes were running on the machine's other processors.
HRTF measurements of a KEMAR dummy head microphone.
Introduction to Fourier Optics.
An integrated approach to sound and motion.
Fundamentals of Acoustics.
On the oscillation of musical instruments.

Graphical modeling and animation of brittle fracture.
Animating fracture.
Nonuniform beams with harmonically related overtones for use in percussion instruments.
Physical modeling by directly solving wave PDE.
Robotic measurement and modeling of contact sounds.

A computer model for bar percussion instruments.
Sound rendering.
Deformable models.
Handbook for Acoustic Ecology.



Physical modeling with the 2D waveguide mesh.
VLSI models for sound synthesis.
Figure 11: A slightly bowed sheet being bent back and forth.
--TR

--CTR
Nikunj Raghuvanshi , Ming C. Lin, Interactive sound synthesis for large scale environments, Proceedings of the 2006 symposium on Interactive 3D graphics and games, March 14-17, 2006, Redwood City, California
Golan Levin , Zachary Lieberman, Sounds from shapes: audiovisual performance with hand silhouette contours in the manual input sessions, Proceedings of the 2005 conference on New interfaces for musical expression, May 26-28, 2005, Vancouver, Canada
Davide Rocchesso , Roberto Bresin , Mikael Fernstrm, Sounding Objects, IEEE MultiMedia, v.10 n.2, p.42-52, March
James F. O'Brien , Chen Shen , Christine M. Gatchalian, Synthesizing sounds from rigid-body simulations, Proceedings of the 2002 ACM SIGGRAPH/Eurographics symposium on Computer animation, July 21-22, 2002, San Antonio, Texas
Ying Zhang , Reza Sotudeh , Terrence Fernando, The use of visual and auditory feedback for assembly task performance in a virtual environment, Proceedings of the 21st spring conference on Computer graphics, May 12-14, 2005, Budmerice, Slovakia
Ying Zhang , Terrence Fernando , Hannan Xiao , Adrian R. L. Travis, Evaluation of auditory and visual feedback on task performance in a virtual assembly environment, Presence: Teleoperators and Virtual Environments, v.15 n.6, p.613-626, December 2006
M. Cardle , S. Brooks , Z. Bar-Joseph , P. Robinson, Sound-by-numbers: motion-driven sound synthesis, Proceedings of the ACM SIGGRAPH/Eurographics symposium on Computer animation, July 26-27, 2003, San Diego, California
Doug L. James , Dinesh K. Pai, DyRT: dynamic response textures for real time deformation simulation with graphics hardware, ACM Transactions on Graphics (TOG), v.21 n.3, July 2002
Kees van den Doel, From physics to sound: Comments on van den Doel, ICAD 2004, ACM Transactions on Applied Perception (TAP), v.2 n.4, p.547-549, October 2005
Laura Ottaviani , Davide Rocchesso, Auditory perception of 3D size: Experiments with synthetic resonators, ACM Transactions on Applied Perception (TAP), v.1 n.2, p.118-129, October 2004
Yoshinori Dobashi , Tsuyoshi Yamamoto , Tomoyuki Nishita, Real-time rendering of aerodynamic sound using sound textures based on computational fluid dynamics, ACM Transactions on Graphics (TOG), v.22 n.3, July
Georg Essl , Stefania Serafin , Perry R. Cook , Julius O. Smith, Musical Applications of Banded Waveguides, Computer Music Journal, v.28 n.1, p.51-63, March 2004
Kees van den Doel, Physically based models for liquid sounds, ACM Transactions on Applied Perception (TAP), v.2 n.4, p.534-546, October 2005
Doug L. James , Jernej Barbi , Dinesh K. Pai, Precomputed acoustic transfer: output-sensitive, accurate sound generation for geometrically complex vibration sources, ACM Transactions on Graphics (TOG), v.25 n.3, July 2006
Cynthia Bruyns, Modal Synthesis for Arbitrarily Shaped Objects, Computer Music Journal, v.30 n.3, p.22-37, September 2006
Kees van den Doel , Dave Knott , Dinesh K. Pai, Interactive simulation of complex audiovisual scenes, Presence: Teleoperators and Virtual Environments, v.13 n.1, p.99-111, February 2004
Georg Essl , Stefania Serafin , Perry R. Cook , Julius O. Smith, Theory of Banded Waveguides, Computer Music Journal, v.28 n.1, p.37-50, March 2004
Kees van den Doel , Paul G. Kry , Dinesh K. Pai, FoleyAutomatic: physically-based sound effects for interactive simulation and animation, Proceedings of the 28th annual conference on Computer graphics and interactive techniques, p.537-544, August 2001
Richard Corbett , Kees van den Doel , John E. Lloyd , Wolfgang Heidrich, Timbrefields: 3d interactive sound models for real-time audio, Presence: Teleoperators and Virtual Environments, v.16 n.6, p.643-654, December 2007
Mashhuda Glencross , Alan G. Chalmers , Ming C. Lin , Miguel A. Otaduy , Diego Gutierrez, Exploiting perception in high-fidelity virtual environmentsAdditional presentations from the 24th course are available on the citation page, ACM SIGGRAPH 2006 Courses, July 30-August 03, 2006, Boston, Massachusetts
animation techniques;finite element method;physically based modeling;sound modeling;simulation;dynamics;surface vibrations