University, Spring 2021, CS
448Z: Physically Based Animation and Sound
#0: Audiovisual code warm-up
Goal: In this preliminary assignment, you
will implement an interactive audiovisual simulation of a
simple particle system. The goal of this
assignment is to ensure that everyone has a basic
audiovisual coding framework setup before we start a nontrivial
Platform: You may implement your audiovisual simulation in any
suitable real-time graphics environment that will support custom sound
generation--not just loading & playing sound clips, or using
fixed-function oscillator models. You may have your own preference, but
some suggestions are:
- OpenFrameworks (C++): Visit the openFrameworks website (http://openframeworks.cc)
and download a version appropriate for your computing
platform. Complete or follow along with their tutorials (http://openframeworks.cc/tutorials)
in order to gain basic understanding.
- Unity (C#): You may be familiar with the Unity game engine (https://unity.com), in which case feel free to use that instead. Here is a talk about Unity's audio system, and a tutorial on how to display procedural audio in Unity using the onAudioFilterRead method (and a related video tutorial).
- JUCE (C++): A multi-platform audio application framework (https://juce.com) that is well-suited to real-time audiovisual synthesis.
- Custom (C++, etc.): You can also write your own
application using a low-level high-performance framework, e.g., OpenGL +
OpenAL, if you have prior experience in graphics and real-time audio.
- Not recommended: There are many great tools for
making interactive audiovisual programs that are not suitable for custom
procedural audio synthesis. Performance is one reason that may make
audio simulation. Another problem is that some APIs, e.g., processing
or p5.js, do not (for performance reasons) support low-level procedural
audio operations wherein you can set the audio buffer directly--which we
will require later.
Instructions: In the following steps, you will implement a simple particle system of N balls (or other objects) bouncing in a box. You can use 2D or 3D depending on
your graphics preferences. For example, here is an OpenProcessing implementation of 2D and 3D examples (without sound).
the basic graphics framework to draw N balls on the
screen (e.g., as circles or spheres), and maintain the position/velocity state of each
the particles move: Implement a symplectic Euler
integrator which (as discussed in class) first updates the
particle velocity using applied forces, and then updates the
particle position. You can use random initial velocities
(of a suitably bounded magnitude) to get started. Include
a downward gravitational force.
collision detection and response: The particles are currently not constrained
to stay inside the box. Implement a scheme
to detect collisions with the four walls, and modify the
velocity to provide simple collision response, using a
(particle-specific) restitution coefficient to attenuate
normal velocity. Be careful how you update your particle
velocity/position or you might find your particles sinking
into the floor.
sound: Every time a ball hits a wall, you should play
a suitable "click" (or other) custom sound or your devising to indicate that an impact was
made, with amplitude proportional to the collision impulse
magnitude, m |dv|. Next, to avoid all collisions during a
time-step producing "clicks" at the same time-step-size
dependent instant, you can estimate the time of ball-wall
collision more accurately, and shift the "click" sound in
the sound buffer. As an implementation detail, you
can use a circular sound buffer to composite
sounds into, then regularly extract the next output sound
clip (of suitable "buffer size"). See the documentation on
how to stream sounds from a real-time application. Your
sound should play clearly and without noticeable audio
- Note: You do not need to resolve collisions between balls.
interactive: You can pull particles around by
applying forces to particles near your mouse position. You
can also change, e.g., rotate, the direction of gravity
using keyboard controls to make all the particles move.
For this starter assignment, feel free to help each other and ask
questions on Piazza.
Submission: Please upload your implementation and a
very brief note about your findings as a zip file to your HW0
entry in Canvas.