2011-11-10

Audio Synthesis from 2D Wave Simulation for Organic Noise Generation

Based on http://www.falstad.com/ripple/

My thought was, since I have a game or three I'm working on that have a interesting menagerie of creatures; I need a way to make the sounds the monster make.
Normally, I find people mixing and splicing sources of sounds together to make monsters, for example, Godzilla's classic roar is simply rubbing a resin-covered leather glove along the loosened strings of a double bass and then slowed down. And a lot of star wars noises are recorded from other things, like cars and animals.
This isn't what I want.

So, I found this nifty ripple tank, and immediately figured "hey, now I can model a synthetic vocal system"

After a bit with the code, I'm attempting to work it out into a non-realtime sound generator; basically performing the ripple simulation above with animations savable and editable so you can "render" a sound.

You would ideally begin by placing "walls" (flesh) around in whatever you think makes sense, usually tubes and baffles, then place some emitters. Place a recording source or a few, and run the simulation. While it is running, you should be able to add/remove walls, which is simulating them moving like a tongue. And you should be able to move the emitters and change their influence, and possibly source data (IE you can record yourself and play it through your system). The output will be greatly distorted and morphed by the complex reflections and interference patterns generated; which is exactly what I want. To get that good monster feel, it'll need different types of emitters and easy wall motion with probably line segments for automated flappings.

Here's a screenshot of my version in action; it needs a LOT of optimization as it's way too slow for my taste. Also, I don't have everything editable yet, I'd like to use png files to define the walls and simulation sizes; which are easy for anyone to edit. Probably have to stick with a text/dxml file for defining listeners since you can have them use wav files and such.

Hopefully I can get something like this in a presentable form, after some testing today I'll see what kinda sounds it currently makes.



The "X" is the listener, the "+" are emitters. Aqua is walls, red is +, green is -. The waveform is shown in the bottom right (on my machine, it scrolled pretty fast). Simulation size is ~ 320x180 cells. It's important to note this is for entirely synthetic creatures, not accurate modeling of real physiology. For making video games or movies, this is great. Not so much good  for modeling how a real animal would sound (could try it I guess, the physics don't change)

I bet somebody is doing a thesis on this.

-Z

No comments: