Audio Synthesis from 2D Wave Simulation for Organic Noise Generation

Based on http://www.falstad.com/ripple/

My thought was, since I have a game or three I'm working on that have a interesting menagerie of creatures; I need a way to make the sounds the monster make.
Normally, I find people mixing and splicing sources of sounds together to make monsters, for example, Godzilla's classic roar is simply rubbing a resin-covered leather glove along the loosened strings of a double bass and then slowed down. And a lot of star wars noises are recorded from other things, like cars and animals.
This isn't what I want.

So, I found this nifty ripple tank, and immediately figured "hey, now I can model a synthetic vocal system"

After a bit with the code, I'm attempting to work it out into a non-realtime sound generator; basically performing the ripple simulation above with animations savable and editable so you can "render" a sound.

You would ideally begin by placing "walls" (flesh) around in whatever you think makes sense, usually tubes and baffles, then place some emitters. Place a recording source or a few, and run the simulation. While it is running, you should be able to add/remove walls, which is simulating them moving like a tongue. And you should be able to move the emitters and change their influence, and possibly source data (IE you can record yourself and play it through your system). The output will be greatly distorted and morphed by the complex reflections and interference patterns generated; which is exactly what I want. To get that good monster feel, it'll need different types of emitters and easy wall motion with probably line segments for automated flappings.

Here's a screenshot of my version in action; it needs a LOT of optimization as it's way too slow for my taste. Also, I don't have everything editable yet, I'd like to use png files to define the walls and simulation sizes; which are easy for anyone to edit. Probably have to stick with a text/dxml file for defining listeners since you can have them use wav files and such.

Hopefully I can get something like this in a presentable form, after some testing today I'll see what kinda sounds it currently makes.

The "X" is the listener, the "+" are emitters. Aqua is walls, red is +, green is -. The waveform is shown in the bottom right (on my machine, it scrolled pretty fast). Simulation size is ~ 320x180 cells. It's important to note this is for entirely synthetic creatures, not accurate modeling of real physiology. For making video games or movies, this is great. Not so much good  for modeling how a real animal would sound (could try it I guess, the physics don't change)

I bet somebody is doing a thesis on this.

-Z

3D Fractals

http://www.youtube.com/user/xlace
http://www.youtube.com/user/bib993
http://www.fractalforums.com/



3D fractals are pretty intimidating. If you don't like this kind of thing, smoke some before you watch it.

Hell, view some in realtime with WebGL: http://fractal.io/
Or make your own: http://www.fractalforums.com/ -> downloads

I made my own, now to explore/video it:

Mandelbulb3Dv1697n:

Mandelbulb3Dv16{
O.....Y/...g2...w....26...EJg46XGin..vJShBjn89.k0M/cLoxG1.gEVm30U7O..3gX6fCIlA.k
................................Cr5ZfPIy602........Y./..................y.2.....
................/M.0/....6Eg1...a4....E2.....YxFZmNi/kfD/..........c./...w1.OaNa
z.UaNadD1E..0..........wz........................................y1...sD...../..
.w1...sDKF9UsFVALu17DWNplc8iyYH2DGggRTej9M9tJY2hcuXQvXRVF4FcyktgZtmSYSfDR/hyWOGR
su1nN6r/023dyeKwHkets6ej......oQ3.....................sD..kz....................
..............................M6S4.4WsN.k8Wb/Uo6S4.sXsN.sFWb/.F7S4..............
.....................wzzz1.U..6.P....M4...EB....W....61....F....8/...I1....UJl22
...U.iVFwxDE./ozPM2Tzz7lz16.mc..zXCc.El18XGQeGyDjvIRhrVAkz1............28.kFrA0.
FWb96aAIVzvh1se7Umvxz0........../6U0.wzzz1................................E.0c..
zzzz.................................2U.8.kzzzD.................................
/6U0.wzzz1...................................2CcN/UvNP6.eeWCNq0.yRii.EJJUk1f..XR
S1.mx3CcN/UvNP6.QsLsUa3.ibhV..bTV1OK.sSq40.ly3CcN/UvNP6.MwLsUa3.ibhV.kqTV1OK.sSq
40.kz3CcN/UvNP6...EsUa3.eeWCNq0.IJ36wk8.wyLsUa3.................................
E....A....E.....I....6....kL/pKMuZaPb7oPs/UQ..........................k/......U.
0..................../........zD...................0./........zDaNaNaNaNKz1.....
..cF.1.......E0E........kz1.....................................................
.....................2.....3....8....woMo34OZFaQjtKG4B3.........................
.....MU/4MU/4..................E........kz1........wz.........zD................
.......................UK/2.......cJ./.......O3E................................
................................/....E/...E.....GZKNh3aPi/ERg74.................
........................................U.2.............................kz1.....
................................................................................
............................................}


WebGL - TED Setups

Try it now!

Current test as a local zip:
http://www.gocaco.com/webgl/test1/Test.zip

Current TED exporter:
http://www.gocaco.com/webgl/igtl_export_ted.py

Added some bugfixes for animations;
Added the ability to configure "setups" which can assign positions, rotations, animations to objects in a scene.

Eventually, I'll replace this system with more game-level functionality, as well as update the method data is loaded in favor of "precompiled" models with external texture references. It will take a while to get a Blender 2.57 exporter for this; but I'll keep trying.

First project I need to try is a robotron clone since it doesn't require advanced collisions.

Here is an example use (Farina II rough level), you can correct file export errors with a setup for proper viewing/interacting.

 However, it is obvious the terrain shader was not loaded because it isn't exported yet; Nor is the glowing gem in the cage. These are serious problems with the pipeline of data exported from blender, but are fixable. The only question is whether to focus on 2.57b+ or 2.49b-. I'll probably stick with 2.5X.

-Z

Least squares fit a sphere to 3D data

I didn't find this online anywhere, and I had some data I needed a least squares fit with a sphere for.

All you have to do is define:

Error = Sum( |Position[n] - Center|^2 - Radius^2 )

Then define the squared error:

Squared Error = Sum( ( |Position[n] - Center|^2 - Radius^2 )^2 )

And solve the summation using a iterative method (like newtons, below) after pulling out the summation terms.
For example, if you do: Sum( (P.x[n] - Cx)^2 ) You get (after Expand):
Sum( P.x[n]^2 - 2*P.x[n]*Cx + Cx^2 )
And you can then split up the sum:
Sum( P.x[n]^2 ) + Sum( P.x[n] ) * -2*Cx + Cx * Nelements
Note you HAVE to ultimately divide the sums by Nelements

Note that "Center" is A,B,C (3D) and I use Rsq as Radius^2.

This method is not fast, but it converges, and the way the code is written it is independent of dataset size, but you do have to compute a number of sums and products before running the algorithm.

Note this method is used to generate the equations used to compute linear and quadratic fits instantly, given you compute some sums first. I suppose it can be extended to any shape with enough working the mathematics. The next shapes are planes, capsules, maybe torii.


//
//Least Squares Fit a sphere A,B,C with radius squared Rsq to 3D data
//
//    P is a structure that has been computed with the data earlier.
//    P.npoints is the number of elements; the length of X,Y,Z are identical.
//    P's members are logically named.
//
//    X[n] is the x component of point n
//    Y[n] is the y component of point n
//    Z[n] is the z component of point n
//
//    A is the x coordiante of the sphere
//    B is the y coordiante of the sphere
//    C is the z coordiante of the sphere
//    Rsq is the radius squared of the sphere.
//
//This method should converge; maybe 5-100 iterations or more.
//
double Xn = P.Xsum/P.npoints;        //sum( X[n] )
double Xn2 = P.Xsumsq/P.npoints;    //sum( X[n]^2 )
double Xn3 = P.Xsumcube/P.npoints;    //sum( X[n]^3 )
double Yn = P.Ysum/P.npoints;        //sum( Y[n] )
double Yn2 = P.Ysumsq/P.npoints;    //sum( Y[n]^2 )
double Yn3 = P.Ysumcube/P.npoints;    //sum( Y[n]^3 )
double Zn = P.Zsum/P.npoints;        //sum( Z[n] )
double Zn2 = P.Zsumsq/P.npoints;    //sum( Z[n]^2 )
double Zn3 = P.Zsumcube/P.npoints;    //sum( Z[n]^3 )

double XY = P.XYsum/P.npoints;        //sum( X[n] * Y[n] )
double XZ = P.XZsum/P.npoints;        //sum( X[n] * Z[n] )
double YZ = P.YZsum/P.npoints;        //sum( Y[n] * Z[n] )
double X2Y = P.X2Ysum/P.npoints;    //sum( X[n]^2 * Y[n] )
double X2Z = P.X2Zsum/P.npoints;    //sum( X[n]^2 * Z[n] )
double Y2X = P.Y2Xsum/P.npoints;    //sum( Y[n]^2 * X[n] )
double Y2Z = P.Y2Zsum/P.npoints;    //sum( Y[n]^2 * Z[n] )
double Z2X = P.Z2Xsum/P.npoints;    //sum( Z[n]^2 * X[n] )
double Z2Y = P.Z2Ysum/P.npoints;    //sum( Z[n]^2 * Y[n] )

//Reduction of multiplications
double F0 = Xn2 + Yn2 + Zn2;
double F1 = 0.5*F0;
double F2 = -8.0*(Xn3 + Y2X + Z2X);
double F3 = -8.0*(X2Y + Yn3 + Z2Y);
double F4 = -8.0*(X2Z + Y2Z + Zn3);

//Set initial conditions:
A = Xn;
B = Yn;
C = Zn;

//First iteration computation:
double A2 = A*A;
double B2 = B*B;
double C2 = C*C;
double QS = A2 + B2 + C2;
double QB = - 2*(A*Xn + B*Yn + C*Zn);

//Set initial conditions:
Rsq = F0 + QB + QS;

//First iteration computation:
double Q0 = 0.5*(QS - Rsq);
double Q1 = F1 + Q0;
double Q2 = 8*( QS - Rsq + QB + F0 );
double aA,aB,aC,nA,nB,nC,dA,dB,dC;

//Iterate N times, ignore stop condition.
int n = 0;
while( n != N ){
    n++;

    //Compute denominator:
    aA = Q2 + 16*(A2 - 2*A*Xn + Xn2);
    aB = Q2 + 16*(B2 - 2*B*Yn + Yn2);
    aC = Q2 + 16*(C2 - 2*C*Zn + Zn2);
    aA = (aA == 0) ? 1.0 : aA;
    aB = (aB == 0) ? 1.0 : aB;
    aC = (aC == 0) ? 1.0 : aC;

    //Compute next iteration
    nA = A - ((F2 + 16*( B*XY + C*XZ + Xn*(-A2 - Q0) + A*(Xn2 + Q1 - C*Zn - B*Yn) ) )/aA);
    nB = B - ((F3 + 16*( A*XY + C*YZ + Yn*(-B2 - Q0) + B*(Yn2 + Q1 - A*Xn - C*Zn) ) )/aB);
    nC = C - ((F4 + 16*( A*XZ + B*YZ + Zn*(-C2 - Q0) + C*(Zn2 + Q1 - A*Xn - B*Yn) ) )/aC);

    //Check for stop condition
    dA = (nA - A);
    dB = (nB - B);
    dC = (nC - C);
    if( (dA*dA + dB*dB + dC*dC) <= Nstop ){ break; }

    //Compute next iteration's values
    A = nA;
    B = nB;
    C = nC;
    A2 = A*A;
    B2 = B*B;
    C2 = C*C;
    QS = A2 + B2 + C2;
    QB = - 2*(A*Xn + B*Yn + C*Zn);
    Rsq = F0 + QB + QS;
    Q0 = 0.5*(QS - Rsq);
    Q1 = F1 + Q0;
    Q2 = 8*( QS - Rsq + QB + F0 );
}

WebGL - Preview local TED files

Try it now!

Current test as a local zip:
http://www.gocaco.com/webgl/test1/Test.zip

Current TED exporter:
http://www.gocaco.com/webgl/igtl_export_ted.py

If you have any local files, you can now browse for and load them.
You can also drag them over into the 3D area, and it will load them automagically.

Some interesting notes about this.

Security wise, I would rather have a notification come up if a javascript script was attempting to "read" any local files. I DO want this to occur, but let me either block ALL of them, or choose as they are requested to be loaded.Firefox understands this, but I cannot find it's setting to make it warn you when a script reads a local file. Apparently the act of dragging and dropping or using the file open dialog is confirmation enough, and that's OK.

Chrome however, is dumb, and will not read any local files unless you add the additional command like switch "--allow-file-access-from-files" switch to it. DO NOT DO THIS if you are online, only do this while developing or playing locally. Because chrome is kinda insecure like that, might as well not use it until they fix this extremely annoying bug. Get Chromium instead.
Examples of chrome stupidness: "There is nothing Gmail can do to defend itself from this attack." "In the future, we hope to further restrict the privileges of local web pages." "Ultimately, we'd like to see all the browser vendors converge on a uniform, secure policy for local web pages."  Seriously? Are these dumbassess tripping? Give us control of some of those flags, and then configure the flags to make your "security" work. Don't generalize and call that good, that's what mac does and why it is so much fail. Give control to advanced uses, set the defaults for your pseudosecurity model, everyone wins. Like firefox.

The option for "Only allow local access with user permission" would be nice, and it could pop up a modal dialogue with timing/input security. Not failproof (did you know you can write code to make fake human input? lol ur h4xed), but it's better than a blanket policy.

An old BomberFan level, made for BGE. Since it was an old blender, blender materials were ignored in favor of face textures, which is why nothing looks right.

-Z

WebGL - I hate chrome

Try out webgl: (you should click the "load" button, and see "Blender", a blue sphere, a red pyramid, in a room.)

Try it now!

Current test as a local zip: 
http://www.gocaco.com/webgl/test1/Test.zip
Current TED exporter:
http://www.gocaco.com/webgl/igtl_export_ted.py

I attempted to test it on Chrome, and after instantly hating chrome, which is severely lacking in features, performance, and stability (as advertised. huh.); It's also missing most all my usual tools that are cleanly implemented in firefox. Chrome is no competition, minus it's nifty threading of scripts. Seems like good ole' google is becoming exactly like microsoft, pointing to the ultimate downfall of all software based empires. It's simply evolutionary that we continually kill off good products with lawsuits, antitrust and other money sucking schemes which hurts everyone because they then have to side with more evil tactics (like online installation, automatic updates, spyware, ads, ect...) just to survive. So long as money is required for software, this will occur.

Anyways, to get chrome to even work, I had to add these three lines to the command console, which I highly recommend NOT adding the last one, the "--allow-file-access-from-files" because that one should only be true for offline files.

"--ignore-gpu-blacklist --enable-webgl --allow-file-access-from-files"

 Interesting note, firefox understands that a local file has a local domain of "./" and can pull things from it. It has no problem. Chrome says this is a cross domain error, when it clearly pulled the file from "./" and then decides it's cross domain because it has a "file://" prefix? No sense there. How about asking me you stupid %@(& browser? And I'll say "get as pure text only." and I'll force a MIME type of text/xml. Then everyone wins right?

Ugh. whatever. Fuck chrome. Though, I suppose getting firefox to use webgl is hard too.

-Z

WebGL - TED DXML importer animation test

Animations and animation playback added. Posing is better, glitch with animation frame fixed, Objects retain independent armature state, small speedup.

Current Test:
Test it right now!

http://www.gocaco.com/webgl/test1/Test.zip

Current Exporter:
http://www.gocaco.com/webgl/igtl_export_ted.py

 This pretty much has everything that's in a TED file, which, for your reference, contains:

Mesh data (coords, texcoords, normals, tangents, matrix palette weights, colors, params)
Textures (including texture data if you want)
Materials (some parameters, illy defined for now.)
Armatures (Includes bones, IK chains, armature splines, lots of data)
Animations (Includes channel blending animations, so channels are seperate and integrated. Includes scripts.)
Scenes (all objects and their properties)

Though there are still more things, this is the basics to get started cleaning this mess up and making it easier to work with. I don;t need the BGE if I can load this stuff into WebGL. That's what's really nice.

Although, it has some really painful disadvantages, mainly input problems, no joystick support, bad keyboard input events, no locking the mouse so no FPS possible... and so on.

Here's 9 marios independently running (bad running animation, I know. took me a minute ok??)

And 25 marios...

Most of the speed is CPU <-> GPU delays. Just remember that 1 gpu "talk" takes 6~100's of CPU cycles because of the bus communication. So, this is a "absolute worse case" naive way to do this, and I still get 15 FPS with 100k fully shaded armatured polygons. As a side note, an average frame of Oblivion as ~200k polygons, but most all of them were static (75% static to 25% dynamic).

-Z

WebGL - TED DXML importer with armatures; animations needed

Yet another update. Now you can mess with the bones of whatever you imported, albeit unoptimized and rather slow, in comparison to the identical C++ program.

Current Test:
http://www.gocaco.com/webgl/test1/Test.zip

Current Exporter:
http://www.gocaco.com/webgl/igtl_export_ted.py

Next up is probably cleaning or testing animations. It is obvious that speed optimization is needed badly, but the math does not really change much; Javascript seems to be missing access to SSE type data. As well as mouse input not based on pixels, and joysticks, and a audio API, ect...

In a less complete sense, I modified the matrix conversion function to be smaller; though it's not computationally faster. The object with matrix animation/ armature animation is as follows:

Parent Armature Space Matrix * Default From Parent Matrix * Current Local Transform = Current Armature Space Matrix

Parent Armature Space Matrix => The transform of the parent, as calculated with current animation data.
Default From Parent Matrix => Upon loading an armature, calculate this by Parent^-1 * Current. This represents the initial rotational and positional offset from the initial bone positions to eachother, which allows your animation local data to be relative to that starting point.
Current Local Transform => The offsets, like position, rotation, scale. Depending on how you want to apply scale this can be tricky.
Current Armature Space Matrix => The final transformed position; send t his to the 'convertforshader' algorithm.

And, in effect, animating is only changing the set of local deformations over time, and possibly adjusting them with things like IK or mouselooks. And all that is best done with SSE operators.

I bent up this Chameleon Boss from Metroid: Other M.

-Z

WebGL - TED DXML importer bones work; needs animations

I got the bones working using some prior work from xcore/mingl.
Unfortunately, it's difficult to show this without a logically consistent animation; However, if you play with the matrix values you can see it works perfectly.

Current Test:
http://www.gocaco.com/webgl/test1/Test.zip

Current Exporter:
http://www.gocaco.com/webgl/igtl_export_ted.py

You probably have wondered how skinning works; Skinning is a procedure where static mesh data is deformed by adjusting the position, rotation, and scale of a transformation matrix. Usually these transform matrices are 4x4 matrices, but 3x4 work just as well.
To achieve a less blocky look, you can apply two or more different matrices to the same vertex; which results in a blending of the motion, controlling the weight of each blend is an art in and of itself. It can take many hours of time to get weights that work for a particular task, elbows and other sharply bending joints being key offenders.

Here is the current unoptimized method I use for matrix palette skinning:



//This procedure works for any input matrix, and converts it for shader use.
var stupid = mat4.create(); //Replace stupid with your armature-space bone matrix.

//Normally, stupid has translate * scale and rotation applied, and scale is applied last.
//You can pass in any matrix you like, just be aware how your modeling program transforms.

//Create some space
sseB = new Float32Array(16);
sseM = new Float32Array(16);
resM = new Float32Array(16);

//Local armaturespace matrix transposed:
sseB[0] = stupid[0];
sseB[1] = stupid[4];
sseB[2] = stupid[8];
sseB[3] = stupid[1];
sseB[4] = stupid[5];
sseB[5] = stupid[9];
sseB[6] = stupid[2];
sseB[7] = stupid[6];
sseB[8] = stupid[10];

//Original armaturespace matrix transposed:
sseM[0] = BAOrigMatrix[0];
sseM[1] = BAOrigMatrix[3];
sseM[2] = BAOrigMatrix[6];
sseM[3] = BAOrigMatrix[1];
sseM[4] = BAOrigMatrix[4];
sseM[5] = BAOrigMatrix[7];
sseM[6] = BAOrigMatrix[2];
sseM[7] = BAOrigMatrix[5];
sseM[8] = BAOrigMatrix[8];

//Multiplication; Transpose result //012 345 678 => 036 147 258
resM[0] = sseB[0]*sseM[0] + sseB[1]*sseM[1] + sseB[2]*sseM[2];
resM[3] = sseB[0]*sseM[3] + sseB[1]*sseM[4] + sseB[2]*sseM[5];
resM[6] = sseB[0]*sseM[6] + sseB[1]*sseM[7] + sseB[2]*sseM[8];

resM[1] = sseB[3]*sseM[0] + sseB[4]*sseM[1] + sseB[5]*sseM[2];
resM[4] = sseB[3]*sseM[3] + sseB[4]*sseM[4] + sseB[5]*sseM[5];
resM[7] = sseB[3]*sseM[6] + sseB[4]*sseM[7] + sseB[5]*sseM[8];

resM[2] = sseB[6]*sseM[0] + sseB[7]*sseM[1] + sseB[8]*sseM[2];
resM[5] = sseB[6]*sseM[3] + sseB[7]*sseM[4] + sseB[8]*sseM[5];
resM[8] = sseB[6]*sseM[6] + sseB[7]*sseM[7] + sseB[8]*sseM[8];

//Load and transpose it, then multiply by inverse original position
resM[12] = stupid[12] - (resM[0]*BAOrigPosition[0] + resM[3]*BAOrigPosition[1] + resM[6]*BAOrigPosition[2]);
resM[13] = stupid[13] - (resM[1]*BAOrigPosition[0] + resM[4]*BAOrigPosition[1] + resM[7]*BAOrigPosition[2]);
resM[14] = stupid[14] - (resM[2]*BAOrigPosition[0] + resM[5]*BAOrigPosition[1] + resM[8]*BAOrigPosition[2]);

//Replace myPaletteIndex with the bone index (for the matrix palette) you want to upload to.
//Remember that armature bones are mapped to the matrix palette, which is often small (28 max bones).
var locpos = 3*myPaletteIndex;

//Get the location of your uniforms
var uploc0 = myshader.pMatrixPaletteUniforms[ locpos ];
var uploc1 = myshader.pMatrixPaletteUniforms[ locpos + 1 ];
var uploc2 = myshader.pMatrixPaletteUniforms[ locpos + 2 ];

//Upload your converted matrix vec4's:
gl.uniform4fv( uploc0, [resM[0], resM[3], resM[6], resM[12]] );
gl.uniform4fv( uploc1, [resM[1], resM[4], resM[7], resM[13]] );
gl.uniform4fv( uploc2, [resM[2], resM[5], resM[8], resM[14]] );


...in your shader, you can now:


//Input vertex position (model space/armature space initial undeformed)
inpos = vec4( vattribPosition.xyz, 1.0 );

if( vattribWeights.x > 0.0 ){
//Use index cumulation (because we cannot upload integers); ie:
//
// w0,w1,w2 => (w0 + 64*w1 + 64*64*w2); floats have perfect 23 bit accuracy for ints.
//
int idex0 = 3*int(mod( vattribWeightIndex.z, 64.0 ));
//Matrix (used?)
rescur.x = vattribWeights.x * dot( inpos, MatrixPalette[ idex0 ] );
rescur.y = vattribWeights.x * dot( inpos, MatrixPalette[ idex0 + 1 ] );
rescur.z = vattribWeights.x * dot( inpos, MatrixPalette[ idex0 + 2 ] );
poscur = rescur;

if( vattribWeights.y > 0.0 ){

int idex1 = 3*int(mod( vattribWeightIndex.z/64.0, 64.0 ));
rescur.x = vattribWeights.y * dot( inpos, MatrixPalette[idex1] );
rescur.y = vattribWeights.y * dot( inpos, MatrixPalette[idex1 + 1] );
rescur.z = vattribWeights.y * dot( inpos, MatrixPalette[idex1 + 2] );
poscur += rescur;

if( vattribWeights.z > 0.0 ){

int idex2 = 3*int(mod( vattribWeightIndex.z/(4096.0), 64.0 ));
rescur.x = vattribWeights.z * dot( inpos, MatrixPalette[idex2] );
rescur.y = vattribWeights.z * dot( inpos, MatrixPalette[idex2 + 1] );
rescur.z = vattribWeights.z * dot( inpos, MatrixPalette[idex2 + 2] );
poscur += rescur;

...ect
}
}
}

//Apply object transform to final position:
gl_Position = uPMatrix * uMVMatrix * vec4(poscur.xyz, 1.0);


Here's a picture of Lara, imported.

Here's the same Gigginox from before, with slight wobble applied via matrix palette. Note this is the exact same file, I am just changing the import into javascript. TED files contain a lot of information.

Animation data is next.

-Z

WebGL - TED DXML importer preliminary bones

Importer reads armature data and assigns armatures to meshes.

Next step is fixing the matrix palette shader, and the way armature instances are updated so that the animations will play properly.

Lucky for me I've already done all the hard work in other C/C++ programs, so it's just a port really.

Just as a note, it is fairly annoying to both apply animations, save the global game transform, then before drawing convert each global transform into shader ready data by a set of matrix multiplications (read matrix palette skinning articles). But it does work, and it'll have to do until I find better solutions.

Need animations...

-Z

WebGL - Convert chars to float

 All info is derived from:
http://www.khronos.org/registry/typedarray/specs/latest/


The basic problem, is you create a vertex buffer object (VBO) and write vertex data into it, such as vertex position, texture coordinates, normals, and weight and weight index values.
In case you have never used a "matrix palette" the basic idea is:

• Construct a VBO with space for your weights and weight indices (4-8 floats)
• Convert your weight values into floats, store in your array,
• For the index values, convert to uint8 values, which refers to the shader' current matrix palette
• This is defined as an array of uniform matrices; usually less than 28 matrices long
• all GPU bone animation routines do this internally, ultimately.
• Then, before you draw a group in your polygon, load the palette with the matrices needed.
• In the shader, simply multiply with your input vertex to compute the output vertex, and multiply each stage with the weight required.

It doesn't take a of lot of inspection to see multiple problems. For one, you are limited by the number of weights applied to a vertex, and you need multiple shaders for each count, such as matrixshd1, matrixshd2, and so on. You also have to load the uniforms in for each group that changes the matrix palette. Also, depending on your shading pipeline, you have to multiply and convert the matrices before sending to the GPU.

All those problems aside, and you suddenly realize that in javascript, there IS no way to convert bytes into a float. In C, this was typecast trickery.

Here is (one way) convert arbitrary bytes into a float for your VBO data:

function utilUCharToFloat( inarray ){

//Step 1: Create a un-editable array buffer (in bytes) of size 4

var n = new ArrayBuffer(4);

//Step 2: Create a "view" of that buffer as a Uint8 view.
//This allows you to write into it as bytes.
//We choose a 0 offset, and a 4 length for explicitness

var vb = new Uint8Array( n, 0, 4 )

//Write into it with your bytes

vb[0] = inarray[0];
vb[1] = inarray[1];
vb[2] = inarray[2];
vb[3] = inarray[3];

//Create a new view as a float.
//This view uses the same data as the vb view, but is read/written to like a float.

vf = new Float32Array( n, 0, 1 );

//The first element, a 32 bit float (8*4 = 32) is returned,

return vf[0];
}

You can do the exact opposite and convert bytes to float using the same logic.

Hope this helps, it's critical to being able to use matrix palette weighting.

-Z

WebGL - TED DXML importer again

The test file: (Run locally)
http://www.gocaco.com/webgl/test1/Archive.zip

The exporter for blender (Export your own model to test)
http://www.gocaco.com/webgl/igtl_export_ted.py

Instructions:

• Make a new folder somewhere.
• Unzip Archive.zip into it. There will be a .html and a.js file.
• Copy the igtl_export_tex.py into your Blender/scripts directory. (check infopanel for location)
• Open Blender (2.49b?) and open a model.
• In blender, File->Export->IGTL TED (.dxml)
• Change your options, do NOT use JSON mode.
• Copy your exported file to the new folder.
• Run the index.html in your browser (doubleclick it)
• Type in the name of your file in the bottom and click load.
• Now you should be able to use left drag, right drag and left+right drag to view around.

Now I have added in scene loading so it can load in lists of objects, all of which can move/translate/scale.

Also multi-texture is working, but no shaders are coded yet that use it. That'll be after animations, since animations are done in the shader. This is why the TED exporter is valuable, it automatically splits up your mesh into groups by material, seams, and matrix palette seams.

Here's a random scene I made. Don't make any comments about old school games, yes I know what "Dungeons of Drax" is. If the shaders were working it would look better.

Animations next...
-Z

WebGL - TED DXML importer

So I redid some javascript; Now I have a stronger preliminary system for loading data from TED files. It's scary fast, and impressively so. (The GL part anyways)

So, make sure you export ONLY data in the TED format, not the JSON format. Use the TED format because it's a lot easier to read and understand, and it's not much less efficient than a JSON file (Maybe 1% tops).

You must export your own test model for now, from blender. Download the source here to try it on your local machine

TED Exporter (test 2.49b, not yet 2.5* compatible):
http://www.gocaco.com/webgl/igtl_export_ted.py

Download the python file, put it in your blender scripts directory (blender/.blender/scripts for me) or add it on to your 2.5 (not tested, API not stable)

Then open any file you want, and select some objects, then File->Export TED.
You may want to poke through the python a little to read something about what it does.
Save your exported ted file (All ted files are UTF-8 ASCII only, so always human readable) and place it wherever you extracted the source index.html.

Then, just enter in the name of the file and click load, nothing nothing breaks.

You should be able to fly around a (currently non-objected) world with left mouse and right mouse drags, left+right mouse is zoom, left is rotate, right is pan.

I do not yet have the scene objects loaded, nor do I have the animations active. That's next. But this is progress, I can now load data into the webgl, that includes models, textures, and animations.

Here's Gigginox from Monster Hunter 3. Not doing much.

-Z

WebGL - Blender Exporter to JSON format

EDIT (2010-04-03): JSON is a bad format. I'm remove support for it in favor of the XML already in place. XML parsers read DXML easily, so the original TED files remain as is. Reason: JSON cannot handle multiple nodes with the same name. Crippling.


Since I already have a very heavy exporter built for blender, and it just so happens to be in the DXML format, I can directly convert it to a JSON with almost no loss of information.

So, if you are frustrated with not having a prebuilt Blender -> JSON exporter, use mine (or copy + improve on it).

The download link is always on the right hand side of my blog, or here since this post is about the exporter:

http://www.gocaco.com/temp/igtl_export_ted.py

You might wonder "why do I want yet another stupid format". GOOD QUESTION.
The "TED" file is designed to mimic hardware formats.
It exports:

• Optimized mesh data (automatically splits along UV seams, and material seams, and matrix palette splits. Exports vertex, texcoord, normal, tangent, weight, weightindex, maybe colors)
• Material information (As much as possible for now, needs improvement)
• Armature data (all of it, including IK chains)
• Animation data (All actions that fit an armature)
• Frame actions (You can make text files with scripts that can act as data for animations)
• Textures (Can be converted to text and put in the file)
• Scenes (Converted exactly as shown)
• Objects (including game properties! And links to data.)

So, it's also all fairly logical. You will have to play with it to get used to it, especially for you novice's out there. Reading through the exporter (find the "const_DTD_ted" declaration) and looking at what the data is should help. Try exporting a regular TED file first, and open it with any text editor. DXML files are regular, very restrictive XML files to promote consistency and easy of reading. It should be immediately obvious how to read them in.

More importantly, for you beginners, the data exporter from the TED exporter is everything you find normally in any commercial game out there. This includes X360, PS3, Wii games, as I have tested and proven myself.
It does NOT export some advanced features, because blender does not have those features yet. Bother blender.org about that if you want more functionality with muscle/keyframe animation.

If you do end up using this, enjoy. Please drop some credit somewhere; there are no restrictions or guarantees to this script. It's 100% free as in free beer.

A early preliminary test:

I will add a animated test soon.

-Z

WebGL - YES!

First, go here to update your Firefox (or use Chrome):

http://www.mozilla.com/en-US/firefox/all.html

In firefox 4.0, you have to enable this feature manually,

First type in "about:config" in the address bar,
     Don't let it scare you,
     Sort the items by name,
     Find the "webgl.force_enabled" and double click that to make it true.

Now, go find some sweet OpenGL ES 2.0 type demos.

!!! ACHTUNG !!!

--> Make absolutely sure you have the NoScript AddOn (http://noscript.net/) for firefox first. This will allow you to STOP content from automatically loading, just in case you are like me and are trigger happy.

http://www.khronos.org/webgl/wiki/Demo_Repository
http://www.chromeexperiments.com/webgl

This is great because it eliminates the need for stupid plugins like Unity, Flash, Shockwave, Java, ect...
Now you can use the ECMA script (Javascript) built into the browser to make full powered 3D games, which now are compliant to any browsers that use them!

This is serious cool. I'm totally making something in this.

-Z

Synology DS 1511+ - More fun

So far, everything has worked like a champ. The Media server is seen by the PS3, and although the PS3 sucks bawls when it comes to format compatibility, when it CAN play something, it does seamlessly.

We ran a blue ray movie off the Synology, to see how fast it could throughput. The router isn't fast enough to even begin to test, I'll have to get a switch and some better networking equipment before I can.

So, to compensate, we watched a blue ray movie WHILE uploading a blu ray, and had two sources pulling and 1 uploading.

No chokes or anything. A consistent, router limited 10 MB/s.

So that was great.

However, if you setup this unit for FTP, make sure to follow synology's instructions and use filezilla, otherwise you'll get frustrated with other shitty FTP programs.

I still like it so far. It's fast, efficient, very quiet, it's amazingly silent for what it's doing, though the clicking is inevitable in any system.

The temperatures are about 98 98 100 100 99 Fahrenheit in the unit itself under blue ray playing mode. So that's really decent. Ambient room temp is 70 Fahrenheit.

HOWEVER

I encountered one problem, but not with the synology DS1511+. The problem is, VLC.

VLC does not seem to support UPnP/DLNA browsing/playing! What the hell!
I googled this, and found either they do not have enabled, since it seems there are builds with it, or it is not working for just windows.
I was able to download XBMC (oh god I hate it) and it worked fine with the synology, no problems, minus I absolutely hate that media player. VLC spoils you, really!

For example, when watching the dark knight on blue ray, we randomly skipped about and sped up/slowed down the playback and added filters. Still good, though seeking took a moment, naturally. But no longer than I would have expected.

Windows XP Pro seems to SEE the UPNP server, but I don;t have any programs that can browse / pull data off it.

This is a crippling problem, and I want VLC to stream from the NAS.

In unrelated news,

I made the candied citrus peel recipe from the "Desserts By Pierre Herme" awesome cookbook I have; And, OH MY GOD they are amazing. I don;t know what I did wrong, if anything, but caramel + orange + candied is absolutely something else. It's incredibly tasty, and super addicting. Once they all dry, I'll coat them in some crappy ass commercial dark chocolate like Hershey's special dark or something. Maybe.

I'm definitely making these things more often. And I'll quantify the recipe so I can replicate it.

-Z

Synology DS 1511+ - Playing around

So, 10TB of a RAID 6 resulted in 5.44 TB of usable space; and it only took about 7 hours to parity check, maybe 1 to build! If you think about it.

10TB / 8hr => 364 MBps

Which, my poor ass router can't go that fast.

So the thing is fast. I started formatting and toying with settings and stuff, uploading movies and music to test it's integrity.

Pending results from the UPnP test with some PS3's.

Remove the DNS so the box is intranet only, which is awesome.

Here's what it looks like in action, using Synology's DSM web interface:

I'm a little hesitant to enable the transcoding on it, but we'll test it out while it's still fresh.

Great job so far guys, this is the best tech product I've ever used. Props Synology!

-Z

Synology DS 1511+ - Initial Setup

So, the unit arrived this afternoon.

after a greedy session of package unwrapping, I uncovered the DS1511+ itself.

It looks a lot better than the picture.

Everything is exactly as stated. I had no problems opening the packaging, the instructions were all in a .pdf, easy to follow, the drive sleds were a little flimsy feeling, you have to really be gentle with them.

Basically, you unlock the bay, push in the bay, and it pops out making it easy to slide out. Then, you screw in the hard drive (rubber grommets don't fit for noise abatement, need that fix.) And slide the sled back in, and push the clip back down.

It took me a few tries on the first drive to get it push back enough before it would click back, slightly frustrating, but  it only happened the first time and was user error.

With all the drives in place, I carted the unit into the NAS Room, and setup the UPS for the unit.
The UPS is manditory for any kind of storage operation, since if the power fluctuates, you need a battery backup to keep the unit running until it can shut down and suspend proporly.

I plugged it all in, and installed the Disk Manager software.

Now this part is a little hairy, but my systems have auto-everything disabled. So, I manually allows the autorun, and a little splash window popped up, and you click a big install button. Wait a minute, then you detect your NAS.

Initially, nothing. I check the ethernet cable, and I guess it had wiggled out. So be careful with those ethernet cables, make sure you have a good connection physically.

After that, it blinked on, and I logged in.

From there, it got easy. The Linux OS they use is really damn clean. It works through a web browser nicely, I had 0 trouble using it. Was like home.

More interestingly, is that using a OS like that in a web browser (aside from being really stupid imo) was insanely easy to do. Kinda like using a strong VNC/RDP connection.

So, I clicked arround, made some user accounts, setup  the RAID6 array, and let it go ahead and check everybody for errors.

So, it should be done tonight.

I'm impressed with this product. Using their software online as a testbed was a good way to test it, and getting the unit in a flawless configuration was a really nice bonus as well.

The next test will be to load it with data, and yank out a drive to see how it rebuilds.

BUT, I probably don't need to do that. It seems perfectly fine to me, I'd rather make sure the UPnP stuff works for the PS3's and the Mac in the house, so I can stream stuff to em.


SUMMARY PROS:

• Easy Setup Physically
• Amazingly easy software setup
• Works so far
• Unusually quiet
• Able to add DX510 units to expand the unit's capacity greatly (26 TB theoretical! 16 TB probably max due to FS problems)

 SUMMARY CONS:

• Drive bays a little flimsy
• No anti-noise grommets for hard drives, nor is there room to add any
• Uses a web interface (However, it has a VGA port and USB ports, I bet you can run it with no web interface. Not yet tested.)
• Filesystem limitations come into play unless you smartly divide the drive out. Which is really hard to do, initially.

SATISFACTION: Yes

Until next update!

-Z

Synology DS 1511+

So I purchased this baby, it should be here in a week. Maybe more since our atmospheric conditions are trending to the less than favorable end of the spectrum. Loaded with 2 TB drives, I'll configure it as a RAID 6 or equivalent level of protection NAS.
This represents the first serious move I have ever made in the attempt to clean up and organize the vast amount of data I have acquired. Having a HD video camera sucked all the little magnetic pikmin out of my drive platters so fast you'd think it was some sort of monster movie. But, since I have about 3.7 TB of data and almost a literal million files, I figured I needed some way to back up this garbage; the important data only amounts to about 400 GB or so (models, music I made, data, programming) and the critical data still fits on a CD (core code, finances).

So, package wise, this NAS seems to be one of the top products out there. When I get the unit setup, I'll do a full review on it. So far it looks great, and very promising in the software end.


in the meantime, I've been attempting to fix a blender problem, which has been exceedingly difficult for no good reason. (I implemented the fix myself in C/C++ in an hour or two, and got it to work fine) I've been fighting it for over a week, and I'm still not anywhere close to done. Well, that's a lie, but it's not perfect yet.
The script, for those of you who use blender, lets you edit an armature, and it then corrects  your animations after you finish editing. That way, you can model, tweak your armature, make a animation to test, and not lose all that work in the process. You will still lose some, in some cases, since moving bones cannot be corrected; but rotating them or changing their roll can.
I know, it sounds simple right? The problem is the data is stored in local keyframes, as you know:
Parent Matrix * Armature Default Matrix * Pose Matrix = Final Matrix
And you simply solve for Pose Matrix:
Pose Matrix = Armature Default Matrix ^ -1 * Parent Matrix^-1 * Desired Final Matrix
So that applying the pose gets you the desired final matrix.
For whatever weird reason (mostly blenders shitty internal representations of rotations & poorly constructed math utilities, I'm sure many people suffered this) it doesn't. I'll figgure it out eventually. I seem to have rotations working, but not positions since positions apparently come from a different space transform or some silly thing.

Platforming game was going fine, as usual, artist hit RL problems, so I don't know what I'll be doing with it yet.

Played with Unity some. Using Unity makes the Blender Game Engine look good. And that is seriously saying something. And that something is that unity sucks*. But, what do you expect for free? It suffers from the same problems all game engines have. Oh well, it's still fun to mess around in. Did everyone just forget BGE had a webplugin too? Or did Unity just steal their source? Who knows.

Also, work is hard, and we're upgrading what I'm doing again. Fun stuff.

Peace out folks!

-Z