WebGL - TED DXML importer animation test

Animations and animation playback added. Posing is better, glitch with animation frame fixed, Objects retain independent armature state, small speedup.

Current Test:
Test it right now!

http://www.gocaco.com/webgl/test1/Test.zip

Current Exporter:
http://www.gocaco.com/webgl/igtl_export_ted.py

 This pretty much has everything that's in a TED file, which, for your reference, contains:

Mesh data (coords, texcoords, normals, tangents, matrix palette weights, colors, params)
Textures (including texture data if you want)
Materials (some parameters, illy defined for now.)
Armatures (Includes bones, IK chains, armature splines, lots of data)
Animations (Includes channel blending animations, so channels are seperate and integrated. Includes scripts.)
Scenes (all objects and their properties)

Though there are still more things, this is the basics to get started cleaning this mess up and making it easier to work with. I don;t need the BGE if I can load this stuff into WebGL. That's what's really nice.

Although, it has some really painful disadvantages, mainly input problems, no joystick support, bad keyboard input events, no locking the mouse so no FPS possible... and so on.

Here's 9 marios independently running (bad running animation, I know. took me a minute ok??)

And 25 marios...

Most of the speed is CPU <-> GPU delays. Just remember that 1 gpu "talk" takes 6~100's of CPU cycles because of the bus communication. So, this is a "absolute worse case" naive way to do this, and I still get 15 FPS with 100k fully shaded armatured polygons. As a side note, an average frame of Oblivion as ~200k polygons, but most all of them were static (75% static to 25% dynamic).

-Z

WebGL - TED DXML importer bones work; needs animations

I got the bones working using some prior work from xcore/mingl.
Unfortunately, it's difficult to show this without a logically consistent animation; However, if you play with the matrix values you can see it works perfectly.

Current Test:
http://www.gocaco.com/webgl/test1/Test.zip

Current Exporter:
http://www.gocaco.com/webgl/igtl_export_ted.py

You probably have wondered how skinning works; Skinning is a procedure where static mesh data is deformed by adjusting the position, rotation, and scale of a transformation matrix. Usually these transform matrices are 4x4 matrices, but 3x4 work just as well.
To achieve a less blocky look, you can apply two or more different matrices to the same vertex; which results in a blending of the motion, controlling the weight of each blend is an art in and of itself. It can take many hours of time to get weights that work for a particular task, elbows and other sharply bending joints being key offenders.

Here is the current unoptimized method I use for matrix palette skinning:



//This procedure works for any input matrix, and converts it for shader use.
var stupid = mat4.create(); //Replace stupid with your armature-space bone matrix.

//Normally, stupid has translate * scale and rotation applied, and scale is applied last.
//You can pass in any matrix you like, just be aware how your modeling program transforms.

//Create some space
sseB = new Float32Array(16);
sseM = new Float32Array(16);
resM = new Float32Array(16);

//Local armaturespace matrix transposed:
sseB[0] = stupid[0];
sseB[1] = stupid[4];
sseB[2] = stupid[8];
sseB[3] = stupid[1];
sseB[4] = stupid[5];
sseB[5] = stupid[9];
sseB[6] = stupid[2];
sseB[7] = stupid[6];
sseB[8] = stupid[10];

//Original armaturespace matrix transposed:
sseM[0] = BAOrigMatrix[0];
sseM[1] = BAOrigMatrix[3];
sseM[2] = BAOrigMatrix[6];
sseM[3] = BAOrigMatrix[1];
sseM[4] = BAOrigMatrix[4];
sseM[5] = BAOrigMatrix[7];
sseM[6] = BAOrigMatrix[2];
sseM[7] = BAOrigMatrix[5];
sseM[8] = BAOrigMatrix[8];

//Multiplication; Transpose result //012 345 678 => 036 147 258
resM[0] = sseB[0]*sseM[0] + sseB[1]*sseM[1] + sseB[2]*sseM[2];
resM[3] = sseB[0]*sseM[3] + sseB[1]*sseM[4] + sseB[2]*sseM[5];
resM[6] = sseB[0]*sseM[6] + sseB[1]*sseM[7] + sseB[2]*sseM[8];

resM[1] = sseB[3]*sseM[0] + sseB[4]*sseM[1] + sseB[5]*sseM[2];
resM[4] = sseB[3]*sseM[3] + sseB[4]*sseM[4] + sseB[5]*sseM[5];
resM[7] = sseB[3]*sseM[6] + sseB[4]*sseM[7] + sseB[5]*sseM[8];

resM[2] = sseB[6]*sseM[0] + sseB[7]*sseM[1] + sseB[8]*sseM[2];
resM[5] = sseB[6]*sseM[3] + sseB[7]*sseM[4] + sseB[8]*sseM[5];
resM[8] = sseB[6]*sseM[6] + sseB[7]*sseM[7] + sseB[8]*sseM[8];

//Load and transpose it, then multiply by inverse original position
resM[12] = stupid[12] - (resM[0]*BAOrigPosition[0] + resM[3]*BAOrigPosition[1] + resM[6]*BAOrigPosition[2]);
resM[13] = stupid[13] - (resM[1]*BAOrigPosition[0] + resM[4]*BAOrigPosition[1] + resM[7]*BAOrigPosition[2]);
resM[14] = stupid[14] - (resM[2]*BAOrigPosition[0] + resM[5]*BAOrigPosition[1] + resM[8]*BAOrigPosition[2]);

//Replace myPaletteIndex with the bone index (for the matrix palette) you want to upload to.
//Remember that armature bones are mapped to the matrix palette, which is often small (28 max bones).
var locpos = 3*myPaletteIndex;

//Get the location of your uniforms
var uploc0 = myshader.pMatrixPaletteUniforms[ locpos ];
var uploc1 = myshader.pMatrixPaletteUniforms[ locpos + 1 ];
var uploc2 = myshader.pMatrixPaletteUniforms[ locpos + 2 ];

//Upload your converted matrix vec4's:
gl.uniform4fv( uploc0, [resM[0], resM[3], resM[6], resM[12]] );
gl.uniform4fv( uploc1, [resM[1], resM[4], resM[7], resM[13]] );
gl.uniform4fv( uploc2, [resM[2], resM[5], resM[8], resM[14]] );


...in your shader, you can now:


//Input vertex position (model space/armature space initial undeformed)
inpos = vec4( vattribPosition.xyz, 1.0 );

if( vattribWeights.x > 0.0 ){
//Use index cumulation (because we cannot upload integers); ie:
//
// w0,w1,w2 => (w0 + 64*w1 + 64*64*w2); floats have perfect 23 bit accuracy for ints.
//
int idex0 = 3*int(mod( vattribWeightIndex.z, 64.0 ));
//Matrix (used?)
rescur.x = vattribWeights.x * dot( inpos, MatrixPalette[ idex0 ] );
rescur.y = vattribWeights.x * dot( inpos, MatrixPalette[ idex0 + 1 ] );
rescur.z = vattribWeights.x * dot( inpos, MatrixPalette[ idex0 + 2 ] );
poscur = rescur;

if( vattribWeights.y > 0.0 ){

int idex1 = 3*int(mod( vattribWeightIndex.z/64.0, 64.0 ));
rescur.x = vattribWeights.y * dot( inpos, MatrixPalette[idex1] );
rescur.y = vattribWeights.y * dot( inpos, MatrixPalette[idex1 + 1] );
rescur.z = vattribWeights.y * dot( inpos, MatrixPalette[idex1 + 2] );
poscur += rescur;

if( vattribWeights.z > 0.0 ){

int idex2 = 3*int(mod( vattribWeightIndex.z/(4096.0), 64.0 ));
rescur.x = vattribWeights.z * dot( inpos, MatrixPalette[idex2] );
rescur.y = vattribWeights.z * dot( inpos, MatrixPalette[idex2 + 1] );
rescur.z = vattribWeights.z * dot( inpos, MatrixPalette[idex2 + 2] );
poscur += rescur;

...ect
}
}
}

//Apply object transform to final position:
gl_Position = uPMatrix * uMVMatrix * vec4(poscur.xyz, 1.0);


Here's a picture of Lara, imported.

Here's the same Gigginox from before, with slight wobble applied via matrix palette. Note this is the exact same file, I am just changing the import into javascript. TED files contain a lot of information.

Animation data is next.

-Z

KDTrees and blender gui synthetics

Remember kids, if you use a synthetic oil, it's not a good idea to switch back to regular.



As far as I understand it, "KDTrees" are simply a data structure that says:

def kdtree:
def node:
x,y,w,h
split
low_node *
high_node *
parent *
direction

And, each node occupies a region of space defined by a x,y,w,h box (or x,y,z) and "splits" that box into TWO boxes, along only one of the axes (either x or y). This way, you can more efficiently partition static objects into a scene; this is like all BSP / octree scene management techniques; However, updating a KD tree is rather expensive, it really is good for storing static scenes though as it requires a far lesser node density and can expand into any other volume as needed.

So, they're used in blender's GUI; I'm getting my mockup GUI system to work slowly, and here are many panels defined by glViewports with text in them. The text system is nothing more than a badass texture font I used some free program to generate with 2 px padding and as a alpha only texture, so the memory required for those letters is quite minimal; although fonts are not conducive to mipmapping, so you must manually provide other LOD fonts for your program. Don't forget that always mipmapping is better for your hardware.



I'm also perfecting my cMesh class, which will provide a full next generation mesh including armature animation and animux system, basically equaling and exceeding the capabilities of all existing high definition games; And doing so in a manner consistent with next gen technology (VBO/Shader) while still retaining the ability to be processed via older cards, at great CPU expense.

It looks something like this:



The concept is present in all games I have hacked, and works like this:

A mesh consists of "tiles", which are separate VA/IA's that contain some amount of triangle strips/triangles/quadstrips with a consistent vertex format and ideally contiguous data. This means each tile can store different vertex parameters, the usual culprits are:

current vertex position (3)
next vertex position + weight (4) //For mesh keyframes only
texcoord 0 (2) //1st texture coordinate
texcoord 1 (2) //2nd texture coordinate (if you have seperate UV mappings, not efficient)
normal (3) //Required for any lighting
binormal (3) //can be shader generated, but space may need to exist in VA array)
matrix indicies (4) //Required to skin a mesh that has more than 1 matrix deforming it
matrix weights (4) //Required to skin a mesh properly

So, each tile can behave as a independent mesh. Above a tile is a tile state, which contains the RC (RenderCommand) array for actually issuing the stripping indicies and material changes and matrix palette updates for this mesh.

Above that is the TileTree, so you can swap/zsort/prioritize and use lod tiles as needed.

the concept of a tile is to load all mesh data into your graphic card, and then merely tell the card what to use to render via a few array binding commands. This is extremely efficient, as it supports mesh instancing to a large degree, and prevents transferring data to the card. The only drawback is you still have to transmit matrix updates to the card, but even a 270 bone character (like Valgirt) this is still far less information that a small VA.

So, once you have tiles setup, we have a generic bone/matrix animation system, with a matrix class (yes, I like quat's too, but matrices require less conversions in a game setting; these matricies can be changed to be only state driven mind you.).

The heart of this system is the base animation classes, which consist of "Keys" which are blocks of static data to interpolate between, "Channels" which are a list of keys and times, and "Animations" which combine channels together to form complete animations.

The Animux is a animation multiplexer, which combines any number of animations together, so that you can use multiple types of animations together, for instance "Run_legs" and "Shoot_Torso" like Quake, or "Face_Phoneme_ma" and "Run". This allows you to make characters that can walk, talk, and run + shoot at the same time, and even use IK calculations as you want.

I've probably done this thrice before, but THIS time, I've got it nailed, and have massive amounts of evidence and experience with the new GLSL to support this type of design. Hopefully, I can get AniStar up and running so I can actually have a program that can make animations for given characters.

Z out.

Thumbs Up!

Okay, I got it. Matrix Palette exporter works fine now.


Next up is adding in textures in MIF/QIF format (already done, copy paste job), and then deciding the structure for spread-optimized armature/matrix palette, and then keyframes using vertex duplicate's.

Of course, now that THIS works, it'll be time to shove it onto the iPhone. Imagine that, "Monster Game" for the iPhone! Course it'll be stripped down, but hey, sounds fun.

Peace ya'll!

-Z

GL_ARB_vertex_program

As the title states, I've been hardcoring this useful tool in order to combat the 'Dancing Lizards' demo performance ratio of 98% CPU / 16 lizards. Now,

We have 35 x Asty's (Asty is just another fat lizard monster, with 800 verts and 1200 poly count. The original dancing lizard was 300 verts with 400 polys.).

The crazy note is, this is a 10% CPU / 35 fully animated characters, including cell shading, deformers, AND whatever else the hell I want to do. This is plenty of performance.
However, if you actually look at this ugly screenshot, you'll notice the statistics in the upper left seem to contradict me; be patient, the IFPS is what is important (~16) which are how many inter-frames there are (1 ms) between render frames. Since all my apps are capped to exactly 50 FPS internal, this means there are 16 ms in there that the game can run and poll for whatever, which is what it does.

Now for some nitty gritty. Let's say you have Asty as a creature in your game (he's quite a friendly fellow!) and you want to make him all sorts of animated, and you overkill the bone count (most of my models have ~120 or so bones, including fingers and IK stuff). Sadly, in the shader model, you cannot feasibly have more than ~28 bones in your shader program at once. (96 parameters available max) This means you have to preprocess groups of verticies that share bones (4 x shader counts, 1 for 1 matrix, 1 for 2 matrix, ect...) so you can actually do all the deformations. This means switching shader programs, which is costly. So, if you want a bunch of fodder enemies, you'll be needing to create some interesting optimization schemes to lower the bone count so you can avoid switching.

In this screenshot, each monster has every vertex with a GLubyte[4] for matrix local palette index (0 to 96, divide by 3) and GLfloat[4] for the weight. I intend to normalize this weight value so I can use bytes, as you generally don't need that accurate of a weigth float. Also, each vertex is renormalized so I can calculate the nifty cell shading value (sum normal dot eye normal = tex coord 0).

Let's look at some assembly:


PARAM K = {1, 0.5, 0, 3.141525968 };

#Skin with any affine matrix:
#PARAM Matrices[] = { program.local[12..84] } <= 28 matricies # [ Xx, Yx, Zx, x pos(1) ] # [ Xy, Yy, Zy, y pos(1) ] #Multiply each axis by it's scale to use scaling, but this will require renormalization for normals. # [ Xz, Yz, Zz, z pos(1) ] #This matrix is CPU computed from: # Let B be the current bone matrix (local to mesh, current pose) # Let M be the original bone matrix (local to mesh, default pose) # Then, W = (B * M^-1) # Send W to shader as matrix in correct form as above #Per vertex in a matrix deformed mesh, # vertex.attrib[6] store n indicies as unsigned bytes (n*1 bytes) # vertex.attrib[7] store n weights as floats (n*4 bytes) #? can we use byte weights? #Requires: RF, R, aMN; ( RF = sum, R = temp, addr = address ) ALIAS R = temp1; #Temp vector ALIAS RF = temp2; #Vector sum (always set to first matrix deform) ALIAS RFN = temp3; #Normal sum (always set to first matrix deform) #For matrix 1 (x)## ARL addr.x, vertex.attrib[6].x; #Get matrix array index DP4 R.x, Matrices[addr.x + 0], wPos; #Rotate & scale local vector (model position) DP4 R.y, Matrices[addr.x + 1], wPos; DP4 R.z, Matrices[addr.x + 2], wPos; MUL RF, R, vertex.attrib[7].x; #Multiply vector by weight, add to summated deformation vector DP3 R.x, Matrices[addr.x + 0], wNorm; #Rotate normal as needed (don't forget about scale...) DP3 R.y, Matrices[addr.x + 1], wNorm; DP3 R.z, Matrices[addr.x + 2], wNorm; MUL RFN, R, vertex.attrib[7].x; #Sum normal with weight as well ################## #For matrix 2 (y)## ARL addr.x, vertex.attrib[6].y; #Get matrix array index DP4 R.x, Matrices[addr.x + 0], wPos; #Rotate & scale local vector (model position) DP4 R.y, Matrices[addr.x + 1], wPos; DP4 R.z, Matrices[addr.x + 2], wPos; MAD RF, R, vertex.attrib[7].y, RF; #Multiply vector by weight, add to summated deformation vector DP3 R.x, Matrices[addr.x + 0], wNorm; #Rotate normal as needed DP3 R.y, Matrices[addr.x + 1], wNorm; DP3 R.z, Matrices[addr.x + 2], wNorm; MAD RFN, R, vertex.attrib[7].y, RFN; #Sum normal with weight as well ################## #For matrix 3 (z)## ARL addr.x, vertex.attrib[6].z; #Get matrix array index DP4 R.x, Matrices[addr.x + 0], wPos; #Rotate & scale local vector (model position) DP4 R.y, Matrices[addr.x + 1], wPos; DP4 R.z, Matrices[addr.x + 2], wPos; MAD RF, R, vertex.attrib[7].z, RF; #Multiply vector by weight, add to summated deformation vector DP3 R.x, Matrices[addr.x + 0], wNorm; #Rotate normal as needed DP3 R.y, Matrices[addr.x + 1], wNorm; DP3 R.z, Matrices[addr.x + 2], wNorm; MAD RFN, R, vertex.attrib[7].z, RFN; #Sum normal with weight as well ################## #For matrix 4 (w)## ARL addr.x, vertex.attrib[6].w; #Get matrix array index DP4 R.x, Matrices[addr.x + 0], wPos; #Rotate & scale local vector (model position) DP4 R.y, Matrices[addr.x + 1], wPos; DP4 R.z, Matrices[addr.x + 2], wPos; MAD RF, R, vertex.attrib[7].w, RF; #Multiply vector by weight, add to summated deformation vector DP3 R.x, Matrices[addr.x + 0], wNorm; #Rotate normal as needed DP3 R.y, Matrices[addr.x + 1], wNorm; DP3 R.z, Matrices[addr.x + 2], wNorm; MAD RFN, R, vertex.attrib[7].w, RFN; #Sum normal with weight as well ################### MOV wNorm, RFN; #Set final normal DP3 R, wNorm, wNorm; #Renormalize normal after deformations (extremely iffy) RSQ R, R.x; MUL wNorm, wNorm, R; MOV wPos.xyz, RF; #Set final position of deformation (could use non-normal weights too...) And that's all you need to make a skinned model in OpenGL using the common ARG vertex program extension. Naturally, people are idiots, and will ask 'why do this instead of GLSL?' and 'This is too old to be useful'. Obviously, if you can do it using the card's assembly language, it's a cinch to move to higher level languages. In fact, it's a incredibly good excercise in understanding not only SIMD instruction stuff, but also general matrix/vector processing units. Plus, this is damned fast and way more portable than GLSL. As some side notes, I have a lot of other nifty shader code, specifically for this things I find important, like: Normal Map colors Cast deformations (spherical) Push-Cast deformations (point -> sphere outward from point)
Water/Perlin noise wobbling
Texture-coordinate lighting
Multiple light cheats
More shading and lighting models

More to come once I give Asty a soul using my IK algorithms and rig him up proper with lotsa delicious bones. THEN we'll see how much CPU we get, and who knows, maybe we'll even get a game demo where you can SSBM other Asty's.

*Note, I also forgot this demo tests a sphere-triangle collision with EVERY loaded triangle. Thus, fixing that, the CPU usage was at MOST 1% for 25 Asty's. So hah. I can't wait to have 1000 lizards dancing. Eat it, GEICO!

-Z