WebGL - YES!

First, go here to update your Firefox (or use Chrome):

http://www.mozilla.com/en-US/firefox/all.html

In firefox 4.0, you have to enable this feature manually,

First type in "about:config" in the address bar,
     Don't let it scare you,
     Sort the items by name,
     Find the "webgl.force_enabled" and double click that to make it true.

Now, go find some sweet OpenGL ES 2.0 type demos.

!!! ACHTUNG !!!

--> Make absolutely sure you have the NoScript AddOn (http://noscript.net/) for firefox first. This will allow you to STOP content from automatically loading, just in case you are like me and are trigger happy.

http://www.khronos.org/webgl/wiki/Demo_Repository
http://www.chromeexperiments.com/webgl

This is great because it eliminates the need for stupid plugins like Unity, Flash, Shockwave, Java, ect...
Now you can use the ECMA script (Javascript) built into the browser to make full powered 3D games, which now are compliant to any browsers that use them!

This is serious cool. I'm totally making something in this.

-Z

2D Platforming Game

Well, I had a dream one day, December 1st or something, where I was instructed how to make a platforming game.
I woke up, coded it, and it worked.
DAMN simple too.

Given a list of AABB's in space, and Actors which contain AABB's for motion against the level,
Simply calculate the current bounds and previous bounds (stored)
then detect any EDGE crossings based on the difference from the previous/current bounds for edges;
Then take the minimum edges found, and move the character out from those edges, with some epsilon (in flash, 0.1 works)

To fire a laser in the game, use a space hash (Make a uniform grid of blocks that store references to the objects) and use a specialized edge traversal algorithm, that moves to the next best slope cell, and check everything in that cell until the laser is exhausted or hits something. Handle the case if it changes a major cell, and check the next two to avoid errors.

Both these ideas work PERFECTLY, and thus any 2D platforming game (without ramps so far) is easily made.

Once I clean up the engine, I'll post it so people can make their own high-powered 2D platformers.

-Z

GHOST, Mingl, XCore, Core, XGFX

Continually working on improving all the above components;

Here is a quick overview of what all this garbage is:

ghost - Lets you make programs for X11, Windows, Mac
mingl - Draws stuff in hardware lingo
core - Does everything you don't want to write yourself
xcore - Does things you should be doing anyways
xgfx - Makes everything look pretty + move around

In more details:

GHOST - Made primarily by the Blender.org guys; this is their generic host operating system toolkit; In other words, this is the code that allows you to receive low level input events and a OpenGL context; And the ability to create windows and draw to them/receive messages from them. I've heavily modified it to include some more game-friendly features, timing looks, multithreading and some specialized input event abstractions, joysticks support and other goodies. It's at least as good as SDL, and in many ways, far superior. Hard to find something better than code that has been attacked by many people.

Mingl - "Minimum Graphics Library"; essentially a wrapper around OpenGL; but, it's designed to specifically mimic OpenGL 3.0 specifications, and does an insanely good job of managing more annoying aspects of OpenGL programming; It makes writing graphical routines trivial, where you may spend hours trying to orient yourself in 3D space using any other toolkit (that includes you, blender. F*** your armature space conventions!). However, it's also interface abstracted just like GHOST, so it can use OpenGL, DirectX, software... it doesn't care. But it is primarily designed to expose low level hardware techniques to force you to write more efficient graphics code; and does a ton of things internally (like managing VBO/IBO/RC, states, transformations, ect...). It is still missing FBO operations, but those are trivial if you have a GL 2.0 + system (just need to sit down and do this)

XCore -  Operating system and hacking level tools. A quite extensive library I made of tools I use frequently, such as texture conversions from DXT to / from anything, mathematical algorithms, VA/IA generation algorithms, numerical methods, statistics, even some unique IK algorithms and other tools. IT also contains agressive memory management and a slew of commonly used structures.

Core - The heart of everything; Based on the std C++ library; core contains separate components that should not rely on any one other component, like vec3 vec4, matrix3x3, matrix4x4, index_array, geo, and a lot of other commonly used, but often not needed tools. Chances are if you need any one of these tools, you need all of them.

XGFX - My rendering system built ontop of using mingl as the renderer interface. IS still under construction for the resource abstraction and management of massive resources.

No good screenshots; Just the city level again after implementing multithreaded loading + mingl improvements.

TED Exporting + Mingl

"Walk the Walk"

I talk a lot of smack about things that I do. It's really nice when I visit artists that I can actually pop open their files and show them how to export them so I can load them in game.

It's much easier to convince people you aren't just blowing smoke when you can show them what you are doing in real time. Hard to dispute that kind of presentation.

So, more improvements on the TED file's; the exporter is more robust, and I'm in the process of verifying the data output is correct. So far this seems to be true, I've seen textures, models, scenes, and just about everything in the file working perfectly, especially animations. It's wonderful to just be able to load in a file directly exported from blender, and import it, then see your work immediately and accurately duplicated in a realtime game.

So here are some example shots; the next long haul is building the xgfx resource management system so I can maintain a logical coherence for all of these vastly different resources + files. Resources are essentially reference counted, but are not cleared until flagged to do so. Instance objects exist physically, so once they terminate they just dump all data. Plenty of re-use optimizations, tons of weird paradigms like "resource-source-instance" ideas with meshes; But this will be demonstrated eventually.

In order from top left clockwise;

Rayne from BloodRayne on PC with a static keyframe applied,
two of my custom levels,
Charizard from Super Smash Brothers Brawl on Wii with standard_wobble applied and hacked texture
another custom level,
The first boss from Dragon Blade on Wii, with wobble, note the material colors are showing now; easily removed.
Bowser Trophy from Super Smash Brothers Brawl on Wii
Rayquaza Boss from Super Smash Brothers Brawl on Wii with wobble.

Looks like I need a damn artist.

-Z

Fencing - Problem Solved

4 days of brain bashing of a very complex error to track it down and slay it.

At first, the program would crash if run outside of debug mode.

"Okay, so it must be a memory overrun, and I'm writing past the end of the array, because the debugger puts padding for detection against clobbering memory"; which looks like this:

uchar myarray[ 10 ];
myarray[ 10 ] = 60; //WHOA, myarray holds 10 ELEMENTS, so position 10 is the 11th element.

Often called a fence, I couldn't find it. What was weirder, std::string was throwing errors. std::map was having kittens. and ntdll seemed to vomit all over the floor at this party from hell.

As I burrowed into the code, seething rage ignited; was I being burned by out of date compilers? Was my code wrong? What had I done? What had THEY done wrong?

This project had it all, nested complex templates, deep inheritance heirarchies, large blocks of data and nasty C algorithms. Luckly, most of my (good) code has __debug_regression defined, so I was able to quickly rule out very large segments of code against error with a single look back at passed test dates.

But my mind was going, Dave.

I ignored the fact that if my code worked in debugging, then only the heap allocation method would cause 0xc0000005 errors thanks to ntdll 's nice memory bounding protection (mainly protects it from 0xBAADFOOD but yeah)

So, I eventually upgraded my compiler, my debugger, brought in the help of my custom designed memory heap manager (igtl_MMHeapSystem; I love you, I LOVE YOU! OH GOD YOU'RE SO SEXY MMMMM*codepronz*) and tracked down the offending bug. Hours of poking and prodding landed me with a really surprisingly simple conclusion.

I was off by one.

I'd been accused of being off kilter, off base, and not even on this world, but one? Off by one causes these insanely weird errors? Why did the debugger not detect the stack corruption? Why did the heap manager skip telling me I was writing one integer past my array? Why did absolutely none of the tools find this till I made my own?

Here's why:

When my array was writing past the end of it's detection, it wrote (coincidentally) over the bad food word. But, in the running of the program, that error PROPOGATED until some time later, when the error detection actually scanned memory; because no one in their right mind scans every single memory allocation (super slowness). By the time a scan actually occured, it was too late and bizzaro values had multiplied. But only in a small well contained region. That is, inside of a system dll I don;t have debug points in. Ironically, since the error affected nothing else in MY code, it really screwed up ntdll, and cause it to throw weird errors all over the place.. So, my code was just missing a +1, but the stack was being blow to pieces thanks to trying to protect itself.

Moral of the story?

Make sure to take a nice drink if you run into a hard problem. Usually, it's just something stupid. I use High Gravity Steel Reserve.

-Z

Here's some preliminary results with a nifty 3 instruction toon shader I made: (<3 Mecha Dragon)

Vertex Buffer Objects - speed testing

For drawing 32,768 meshes of 12*16*2 triangles each (1 triangle strip):

OpenGL 2.1 hardware:
VA/IA: ~230ms
VBO: ~30ms

OpenGL 1.4 hardware:
VA/IA: ~230ms
VBO: ~240ms

The note is, although your card may say it supports the VBO extension, or support VBO as per the OpenGL 1.5 specs, it doesn't.

VBO's only work in 1.5 GL's or later, due to hardware restrictions.

Remember; If you worry about optimization, you'd better have timing tests to verify that it even matters. Saving about 10% or more is useful.
Also, small frame optimizations are useless; Anyone can do those, and they generally introduce unmaintainable hacks. Worry about your high level optimizations, like using "most common failed test first" and algorithmic and design changes to maximize your logical speed.

Anyways. Here's a buncha UV spheres, as mentioned (generated using xcore + mingl):



Currently working on the texture functions in mingl. Having to write compressors/decompressors for all the DXT formats. It'll be quite useful though.

Z out. Happy Halloween.

xcode, mingl, xgfx

Continuing my great adventure to construct the engine I need;
Yes Unity is now free. Who cares; What we need is a blender Synthesis program so I can use all my high def animations and levels.

xcore is moving along nicely, most data structure and memory alignment things work perfectly. Still some issues with float hacks.

mingl is quickly being put together, the abstraction of all mesh data into the VBO/IBO/VAO paradigm makes using OpenGL very very nice. In fact, learning a mingl type system is far easier than learning OpenGL fixed pipeline crap.

Vertex Buffer Objects:

They don't help unless you have OpenGL 1.5 or higher. Go figure.
And instancing doesn't really help unless you have a very specialized, small geometry item.
Here's 32,768 cubes:



If you stare too long, you begin to see a n-dimensional triangulation or worse, a shadow of a n-dimensional object.

I'll press on until I can have dancing lizard demo # 4. Or in this case, it might be a higher res female model.

Peace out.
-Z

xcore, mingl, xgfx

Not much to update today; Started some new projects.

In the spirit of the IGTL::Sys classes I made (mostly copy paste from them since they're so dang useful), I am building xcore, which is responsible for:

1. OS level byte types
2. exceptions and error reporting
3. CPU identification, SSE intrinsic math and special instructions
4. Memory manager
5. Specialized templated common file structures, like index_array, carray, index_deque, string_table
6. Directory I/O and file I/O

XCore is the minimal utility toolkit that drives all of my programming. On top of XCore is an important layer called GHOST, which was coded by the blender foundation and furthur extended by me for my own projects. GHOST is a cross platform windowing toolkit that accomplishes some of these things, but not all of them. I'm using it religiously since it avoids stupid SDL licensing issues, and has support for context sharing, multiple windows, and a better event system. The nice thing is, since I use blender anyways to make all of my media, it'll be a cinch to always mention i am using code from blender, thus keeping in line with their license for distribution.

mingl is a minimum wrapper on top of OpenGL that completly abstracts it, preventing me from making any non-suppported OpenGL calls. For example, it wraps all geometry data as a vbo/ibo/vao class (read up on OpenGL 3.0 if you don't know what these are) and internally can down-support any openGL version using this system. It cannot abstract shades really well, so it looks like it will require GL 2.0+ for any shaders to even work. I'm also not sure yet if I need to write my own shader coders so that the engine can generate shaders for vertex/fragment program aas well as GLSL; This seems a little difficult at the moment so I'll defer it.

xgfx is the graphics engine that performs culling, scene management, mesh updating, keyframing, armature animations, lighting, shadowing, and all the regular crap you expect to have in a game engine.

So, with this toolchain of xcore/ mingl/ xgfx /xoal/ xnet, it should abstract all the things I need for any game, just like I did back in late 2005. Except this time, it'll be my engine with no rules and no hangups or lies to slow me down. Of other importance, it's all using C++ as much as possible, so each component should be independent as much as possible.

Z out.

KDTrees and blender gui synthetics

Remember kids, if you use a synthetic oil, it's not a good idea to switch back to regular.



As far as I understand it, "KDTrees" are simply a data structure that says:

def kdtree:
def node:
x,y,w,h
split
low_node *
high_node *
parent *
direction

And, each node occupies a region of space defined by a x,y,w,h box (or x,y,z) and "splits" that box into TWO boxes, along only one of the axes (either x or y). This way, you can more efficiently partition static objects into a scene; this is like all BSP / octree scene management techniques; However, updating a KD tree is rather expensive, it really is good for storing static scenes though as it requires a far lesser node density and can expand into any other volume as needed.

So, they're used in blender's GUI; I'm getting my mockup GUI system to work slowly, and here are many panels defined by glViewports with text in them. The text system is nothing more than a badass texture font I used some free program to generate with 2 px padding and as a alpha only texture, so the memory required for those letters is quite minimal; although fonts are not conducive to mipmapping, so you must manually provide other LOD fonts for your program. Don't forget that always mipmapping is better for your hardware.



I'm also perfecting my cMesh class, which will provide a full next generation mesh including armature animation and animux system, basically equaling and exceeding the capabilities of all existing high definition games; And doing so in a manner consistent with next gen technology (VBO/Shader) while still retaining the ability to be processed via older cards, at great CPU expense.

It looks something like this:



The concept is present in all games I have hacked, and works like this:

A mesh consists of "tiles", which are separate VA/IA's that contain some amount of triangle strips/triangles/quadstrips with a consistent vertex format and ideally contiguous data. This means each tile can store different vertex parameters, the usual culprits are:

current vertex position (3)
next vertex position + weight (4) //For mesh keyframes only
texcoord 0 (2) //1st texture coordinate
texcoord 1 (2) //2nd texture coordinate (if you have seperate UV mappings, not efficient)
normal (3) //Required for any lighting
binormal (3) //can be shader generated, but space may need to exist in VA array)
matrix indicies (4) //Required to skin a mesh that has more than 1 matrix deforming it
matrix weights (4) //Required to skin a mesh properly

So, each tile can behave as a independent mesh. Above a tile is a tile state, which contains the RC (RenderCommand) array for actually issuing the stripping indicies and material changes and matrix palette updates for this mesh.

Above that is the TileTree, so you can swap/zsort/prioritize and use lod tiles as needed.

the concept of a tile is to load all mesh data into your graphic card, and then merely tell the card what to use to render via a few array binding commands. This is extremely efficient, as it supports mesh instancing to a large degree, and prevents transferring data to the card. The only drawback is you still have to transmit matrix updates to the card, but even a 270 bone character (like Valgirt) this is still far less information that a small VA.

So, once you have tiles setup, we have a generic bone/matrix animation system, with a matrix class (yes, I like quat's too, but matrices require less conversions in a game setting; these matricies can be changed to be only state driven mind you.).

The heart of this system is the base animation classes, which consist of "Keys" which are blocks of static data to interpolate between, "Channels" which are a list of keys and times, and "Animations" which combine channels together to form complete animations.

The Animux is a animation multiplexer, which combines any number of animations together, so that you can use multiple types of animations together, for instance "Run_legs" and "Shoot_Torso" like Quake, or "Face_Phoneme_ma" and "Run". This allows you to make characters that can walk, talk, and run + shoot at the same time, and even use IK calculations as you want.

I've probably done this thrice before, but THIS time, I've got it nailed, and have massive amounts of evidence and experience with the new GLSL to support this type of design. Hopefully, I can get AniStar up and running so I can actually have a program that can make animations for given characters.

Z out.

Graphical 'Lag' Problem resolved

Here's the problem:

You code up a nice 3D game, and find that every time you call 'swapBuffers' your program is blocked from execution. You've tried threads, weird API's, and even low level OS hacking tricks; And nothing fixes it! Your game will still be forced to run at 60 FPS, no matter what, and every buffer swap lags your code at least 15 ms! Which is total burned CPU! And tons of dropped packets! And messages!

If you happen to have a graphics card, and specifically a NVidia one, here is the problem:

Go to start->Control Panel-> NVidia Control Panel(Display; Move to the display tab, advanced... Open the Nvidia Control Panel)

Make sure this is off:



What is happening, is it is making a crit section/thread safe block for buffer swaps, which if you write high performance games, this kills your testing phase since your game is force-lagged out.

This easily pushed my FPS and IFPS from 60 to well in the hundreds. In fact, in my GHOST test, I went from 15 RFms with 1 dropped frame to 10 RFms with 0 dropped frames. This translates to 0 dropped frames per second from 60 dropped frames per second.

I have fought this problem for a while; glad it wasn't my problem. The moral of the story? Sometimes you just have to step back and learn something new.

- Imaginary Z

Hacking Success; Dragon Quest Swords broken!

Add:

Godzilla: Unleashed
Rampage: Total Destruction
Dragon Quest Swords
Monster Hunter G
Monster Hunter Tri
Super Smash Bros. Brawl
Carnival Games *
Super Mario Galaxy
NiGHTS
ect...

to the list of games I can extract models from.

I wish I knew who to give credit to, but all I needed was this little function here to finally break apart the .fpk / Dragon Quest Swords compression format (I was really close too, ironically)


typedef unsigned int u32;
typedef unsigned short u16;
typedef unsigned char u8;


int endian=1;

u32 BE32(u32 data)
{
if(endian)
return ( (data<<24) | ((data<<8)&0x00ff0000) |
((data>>8)&0x0000ff00) | (data>>24) );
else
return data;
}

int blen;
int fbuf;

/* PRS get bit form lsb to msb, FPK get it form msb to lsb */
int get_bits(int n, char *sbuf, int *sptr)
{
int retv;

retv = 0;
while(n){
retv <<= 1;
if(blen==0){
fbuf = sbuf[*sptr];
//if(*sptr<256)
//{ printf("[%02x] ", fbuf&0xff); fflush(0); }
(*sptr)++;
blen = 8;
}

if(fbuf&0x80)
retv |= 1;

fbuf <<= 1;
blen --;
n --;
}

return retv;
}

int uncomp(char *dbuf, int dlen, char *sbuf, int slen)
{
int sptr;
int dptr;
int i, flag, len, pos;

blen = 0;

sptr = 0;
dptr = 0;
while(sptr < slen){
flag = get_bits(1, sbuf, &sptr);
if(flag==1){
//if(sptr<256)
//{ printf("%02x ", (u8)sbuf[sptr]); fflush(0); }
if(dptr < dlen)
dbuf[dptr++] = sbuf[sptr++];
}else{
flag = get_bits(1, sbuf, &sptr);
if(flag==0){
len = get_bits(2, sbuf, &sptr)+2;
pos = sbuf[sptr++]|0xffffff00;
}else{
pos = (sbuf[sptr++]<<8)|0xffff0000;
pos |= sbuf[sptr++]&0xff;
len = pos&0x07;
pos >>= 3;
if(len==0){
len = (sbuf[sptr++]&0xff)+1;
}else{
len += 2;
}
}
//if(sptr < 256)
//{ printf("< %08x(%08x): %08x %d> \n", dptr, dlen, pos, len); fflush(0); }
pos += dptr;
for(i=0; i< len; i++){
if(dptr < dlen)
dbuf[dptr++] = dbuf[pos++];
}
}
}

return dptr;
}



Thanks to that mess, I could use my own BRRESToOBM and the Pipeworks1.4ToOBM converters to completely extract the models and import them into blender.

Why do this, you ask?

Simple! Since NOT A SINGLE DAMN PERSON here is competent enough to help me, my only method of QA/QOS/QMS/verification is checking my work against commercial products. It's a very useful method as you often get to see techniques for animation, texturing and shading that are well known to industry but not well documented, or just plain hard to find.

Some of the cool things I've learned:

*Use additional bones inside of the arm joints on your characters; This allows you to retwist the arm visually while still keeping the IK chain unbroken. This is a classic character animation technique that I never had seen in practice.

*Use lots of matrices and bleed the weights out as much as possible, trying not to exceed 4 weights per vertex; Don't just use 0.5/0.5 blending factors (Valgirt has 272 bones...)

*If you are applying a bump/normal map to a character, use the I8A8 texture format and simply calculate the z component in the shader; This costs no additional instructions AND guarantees a normalized z component which you would have to do anyways.

*Use well painted textures, and don;t be afraid to use non-contiguous models (fins, talons, tongues, even neck/finger joints do not always have to be connected to the base mesh!

*Make sure you add in additional transform bones to characters, such as a Root Node and X/Y rotation nodes.

*for mesh modeling, targeting 3k verticies isn't really an issue.

*Always sort your meshes by: Material, Matrix Deformation Pool, Index array type.

*Always convert your final mesh pools into triangle strips if you can; Avoid quads and triangles. If you must use triangles, sort them in a indexed triangle list so that each next triangle shares adjacent verticies. Note that on file loading, you can always convert strips back to triangles; but not the other way around.

*Matrix deformation systems allow you to do things you could never do with bone animation systems; Since you just get a 3x4 float matrix per 'bone', you can apply translation, rotation, scale, and shear transformations. Since the hardware is unaffected by the values you put in those matrices (same number of ops) you're allowed to really have fun animating and be guaranteed consistent results anywhere.

*Mesh keyframes are A-OK and should be combined into your matrix mesh; for example facial poses and stuff. Simply apply those before running your mesh through the shader; You can get very awesome character animations this way.

*Try and avoid making your matricies match a 'skeleton' of you character; This often results in hard mechanical animation that looks not fun. It also makes your artists have to work harder, because a 'bone' does not behave like a matrix. Usually, with matrix deformations, you want the matrix to be at the 'ring center' of the mesh data, so that scaling and transforms will behave as you expect. Interestingly, since matrices have no constraints, you can always make this look like the hard mechanical, but you can't do that the other way around.

*Add additional target matricies; It only costs a small amount of CPU and gives your models flexibility for grapple animations, picking up objects, and easier cinematic animations

*I've seen 11,000 vertex models fully shaded on the Wii. No kidding. With 200 + bones. Think about that for a second. Think about how much more powerful your PC is than that.

*Most importantly, animate/design for fun. Don't make things realistic; It's a waste of your time because commercial studios can always do that better than you can. And who wants to play a realistic game anyways?

I'll post pics later, today I have more beer to acquire!

-Z