2010-05-14

TED file format; Restricted XML

Ever notice how awful XML documents are?

Well, I upgraded my model exporter from blender to 2.49b; so now, instead of the annoying variety of formats I used to use (obm, arf, brf arf2.0, brf3.0, grf, iRF) I decided to spec out a more logical, easier to understand but far more byte/size inefficient DXML format.

I call it "TED" (Text Extracted Data).

The goal of TED files is to export the maximum usable, useful information out of Blender 2.49b+.
  • This includes Meshes, Armatures, Animations, Objects, Materials, Textures, and some scripting and logic elements. Meshes should contain enough data to be GLSL renderable (IE normals, tangents, uv layers, parameters).
  • TED files must also conform to a very strict file standard, so there will be NO allowed coordinate systems other than the default universal correct system (RHR, +X forward, +Y left, +Z up; This is universal for ALL transforms. Matrices also have a standard definition of Xx Xy Xz Yx Yy Yz Zx Zy Zz where X is the forward axis, Y is the left axis, Z is the up axis. Quaternions follow these definitions as well.) 
  • TED files must be easy to read in; Though they are not (byte) efficient, they should contain "hints" for a loader so it may allocate elements beforehand; for example declaring the number of elements in an array before reading it in. You might notice .TED files can be
  • TED files must maintain a logical and render-system friendly data model; And must encapsulate all data elements; No element can have differing types of data; ergo a tag can store an array of strings, an array of ints, an array of floats, or other tags ONLY.
Why TED files? Because my old exporters did not generate shader code or export tangent vectors. Also, the old exporters somewhat obsfuscated the data, and did not present it in a correct hardware manner; There are fixed graphical paradigms in working with 3D polygonal data; TED files will adhere to them strictly and yet maintain enough data to be edit-friendly.

What TED files should NOT be used for:
  • Final game data (you should convert to your own or local host format)
  • Final Level descriptions (TED files do not provide hashing or optimizations or portals of scenes)
  • Replacing something you already understand (unless it has something you need; just make a converter to convert TED to whatever)
TED files could possibly be placed in direct competition with other formats, such as "OGRE XML", "COLLADA", "Alias Wavefront .obj", "Direct X 8.0 .x" and other such lexical, human readable formats. None of those formats contained all of data I wanted to use, therefore I made .TED files. It would be a trivial exercise to convert FROM TED files to those formats, but the other way around is much more difficult due to those other formats inefficiencies and shortfalls in dealing with hardware level data. Yes, I've used all those formats before, and yes I have used and loaded .3ds, .mshp, .md2 .md5 and many many others. So no, the format you are thinking of also sucks.

I have not made the exporter for 2.5.2 yet because I have been burned many times by blender changing specifications mid-stream on me. Luckily, the ted exporter is written to avoid this as much as possible; but when more alpha versions come out I will snap one in easily. Until then, 2.49b it is.


If I could show you a .TED file I would; but HTML can't understand < > signs.

Have a dragon instead (All credit goes to DragonBlade from the Wii; Obviously I'll never use this model, it just happens to be the first example I picked)


Blender on the left, mingl without GLSL on the right (in-game)

-Z

No comments: