[DIRECTX] About X-File (or other) mesh optimization

[mikorians]

Moderator ---Please make this a reference or sticky type of file for posterity---
Tired of exporting raw triangles and want to know how you can optimize your mesh?
Especially those pesky X-Files? We just figured out how to do it by staring at it long enough,
and felt it necessary to explain it in simple terms for those having trouble wrapping their heads
around it -- X-file descriptions and standards weren't very informative at all, so here it is ---
Feel free to offer any tips or corrections, but this is with regards to DirectX version 8 which was
capable of LOADING them automatically.

OPTIMIZING MESHES AND USING VERTEXDUPLICATIONINDEXES IN X-FILES (DIRECTX .X FILES)
VertexDuplicationIndexes is/are meant to indicate where two vertexes have the same position
but different texture coordinates. It is noted that if you get this wrong in any way that
the mesh will probably fail to load properly.

It appears after the MeshMaterialList section in the mesh, and is roughly formatted thus:
1. "VertexDuplicationIndices {"
2. n; n = Actual current vertex count from the 1st mesh section
3. n; n = The quantity of mesh vertexes without any duplicates in the above (or optimal mesh);
3. Vert#n, OR DuplicateOfVert#n, ........
4. Vert#Last;
5. } and closing bracket

By checking the mesh's vertex coordinates we can find duplicates.
Where if any vert position is the same as any other vert, the other vert#
is listed in the duplication index.

Assuming in my example that ALL 6 vertexes have DIFFERENT texture coordinates...
In the 2 tri bowtie object we have:
6; size of 2 tris (2 sets of 3 verts)
5; size of the bowtie (5 verts, with 1 dup)
0, point 0 is point 0 -- the center point (in my mesh example)
1, point 1 is point 1
2, point 2 is point 2
3, point 3 is point 3
1, point 4 is same as point 1 -- center point again
5; point 5 is point 5 -- count continues and is sequential for non-dup and not reduced by 1

In the 2 tri square object we have:
6; size of 2 tris
4; size of the square (4 verts)
0, point 0 is point 0 -- UL corner
1, point 1 is point 1 -- LR corner
2, point 2 is point 2
0, point 3 is same as point 0 -- UL corner
1, point 4 is same as point 1 -- LR corner
5; point 5 is point 5 -- count continues and is sequential and not reduced

Note that the dups can appear anywhere and in any order, just so long as the non-dup sequence is sequential and matches.

A TOTAL MESH OPTIMIZATION PROCEDURE
When making vertex tables, it's important to remember that verts are separate entities from faces, and that faces
refer TO the vertexes, but that vertexes and texture coordinates are TIED TOGETHER and must not be out of order.
Likewise are ALL faces TIED TOGETHER and should be in the same order.

Variable Array Table Examples
A good kind of table for duplicate elimination could read like this:
Verts:
(x,y,z,tu,tv)...
faces:
f1v1,f1v2,f1v3,f2v1,f2v2,f2v3,f3v1,f3v2,f3v3...
See that the face vert numbers can be easily remapped to different vert numbers as a 1D array
Verts can be checked against each other and removed easily since they're in a 2D array.
You'll need a second vert index for cross-referencing, maybe a third for marking dups for deletion.

A. Physically reducing file size first.

1. Carefully build a variable array of all mesh data first so that it may be changed quickly.
2. Reduce material and texture count by eliminating any duplicates and re-mapping the faces in MeshMaterialList.
While you're at it, re-sort all the faces by material ID number (attribute sort)
so that the MeshMaterialList is sequential. This 2nd step is very important to DX efficiency.
Keep track of your Normals faces...
3. Find vertexes with totally matching Positions AND Texture Coordinates and physically REMOVE these duplicates
from the array while remapping the relevant faces to the new compressed vertex table.
Don't forget this will affect any bone influences and skinweights also... So you'll need the Original
Vertex Table Xref'd to this new one...
4. An abbreviation for meshes with only 1 material ONLY is to use:
MeshMaterialList {
1; //<--1 material
0;; //<--Note the 0 for the face count (means all) and the closing extra ";"
Material A {0.341176;0.878431;0.341176;1.000000;;0.099999;1.000000;1.000000;1.000000;;0.039215;0.039215;0.03921 5;;
TextureFilename {"TILEBLU.bmp";}}
}
B. VertexDuplicationIndexes and MeshNormals

1. The Vertex Duplication Index is for vertexes that match in position but have different texture
coordinates on the new final compressed table after step A3.
2. Normals are a separate array of verts and faces. Duplicates can be reduced to even 1 Normal vertex,
just so long as the face count for Normals matches the face count for the mesh.
3. Remember to keep your face count the same in all of the mesh sections, even though the normal faces
point to the set of normal vertexes which don't match the mesh vertexes at all.
DX Indexes normals by face seperately when you load the file.

This SHOULD produce the most efficient and fairly compact output for a text formatted x-file.

Tips: Watch how your materials are applied (wrapped) to your objects as this will drastically affect the number
of texture coordinates that may or may not match each other. If you're flexible, play with it a bit.
If you don't mind facets and a lack of smoothing, use face-based normals instead of vertex-based normals.
This means only that the three vertexes making up the face have the same value instead of separate ones.
Larger numbers of materials/textures applied to a mesh will also slow things down quite a bit.
Textures can be much more taxing than materials, especially larger ones, though this can be debatable, since
you will want to use sprite sheets- lots of little pictures stamped on 1 big one, instead the numerous little ones.
And always, always, keep your mesh as simple as possible to convey the detail you are aiming for.
Reduce polygons over large, flat areas, and keep details where they're important.
Use your 3D Editing package's optimization features to help you.
Learn to settle for less if you want a fast frame rate.
30 years of almost continual work and effort has taught me these things.

Example MaxScript file for 3D Studio Max 5 --- doesn't do any file writing, just prepares the necessary variable arrays and processes them-
Hey! All the hard work done for you!

[mikorians]

Oops. I noticed also a good way to optimize is to also re-order the vertexes in your table by the face order so that the vertexes are in sequential order.
The routine I wrote was too slow for my larger meshes, so I'm going to re-do it in Assy Language. WHY BOTHER? You ask, since DX has an optimze function and so on...
Well, because I noticed it didn't do quite as good a job as Pandasoft's exporter, and I was right to think so. My macro had beaten Pandasoft slightly, so I thought a re-write would be in order.

I do also have a rudimentary material optimizer also if anybody's interested--- A necessary 1st step before we export....
More much later...