   Extracting mesh data from data file
  Hi all,I wonder if any of you have attempted to extract meshes from a game's data file, where the data is in an unknown form. It seems like it would be a fun thing to try. Does anyone have any tips/tutorials for scanning for floating-point values? Any good hex editors for looking at floating point values?For some of the earlier 3D games, I think I might be able to recognize interleaved positional data and tex coords, etc, maybe make a visualization tool. I realize this would be very hard if the data were compressed in any way.:)   
  Most games are built on top of some existing engine, or iterations of some previous code base, where the file format is known. Thus, it's usually possible to get a good head start by knowing where the technology came from.One thing you can do is to put a breakpoint inside CreateVertexBuffer, or VertexBuffer::Lock, and then disassemble the application to see where the data is coming from. It may take a couple of tries of back-tracking, but it ought to get you to the point where the data is being read and parsed from file.Another way to do it is to just trap Unlock(), and rip the decoded data out that way, without having to decode the files.Easiest, of course, is to find a programmer on the game, and ask him :-)Good luck!;  I'll try those, thanks, that should be fun. But I'm talking about older games; I'm actually looking at a Playstation one right now (TMS file). There seem to be a few util's for TMD files, not TMS. But I really just would like to figure one out anyways.   ;  I don't have much experience with extracting meshes, but a few comments:With a bit of practice, it's not that hard to recognise floating-point values in a hex dump - stuff like "00 00 80 3F" is a dead giveaway, and most other numbers are three random-looking bytes typically followed by one like (3|B)(F|E|D|C). Usually there's a big chunk of four-byte values which tend to vaguely follow that pattern, so you can assume they're all floats. (I like BIEW for looking through files, since it's easy to see the high-level detail in the 'binary' mode (more like a text editor, minus line endings) and then change to hex mode to see specific values). (If there's not any visible pattern at all in the file, then it's probably compressed - have fun working out what compression algorithm they used, if it's not zlib...)Some hex editors make it easy to view the data as something more structured than a set of bytes, but I've usually found it easier to write a quick program that reads the data file and jumps over the bits I don't understand and dumps the rest to a text file. (Actually, I wrote (/hacked together) a small language for describing file formats and a program to parse files based on that, which makes it quicker to experiment with things). Then it's obvious when you've got floats lined up correctly, because they're nice sensible values like "0.2979476749897" instead of "6.18519013327578e+034", and you can tweak the program until it's just right. (But I've seen one game (Age of Mythology) where all the data was truncated to 16-bit floats, so you could be trying to read 32-bit floats at two different offsets and still get sensible-looking (but wrong) output. (It was more confusing because the beta (I think) version did store all 32 bits, and it took a while to realise they'd changed it.))If a certain section has values all between -1 and 1, that's probably normals (assuming they're not interleaved with the other data). Depending on the game, there might be one normal/position/texcoord per vertex, in which case you can count the number of normals to see how many vertices there are, then look at the beginning of the file for an int which is about the same number. Then look for the right number of floats for the vertex positions and texture coordinates, and try loading stuff into a 3d modelling program as a cloud of points to see if it looks roughly object-shaped. Triangles might stand out as a load of 2-byte integers, if they're indexes into the vertex data. Then there's all the fun with animation (though maybe they just store the whole mesh for every frame), texturing and other effects, and lots of random junk that they left in the file format even though it's never used because they probably didn't look at their own data files thoroughly in a hex editor [smile]But there's a lot of variety in file formats between different game engines, with varying requirements/effort/competence having gone into them, so the one you're looking at might be completely different to what I've seen. But I still favour the approach of looking for patterns using intuition, automating the process of reading that part of the data once you think you understand some more, then repeating until finished or bored.   ;  Thanks for the very good tips! I think I'll take a look at some binary .X files (they have a compressed mode, so I'm pretty sure their binary mode does no compression), and get some practice.Quote:Original post by ExcorsThen look for the right number of floats for the vertex positions and texture coordinates, and try loading stuff into a 3d modelling program as a cloud of points to see if it looks roughly object-shaped.That was may naive vision of how I would recognize shapes. :) Thanks![Edited by - discman1028 on September 24, 2006 4:56:37 PM];  For completeness-sake, I found a pretty well-expained tutorial for beginners: http://www.xentax.com/downloads/multiex/Definitive_Guide_To_Exploring_File_Formats_MW_2004.pdf( Well, at least for archives... not 3D data specifically )   