   3d format proposal
  Warning: I don´t know if i´m making a fool of myself proposing this but i think it is worth the risk :P.I´ve been always wondering why finding a good 3d format is so hard, since many of them are either too complex and bloated or so simple they´re nearly useless. It is also hard to find info on how to write importers/exporters. Collada for example is an industry standard but it is so generic and complex that several "dialects" exist and you are never sure if your implementation will go boom when someone tries to load a 3dsmax collada instead of a maya collada.Because of all that, I propose to create a simple open format, some sort of "amateur/hobbyist" standard that supports all the usually needed stuff in games (textures! animation data! tangents!), but none of the overly complex things (cameras, lights, ik, shaders). Ideally, it should be the format you use when you need something reliable to import a character into your engine but you don´t want to spend too much time and you are not a 3d format guru.  I propose this: Instead of cramming everything into a mastodontic format, have separate files(like Ogre does): a geometry format, a material format, a skeleton format, a skin weights format and an animation format. --------For geometry/textures, an extension of the .obj file with support for tangents. For materials, something like .mtl. They´re simple enough and they work well.--------For skeletons, a format very much like .obj:rb joint1  -0.5 0.6 0  0 0 0  0 0 -22  1 1 1 b joint2 joint1  1.8 0 0.3  0 0 0  0 0 -66  1 1 1 ...etcEach line begins with rb or b to denote a root bone (parentless) or a bone. Then the name of the bone, the name of the parent bone (if it is not a rb), and then: position xyz, rotation xyz, orientation xyz, scale xyz, the parent bone as coordinate origin.--------For skin weights, another relative of .obj:vw 5  joint1 0.4  joint5 0.6  # 0  # 0 vw 6  joint3 0.2  joint5 0.2  joint6 0.2  joint8 0.4 ..etcAll lines begin with "vw", vertex weight. Then, the vertex number (numbered as they appear in the geometry file), and four weights: <joing name, weight>. If the weight is not used, the joint name is # and the weight is 0. Yes, a maximum of 4 weights because you rarely need more, no sense in making it "generic" if it makes it more complex.--------For animation:b joint1k 1.5  0 0 0  -0.01 0.65 0  1 1 1k 2.3  0 0 0  -0.05 0.68 0  1 1 1b joint2k 1.5  0 0 0  -0.04 0.65 0  1 1 1k 4.8  0 0 0  -0.06 0.62 0  1 1 1...etcSome lines begin with "b", then the name of a bone. That means the beginning of a list of keyframes: "k", then time, then position xyz, rotation xyz, scale xyz.These formats are easy to read/write/understand/debug and it is possible to add more mini-formats if needed (particle system format, 2D tilemap format...), and they won´t pollute the existing ones. I´m willing to write example importers in c++ and java (libraries), and exporters for maya/3dsmax/blender. Obviously a good format specification document and a webpage to hold it all.Do you think it is a good idea? would you benefit from it? would people support such a family of formats?[Edited by - ArKano22 on April 4, 2010 10:40:23 AM]
  Sounds a lot like id's MD5mesh/MD5Anim formats, though I don't think they store tangents.   ;Quote:Original post by InvalidPointerSounds a lot like id's MD5mesh/MD5Anim formats, though I don't think they store tangents.Yep, it is in the same line of simplicity. However this is even simpler (i think) and it would add animation data, not only skeleton/weights. Plus it would be easy to extend and create more files with different data.;  3D is very versatile. Every format that exists was made to serve someone's unique needs. A format isn't bloated if the information is required to be there. A format isn't tool lightweight if it stores all that is needed.It takes 2 seconds to open a scripting window in your 3D app and type out a script to dump the info you need into a file.;Quote:Original post by Daaark3D is very versatile. Every format that exists was made to serve someone's unique needs. A format isn't bloated if the information is required to be there. A format isn't tool lightweight if it stores all that is needed.It takes 2 seconds to open a scripting window in your 3D app and type out a script to dump the info you need into a file.I know. However, there is people that do not want/dont have the time to learn scripting in mel/python/maxscript just to dump the data they need. Plus many people get angry when they ask for an exporter/importer and the answer is "write your own". There is always going to be someone who needs to export some funky spherical harmonics data with their mesh or to add special constraints to their bones, then it is justified to use a custom format. If you want to work with basic data (a mesh with an animation and some textures) the solution usually is some esoteric format with no exporters available, collada,3ds,fbx which aren´t exactly easy to work with (try writing a collada importer with colladaDOM and you´ll enter a world of pain :P), or custom stuff. My point is: wouldn´t it be good to create a well documented, simple format, supported across the most important 3d packages, so that the answer to "what format do i use to get my character in my engine?" changes from "write your own, it is easy" to "use blah and you´re done, here´s the link to the library:"?PD: "bloated" and "lightweight" are relative as you say. But having to write a 3ds importer for many people is overkill, that is what i really mean with bloated: some formats hold data that is usually not needed for hobby purposes.   ;Quote:Original post by ArKano22what i really mean with bloated: some formats hold data that is usually not needed for hobby purposes.If you want the format to be widely adopted, you need to be flexible enough to cover 90% of users' needs.As far as I can tell, on the mesh end that means a allowing arbitrary streams of vertex data to be associated with index data - i.e. the user may need extra sets of texture coords, vertex weights, or some other per-vertex metadata that we don't yet know the use of.   ;Quote:Original post by swiftcoderQuote:Original post by ArKano22what i really mean with bloated: some formats hold data that is usually not needed for hobby purposes.If you want the format to be widely adopted, you need to be flexible enough to cover 90% of users' needs.As far as I can tell, on the mesh end that means a allowing arbitrary streams of vertex data to be associated with index data - i.e. the user may need extra sets of texture coords, vertex weights, or some other per-vertex metadata that we don't yet know the use of.Yes, it would have to allow arbitrary per-vertex data. That´s why plain .objwouldn´t cut it. Many things i wrote in the first post have little sense (i wrote it on the fly), like having a separate skin file instead of putting the weight data with the mesh.I guess i will take a shot at it, and see how it ends up.   ;Quote:Original post by ArKano22Plus many people get angry when they ask for an exporter/importer and the answer is "write your own". Let them get angry.It's not possible to write a universal format.Almost every 3D cgi movie that comes out has to write custom software and file formats to achieve certain effects.Most games will need custom formats to store things exactly how they want/need it. There is no one size fits all here. Often, files will be mapped out directly to in game data structures so they can be loaded as fast as possible, with 0 translation as they are being read.There's no reason to even try to make a standard way of describing 3d data. It's way to simple to write your own format as you need it. People who whine about it won't get very far.;Quote:Original post by DaaarkQuote:Original post by ArKano22Plus many people get angry when they ask for an exporter/importer and the answer is "write your own". Let them get angry.It's not possible to write a universal format.Almost every 3D cgi movie that comes out has to write custom software and file formats to achieve certain effects.Most games will need custom formats to store things exactly how they want/need it. There is no one size fits all here. Often, files will be mapped out directly to in game data structures so they can be loaded as fast as possible, with 0 translation as they are being read.There's no reason to even try to make a standard way of describing 3d data. It's way to simple to write your own format as you need it. People who whine about it won't get very far.You are right. Still, i think it would be kinda cool to help people out with the issues everyone has had when first trying to find a format to use. I´m aware that it is downright impossible to please everyone or create an universal format. But data common to nearly all games is still tricky to export, so i wanted to unify it a bit more, making it simple for people that are just creating their own shader framework and need model loading, for example, or a simple game to test some new concepts... imagine that as part of the driving license exam, they required you to build your own car from scratch...it is fun and didactic, sure, but quite cumbersome. That is what people feel when everyone is driving their own 3d format and they are told to build their own too.thank you for your oppinion :). Maybe it is best to let people build their own car so that they know better what they´re doing with it.   ;  I think COLLADA is the perfect middle ground solution for 3D overall.Most have custom formats for whatever they are doing, but pulling the info out of a package is always different, and there are many ways to do it. COLLADA has plugins for most major packages (http://www.opencollada.org/download.html) , and it is an interchange format, its almost fully featured or maybe is now depending on what youre doing. I say support COLLADA,as the layer between 3D packages and your final purpose for the data.Its XML so can be augmented without destroying the original file, and can be read and changed by hand if need be.Although it is a large spec it has to be to support all that it does. The only problem with it is that its relatively new and still has some gotchas here and there depending on the export/package it came from but with support, tools and time it will only get better. I think it would be better if everyone supports a format like COLLADA then to create another format in a long list of formats.   