   3ds Max - DAE animation export - bone matrices
  HiI've been trying to get my animation system working and I'm having a few issues with my Collada importer.  I've created a simple skinned model within 3ds max consisting of a long segmented box and 5 bones.The DAE export within 3ds Max (2011) gives you a visual_scene node which basically contains the skeleton in the form of JOINT nodes.  Underneath each JOINT node there is a 'matrix' element containing a matrix.Inside the controller instance which contains the skinning info and joint information, there is an INV_BIND_MATRIX reference which references a number of matrices (which I assume are the inverse bind matrices per joint).I then take the DAE file exported from Max and fire out an engine-ready binary file.  Everything loads okay, all values are as expected.  When I draw my model using Identity matrices for the joints, everything looks good and also if I add simple rotations to those matrices, the geometry deforms correctly.However, when I use either of the matrices from the DAE file, my results are completely unexpected, geometry ends up everywhere and is clearly not right.  The skeleton matrices (from the Node elements under visual scene), when Col/Row flipped, look vaguely correct (i.e. I can see elements of my original model, but areas of the geometry not linked to the first bone are still messed up).Does anyone know exactly what these matrices that get exported from Max represent and whether they are cumulative or not with respect to the joint hierarchy?  I have tried multiplying them together up the hierarchy but that just makes it worse.  I was under the impression that with skinning, you only needed the inverse bind matrix of each bone but I may be wrong..?Here's a snippet of my collada file - the hierarchy of nodes/joints (all the other skinning info including the INV_BIND_MATRIX is stored within the controller shown referenced):<visual_scene id="RootNode" name="RootNode"> <node id="Box001" name="Box001">   <instance_controller url="#Box001Controller"/> </node> <node id="Bone001" name="Bone001" type="JOINT">   <matrix sid="matrix">-0.846815 -0.531888 -0.000000 24.748735 0.000000 0.000000 -1.000000 -0.000000 0.531888 -0.846815 0.000000 0.045916 0.000000 0.000000 0.000000 1.000000</matrix>   <node id="Bone002" name="Bone002" type="JOINT"><matrix sid="matrix">0.860559 -0.450718 0.237257 9.826459 0.454657 0.889718 0.041109 -0.000000 -0.229620 0.072494 0.970577 0.000001 0.000000 0.000000 0.000000 1.000000</matrix><node id="Bone003" name="Bone003" type="JOINT">  <matrix sid="matrix">0.870005 -0.388983 -0.302957 8.356716 0.429128 0.899972 0.076809 0.000002 0.242775 -0.196831 0.949904 0.000001 0.000000 0.000000 0.000000 1.000000</matrix>  <node id="Bone004" name="Bone004" type="JOINT">    <matrix sid="matrix">0.749533 -0.661815 0.014176 8.358843 0.660073 0.745599 -0.091571 0.000000 0.050033 0.077992 0.995698 0.000000 0.000000 0.000000 0.000000 1.000000</matrix>    <node id="Bone005" name="Bone005" type="JOINT"> <matrix sid="matrix">0.159535 0.987183 0.004340 10.380652 -0.986697 0.159314 0.032373 0.000000 0.031266 -0.009447 0.999466 -0.000000 0.000000 0.000000 0.000000 1.000000</matrix>    </node>  </node></node>   </node> </node>    </visual_scene> Thanks in advance   
  I've done some more research on this and it's got me really confused about skeletons and skinning so hopefully someone could shed some light…(For this, my simple model is created with its origin at world space 0, 0, 0 and I'll call model space MS and world space WS)Firstly, I have only one bone/joint positioned is at MS 50, 0, 0 and pointing directly along the negative x axis (so effectively, and assuming a notional length of 50, it's end would be at MS 0, 0, 0.Now am I correct in thinking that multiplying a vertex in MS by the "inverse bind matrix" of this joint would take a vertex into bone space?  So if I had a mesh vertex point at MS 25, 10, 0 the point should end up at -25, 10, 0…?And that the "bind matrix" of this joint would take a vertex from bone space and move it into MS?  So my vertex that is now in bone space at -25, 10, 0, would move back to MS 25, 10, 0 after being multiplied by the bind matrix?That's my first question, secondly:In order to create a matrix to move my vertices to the correct position with regard to this bone, I need to do this:finalPositionMatrix = bindMatrix * someBoneRotation * inverseBindMatrix Effectively, the matrix moves the vertex to bone space, then applies any necessary rotation (animation), and then moves it back to model space.Is this correct?  If it is correct, then I don't believe the DAE exporter is giving me the correct inverse_bind_matrix values as these don't appear to go from model space into bone space.I'd really appreciate any help.Thanks   ;First of all, I'm no expert on the subject and am just going on my XNA tinkerings. It may well be that FBX and/or XNA approaches things differently.Quote:Original post by RobMaddisonNow am I correct in thinking that multiplying a vertex in MS by the "inverse bind matrix" of this joint would take a vertex into bone space?  So if I had a mesh vertex point at MS 25, 10, 0 the point should end up at -25, 10, 0…?As I understand it the inverse bind matrix serves to 'remove' the bind pose transformation, effectively placing the vertex at the origin. After collapsing relative transformations (basically including parent transformations) a bone matrix contains the absolute transform from the origin and can be applied to vertices that have already been transformed by the inverse bind matrix.Quote:finalPositionMatrix = bindMatrix * someBoneRotation * inverseBindMatrix Is this correct?In my code (which was largely ripped from the MS sample) the transformation matrix is calulated like this:finalPositionMatrix = inverseBindMatrix * someBoneRotationThis seems to match my understanding of the subject in that someBoneRotation contains the absolute transform, which necessitates that the initial bind pose position of the vertices is reset by multiplying them with the inverse bind pose first, or rather by merging the transformations into a single matrix as described above.;Quote:In my code (which was largely ripped from the MS sample) the transformation matrix is calulated like this:finalPositionMatrix = inverseBindMatrix * someBoneRotationThanks for the response.So is 'someBoneRotation' a local rotation about a bone's axis?  If so, how do you get your vertex back to modelspace?More research reveals that the <node> entries in the Collada doc contain the skeleton at bind time.  Each bone down through the hierarchy declares its own local coordinate system.  It looks like the inv_bind_matrix entries are correct, but each bone's inv_bind_matrix is relevant only to the matching bone's matrix in the skeleton hierarchy, so in order to get to bone space, you combine the inv_bind_matrix for a bone with its matching bone in the skeleton.  I was looking at this list of inv_bind_matrix matrices as single entities which is why I got confused.   ;Quote:Original post by RobMaddisonSo is 'someBoneRotation' a local rotation about a bone's axis?  If so, how do you get your vertex back to modelspace?...Each bone down through the hierarchy declares its own local coordinate system. Nope, 'someBoneRotation' should actually be the absolute transformation you want to apply for this particular bone. As the data is read each bone does in fact declare its own local coordinate system, but it's key to note that this coordinate system is relative to the parent transforms all the way up the hierarchy. Clobbed together from C# that makes:[source lang=csharp]// just your regular world matrix for moving the entire model aroundMatrix rootTransform = Matrix.Identity;// worldTransforms[] contains all absolute transforms which will be // used to compute the skinning transforms. boneTransforms[] contains// the transforms read from an animation clip.// First compute the transform for the root bone of the skeletonworldTransforms[0] = boneTransforms[0] * rootTransform;// The bones are ordered by hierarchy, meaning deeper nodes have// higher indices in the arrays. This means we can always refer// to the parentBone index and rely on it already being computedfor (int bone = 1; bone < worldTransforms.Length; bone++){    // this simply retrieves the index of the parent to the current bone    int parentBone = skinningDataValue.SkeletonHierarchy[bone];    worldTransforms[bone] = boneTransforms[bone] *   worldTransforms[parentBone];}// Finally now that we have the absolute transforms, we can incorporate// the per-bone inverse bind post matrix. This could be combined with the// loop above, but XNA keeps the intermediate worldTransforms[] aroundfor (int bone = 0; bone < skinTransforms.Length; bone++){    // skinTransforms[] is the actual matrix array set on the shader to do skinning    skinTransforms[bone] = skinningDataValue.InverseBindPose[bone] *  worldTransforms[bone];}So there you have it. The code basically walks the path back to the root for each bone and constructs the absolute transform per bone. It may help to visualize these transforms as a point at the absolute origin being transformed to the tip of the bone. Next all matrices are premultiplied with the inverse bind pose matrix, which makes sure the original bind pose offsets (which are backed into the initial vertex positions) are removed and all transformations are relative to the origin, as absolute transforms should.I'm not sure what you mean by model space, but when using the identity matrix for the root transform, the model vertices are effectively transformed back into model space. I'm not sure how to explain this better since I'm ending up repeating myself, but perhaps it's useful to consider that the relative bone transformations (boneTransforms[] in the code, ie those you load from an animation) are typically initialized to the regular bindPose matrices. That way you get:// see above, this is not useful code by itselffinalPositionMatrix = inverseBindMatrix * bindMatrix = identityMatrixSince the bindPose offsets are also baked into the vertex positions, this will effectively give you your original bind pose without any special code. This is not particularly useful in a game, but it's as good an initialization as any and it helps to verify the model is looking like it should. The most important take-away here however is that is shows show the bind matrix, inverseBind matrix and base vertex positions are related.If you have any further questions please go ahead and ask them. This is actually a good way for me to find out if I understand it properly myself [smile] ;  Thanks again for the great info, but I do have some more questions!Quote:Nope, 'someBoneRotation' should actually be the absolute transformation you want to apply for this particular boneWhat exactly do you mean by 'absolute transformation'?I understand the concept of your code snippet, i.e. traversing up the hierarchy combining the transforms, but it would be useful to know exactly what 'worldTransforms[n]' are (should be made clearer by the above question hopefully!).I'm also struggling to get my head around this:Quote:Next all matrices are premultiplied with the inverse bind pose matrix, which makes sure the original bind pose offsets (which are backed into the initial vertex positions) are removed and all transformations are relative to the origin, as absolute transforms should.I'm not sure what you mean by 'baked (I assume that's what you meant) into the initial vertex positions'.  And also why you can, after performing the code in the snippet, multiply all matrices by the inverse bind pose matrix.Thanks   ;  Okay I think I'm starting to get it now.  The only thing that I have an issue with now are the inverse bind matrices.  In my setup, each bone has an inverse bind matrix, but that matrix only transforms to world space when it is combined with its counterpart bind matrix (or what I think you mean by worldTransform) in the skeleton hierarchy.  If there are no, as you call them, boneTransforms (i.e. no animation data), I can see how this would work.  But if, say for example, the root bone is rotated (by some animation value) and that rotation is carried down through the skeleton hierarchy, won't that make all inverse bind matrices further down in the hierarchy wrong?In my mind, each inverse bind matrix further down the hierarchy would need to be multiplied by the inverse of the previous local rotations (boneTransform) - I'm sure I'm missing a trick here...   ;Quote:Original post by RobMaddisonWhat exactly do you mean by 'absolute transformation'?I understand the concept of your code snippet, i.e. traversing up the hierarchy combining the transforms, but it would be useful to know exactly what 'worldTransforms[n]' are (should be made clearer by the above question hopefully!).Basically what I mean by this are the transforms you obtain by moving up the hierarchy and combining the transforms. These combined transforms are the absolute transformations for each bone, as opposed to the relative transforms (relative to the bone's parent's transform up the hierarchy) you'd typically get from reading in an animation. Perhaps this will clear things up:The more I look at it the more I think it probably isn't the best example, but let's roll with it to see if it helps. The solid lines indicate the bones and the arrowheads provide some graphical representation of the transformations associated with the bones. In the drawing bone0 is the root bone. Assuming you're using an identity matrix for the world matrix, the boneTransform[0] you have for it is also worldTransform[0]. After all it's the root bone and it doesn't have any parents, so it's absolute by definition.Now if you look to bone1, let's assume that it's parent is bone0. As you can see from the drawing, this relation is not at all obvious from the transformations (in the code, you see that a bone's parent's index is looked up in a seperate table/array). This matches what your investigation pointed out, that each bone defines it's own local coordinate system, ie transform. By convention this local transform is considered relative to the transform of the bone's parent, so we need to walk up the hierarchy to construct the absolute transform, specific to the bone and not dependent on its parent and further ancestors. This is what the purple dashed vector is meant to represent.Another way to think about this is that those local transforms can't be used to transform the vertices later on. Each vertex typically has multiple bone influences, but if we only had local transforms we couldn't combine them since they're in seperate coordinate systems. By going up the hierarchy and combining transforms, we define each bone transform in a common coordinate system. By using the rootTransform like in the code snippet, we end up in world space (which is where the name for the array comes from). If we'd have omitted this or use the identity matrix, we'd end up in model space.Quote:I'm not sure what you mean by 'baked (I assume that's what you meant) into the initial vertex positions'.  And also why you can, after performing the code in the snippet, multiply all matrices by the inverse bind pose matrix.What I meant here is that the transforms specified by the bind pose matrices are conceptually already in your vertex data. When you export any model, the vertices are exported with positions. These positions match the positions the vertices had in your modelling application and for skinned meshes the model is virtually always in the bind pose position. So if you were to render the model without any matrices whatsoever, you'd see the model as it was in the modelling application, that is, in the bind pose.For skinning we don't want our vertices to already be at bindpose positions though. We have constructed absolute world space transforms that specify exactly how vertices should be transformed, so we need the vertices to be in an appropriate base space to apply these. Since the vertices are at bindpose positions, they effectively are where they would be when you would hypothetically apply the absolute bindpose transforms as bone transformations. So to get them back into the space we need, we basically undo the bine pose transformation that is implicitly in the vertex positions, which is done by multiplying the by the inverse bind pose matrix.Since each bone has its own bind pose matrix, each bone also has its inverse bind pose matrix. Conveniently each bone also has its own absolute bone transform, so for each bone we can calculate inverseBindPoseTransform[bone] * absoluteTransform[bone] to get a single matrix (for a particular time in the animation) per bone representing the entire skinning transform it will apply to any vertex under its influence. I'd have to delve a bit in the XNA code to make sure, but this would imply that the inverse bind pose transform I'm using has already been made absolute.I hope this clears things up a bit and isn't utterly confusing [smile];  I really appreciate your help so far and your descriptions really are great, it's just me not grasping it.I think I'm getting wires crossed with boneTransforms and worldTransforms.  In your code snippet, you do this:for (int bone = 1; bone < worldTransforms.Length; bone++){    // this simply retrieves the index of the parent to the current bone    int parentBone = skinningDataValue.SkeletonHierarchy[bone];    worldTransforms[bone] = boneTransforms[bone] *   worldTransforms[parentBone];}Now you've mentioned that 'boneTransforms' represent data from an animation (a local rotation).  If you draw the mesh with no animation applied, all these boneTransforms would be Identities effectively, right?With that said, I'd have thought the code would be:for (int bone = 1; bone < worldTransforms.Length; bone++){    // this simply retrieves the index of the parent to the current bone    int parentBone = skinningDataValue.SkeletonHierarchy[bone];    worldTransforms[bone] = (boneTransforms[bone] * worldTransforms[bone]) *   worldTransforms[parentBone];}... because the code snippet you sent looks like it effectively does the following (for drawing the mesh with NO animation, i.e. Identity boneTransforms):worldTransforms[n] = n's parent worldTransformI don't see any cumulative multiplication there.  Unless boneTransforms contains more than just the animation data?  I know your way obviously works, I'm just trying to understand exactly what 'boneTransforms[bone]' contains.So when you load up all the skinning data from an imported model, you will have a hierarchy of bones, like this:Root bone A - bind matrix = a position and direction | -> Bone B - bind matrix = bone B's pos and dir multiplied by A's bind matrix|-> Bone C - bind matrix = bone C's pos and dir multiplied by B's bind matrixand so on.  Bone C's inverse bind matrix is the inverse of its bind matrix (A * B * C), bone B's inverse bind matrix is the inverse of its bind matrix (A * B), etc?Quote:By convention this local transform is considered relative to the transform of the bone's parent, so we need to walk up the hierarchy to construct the absolute transform, specific to the bone and not dependent on its parent and further ancestors. This is what the purple dashed vector is meant to represent.By 'absolute transform' here do you mean that if you transformed a vertex by (from above) bone C's absolute transform matrix (aggregated bind matrix), it would move into the coordinate space created by the full hierarchy?I'm really sorry for asking so many more questions - you've been more than helpful, I'm just still struggling to grasp it - I think it might be more down to terminology than anything else.Thanks   ;  remigius, I've just carefully re-read your posts a couple of times and everything's finally clicked into place. Thanks so much for taking the time to explain everything so well. I was getting confused because I thought boneTransforms[] contained actual animation data, not the local bind pose matrix of a bone.Thanks again   