Bone animation with normalized bones and raw bones

[alankent]

Hi all!

Advice appreciated!

I am working on a project where I create a scene with a combination of procedural animation (look at), physics (hair bounces with springbone), and bone animation clips (e.g. from mixamo.com) with mixing and blends between clips. I then play everything and take copies of all the bone rotations etc to make a final "baked" animation clip. This final clip I can play back in three.js or export to OpenUSD (so I can render in other tools).

With this "baking" approach, the end result is smooth, and I can scrub the resultant "baked" animation clip and see all the physics correctly (no weird going backwards in time making hair bouncing strangely - the hair bouncing physics was worked out correctly in the first "baking" pass).

So I have two rounds of animation:

1. A first "baking" round with lots of mixers, look-at, physics, etc; and
2. A second round where everything is a single time-sampled animation clip (sampled from the first round).

I have it all working if I animate the raw bones, but then I cannot call vrm.update() as it copies the normalized bones over to the raw bones. Here is the basic approach I am currently trying (which does not work!)

• On the first pass, instead of using J_Bip_C_Neck etc, I animate Normalized_J_Bip_C_Neck.
• Root however is still root (there is no Normalized_Root)
• I used to animate raw blend shapes, but I just moved to VRMExpressions for animating blend shapes for the face as it also animates texture tiling.
• I then capture the raw bone positions and rotations per frame (the raw bones here includes all the hair etc, which are not normalized bones - capturing only normalized bones is not enough), and I also capture the expression weights
• On the second pass I then use the raw bone names (like J_Bip_C_Neck).
  • But do I call vrm.update() (which overrides raw bones with the normalized bones)? (so does not work)
  • Or do I NOT call vrm.update() (which means the expression tiling etc won't be updated.
• I was going to capture raw blend shape names, but I also need texture tiling offsets - I was rely on the vrm.expressionManager to do that. So I am currently capturing expression weights rather than individual blend shapes

But this leaves me in a quandary: for the second pass, I want to call vrm.update() for the facial blend shapes to work, but not have it copy across normalized bone to the raw bones.

I am trying to work out the best way to do this. E.g. can I turn off the normalized to raw bone copying logic for the second playback pass?

[ke456-png]

If synchronization from normalizedHumanBones is not needed, please set VRMHumanoid.autoUpdateHumanBones to false.

[alankent]

Thank you! I think this is probably enough.

I still need to sit down and plan everything through carefully as things are not behaving quite as expected (I thought I was baking hair animations but they don't play back later - I have to use spring bone manager at playback). But it is probably complexity at my end. I need to get the order of things right and clear in my mind, like expressions/blend shape animation, lipsync overrides for the mouth, bone animation (e.g. mixamo clips), look-at animation (needs to be after bone animation to override it), spring bone (needs to be after head turn logic so head turns taken into account?). I then need to work out the right values to capture during baking, then have the right mode at playback time so VRM does not override the animation clip (and also have the information I need to export to USD).

Sorry, talking to myself. In my case I think turning off the humanoid bone updates will help make things more consistent, especially when combining different bone animations together (including non-humanoid bones - like hair and ribbons).