[Bf-committers] Notes on Action Baking FTR

[Roland Hess]

The Action Baking patch has been marked as postponed, pending a 
(quite justified) review by Ton of the code (whose quite accurate and 
probably more-than-polite comment was that the implementation is 
"disputable"). This is a detailed, For-The-Record explanation of what 
went into the patch, so that Ton or whoever ends up working on this 
in the future can know better what they're dealing with, whether or 
not they try to rescue this code or start from scratch.

1. Action Baking was in the back of my mind for BlenderPeople, but 
the current implementation came from irc conversations with Toni 
Alatalo, who was working on the same thing for Orange.

2. It was determined that Action Baking needed to do two things: bake 
object-level motion, keyframed or path based, into bone-level 
actions; and bake NLA into a single Action strip.

3. At first, the theory was to simply convert the object-level 
keyframes, or curve points if using path animation, into keys for 
bones in the appropriate bone spaces. It turned out that this does 
not produce correct results. When recreating object-level motion at 
bone level, it is necessary to use a sampling rate (1 sample every 10 
frames produces passable results), in order to achieve good 
approximate motion. Sampled locations can be taken from the object's 
matrix every n frames, converted into bone space, and applied to an 
armature's root bones. You only need to set keys for root bones (or 
bones for which Hinge is set), as the rest of the bones down the 
chain will follow the roots appropriately.

4. There were no NLA blending modes that accurately incorporated this 
Baked motion into the pipeline, so a new one was created: Multiply. 
Currently, Add mode blends bone position and rotations between 
overlapping strips, while the default mode simply replaces the motion 
of bones further back in the chain with those further forward. 
Multiply mode adds the motion, using the previous strips' transform 
as the start of transformation for the subsequent Multiplied strip. 
The addition of this mode allowed motion baked from the object level 
to be used properly in NLA.

5. I quickly realized that building a single Action to encompass 
everything in NLA was a non-trivial task. The goal was to produce as 
few keyframes as possible while retaining the original motion. The 
more keyframes that Baking produced beyond what originally appeared 
in the Actions, the less useful it would be. Here is the methodology 
I ended up with:

NLA Baking Method

A. Create list of all keyframes set throughout NLA, taking into 
account strip scaling, mode (Add, Hold, etc.) and layering.

B. Step through NLA at the keyframes generated by step A.

C. At each step, set keyframes for bones only if they were keyed at 
this location in their original actions (adjusted for NLA 
time/scaling)

D. Append new Action to NLA

Problems presented in this process were:

A. There was no simple way to get a list of keyframes from NLA. The 
following method was devised:
i. Loop through linked list of NLA strips
ii. Create list of keyframes along current strip
iii. Convert list to NLA time
iv. Depending on blending modes and is any future strips obscure this 
one, remove a portion of the keyframes
v. Sort list and remove duplicate entries
vi. Add to master list
vii. Repeat i-vi for all strips
viii. Sort master list and remove duplicate entries

B. Upon stepping through NLA, there was no existing way to determine 
if a certain bone's Ipo value was based on an exact keyframe, or on 
an interpolated value, without paging through each Ipo curve a second 
time, examining each key poisition and adjusting for NLA time. To 
remedy this, the following was done:
i. a "key" flag was added to the SDNA for bones
ii. upon the calculation of bone location/orientation in the already 
existing code, a new check is made to see whether or not the location 
is the result of a direct keyframe, or of interpolation.
iii. if it is the result of a direct keyframe, the key flag is set 
for the appropriate transform of the bone
iv. during NLA Baking, the key flag is used, per bone per frame, to 
determine which keys to set

C. Inaccuracies in NLA sampling were causing keyframes to disappear. 
The actual frame time of a key in a scaled, moved NLA strip might be 
something like 124.62. NLA was set up to only sample on whole 
numbered frames. This caused bad sampling. For this reason, do_nla 
was renamed do_nla_float, and altered to accept a float value for 
it's calculations. do_nla was then added as a wrapper, pointing to 
the do_nla_float function, and passing the current frame value as a 
float, allowing coders to choose between using the old functionality 
(simply calling do_nla), or calling do_nla_float directly, with a 
more exact frame value. I know Ton has a big problem with this, as he 
wants the frame to be the smallest slice of Blender time, and would 
rather solve the time problem some other way, but this was the only 
solution that presented itself for the animators' immediate 
production needs.

6. The interface is fairly simple at the moment. The user selects and 
Armature and an NLA strip, hits Shift-B, then chooses from the popup 
menu: "Bake Object to Action; Bake NLA to Action; Bake All to 
Action". The first bakes only object-level motion into an Action, 
appends it to NLA and sets its mode to Multiply. It also unlinks 
object-level Ipos and clears any parents the object may have had that 
affected it's motion, so as not to have the motion doubled. The 
second bakes the current NLA state into a single Action strip, 
appending it to NLA in Replace mode. The third option does the first, 
followed by the second, in effect Baking both object level and NLA 
motion into a single Action strip.

7. A lack of nicety abounds in the actual implementation:

A. Although I have handled (I think) situations that would cause 
crashes, there are probably a lot of fail conditions that I did not 
address (didn't check all possible Action/NLA combinations for how 
they react to the command, edit mode, etc.).

B. Newly created actions are given crappy default names, and sampling 
resolution for object->bone baking is hard coded. Ideally, something 
like this would have it's own floating panel, where the user could 
set Action names ahead of time, set a resolution, perhaps even 
designate an armature object, then hit a panel button to begin the 
process.

C. Due to rounding errors and problems with some sections of the 
animation code only dealing in whole frame numbers, I had to 
introduce a fudge factor into the section where keyframes flags are 
generated, leading to extra keys in some situations.

D. The determination of when to start baking object-level motion and 
when to stop isn't the best. The current implementation attempts to 
find object-level keyframes, then uses that range. Failing that, it 
uses the range from first NLA key to last. If you're using path 
animation, there is currently no guarantee that your path will fall 
within this range.

E. The current code completely ignores bone scaling. Maybe it 
shouldn't, but honest to Pete, you shouldn't be scaling bones in 
character animation anyway!

F. There is almost certainly a more elegant way to do the function 
world2bonespace that a better mind than mine is invited to come up 
with.

F. I suspect that I abused the portion of the SDNA for Action strips 
that designates strip mode (Add, Replace, Multiply). I do know that 
what I did works. I also know enough to realize that it's not the 
preferred method, but not enough to change it.

G. I believe I eliminated memory leaks from the function that builds 
the list of NLA keyframes, but as this was my first time dealing with 
creating, manipulating and destroying lists, I may have missed 
something.

So, to whoever works on this problem in the future, I hope these 
notes will be helpful!
-- 
Roland Hess - harkyman