[Bf-animsys] Depsgraph Refactor - GSoC2013 System Design Proposal

[Joshua Leung]

Hi Brecht,

See replies below...

On Sat, Jul 6, 2013 at 8:07 AM, Brecht Van Lommel
<brechtvanlommel at pandora.be> wrote:
> Looks quite good, some questions:
>
> * What is the purpose of ID Groups?

Currently the main purpose is really during the graph building
process, when we use this as a bit of a shortcut/aid for identifying
clusters of datablocks which are mutually dependent on each other - on
a datablock level, these datablocks cannot be evaluated separate from
each other, and it's helpful to be able to keep track of that (if we
also keep track of ID blocks).

Originally though, it served several other purposes:
1) For cyclic dependencies between ID blocks which couldn't be
resolved by the granularity measures, it would act as a convenient
mechanism to say that just the inner nodes from that group could get
evaluated more than once (up to a "max iterations" value set by users
for that scene, or perhaps provided as a property on the affected
datablocks?).

At this stage, I've decided to leave out this idea from the system
since it doesn't seem like such a high priority item (from discussions
last week). However, I should also note that there is one case where
it would be really helpful: many new users creating mechanical rigs
often try using a series of limit distance constraints pointing
between pairs of objects to get them to hold together while still
being free to flex and move. It seems that allowing users to specify
how many times such cycles get evaluated (obviously, we'd try to use
some flag to differentiate between ID groups where there true
unresolved cycles vs phantom cycles), they could still have some
control over the performance of how these setups are resolved, and
that hopefully with most of these setups they will converge to some
reasonable setup.

2) At one stage (even up to when I was preparing the first diagram
showing these nodes in the doc), you couldn't have single ID-block
nodes in the graph. Rather, ID Groups were the only sort of "outer"
nodes which really existed. ID blocks (as shown in the diagrams) at
that stage were simply ID Groups which had only a single ID attached
to them. Things like the "Rigidbody Sim" outer node would thus be an
ID Group with no ID blocks attached to it - it would only have some
inner node(s).

== So, current version ==
Despgraph Node List = [
--+ DepsNode
----+ ID Group [ID1, ID2, ...]
----+ ID (Single ID)
----+ Blackbox Multi-ID Operation (No ID's, but maybe an alternative identifier)

-- + DepsNode
---- + ...

-- + DepsNode
---- + ...
]

== Original version ==
Depsgraph Node List = [
-- + DepsNode/ID Group [(ID1), (ID2), ...]

--+ DepsNode/ID Group [(ID3), (ID4), ...]

-- + ...
]

>
> * Scheduling Nodes for Evaluation: I read this in the libEE paper as
> well, but I don't really understand the usefulness of caching the
> nodes needed for editing some particular value, this seems like a very
> quick operation if the dependency graph is already built. Maybe they
> aren't caching the graph at all?
>

Hmm... Thinking about this a bit, I suspect that this short of caching
might only be really useful in cases where user is doing some kind of
modal property-tweaking/transform operator editing, where this sort of
thing would just amount to saving off the set of nodes that need to be
tagged on every refresh, instead of setting and flushing those flags
everytime from scratch.

It's probably a very minor cost compared with things like recomputing
all mesh geometry for example. Having said that, IIRC some operators
we have do or used to do this, and the flushing step (which would
presumably go over the whole graph looking for tagged nodes to start
flushing from) may be something that could be a source of some small
optimisations if/when they're needed.

> * What is the querying API exactly? Reading this it's not clear to me:
>
>>> Tools should be able to ask the depsgraph questions like: “What objects/datablocks depend on this thing I'm editing?”, or “If I'm updating this piece of data, >> what other objects/drivers/constraints do I need to update to correctly evaluate this?”
>
> Later it mentions that only the outer nodes would be part of the
> querying API, but to me it seems like if e.g. an operator edits an
> object transform, it should be able to tag just the transform as
> modified. Same if you would have e.g. a mesh modifier node, you want
> to only tag the node for that modifier. But as I understand it these
> are inner nodes?
>
> Not all inner nodes maybe be interesting to be able to query, but it
> should be finer then just ID block level I think?
>

Good points.

With the querying API, I was really looking at some way to ask the
despgraph about what other pieces of data were likely to need to be
updated/checked when tools make changes.

Some concrete examples of what I was envisaging it being used for:
1) When renaming a bone, being able to ask the despgraph to provide a
set of PointerRNA's + Props of things that refer to that setting, so
that those places can be checked and edited without manually having to
remember to add each case to ED_bone_rename() or whatever it's called.

2) Making it possible for an armature to check what objects/meshes use
it. This isn't something that's exactly needed now, but for a while
now I've wanted to be able to do this for some nicer onion skinning
controls for characters, where armature ghosting settings can be
dynamically applied over to the rig's geometry...

3) Similarly, checking what dependencies some data has...


So, I guess you might not always want just ID nodes out from queries.
However, I'm slightly hesitant to have it just spewing out all inner
nodes, since some of those are things like
"init_armature_evaluation()" and "do_rigidbody_sim()", which don't
quite provide the expected level of detail.


> * "Filtering", I don't understand what that means here?
>

The idea behind this is that from the main depsgraph, we can pick out
a subgraph/copy of that graph containing just the nodes needed for
computing some result.

So, for example, if we wanted to calculate motion paths for a
particular bone, the filtering capabilities should make it possible to
construct a special-purpose graph that only contains nodes which lead
to the transform matrices/values (i.e. what we need from that bone)
having the correct values. Nodes which lead to other side effects
(such as unnecessary geometry/particles/other bone chains being
calculated, but not actually affecting the bone we're interested in)
are not included, and hence, those kinds of things are more efficient.

Furthermore, most of the time, these filtered graphs should probably
be able to be evaluated separately from the main depsgraph, as most of
the time when you need subsets of the main depsgraph for performing
updating work like this, you're doing so in a background process to
bake results for some specific purpose.


> * Maybe more related to proxies, but I think we should make a clear
> distinction between groups/objects that are instanced (evaluated at
> their original location and duplicated into a new place without any
> evaluation, so fast and memory efficient) or proxied (evaluated in
> their new place, possibly interacting with other objects there).
>

That's true. I probably made it sound like all of them are what you
call "proxied", when in fact, a good number of cases (e.g. a forest of
pine trees, with no special transforms/overrides applied to individual
trees) are actually of the "instanced" variety.


Regards,
Joshua