[Soc-2013-dev] Weekly Report #2 Depsgraph Refactor

Sat Jun 29 21:12:38 CEST 2013

I'm not sure what kind of granularity you think should be avoided
then. As far as I can tell the most important thing is to split
armatures into bones, and objects in object transform, object data and
particle systems. Is that too granular? These are pretty much required
if we want to solve the common dependency cycles, the fact that once
they are split they can be multithreaded is a bonus.

I do think it would be good to keep this system generic so it can
become more granular as needed, but until we have e.g. constraint or
modifier nodes there isn't really that much that could be parallelized
at the dependency graph level. Maybe I missed something in the blog
posts or we are talking about different things?

(For optimal threading we will need to do more parallelization inside
modifiers, but that's not a depsgraph problem so doesn't impact the
design much and would be outside of the scope of these gsoc projects I
think.)

On Sat, Jun 29, 2013 at 8:34 PM, Ton Roosendaal <ton at blender.org> wrote:
> Hi,
>
> I think more of cases with many characters, or many animated entities, or large complex rigs consisting of various hair systems, multiple meshes, etc.
>
> The slowness of animation is not really in the object/bone matrix transforms or driver evaluation code, it's almost all in mesh modifiers anyway.
>
> -Ton-
>
> --------------------------------------------------------
> Ton Roosendaal  -  ton at blender.org   -   www.blender.org
> Chairman Blender Foundation - Producer Blender Institute
> Entrepotdok 57A  -  1018AD Amsterdam  -  The Netherlands
>
>
>
> On 29 Jun, 2013, at 20:27, Brecht Van Lommel wrote:
>
>> Hi,
>>
>> I think if you want to make things optimal threadable then granularity
>> is a requirement I think? If you have 8, 16, 32, 64, .. threads,
>> you're not going to have enough datablocks to keep all those busy
>> unless you split up datablocks.
>>
>> The way I'd like to see this done is that you start from the whole
>> datablock, just have an evaluate function for each type that does the
>> whole thing. Then later it can be refined as necessary. For example
>> for armatures splitting it up into bones is logical and would be done
>> soon. For objects you would probably split it into object transform,
>> object data and particle systems. That solves the most common
>> problems, but it would be nice if the system was generic enough that
>> you could add more granularity later on as needed (e.g. mesh modifier
>> nodes), so not have assumptions about it just being datablocks and
>> bones.
>>
>>
>> Regarding dependency cycles, I wouldn't put too much effort into
>> trying to solve those. We should try to avoid having them in the first
>> place by making the system sufficiently granular. If there's still a
>> cycle then, I suggest no to try to be smart and evaluate things twice
>> to work around it. Otherwise it becomes difficult for riggers to
>> control performance I think, better not do hidden trickery behind the
>> scenes, just evaluate everything once.
>>
>> Brecht.
>>
>> On Sat, Jun 29, 2013 at 7:22 PM, Ton Roosendaal <ton at blender.org> wrote:
>>> Hi Joshua,
>>>
>>> It's certainly a very detailed research there already. I spent an hour on reading it, and it would more time to digest the full flow of though processes.
>>>
>>> What we should prevent though is to have too much complexity here. So I'd like you to set the analysis aside for a moment and think about the actual requirements we aim for.
>>>
>>> BTW:
>>> Requirements: the goals we to achieve
>>> Specifications: how to realize this
>>>
>>> For example, I'm not so much convinced yet if a fine granularity is even a good requirement. Blender's current design in ridid blocks is just a given fact (with its drawbacks), we should and still can live with that.
>>>
>>> Also the cycle solving issue is could be treated by allowing riggers/animators sufficient ways to prevent this - for the useful cases. If you can achieve an animated result avoiding cycles, cycle solving is really secondary.
>>>
>>> I really believe in simplicity here too - which would benefit threading in the end too. The larger the 'blocks' or jobs, the better.
>>>
>>> Here's what I think are the key requirements:
>>>
>>> - Allow dependencies between bones and objects (at least inside group)
>>> - Handle dependencies of (non-Object) RNA data drivers well
>>> - Solve our problems with duplicators and proxy usage
>>> - Make it optimal threadable
>>>
>>> Ease of use, speed. And keep it very simple. :)
>>>
>>> If a user makes weird cyclic dependencies (like material color changing a bone position that drives an lamp that changes material again) you just don't have to support this. That's where post-evaluation scripting hooks can work too.
>>>
>>> -Ton-
>>>
>>> --------------------------------------------------------
>>> Ton Roosendaal  -  ton at blender.org   -   www.blender.org
>>> Chairman Blender Foundation - Producer Blender Institute
>>> Entrepotdok 57A  -  1018AD Amsterdam  -  The Netherlands
>>>
>>>
>>>
>>> On 28 Jun, 2013, at 16:03, Joshua Leung wrote:
>>>
>>>> Hi All,
>>>>
>>>> == This Week ==
>>>> I spent most of the week polishing/writing up a bunch of notes about
>>>> various fragments/topics about various design issues and
>>>> considerations. Links to these are included at the end of this report,
>>>> and are also linked from the main wiki page (TODO: copy over the
>>>> contents to wiki pages; I'll probably leave this for September, when
>>>> I'll almost certainly be fully occupied by a paper submission
>>>> deadline).
>>>>
>>>> In the process of writing these, I now have a clearer perspective on
>>>> some aspects of the design which had previously still been quite
>>>> fuzzy/tricky to pin down (notably, how exactly to group/keep track of
>>>> links between coarse and fine grained nodes).
>>>>
>>>> As an added bonus, we now finally have a nice intro to how the
>>>> internals of the armature evaluation process works (at last! - and
>>>> just before it's soon to get a good shake-up), and some annotated
>>>> reading material for anyone getting into this (dependency graph) work.
>>>> :)
>>>>
>>>>
>>>> == Next Week ==
>>>> I'm currently aiming to have an initial design doc/proposal (maybe not
>>>> fully fleshed out with regards to a few areas) outlining the general
>>>> direction of the system design ready by the end of this weekend (+/-
>>>> 1-2 days). Core devs (and in particular, those involved with the
>>>> various physics sims) and riggers are strongly encouraged to have a
>>>> good look and try to pick up any problems which we may run into. At
>>>> this stage, I'll probably be posting this to the main developers list,
>>>> though it might be better to move/hold the discussion on the
>>>> bf-animsys (http://lists.blender.org/mailman/listinfo/bf-animsys)
>>>> list, which was originally set up for discussions about animation
>>>> system stuff during early 2.5 days.
>>>>
>>>> I'll also start prototyping/mocking up the core data-structures/API's
>>>> for implementing the proposed design. Hopefully I won't encounter any
>>>> significant roadblocks here requiring a serious rethink about where we
>>>> go with this. It's always a possibility though with something like
>>>> this.
>>>>
>>>> Depending on how I go for time, more likely than not, I'll have a few
>>>> more posts up examining some of the trickier issues (e.g. physics
>>>> integration, layer-update stuff).
>>>>
>>>>
>>>> == Questions ==
>>>> Which list should we hold feedback/discussions about the design on?
>>>> bf-developers/committers or bf-animsys? In any case, I'll probably
>>>> just cross-post the initial link(s) across both.
>>>>
>>>>
>>>> == Links ==
>>>> In chronological/logical order:
>>>> * http://aligorith.blogspot.com/2013/06/depsgraph-know-thy-enemy.html
>>>> * http://aligorith.blogspot.com/2013/06/depsgraph-granularity-problem.html
>>>> * http://aligorith.blogspot.com/2013/06/depsgraph-whats-in-node.html
>>>> * http://aligorith.blogspot.com/2013/06/depsgraph-id-groups.html
>>>>
>>>>
>>>> Regards,
>>>> Joshua
>>>> _______________________________________________
>>>> Soc-2013-dev mailing list
>>>> Soc-2013-dev at blender.org
>>>> http://lists.blender.org/mailman/listinfo/soc-2013-dev
>>>
>>> _______________________________________________
>>> Soc-2013-dev mailing list
>>> Soc-2013-dev at blender.org
>>> http://lists.blender.org/mailman/listinfo/soc-2013-dev
>> _______________________________________________
>> Soc-2013-dev mailing list
>> Soc-2013-dev at blender.org
>> http://lists.blender.org/mailman/listinfo/soc-2013-dev
>
> _______________________________________________
> Soc-2013-dev mailing list
> Soc-2013-dev at blender.org
> http://lists.blender.org/mailman/listinfo/soc-2013-dev