[Soc-2013-dev] Weekly Report #2 Threaded Dependency Graph

[Brecht Van Lommel]

Hi Sergey,

I'm not sure this design with threaded groups is ideal. Maybe it's
temporary, but as I understand it you're trying to divide the work in
advance. This is not going to give good scaling across threads as the
amount of work for each object or group is difficult to predict. I
think the way this should work is that you have a queue of objects
that are ready to be updated, and threads will each pick an object
from it, and when done add any objects to the queue that are now ready
to be updated because all their dependencies are done.

It also more naturally fits with e.g. dupligroups or armatures I
think. You may need to evaluate duplis before you can find out if some
proxy object needs to be updated. Instead of having to figure out all
that stuff in the main thread in advance, a thread can go into the
dupli group and dynamically add more work to the queue as needed. Same
with armatures, if the main thread needs to figure out in advance
which bones need to be evaluated that's no ideal, it's better if a
worker thread can do that and figure it out while other worker threads
already start evaluating other objects.

Some time ago I did a test porting the Cycles task scheduler to C. I
almost forgot this, maybe it's useful to you. It's basically the
mechanism described, you push tasks into a queue and threads pick them
off that queue and spawn their own if needed. It's possible to make a
better implementation, but I think multithreaded graph or game engine
implementations are almost always task queue based, I'm not sure how
else you can do proper load balancing.

http://www.pasteall.org/43629/diff

Brecht.

On Fri, Jun 28, 2013 at 11:58 PM, Sergey Sharybin <sergey.vfx at gmail.com> wrote:
> Hi,
>
> From the code side this week i did:
>
> * Experimental feature to lock the interface while rendering, which locks
> the whole interface forbidding changing any data and preventing viewport
> from running into conflict with render thread.
>
> Also implemented small feature which will allow panning/zooming in image
> editor while interface is locked.
>
> * Worked on running object_handle_update from multiple threads.
>
> Basis code is here, need to be cleaned up a bit, but it works in general.
> However this change unleashed some more areas which are not safe for
> threading, read about this in plans for the next week.
>
> From the documentation point of view, not so much done actually. Made some
> discussions with Campbell about how to keep memory usage sane when copying
> all the scene graph, but it's still some black holes in the design i'd say.
>
> = Next week =
> Would work on making this areas thread safe:
>
> * Curves, which are likely still using some pointers stored in Curve
> datablock, which makes it unsafe to create displist from multiple threads
> when curve is used by multiple objects.
> * Virtual modifier list, which seems to be smallest change for the next
> week.
> * Look into armature modifier and hope to change storing runtime data in
> armature itself, so armature modifier becomes safe for threading.
> * And hope to finish design document for safe and nice local graphs.
>
> = Questions =
> No new question actually,
>
> P.S. Just in case, reports are also there:
> http://wiki.blender.org/index.php/User:Nazg-gul/GSoC-2013/WeeklyProgress
>
> --
> With best regards, Sergey Sharybin
>
> _______________________________________________
> Soc-2013-dev mailing list
> Soc-2013-dev at blender.org
> http://lists.blender.org/mailman/listinfo/soc-2013-dev
>