[Bf-codereview] Threaded object update and EvaluationContext (issue 15340044)

[sergey.vfx at gmail.com]

Reviewers: bf-codereview_blender.org, brechtvl, ideasman42,

Description:
Made objects update happening from multiple threads. It is a task-based
scheduling system which uses current dependency graph for spawning new
tasks. This means threading happens on object level, but the system is
flexible enough for higher granularity.

Technical details:

- Uses task scheduler which was recently commited to trunk (that one
which Brecht ported from Cycles).
- Added two utility functions to dependency graph:
   * DAG_threaded_update_begin, which is called to initialize threaded
objects update. It will also schedule root DAG node to the queue, hence
starting evaluation process.

     Initialization will fill in DAG nodes valency (valency is how much
dependencies which are not updated yet current node does have). This
valency is used by task threads for faster detecting which nodes might
be scheduled.

   * DAG_threaded_update_handle_node_updated which is called from task
thread function when node was fully handled.

     This function decreases valency of node childs an schedules childs
with zero valency.

   As it might have become clear, task thread receives DAG nodes and
decides which callback to call for it. Currently only
BKE_object_handle_update is called for object DAG nodes.

   In the future it'll call node->callback() from Ali's new dependency
graph.

- This required adding paradigma of derivedRender, which is used by
modifiers when preparing scene for rendering.

   This is mainly needed to stop using get_object_dm() from modifiers'
apply callback. Such a call was only a workaround for dependency graph
glitch when rendering scene with, say, boolean modifiers before
displaying this scene.

   This derivedReder is stored in Object and being created by
database_init_objects and it happens in threads (the same way as it's
done for scene update).

   Unfortunately, this bumps memory usage, but could not see how to
actually solve it atm. using Locked interface from GSoC branch will help
here,

- Added paradigma of EvaluaitonContext. Currently it's more like just a
replacement for G.is_rendering which fails in some circumstances.

   Future idea of this context is to also store all the local data needed
for objects evaluation such as local time, copied objects (by
Copy-on-Write system).

   So now there're two types of EvaluationContext:
   * Context used for viewport updated and owned by Main. in the future
this context might be easily moved to Window or Screen to allow
per-window/per-screen local time.

   * Context used by render engines to evaluate objects for render
purposes. Render engine is an owner of this context.

   This context is passed to all object update routines.

There still might be some TODOs in the patch, but no actual stoppers i
think. And it might be some left-overs from unfinished work in the
branch. Also we might want to have "locked interface rendering" perhaps?

Please review this at https://codereview.appspot.com/15340044/

Affected files (+911, -249 lines):
   b/source/blender/blenkernel/BKE_DerivedMesh.h
   b/source/blender/blenkernel/BKE_anim.h
   b/source/blender/blenkernel/BKE_depsgraph.h
   b/source/blender/blenkernel/BKE_group.h
   b/source/blender/blenkernel/BKE_library.h
   b/source/blender/blenkernel/BKE_main.h
   b/source/blender/blenkernel/BKE_object.h
   b/source/blender/blenkernel/BKE_scene.h
   b/source/blender/blenkernel/BKE_sequencer.h
   b/source/blender/blenkernel/BKE_shrinkwrap.h
   b/source/blender/blenkernel/BKE_smoke.h
   b/source/blender/blenkernel/depsgraph_private.h
   b/source/blender/blenkernel/intern/DerivedMesh.c
   b/source/blender/blenkernel/intern/anim.c
   b/source/blender/blenkernel/intern/blender.c
   b/source/blender/blenkernel/intern/constraint.c
   b/source/blender/blenkernel/intern/depsgraph.c
   b/source/blender/blenkernel/intern/dynamicpaint.c
   b/source/blender/blenkernel/intern/group.c
   b/source/blender/blenkernel/intern/library.c
   b/source/blender/blenkernel/intern/object.c
   b/source/blender/blenkernel/intern/pointcache.c
   b/source/blender/blenkernel/intern/scene.c
   b/source/blender/blenkernel/intern/sequencer.c
   b/source/blender/blenkernel/intern/shrinkwrap.c
   b/source/blender/blenkernel/intern/smoke.c
   b/source/blender/blenloader/intern/readfile.c
   b/source/blender/collada/AnimationExporter.cpp
   b/source/blender/editors/mesh/editmesh_extrude.c
   b/source/blender/editors/mesh/editmesh_utils.c
   b/source/blender/editors/object/object_add.c
   b/source/blender/editors/render/render_internal.c
   b/source/blender/editors/render/render_opengl.c
   b/source/blender/editors/screen/screen_edit.c
   b/source/blender/editors/sound/sound_ops.c
   b/source/blender/editors/space_sequencer/sequencer_draw.c
   b/source/blender/editors/space_view3d/view3d_draw.c
   b/source/blender/editors/space_view3d/view3d_view.c
   b/source/blender/editors/transform/transform_conversions.c
   b/source/blender/editors/transform/transform_snap.c
   b/source/blender/gpu/intern/gpu_material.c
   b/source/blender/makesdna/DNA_object_types.h
   b/source/blender/makesrna/intern/rna_object_api.c
   b/source/blender/makesrna/intern/rna_scene_api.c
   b/source/blender/modifiers/intern/MOD_array.c
   b/source/blender/modifiers/intern/MOD_boolean.c
   b/source/blender/modifiers/intern/MOD_shrinkwrap.c
   b/source/blender/modifiers/intern/MOD_smoke.c
   b/source/blender/modifiers/intern/MOD_util.c
   b/source/blender/modifiers/intern/MOD_util.h
   b/source/blender/render/intern/include/render_types.h
   b/source/blender/render/intern/source/convertblender.c
   b/source/blender/render/intern/source/external_engine.c
   b/source/blender/render/intern/source/pipeline.c
   b/source/blender/windowmanager/intern/wm_event_system.c
   b/source/blender/windowmanager/intern/wm_operators.c
   b/source/blender/windowmanager/intern/wm_playanim.c
   b/source/creator/creator.c
   b/source/gameengine/GamePlayer/ghost/GPG_ghost.cpp