[Bf-blender-cvs] [802027f3f8f] blender2.8: Depsgraph: Initial groundwork for copy-on-write support
Sergey Sharybin
noreply at git.blender.org
Mon Jun 19 13:33:16 CEST 2017
Commit: 802027f3f8f9a83a77134a2b104a25ff3a4ac013
Author: Sergey Sharybin
Date: Wed Jun 14 10:26:24 2017 +0200
Branches: blender2.8
https://developer.blender.org/rB802027f3f8f9a83a77134a2b104a25ff3a4ac013
Depsgraph: Initial groundwork for copy-on-write support
< Dependency graph Copy-on-Write >
--------------------------------
\ ^__^
\ (oo)\_______
(__)\ )\/\
||----w |
|| ||
This is an initial commit of Copy-on-write support added to dependency graph.
Main priority for now: get playback (Alt-A) and all operators (selection,
transform etc) to work with the new concept of clear separation between
evaluated data coming from dependency graph and original data coming from
.blend file (and stored in bmain).
= How does this work? =
The idea is to support Copy-on-Write on the ID level. This means, we duplicate
the whole ID before we cann it's evaluaiton function. This is currently done
in the following way:
- At the depsgraph construction time we create "shallow" copy of the ID
datablock, just so we know it's pointer in memory and can use for function
bindings.
- At the evaluaiton time, the copy of ID get's "expanded" (needs a better
name internally, so it does not conflict with expanding datablocks during
library linking), which means the content of the datablock is being
copied over and all IDs are getting remapped to the copied ones.
Currently we do the whole copy, in the future we will support some tricks
here to prevent duplicating geometry arrays (verts, edges, loops, faces
and polys) when we don't need that.
- Evaluation functions are operating on copied datablocks and never touching
original datablock.
- There are some cases when we need to know non-ID pointers for function
bindings. This mainly applies to scene collections and armatures. The
idea of dealing with this is to "expand" copy-on-write datablock at
the dependency graph build time. This might introduce some slowdown to the
dependency graph construction time, but allows us to have minimal changes
in the code and avoid any hash look-up from evaluation function (one of
the ideas to avoid using pointers as function bindings is to pass name
of layer or a bone to the evaluation function and look up actual data based
on that name).
Currently there is a special function in depsgraph which does such a
synchronization, in the future we might want to make it more generic.
At some point we need to synchronize copy-on-write version of datablock with
the original version. This happens, i.e., when we change active object or
change selection. We don't want any actual evaluation of update flush happening
for such thins, so now we have a special update tag:
DEG_id_tag_update((id, DEG_TAG_COPY_ON_WRITE)
- For the render engines we now have special call for the dependency graph to
give evaluated datablock for the given original one. This isn't fully ideal
but allows to have Cycles viewport render.
This is definitely a subject for further investigation / improvement.
This call will tag copy-on-write component tagged for update without causing
updates to be flushed to any other objects, causing chain reaction of updates.
This tag is handy when selection in the scene changes.
This basically summarizes ideas underneath this commit. The code should be
reasonably documented.
Here is a demo of dependency graph with all copy-on-write stuff in it:
https://developer.blender.org/F635468
= What to expect to (not) work? =
- Only meshes are properly-ish aware of copy-on-write currently, Non-mesh
geometry will probably crash or will not work at all.
- Armatures will need similar depsgraph built-time expansion of the copied
datablock.
- There are some extra tags / relations added, to keep things demo-able but
which are slowing things down for evaluation.
- Edit mode works for until click selection is used (due to the selection
code using EditDerivedMesh created ad-hoc).
- Lots of tools will lack tagging synchronization of copied datablock for
sync with original ID.
= How to move forward? =
There is some tedious work related on going over all the tools, checking
whether they need to work with original or final evaluated object and make
the required changes.
Additionally, there need synchronization tag done in fair amount of tools
and operators as well. For example, currently it's not possible to change
render engine without re-opening the file or forcing dependency graph for
re-build via python console.
There is also now some thoughts required about copying evaluated properties
between objects or from collection to a new object. Perhaps easiest way
would be to move base flag flush to Object ID node and tag new objects for
update instead of doing manual copy.
here is some WIP patch which moves such evaluaiton / flush:
https://developer.blender.org/F635479
Lots of TODOs in the code, with possible optimization.
= How to test? =
This is a feature under heavy development, so obviously it is disabled by
default. The only reason it goes to 2.8 branch is to avoid possible merge
hell.
In order to enable this feature use WITH_DEPSGRAPH_COPY_ON_WRITE CMake
configuration option.
===================================================================
M CMakeLists.txt
M intern/cycles/blender/blender_shader.cpp
M source/blender/blenkernel/CMakeLists.txt
M source/blender/blenkernel/intern/object.c
M source/blender/blenkernel/intern/object_update.c
M source/blender/depsgraph/CMakeLists.txt
M source/blender/depsgraph/DEG_depsgraph.h
M source/blender/depsgraph/DEG_depsgraph_query.h
M source/blender/depsgraph/intern/builder/deg_builder.cc
M source/blender/depsgraph/intern/builder/deg_builder_nodes.cc
M source/blender/depsgraph/intern/builder/deg_builder_nodes.h
M source/blender/depsgraph/intern/builder/deg_builder_nodes_layer.cc
M source/blender/depsgraph/intern/builder/deg_builder_nodes_scene.cc
M source/blender/depsgraph/intern/builder/deg_builder_relations.cc
M source/blender/depsgraph/intern/builder/deg_builder_relations.h
M source/blender/depsgraph/intern/builder/deg_builder_relations_scene.cc
M source/blender/depsgraph/intern/debug/deg_debug_graphviz.cc
M source/blender/depsgraph/intern/depsgraph.cc
M source/blender/depsgraph/intern/depsgraph.h
M source/blender/depsgraph/intern/depsgraph_build.cc
M source/blender/depsgraph/intern/depsgraph_query.cc
M source/blender/depsgraph/intern/depsgraph_tag.cc
M source/blender/depsgraph/intern/depsgraph_type_defines.cc
M source/blender/depsgraph/intern/depsgraph_types.h
A source/blender/depsgraph/intern/eval/deg_eval_copy_on_write.cc
A source/blender/depsgraph/intern/eval/deg_eval_copy_on_write.h
M source/blender/depsgraph/intern/eval/deg_eval_debug.cc
M source/blender/depsgraph/intern/eval/deg_eval_flush.cc
M source/blender/depsgraph/intern/nodes/deg_node.cc
M source/blender/depsgraph/intern/nodes/deg_node.h
M source/blender/depsgraph/intern/nodes/deg_node_component.cc
M source/blender/depsgraph/intern/nodes/deg_node_component.h
M source/blender/depsgraph/intern/nodes/deg_node_operation.cc
M source/blender/depsgraph/intern/nodes/deg_node_operation.h
M source/blender/editors/object/object_edit.c
M source/blender/makesdna/DNA_object_types.h
M source/blender/makesrna/intern/rna_depsgraph.c
===================================================================
diff --git a/CMakeLists.txt b/CMakeLists.txt
index a4a668fcdee..3da27fb52d5 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -510,6 +510,10 @@ if(CMAKE_COMPILER_IS_GNUCC)
mark_as_advanced(WITH_LINKER_GOLD)
endif()
+# Dependency graph
+option(WITH_DEPSGRAPH_COPY_ON_WRITE "Build Blender with copy-on-write support for dependency graph" OFF)
+mark_as_advanced(WITH_DEPSGRAPH_COPY_ON_WRITE)
+
if(WIN32)
# Use hardcoded paths or find_package to find externals
option(WITH_WINDOWS_FIND_MODULES "Use find_package to locate libraries" OFF)
diff --git a/intern/cycles/blender/blender_shader.cpp b/intern/cycles/blender/blender_shader.cpp
index bdbab1006c0..c96727d5f64 100644
--- a/intern/cycles/blender/blender_shader.cpp
+++ b/intern/cycles/blender/blender_shader.cpp
@@ -1177,13 +1177,18 @@ void BlenderSync::sync_materials(bool update_all)
{
shader_map.set_default(scene->default_surface);
- /* material loop */
- BL::BlendData::materials_iterator b_mat;
-
TaskPool pool;
set<Shader*> updated_shaders;
- for(b_data.materials.begin(b_mat); b_mat != b_data.materials.end(); ++b_mat) {
+ /* material loop */
+ BL::BlendData::materials_iterator b_mat_orig;
+ for(b_data.materials.begin(b_mat_orig);
+ b_mat_orig != b_data.materials.end();
+ ++b_mat_orig)
+ {
+ /* TODO(sergey): Iterate over evaluated data rather than using mapping. */
+ BL::Material b_mat_(b_depsgraph.evaluated_id_get(*b_mat_orig));
+ BL::Material *b_mat = &b_mat_;
Shader *shader;
/* test if we need to sync */
@@ -1343,9 +1348,14 @@ void BlenderSync::sync_lamps(bool update_all)
shader_map.set_default(scene->default_light);
/* lamp loop */
- BL::BlendData::lamps_iterator b_lamp;
-
- for(b_data.lamps.begin(b_lamp); b_lamp != b_data.lamps.end(); ++b_lamp) {
+ BL::BlendData::lamps_iterator b_lamp_orig;
+ for(b_data.lamps.begin(b_lamp_orig);
+ b_lamp_orig != b_data.lamps.end();
+ ++b_lamp_orig)
+ {
+ /* TODO(sergey): Iterate over evaluated data rather than using mapping. */
+ BL::Lamp b_lamp_(b_depsgraph.evaluated_id_get(*b_lamp_orig));
+ BL::Lamp *b_lamp = &b_lamp_;
Shader *shader;
/* test if we need to sync */
diff --git a/source/blender/blenkernel/CMakeLists.txt b/source/blender/blenkernel/CMakeLists.txt
index 66c71dcfab1..1d4565f1ec6 100644
--- a/source/blender/blenkernel/CMakeLists.txt
+++ b/source/blender/blenkernel/CMakeLists.txt
@@ -547,4 +547,8 @@ endif()
# set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} /WX")
#endif()
+if(WITH_DEPSGRAPH_COPY_ON_WRITE)
+ add_definitions(-DWITH_COPY_ON_WRITE)
+endif()
+
blender_add_lib(bf_blenkernel "${SRC}" "${INC}" "${INC_SYS}")
diff --git a/source/blender/blenkernel/intern/object.c b/source/blender/blenkernel/intern/object.c
index 14d93d75440..68e5ddf8583 100644
--- a/source/blender/blenkernel/intern/object.c
+++ b/source/blender/blenkernel/intern/object.c
@@ -363,7 +363,16 @@ void BKE_object_free_derived_caches(Object *ob)
ob->derivedDeform->release(ob->derivedDeform);
ob->derivedDeform = NULL;
}
-
+
+ if (ob->mesh_evaluated != NULL) {
+ /* Evaluated mesh points to edit mesh, but does not own it. */
+ ob->mesh_evaluated->edit_btmesh = NULL;
+ BKE_mesh_free(ob->mesh_evaluated);
+ BKE_libblock_free_data(&ob->mesh_evaluated->id, false);
+ MEM_freeN(ob->mesh_evaluated);
+ ob->mesh_evaluated = NULL;
+ }
+
BKE_object_free_curve_cache(ob);
}
diff --git a/source/blender/blenkernel/intern/object_update.c b/source/blender/blenkernel/intern/object_update.c
index 0c53ee8818a..70e1f434388 100644
--- a/source/blender/blenkernel/intern/object_update.c
+++ b/source/blender/blenkernel/intern/object_update.c
@@ -32,6 +32,7 @@
#include "DNA_group_types.h"
#include "DNA_key_types.h"
#include "DNA_material_types.h"
+#include "DNA_mesh_types.h"
#include "DNA_scene_types.h"
#include "BLI_blenlib.h"
@@ -51,6 +52,7 @@
#include "BKE_key.h"
#include "BKE_lamp.h"
#include "BKE_lattice.h"
+#include "BKE_library.h"
#include "BKE_editmesh.h"
#include "BKE_object.h"
#include "BKE_particle.h"
@@ -60,6 +62,7 @@
#include "BKE_mesh.h"
#include "BKE_image.h"
+#include "MEM_guardedalloc.h"
#include "DEG_depsgraph.h"
#define DEBUG_PRINT if (G.debug & G_DEBUG_DEPSGRAPH) printf
@@ -345,6 +348,51 @@ void BKE_object_eval_uber_data(EvaluationContext *eval_ctx,
break;
}
+#ifdef WITH_COPY_ON_WRITE
+ if (ob->type == OB_MESH) {
+ /* Quick hack to convert evaluated derivedMesh to Mesh. */
+ DerivedMesh *dm = ob->derivedFinal;
+ if (dm != NULL) {
+ Mesh *mesh = (Mesh *)ob->data;
+ Mesh *new_mesh = BKE_libblock_alloc_notest(ID_ME);
+ BKE_mesh_init(new_mesh);
+ /* Copy ID name so GS(new_mesh->id) works correct later on. */
+ BLI_strncpy(new_mesh->id.name, mesh->id.name, sizeof(new_mesh->id.name));
+ /* Copy materials so render engines can access them. */
+ new_mesh->mat = MEM_dupallocN(mesh->mat);
+ new_mesh->totcol = mesh->totcol;
+ DM_to_mesh(dm, new_mesh, ob, ob->lastDataMask, true);
+ new_mesh->edit_btmesh = mesh->edit_btmesh;
+ /* Store result mesh as derived_mesh of object. This way we have
+ * explicit way to query final object evaluated data and know for sure
+ * who owns the newly created mesh datablock.
+ */
+ ob->mesh_evaluated = new_mesh;
+ /* TODO(sergey): This is kind of compatibility thing, so all render
+ * engines can use object->data for mesh data for display. This is
+ * something what we might want to change in the future.
+ */
+ ob->data = new_mesh;
+ /* Save some memory by throwing DerivedMesh away. */
+ /* NOTE: Watch out, some tools might need it!
+ * So keep around for now..
+ */
+ }
+#if 0
+ if (ob->derivedFinal != NULL) {
+ ob->derivedFinal->needsFree = 1;
+ ob->derivedFinal->release(ob->derivedFinal);
+ ob->derivedFinal = NULL;
+ }
+ if (ob->derivedDeform != NULL) {
+ ob->derivedDeform->needsFree = 1;
+ ob->derivedDeform->release(ob->derivedDeform);
+ ob->derivedDeform = NULL;
+ }
+#endif
+ }
+#endif
+
ob->recalc &= ~(OB_RECALC_DATA | OB_RECALC_TIME);
}
diff --git a/source/blender/depsgraph/CMakeLists.txt b/source/blender/depsgraph/CMakeLists.txt
index 5eb5d69d62e..17a13b66aaa 100644
--- a/source/blender/depsgraph/CMakeLists.txt
+++ b/source/blender/depsgraph/CMakeLists.txt
@@ -55,6 +55,7 @@ set(SRC
intern/builder/deg_builder_transitive.cc
intern/debug/deg_debug_graphviz.cc
intern/eval/deg_eval.cc
+ intern/eval/deg_eval_copy_on_write.cc
intern/eval/deg_eval_debug.cc
intern/eval/deg_eval_flush.cc
intern/nodes/deg_node.cc
@@ -80,6 +81,7 @@ set(SRC
intern/builder/deg_builder_relations.h
intern/builder/deg_builder_transitive.h
intern/eval/deg_eval.h
+ intern/eval/deg_eval_copy_on_write.h
intern/eval/deg_eval_debug.h
intern/eval/deg_eval_flush.h
intern/nodes/deg_node.h
@@ -126,4 +128,8 @@ if(WITH_OPENSUBDIV)
add_definitions(-DWITH_OPENSUBDIV)
endif()
+if(WITH_DEPSGRAPH_COPY_ON_WRITE)
+ add_definitions(-DWITH_COPY_ON_WRITE)
+endif()
+
blender_add_lib(bf_depsgraph "${SRC}" "${INC}" "${INC_SYS}")
diff --git a/source/blender/depsgraph/DEG_depsgraph.h b/source/blender/depsgraph/DEG_depsgraph.h
index 63c02d007dc..9caf7635ccf 100644
--- a/source/blender/depsgraph/DEG_depsgraph.h
+++ b/source/blender/depsgraph/DEG_depsgraph.h
@@ -144,10 +144,31 @@ void DEG_graph_data_tag_update(Depsgraph *graph, const struct PointerRNA *ptr);
void DEG_graph_property_tag_update(Depsgraph *graph, const struct PointerRNA *ptr, const struct PropertyRNA *prop);
/* Tag given ID for an update in all the dependency graphs. */
-void DEG_id_tag_update(struct ID *id, short flag);
+enum {
+ /* Object transformation changed, corresponds to OB_RECALC_OB. */
+ DEG_TAG_TRANSFORM = (1 << 0),
+
+ /* Object geoemtry changed, corresponds to OB_RECALC_DATA. */
+ DEG_TAG_GEOMETRY = (1 << 1),
+
+ /* Time changed and animation is to be re-evaluated, OB_RECALC_TIME. */
+ DEG_TAG_TIME = (1 << 2),
+
+ /* Particle system changed. */
+ DEG_TAG_PSYSC_REDO = (1 << 3),
+ DEG_TAG_PSYS_RESET = (1 << 4),
+ DEG_TAG_PSYS_TYPE = (1 << 5),
+ DEG_TAG_PSYS_CHILD = (1 << 6),
+ DEG_TAG_PSYS_PHYS = (1 << 7),
+ DEG_TAG_PSYS = ((1 << 3) | (1 << 4) | (1 << 5) | (1 << 6) | (1 << 7)),
+
+ /* Update copy on write component without flushing down the road. */
+ DEG_TAG_COPY_ON_WRITE = (1 << 8),
+};
+void DEG_id_tag_update(struct ID *id, int flag);
void DEG_id_tag_update_ex(struct Main *bmain,
struct ID *id,
- short flag);
+ int flag);
/* Tag given ID type for update.
*
diff --git a/source/blender/depsgraph/DEG_depsgraph_query.h b/source/blender/depsgraph/DEG_depsgraph_query.h
index 649e788a0ef..674389ddd6c 100644
--- a/source/blender/depsgraph/DEG_depsgraph_query.h
+++ b/source/blender/depsgraph/DEG_depsgraph_query.h
@@ -64,6 +64,9 @@ struct SceneLayer *DEG_get_scene_layer(struct Depsgraph *graph);
/* Get the object as properly evaluated by depsgraph. */
struct Object *DEG_get_object(struct Depsgraph *depsgraph, struct Object *ob);
+/* Get evaluated version of given ID datablock. */
+struct ID *DEG_get_evaluated_id(struct Depsgraph *depsgraph, struct ID *id);
+
/* ************************ DAG iterators ********************* */
enum {
diff --git a/source/blender/depsgraph/intern/builder/deg_builder.cc b/source/blender/depsgraph/intern/builder/deg_builder.cc
index e7f54f9b0cb..92c79388657 100644
--- a/source/blender/depsgraph/intern/builder/deg_builder.cc
+++ b/source/blender/depsgraph/intern/builder/deg_builder.cc
@@ -50,7 +50,8 @@ void deg_graph_build_finalize(Depsgraph *graph)
*/
GHASH_FOREACH_BEGIN(IDDepsNode *, id_node, graph->id_hash)
{
- ID *id = id_node->id;
+ ID *id = id_node->id_orig;
+ id_node->finalize_build(graph);
if ((id->tag & LIB_TAG_ID_RECALC_ALL)) {
id_node->tag_update(graph);
}
@@ -60,7 +61,11 @@ void deg_graph_build_finalize(Depsgraph *graph)
id_node->tag_update(graph);
}
}
- id_node->finalize_build();
+ /* XXX: This is only so we've got proper COW IDs after rebuild. */
+ /* TODO(sergey): Ideally we'll need to copy evaluated CoW from previous
+ * depsgraph, so we don't need to re-tag anything what we already have.
+ */
+ id_node->tag_update(graph);
}
GHASH_FOREACH_END(
@@ Diff output truncated at 10240 characters. @@
More information about the Bf-blender-cvs
mailing list