[Bf-blender-cvs] [4130f1e674f] master: Geometry Nodes: new evaluation system
Jacques Lucke
noreply at git.blender.org
Tue Sep 13 08:52:00 CEST 2022
Commit: 4130f1e674f83fc3d53979d3061469af34e1f873
Author: Jacques Lucke
Date: Tue Sep 13 08:44:26 2022 +0200
Branches: master
https://developer.blender.org/rB4130f1e674f83fc3d53979d3061469af34e1f873
Geometry Nodes: new evaluation system
This refactors the geometry nodes evaluation system. No changes for the
user are expected. At a high level the goals are:
* Support using geometry nodes outside of the geometry nodes modifier.
* Support using the evaluator infrastructure for other purposes like field evaluation.
* Support more nodes, especially when many of them are disabled behind switch nodes.
* Support doing preprocessing on node groups.
For more details see T98492.
There are fairly detailed comments in the code, but here is a high level overview
for how it works now:
* There is a new "lazy-function" system. It is similar in spirit to the multi-function
system but with different goals. Instead of optimizing throughput for highly
parallelizable work, this system is designed to compute only the data that is actually
necessary. What data is necessary can be determined dynamically during evaluation.
Many lazy-functions can be composed in a graph to form a new lazy-function, which can
again be used in a graph etc.
* Each geometry node group is converted into a lazy-function graph prior to evaluation.
To evaluate geometry nodes, one then just has to evaluate that graph. Node groups are
no longer inlined into their parents.
Next steps for the evaluation system is to reduce the use of threads in some situations
to avoid overhead. Many small node groups don't benefit from multi-threading at all.
This is much easier to do now because not everything has to be inlined in one huge
node tree anymore.
Differential Revision: https://developer.blender.org/D15914
===================================================================
A source/blender/blenkernel/BKE_compute_contexts.hh
M source/blender/blenkernel/BKE_node_runtime.hh
M source/blender/blenkernel/CMakeLists.txt
A source/blender/blenkernel/intern/compute_contexts.cc
M source/blender/blenkernel/intern/node.cc
M source/blender/blenkernel/intern/node_runtime.cc
A source/blender/blenlib/BLI_compute_context.hh
M source/blender/blenlib/BLI_multi_value_map.hh
M source/blender/blenlib/CMakeLists.txt
A source/blender/blenlib/intern/compute_context.cc
M source/blender/blenlib/intern/cpp_type.cc
M source/blender/depsgraph/intern/builder/deg_builder_nodes.cc
M source/blender/editors/include/UI_interface.hh
M source/blender/editors/interface/interface_template_attribute_search.cc
M source/blender/editors/space_node/node_draw.cc
M source/blender/editors/space_node/node_geometry_attribute_search.cc
M source/blender/editors/space_spreadsheet/spreadsheet_data_source_geometry.cc
M source/blender/functions/CMakeLists.txt
M source/blender/functions/FN_field.hh
M source/blender/functions/FN_field_cpp_type.hh
A source/blender/functions/FN_lazy_function.hh
A source/blender/functions/FN_lazy_function_execute.hh
A source/blender/functions/FN_lazy_function_graph.hh
A source/blender/functions/FN_lazy_function_graph_executor.hh
M source/blender/functions/FN_multi_function.hh
M source/blender/functions/intern/cpp_types.cc
A source/blender/functions/intern/lazy_function.cc
A source/blender/functions/intern/lazy_function_execute.cc
A source/blender/functions/intern/lazy_function_graph.cc
A source/blender/functions/intern/lazy_function_graph_executor.cc
A source/blender/functions/tests/FN_lazy_function_test.cc
M source/blender/makesdna/DNA_node_types.h
M source/blender/modifiers/CMakeLists.txt
M source/blender/modifiers/intern/MOD_nodes.cc
D source/blender/modifiers/intern/MOD_nodes_evaluator.cc
D source/blender/modifiers/intern/MOD_nodes_evaluator.hh
M source/blender/nodes/CMakeLists.txt
M source/blender/nodes/NOD_geometry_exec.hh
D source/blender/nodes/NOD_geometry_nodes_eval_log.hh
A source/blender/nodes/NOD_geometry_nodes_lazy_function.hh
A source/blender/nodes/NOD_geometry_nodes_log.hh
M source/blender/nodes/NOD_multi_function.hh
M source/blender/nodes/geometry/node_geometry_exec.cc
M source/blender/nodes/geometry/nodes/node_geo_boolean.cc
M source/blender/nodes/geometry/nodes/node_geo_geometry_to_instance.cc
M source/blender/nodes/geometry/nodes/node_geo_input_named_attribute.cc
M source/blender/nodes/geometry/nodes/node_geo_join_geometry.cc
M source/blender/nodes/geometry/nodes/node_geo_remove_attribute.cc
M source/blender/nodes/geometry/nodes/node_geo_store_named_attribute.cc
M source/blender/nodes/geometry/nodes/node_geo_string_join.cc
D source/blender/nodes/intern/geometry_nodes_eval_log.cc
A source/blender/nodes/intern/geometry_nodes_lazy_function.cc
A source/blender/nodes/intern/geometry_nodes_log.cc
M source/blender/nodes/intern/node_geometry_exec.cc
M source/blender/nodes/intern/node_multi_function.cc
===================================================================
diff --git a/source/blender/blenkernel/BKE_compute_contexts.hh b/source/blender/blenkernel/BKE_compute_contexts.hh
new file mode 100644
index 00000000000..a8f0022f49b
--- /dev/null
+++ b/source/blender/blenkernel/BKE_compute_contexts.hh
@@ -0,0 +1,46 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+
+#pragma once
+
+/**
+ * This file implements some specific compute contexts for concepts in Blender.
+ */
+
+#include "BLI_compute_context.hh"
+
+namespace blender::bke {
+
+class ModifierComputeContext : public ComputeContext {
+ private:
+ static constexpr const char *s_static_type = "MODIFIER";
+
+ /**
+ * Use modifier name instead of something like `session_uuid` for now because:
+ * - It's more obvious that the name matches between the original and evaluated object.
+ * - We might want that the context hash is consistent between sessions in the future.
+ */
+ std::string modifier_name_;
+
+ public:
+ ModifierComputeContext(const ComputeContext *parent, std::string modifier_name);
+
+ private:
+ void print_current_in_line(std::ostream &stream) const override;
+};
+
+class NodeGroupComputeContext : public ComputeContext {
+ private:
+ static constexpr const char *s_static_type = "NODE_GROUP";
+
+ std::string node_name_;
+
+ public:
+ NodeGroupComputeContext(const ComputeContext *parent, std::string node_name);
+
+ StringRefNull node_name() const;
+
+ private:
+ void print_current_in_line(std::ostream &stream) const override;
+};
+
+} // namespace blender::bke
diff --git a/source/blender/blenkernel/BKE_node_runtime.hh b/source/blender/blenkernel/BKE_node_runtime.hh
index f2e551a9f32..194820aa4ba 100644
--- a/source/blender/blenkernel/BKE_node_runtime.hh
+++ b/source/blender/blenkernel/BKE_node_runtime.hh
@@ -21,6 +21,7 @@ struct bNodeType;
namespace blender::nodes {
struct FieldInferencingInterface;
class NodeDeclaration;
+struct GeometryNodesLazyFunctionGraphInfo;
} // namespace blender::nodes
namespace blender::bke {
@@ -48,6 +49,15 @@ class bNodeTreeRuntime : NonCopyable, NonMovable {
/** Information about how inputs and outputs of the node group interact with fields. */
std::unique_ptr<nodes::FieldInferencingInterface> field_inferencing_interface;
+ /**
+ * For geometry nodes, a lazy function graph with some additional info is cached. This is used to
+ * evaluate the node group. Caching it here allows us to reuse the preprocessed node tree in case
+ * its used multiple times.
+ */
+ std::mutex geometry_nodes_lazy_function_graph_info_mutex;
+ std::unique_ptr<nodes::GeometryNodesLazyFunctionGraphInfo>
+ geometry_nodes_lazy_function_graph_info;
+
/**
* Protects access to all topology cache variables below. This is necessary so that the cache can
* be updated on a const #bNodeTree.
@@ -70,6 +80,7 @@ class bNodeTreeRuntime : NonCopyable, NonMovable {
MultiValueMap<const bNodeType *, bNode *> nodes_by_type;
Vector<bNode *> toposort_left_to_right;
Vector<bNode *> toposort_right_to_left;
+ Vector<bNode *> group_nodes;
bool has_link_cycle = false;
bool has_undefined_nodes_or_sockets = false;
bNode *group_output_node = nullptr;
@@ -148,6 +159,12 @@ class bNodeRuntime : NonCopyable, NonMovable {
namespace node_tree_runtime {
+/**
+ * Is executed when the depsgraph determines that something in the node group changed that will
+ * affect the output.
+ */
+void handle_node_tree_output_changed(bNodeTree &tree_cow);
+
class AllowUsingOutdatedInfo : NonCopyable, NonMovable {
private:
const bNodeTree &tree_;
@@ -241,6 +258,18 @@ inline blender::Span<bNode *> bNodeTree::all_nodes()
return this->runtime->nodes;
}
+inline blender::Span<const bNode *> bNodeTree::group_nodes() const
+{
+ BLI_assert(blender::bke::node_tree_runtime::topology_cache_is_available(*this));
+ return this->runtime->group_nodes;
+}
+
+inline blender::Span<bNode *> bNodeTree::group_nodes()
+{
+ BLI_assert(blender::bke::node_tree_runtime::topology_cache_is_available(*this));
+ return this->runtime->group_nodes;
+}
+
inline bool bNodeTree::has_link_cycle() const
{
BLI_assert(blender::bke::node_tree_runtime::topology_cache_is_available(*this));
@@ -413,7 +442,6 @@ inline blender::Span<const bNodeLink *> bNode::internal_links_span() const
inline const blender::nodes::NodeDeclaration *bNode::declaration() const
{
- BLI_assert(this->runtime->declaration != nullptr);
return this->runtime->declaration;
}
diff --git a/source/blender/blenkernel/CMakeLists.txt b/source/blender/blenkernel/CMakeLists.txt
index b982c69a378..2f1e1897f8d 100644
--- a/source/blender/blenkernel/CMakeLists.txt
+++ b/source/blender/blenkernel/CMakeLists.txt
@@ -98,6 +98,7 @@ set(SRC
intern/collision.c
intern/colorband.c
intern/colortools.c
+ intern/compute_contexts.cc
intern/constraint.c
intern/context.c
intern/crazyspace.cc
@@ -352,6 +353,7 @@ set(SRC
BKE_collision.h
BKE_colorband.h
BKE_colortools.h
+ BKE_compute_contexts.hh
BKE_constraint.h
BKE_context.h
BKE_crazyspace.h
diff --git a/source/blender/blenkernel/intern/compute_contexts.cc b/source/blender/blenkernel/intern/compute_contexts.cc
new file mode 100644
index 00000000000..026706d363e
--- /dev/null
+++ b/source/blender/blenkernel/intern/compute_contexts.cc
@@ -0,0 +1,38 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+
+#include "BKE_compute_contexts.hh"
+
+namespace blender::bke {
+
+ModifierComputeContext::ModifierComputeContext(const ComputeContext *parent,
+ std::string modifier_name)
+ : ComputeContext(s_static_type, parent), modifier_name_(std::move(modifier_name))
+{
+ hash_.mix_in(s_static_type, strlen(s_static_type));
+ hash_.mix_in(modifier_name_.data(), modifier_name_.size());
+}
+
+void ModifierComputeContext::print_current_in_line(std::ostream &stream) const
+{
+ stream << "Modifier: " << modifier_name_;
+}
+
+NodeGroupComputeContext::NodeGroupComputeContext(const ComputeContext *parent,
+ std::string node_name)
+ : ComputeContext(s_static_type, parent), node_name_(std::move(node_name))
+{
+ hash_.mix_in(s_static_type, strlen(s_static_type));
+ hash_.mix_in(node_name_.data(), node_name_.size());
+}
+
+StringRefNull NodeGroupComputeContext::node_name() const
+{
+ return node_name_;
+}
+
+void NodeGroupComputeContext::print_current_in_line(std::ostream &stream) const
+{
+ stream << "Node: " << node_name_;
+}
+
+} // namespace blender::bke
diff --git a/source/blender/blenkernel/intern/node.cc b/source/blender/blenkernel/intern/node.cc
index 2ae0b456b0d..b82cf30416a 100644
--- a/source/blender/blenkernel/intern/node.cc
+++ b/source/blender/blenkernel/intern/node.cc
@@ -71,6 +71,7 @@
#include "NOD_composite.h"
#include "NOD_function.h"
#include "NOD_geometry.h"
+#include "NOD_geometry_nodes_lazy_function.hh"
#include "NOD_node_declaration.hh"
#include "NOD_shader.h"
#include "NOD_socket.h"
diff --git a/source/blender/blenkernel/intern/node_runtime.cc b/source/blender/blenkernel/intern/node_runtime.cc
index a8281820a0b..00b78284791 100644
--- a/source/blender/blenkernel/intern/node_runtime.cc
+++ b/source/blender/blenkernel/intern/node_runtime.cc
@@ -10,8 +10,22 @@
#include "BLI_task.hh"
#include "BLI_timeit.hh"
+#include "NOD_geometry_nodes_lazy_function.hh"
+
namespace blender::bke::node_tree_runtime {
+void handle_node_tree_output_changed(bNodeTree &tree_cow)
+{
+ if (tree_cow.type == NTREE_GEOMETRY) {
+ /* Rebuild geometry nodes lazy function graph. */
+ {
+ std::lock_guard lock{tree_cow.runtime->geometry_nodes_lazy_function_graph_info_mutex};
+ tree_cow.runtime->geometry_nodes_lazy_function_graph_info.reset();
+ }
+ blender::nodes::ensure_geometry_nodes_lazy_function_graph(tree_cow);
+ }
+}
+
static void double_checked_lock(std::mutex &mutex, bool &data_is_dirty, FunctionRef<void()> fn)
{
if (!data_is_dirty) {
@@ -36,11 +50,15 @@ static void update_node_vector(const bNodeTree &ntree)
{
bNodeTreeRuntime &tree_runtime = *ntree.runtime;
tree_runtime.nodes.clear();
+ tree_runtime.group_nodes.clear();
tree_runtime.has_undefined_nodes_or_sockets = false;
LISTBASE_FOREACH (bNode *, node, &ntree.nodes) {
node->runtime->index_in_tree = tree_runtime.nodes.append_and_get_index(node);
node->runtime->owner_tree = const_cast<bNodeTree *>(&ntree);
tree_runtime.has_undefined_nodes_or_sockets |= node->typeinfo == &NodeTypeUndefined;
+ if (node->is_group()) {
+ tree_runtime.group_nodes.append(node);
+ }
}
}
diff --git a/source/blender/blenlib/BLI_compute_context.hh b/source/blender/blenlib/BLI_compute_context.hh
new file mode 100644
index 00000000000..7422467e400
--- /dev/null
+++ b/source/blender/blenlib/BLI_compute_context.hh
@@ -0,0 +1,173 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+
+#pragma once
+
+/** \file
+ * \ingroup bli
+ *
+ * When logging computed values, we generally want to know where the value was computed. For
+ * example, geometry nodes logs socket values so that they can be displayed in the ui. For that we
+ * can combine the logged value with a `ComputeContext`, which identifies the place where the value
+ * was computed.
+ *
+ * This is not a trivial problem because e.g. just storing storing a pointer to the socket a value
+ * belongs to is not enough. That's because the same socket may correspond to many different values
+ * when the socket is used in a node group that is used multiple times. In this case, not only does
+ * the socket have to be stored but also the entire nested node group path that led to the
+ * evaluation of the socket.
+ *
+ * Storing the entire "context path" for every logged value is not feasible, because that path can
+ * become quite long. So that would need much more memory, more compute overhead and makes it
+ * complicated to compare if two contexts are the same. If the identifier for a compute context
+ * would have a variable size, it would also be much harder to create a map from context to values.
+ *
+ * The solution implemented below uses the following key ideas:
+ * - Every compute context can be hashed to a unique fixed size value (`ComputeContextHash`). Whi
@@ Diff output truncated at 10240 characters. @@
More information about the Bf-blender-cvs
mailing list