[Bf-committers] Separating node systems

[Lukas Tönne]

Hi,

As i mentioned last sunday, i already started this separation work on
the particles-2010 branch. IMO these changes can be merged to trunk
will little risk and effort, since they don't really add new
functionality, but rather move code to separate files and clean up
some of the function interfaces. Let me try to outline here what i
did:

* Each tree type gets it's own .c file (CMP_nodetree.c,
SHD_nodetree.c, TEX_nodetree.c). The goal is to eventually have the
core node code free of all tree type and node type references.
* For each case where a functionality is specific to one or some of
the trees, a generic callback method is introduced. These are stored
in a bNodeTreeType interface struct, similar to bNodeType.
* The most common callback is for iterating over all node trees of a
type, which accesses scenes, materials and textures respectively.
Others can be used for only compositor nodes (or another tree type),
such as management of preview and "cache" data. Callbacks should be
kept as generic as possible, which forces a bit of abstraction work (a
good thing!).
* The way node types (i.e. bNodeType structs) are defined has already
been changed in trunk recently. This now uses initialization functions
instead of simply writing down values of the struct. This means that
changing and extending the node type struct is much easier because one
doesn't have to update all node .c files every time.
* Right now i'm working on moving all references to concrete node
types out of the core code as well. E.g. there are often function
calls only for viewer or output nodes and especially group nodes are
handled differently. Replacing these by generic callbacks in bNodeType
makes code much easier to understand and maintain.
* The design- vs. execution code separations is still a bit
incomplete. My idea of a clean system would be that the "design-time"
code knows as little about execution data as possible. When a tree is
executed it should create the execution data "on-top" of the
bNodeTree, bNode and bNodeSocket structures.

Basically i think execution should work in these rough steps:
1) For each instance of an execution call to a node tree, an
"execution data" struct is created. This is independent from the
bNodeTree (the tree doesn't know about this, unlike currently where
the tree stores the actual data stack!). The execution instance
references the bNodeTree, but not the other way around.
2) In order to prevent conflicts between editing and reading the tree
structure, a local copy of the node tree (and all trees used in
groups, etc.) is created. Currently this only happens in compo trees
afaik, but could be done in general.
3) Before a tree is actually executed, some preparation work might be
necessary (the details from here on may differ from current
implementation). Currently this includes generating a "stack index"
for each output buffer in the tree. This would be executed for all
trees used in groups and the like in advance. The management data (a
"stack" of socket data pointers) can then be allocated.
4) The actual execution code works on the execution data only. The
underlying design-time structure should be strictly read-only!
5) After execution some results may be copied back to the node tree
(previews in particular). Then the execution data can be freed.

Cheers,
Lukas

On Tue, Feb 15, 2011 at 8:33 AM, j.bakker at atmind.nl <j.bakker at atmind.nl> wrote:
> Currently all node systems are implemented in the space_node (design-time)
> and in the blenkernel/node.c (execution-time) and the makesdna
> DNA_node_types.h (datastore/database).
>
> There are 3 node systems and a new one is on its way:
>  * Compositor
>  * Material
>  * Texture
>  * [[http://phonybone.planetblender.org/ | Particles]]
>
> The node system has initially designed for the compositor. Since then, the
> Material and Texture node systems have been implemented on top of the
> original code. The Materials and Textures node system uses different styles
> when executing.
>
> Data needed for design-time and data needed for execution-time are both
> stored in the same structure. As the different node systems have different
> execution styles it is not clear what data is meant for what node system.
> The new compositor design changes totally the initial implementation of the
> node system execution (node.c).
>
> What I want to propose is to start separating the different node systems
> execution and the data.
>
> For the compositor I will start developing in a separate library
> ("bf_compositor"). During execution of the node system, the design-time
> node system is converted to the execution-time node system. This
> execution-time node system is then calculated. The implementation of the
> other node systems I will leave unchanged inside the (node.c).
>
> This way I want to ensure several things:
>  - Keep the code clean; the node systems are technically not the same, so
> why place them in a single file.
>  - Make sure that UI improvements can be done with limited impact to the
> execution engine. The user interface will evolve in a different speed than
> the execution engine.
>  - During conversion (design-time to execution-time) the node system can
> be optimized and modified
>    * Different node grouping based on the internal behavior of nodes.
>    * Filter node and bokeh blurs can be converted to an optimized
> convolution node.
>    * Data type conversion are added to the node system.
>  - In my opinion CPP is better in handling the needed complexity
> (balancing algorithms & graph execution optimizations). but I don't like
> the idea of placing CPP code inside the blenkernel library.
>
> Reusable components like opencl driver manager, or the new memory manager
> will be placed inside the blender-kernel or blender-library.
>
> Jeroen.
>
> --------------------------------------------------------------------
> mail2web.com - Microsoft® Exchange solutions from a leading provider -
> http://link.mail2web.com/Business/Exchange
>
>
> _______________________________________________
> Bf-committers mailing list
> Bf-committers at blender.org
> http://lists.blender.org/mailman/listinfo/bf-committers
>