[Bf-committers] Custom Nodes branch review

[Lukas Tönne]

At long last, the "custom nodes" branch (aka pynodes) is ready for
review. Probably it is too late to include this in the 2.66 release
(provided there are no major issues, which is still possible due to
the complexity), so i would aim for merge after the 2.66 release in
about 3-4 weeks.

Unfortunately the codereview tool (https://codereview.appspot.com/) is
not capable of handling the entire diff file (>1mb). No datatoc stuff,
only the repetitive changes to node .c files and moved code etc. add
up to a lot of lines ...

So for now, here is just the full diff of the complete squashed patch
of the branch. If anybody knows how to upload this to codereview
anyway please let me know.

The full git repository can be found here:
website: https://gitorious.org/~lukastoenne/blenderprojects/blender-lukastoenne
git repository:
git://gitorious.org/~lukastoenne/blenderprojects/blender-lukastoenne.git
(master = trunk, "customnodes" branch contains the actual branch.)

Squashed svn patch:
http://www.pasteall.org/38982/diff (WARNING 30k lines!)
or http://dl.dropbox.com/u/18763916/customnodes_01.diff

Main areas that have been changed:
* Dynamic registration of node types, node tree types and socket types
in independent global hashes
* Support for python class registration
* Support for ID properties in nodes and sockets (for python types mostly)
* Node editor tree stack, instead of base tree vs. edit group, to
allow editing nested groups
* Pinning in the node editor (disable context selection)

A few documentation links:
* Overview of the DNA/RNA/BKE structure with a few remarks on changes:
http://wiki.blender.org/index.php/User:Phonybone/Nodes_Documentation
* Python nodes: http://wiki.blender.org/index.php/User:Phonybone/Python_Nodes

The patch needs to add a lot of really nasty compatibility code in
readfile.c/writefile.c to make it forward compatible (old Blender
versions can open newer files). To keep this somewhat manageable i
wrapped all the forward compat code in a USE_NODE_COMPAT_CUSTOMNODES
define. One day (probably with the 2.7 version) we can then drop
compatibility for older versions and remove all that code easily.

TODO:
Node Groups could still use a few improvements. They work ok in most
situations for now, but in the long run i'd like to tackle a number of
issues:
1) Using node socket "interface" types in addition to regular node
socket types is a flawed concept. Interface types are part of the node
group type and should be implemented by that type: the node group
creates an interface definition and in turn adds sockets based on it.
Having interface types somewhat independent from group nodes just
makes things more complicated.
2) Update tagging and notifiers for node groups may not work in all
cases. In trunk all updates rely heavily on the "one edited group"
limitation. The new node editor design for groups requires a better
way of tagging nodes in many different groups depending on visibility.
3) Value updates for node properties from python rely on a "node"
pointer in the context (otherwise would have to look up the node in
node tree data, which does not work well for node group properties).
Problem is that popup widgets often lose all context storage info,
this needs to be fixed.
4) Operators for grouping/ungrouping/inserting/extracting perform some
very low-level hacks on the DNA linked lists and anim data. This
should be moved into the RNA and cleaned up as much as possible, so
that operators in turn can access the API and user-defined operators
can be created for managing node groups.

Ideally node groups should be implemented in python at some point,
this way they can nicely use the API (avoiding low-level hacks) and
ensure the concept of group nodes can be extended in the future.

Regards,
Lukas Tönne