[Bf-committers] Some Ideas for a Blender Plugin System (hewi)

Tue Nov 10 08:34:20 CET 2015

developers, coders, all,

Saw the mail about the Blender plugin system (Bps, Martin Felke), must say it's best idea I have seen towards restructuring Blender source and preparing it for, as Ton would say, 2020!  IMHO the discussion is not on yes or no to implement the plugins, the discussion is on how to nicely do it.  This (little) text is on how I see the Bps being a good start, a fundamental way forward and the best long term strategy you can decide on.

If you would wonder why the plugins will work, I always explain things in terms of cars and engines and car dashboard or even ... Tanks.  (Knowing you guys probably do not need explanation but this is just the way how I like to explain to myself.)  Remember the old school tanks, why were the no good?  Because if they broke, they had to be completely dis-assembled, repaired and re-assembled again, way too time consuming, however good looking and strong they were.  engine was on the inside, suspension was merged with hull, turrets were not to be dis-assembled. The tanks nowadays are completely modular.  Auch, maybe this is a wrong time to use the word modular :)

Let's continue on the tank idea.  The engine is the core black box, which powers all the rest.  Is the engine a module, or is it a plugin?  Well, everything that dis-assembles as one single entity is a plugin.  A black box, as Piotr mentioned. Sure when the engine is taken out, you can dis-assemble the engine into single modules, the spark plugs being a good example, but also cylinders, drive shank, oil carter, pumps, cooling, etc ...  but you see the engine as an entity, I think no discussion?  you would not be using the radiator for anything else, but you could fit the whole engine into another tank, yes.  Of course all the plugins have dependencies, but the are limited to a core minimum.  What does the engine need?  Two inputs, one output (exhaust not taken into account).  The engine needs a fuel line and a throttle, the output is power.  That's it, but imagine a tank without an engine, no go.  Once you have the engine running, you can start plugin things in: transmission, the suspension, the tracks, the hull, the turret, the gun, all base plugins. All possible to be plugged in into another tank, but unlikely individually to be disassemble unless erroneous. Absolute pro is that you can test every single plugin on it's own, outside the tank.  The gun should fire, the engine should run.  Take this idea into Blender, imagine what a time-saver it would be to get the core running and then say:"plug everything in".  While plugin things in, the testers would say A plugged, OK; B plug OK; C plug error; so you would only need to search what 's wrong with C (haha, whats wrong with C, well, where to start).

So plugins should touch base or core, however you want to interpret.  The python API is cool if you want to create a button for the radio to go louder, or what we would call a volume button.  That is also how the API should be used.  It does not create the radio to be plugged in (event though possible, it is not there to do that). Of course the radio depends on other systems and it cannot function on its own.  Imagine you do not plug power, mmmh.  Or that the radio has power but you do not connect the speakers, pretty useless.  The radio as plugin, the volume button, jack output, power input are modules of the radio.   

In an ideal world, you should (re)write Blender's core and dived all the rest into plugins, unfortunately that is not possible.  Way too many circular dependencies, even way down to the core (GHOST depending on <string.h>, what the F**).  So what you/we need to do is write simple plugins, which do touch core and are based on the foundations of blender code (RNA, DNA, editors, ...).  Gradually, in this way, we can start pulling things out of the core code base, eventually you are left with a solid black box, which will never fail, where you can plug in all system or applications goodies.  With Blender, the only way to do this is reverse engineering.  That is were this idea fits in perfectly.  

Taking the advice of matmenu, it would be a good initiative and good trial to implement this array modifier, touching core, pluggable, fast, rock solid, ...  for this, I think we best come up with an action plan.  How would one co-ordinate such project?  An other nice example of a good plug-in would be a browser.  Wouldn't that be cool to be able to browse inside Blender :)

Where do we start? (mostly addressing Mr Felke at this point), do you feel like implementing the array idea, what are your thoughts?

Addendum: 
- Good effort on the 2.8 wiki page, just try to keep this one clean and structured.

- Small side note about testing, just make sure you keep the three levels of testing seperated: 
 1 the CMake testers, regarding platform and linkage and presence of certain libs and compilers and flags.
 2 the code testers, who check functions andd do static tests of all internal functions and stuff
 3 the runtime tests, to check if the lama still has furr and eyes after rendering ;)

------------------------------

Message: 3
Date: Thu, 5 Nov 2015 15:16:07 +0100
From: Fran?ois T. <francoistarlier at gmail.com>
Subject: Re: [Bf-committers] Some Ideas for a Blender Plugin System
To: bf-blender developers <bf-committers at blender.org>
Message-ID:
        <CABE+dDbK135PhpGS165unfMU4Gc8oLDHr5uD6RcHBxAbQs7pBQ at mail.gmail.com>
Content-Type: text/plain; charset=UTF-8

- Cython does not have as much performance as pure C/C++
- Python is still the best choice for prototyping and pipeline stuff
- C/C++ API can be used for better performance without having the nightmare
to dive into Blender core code to just add one modifier

Most of the modifier we are developping in Maya are first done in Python,
but then it is port to C++ for performance matter. Same approach would be
nice in Blender.

Of course we can modify Blender code and recompile it but :
- this is low level coding compare to the use of an API
- can make future merge more complicated for a studio

And in some cases like sharing code for importer/exporter in alembic, fbx
would be more easier if it was done as a plugin rather than being embeded
in our blender build

2015-11-04 14:44 GMT+01:00 Diego Gangl <dnicolas at gmail.com>:

> Hi all,
>
> What would be the benefit of doing this, instead of expanding the python
> api?
>
> The only case I can think of is doing really complex computations or
> working with large sets of data, and you already have three options for
> that:
>
> - Use numpy (for some things)
> - Use Cython
> - Start a process from python and communicate with pipes
>
> A plugin/module system might sound great on paper, but I think it would be
> a nightmare of corner cases.
>
>
>
>
>
>
>
>
>
>
>
>
> 2015-11-04 4:58 GMT-03:00 Sybren A. St?vel <sybren at stuvel.eu>:
>
> > On Tue, Nov 03, 2015 at 08:04:03PM +0100, matmenu wrote:
> > > 2) Do you have a proof for this claim? I mean, modifier/operator
> > > combinations already lead to bugs, I don't see why having them as
> > > plugins/modules would make those problems arise more often?
> >
> > I don't understand your reasoning. To me it reads like "we get bugs
> > now, so why would we get more when we complicate the software?" In the
> > current situation there is a fixed set of functionality in C++
> > (ignoring precompiler directives). In the case of a plugin structure,
> > not only do we have the same set of functionality, but also situations
> > where that functionality is only partially loaded. More different
> > possible situations -> more complex -> more bugs.
> >
> > > 3) That's true, but then the user decide to install this module, so
> > > he will report the bug to the original dev
> >
> > This is only true if the bug is trivial to to identify by end-users.
> > Plenty of bugs won't be. Think about double-free bugs, where the first
> > free is performed in the plugin. A crash may happen when Blender
> > itself frees the pointer, and thus any traceback (if that's shown at
> > all) will indicate a problem in Blender, not the plugin.
> >
> > > 4) Also true, but we now have a buildbot that can make builds on
> > > demand for all supported OS. It could be used further by
> > > module/plugin devs and avoid having to ship a compiler with Blender.
> >
> > AFAIK FreeBSD is not built by this system. Furthermore, allowing
> > arbitrary third party code to be run on a BI/BF-hosted machine is an
> > invitation for trouble. This already happens, but through the Git
> > repository, so at least it's only done by people that have been vetted
> > by receiving commit access.
> >
> > --
> > Sybren A. St?vel
> >
> > http://stuvelfoto.nl/
> > http://stuvel.eu/
> >
> > _______________________________________________
> > Bf-committers mailing list
> > Bf-committers at blender.org
> > http://lists.blender.org/mailman/listinfo/bf-committers
> >
> >
>
>
> --
> --
> Diego Gangl - sinestesia.co
> _______________________________________________
> Bf-committers mailing list
> Bf-committers at blender.org
> http://lists.blender.org/mailman/listinfo/bf-committers
>

--
____________________
Fran?ois Tarlier
www.francois-tarlier.com
www.linkedin.com/in/francoistarlier

------------------------------