[Bf-cycles] Render API

[Paulo Perbone]

Hi Brecht,

>> I'm assuming we need to support a real set of tools and API/SPI for
>> developers to allow them to integrate and extend Blender using C++. In
>> special for the Renderer we need a plug-in interface for shared library,
>> with loaders, factories etc. This SDK should not depend on Blender's source
>> code and should be ABI/API compatible with the matching version of its
>> release and Blender's.
>
> Right, we basically need a C++ SDK if external renderers want good
> export performance.

Great! Any restriction on C++11? Blender is moving towards VS2013 right?

> It would be good to support a plugin interface with support for
> loading dynamic libraries. However this is not necessarily needed to
> start, it is also possible to use Python scripts and let those scripts
> load Python extensions that then access the Blender API. Without this
> Python bridge the code would be cleaner though.

I think it's better to start with dynamic libraries. If ok with you 
I'll try to keep pure C++ and avoid Python as much as possible.

> Not sure what you mean by not depending on Blender's source code, if
> you mean that the SDK contains a set of header files that plugins can
> compile against without having access to the rest of the source code
> then yes, that's how it should be.

Yes, the only dependency the plug-in will need to compile will be the
SDK headers files.

>> Internal features used outside Blender's blob
>>
>> Cycles make use of lots of internal types and some times it uses its own
>> with similar purpose. Concepts like Scene, Camera, Mesh etc exists inside
>> Blender and they are the source of data for the Renderer. The way a SDK
>> should work we should not have dependencies of internal types like this for
>> the plug-in to compile and build. I can't think another way other than
>> redefining all this concepts again into the SDK so that Blender can manage
>> the translation of its internal state to a more public representation
>> through the API/SPI interfaces.
>
> It's not clear to me what you mean here. We have the RNA API that
> abstracts away the internal Blender types and can preserve
> compatibility up to a point as internal structures changes, until they
> change so much that the abstraction is no longer valid. This is the
> API that Cycles and the Python API use.
> 
> I wouldn't say this uses internal types, the RNA types match the
> internal data structures pretty well, but that seems like reasonable
> API design. So it's not clear to me what it means then to redefine all
> these concepts or why it would be done.

I understand what you mean but I still don't get it. How can we make
the SDK the only dependency for the plug-in to compile if we are going to use
RNA types? I mean, they are located in the Blender's source code tree, right?

Just to be clear, I'm talking about for example cycles/blender/blender_session.h
It uses lots of RNA types (BL::RenderEngine, BL::UserPreferences, etc)

Can we refactor Blender's source code and move all commons types into the SDK?
By common I mean types that both Blender and plug-ins depends on to compile.

> 
>> How to deal with Python initialization vs C++ plug-in initialization
>>
>> The way things works now is pretty straightforward. The module's Python code
>> gain access to all sort of internal state and relay them to the C++ code.
>> Once we separate the C++ to its own blob with its own roles things can
>> become more evolved and complex. We could use a thin glue of Python for each
>> plug-in but I'm not sure if I like this approach. Not to mention we need
>> performance here so Python much probably would be in your way.
>
> The Python glue doesn't have a significant impact on Cycles
> performance so it's not a big deal. But the mix of C++ and Python code
> is not easy to understand. There are a bunch of things in the Python
> API like defining properties, user interfaces, subclassing types that
> are not possible in C++ now though, so I imagine any plugin will
> likely need to have some Python code unless all these things are
> somehow supported from a first SDK release.

I see. Let me try to put this differently. Currently in the User Preferences we can
choose for example the type of Compute Device we want to use. This choice is sent way
down to Cycles so it can give us the list of compatible devices available. Finally
this list is displayed in the UI (It's not that simple I know but bear with me).

If I'm not mistaken this is a kind of cyclic dependency where both parts has great
knowledge of each other. With that in mind, how can we initialize both parts of the
plug-in (UI and the engine) without knowing about this cyclic dependency?

Simple put I would like to have no dependency at all between them (not explicit I mean)
and somehow Blender+SDK can make the IPC happens.


>> Events
>>
>> We need a way to expose internal events to the outside so the plug-in can
>> connect and consume event signals. The same way we need to be able to fire
>> events inside the plug-in into Blender for any target consumer.
>
> The way this works in the Python API is that you subclass the
> RenderEngine class (or Panel, Operator, Node, ..). Then you have a
> number of member functions that will get called: "update", "render",
> "viewport_draw", etc. I think that subclassing system works well as a
> general mechanism for Blender to call into plugins, it's a pretty
> standard OO programming technique.
> 
> There are other types of events that are supported in the Python
> (bpy.app.handlers), though these are not needed for render engines at
> the moment, so I wouldn't worry about that. If you want to avoid the
> Python glue here I would try to implementing this subclassing system
> for the RNA C++ API. 

I completely understand what you mean. I just think that it all depends on how 
the UI vs engine dependency thing (as put above) will works out. Am I wrong?

So... If you're willing to we can keep this mentoring but I would like to start
coding. I'm thinking some thing like this for the repository tree:

blender-sdk.git
cycles.git (refactored from blender tree)
blender.git (refactoring branch)

I'll need some place to stage all the code. For the time being I'll use my
Github repository. Is that ok with you?

Again thanks for your time.
Paulo