[Bf-taskforce25] Blender API

[vekoon]

Well, the central point of my idea was that in the end you can't put 
everything high level (operator level). Especially for things like the 
game engine and extensibility you'll need to provide the user with some 
finer controls. Operators are good for end users as they provide a very 
abstracted view on actions which makes it more predictable (and 
constrained).

The problem as I said arises when you try to satisfy let's say more 
'demanding' needs. As far as end-users go they're fine with operators 
and abstracted data access. You can even tell them about RNA but there's 
no need to and they generally don't care. They just use their natural 
insight and see clearly what for instance 
main.objects['cube'].materials[0].color means. This provides them with a 
generic interface for them to use and get used to but they'll generally 
just use it in the limitations provided by the application itself 
(macros, custom shortcuts, bindings etc).

On the other hand more experienced users or simply developers hired to 
accomplish some specific task surely want to easily *extend* some 
functionality and there's where a more specific API is needed. They're 
also perfectly fine with having to learn some specs or implementation 
details but they then want to be able to implement any idea they have in 
the way they'd like to and too much abstraction tends to make things 
harder for devs not only because it's slower and devs get frustrated 
when they have to optimize but also because it gives you much less 
control on specific aspects of the functionality.

For instance there will surely be need for custom editors to be 
implemented easily. How to accomplish this? In no way high abstraction 
can tackle this as it is not really suited for something of this entity 
(there's too much unpredictability attached to it). This means you'll 
have to provide access to this functionality somehow which in the end 
will always create a parallel system to the high interface part.

So starting from this assumption, because I noted we had to provide 
something custom for specific tasks anyway, I tried to find a way that 
blended in nicely with the rest of blender's design. Why to create 
something separate instead of some extension to RNA? Because RNA means 
data and it should be kept this way imo because data and functionality 
are two very different things with data being "how some stuff looks 
like" and functionality "what you can do over some stuff when you 
interpret this stuff in this way". If it really turns out to be 
unmanageable we could merge the API stuff into RNA but I'd really like 
not to. Yes there's some additional API information to define (so 
basically core C side) but makes it more powerful and flexible for the 
end programmers (extension writers). Even if you write some C plugin 
modules (which by the way I think should also be a goal of 2.5 as I 
don't think py plugins are suitable for everything) you ought to be able 
to export your custom API so that this is immediately accessible to py 
or any other extension. This way for C plugins you just have to provide 
a couple headers for RNA and API and all the provided functionality will 
be accessible. Forcing the plugin developer to export operators is a bad 
decision in my view of things as it limits him as I said above forcing 
an higher level onto the developer is always frustrating on his end as 
it forces him to adapt to an entire point of view instead of the other 
way around.
Of course as I already mentioned in channel I don't want to 'force' in 
any way my design, I just implemented it directly instead of simply 
doing a proposal as it's the way I prefer to do things and it was the 
most direct way of showing my ideas.


Ton Roosendaal wrote, on 01/03/2009 13.33:
> Hi Vekoon,
>
> I think you point at a good problem in our design, but the solution 
> confuses me. It seems you solve things on a too low level, introducing 
> a new programming api in our code.
>
> What I tried to achieve with operators was to make it as high level as 
> possible, as close as possible to the end-user (hotkeys, menus, 
> toolbars, history stacks, command-line ops), and allowing to re-use 
> existing code relatively flexible. It also provides some design 
> paradagms; like separating context (the "user") from 
> settings/properties (the "state") to achieve a good level of 
> re-usablity.
>
> The open issues are - as you note already - on a lot of UI options we 
> have in Blender for managing data with buttons; like constraints, 
> modifiers, image types and so on. There we could find solutions for on 
> a very high level as well; probably using the same rna-path convention 
> as for the animation system (as provided context to operator), and some 
> method to extend or integrate operators with UIs.
>
> For the next days I wanted to give the operator design a good review 
> and propose some ideas for discussion on how to extend it further.
>
> -Ton-
>
> ------------------------------------------------------------------------
> Ton Roosendaal  Blender Foundation   ton at blender.org    www.blender.org
> Blender Institute BV  Entrepotdok 57A  1018AD Amsterdam The Netherlands
>
> On 27 Feb, 2009, at 17:32, vekoon wrote:
>
>   
>> Hey guys,
>> lately I've seen discussions especially on channel about what should be
>> made into operators and what should not and how to eventually solve or
>> provide access to the functionality not covered by operators.
>> Is obvious that operatorifying everything is a mistake or even
>> impossible in some circumstances (like direct UI) but personally I'd
>> also prefer to make into operators only what is strictly a tool of some
>> kind. I'm not sure "everything user-accessible" is a good measure in
>> that the user is in the end able to access almost everything. There are
>> various gray areas on top of this argument. For instance moving a
>> space's splitter, is that a "tool"? No, but it's definitively
>> user-accessible in the sense that the user can actually interact with 
>> it
>> and change every aspect of it (not that it has many aspects though). 
>> Now
>> this is the case of a grey area. If you resize the splitter while
>> registering a macro and the action is registered and then you run the
>> macro with a different screen layout, what happens? Of course someone
>> could argue that you can make the operator non-registrable but of 
>> course
>> you don't want to have a moved splitter as undoable (you move an 
>> object,
>> resize window, now you have to undo twice to reset the object to the 
>> old
>> location?), so what's the point of having it as an operator at all if
>> you're not using operators' features?
>> So probably operators should be either tools or actions directly linked
>> to tool customization/interactivity, which in the end means nothing,
>> other than there's the need for an API parallel to operators.
>>
>> For this reason in the last few days I tried to implement an API 
>> system.
>> The main point of such a system was to achieve some kind of abstraction
>> over *functionality* and to provide a common interface to make it
>> accessible. This description looks pretty much the same as RNA's if you
>> replace "functionality" with "data" and indeed they're very connected
>> components. So in the end my idea was to create a wrapper exactly like
>> RNA but which would wrap functionality instead of data. This system
>> integrates perfectly with RNA by using RNA structs as basic type
>> definition and associating functionality to these structs by defining
>> generic "types" that use RNA structs as their data structurization. The
>> implementation is much simpler than it may sound especially because I'm
>> tired and probably I'm explaining it badly.
>>
>> patch: http://vekoon.googlecode.com/files/blender_api_0.1.patch
>> Tried on scons/mingw and cmake/msvc8
>>
>> How the definition looks like (very similar to RNA):
>>
>> void API_def_object(struct BlenderAPI *bapi)
>> {
>>     TypeAPI *tapi;
>>     FunctionAPI *func;
>>     ParameterAPI *parm;
>>
>>     tapi= API_def_type(bapi, "Object", "ID");
>>     API_def_type_struct(tapi, "Object");
>>     API_def_type_ui_description(tapi, "Object type");
>>
>>
>>     func= API_def_function(tapi, "move");
>>     API_def_function_call(func, "api_Object_move_call");
>>     API_def_function_description(func, "Move the object by an offset
>> relative to its current position.");
>>
>>     parm= API_def_parameter(func, "x");
>>     API_def_parameter_type(parm, "Float");
>>     API_def_parameter_ui_description(parm, "Offset on the X axis");
>>
>>     parm= API_def_parameter(func, "y");
>>     API_def_parameter_type(parm, "Float");
>>     API_def_parameter_ui_description(parm, "Offset on the Y axis");
>>
>>     parm= API_def_parameter(func, "z");
>>     API_def_parameter_type(parm, "Float");
>>     API_def_parameter_ui_description(parm, "Offset on the Z axis");
>>
>>
>>     func= API_def_function(tapi, "swap");
>>     API_def_function_call(func, "api_Object_swap_call");
>>     API_def_function_description(func, "Swap this object's location 
>> with
>> another object.");
>>
>>     parm= API_def_parameter(func, "ob");
>>     API_def_parameter_type(parm, "Object");
>>     API_def_parameter_flag(parm, PARAM_POINTER);
>>     API_def_parameter_ui_description(parm, "The target object this
>> object's location must swap with.");
>> }
>>
>> How it looks in usage:
>>
>> camera = bpy.objects['Camera']
>> cube = bpy.objects['Cube']
>> camera.move(0.8,0.7,0.4)
>> camera.swap(cube)
>> cube.rename('Camera');
>> camera.rename('Cube');
>> cube.rename('Camera');
>> # update changes
>> for scene in bpy.scenes:
>>     scene.refresh()
>>
>> Note: the script above already works if you apply the patch although as
>> a script it doesn't make a lot of sense but it's just to show 
>> something.
>> The definition may look a bit verbose I know, but in the end there's a
>> lot of copy/paste involved. I also initially thought of using RNA or
>> other form of structurization for parameter lists but then decided to
>> instead redefine parameters for every function as it introduced too 
>> many
>> difficulties not to do so. At most one can define a function
>> API_def_something_params(...) which defines a common list of parameters
>> and reuse it for similar function and the result will more or less be
>> the same.
>>
>> Note also that this is just a demo implementation far from being 
>> perfect
>> or even complete. Various things are missing or yet to be decided or
>> changed etc. It's just to show how it could work and that it works but
>> it's just an idea, I just thought some functional code was better than
>> written theories. There are some low level things like parameter
>> allocation/handling which may look bad but they work and they're
>> probably faster than doing it in other ways. I also initially thought 
>> of
>> using ID properties groups as operators do but then dropped as it was
>> not performance friendly. In the end operators need this because they
>> have to be savable while instead API parameters don't and that's the
>> point of having a parallel API for things that don't need operator
>> features so they can be faster.
>>
>> To conclude this message I'd also like to point out some of the nice
>> advantages/features of this system. First that comes to mind is that
>> this system is generalized exactly like RNA so in theory can be ported
>> automatically to any other scripting language and probably the most
>> interesting part is the reverse where you can automatically import into
>> the system stuff defined into the scripts and thereby for instance use 
>> a
>> class defined in python from withing a Lua script. This although
>> possible is not implemented yet as I'm not a python expert and I don't
>> know how much time that would have required. Connected to this is that
>> API defined through this system would of course be accessible
>> automatically from the GE so you could theoretically create a plugin in
>> C (or even in python if we export the API access calls themselves!) 
>> that
>> allows Lua access and then code your game entirely in Lua.
>> Another advantage is automatic extension. For instance you could just
>> create a class in py that inherits from GraphSpace thus creating a new
>> space type and extend it by overriding its methods and there you have a
>> custom editor. This again is not implemented and will require some
>> thinking but I'm pretty sure is feasible.
>>
>> Thanks for reading and any comment/critique is well accepted.
>>
>>
>> _______________________________________________
>> Bf-taskforce25 mailing list
>> Bf-taskforce25 at blender.org
>> http://lists.blender.org/mailman/listinfo/bf-taskforce25
>>
>>     
>
> _______________________________________________
> Bf-taskforce25 mailing list
> Bf-taskforce25 at blender.org
> http://lists.blender.org/mailman/listinfo/bf-taskforce25
>
>