[Bf-taskforce25] context/operator changes

[Ton Roosendaal]

Hi,

> How would that answer my question?

I just said that we just don't make a plugin API (yet), so advanced  
features like motion tracking have to be coded as integrated C/C++  
tool.

> I asked *how* you create a custom editor?

That's not designed yet. Most likely it'll be a special WM level python  
api, not operators. Would also allow to configure keymaps, add  
handlers, etc.

-Ton-

------------------------------------------------------------------------
Ton Roosendaal  Blender Foundation   ton at blender.org    www.blender.org
Blender Institute BV  Entrepotdok 57A  1018AD Amsterdam The Netherlands

On 19 Mar, 2009, at 13:29, vekoon wrote:

> How would that answer my question? That direct API would be used from C
> side as well. What you do otherwise? You export all the Blender's
> symbols into separate libs and link against those and include 20  
> headers
> in your C plugin? Then how do you make your plugin's functionality
> available to scripts? You write 100 lines operators instead of 10  
> simple
> API_def lines?
> Also it seems like you answered to a different question regarding  
> custom
> editors. I asked *how* you create a custom editor? That's not relative
> to data access for the editor, it's just an example of how operators
> alone can't handle this. You'll need to like extend from a specific  
> type
> to create a custom editor or ui template or whatever? Then why make  
> this
> distinct and specific for every situation when you can simply create a
> more consistent and homogeneous API that does this for all (and more)?
> What about creating an additional region with custom stuff in it? Is
> that advanced functionality? You create a
> WM_OT_add_region_with_stuff_in_it operator? Again even here the most
> immediate and logic way of doing it is by extending an existing type  
> and
> again why make it a distinguished (custom) thing and how making this
> every time you need to extend something would be better (or "higher
> level") than a more consistent generic API.
>
>
> Ton Roosendaal wrote, on 19/03/2009 12.41:
>> Hi Vekoon,
>>
>> Simple answer: if you want to add advanced new functionality in  
>> Blender
>> you better code this C or C++ :)
>>
>> How custom editors ("Space types") will work is unresolved though...  
>> in
>> theory these editors can define an entire own local context, and use
>> py-opengl to draw stuff and all kinds of own defined unique operator
>> types.
>>
>> However, when such a custom editor likes to access Blender data or
>> operators, it just has to respect and follow the context specs as
>> defined for them.
>>
>> -Ton-
>>
>> ---------------------------------------------------------------------- 
>> --
>> Ton Roosendaal  Blender Foundation   ton at blender.org     
>> www.blender.org
>> Blender Institute BV  Entrepotdok 57A  1018AD Amsterdam The  
>> Netherlands
>>
>> On 19 Mar, 2009, at 12:34, vekoon wrote:
>>
>>
>>> Well, but my point is that if you leave the operators as they are it
>>> will still be an artists' tool and you solve the context problem  
>>> simply
>>> by passing the Contentx the script was run into (as it works right
>>> now).
>>> This way they'll use this operator functionality exactly as they use
>>> normal blender but in an automated (script) way. In normal
>>> functionality
>>> they select something and then act upon it (with Tools-Operators that
>>> is), so doing the same in script side will be fine for them I guess.  
>>> If
>>> you do decouple context instead, you're creating 2 different ways of
>>> using operators, which was exactly what you were willing to avoid.
>>> Also, what if they want to create their own tool that chains some  
>>> other
>>> tools and does something more, writing a custom Operator that works  
>>> for
>>> scripting will be a greater pain for them and using it as well would  
>>> be
>>> such if I'm not mistaken as you have to pass additional information  
>>> to
>>> the operator.
>>>
>>> On the other hand, you can't ignore developers or extensions writers  
>>> as
>>> those are generally tightly related to artists/users but this way, by
>>> not providing a generic and direct API, you're making things way more
>>> painful for them or even impossible in some cases.
>>>
>>> For instance, how would a custom editor be implemented using the
>>> operator system? Like let's say I want to add a match moving kind of
>>> functionality to Blender, how would I do it? (not that I'd have the
>>> skill for it, it's just an example)
>>> Or let's say that for some reason I want to draw a background image
>>> into
>>> the Graph Editor, or adding a region in 3d view to manage parametric
>>> objects or custom rig controls or I want to do something like Grase
>>> Pencil did. Yes you can integrate that into the sources but that
>>> creates
>>> huge problems an extension writer generally doesn't want to deal  
>>> with,
>>> especially when he's willing to implement a feature that unlikely  
>>> will
>>> be included into default blender as it's too much field-specific.
>>>
>>> Having a powerful parallel API for those type of things is somehow
>>> needed in my opinion as the possibilities it creates are way superior
>>> to
>>> the complexity it causes. This is as much true for extensions writers
>>> as
>>> it is for artists because easier to write extensions in the end just
>>> means more tools for end users.
>>> Basically, the way I see it, operators would just be high level API
>>> wrappers for artists to use in a very simple way and without  
>>> excessive
>>> scripting knowledge. Direct API access instead would still be  
>>> possible
>>> for more advanced usages and extensibility but still maintaining an
>>> intrinsic simplicity in its approach.
>>>
>>>
>>> Brecht Van Lommel wrote, on 19/03/2009 3.06:
>>>
>>>> Hi,
>>>>
>>>> vekoon wrote:
>>>>
>>>>
>>>>> Maybe I'm missing something but I'll try to express my opinion  
>>>>> anyway
>>>>> (it'll be long and boring, sorry).
>>>>>
>>>>> As stated in the design documents for the new operator system the
>>>>> idea
>>>>> was that Context + Operator creates an Action, but that doesn't
>>>>> necessarily mean this is the only way to achieve the Action.
>>>>> In fact from my point of view it was exactly the opposite, i.e. it
>>>>> was
>>>>> the answer to the question "How could we achieve an Action which is
>>>>> data-independent (so that it's reusable)?" and the answer was
>>>>> Context,
>>>>> because context is an abstract representation of state-dependent
>>>>> data.
>>>>> Meaning we could actually translate the above equation to (Data *
>>>>> State)
>>>>> + Operator = Action. Ok it's not meant to be mathematically  
>>>>> accurate,
>>>>> just to explain.
>>>>>
>>>>>
>>>> This is a good description of the original design.
>>>>
>>>>
>>>>
>>>>> The reason Context is used (and thought of) in an atomic way is  
>>>>> that
>>>>> as
>>>>> I said State is an abstract concept, you can't really manipulate it
>>>>> by
>>>>> itself but it represents the current State in which the Data is and
>>>>> in
>>>>> which the Action should be executed in. It's like the environment  
>>>>> for
>>>>> the action. So by keeping the Action dependent on the State,  
>>>>> because
>>>>> at
>>>>> any given time you always know what the current State is, you have  
>>>>> a
>>>>> way
>>>>> to keep both separated and to repeat the same Action under  
>>>>> different
>>>>> State conditions. This was the main reason for having  
>>>>> State-dependent
>>>>> Data in the first place.
>>>>>
>>>>> On the other hand, as much as it's true the above statement, it's
>>>>> also
>>>>> true that Data + Function = Action. In this case the State is  
>>>>> ignored
>>>>> which means the Action is directly binded to the Data. This is
>>>>> perfectly
>>>>> fine when you have some direct Data you want to act upon which by
>>>>> consequence means you can only repeat the Action when having access
>>>>> to
>>>>> that specific Data.
>>>>>
>>>>>
>>>> I guess here you are making the distinction between using an  
>>>> operator
>>>> as a tool, and in a script/plugin/node. The second use case is  
>>>> indeed
>>>> what we are trying to improve.
>>>>
>>>>
>>>>
>>>>> Now by giving access only to State-dependent data you're actually
>>>>> forcing a State change every time you want to act upon some Data,
>>>>> i.e.
>>>>> select objects A and B and then move selected objects. This is a  
>>>>> big
>>>>> limitation in many real-world circumstances.
>>>>> For people who programmed in Win32 or windows in general this is
>>>>> already
>>>>> obvious by considering the limitations a control like RichEdit  
>>>>> forces
>>>>> upon the developer for which you actually have to first mark the
>>>>> selection you want to act upon (text of course in this case) and  
>>>>> then
>>>>> invoke the command you'd like to apply (i.e. Make Bold) and which
>>>>> reflects how you act on it as a user.
>>>>>
>>>>>
>>>> This is exactly why we are making a system that decouples the state
>>>> from the operator better. We can't immediately do that to the finest
>>>> level, since there is no time for it. But text editing operators can
>>>> be extended to take selection as an input instead of the user
>>>> selection. Similarly for mesh editing and other systems.
>>>>
>>>>
>>>>
>>>>> This is the reason why other types of more complex piece of  
>>>>> software
>>>>> like firefox, internet explorer itself and other browsers in  
>>>>> general
>>>>> had
>>>>> to adopt a DOM centered way of manipulating data (those in web
>>>>> development are perfectly aware of this).
>>>>> This does not mean they dropped Context-based interactivity (which  
>>>>> in
>>>>> their case just means Selection-based). The latter indeed allows
>>>>> those
>>>>> softwares to generalize an idea of Command (which really resembles
>>>>> the
>>>>> idea of Tool-Operator in the end) which acts on current Context,  
>>>>> can
>>>>> be
>>>>> automatically described, associated with custom keystrokes,
>>>>> undone/redone or could even be macroed although they didn't  
>>>>> implement
>>>>> macros as they were really of no use in their case.
>>>>>
>>>>> My "Blender API" proposal actually tried to provide this. It would
>>>>> not
>>>>> be code duplication in that the idea would be to create a generic  
>>>>> API
>>>>> function like "move object" and then reuse this in both the
>>>>> direct-data
>>>>> and operator API. Also with a well thought generic API and some  
>>>>> minor
>>>>> changes to my BlenderAPI code we could actually automatize the
>>>>> creation
>>>>> of bridge functions between the BlenderAPI calling system and  
>>>>> normal
>>>>> C
>>>>> one by generating the corresponding C functions within the makesapi
>>>>> module which mostly consists of parameters creation/conversion and
>>>>> should be easy to do.
>>>>>
>>>>>
>>>> This is similar to what we are trying to as I see it, but with a
>>>> different implementation. The difference is that you make a
>>>> distinction between an operator and a low level function, while with
>>>> this design it is all operators.
>>>>
>>>> The reason I think doing it at the operator level is good, is  
>>>> because
>>>> then the scripting API closely matches what the user does. Blender  
>>>> is
>>>> an artists tool, not a development API. Having the scripting API  
>>>> close
>>>> to the tools can make scripting easier, and also means that the  
>>>> tools
>>>> you use can directly be used in scripts, or as nodes in the future.
>>>>
>>>>
>>>>
>>>>> I don't really see how this lower level interaction would be a
>>>>> problem
>>>>> or how this even is low level for that matter. You can't have a  
>>>>> good
>>>>> level of extensibility if you can't access the data you want where
>>>>> you
>>>>> want and act directly upon it. Other advanced programs do this and
>>>>> yes
>>>>> this adds complications, but avoiding the problem is not a solution
>>>>> especially if it means limiting the functionality a lot.
>>>>>
>>>>>
>>>> The context-decoupled operators would allow you to act on data as  
>>>> much
>>>> as the Blender data structures and code support it I think. Even
>>>> inside Blender you should not change any data at any point.
>>>>
>>>>
>>>>
>>>>> Not to be rough or anything but to me what "context decoupling"
>>>>> really
>>>>> looks like is a nicer way of calling an hack to a system that was
>>>>> good
>>>>> for its purpose in order to make it fit into something it is not
>>>>> suited
>>>>> for, attaching lot of added complexity to it.
>>>>> My idea is simple: when a py script is run it can get current
>>>>> Context,
>>>>> i.e. the Context the script was run into. From context it can pull
>>>>> out
>>>>> the context-dependent stuff it needs (like selected objects)or use
>>>>> the
>>>>> Context directly to call Operators but because operators are just
>>>>> high
>>>>> level wrappers around a generic API it really wouldn't need to
>>>>> unless he
>>>>> wants those commands to be macroed for instance. This way you have  
>>>>> a
>>>>> dynamic system, highly functional and extensible; you make  
>>>>> operators
>>>>> a
>>>>> lot simpler (removing the need of context decoupling) and you let
>>>>> devs
>>>>> take care of the full potential and the problems this potential may
>>>>> cause.
>>>>>
>>>>>
>>>> Here you describe what operators do currently. If you don't need the
>>>> operator to be used for scripting there is no point in trying to
>>>> decouple the context. If you do decouple it, then I don't see how  
>>>> this
>>>> is more work than the BlenderAPI you propose.
>>>>
>>>>
>>>>
>>>>> On a final note (also to kinda strengthen my point) look at Firefox
>>>>> and
>>>>> all the problems it has: badly documented and full-of-issues APIs,
>>>>> huge
>>>>> memory usage, complex trickeries to get more advanced stuff  
>>>>> working.
>>>>> And
>>>>> yet it's eating up on market share on every other browser  
>>>>> (especially
>>>>> IE). This is because its add-ons are extremely powerful yet
>>>>> relatively
>>>>> easy to write, and that's the key. Users search for good
>>>>> functionality,
>>>>> devs for simplicity so you have to combine both elements in order  
>>>>> to
>>>>> "succeed", even if this means not having a perfect design, which is
>>>>> impossible to get anyway.
>>>>> Probably I should have avoided this final note as it's mostly
>>>>> irrelevant
>>>>> but my point is that a complex "low level" API is always needed  
>>>>> for a
>>>>> complex application, you can't avoid it. Otherwise you end up
>>>>> creating
>>>>> various APIs for various purposes because you find you need  
>>>>> something
>>>>> for custom GUIs, so you create UI templates, then maybe skinning,  
>>>>> so
>>>>> you
>>>>> create UI styles; then you'll surely need a way to create and  
>>>>> manage
>>>>> screen layouts and there's another differently custom API. Then
>>>>> custom
>>>>> toolbars buttons creation, then menus, then custom editors, then
>>>>> what?
>>>>> You end up with tons of different custom APIs for doing all the
>>>>> different things you need to be able to do which in the end will  
>>>>> look
>>>>> exactly like having a single "low level" API but it'll be  
>>>>> subdivided
>>>>> and
>>>>> inconsistent.
>>>>>
>>>>>
>>>> For such low level API stuff, I would still like to use operators,  
>>>> but
>>>> ones that you don't use as a tool. They'd work quite similar to the
>>>> BlenderAPI probably?
>>>>
>>>>
>>>> Context decoupling does add complexity. I am however not convinced
>>>> that it adds more complexity than using the BlenderAPI concept. It
>>>> would be interesting to compare the code for an operator +  
>>>> associated
>>>> Blender API function, and the code for a context-decoupled operator.
>>>> Perhaps something better than either of both can come out of it..
>>>>
>>>> Brecht.
>>>> _______________________________________________
>>>> Bf-taskforce25 mailing list
>>>> Bf-taskforce25 at blender.org
>>>> http://lists.blender.org/mailman/listinfo/bf-taskforce25
>>>>
>>>>
>>>>
>>> _______________________________________________
>>> Bf-taskforce25 mailing list
>>> Bf-taskforce25 at blender.org
>>> http://lists.blender.org/mailman/listinfo/bf-taskforce25
>>>
>>>
>>
>> _______________________________________________
>> Bf-taskforce25 mailing list
>> Bf-taskforce25 at blender.org
>> http://lists.blender.org/mailman/listinfo/bf-taskforce25
>>
>>
> _______________________________________________
> Bf-taskforce25 mailing list
> Bf-taskforce25 at blender.org
> http://lists.blender.org/mailman/listinfo/bf-taskforce25
>