[Bf-python] API overhaul - irc meeting?

[Gilbert, Joseph T.]

"""This includes that all struct elements are available as 
attributes and derived values are available via methods"""
>>>This sounds about right. I think we've been moving towards doing this.

"""Early backward compatibility 
constraints do not allow you to correct errors or improve the interface after 
the first few iterations, although the api is still far away from convergence 
to optimum"""
>>>This reminds me that we need to decide on a clear solution regarding deprecation without breaking things in the api. One of the biggest problems with the python API is that we are always 'fixing' things and making the API better. This is good. However, we are also constantly breaking things. It gives the impression that the API is never finished so y bother writing scripts. Periodically you hear people say things like 'i'm coding this in C and avoiding python' - the reason being that people dont want to put effort into something that might break 2 months from now. 
To really get the API on track I suggest we really define how to develop the API AND not break older scripts. 

"""BPy is a thin wrapper to the C-structs. There is no caching inbetween. """
>>> In theory this is great, in practice it has run into problems but I think it can be done.
If python malloc's something blender should handle the clean up. For large objects like Object,Mesh,Armature this works fine. However for 'subojbects' like Bone, Vert etc. these objects are not 'directly' cleaned up by blender. i.e. creating unlinked subobjects is not cleaned up by blender.
The result is that bpy has to do cleanup of these objects - which means if tp_dealloc is not called due to ref count issues this object is leaked in memory. In the past we have avoided this however its doable if we make sure *never have refcount issues* on that object.

-Thinking about this a little there IS something that we could do....
It's possible that we could have tp_init/tp_new set for objects that blender will cleanup internally and use factory functions (e.g. myobject.newSubobject()) to generate subobjects. In many cases this might help the API. 

"""This avoids 
snycing issues resp. spreading of BPy code all over the codebase"""
>>>Actually there are more trees here in this forest. Sync'ing is a problem even with thin wrappers. If blender deletes a pointer somewhere we have a sync'ing problem. The is a particular issue with GUI scripts where the user can interact with blender while python is caching some thin-wrapped pointer somewhere.  Pointers can become disconnected internally but python still has a wrapped pointer to the subobject , which happens to be no-longer linked to it's parent, etc. etc.

""""The same design as Blender's codebase""""
>>>This would be SO much easier if blender was designed with having an API in mind. A good example of not thinking about an API is stuff like OBACT or G.edbo. Setting bones by vector, head and tail, etc. The API typically has to find work arounds for this stuff. However if blender did have a wrappable api we would be using SWIG and I wouldn't be writing this :)


-----Original Message-----
From: bf-python-bounces at projects.blender.org on behalf of Michael Reimpell
Sent: Sun 1/8/2006 9:43 AM
To: Blender Foundation Python list
Subject: Re: [Bf-python] API overhaul - irc meeting?
 
> 6) The start: define API before coding it, create (epy)docs and tests, etc.

This is a very good idea! Blender's BPy development is definitely missing some 
tests (aka "use cases"). For example, take the Key module. You have access to 
the Key Ipo and to to Key KeyBlocks, but there is no way of getting the 
IpoCurve that belongs to a KeyBlock. A simple use case "read the ipo curve 
belonging to a relative shape" would have revealed it.

A good examples of the problems with the current BPy implementation is the 
NMesh.getMaterials() method. Due to a missing use case "get a material by 
index" the getMaterials method returned a material list with "None" materials 
removed. Of course that led to wrong material associations and index errors. 
This behaviour is still available with the what=1 option, which shows another 
problem with BPy: "premature backward compatibility".

Software development is an iterative process. This is especially true for open 
source software, where developers just add what they need and don't bother 
about other peoples usage of the software. Early backward compatibility 
constraints do not allow you to correct errors or improve the interface after 
the first few iterations, although the api is still far away from convergence 
to optimum. (In the getMaterials() case, backward compatibility just results 
in the survival of an error-prone option, see the discussion to 
http://projects.blender.org/tracker/?func=detail&atid=127&aid=3701&group_id=9 
for another example where this problem arises)

The NMesh.getMaterials() methods also shows another problem with the api: You 
can get the list of materials of the underlying Python object, or that of the 
Blender object. In other words: Although Blender has a data driven design, 
the python data is out of sync. This is obviously a problem, especially if 
you want to interact with the user input from the 3D view.

That the NMesh.getMaterials() method needs a "what" argument should ring the 
alarm bells of the BPy developers. Such crutches are a warning for problems 
with the api design. I'm glad that this was realized and led to the Mesh 
module.

To avoid such issues in the future, I propose the following design guidelines:

1) There is a one-to-one correspondence between Blender internal C-structs and 
BPy's classes. This includes that all struct elements are available as 
attributes and derived values are available via methods (I will address the 
"C-struct element names are too cryptic" and "attributes above methods" 
arguments later)

2) BPy is a thin wrapper to the C-structs. There is no caching inbetween. 
Elements created in BPy are immediately created in Blender. This avoids 
snycing issues resp. spreading of BPy code all over the codebase (e.g., into 
to dependency graph).

3) BPy classes can (and should) provide all kind of convenient methods to make 
life easier. That is, although the BPy class attributes are limited to the 
C-struct elements, there can (and should) be getters or setters for derived 
values. For example an armature bone could have a setter 
setTransformation(matrix) which changes the head, tail and roll values 
according to the given matrix.

Benefits of this approach are:
Due to the 1-1 correspondence, there are no assumptions about the use case of 
the BPy class. In principle, coding in BPy becomes as powerful as coding 
within Blender's C code. This is as flexible and feature save as possible. 
Also, due to the thin wrapper, there are no syncronization issues and BPy 
code stays located in contrast to spreading all over Blender's codebase. 
Another advantage is that script writers can more easily contribute to 
Blender's internal code, as they allready know the corresponding C-structs 
and connections from BPy.

As promised, here my comment on "most script writers are not interested in the 
internal representation". Although I can't know what most script writers are 
interested in, it is simply not possible to write any nontrivial script in 
BPy without looking into Blender's interal codebase: You want to know what 
the camera "lens" value is? You have to look into the corresponding C code. 
You want to know what the light attenuation function is? You have to look 
into the corresponding C code. In fact, having a different BPy representation 
compared to Blender's C-structs is a severe slowdown: After you checked 
Blender's C-code you have to additional check the BPy code, just to see how 
to access Blender's internals (or if at all it's possible). If you think that 
the C-struct elements are cryptic, add a meaningful comment in the api 
documentaiton! "Camera.getLens() Get the lens value" has the same information 
as no documentation at all. C developers may even start to look into the BPy 
documentation to see what the C struct elements mean.

Also, I don't understand the "attributes above methods" philosophy. Don't let 
a language dictate your design. In the proposed design attributes have a 
clear meaning  as Blender's internal data, and this meaning fits very well 
with Blender's data driven design.

The proposed BPy design can be summerized into one sentence: "The same design 
as Blender's codebase". On the one hand this concept is extremly easy to 
grab, on the other hand it is flexible and feature save. Even if you come up 
with another BPy design in the end, please check that with the arguments 
above.

Cheers,
Michael

_______________________________________________
Bf-python mailing list
Bf-python at projects.blender.org
http://projects.blender.org/mailman/listinfo/bf-python

-------------- next part --------------
A non-text attachment was scrubbed...
Name: winmail.dat
Type: application/ms-tnef
Size: 7180 bytes
Desc: not available
URL: <http://lists.blender.org/pipermail/bf-python/attachments/20060108/8a112ac2/attachment.bin>