[Bf-committers] Blender 2.5 api proposal

[Campbell Barton]

On Mon, Dec 1, 2008 at 9:56 PM, Toni Alatalo <antont at kyperjokki.fi> wrote:
> I agree with Cambpell and Joe that dropping the old api in 2.50, but
> helping with a py written Blender module with functions people are
> familiar with, is probably the way to go. People depending on existing
> scripts that are not ported can stick to 2.4x in the time being.
>
> Without knowing the 2.50 work well but based on the reading have done
> in the wikis etc. the plans seem good to me otherwise too.
>
> One concern I have is PyRNA's ability to support NumPy, i.e.
> fast/direct access to memory buffers like the positions of all vertices
> of a mesh, all particles of a particle system, or the values of all
> pixels in a compositing node, to process them efficiently from py
> (change all of them in one py call).
Your right, PyRNA does not give especially fast access.
Id prefer de

> I am not a great expert there, but recently got more experience by
> writing a real-time thing with pygame & numpy for an artist friend
> where am manipulating all pixels individually with numpy ops, and after
> got the hang of it found it a great way to work. So am still guessing
> that it'll be a killer feat for at least compositing nodes, but
> possibly for 3d things too (would be fun to try to manipulate mesh
> geoms, but afaik there is no tool yet that supports that (unless
> pyopengl directly somehow?)).
>
> As Theo de Ridder told and showed us in the 2007 conf, all that's
> needed to enable numpy manipulation is to implement the minimal Python
> buffer protocol on the c side, which basically is just returning a
> pointer to the array. It was revised for Python 3000, which is also
> backported to 2.6 and documented in
> http://docs.python.org/c-api/objbuffer.html .. the current 2.5.2 docs
> at http://www.python.org/doc/api/bufferObjects.html tell more about the
> usage. For those who were not there, Theo had added buffer access to
> shape keys, so he could animate a swarm of fishes swimming by having
> the fishes as dupliverts and manipulating the keybuffer directly in
> memory using numpy. I asked Jahka and the particles in a particle
> systems are there similarily as a continuous memory chunk, so the same
> could be done there.
>
> For something that is not a buffer but e.g. a linked list, I guess it
> may still make sense to implement an interface to numpy in c, in order
> to not iterate on the py side (which is slow) but just define the
> operations there, and iterate in native code instead. But I haven't
> looked into that yet .. dunno if exposing even something like
> scene.objects as a numpy array might be interesting for some cases? (at
> least if Blender is restructured internally so that it'd support
> hundreds of thousands of objects more efficiently ;)
>
> So my 2 cents from outside of the inner loop is that please don't make
> it difficult to use numpy, but consider where it'd be good to support -
> thank you :) .. and do note that the support for it is *optional*, it
> does not add any runtime dependencies (that's how pygame does it too -
> the surfarray module works if you have numpy installed, but you can
> ignore it otherwise).
>
> ~Toni

Hi Toni, your right, though this isnt a problem Im trying to solve :)
Fast access to memory buffers with numpy has merits but 2.5 wont make
easier or harder, data structures are unchanged.

Theo's example only worked because shape key buffers are an array of
floats, this is not the case for most other areas of blenders mesh
storage.
If blender was more python-centric we could make sure the data was
structured in a way that Numpy could read easily.
But this can be limiting too if we switched to tile based ImLib,
access to image buffers would not be contiguous breaking Numpy access.
Further, we would need to agree on including Numpy with blender for it
to be truly useful average blender-heads.

Personally I prefer to focus on really great high level python api
that can do things like record user actions and play them back to
create custom tools.

This way python can make use of functions written in C, rather then
having to rewrite them in py like many scripts do now.

If we want Numpy integration for compositor, this can be added as a
irrespective of the 2.5 API, though I'd hold out for a GPU compositor
:-)

-- 
- Campbell