[Bf-committers] Blender and Asyncio

Andreas Klostermann andreasklostermann at gmail.com
Mon Mar 27 12:50:13 CEST 2017


>
>
>
>     - A different way to use dialogs; from just browsing through the
>       code it's hard to see the end-user's benefit here. I also don't
>       quite understand the need to define and register operators from
>       inside an async function.
>
> No benefit to the end user. It's just for scripting pleasure. It enables
you to express a fairly complex workflow or even an automated UI test with
the convenient "async/await" syntax. The operators needed to be registered
because operators have severe limitations like no return value and no
callbacks. I think I know a way around this by using one-time tokens, but
well, it works this way too.


>     - The _run_once() function is interesting, since it allows running
>       asyncio tasks for a short period of time. This hooks into:
>
>     - The web server acts as an example of an asyncio task. Have you
>       compared the web server's performance with an aiohttp server
>       running outside of Blender? And have you compared your aiohttp
>       approach to running some other HTTP server implementation in,
>       say, another thread or process? I'm also very interested in
>       seeing the effect on Blender itself when the asyncio webserver
>       is being stressed.
>
>
I'm completely disinterested in the performance of the http server, or
anything run in asyncio for that matter. If the workload is 90%
asyncio/http and 10% blender, then it shouldn't be done inside blender or
distributed more wisely. Never ever would I suggest connecting this http
server to the internet, and performance is the least of the concerns there.
That said, aiohttp is among the most performant options in Python, has the
most community support compared to other asyncio http frameworks and it
comes with the least baggage.

In general I haven't seen any real performance issues with asyncio and
blender. The animations still run smoothly if they would without asyncio.
If a particular slice of a task takes too much time, of course you will see
a reduction in fps. I haven't encountered that problem yet, but it then
would be easy to throw in a couple of extra await calls to distribute the
workload over time. The loop can also be adapted to check how many time per
iteration is spent inside or outside the asyncio callbacks, so as to give
other parts of blender more space in the main loop cycle.


> I really like asyncio, but I think personal interest is not enough to
> warrant building things into Blender. Building web servers into
> Blender isn't something I see a wide need for. However, downloading
> things asynchronously is something we certainly have a use for


>
    - the package manager
>     - Blender Cloud add-on
>     - add-ons that now bundle large datasets in their scripts
>       directory (which they shouldn't)
>
>
I don't want to include the http server stuff into Blender, either. It
should be an optional plugin. There are two major ways to use an http
server from inside a blender instance:
- web user interfaces for tasks in which blender's UI is really bad at
(text editing, browsing data, documenting)
- RPC through REST API

The latter is the more useful, actually.
- A website with a shop could for example keep a few blender instances
running as workers, and when the site needs to render a specially
configured product, the options are passed via rpc and the image is then
"downloaded" as response.
- An external asset manager (usually a webapp I guess) can keep blender
workers around to render previews or find out what assets are where. The
alternative, without RPC or inter-process communication, would have to
create a new blender instance for each call, run a script and take
parameters from the command line or a temporary file.
- A render farm could get more insight into the progress and state of a
blender instance.
- Blend4Web could get resources or other information directly from inside
an instance. Maybe even update on-the-fly
- A plugin for the VSE, run inside one blender instance, could start and
control another instance in background mode, with a blend file to create
animated captions
- it may be a good idea to implement certain integration tests via RPC,
especially across multiple blender builds to look for regressions. But this
can also be done through a script with asyncio and no RPC.

The main reason to do this as an http-based REST API is because it is a
well understood paradigm with a large ecosystem. There are other options,
like zeromq, which are often faster and more efficient, but they have
problems of their own. For example, the REST API makes it relatively easy
to access it from javascript, for example in a web-ui. Another benefit is,
that other languages, or other Python environments can get better access to
controlling Blender, but I consider that a very minor advantage.

Examples for useful web-uis:
- Image/Texture browser
- better Python editor
- "second screen" controls for animation, for example on a tablet
- a cross-platform way to access certain sensors, like camera, microphone
or joysticks from the corresponding web apis

Another big use of asyncio would be interprocess communication with foreign
processes started/controlled from blender. For example a video encoder, an
external renderer, or even a python process which does some heavy computing
using Numba, Cython or Theano, maybe even using the GPU, from another
Python installation (thus avoiding the compatibility problems). This is a
good way to escape the usual one thread/process/core paradigm in Blender
scripting.

Now, as to the question for why multi-threading is not such a good idea
here: multi-threading is really hard. Especially because the Blender C API,
at least in combination with Python isn't that threadsafe at all. There
would always have to be a layer of indirection to pass data from one thread
to another in order to not mess things up. Same with putting the
asynchronous stuff inside a child process.

When an asyncio callback is processed, the developer can assume blender is
not doing anything else. Which makes the entire system easier to reason
about. Also, the developer decides where to break up tasks, instead of
letting the operating system decide. Much more predictable.

Blender already uses the paradigm of an event loop. Modal Operators work
that way, too. When code inside an operator is run, nothing else happens.
It's just that blender so far (as far as I know) doesn't use asynchronous
non-blocking IO with its event loop.

Sorry for the very long post. I don't intend to implement all or most of
the ideas above. My interests in Blender currently are: Video Editing, 3D
Printing and Python hacking, in that order. The integration with the
blender-cloud is probably the thing that will interest/benefit most users,
with the rest I described being mainly of interest to scripters in studios
for implementing specific workflows.

Greetings,
Andreas


More information about the Bf-committers mailing list