[Bf-committers] Blender and Asyncio

[Jacob Merrill]

Maybe you can apply this to blender verse?

it really needs it*

On Mon, Mar 27, 2017 at 3:50 AM, Andreas Klostermann <
andreasklostermann at gmail.com> wrote:

> >
> >
> >
> >     - A different way to use dialogs; from just browsing through the
> >       code it's hard to see the end-user's benefit here. I also don't
> >       quite understand the need to define and register operators from
> >       inside an async function.
> >
> > No benefit to the end user. It's just for scripting pleasure. It enables
> you to express a fairly complex workflow or even an automated UI test with
> the convenient "async/await" syntax. The operators needed to be registered
> because operators have severe limitations like no return value and no
> callbacks. I think I know a way around this by using one-time tokens, but
> well, it works this way too.
>
>
> >     - The _run_once() function is interesting, since it allows running
> >       asyncio tasks for a short period of time. This hooks into:
> >
> >     - The web server acts as an example of an asyncio task. Have you
> >       compared the web server's performance with an aiohttp server
> >       running outside of Blender? And have you compared your aiohttp
> >       approach to running some other HTTP server implementation in,
> >       say, another thread or process? I'm also very interested in
> >       seeing the effect on Blender itself when the asyncio webserver
> >       is being stressed.
> >
> >
> I'm completely disinterested in the performance of the http server, or
> anything run in asyncio for that matter. If the workload is 90%
> asyncio/http and 10% blender, then it shouldn't be done inside blender or
> distributed more wisely. Never ever would I suggest connecting this http
> server to the internet, and performance is the least of the concerns there.
> That said, aiohttp is among the most performant options in Python, has the
> most community support compared to other asyncio http frameworks and it
> comes with the least baggage.
>
> In general I haven't seen any real performance issues with asyncio and
> blender. The animations still run smoothly if they would without asyncio.
> If a particular slice of a task takes too much time, of course you will see
> a reduction in fps. I haven't encountered that problem yet, but it then
> would be easy to throw in a couple of extra await calls to distribute the
> workload over time. The loop can also be adapted to check how many time per
> iteration is spent inside or outside the asyncio callbacks, so as to give
> other parts of blender more space in the main loop cycle.
>
>
> > I really like asyncio, but I think personal interest is not enough to
> > warrant building things into Blender. Building web servers into
> > Blender isn't something I see a wide need for. However, downloading
> > things asynchronously is something we certainly have a use for
>
>
> >
>     - the package manager
> >     - Blender Cloud add-on
> >     - add-ons that now bundle large datasets in their scripts
> >       directory (which they shouldn't)
> >
> >
> I don't want to include the http server stuff into Blender, either. It
> should be an optional plugin. There are two major ways to use an http
> server from inside a blender instance:
> - web user interfaces for tasks in which blender's UI is really bad at
> (text editing, browsing data, documenting)
> - RPC through REST API
>
> The latter is the more useful, actually.
> - A website with a shop could for example keep a few blender instances
> running as workers, and when the site needs to render a specially
> configured product, the options are passed via rpc and the image is then
> "downloaded" as response.
> - An external asset manager (usually a webapp I guess) can keep blender
> workers around to render previews or find out what assets are where. The
> alternative, without RPC or inter-process communication, would have to
> create a new blender instance for each call, run a script and take
> parameters from the command line or a temporary file.
> - A render farm could get more insight into the progress and state of a
> blender instance.
> - Blend4Web could get resources or other information directly from inside
> an instance. Maybe even update on-the-fly
> - A plugin for the VSE, run inside one blender instance, could start and
> control another instance in background mode, with a blend file to create
> animated captions
> - it may be a good idea to implement certain integration tests via RPC,
> especially across multiple blender builds to look for regressions. But this
> can also be done through a script with asyncio and no RPC.
>
> The main reason to do this as an http-based REST API is because it is a
> well understood paradigm with a large ecosystem. There are other options,
> like zeromq, which are often faster and more efficient, but they have
> problems of their own. For example, the REST API makes it relatively easy
> to access it from javascript, for example in a web-ui. Another benefit is,
> that other languages, or other Python environments can get better access to
> controlling Blender, but I consider that a very minor advantage.
>
> Examples for useful web-uis:
> - Image/Texture browser
> - better Python editor
> - "second screen" controls for animation, for example on a tablet
> - a cross-platform way to access certain sensors, like camera, microphone
> or joysticks from the corresponding web apis
>
> Another big use of asyncio would be interprocess communication with foreign
> processes started/controlled from blender. For example a video encoder, an
> external renderer, or even a python process which does some heavy computing
> using Numba, Cython or Theano, maybe even using the GPU, from another
> Python installation (thus avoiding the compatibility problems). This is a
> good way to escape the usual one thread/process/core paradigm in Blender
> scripting.
>
> Now, as to the question for why multi-threading is not such a good idea
> here: multi-threading is really hard. Especially because the Blender C API,
> at least in combination with Python isn't that threadsafe at all. There
> would always have to be a layer of indirection to pass data from one thread
> to another in order to not mess things up. Same with putting the
> asynchronous stuff inside a child process.
>
> When an asyncio callback is processed, the developer can assume blender is
> not doing anything else. Which makes the entire system easier to reason
> about. Also, the developer decides where to break up tasks, instead of
> letting the operating system decide. Much more predictable.
>
> Blender already uses the paradigm of an event loop. Modal Operators work
> that way, too. When code inside an operator is run, nothing else happens.
> It's just that blender so far (as far as I know) doesn't use asynchronous
> non-blocking IO with its event loop.
>
> Sorry for the very long post. I don't intend to implement all or most of
> the ideas above. My interests in Blender currently are: Video Editing, 3D
> Printing and Python hacking, in that order. The integration with the
> blender-cloud is probably the thing that will interest/benefit most users,
> with the rest I described being mainly of interest to scripters in studios
> for implementing specific workflows.
>
> Greetings,
> Andreas
> _______________________________________________
> Bf-committers mailing list
> Bf-committers at blender.org
> https://lists.blender.org/mailman/listinfo/bf-committers
>