[Bf-committers] Script files layout (proposed changes).

[Martin Poirier]

Hey,

Didn't you say you were going to get more concrete data on loading speed the last time we talked about this? If you did, now would be a good time to show it.

--- On Tue, 3/15/11, Campbell Barton <ideasman42 at gmail.com> wrote:

> There are a few problems with the
> current way we load scripts on startup
> * too many modules in sys.modules (96 for blender alone),
>   in most cases this is only to make use of
> auto-loading feature and
> we don't need them directly imported, they only pollute the
> namespace.

sys.modules is a dictionary, it doesn't matter how many there are, access speed is constant.

What namespace do they pollute?

> * too many search paths in sys.path, causes python to do
> over 4000
> open(...) commands (on my system, at least).
>   rough guess is we could get this down to 1500-2000.
> 
> * failed stat/open calls slow startup. see below for
> example below.
>   rough guess based on previous tests - 8% - 15%
> overall slowdown, so
> not huge but would vary depending on the OS, drivers,
> hardware.

Is that cold start stats or average stats?
Did any one else do some timings?
Did you try on more than one machine?
Do we have stats for other OS?

Aren't we drawing conclusions with a very limited set of data...

> ---
> 
> Proposal:
> Have a single auto-load directory and group scripts as
> packages.
> *note* this won't effect addons at all and wont break
> scripts so its
> mostly an internal blender/python topic.
> 
> existing layout
>  - scripts/addons,
>  - scripts/keyingsets,
>  - scripts/io,
>  - scripts/op,
>  - scripts/ui,
>  - scripts/modules,
> 
> proposed layout
>  - scripts/addons,
>  - scripts/startup, <-- only auto-load dir
>  - scripts/modules,
> 
> This way we can have...
> "scripts/ui" --> would move to "scripts/startup/ui" and
> become its own
> package-module, with __init__.py
> 
> Any scripts which don't make sense to have as packages
> could be placed
> directly in scripts/startup/ as a file though I'd like to
> avoid this.

This will not reduce the amount of entries in the module cache (each imported subpackage gets an entry) and won't reduce the number of unique import calls: subpackage defining RNA subtypes will have to be imported anyway, back to square one (or very near). Not without serious repackaging of all of these modules at least.

The only thing this does is remove three entries from sys.path, which is fine but is it worth moving everything around?

Now, I could very well be wrong, but it's hard to be sure without any kind of data on what that change accomplishes.

Martin
PS: I didn't want this to sound so negative. On second reading, it does a bit. C'est la vie.