[Bf-committers] Script files layout (proposed changes).

Campbell Barton ideasman42 at gmail.com
Wed Mar 16 00:30:58 CET 2011


There are a few problems with the current way we load scripts on startup
* too many modules in sys.modules (96 for blender alone),
  in most cases this is only to make use of auto-loading feature and
we don't need them directly imported, they only pollute the namespace.

* too many search paths in sys.path, causes python to do over 4000
open(...) commands (on my system, at least).
  rough guess is we could get this down to 1500-2000.

* failed stat/open calls slow startup. see below for example below.
  rough guess based on previous tests - 8% - 15% overall slowdown, so
not huge but would vary depending on the OS, drivers, hardware.

---

Proposal:
Have a single auto-load directory and group scripts as packages.
*note* this won't effect addons at all and wont break scripts so its
mostly an internal blender/python topic.

existing layout
 - scripts/addons,
 - scripts/keyingsets,
 - scripts/io,
 - scripts/op,
 - scripts/ui,
 - scripts/modules,

proposed layout
 - scripts/addons,
 - scripts/startup, <-- only auto-load dir
 - scripts/modules,

This way we can have...
"scripts/ui" --> would move to "scripts/startup/ui" and become its own
package-module, with __init__.py

Any scripts which don't make sense to have as packages could be placed
directly in scripts/startup/ as a file though I'd like to avoid this.

--- details (if you're interested) ---

counted 96 blender modules with:
  len([m for m in __import__("sys").modules.values() if '/scripts/' in
getattr(m, "__file__", "")])


Example of open() being called a lot for a single module.
open("/data/src/blender/cmake_debug/bin/2.56/scripts/io/urllib.cpython-32d.so",
O_RDONLY) = -1 ENOENT (No such file or directory)
open("/data/src/blender/cmake_debug/bin/2.56/scripts/io/urllibmodule.cpython-32d.so",
O_RDONLY) = -1 ENOENT (No such file or directory)
open("/data/src/blender/cmake_debug/bin/2.56/scripts/io/urllib.abi3.so",
O_RDONLY) = -1 ENOENT (No such file or directory)
open("/data/src/blender/cmake_debug/bin/2.56/scripts/io/urllibmodule.abi3.so",
O_RDONLY) = -1 ENOENT (No such file or directory)
open("/data/src/blender/cmake_debug/bin/2.56/scripts/io/urllib.so",
O_RDONLY) = -1 ENOENT (No such file or directory)
open("/data/src/blender/cmake_debug/bin/2.56/scripts/io/urllibmodule.so",
O_RDONLY) = -1 ENOENT (No such file or directory)
open("/data/src/blender/cmake_debug/bin/2.56/scripts/io/urllib.py",
O_RDONLY) = -1 ENOENT (No such file or directory)
open("/data/src/blender/cmake_debug/bin/2.56/scripts/io/urllib.pyc",
O_RDONLY) = -1 ENOENT (No such file or directory)
open("/data/src/blender/cmake_debug/bin/2.56/scripts/op/urllib.cpython-32d.so",
O_RDONLY) = -1 ENOENT (No such file or directory)
open("/data/src/blender/cmake_debug/bin/2.56/scripts/op/urllibmodule.cpython-32d.so",
O_RDONLY) = -1 ENOENT (No such file or directory)
open("/data/src/blender/cmake_debug/bin/2.56/scripts/op/urllib.abi3.so",
O_RDONLY) = -1 ENOENT (No such file or directory)
open("/data/src/blender/cmake_debug/bin/2.56/scripts/op/urllibmodule.abi3.so",
O_RDONLY) = -1 ENOENT (No such file or directory)
open("/data/src/blender/cmake_debug/bin/2.56/scripts/op/urllib.so",
O_RDONLY) = -1 ENOENT (No such file or directory)
open("/data/src/blender/cmake_debug/bin/2.56/scripts/op/urllibmodule.so",
O_RDONLY) = -1 ENOENT (No such file or directory)
open("/data/src/blender/cmake_debug/bin/2.56/scripts/op/urllib.py",
O_RDONLY) = -1 ENOENT (No such file or directory)
open("/data/src/blender/cmake_debug/bin/2.56/scripts/op/urllib.pyc",
O_RDONLY) = -1 ENOENT (No such file or directory)
open("/data/src/blender/cmake_debug/bin/2.56/scripts/ui/urllib.cpython-32d.so",
O_RDONLY) = -1 ENOENT (No such file or directory)
open("/data/src/blender/cmake_debug/bin/2.56/scripts/ui/urllibmodule.cpython-32d.so",
O_RDONLY) = -1 ENOENT (No such file or directory)
open("/data/src/blender/cmake_debug/bin/2.56/scripts/ui/urllib.abi3.so",
O_RDONLY) = -1 ENOENT (No such file or directory)
open("/data/src/blender/cmake_debug/bin/2.56/scripts/ui/urllibmodule.abi3.so",
O_RDONLY) = -1 ENOENT (No such file or directory)
open("/data/src/blender/cmake_debug/bin/2.56/scripts/ui/urllib.so",
O_RDONLY) = -1 ENOENT (No such file or directory)
open("/data/src/blender/cmake_debug/bin/2.56/scripts/ui/urllibmodule.so",
O_RDONLY) = -1 ENOENT (No such file or directory)
open("/data/src/blender/cmake_debug/bin/2.56/scripts/ui/urllib.py",
O_RDONLY) = -1 ENOENT (No such file or directory)
open("/data/src/blender/cmake_debug/bin/2.56/scripts/ui/urllib.pyc",
O_RDONLY) = -1 ENOENT (No such file or directory)
open("/data/src/blender/cmake_debug/bin/2.56/scripts/urllib.cpython-32d.so",
O_RDONLY) = -1 ENOENT (No such file or directory)
open("/data/src/blender/cmake_debug/bin/2.56/scripts/urllibmodule.cpython-32d.so",
O_RDONLY) = -1 ENOENT (No such file or directory)
open("/data/src/blender/cmake_debug/bin/2.56/scripts/urllib.abi3.so",
O_RDONLY) = -1 ENOENT (No such file or directory)
open("/data/src/blender/cmake_debug/bin/2.56/scripts/urllibmodule.abi3.so",
O_RDONLY) = -1 ENOENT (No such file or directory)
open("/data/src/blender/cmake_debug/bin/2.56/scripts/urllib.so",
O_RDONLY) = -1 ENOENT (No such file or directory)
open("/data/src/blender/cmake_debug/bin/2.56/scripts/urllibmodule.so",
O_RDONLY) = -1 ENOENT (No such file or directory)
open("/data/src/blender/cmake_debug/bin/2.56/scripts/urllib.py",
O_RDONLY) = -1 ENOENT (No such file or directory)
open("/data/src/blender/cmake_debug/bin/2.56/scripts/urllib.pyc",
O_RDONLY) = -1 ENOENT (No such file or directory)
open("/data/src/blender/cmake_debug/bin/2.56/scripts/modules/urllib.cpython-32d.so",
O_RDONLY) = -1 ENOENT (No such file or directory)
open("/data/src/blender/cmake_debug/bin/2.56/scripts/modules/urllibmodule.cpython-32d.so",
O_RDONLY) = -1 ENOENT (No such file or directory)
open("/data/src/blender/cmake_debug/bin/2.56/scripts/modules/urllib.abi3.so",
O_RDONLY) = -1 ENOENT (No such file or directory)
open("/data/src/blender/cmake_debug/bin/2.56/scripts/modules/urllibmodule.abi3.so",
O_RDONLY) = -1 ENOENT (No such file or directory)
open("/data/src/blender/cmake_debug/bin/2.56/scripts/modules/urllib.so",
O_RDONLY) = -1 ENOENT (No such file or directory)
open("/data/src/blender/cmake_debug/bin/2.56/scripts/modules/urllibmodule.so",
O_RDONLY) = -1 ENOENT (No such file or directory)
open("/data/src/blender/cmake_debug/bin/2.56/scripts/modules/urllib.py",
O_RDONLY) = -1 ENOENT (No such file or directory)
open("/data/src/blender/cmake_debug/bin/2.56/scripts/modules/urllib.pyc",
O_RDONLY) = -1 ENOENT (No such file or directory)
open("/opt/py32/lib/python3.2/urllib/__init__.cpython-32d.so",
O_RDONLY) = -1 ENOENT (No such file or directory)
open("/opt/py32/lib/python3.2/urllib/__init__module.cpython-32d.so",
O_RDONLY) = -1 ENOENT (No such file or directory)
open("/opt/py32/lib/python3.2/urllib/__init__.abi3.so", O_RDONLY) = -1
ENOENT (No such file or directory)
open("/opt/py32/lib/python3.2/urllib/__init__module.abi3.so",
O_RDONLY) = -1 ENOENT (No such file or directory)
open("/opt/py32/lib/python3.2/urllib/__init__.so", O_RDONLY) = -1
ENOENT (No such file or directory)
open("/opt/py32/lib/python3.2/urllib/__init__module.so", O_RDONLY) =
-1 ENOENT (No such file or directory)
open("/opt/py32/lib/python3.2/urllib/__init__.py", O_RDONLY) = 18
open("/opt/py32/lib/python3.2/urllib/__pycache__/__init__.cpython-32.pyc",
O_RDONLY) = 19

-- 
- Campbell


More information about the Bf-committers mailing list