[Bf-committers] import_obj.py - Reimplementation

[Isaac Lenton]

Hi

A few days ago I was using Blender's import_obj.py script to import a few
rather bulky OBJ files into blender (200MB and above).  On my little system
it takes about 2 minutes for every file.  I have been looking into improving
this script with simple optimisation, splitting the work amongst multiple
workers and changing the amount of memory the script uses.  Some of these
methods are successful, but the results are only minor.  I did manage to
reduce the loading time of the 200MB file to 1 minute 40 seconds.

Although a quicker method for importing my data into Blender would be to
use another format, I quite like the OBJ format and I am "in the mood" for
optimising.

In order to significantly improve the speed of Blender's OBJ importer I can
see a couple of options:

    - Implement the importer straight in C/C++ and build into Blender.
        I think this option is probably undesirable considering it takes
        away one of the really nice things about Blender's add-ons (the
        add-on part!).

    - Keep the implementation in pure python.
        Also, in my opinion, undesirable.  The Python is great, but it can
        slither only so fast.  (Also, what are your opinions on using Python's
        multiprocessing module in import scripts?)

    - Keep the python script but implement the most expensive methods in
        a C dynamic library.  This option will keep the interface to the
        exporter exactly the same, the module will still be able to be
        imported using pythons `import` and Blender will not even notice
        the difference.  The library would be placed in the same directory
        as the script (scripts/addons/io_scene_obj) and the script would
        simply add the line `from . import TheNewLibrary` at the start.
        Downside: using plain C and python can be painful (but fun).

    - Keep the python script but implement the most expensive methods in
        a boost::python C++ dynamic library.  Same as above.
        Downside: Boost::Python libraries have a big overhead.

I find for my own code, when size is not important, I prefer the last option.

I am interested to hear any opinions on the above ideas, and any ideas anyone
may have on generally improving the speed of Blender's importers.  Are any
of these options Blender compatible (as in, could they be included in the
official release)?

Thanks, 
Isaac Lenton
(ilent2)