[Bf-committers] import_obj.py - Reimplementation

[Kursad Karatas]

The mesh is around 3.5gb single mesh obj.  Blender puffed up all the way 
to 26.8gb in ram and stopped loading  after sometime. I have 32gb here 
so that should be ok. In any case 26.8 gb for this mesh sounds a bit 
excessive to me.

Keeping the vert order definitely helped. Blender finally showed some 
signs of life after  sometime. But I was not able to interact with 
Blender after that point (massive slowdown due to mesh size I guess) . 
Maybe that is the best it can do at the moment. Just load it. Once it 
loaded the mesh the memory use went down to 11.08 gb although Blender is 
not in usable interaction state once that mesh was in the viewport.

thanks

k




On 01/03/2013 10:58 AM, CoDEmanX wrote:
> OBJ import can take quite some time and use a lot memory, but only if
> you use splitting ("keep vert order" imports way faster). Is there
> something else causing the bad performance?
>
> Multiprocessing would be great, for splitting it might work. But
> probably not for any code that calls bpy api functions...
>
> C/C++ importer: This year's GSoC project "FBX importer" wasn't actually
> about FBX, but assimp integration (= C/C++ importers for FBX, OBJ etc.).
> There are builds available at graphicall.org. However, I'd keep the
> python addons in trunk.
>
> Python + Lib: Why improve performance of a python script if a C/C++
> importer is available (assimp) and faster than the script will ever be?
> In addition, I/O scripts should be kept simple IMO, so that scripters
> can learn from them for their own addons.
>
> If there are any simple python hacks to improve performance, they should
> be applied of course.
>
>
> Am 03.01.2013 03:16, schrieb Isaac Lenton:
>> Hi
>>
>> A few days ago I was using Blender's import_obj.py script to import a few
>> rather bulky OBJ files into blender (200MB and above).  On my little system
>> it takes about 2 minutes for every file.  I have been looking into improving
>> this script with simple optimisation, splitting the work amongst multiple
>> workers and changing the amount of memory the script uses.  Some of these
>> methods are successful, but the results are only minor.  I did manage to
>> reduce the loading time of the 200MB file to 1 minute 40 seconds.
>>
>> Although a quicker method for importing my data into Blender would be to
>> use another format, I quite like the OBJ format and I am "in the mood" for
>> optimising.
>>
>> In order to significantly improve the speed of Blender's OBJ importer I can
>> see a couple of options:
>>
>>       - Implement the importer straight in C/C++ and build into Blender.
>>           I think this option is probably undesirable considering it takes
>>           away one of the really nice things about Blender's add-ons (the
>>           add-on part!).
>>
>>       - Keep the implementation in pure python.
>>           Also, in my opinion, undesirable.  The Python is great, but it can
>>           slither only so fast.  (Also, what are your opinions on using Python's
>>           multiprocessing module in import scripts?)
>>
>>       - Keep the python script but implement the most expensive methods in
>>           a C dynamic library.  This option will keep the interface to the
>>           exporter exactly the same, the module will still be able to be
>>           imported using pythons `import` and Blender will not even notice
>>           the difference.  The library would be placed in the same directory
>>           as the script (scripts/addons/io_scene_obj) and the script would
>>           simply add the line `from . import TheNewLibrary` at the start.
>>           Downside: using plain C and python can be painful (but fun).
>>
>>       - Keep the python script but implement the most expensive methods in
>>           a boost::python C++ dynamic library.  Same as above.
>>           Downside: Boost::Python libraries have a big overhead.
>>
>> I find for my own code, when size is not important, I prefer the last option.
>>
>> I am interested to hear any opinions on the above ideas, and any ideas anyone
>> may have on generally improving the speed of Blender's importers.  Are any
>> of these options Blender compatible (as in, could they be included in the
>> official release)?
>>
>> Thanks,
>> Isaac Lenton
>> (ilent2)
>> _______________________________________________
>> Bf-committers mailing list
>> Bf-committers at blender.org
>> http://lists.blender.org/mailman/listinfo/bf-committers
>>
> _______________________________________________
> Bf-committers mailing list
> Bf-committers at blender.org
> http://lists.blender.org/mailman/listinfo/bf-committers
>


-- 
Kursad Karatas
http://portfolio.plecxus.com/