[Bf-committers] jemalloc blender tests.

[Campbell Barton]

In my tests Guardedalloc was still being used, just with jemalloc
rather then the systems.
To test on *nix you dont even need to rebuild :)

LD_PRELOAD=/usr/local/lib/libjemalloc.so ./blender.bin

On Wed, Apr 14, 2010 at 2:14 PM, joe <joeedh at gmail.com> wrote:
> Guardedalloc adds a great deal more overhead to allocations then
> system allocators do, which I think explains much of it (and too
> fine-grained allocations of course).
>
> Joe
>
> On Wed, Apr 14, 2010 at 2:08 AM, Campbell Barton <ideasman42 at gmail.com> wrote:
>> Recently I was looking into blender using a lot more memory then it
>> should, where blender reported ~75mb, and the OS reported 750mb.
>> This made one of the animators run out of ram, so I did a lot of
>> checks and made sure it wasn't a leak (calling exit() and have
>> valgrind report all unfree'd memory works pretty well).
>>
>> It turns out that this is *(most likely) because of memory
>> fragmentation, so I tried malloc replacements 'hoard' and 'jemalloc',
>> which had tried before and found gave minimal/no performance gains.
>> But they do help with using less memory from the OS.
>> (Keep in mind, using 64bit linux - ubuntu karmic, mileage would vary
>> depending on the OS ofcourse).
>>
>> jemalloc gave better results so did some further tests.
>> On loading the same file it used ~500mb rather then ~750mb. This was a
>> scene with sintel so I suspect fragmentation could be because of
>> vertex groups but not 100% sure.
>>
>> Then tested the alley scene, jemalloc <-> linux's default malloc,
>> interesting results!
>> (scene http://durian.blender.org/wp-content/uploads/2010/03/alley.png)
>> (maximum memory used, from the OS's memory monitor. unscientifically
>> took notes while rendering)
>> new file (idle): 53mb <-> 55mb
>> loaded alley scene (idle): 671mb <-> 1300mb
>> render shadowbuffer: 5.5gig <-> 6.8gig
>> raytree: 3.6gig <-> 4.0gig
>> sss: 3.1gig <-> 4.1gig
>> rendering (/w gi): 6.2gig <-> 7.2gig
>> after rendering (idle): 4.0gig <-> 6.7gig
>> new file after render (idle): 140mb <-> 3.1gig
>>
>> >From what I have read, linux malloc implementation is considered to be
>> quite good, (not worth replacing), so I'm guessing blender is using
>> more fine grained malloc's then it should be.
>>
>> Im not sure of the details, if libc is keeping the ram for the
>> application, but when you want to have more then 1 blender loaded at a
>> time, its annoying to have to restart it just to free the memory.
>>
>> So I'd be interested if someone could test memory usage on windows
>> with jemalloc since we get a lot of complaints about memory on windows
>> systems,
>>
>> see: http://www.canonware.com/jemalloc/
>> note: needed to compile with "./configure --disable-tcache", it was
>> crashing when rendering. (bug reported)
>> note: rendertimes were not noticeably different.
>>
>> According to the site http://www.canonware.com/jemalloc/
>> facebook server, firefox and FreeBSD use jemalloc.
>>
>> Could be worth including with blender, especially if it helps windows
>> users from running out of ram so easily.
>>
>> --
>> - Campbell
>> _______________________________________________
>> Bf-committers mailing list
>> Bf-committers at blender.org
>> http://lists.blender.org/mailman/listinfo/bf-committers
>>
> _______________________________________________
> Bf-committers mailing list
> Bf-committers at blender.org
> http://lists.blender.org/mailman/listinfo/bf-committers
>



-- 
- Campbell