[Bf-committers] jemalloc blender tests.

[Campbell Barton]

Recently I was looking into blender using a lot more memory then it
should, where blender reported ~75mb, and the OS reported 750mb.
This made one of the animators run out of ram, so I did a lot of
checks and made sure it wasn't a leak (calling exit() and have
valgrind report all unfree'd memory works pretty well).

It turns out that this is *(most likely) because of memory
fragmentation, so I tried malloc replacements 'hoard' and 'jemalloc',
which had tried before and found gave minimal/no performance gains.
But they do help with using less memory from the OS.
(Keep in mind, using 64bit linux - ubuntu karmic, mileage would vary
depending on the OS ofcourse).

jemalloc gave better results so did some further tests.
On loading the same file it used ~500mb rather then ~750mb. This was a
scene with sintel so I suspect fragmentation could be because of
vertex groups but not 100% sure.

Then tested the alley scene, jemalloc <-> linux's default malloc,
interesting results!
(scene http://durian.blender.org/wp-content/uploads/2010/03/alley.png)
(maximum memory used, from the OS's memory monitor. unscientifically
took notes while rendering)
new file (idle): 53mb <-> 55mb
loaded alley scene (idle): 671mb <-> 1300mb
render shadowbuffer: 5.5gig <-> 6.8gig
raytree: 3.6gig <-> 4.0gig
sss: 3.1gig <-> 4.1gig
rendering (/w gi): 6.2gig <-> 7.2gig
after rendering (idle): 4.0gig <-> 6.7gig
new file after render (idle): 140mb <-> 3.1gig

>From what I have read, linux malloc implementation is considered to be
quite good, (not worth replacing), so I'm guessing blender is using
more fine grained malloc's then it should be.

Im not sure of the details, if libc is keeping the ram for the
application, but when you want to have more then 1 blender loaded at a
time, its annoying to have to restart it just to free the memory.

So I'd be interested if someone could test memory usage on windows
with jemalloc since we get a lot of complaints about memory on windows
systems,

see: http://www.canonware.com/jemalloc/
note: needed to compile with "./configure --disable-tcache", it was
crashing when rendering. (bug reported)
note: rendertimes were not noticeably different.

According to the site http://www.canonware.com/jemalloc/
facebook server, firefox and FreeBSD use jemalloc.

Could be worth including with blender, especially if it helps windows
users from running out of ram so easily.

-- 
- Campbell