[Bf-committers] jemalloc blender tests.

[joe]

Guardedalloc adds a great deal more overhead to allocations then
system allocators do, which I think explains much of it (and too
fine-grained allocations of course).

Joe

On Wed, Apr 14, 2010 at 2:08 AM, Campbell Barton <ideasman42 at gmail.com> wrote:
> Recently I was looking into blender using a lot more memory then it
> should, where blender reported ~75mb, and the OS reported 750mb.
> This made one of the animators run out of ram, so I did a lot of
> checks and made sure it wasn't a leak (calling exit() and have
> valgrind report all unfree'd memory works pretty well).
>
> It turns out that this is *(most likely) because of memory
> fragmentation, so I tried malloc replacements 'hoard' and 'jemalloc',
> which had tried before and found gave minimal/no performance gains.
> But they do help with using less memory from the OS.
> (Keep in mind, using 64bit linux - ubuntu karmic, mileage would vary
> depending on the OS ofcourse).
>
> jemalloc gave better results so did some further tests.
> On loading the same file it used ~500mb rather then ~750mb. This was a
> scene with sintel so I suspect fragmentation could be because of
> vertex groups but not 100% sure.
>
> Then tested the alley scene, jemalloc <-> linux's default malloc,
> interesting results!
> (scene http://durian.blender.org/wp-content/uploads/2010/03/alley.png)
> (maximum memory used, from the OS's memory monitor. unscientifically
> took notes while rendering)
> new file (idle): 53mb <-> 55mb
> loaded alley scene (idle): 671mb <-> 1300mb
> render shadowbuffer: 5.5gig <-> 6.8gig
> raytree: 3.6gig <-> 4.0gig
> sss: 3.1gig <-> 4.1gig
> rendering (/w gi): 6.2gig <-> 7.2gig
> after rendering (idle): 4.0gig <-> 6.7gig
> new file after render (idle): 140mb <-> 3.1gig
>
> >From what I have read, linux malloc implementation is considered to be
> quite good, (not worth replacing), so I'm guessing blender is using
> more fine grained malloc's then it should be.
>
> Im not sure of the details, if libc is keeping the ram for the
> application, but when you want to have more then 1 blender loaded at a
> time, its annoying to have to restart it just to free the memory.
>
> So I'd be interested if someone could test memory usage on windows
> with jemalloc since we get a lot of complaints about memory on windows
> systems,
>
> see: http://www.canonware.com/jemalloc/
> note: needed to compile with "./configure --disable-tcache", it was
> crashing when rendering. (bug reported)
> note: rendertimes were not noticeably different.
>
> According to the site http://www.canonware.com/jemalloc/
> facebook server, firefox and FreeBSD use jemalloc.
>
> Could be worth including with blender, especially if it helps windows
> users from running out of ram so easily.
>
> --
> - Campbell
> _______________________________________________
> Bf-committers mailing list
> Bf-committers at blender.org
> http://lists.blender.org/mailman/listinfo/bf-committers
>