[Bf-committers] PATCH to limit memory usage in sequencer
ton at blender.org
Tue Oct 25 20:40:48 CEST 2005
Thanks a lot for the very cool & cogent reply. :)
At this moment I am doing other things... will come back to it! And for
everyone, give the patch a good test!
On 24 Oct, 2005, at 20:17, Peter Schlaile wrote:
> Hi Ton,
>> the API for it has no comments included either.
> OK. I'll add this later. For the moment:
> There are basically 4 operations you can do with a managed object:
> * insert: Insert the object into the management system. I don't like
> idea of managing _every_ object in a cache limiter. The system
> force you, to manage an object. To express your wish, you need to say
> That's what we do in
> The implementation in IMB_cache_limiter_insert(...) is robust enough
> notice, if we are already managing the given object and then does
> nothing at all.
> So we can call it here, to be on the safe side. (The code in the
> sequencer has some side effects by copying pointers around, so this
> the safe way to go, if you don't want to get totally confused...)
> IMB_cache_limiter_insert(...) enforces the memory cache limit at the
> same time. In contrast, the generic cache limiter interface makes
> this a
> seperate step.
> * ref: If you want to lock a managed object in memory, you have to
> increment the reference counter using "ref". I didn't want to call
> this "lock", since I wanted to make it clear, that you can ref
> and unref several times and we can free this object again, if
> refcount == 0.
> We are working on this IMBuf and we can not be sure, if some effect
> filter some layers up wants to use our result. So here we go:
> the refcounter using
> * touch: Every clever cache management system has some notion of "page
> aging". You don't want to throw out objects which are used
> Since we obviously used this object in this moment (remember, the
> could easily have been a no-op), we just "touch" it to make sure it
> at the tail of the deletion list.
> * unref: decrement the ref counter. This is done at the end of the
> processing in a seperate function.
> Hope that explains it somewhat better than my short example in the
>> I'm also not very fond of invisible STL memory management, in Blender
>> we use our secure MEM_mallocN almost everywhere, which as saved us in
>> lot of situations. Could you use that too?
> Should be rather easy using a seperate STL Allocator class, right? I
> add this to memutil, since someone will need it sooner or later.
>> Then, knowing sequencer quite well, the current implementation was
>> never aimed at allowing longer videos to edit...
> Uuhm. The biggest show stopper right now is the global frame count
> of 32000 frames, if you ask me. The evening I wrote the patch, I just
> cinelerra before and must say, that blender's sequencer is the closest
> thing to a video editing system that I always wanted to have.
>> it has quite some
>> design flaws which we could tackle too, for example by freeing up
>> memory of strips get layered (only the end result needs storage).
>> This is worth testing too.
> Why? Let the cache manager do the trick. If I understand the sequencer
> code correctly, end results are rendered once and cached until
> change. Since we do caching and cache limiting, we are done.
>> I would also like to know if such a cache system is what other video
>> edit programs use too?
> I don't really know. I liked the concept and I definitely disliked most
> other video editing systems, since they all pretend to be a multitrack
> recorder with all it's limitations.
>> The current speed bottleneck is also that
>> calculation of effects just takes cpu time, making it not realtime.
> Well. You are at the editing process of some small sequence of your
> You want to test it. Easy: Just view the sequence twice. The first
> time to
> render and cache it, the second time to judge the result.
> I don't want to let the system waste too much memory on importing,
> and doing other crazy things like "Adobe Premiere" or "Avid". It is the
> very impressive way, the blender sequencer handles input files, that
> it a sensible choice for video editing on _large_ movie projects.
> The only thing at this point that could be usefull, is rendering a
> seperate low res "preview" track.
> Additionally, it could be worth, building a dependency tree. Think of
> the following: You have several tracks camera 1-4 and want to just
> crossfade between them. The crossfade filters should just need to
> one camera and not all of them for every given frame. This could be
> easy, if we start at the top strip and work our way down, letting the
> filters decide, which input they actually need.
> And while we are at it: we need some way in the input file selector to
> choose an in- and out-point. It is rather frustrating to work on 2 hour
> timelines all the time...
> Next thing missing: we need a lot of usefull video filters. I'll try to
> look into a "virtual dub"-plugin-converter or interface-thingy.
> I'll submit patches for this in the next few weeks, since I definitely
> need these for my own project.
> I already wrote a "ffmpeg"-import filter, but I don't know if it
> with the filter some other guy wrote. I think, I'll add it to the patch
> tracker and hope the better patch of the two gets included... ;-)
> And right: the guy at the plugin repository silently dumped my generic
> "background-keyer" (luma- and chromakeyer on arbitrary backgrounds).
> anyone know, where I can submit it?
> Peter Schlaile
> Bf-committers mailing list
> Bf-committers at projects.blender.org
Ton Roosendaal Blender Foundation ton at blender.org
More information about the Bf-committers