[Bf-committers] PATCH to limit memory usage in sequencer

[Peter Schlaile]

Hi Ton,

> the API for it has no comments included either.

OK. I'll add this later. For the moment:

There are basically 4 operations you can do with a managed object:

* insert: Insert the object into the management system. I don't like the
  idea of managing _every_ object in a cache limiter. The system doesn't
  force you, to manage an object. To express your wish, you need to say
  it.

  That's what we do in

  IMB_cache_limiter_insert(...)

  The implementation in IMB_cache_limiter_insert(...) is robust enough to
  notice, if we are already managing the given object and then does
  nothing at all.

  So we can call it here, to be on the safe side. (The code in the
  sequencer has some side effects by copying pointers around, so this is
  the safe way to go, if you don't want to get totally confused...)

  IMB_cache_limiter_insert(...) enforces the memory cache limit at the
  same time. In contrast, the generic cache limiter interface makes this a
  seperate step.

* ref: If you want to lock a managed object in memory, you have to
  increment the reference counter using "ref". I didn't want to call
  this "lock", since I wanted to make it clear, that you can ref
  and unref several times and we can free this object again, if
  refcount == 0.

  We are working on this IMBuf and we can not be sure, if some effect
  filter some layers up wants to use our result. So here we go: increment
  the refcounter using

  IMB_cache_limiter_ref(...).

* touch: Every clever cache management system has some notion of "page
  aging". You don't want to throw out objects which are used frequently.
  Since we obviously used this object in this moment (remember, the insert
  could easily have been a no-op), we just "touch" it to make sure it is
  at the tail of the deletion list.

* unref: decrement the ref counter. This is done at the end of the layer
  processing in a seperate function.

Hope that explains it somewhat better than my short example in the header
file.

> I'm also not very fond of invisible STL memory management, in Blender
> we use our secure MEM_mallocN almost everywhere, which as saved us in a
> lot of situations. Could you use that too?

Should be rather easy using a seperate STL Allocator class, right? I can
add this to memutil, since someone will need it sooner or later.

> Then, knowing sequencer quite well, the current implementation was
> never aimed at allowing longer videos to edit...

Uuhm. The biggest show stopper right now is the global frame count limit
of 32000 frames, if you ask me. The evening I wrote the patch, I just tested
cinelerra before and must say, that blender's sequencer is the closest
thing to a video editing system that I always wanted to have.

> it has quite some
> design flaws which we could tackle too, for example by freeing up
> memory of strips get layered (only the end result needs storage).
> This is worth testing too.

Why? Let the cache manager do the trick. If I understand the sequencer
code correctly, end results are rendered once and cached until parameters
change. Since we do caching and cache limiting, we are done.

> I would also like to know if such a cache system is what other video
> edit programs use too?

I don't really know. I liked the concept and I definitely disliked most
other video editing systems, since they all pretend to be a multitrack
recorder with all it's limitations.

> The current speed bottleneck is also that
> calculation of effects just takes cpu time, making it not realtime.

Well. You are at the editing process of some small sequence of your video.
You want to test it. Easy: Just view the sequence twice. The first time to
render and cache it, the second time to judge the result.
I don't want to let the system waste too much memory on importing, copying
and doing other crazy things like "Adobe Premiere" or "Avid". It is the
very impressive way, the blender sequencer handles input files, that makes
it a sensible choice for video editing on _large_ movie projects.

The only thing at this point that could be usefull, is rendering a
seperate low res "preview" track.

Additionally, it could be worth, building a dependency tree. Think of
the following: You have several tracks camera 1-4 and want to just
crossfade between them. The crossfade filters should just need to render
one camera and not all of them for every given frame. This could be rather
easy, if we start at the top strip and work our way down, letting the
filters decide, which input they actually need.

And while we are at it: we need some way in the input file selector to
choose an in- and out-point. It is rather frustrating to work on 2 hour
timelines all the time...

Next thing missing: we need a lot of usefull video filters. I'll try to
look into a "virtual dub"-plugin-converter or interface-thingy.

I'll submit patches for this in the next few weeks, since I definitely
need these for my own project.

I already wrote a "ffmpeg"-import filter, but I don't know if it collides
with the filter some other guy wrote. I think, I'll add it to the patch
tracker and hope the better patch of the two gets included... ;-)

And right: the guy at the plugin repository silently dumped my generic
"background-keyer" (luma- and chromakeyer on arbitrary backgrounds). Does
anyone know, where I can submit it?

Greetings,
Peter

----
Peter Schlaile