[Bf-committers] New VSE proxies, feedback appreciated
Peter Schlaile
peter at schlaile.de
Sun May 29 15:01:26 CEST 2011
Hi,
I'm currently coding a new Video Sequencer Proxy and indexing system on
gitorious:
https://gitorious.org/~schlaile/blenderprojects/blender-vse-schlaile
vse-proxies branch
Usage:
git clone git://gitorious.org/~schlaile/blenderprojects/blender-vse-schlaile.git
cd blender-vse-schlaile/blender
git checkout vse-proxies
scons / cmake as usual.
Before I do a merge into trunk, I'd like to have some feedback, if the
code works like expected.
The problems:
* VSE proxies are still missing from Blender 2.5
* FFMPEG seeking heuristics works currently "most of the time" at best,
sometimes it breaks down completely (depending on the mood of ffmpeg
internals)
Those are not "a let's hack on the interface and finally we figure out
a way to do it" kind of problems. There are fundamental ones:
The heuristics are DTS based (in case you don't know,
non-intra-formats code a "decoding timestamp" and a "presentation
timestamp" (PTS) into every data packet. Decoding timestamps are
(most of the time) monotonous, so a very good candidate for binary
searches, but: they have very little in common with the frame, we are
actually looking for, since that is determined by the PTS.
Fundamental flaws:
* there are (commonly used!) formats, like HDV, where the DTS
doesn't have to be monotonous. So: seeking can (and will!) break down
completely.
* the DTS is only a rough idea where the frame, we are actually
looking for is probably located. So: we are working with rough
preseeks in the hope, the right PTS will appear (which probably
will never happen)
* track duration is a wild guess, since ffmpeg uses the average frame
and bitrate of the beginning of the file, to deduce the right
duration. That can (and will) fail completely on certain files for
obvious reasons.
* it's rather tricky to determine the right offset between DTS and
PTS, since start offsets are specified for DTS and the PTS has
an unspecified time offset (which could be detected by decoding
the first view frames): so, even if we got the right DTS and even
if we are at the right place within the file, we are probably looking
for the wrong PTS from the start...
* sometimes, the DTS/PTS pair written into the stream, isn't the
timecode we were looking for in the first place. Camcorders
can write seperate timecode tracks (and do so, even the most
cheep ones). Sometimes, you can only deduce from the timecode
track, that some dropouts happened. And: if you work with
muliple cams, and your cam operators tend to start/stop a lot,
you will go crazy syncing the timeline without timecode support.
The idea:
* let's optionally build seperate timecode indices, which specify the
exact position (absolute file position, DTS to seek to, PTS to search
for) for every possible frame in the file.
* let's build that index together with the proxies, since for them, we
have to scan and decode the file anyways. And: a proxy only makes sense
within the context of an index. Another good reason, why those should
be build in the same pass. (Besides the fact, that we can do it a heck
lot faster, when only using ffmpeg functions, since color conversion
steps can be optimized away.)
The implementation:
* currently, proxy building (in background) and "Record run" indices are
implemented, other time code indices following later.
To use them: build the branch (see above), open a movie file, go to
N-keys panel "Proxies", check "Use Proxy", by default 25% proxies are
the only one build, but you can change that (or build no proxy at all
and use timecode indices only).
Do that for as much movie clips as you like.
Select all of them.
Go to "Strip -> Rebuild proxy"
On the status bar, there should be a progress bar (which currently shows
per file progress, total progress is TODO).
Blender is building proxies in the background, so you can continue
working.
After the build process finishes, you can select different time codes
(currently only "record run" is implemented)
and use the Proxy preview (N-keys dialog in preview window) again.
* seeking works perfectly for the first time in blender (no stuttering,
failing completely on broken files, etc.) since with indices, the
system now works completely PTS based. And: the preseek heuristics isn't
needed anymore, so blender will always seek to the right I-Frame, which
means: faster feedback on CPU-heavy footage (like Sony EX3) e.g.!
* currently missing:
* other timecode tracks than "record run".
In case you don't know: "record run" means: use the frames as they
are recorded, ignore gaps, if someone hits the stop button on the
camcorder.
"free run" means: use a global time counter (configurable on the
camcorder), if someone hits stop, consider those frames, and display
them as black gaps, frozen pictures.
"free run (interpolated)" means: if your camcorder is too cheap,
use record time/date to simulate a real "free run"-mode, usually only
found in prosumer camcorders.
* proper variable frame rate support (could be considered as some sort
of "free run", since the idea has a lot of similarities.
* audio code doesn't use the same seeking code path, leading to A/V sync
problems.
If things don't work out for you, using blender with "blender -d" will
print a lot of debugging output within the seeking code.
Please send me an email with the debugging output pasteall-ed, your
ffmpeg version, if you want to report problems.
Otherwise: enjoy!
And: I'm eager to hear your feedback, also code / design suggestions.
Cheers,
Peter
----
Peter Schlaile
More information about the Bf-committers
mailing list