[Bf-vfx] Improving the 2D stabilization tool

[Ichthyostega]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Am 04.06.2014 11:49, schrieb Thomas Beck:
> I really like your ideas that you're pushing forward. I read your 
> explanations on the github page and have to say that I found the current 
> stabilization sometimes frustrating too. As it was more or less a "plus" on
>  our projects I didn't care that much though.

> To continue with your voyage, consider to subscribe to the BF-VFX list...


Hello all,

so let's move a more focussed discussion to the VFX-list.

As mentioned in my initial post to BF-committers, I started looking into
the implementation of Blender's 2D stabilisation tool, prompted by problems
encountered when using this feature on my own footage. I found it surprisingly
difficult to get the desired stabilisation result -- in fact to get any decent
result at all, especially when the shot involves more elaborate camera and
object movement, like travelling, panning and perspective changes.

> 2014-06-02 7:27 GMT+02:00 Ichthyostega <prg at ichthyostega.de>:

> You can find my code as a branch on top of current master 
> https://github.com/Ichthyostega/blender.git

> Branch: stabilizer_rework

> (note: you might want to skip the "[private]..." commits at start of the
> branch; I am building on Debian-stable


> I've written some notes (daft version) to the github wiki:

> https://github.com/Ichthyostega/Blender/wiki

> In the same github repo, there is a branch "stabilizer_demo" It contains a
> blend file to check out my new features.



Meanwhile I've finished an initial round of rework, which I'd like to present
as starting point for discussion. With these changes, the original issues
are resolved for me, but there remains some integration work, where I'd
need some help.

- - keyframes and animation on track weights and parameters of 2D stabilisation
  can be defined and play back as expected in MovieClipEditor, but can't be
  handled in the usual way for automation (NLA, F-Curves), nor does the
  animation work at all for playback in the compositor, sequence editor
  or final render.

  My guess is that a "movie clip" is not a scene object in the usual sense


- - to combine autoscale (and further ideas for development) with such
  animated parameters, I'd need a way to "cue" the automation to other
  values than the current frame (and this in a subroutine which might
  be called e.g. for initialisation while in playback)


- - also, I am not so familiar with the compositor nodes and I still have
  to find out how the final rendering is actually performed; I need to
  do the same modifications to the transformation function I did
  successfully to the display in the MovieClipEditor


Beyond that, my approach of systematically summing up contributions to
various parameter measurements opens some perspectives for further work:

- - the problem with the pivot point, what options to offer?
- - maybe include a solving for the pivot point? And if so, then globally,
  per single frame or per range of frames?
- - in theory, the pivot point for zooming could be different to the
  one used to measure and compensate rotation. Not sure how noticeable
  the differences are we get when ignoring this distinction.

- - a dynamically calculated pivot point would imply that this pivot point
  needs to be part of the "stabilisation data". This would mean to
  extend the public API and esp. it would mean to extend the compositing
  node to deliver this pivot information alongside with position, scale
  and rotation.

- - the "autoscale" function is a nice idea, but in its current shape
  it is unusable in practice: Usually, it scales away almost everything.
- - What would be needed is a dynamic autoscale, which could also get us a
  starting point for the target position / target rotation / target zoom
  setting automatically.
- - to get that to work, my guess is that we'd need to build in some amount
  of damping. For example considering the current frame and the neighbouring
  frames, but use a simple window function (e.g. gaussian bell curve) to
  weight the neighbours.

- - are there further parameter which could be measured and then compensated?
  In my experiments -- since the basic stabilisation works very well now --
  typically I saw a residual shearing movement, which is due to 3D perspective
  and possibly due to lens distortion. In theory, any systematic distortion
  or movement that can be modelled can be measured and compenstated

- - BUT the glaring question is where to stop with adding features.
  A really good tool always requires some limitation and focus to
  the reasonable and typical patterns of usage.

Any discussion, review, critique, thoughts and proposals appreciated


Cheers,
Hermann Vosseler
(aka "Ichthyostega")





-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.12 (GNU/Linux)
Comment: Using GnuPG with Icedove - http://www.enigmail.net/

iEYEARECAAYFAlOPwM4ACgkQZbZrB6HelLI65QCgtJCR8tYsoulN6nKN4Q9RufgT
WBEAn0qW+HB6vNXHKqJjuR8Glfnar/oA
=LmW3
-----END PGP SIGNATURE-----