[Bf-vfx] Improving the 2D stabilization tool

Thu Jun 5 08:10:15 CEST 2014

Hi,

Nice to hear someone finally goes solving stupidness of the current 2D
stabilizer.

Didn't check the code but ideas seems indeed much better than the current
ones. Would be cool to see them in action. Do you have some videos which
you can show to Sebastian and Sean? :)

Also, do you mind putting the patch to the review at developer.blender.org?
Easiest way to do it is to use arcanist [1]. This way we can start check on
the code and probably point into some better solutions there. Doing early
code review always helps.

It'll also make it so more users can apply the patch and play around with
it.

Also, just for the ideas, you might also consider using information from 3D
solver and do some sort of frame re-projection into a camera after applying
some momentum to it (applying some sort of low-pass filter on the camera
trajectory).

Now, about some questions you're raising:

- Not sure why animated weight is not applied in compositor. It should work
just fine, seems i'll need to have a closer look into the code. And yes,
it's not editable in animation editors. This is something to implement
still.

- Compensating distortion -- that's a separate topic which is currently
being handled by either runtime frame undistortion or building undistorted
proxy. I don't think dealing with this should be a part of 2D stabilization.

- As for other lens effects which you might want to compensate is the
rolling shutter. But that's completely separate topic as well. We don't
deal with this in the solver pipeline and dealing with this is rather a
tricky business. You need to be very careful choosing what you're gonna to
do with this in order to preserve solvability of the footage.

And i agree, better to stop adding features, wrap up current improvements
and continue doing improvements when current ones are in the master branch.
For this it'll help having the patch in the code review tool as was
mentioned above. Assign me (i'm "sergey" in the developer.b.o) as a
reviewer there and i'll have a closer look.

[1] http://wiki.blender.org/index.php/Dev:Doc/Tools/Code_Review#Use_Arcanist

On Thu, Jun 5, 2014 at 7:55 AM, Sean Kennedy <mack_dadd2 at hotmail.com> wrote:

> I just read over everything pretty quickly, and it sounds very promising!
> I too have tried to use stabilization in a production environment and found
> it slightly lacking. A long time ago (maybe a year?) I was pestering a
> certain developer (one I pester a lot!) about why there isn't a stabilize
> scale option. Seems to make sense there should be one since there's already
> position and rotation. But anyway... what I really wanted to comment on was
> the idea of a pivot point. This would be SO useful, having access to a
> definable pivot point in the compositor. It's almost necessary when doing
> common compositing things like removals and replacements. I don't know
> about the idea of a "dynamically calculated" pivot point, if that means
> it's going to be moving around a lot and animating itself. That could
> present some issues in a compositing environment where a pivot point is
> useful.
>
> Great ideas, though! Glad to see stabilization getting some love!
>
> Sean
>
> > Date: Thu, 5 Jun 2014 02:58:54 +0200
> > From: prg at ichthyostega.de
> > To: bf-vfx at blender.org
> > Subject: [Bf-vfx] Improving the 2D stabilization tool
>
> >
> > -----BEGIN PGP SIGNED MESSAGE-----
> > Hash: SHA1
> >
> > Am 04.06.2014 11:49, schrieb Thomas Beck:
> > > I really like your ideas that you're pushing forward. I read your
> > > explanations on the github page and have to say that I found the
> current
> > > stabilization sometimes frustrating too. As it was more or less a
> "plus" on
> > > our projects I didn't care that much though.
> >
> > > To continue with your voyage, consider to subscribe to the BF-VFX
> list...
> >
> >
> > Hello all,
> >
> > so let's move a more focussed discussion to the VFX-list.
> >
> > As mentioned in my initial post to BF-committers, I started looking into
> > the implementation of Blender's 2D stabilisation tool, prompted by
> problems
> > encountered when using this feature on my own footage. I found it
> surprisingly
> > difficult to get the desired stabilisation result -- in fact to get any
> decent
> > result at all, especially when the shot involves more elaborate camera
> and
> > object movement, like travelling, panning and perspective changes.
> >
> > > 2014-06-02 7:27 GMT+02:00 Ichthyostega <prg at ichthyostega.de>:
> >
> > > You can find my code as a branch on top of current master
> > > https://github.com/Ichthyostega/blender.git
> >
> > > Branch: stabilizer_rework
> >
> > > (note: you might want to skip the "[private]..." commits at start of
> the
> > > branch; I am building on Debian-stable
> >
> >
> > > I've written some notes (daft version) to the github wiki:
> >
> > > https://github.com/Ichthyostega/Blender/wiki
> >
> > > In the same github repo, there is a branch "stabilizer_demo" It
> contains a
> > > blend file to check out my new features.
> >
> >
> >
> > Meanwhile I've finished an initial round of rework, which I'd like to
> present
> > as starting point for discussion. With these changes, the original issues
> > are resolved for me, but there remains some integration work, where I'd
> > need some help.
> >
> > - - keyframes and animation on track weights and parameters of 2D
> stabilisation
> > can be defined and play back as expected in MovieClipEditor, but can't be
> > handled in the usual way for automation (NLA, F-Curves), nor does the
> > animation work at all for playback in the compositor, sequence editor
> > or final render.
> >
> > My guess is that a "movie clip" is not a scene object in the usual sense
> >
> >
> > - - to combine autoscale (and further ideas for development) with such
> > animated parameters, I'd need a way to "cue" the automation to other
> > values than the current frame (and this in a subroutine which might
> > be called e.g. for initialisation while in playback)
> >
> >
> > - - also, I am not so familiar with the compositor nodes and I still have
> > to find out how the final rendering is actually performed; I need to
> > do the same modifications to the transformation function I did
> > successfully to the display in the MovieClipEditor
> >
> >
> > Beyond that, my approach of systematically summing up contributions to
> > various parameter measurements opens some perspectives for further work:
> >
> > - - the problem with the pivot point, what options to offer?
> > - - maybe include a solving for the pivot point? And if so, then
> globally,
> > per single frame or per range of frames?
> > - - in theory, the pivot point for zooming could be different to the
> > one used to measure and compensate rotation. Not sure how noticeable
> > the differences are we get when ignoring this distinction.
> >
> > - - a dynamically calculated pivot point would imply that this pivot
> point
> > needs to be part of the "stabilisation data". This would mean to
> > extend the public API and esp. it would mean to extend the compositing
> > node to deliver this pivot information alongside with position, scale
> > and rotation.
> >
> > - - the "autoscale" function is a nice idea, but in its current shape
> > it is unusable in practice: Usually, it scales away almost everything.
> > - - What would be needed is a dynamic autoscale, which could also get us
> a
> > starting point for the target position / target rotation / target zoom
> > setting automatically.
> > - - to get that to work, my guess is that we'd need to build in some
> amount
> > of damping. For example considering the current frame and the
> neighbouring
> > frames, but use a simple window function (e.g. gaussian bell curve) to
> > weight the neighbours.
> >
> > - - are there further parameter which could be measured and then
> compensated?
> > In my experiments -- since the basic stabilisation works very well now --
> > typically I saw a residual shearing movement, which is due to 3D
> perspective
> > and possibly due to lens distortion. In theory, any systematic distortion
> > or movement that can be modelled can be measured and compenstated
> >
> > - - BUT the glaring question is where to stop with adding features.
> > A really good tool always requires some limitation and focus to
> > the reasonable and typical patterns of usage.
> >
> > Any discussion, review, critique, thoughts and proposals appreciated
> >
> >
> > Cheers,
> > Hermann Vosseler
> > (aka "Ichthyostega")
> >
> >
> >
> >
> >
> > -----BEGIN PGP SIGNATURE-----
> > Version: GnuPG v1.4.12 (GNU/Linux)
> > Comment: Using GnuPG with Icedove - http://www.enigmail.net/
> >
> > iEYEARECAAYFAlOPwM4ACgkQZbZrB6HelLI65QCgtJCR8tYsoulN6nKN4Q9RufgT
> > WBEAn0qW+HB6vNXHKqJjuR8Glfnar/oA
> > =LmW3
> > -----END PGP SIGNATURE-----
> > _______________________________________________
> > Bf-vfx mailing list
> > Bf-vfx at blender.org
> > http://lists.blender.org/mailman/listinfo/bf-vfx
>
> _______________________________________________
> Bf-vfx mailing list
> Bf-vfx at blender.org
> http://lists.blender.org/mailman/listinfo/bf-vfx
>
>

-- 
With best regards, Sergey Sharybin
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.blender.org/pipermail/bf-vfx/attachments/20140605/4c16b32d/attachment.htm 