[Bf-vfx] tracker constraints and camera reconstruction...

[Francesco Callari]

On Sat, Aug 17, 2013 at 10:38 PM, David Jeske <davidj at gmail.com> wrote:

> On Sat, Aug 17, 2013 at 3:39 PM, Francesco Callari <fgcallari at gmail.com>wrote:
>
>> It is conceptually quite easy, and in fact the easiest implementation is
>> the most general one: a scene graph solver.
>>
>
> Thanks for the detailed writeup. Reading that, I think expressing the
> solver constraints which are most useful may be easier to work with using a
> hybrid approach. For example, it seems much easier (from a user-UI
> perspective) to bundle a bunch of trackers and say they are on a plane, or
> express a constraint about which is closer to the camera -- without
> involving the 3d scene graph.
>

As for the UI, the "attachment" of point constraints to user-modelled
geometry can be automated (e.g. by nearest neighbor).

Not involving the scene graph implies adding special-case code to the
solver, and along that way lies madness :-)

Continuing on the plane example, if the plane (the unknown) you want to fit
is not an element of the scene graph, then code for fitting it must be
added as a special case to the solver, and the solved plane thrown away
after the solve. A lot of code must be replicated in the solver to do
things (like chaining of coordinate transforms) that the scene graphs knows
best about - including caching. The user cannot manipulate the plane prior
to the solve to give a "visually good" initial estimate, therefore the
success rate will be less than ideal, and the special-case code-path in the
solver must have all sort of sanity checks that likely will not apply
anywhere else. Worse, the user cannot "see" it in the scene graph after the
solution to judge the quality of the solution itself, nor can he tell where
adding further tracks would be most useful to refine the solution.

Special-case code by definition does not scale. If all you want is a solver
for planes, sure, a special case approach may get you there faster. But the
problem is that you always need more. When the least-squares solver is
embedded in the scene graph, lots of things become possible besides just
solving for the matchmove (camera motion). Things like

   - Adding constraints in a way that's "natural" for the matchmover and
   matches their goal for the shot. For example, to do a set extension, he may
   have to model buildings in the scene 20 story higher than those filmed,
   then solve for the camera motion that matches them to the real ones. The
   set extensions are the goal, the tracks are just something he needs to use
   to fit them. By concentrating on the goal the matchmover can produce the
   best possible solution.
   - I once worked on an exotic lens distortion problem. Director decided
   to shoot scene with an anamorphic lens about which nothing else was known
   (camera notes lost or never written up). I modeled the lens distortion as a
   thin-plate spline deformation on a grid, then fit the grid edges to tracks
   on the side of buildings (thankfully the shoot was a slow pan covering
   skyskrapers).
   - Register a point cloud on an actor's face produced by a hi-res (laser
   or stereo) scanner to another captured on set by a lower-res stereo camera
   pair, then use it to match-animate a mesh fitted to both by solving for the
   mixing parameters of animation shapes - that's at the core of the
   "animation transfer" for Avatar's characters.

In all these cases, the priors about the scene geometry were expressed
explicitly in the scene graph (often via point constraints), and thus
available to the solver.




> _______________________________________________
> Bf-vfx mailing list
> Bf-vfx at blender.org
> http://lists.blender.org/mailman/listinfo/bf-vfx
>
>


-- 
Franco Callari <fgcallari at gmail.com>

            EC67 BEBE 62AC 8415 7591  2B12 A6CD D5EE D8CB D0ED

I am not bound to win, but I am bound to be true. I am not bound to
succeed, but I am bound to live by the light that I have. (Abraham Lincoln)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.blender.org/pipermail/bf-vfx/attachments/20130818/32e230ea/attachment.htm