[Bf-vfx] tracker constraints and camera reconstruction...

[Francesco Callari]

On Sat, Aug 17, 2013 at 3:54 PM, David Jeske <davidj at gmail.com> wrote:

>
> This is interesting and goes far beyond what I was thinking with
> plane-constraints (though not necessarily in a complex way).
>
> If there were a mechanism for specifying persistent constraints between
> trackers and 3d-models WRT camera-solving, it might also be practical to
> use this to handle origin/floor/scale as well. (current origin/floor/scale
> is frustrating, as they are not persistent constraints but actions which
> need to be repeated after every camera solve)
>

It is conceptually quite easy, and in fact the easiest implementation is
the most general one: a scene graph solver. But  I don't know much about
the implementation of the scene graph in Blender, and if it's not designed
so to compute a function between any two nodes (and between any attributes
of two nodes) it may be hard to do it in practice.

Start with the simplest case: the point constraint. You have one (possibly
animated) 3D point P(t) which is the solution of a track. Call it a
"bundle", because It is theoretically the intersection of a bundle of rays
going through it an the camera centers at each frame in the track. Let it
be an object in the scene graph, whose attributes are its coordinates
X(t),Y(t), Z(t) at each frame time t, defined with respect to some parent
transform node.

Now, the important thing is that the bundle's coordinates are used to
compute the re-projection error w.r.t. the tracks, which drives the solver,
but are not necessarily the unknowns solved for by the solver itself. In
fact, the unknowns we want to solve for could be something completely
different elsewhere in the scene graph. To get a solution it is necessary
that a "solvable" relationship exists between the two. In the simple
example of my previous message, the unknowns would be the 6 scalar
attributes defining the pose and location of the box (6 functions of time).

To define the above relationship you need a way to "attach" the bundle,
which provides the error signal driving the solver, to the unknowns you
want to solve for. Consider then another scene graph object representing a
3D point, call it a "point constraint". It has 3 coordinates at each frame
as well, but unlike the bundle's they are *assigned *by the scene graph,
i.e they are knowns. For example, if the thing you are solving for is the
pose/position of a car, whose shape you have modeled with a mesh, and the
track is for the spoke of a wheel, you attach the point constraint to a
mesh vertex on the spoke. Now the point constraint's coordinates with
respect to the unknowns are well defined. Next, you "attach" the point
constraint to the bundle, effectively establishing a relationship in the
scene graph that says "this bundle shall have the position of this point
constraint". That's it, you are done. Assuming the problem is balanced and
solvable (i.e. that you have enough good tracks), the solver will try to
find the values of the unknowns that cause the point constraint to drive
the bundle to places where the error is small, effectively matching the
tracks.

This definition of point constraint neatly separates the modeling of the
geometry (which is done with ordinary meshes) from the solver. You don't
need to complicate the mesh software with "special" vertices or facets that
know about markers and tracks. You only need a 3D point thing (the point
constraint) that's under the same parent transform node as the mesh. And
this thing, likewise, needs not know about cameras and reprojection errors,
it only needs to know how to drive (i.e. "force") the position of another
3D point thing, the bundle, which knows about cameras seeing it and markers
reprojecting it (so that it can compute an error term).

Yes, I have left out plenty of details (in particular, how to compute
derivatives of the error term in the scene graph), but hope the idea is
somewhat clear.

Ciao
Franco




>
> _______________________________________________
> Bf-vfx mailing list
> Bf-vfx at blender.org
> http://lists.blender.org/mailman/listinfo/bf-vfx
>
>


-- 
Franco Callari <fgcallari at gmail.com>

            EC67 BEBE 62AC 8415 7591  2B12 A6CD D5EE D8CB D0ED

I am not bound to win, but I am bound to be true. I am not bound to
succeed, but I am bound to live by the light that I have. (Abraham Lincoln)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.blender.org/pipermail/bf-vfx/attachments/20130817/80bbca3d/attachment.htm