[Soc-2011-dev] Weekly Progress Report #12 - libmv Tracker

[Matthias Fauconneau]

Hi,

Complete formatted version of the report (with screenshots):
 http://wiki.blender.org/index.php/User:Matthias.fauconneau/GSoC2011

=== Week 12: August 5 - August 12 ===

* Sum of Absolute Differences Tracker.
** This method is simpler, more robust, faster and accurate.
** Improve subpixel precision.
** SSE2 optimization.
** API for affine tracking (implementation on hold).
* Collaborate with Tomato.
** Synchronize trees.
** Adapt and Merge simple_pipeline patches.
** Re-merge deprecated KLT tracker.
** Discuss priorities and issues to resolve before end of GSoC.
** Improve CameraIntrinsics.
* Lens distortion estimation
** Research various existing methods.
** Select "Algebraic Lens Distortion Estimation".
** Implement a reference application.

It seems the KLT tracker doesn't work correctly.
For example,[http://www.youtube.com/watch?v=GSbccckZMn4 this video]
doesn't track at all.
I couldn't find any obvious issue in libmv KLT implementation so its
model might be too limited.

So I took the initiative to implement a much simpler (and easier to
debug) [https://github.com/Matthias-Fauconneau/libmv/blob/master/src/libmv/tracking/sad.cc
tracker].

Right now, this tracker simply computes the Sum of Absolute
Differences (SAD) for each integer pixel position in the search region
and then refine subpixel position using a simple square search. This
algorithm is used for motion estimation in video encoders.

In my tests, I've found this method more robust, faster and accurate.
I think we should test further and provide this alternative to Tomato users.


Tomato users are now trying to solve more complex scenes where
approximate calibration of camera intrinsic parameters prevent
accurate reconstruction even with precise tracks.
The new defined priority is to implement manual straight line
calibration, to allow estimating intrinsic footage from the tracked
footage instead of relying on external tools to perform full
calibration on checkerboard images.
I researched various methods to perform lens distortion estimation and
I had to learn the more general techniques which are applied on this
problem.
The easiest solution would be to use the "Algebraic Lens Distortion
Estimation"[1] method which has sample code available. This project
was under GPLv3 but the authors agreed to release it under GPLv2 for
Blender usage.

I'm developing an application[2] to test the method and provide a
reference which will allow the interface to be easily ported to
Blender.

[1][http://www.ipol.im/pub/algo/ags_algebraic_lens_distortion_estimation
Algebraic Lens Distortion Estimation]
[2][https://gitorious.org/qt-lens-distortion/ Reference Application]


=== Week 13: August 12 - August 19 ===

Over the week-end, the lens distortion estimation application will be completed.
If time allows it, a method to "snap" user input to image lines will
be implemented.
A way to measure the physical focal length from the footage might also
be necessary.

Since libmv currently doesn't support affine tracking, rotation and
scaling cannot be tracked. Thus patterns need to be adapted often to
avoid drift or even complete loss of the marker.
A planar tracker would be able track affine movements in the common
cases (visible planar surfaces) with efficient of computing resources,
while providing additional robustness against track loss.

Some time will be reserved to ensure the code is maintainable by
future contributors.
API documentation will be improved and more comments will be added.