[Soc-2011-dev] Weekly Progress Report #13 - libmv Tracker

Matthias Fauconneau matthias.fauconneau at gmail.com
Fri Aug 19 22:17:36 CEST 2011


We have affine tracking !

Complete formatted version of the report (with screenshots):

=== Week 13: August 12 - August 20 ===

* Lens distortion estimation.
** Complete reference implementation.
** Create and document a simple C interface.
* "Moravec"(SAD) detector.
** Detect features which are unlikely to drift.
** Detect features similar to a given pattern.
* Affine Tracking!
** Support in Tracker API.
** Support in Qt Tracker UI.
** Implemented for SAD Tracker using coordinate descent.
** Optimize integer pixel search.
** Tweak coordinate descent.
** Added regularization (area and condition number).
* Add Laplace filter to be used to avoid failure from smooth lighting changes.
* Improve UI.
* Document API.

The SAD detector is more suited to tracking since it try to choose
patterns which are unlikely to drift by computing SAD with neighboring
It could be improved to better avoid edges.

The SAD Tracker now supports full affine transforms (rotation,scale,skew).
This was necessary to make tracking usable without adapting at each frame.
Since there wasn't much time left for tracker improvements (the KLT
tracker was good enough until recently), I implemented a simple
brute-force solution:
To optimize all 6 affine parameters, I use the coordinate descent method.
While this made the tracking 10 times slower, there is much potential
for optimization if ever necessary. In any case, the performance is
still comparable to the slow pan only KLT tracker (10ms/marker).

=== Later ===

Blender will need to integrate straight line lens distortion
estimation, image distortion/undistortion, make affine tracking
interface easy to use and expose the laplace filter.

Libmv will need to be refactored to remove all unused and unsupported parts.
It should focus on its initial goal: structure from motion and
multiview reconstruction.
This simplification of libmv design will ease the maintenance, reduce
compile time,
decrease binary size, increase performance, and help attract contributors.

More information about the Soc-2011-dev mailing list