[Bf-blender-cvs] SVN commit: /data/svn/bf-blender [53220] branches/soc-2011-tomato/extern/ libmv: Tomato: syncronize with changes in libmv branch with assorted fixes

Sergey Sharybin sergey.vfx at gmail.com
Thu Dec 20 20:40:31 CET 2012


Revision: 53220
          http://projects.blender.org/scm/viewvc.php?view=rev&root=bf-blender&revision=53220
Author:   nazgul
Date:     2012-12-20 19:40:30 +0000 (Thu, 20 Dec 2012)
Log Message:
-----------
Tomato: syncronize with changes in libmv branch with assorted fixes

- Biggest error was in cost functors used for F and H refirement,
  they were just wrong.

- Use natural logarithms, since it's actually makes sense from
  math papers point of view and error is somewhere else.

- Disabled rho for GRIC, for now use non-clamped error for tests.

- Made SymmetricEpipolarDistance returning non-squared distance
  Keyframe selection is currently the only used of this function
  and it seems using non-squared distance makes much more sense.

  Also would think to append suffix "Squared" to functions which
  returns squared distances.

- Removed templated version of SymmetricEpipolarDistance, since
  it's not needed actually.

- Changed main keyframe selection cycle, so in cases there're no
  more next keyframes for current keyframe could be found in the
  image sequence, current keyframe would be moved forward and
  search continues.

  This helps in cases when there's poor motion in the beginning
  of the sequence, then markers becomes occluded. There could be
  good keyframes in the middle of the shot still.

- Moved correspondences constraint to the top, so no H/F estimation
  happens if there's bad correspondence. Makes algorithm go a bit
  faster.

Selection still behaves very much fuzzy and still working on this.

Modified Paths:
--------------
    branches/soc-2011-tomato/extern/libmv/ChangeLog
    branches/soc-2011-tomato/extern/libmv/libmv/multiview/fundamental.h
    branches/soc-2011-tomato/extern/libmv/libmv/simple_pipeline/keyframe_selection.cc

Modified: branches/soc-2011-tomato/extern/libmv/ChangeLog
===================================================================
--- branches/soc-2011-tomato/extern/libmv/ChangeLog	2012-12-20 19:26:57 UTC (rev 53219)
+++ branches/soc-2011-tomato/extern/libmv/ChangeLog	2012-12-20 19:40:30 UTC (rev 53220)
@@ -1,3 +1,55 @@
+commit 770eb0293b881c4c419c587a6cdb062c47ab6e44
+Author: Sergey Sharybin <sergey.vfx at gmail.com>
+Date:   Fri Dec 21 00:43:30 2012 +0600
+
+    Improvements for keyframe selection
+    
+    - Changed main keyframe selection cycle, so in cases there're no
+      more next keyframes for current keyframe could be found in the
+      image sequence, current keyframe would be moved forward and
+      search continues.
+    
+      This helps in cases when there's poor motion in the beginning
+      of the sequence, then markers becomes occluded. There could be
+      good keyframes in the middle of the shot still.
+    
+    - Extended keyframe_selection_test with real world cases.
+    
+    - Moved correspondences constraint to the top, so no H/F estimation
+      happens if there's bad correspondence. Makes algorithm go a bit
+      faster.
+    
+    Strangely, but using non-squared distances makes neighbor frames
+    test fail, using squared distances makes this tests pass.
+    
+    However, using non-squared distances seems to be working better
+    in real tests i've been doing. So this requires more investigation/
+
+commit 7415c62fbda36c5bd1c291bc94d535a66da896d0
+Author: Sergey Sharybin <sergey.vfx at gmail.com>
+Date:   Thu Dec 20 18:46:09 2012 +0600
+
+    Cosmetic change to correspondences reports in keyframe selection
+
+commit ceaf80c987ec0338e7e83965bc808411453eb755
+Author: Sergey Sharybin <sergey.vfx at gmail.com>
+Date:   Thu Dec 20 18:08:03 2012 +0600
+
+    Various fixes:
+    
+    - That was a typo in symmetric geometric cost functor, which
+      computed inverse distance in a wrong way.
+    
+    - Fixed compilation of unit tests
+    
+    - Added simple test for keyframe selection. Currently only
+      covers case that neighbor frames with only translation
+      (homography should be better than fundamental) are not
+      considered a keyframes.
+    
+      Still need to be investigated why it only works if tracks
+      are in pixel space and why doesn't work in normalized space.
+
 commit cfabdfe48df2add3d1f30cf4370efd0b31990ab0
 Author: Sergey Sharybin <sergey.vfx at gmail.com>
 Date:   Thu Dec 20 05:46:53 2012 +0600
@@ -702,23 +754,3 @@
 Date:   Fri Feb 17 20:54:13 2012 +0600
 
     More Windows -> Unix EOL conversions
-
-commit 18aeda58bec9556140ba617724e31ada6f5b67c0
-Author: Sergey Sharybin <sergey.vfx at gmail.com>
-Date:   Fri Feb 17 20:15:42 2012 +0600
-
-    Looks like this debug output was removed accidentally.
-
-commit 189dc0cacdee3c1eab68c43263ecb038ed244c09
-Author: Sergey Sharybin <sergey.vfx at gmail.com>
-Date:   Fri Feb 17 20:11:56 2012 +0600
-
-    Made V3D verbose again by default
-
-commit 8b3422d3eec5e450d76243886bf07fb0a3e83a81
-Author: Sergey Sharybin <sergey.vfx at gmail.com>
-Date:   Fri Feb 17 20:08:01 2012 +0600
-
-    SAD tracker now can deal with pattern size any size,
-    Very quick implementation came from Blender before Hybrid tracker was added.
-    Better to be replaced with brute tracker.

Modified: branches/soc-2011-tomato/extern/libmv/libmv/multiview/fundamental.h
===================================================================
--- branches/soc-2011-tomato/extern/libmv/libmv/multiview/fundamental.h	2012-12-20 19:26:57 UTC (rev 53219)
+++ branches/soc-2011-tomato/extern/libmv/libmv/multiview/fundamental.h	2012-12-20 19:40:30 UTC (rev 53220)
@@ -81,26 +81,6 @@
 double SymmetricEpipolarDistance(const Mat &F, const Vec2 &x1, const Vec2 &x2);
 
 /**
- * Calculates the sum of the distances from the points to the epipolar lines.
- *
- * Templated and not optimized version of function above
- */
-template<typename T> inline
-T SymmetricEpipolarDistance(const Eigen::Matrix<T, 3, 3> &F,
-                            const Eigen::Matrix<T, 2, 1> &x1,
-                            const Eigen::Matrix<T, 2, 1> &x2) {
-  Eigen::Matrix<T, 3, 1> x(x1(0), x1(1), T(1.0));
-  Eigen::Matrix<T, 3, 1> y(x2(0), x2(1), T(1.0));
-
-  Eigen::Matrix<T, 3, 1> F_x = F * x;
-  Eigen::Matrix<T, 3, 1> Ft_y = F.transpose() * y;
-  T y_F_x = y.dot(F_x);
-
-  return Square(y_F_x) * (  T(1) / F_x.head(2).squaredNorm()
-                          + T(1) / Ft_y.head(2).squaredNorm());
-}
-
-/**
  * Compute the relative camera motion between two cameras.
  *
  * Given the motion parameters of two cameras, computes the motion parameters

Modified: branches/soc-2011-tomato/extern/libmv/libmv/simple_pipeline/keyframe_selection.cc
===================================================================
--- branches/soc-2011-tomato/extern/libmv/libmv/simple_pipeline/keyframe_selection.cc	2012-12-20 19:26:57 UTC (rev 53219)
+++ branches/soc-2011-tomato/extern/libmv/libmv/simple_pipeline/keyframe_selection.cc	2012-12-20 19:40:30 UTC (rev 53220)
@@ -27,53 +27,54 @@
 namespace libmv {
 
 namespace {
-template<typename T>
-T SymmetricGeometricDistance(const Eigen::Matrix<T, 3, 3> &H,
-                                  const Eigen::Matrix<T, 2, 1> &x1,
-                                  const Eigen::Matrix<T, 2, 1> &x2) {
-  Eigen::Matrix<T, 3, 1> x(x1(0), x1(1), T(1.0));
-  Eigen::Matrix<T, 3, 1> y(x2(0), x2(1), T(1.0));
+double SymmetricGeometricDistance(Mat3 &H, Vec2 &x1, Vec2 &x2) {
+  Vec3 x(x1(0), x1(1), 1.0);
+  Vec3 y(x2(0), x2(1), 1.0);
 
-  Eigen::Matrix<T, 3, 1> H_x = H * x;
-  Eigen::Matrix<T, 3, 1> Hinv_y = H.inverse() * y;
+  Vec3 H_x = H * x;
+  Vec3 Hinv_y = H.inverse() * y;
 
   H_x /= H_x(2);
   Hinv_y /= Hinv_y(2);
 
-  return (H_x.head(2) - y.head(2)).squaredNorm() +
-         (Hinv_y.head(2) - x.head(2)).squaredNorm();
+  return (H_x.head<2>() - y.head<2>()).norm() +
+         (Hinv_y.head<2>() - x.head<2>()).norm();
 }
 
 class HomographySymmetricGeometricCostFunctor {
  public:
-  HomographySymmetricGeometricCostFunctor(Mat &x1, Mat &x2)
-      : x1_(x1),
-        x2_(x2) {
+  HomographySymmetricGeometricCostFunctor(Vec2 x, Vec2 y)
+      : x_(x),
+        y_(y) {
   }
 
   template<typename T>
   bool operator()(const T *homography_parameters, T *residuals) const {
     typedef Eigen::Matrix<T, 3, 3> Mat3;
-    typedef Eigen::Matrix<T, 2, 1> Vec2;
+    typedef Eigen::Matrix<T, 3, 1> Vec3;
 
     Mat3 H(homography_parameters);
 
-    T symmetric_error = T(0.0);
+    Vec3 x(T(x_(0)), T(x_(1)), T(1.0));
+    Vec3 y(T(y_(0)), T(y_(1)), T(1.0));
 
-    for (int i = 0; i < x1_.cols(); i++) {
-      Vec2 x1(T(x1_(0, i)), T(x1_(1, i)));
-      Vec2 x2(T(x2_(0, i)), T(x2_(1, i)));
+    Vec3 H_x = H * x;
+    Vec3 Hinv_y = H.inverse() * y;
 
-      symmetric_error += SymmetricGeometricDistance(H, x1, x2);
-    }
+    H_x /= H_x(2);
+    Hinv_y /= Hinv_y(2);
 
-    residuals[0] = symmetric_error;
+    residuals[0] = H_x(0) - T(y_(0));
+    residuals[1] = H_x(1) - T(y_(1));
 
+    residuals[2] = Hinv_y(0) - T(x_(0));
+    residuals[3] = Hinv_y(1) - T(x_(1));
+
     return true;
   }
 
-  const Mat &x1_;
-  const Mat &x2_;
+  const Vec2 x_;
+  const Vec2 y_;
 };
 
 void ComputeHomographyFromCorrespondences(Mat &x1, Mat &x2, Mat3 *H) {
@@ -83,16 +84,18 @@
   // Refine matrix using Ceres minimizer
   ceres::Problem problem;
 
-  HomographySymmetricGeometricCostFunctor *homography_symmetric_geometric_cost_function =
-       new HomographySymmetricGeometricCostFunctor(x1, x2);
+  for (int i = 0; i < x1.cols(); i++) {
+    HomographySymmetricGeometricCostFunctor *homography_symmetric_geometric_cost_function =
+        new HomographySymmetricGeometricCostFunctor(x1.col(i), x2.col(i));
 
-  problem.AddResidualBlock(
-      new ceres::AutoDiffCostFunction<
-          HomographySymmetricGeometricCostFunctor,
-          1, /* num_residuals */
-          9>(homography_symmetric_geometric_cost_function),
-      NULL,
-      H->data());
+    problem.AddResidualBlock(
+        new ceres::AutoDiffCostFunction<
+            HomographySymmetricGeometricCostFunctor,
+            4, /* num_residuals */
+            9>(homography_symmetric_geometric_cost_function),
+        NULL,
+        H->data());
+  }
 
   // Configure the solve.
   ceres::Solver::Options solver_options;
@@ -111,34 +114,33 @@
 
 class FundamentalSymmetricEpipolarCostFunctor {
  public:
-  FundamentalSymmetricEpipolarCostFunctor(Mat &x1, Mat &x2)
-      : x1_(x1),
-        x2_(x2) {
+  FundamentalSymmetricEpipolarCostFunctor(Vec2 x, Vec2 y)
+      : x_(x),
+        y_(y) {
   }
 
   template<typename T>
   bool operator()(const T *fundamental_parameters, T *residuals) const {
     typedef Eigen::Matrix<T, 3, 3> Mat3;
-    typedef Eigen::Matrix<T, 2, 1> Vec2;
+    typedef Eigen::Matrix<T, 3, 1> Vec3;
 
     Mat3 F(fundamental_parameters);
 
-    T error = T(0.0);
+    Vec3 x(T(x_(0)), T(x_(1)), T(1.0));
+    Vec3 y(T(y_(0)), T(y_(1)), T(1.0));
 
-    for (int i = 0; i < x1_.cols(); i++) {
-      Vec2 x1(T(x1_(0, i)), T(x1_(1, i)));
-      Vec2 x2(T(x2_(0, i)), T(x2_(1, i)));
+    Vec3 F_x = F * x;
+    Vec3 Ft_y = F.transpose() * y;
+    T y_F_x = y.dot(F_x);
 
-      error += SymmetricEpipolarDistance(F, x1, x2);
-    }
+    residuals[0] = y_F_x * T(1) / F_x.head(2).norm();
+    residuals[1] = y_F_x * T(1) / Ft_y.head(2).norm();
 
-    residuals[0] = error;
-
     return true;
   }
 
-  const Mat &x1_;
-  const Mat &x2_;
+  const Mat x_;
+  const Mat y_;
 };
 
 void ComputeFundamentalFromCorrespondences(Mat &x1, Mat &x2, Mat3 *F)
@@ -149,16 +151,18 @@
   // Refine matrix using Ceres minimizer
   ceres::Problem problem;
 
-  FundamentalSymmetricEpipolarCostFunctor *fundamental_symmetric_epipolar_cost_function =
-       new FundamentalSymmetricEpipolarCostFunctor(x1, x2);
+  for (int i = 0; i < x1.cols(); i++) {
+    FundamentalSymmetricEpipolarCostFunctor *fundamental_symmetric_epipolar_cost_function =
+        new FundamentalSymmetricEpipolarCostFunctor(x1.col(i), x2.col(i));
 
-  problem.AddResidualBlock(
-      new ceres::AutoDiffCostFunction<
-          FundamentalSymmetricEpipolarCostFunctor,
-          1, /* num_residuals */
-          9>(fundamental_symmetric_epipolar_cost_function),
-      NULL,
-      F->data());
+    problem.AddResidualBlock(
+        new ceres::AutoDiffCostFunction<
+            FundamentalSymmetricEpipolarCostFunctor,
+            2, /* num_residuals */
+            9>(fundamental_symmetric_epipolar_cost_function),
+        NULL,

@@ Diff output truncated at 10240 characters. @@



More information about the Bf-blender-cvs mailing list