[Bf-blender-cvs] SVN commit: /data/svn/bf-blender [53631] branches/soc-2011-tomato/extern/ libmv: Fixed for keyframe selection
Sergey Sharybin
sergey.vfx at gmail.com
Mon Jan 7 13:20:57 CET 2013
Revision: 53631
http://projects.blender.org/scm/viewvc.php?view=rev&root=bf-blender&revision=53631
Author: nazgul
Date: 2013-01-07 12:20:51 +0000 (Mon, 07 Jan 2013)
Log Message:
-----------
Fixed for keyframe selection
- Calculate residuals for GRIC in pixel space rather than
in normalized space.
This seems to be how it's intended to be used.
Algebraic H and F will still use normalized coordinates which
are more stable, after this matrices are converted to pixel
space and Ceres refinement happens in pixel space.
- Standard deviation calculation was wrong in GRIC. It shouldn't
be deviation of residuals, but as per Torr it should be deviation
of measurement error, which is constant (in our case 0.1)
Not sure if using squared cost function is correct for GRIC,
but cost function is indeed squared and in most papers cost
function is used for GRIC. After some further tests we could
switch GRIC residuals to non-squared distance.
- Bring back rho part of GRIC, in unit tests it doesn't make
sense whether it's enabled or not, lets see how it'll behave
in real-life footage.
- Reduce minimal correspondence to match real-world manually
tracked footage
- Returned back squares to SymmetricEpipolarDistance and
SymmetricGeometricDistance -- this is actually a cost
functions, not distances and they shall be squared.
Modified Paths:
--------------
branches/soc-2011-tomato/extern/libmv/ChangeLog
branches/soc-2011-tomato/extern/libmv/libmv/multiview/fundamental.cc
branches/soc-2011-tomato/extern/libmv/libmv/multiview/homography.cc
branches/soc-2011-tomato/extern/libmv/libmv/multiview/homography.h
branches/soc-2011-tomato/extern/libmv/libmv/simple_pipeline/keyframe_selection.cc
branches/soc-2011-tomato/extern/libmv/libmv/simple_pipeline/keyframe_selection.h
branches/soc-2011-tomato/extern/libmv/libmv-capi.cpp
Modified: branches/soc-2011-tomato/extern/libmv/ChangeLog
===================================================================
--- branches/soc-2011-tomato/extern/libmv/ChangeLog 2013-01-07 12:19:00 UTC (rev 53630)
+++ branches/soc-2011-tomato/extern/libmv/ChangeLog 2013-01-07 12:20:51 UTC (rev 53631)
@@ -1,3 +1,49 @@
+commit 575336f794841ada90aacd783285014081b8318c
+Author: Sergey Sharybin <sergey.vfx at gmail.com>
+Date: Mon Jan 7 15:58:40 2013 +0600
+
+ Fixed for keyframe selection
+
+ - Calculate residuals for GRIC in pixel space rather than
+ in normalized space.
+
+ This seems to be how it's intended to be used.
+
+ Algebraic H and F will still use normalized coordinates which
+ are more stable, after this matrices are converted to pixel
+ space and Ceres refinement happens in pixel space.
+
+ - Standard deviation calculation was wrong in GRIC. It shouldn't
+ be deviation of residuals, but as per Torr it should be deviation
+ of measurement error, which is constant (in our case 0.1)
+
+ Not sure if using squared cost function is correct for GRIC,
+ but cost function is indeed squared and in most papers cost
+ function is used for GRIC. After some further tests we could
+ switch GRIC residuals to non-squared distance.
+
+ - Bring back rho part of GRIC, in unit tests it doesn't make
+ sense whether it's enabled or not, lets see how it'll behave
+ in real-life footage.
+
+ - Added one more unit test based on elevator scene and manual
+ keyframe selection.
+
+commit 24117f3c3fc5531beb6497d79bb6f1780a998081
+Author: Sergey Sharybin <sergey.vfx at gmail.com>
+Date: Sun Jan 6 19:07:06 2013 +0600
+
+ Added test for keyframe selection based on manual selection
+
+ Additional changes:
+
+ - Reduce minimal correspondence to match real-world manually
+ tracked footage
+
+ - Returned back squares to SymmetricEpipolarDistance and
+ SymmetricGeometricDistance -- this is actually a cost
+ functions, not distances and they shall be squared.
+
commit 770eb0293b881c4c419c587a6cdb062c47ab6e44
Author: Sergey Sharybin <sergey.vfx at gmail.com>
Date: Fri Dec 21 00:43:30 2012 +0600
@@ -742,15 +788,3 @@
Date: Fri Feb 17 21:32:05 2012 +0600
Picky edits: corrected EOL
-
-commit 3f2a4205ec5adadcdfa306b161c705c868a7be93
-Author: Sergey Sharybin <sergey.vfx at gmail.com>
-Date: Fri Feb 17 21:30:07 2012 +0600
-
- Fixed incorrect access to ucontext on linux. Caused by incorrect merge conflict resolve.
-
-commit d360a21a5aa125cf9e83dd26b302508688ff7007
-Author: Sergey Sharybin <sergey.vfx at gmail.com>
-Date: Fri Feb 17 20:54:13 2012 +0600
-
- More Windows -> Unix EOL conversions
Modified: branches/soc-2011-tomato/extern/libmv/libmv/multiview/fundamental.cc
===================================================================
--- branches/soc-2011-tomato/extern/libmv/libmv/multiview/fundamental.cc 2013-01-07 12:19:00 UTC (rev 53630)
+++ branches/soc-2011-tomato/extern/libmv/libmv/multiview/fundamental.cc 2013-01-07 12:20:51 UTC (rev 53631)
@@ -254,8 +254,8 @@
Vec3 Ft_y = F.transpose() * y;
double y_F_x = y.dot(F_x);
- return y_F_x * ( 1 / F_x.head<2>().norm()
- + 1 / Ft_y.head<2>().norm());
+ return Square(y_F_x) * ( 1 / F_x.head<2>().squaredNorm()
+ + 1 / Ft_y.head<2>().squaredNorm());
}
// HZ 9.6 pag 257 (formula 9.12)
Modified: branches/soc-2011-tomato/extern/libmv/libmv/multiview/homography.cc
===================================================================
--- branches/soc-2011-tomato/extern/libmv/libmv/multiview/homography.cc 2013-01-07 12:19:00 UTC (rev 53630)
+++ branches/soc-2011-tomato/extern/libmv/libmv/multiview/homography.cc 2013-01-07 12:20:51 UTC (rev 53631)
@@ -264,4 +264,19 @@
return false;
}
}
+
+double SymmetricGeometricDistance(Mat3 &H, Vec2 &x1, Vec2 &x2) {
+ Vec3 x(x1(0), x1(1), 1.0);
+ Vec3 y(x2(0), x2(1), 1.0);
+
+ Vec3 H_x = H * x;
+ Vec3 Hinv_y = H.inverse() * y;
+
+ H_x /= H_x(2);
+ Hinv_y /= Hinv_y(2);
+
+ return (H_x.head<2>() - y.head<2>()).squaredNorm() +
+ (Hinv_y.head<2>() - x.head<2>()).squaredNorm();
+}
+
} // namespace libmv
Modified: branches/soc-2011-tomato/extern/libmv/libmv/multiview/homography.h
===================================================================
--- branches/soc-2011-tomato/extern/libmv/libmv/multiview/homography.h 2013-01-07 12:19:00 UTC (rev 53630)
+++ branches/soc-2011-tomato/extern/libmv/libmv/multiview/homography.h 2013-01-07 12:20:51 UTC (rev 53631)
@@ -79,6 +79,14 @@
Mat4 *H,
double expected_precision =
EigenDouble::dummy_precision());
+
+/**
+ * Calculate symmetric geometric cost:
+ *
+ * D(H * x1, x2)^2 + D(H^-1 * x2, x1)
+ */
+double SymmetricGeometricDistance(Mat3 &H, Vec2 &x1, Vec2 &x2);
+
} // namespace libmv
#endif // LIBMV_MULTIVIEW_HOMOGRAPHY_H_
Modified: branches/soc-2011-tomato/extern/libmv/libmv/simple_pipeline/keyframe_selection.cc
===================================================================
--- branches/soc-2011-tomato/extern/libmv/libmv/simple_pipeline/keyframe_selection.cc 2013-01-07 12:19:00 UTC (rev 53630)
+++ branches/soc-2011-tomato/extern/libmv/libmv/simple_pipeline/keyframe_selection.cc 2013-01-07 12:20:51 UTC (rev 53631)
@@ -27,20 +27,28 @@
namespace libmv {
namespace {
-double SymmetricGeometricDistance(Mat3 &H, Vec2 &x1, Vec2 &x2) {
- Vec3 x(x1(0), x1(1), 1.0);
- Vec3 y(x2(0), x2(1), 1.0);
- Vec3 H_x = H * x;
- Vec3 Hinv_y = H.inverse() * y;
+Vec2 NorrmalizedToPixelSpace(Vec2 vec, CameraIntrinsics &intrinsics) {
+ Vec2 result;
- H_x /= H_x(2);
- Hinv_y /= Hinv_y(2);
+ result(0) = vec(0) * intrinsics.focal_length_x() + intrinsics.principal_point_x();
+ result(1) = vec(1) * intrinsics.focal_length_y() + intrinsics.principal_point_y();
- return (H_x.head<2>() - y.head<2>()).norm() +
- (Hinv_y.head<2>() - x.head<2>()).norm();
+ return result;
}
+Mat3 IntrinsicsNormalizationMatrix(CameraIntrinsics &intrinsics) {
+ Mat3 T = Mat3::Identity(), S = Mat3::Identity();
+
+ T(0, 2) = -intrinsics.principal_point_x();
+ T(1, 2) = -intrinsics.principal_point_y();
+
+ S(0, 0) /= intrinsics.focal_length_x();
+ S(1, 1) /= intrinsics.focal_length_y();
+
+ return S * T;
+}
+
class HomographySymmetricGeometricCostFunctor {
public:
HomographySymmetricGeometricCostFunctor(Vec2 x, Vec2 y)
@@ -77,16 +85,23 @@
const Vec2 y_;
};
-void ComputeHomographyFromCorrespondences(Mat &x1, Mat &x2, Mat3 *H) {
- // Algebraic homography estimation
- Homography2DFromCorrespondencesLinear(x1, x2, H);
+void ComputeHomographyFromCorrespondences(Mat &x1, Mat &x2, CameraIntrinsics &intrinsics, Mat3 *H) {
+ // Algebraic homography estimation, happens with normalized coordinates
+ Homography2DFromCorrespondencesLinear(x1, x2, H, 1e-12);
- // Refine matrix using Ceres minimizer
+ // Convert homography to original pixel space
+ Mat3 N = IntrinsicsNormalizationMatrix(intrinsics);
+ *H = N.inverse() * (*H) * N;
+
+ // Refine matrix using Ceres minimizer, it'll be in pixel space
ceres::Problem problem;
for (int i = 0; i < x1.cols(); i++) {
+ Vec2 pixel_space_x1 = NorrmalizedToPixelSpace(x1.col(i), intrinsics),
+ pixel_space_x2 = NorrmalizedToPixelSpace(x2.col(i), intrinsics);
+
HomographySymmetricGeometricCostFunctor *homography_symmetric_geometric_cost_function =
- new HomographySymmetricGeometricCostFunctor(x1.col(i), x2.col(i));
+ new HomographySymmetricGeometricCostFunctor(pixel_space_x1, pixel_space_x2);
problem.AddResidualBlock(
new ceres::AutoDiffCostFunction<
@@ -143,17 +158,24 @@
const Mat y_;
};
-void ComputeFundamentalFromCorrespondences(Mat &x1, Mat &x2, Mat3 *F)
+void ComputeFundamentalFromCorrespondences(Mat &x1, Mat &x2, CameraIntrinsics &intrinsics, Mat3 *F)
{
- // Algebraic fundamental estimation
+ // Algebraic fundamental estimation, happens with normalized coordinates
NormalizedEightPointSolver(x1, x2, F);
- // Refine matrix using Ceres minimizer
+ // Convert fundamental to original pixel space
+ Mat3 N = IntrinsicsNormalizationMatrix(intrinsics);
+ *F = N.inverse() * (*F) * N;
+
+ // Refine matrix using Ceres minimizer, it'll be in pixel space
ceres::Problem problem;
for (int i = 0; i < x1.cols(); i++) {
+ Vec2 pixel_space_x1 = NorrmalizedToPixelSpace(x1.col(i), intrinsics),
+ pixel_space_x2 = NorrmalizedToPixelSpace(x2.col(i), intrinsics);
+
FundamentalSymmetricEpipolarCostFunctor *fundamental_symmetric_epipolar_cost_function =
- new FundamentalSymmetricEpipolarCostFunctor(x1.col(i), x2.col(i));
+ new FundamentalSymmetricEpipolarCostFunctor(pixel_space_x1, pixel_space_x2);
problem.AddResidualBlock(
new ceres::AutoDiffCostFunction<
@@ -195,28 +217,20 @@
// lambda3 limits the residual error, and this paper
// http://elvera.nue.tu-berlin.de/files/0990Knorr2006.pdf
// suggests using lambda3 of 2
+ // same value is used in Torr's Problem of degeneracy in structure and motion recovery
+ // from uncalibrated image sequences
+ // http://www.robots.ox.ac.uk/~vgg/publications/papers/torr99.ps.gz
double lambda3 = 2.0;
- // Compute squared standard deviation sigma2 of the error
- double mean_value = 0;
- for (int i = 0; i < n; i++)
- mean_value += e(i);
- mean_value /= (double) n;
+ // measurement error of tracker
+ double sigma2 = 0.01;
- double sigma2 = 0.0;
- for (int i = 0; i < n; i++)
- sigma2 += Square(e(i) - mean_value);
- sigma2 /= n;
-
// Actual GRIC computation
double gric_result = 0.0;
for (int i = 0; i < n; i++) {
- // disable rho stuff for now since it seems to be leading to wrong
- // results in some cases
- //double rho = std::min(e(i) * e(i) / sigma2, lambda3 * (r - d));
- //gric_result += rho;
- gric_result += e(i) * e(i) / sigma2;
+ double rho = std::min(e(i) * e(i) / sigma2, lambda3 * (r - d));
+ gric_result += rho;
}
gric_result += lambda1 * d * n;
@@ -227,7 +241,9 @@
} // namespace
-void SelectkeyframesBasedOnGRIC(Tracks &tracks, vector<int> &keyframes) {
+void SelectkeyframesBasedOnGRIC(Tracks &tracks,
+ CameraIntrinsics &intrinsics,
@@ Diff output truncated at 10240 characters. @@
More information about the Bf-blender-cvs
mailing list