[Bf-blender-cvs] SVN commit: /data/svn/bf-blender [56265] branches/soc-2011-tomato/extern/ libmv: Initial commit of reconstruction variance criteria
Sergey Sharybin
sergey.vfx at gmail.com
Wed Apr 24 16:30:42 CEST 2013
Revision: 56265
http://projects.blender.org/scm/viewvc.php?view=rev&root=bf-blender&revision=56265
Author: nazgul
Date: 2013-04-24 14:30:42 +0000 (Wed, 24 Apr 2013)
Log Message:
-----------
Initial commit of reconstruction variance criteria
which is an addition for GRIC-based keyframe selection.
Uses paper Keyframe Selection for Camera Motion and Structure
Estimation from Multiple Views,
ftp://ftp.tnt.uni-hannover.de/pub/papers/2004/ECCV2004-TTHBAW.pdf
as a basis.
Currently implemented camera positions reconstructions,
bundle positions estimation and bundle adjustment step.
Covriance estimation is implemented in very basic way
and need to be cleaned up, speed up and probably fixed.
Covariance matrix is estimating using generalized inverse
with 7 (by the number of gauge freedoms) zeroed eigen values
of J^T * J. Use value of 7 because we've got 3 translations,
3 rotations and 1 scale freedoms.
Additional changes:
- Added utility function FundamentalToEssential to extract
E from F matrix, used by both final reconstruction pipeline
and reconstruction variance code.
- Refactored bundler a bit, so now it's possible to return
different evaluation data, such as number of cameras and
points being minimized and also jacobian.
Jacobian currently contains only camera and points columns,
no intrinsics there yet. It is also currently converting to
an Eigen dense matrix. A bit weak, but speed is nice for
tests.
Columns in jacobian are ordered in the following way:
first columns are cameras (3 cols for rotation and 3 cols
for translation), then goes 3D point columns.
- Switched F and H refining to normalized space. Apparently,
refining F in pixel space squeezes it a lot making it wrong.
- EuclideanIntersect will not add point to reconstruction if
it happened to be behind the camera.
- Cleaned style a bit.
Modified Paths:
--------------
branches/soc-2011-tomato/extern/libmv/ChangeLog
branches/soc-2011-tomato/extern/libmv/libmv/multiview/fundamental.cc
branches/soc-2011-tomato/extern/libmv/libmv/multiview/fundamental.h
branches/soc-2011-tomato/extern/libmv/libmv/simple_pipeline/bundle.cc
branches/soc-2011-tomato/extern/libmv/libmv/simple_pipeline/bundle.h
branches/soc-2011-tomato/extern/libmv/libmv/simple_pipeline/initialize_reconstruction.cc
branches/soc-2011-tomato/extern/libmv/libmv/simple_pipeline/intersect.cc
branches/soc-2011-tomato/extern/libmv/libmv/simple_pipeline/keyframe_selection.cc
branches/soc-2011-tomato/extern/libmv/libmv/simple_pipeline/keyframe_selection.h
branches/soc-2011-tomato/extern/libmv/libmv/simple_pipeline/pipeline.cc
branches/soc-2011-tomato/extern/libmv/libmv-capi.cpp
Modified: branches/soc-2011-tomato/extern/libmv/ChangeLog
===================================================================
--- branches/soc-2011-tomato/extern/libmv/ChangeLog 2013-04-24 14:14:54 UTC (rev 56264)
+++ branches/soc-2011-tomato/extern/libmv/ChangeLog 2013-04-24 14:30:42 UTC (rev 56265)
@@ -1,3 +1,79 @@
+commit a9006989e45afe6179b22f89f7245d4f6fffd6f2
+Author: Sergey Sharybin <sergey.vfx at gmail.com>
+Date: Thu Mar 14 14:53:42 2013 +0600
+
+ Initial commit of reconstruction variance criteria
+ which is an addition for GRIC-based keyframe selection.
+
+ Uses paper Keyframe Selection for Camera Motion and Structure
+ Estimation from Multiple Views,
+ ftp://ftp.tnt.uni-hannover.de/pub/papers/2004/ECCV2004-TTHBAW.pdf
+ as a basis.
+
+ Currently implemented camera positions reconstructions,
+ bundle positions estimation and bundle adjustment step.
+
+ Covariance matrix is estimating using generalized inverse
+ with 7 (by the number of gauge freedoms) zeroed eigen values
+ of J^T * J.
+
+ Additional changes:
+ - Added utility function FundamentalToEssential to extract
+ E from F matrix, used by both final reconstruction pipeline
+ and reconstruction variance code.
+
+ - Refactored bundler a bit, so now it's possible to return
+ different evaluation data, such as number of cameras and
+ points being minimized and also jacobian.
+
+ Jacobian currently contains only camera and points columns,
+ no intrinsics there yet. It is also currently converting to
+ an Eigen dense matrix. A bit weak, but speed is nice for
+ tests.
+
+ Columns in jacobian are ordered in the following way:
+ first columns are cameras (3 cols for rotation and 3 cols
+ for translation), then goes 3D point columns.
+
+ - Switched F and H refining to normalized space. Apparently,
+ refining F in pixel space squeezes it a lot making it wrong.
+
+ - EuclideanIntersect will not add point to reconstruction if
+ it happened to be behind the camera.
+
+ - Cleaned style a bit.
+
+commit 2b7d2b44e6446bbcc23038f5dbb824feca888069
+Author: Sergey Sharybin <sergey.vfx at gmail.com>
+Date: Wed Apr 24 19:40:39 2013 +0600
+
+ Reconstructed scene scale ambiguity improvement
+
+ Added a function EuclideanScaleToUnity() which is
+ aimed to solve scale ambiguity by scaling solution
+ in a way cameras centers variance in unity.
+
+ Currently only available for euclidean pipeline,
+ projective one is not finished anyway.
+
+commit ed1f650576dc6f5b648a026a2861c54827b0f5c9
+Author: Sergey Sharybin <sergey.vfx at gmail.com>
+Date: Tue Apr 23 01:41:29 2013 +0600
+
+ Use epsilon in modal solver test
+
+ Default epsilon for isApprox was too small,
+ leading to some false test failures.
+
+commit c44679a9a0fafdde6a0a22e7e5c8496fc9c93cd0
+Author: Sergey Sharybin <sergey.vfx at gmail.com>
+Date: Tue Apr 23 01:35:56 2013 +0600
+
+ Update Ceres to current HEAD
+
+ Brings optimization for DENSE_NORMAL_CHOLESKY and
+ also fixes threading issues with BLAS.
+
commit 03cbc88ce7f51aa26ba503acea2e984bcb78873c
Author: Sergey Sharybin <sergey.vfx at gmail.com>
Date: Mon Apr 15 05:35:33 2013 +0600
@@ -612,82 +688,3 @@
Still need to be investigated why it only works if tracks
are in pixel space and why doesn't work in normalized space.
-
-commit cfabdfe48df2add3d1f30cf4370efd0b31990ab0
-Author: Sergey Sharybin <sergey.vfx at gmail.com>
-Date: Thu Dec 20 05:46:53 2012 +0600
-
- Assorted fixes for keyframe selection:
-
- - Biggest error was in cost functors used for F and H refirement,
- they were just wrong.
-
- - Use natural logarithms, since it's actually makes sense from
- math papers point of view and error is somewhere else.
-
- - Disabled rho for GRIC, for now use non-clamped error for tests.
-
- - Made SymmetricEpipolarDistance returning non-squared distance
- Keyframe selection is currently the only used of this function
- and it seems using non-squared distance makes much more sense.
-
- Also would think to append suffix "Squared" to functions which
- returns squared distances.
-
- - Removed templated version of SymmetricEpipolarDistance, since
- it's not needed actually.
-
- This is actually even worse working than previous implementation,
- but commit it needed for further review.
-
-commit 35d8c57626ad74818f155e6e5960c663ea84e032
-Author: Sergey Sharybin <sergey.vfx at gmail.com>
-Date: Thu Dec 20 03:00:40 2012 +0600
-
- Euclidean resection cost function didn't use correct constructor
-
- It was storing a reference to initial rotation passed by value,
- leading to pointer being pointing to a stack variable, leading to
- wrong memory access in residuals computing.
-
- Apparently was visible in optimized builds only with inline
- substitution allowed.
-
-commit 0798d3162bb49cee7e1c423ceccbca1326ad5650
-Author: Sergey Sharybin <sergey.vfx at gmail.com>
-Date: Thu Dec 20 02:50:52 2012 +0600
-
- Automatic keyframe selection based on Pollefeys's criteria
-
- This commit implements automatic keyframe selection algorithm
- based on Pollefeys's criteria (F-GRIC is smaller than H-GRIC
- and correspondence ratio is more then 90%).
-
- It is implemented as a part of simple pipeline and returns
- vector of keyframe images for a given Tracks structure.
-
- For simple pipeline reconstruction two best keyframes are
- expected to be selected from all detected candidates.
- Criteria for this selection could be reprojection error of
- solution from two candidate keyfames.
-
- Unfortunately, it's not fully workable yet, hopefully would
- be fixed soon.
-
-commit e943985552f0598ae122252876f305d72c25c2f9
-Author: Sergey Sharybin <sergey.vfx at gmail.com>
-Date: Thu Dec 6 17:47:11 2012 +0600
-
- Camera Tracking: allow fallback to reprojection resection
- by user demand
-
- This fixes some "regressions" introduced in previous commit
- which lead to much worse solution in some cases. Now it's
- possible to bring old behavior back.
-
- Perhaps it's more like temporal solution for time being smarter
- solution is found. But finding such a solution isn't so fast,
- so let's bring manual control over reprojection usage.
-
- But anyway, imo it's now nice to have a structure which could
- be used to pass different settings to the solver.
Modified: branches/soc-2011-tomato/extern/libmv/libmv/multiview/fundamental.cc
===================================================================
--- branches/soc-2011-tomato/extern/libmv/libmv/multiview/fundamental.cc 2013-04-24 14:14:54 UTC (rev 56264)
+++ branches/soc-2011-tomato/extern/libmv/libmv/multiview/fundamental.cc 2013-04-24 14:30:42 UTC (rev 56265)
@@ -388,4 +388,23 @@
}
}
+void FundamentalToEssential(const Mat3 &F, Mat3 *E) {
+ Eigen::JacobiSVD<Mat3> svd(F, Eigen::ComputeFullU | Eigen::ComputeFullV);
+
+ // See Hartley & Zisserman page 294, result 11.1, which shows how to get the
+ // closest essential matrix to a matrix that is "almost" an essential matrix.
+ double a = svd.singularValues()(0);
+ double b = svd.singularValues()(1);
+ double s = (a + b) / 2.0;
+
+ LG << "Initial reconstruction's rotation is non-euclidean by "
+ << (((a - b) / std::max(a, b)) * 100) << "%; singular values:"
+ << svd.singularValues().transpose();
+
+ Vec3 diag;
+ diag << s, s, 0;
+
+ *E = svd.matrixU() * diag.asDiagonal() * svd.matrixV().transpose();
+}
+
} // namespace libmv
Modified: branches/soc-2011-tomato/extern/libmv/libmv/multiview/fundamental.h
===================================================================
--- branches/soc-2011-tomato/extern/libmv/libmv/multiview/fundamental.h 2013-04-24 14:14:54 UTC (rev 56264)
+++ branches/soc-2011-tomato/extern/libmv/libmv/multiview/fundamental.h 2013-04-24 14:30:42 UTC (rev 56265)
@@ -139,6 +139,11 @@
Mat3 *R,
Vec3 *t);
+/**
+ * Find closest essential matrix E to fundamental F
+ */
+void FundamentalToEssential(const Mat3 &F, Mat3 *E);
+
} // namespace libmv
#endif // LIBMV_MULTIVIEW_FUNDAMENTAL_H_
Modified: branches/soc-2011-tomato/extern/libmv/libmv/simple_pipeline/bundle.cc
===================================================================
--- branches/soc-2011-tomato/extern/libmv/libmv/simple_pipeline/bundle.cc 2013-04-24 14:14:54 UTC (rev 56264)
+++ branches/soc-2011-tomato/extern/libmv/libmv/simple_pipeline/bundle.cc 2013-04-24 14:30:42 UTC (rev 56265)
@@ -235,6 +235,29 @@
}
}
+// Converts sparse CRSMatrix to Eigen matrix, so it could be used
+// all over in the pipeline
+//
+// TODO(sergey): currently uses dense Eigen matrices, best would
+// be to use sparse Eigen matrices
+void CRSMatrixToEigenMatrix(const ceres::CRSMatrix &crs_matrix,
+ Mat *eigen_matrix) {
+ eigen_matrix->resize(crs_matrix.num_rows, crs_matrix.num_cols);
+ eigen_matrix->setZero();
+
+ for (int row = 0; row < crs_matrix.num_rows; ++row) {
+ int start = crs_matrix.rows[row];
+ int end = crs_matrix.rows[row + 1] - 1;
+
+ for (int i = start; i <= end; i++) {
+ int col = crs_matrix.cols[i];
+ double value = crs_matrix.values[i];
+
+ (*eigen_matrix)(row, col) = value;
+ }
+ }
+}
+
} // namespace
void EuclideanBundle(const Tracks &tracks,
@@ -242,15 +265,18 @@
CameraIntrinsics intrinsics;
EuclideanBundleCommonIntrinsics(tracks,
BUNDLE_NO_INTRINSICS,
+ BUNDLE_NO_CONSTRAINTS,
reconstruction,
- &intrinsics);
+ &intrinsics,
+ NULL);
}
void EuclideanBundleCommonIntrinsics(const Tracks &tracks,
int bundle_intrinsics,
+ int bundle_constraints,
EuclideanReconstruction *reconstruction,
CameraIntrinsics *intrinsics,
- int bundle_constraints) {
+ BundleEvaluation *evaluation) {
@@ Diff output truncated at 10240 characters. @@
More information about the Bf-blender-cvs
mailing list