[Bf-committers] GSOC Ideas: 3D Reconstruction

[Sergey Sharybin]

Hi,

Point cloud reconstruction could be quite a benefitial project for blender
if fully/properly done. There're basis stuff for this already present in
libmv (in terms of reconstruction/bundling), would need to implement
correspondence matching tho and outlier filtering (which would also be
needed for automated tracking workflow). But this will also imply quite
enough of improvements in blender side -- being able to manipulate with
unordered image collections, point clouds structure (afraid mesh will be
too much expensive for this), tools to deal with point clouds (surface
reconstruction/optimization/texturing).. This kind of glues two your
proposals into one.

Ends up in quite bigger project, but think delivering new tools to artists
is more priority than adding extra ground for them when speaking of GSoC
project. So think better thing would be to first aim to surface
reconstruction tools, texturing, orientation  (bundles could be obtained
from several cameras) and so on, and then going to "depths" in terms adding
SfM from unordered image sounds more awesome. :)

Motion capture i'm not sure. This is rather challenging project for nice
integration into blender/libmv, not sure GSoC's time frame is enough for
this?


On Fri, Apr 26, 2013 at 8:57 PM, Shabbir Marzban <mshabbir66 at gmail.com>wrote:

> Hi,
> I wish to participate in upcoming GSOC. I want to discuss couple of ideas.
> It would be great if I can have some feedback on these :).
>
> I have described these briefly below. Kindly let me know if further
> details/ clarifications are required.
>
> 1) Automated rigid structure reconstruction from set of un-ordered images:
> A very common structure from motion (SFM) tool, Bundler[1] can be used to
> recover 3D point cloud from such images. Its output can
> further transformed into dense patches[2] on which schematic reconstruction
> can be done[3]. See the video here: <
> http://www.youtube.com/watch?v=UwSGtqUZx3M>. Developing this pipeline into
> Blender will be a fun project.
>
> 2) An interactive tool for 3D modelling: A tool can be made
> and integrated into blender as indicated in [4] which uses bundler's[1]
> point cloud. They use plane estimation for modelling which is quite
> accurate. They have presented a pretty neat and robust way of estimating
> planes using points clouds and vanishing points. See the video here:
> http://www.youtube.com/watch?v=RhNNLxHwz6s.
>
> 3)Non-rigid reconstruction (motion capture) from set of static cameras:This
> reconstruction is for slightly different domain, mainly motion capture.
> This is actually one of the methods I have coauthored[5]. It requires
> labelled track points in all the cameras feeds as input. The salient
> feature of this method is that it can handle occlusions. This method has
> been extensively tested on CMU mocap database.
>
> Poject website: http://cvlab.lums.edu.pk/MultiviewNRSFM/
> Video: http://www.youtube.com/watch?v=a1XUPI9L9vA
>
> Reconstruction from this method can be used in motion re-targeting like in
> Avator and Pirates of the Caribbean. Including this into blender will be a
> great GSOC project.
>
> Kind Regards.
> Shabbir Marzban,
>
> I am first year grad-student at School of Science and Engineering, Lahore
> University of Management Sciences, Pakistan. My resume and sample of some
> old projects can be seen here: http://shabbirm.weebly.com/index.html
>
>
> [1]. Snavely, Noah, Steven M. Seitz, and Richard Szeliski. "Modeling the
> world from internet photo collections." *International Journal of Computer
> Vision* 80.2 (2008): 189-210.
>
> [2]Furukawa, Yasutaka, et al. "Towards internet-scale multi-view
> stereo."*Computer
> Vision and Pattern Recognition (CVPR), 2010 IEEE Conference on*. IEEE,
> 2010.
>
> [3] Wu, Changchang, et al. "Schematic surface reconstruction." *Computer
> Vision and Pattern Recognition (CVPR), 2012 IEEE Conference on*. IEEE,
> 2012.
>
> [4] Sinha, Sudipta N., et al. "Interactive 3D architectural modeling from
> unordered photo collections." *ACM Transactions on Graphics (TOG)*. Vol.
> 27. No. 5. ACM, 2008.
>
> [5] Zaheer, Aamer, et al. "Multiview structure from motion in trajectory
> space."*Computer Vision (ICCV), 2011 IEEE International Conference on*.
> IEEE, 2011.
> _______________________________________________
> Bf-committers mailing list
> Bf-committers at blender.org
> http://lists.blender.org/mailman/listinfo/bf-committers
>



-- 
With best regards, Sergey Sharybin