[Bf-committers] GSOC Ideas: 3D Reconstruction

[Jace Priester]

Meshes are very sufficient for handling point cloud data as I have already
discussed. They are a collection of vertices, stored efficiently and
rendered to the viewport efficiently. They are simply lacking per-vertex
color and size, which is what my code adds. There is no need for a new
object type to accommodate point cloud data. That's actually a really good
thing, because all the existing mesh tools continue to work.

I agree this all sounds like too big of a project. If there were a library
out there that did this type of work well and it could be integrated with
Blender, that would be great, but there isn't. Bundler is great but it's
only one step in the pipeline. That said, if this does end up getting
underway I'm very interested in working on it with whoever is involved. And
like I said I'm working on putting together a patch for the little bit that
I have, and I'll get that part out in a couple days here.


On Tue, Apr 30, 2013 at 12:41 PM, Sergey Sharybin <sergey.vfx at gmail.com>wrote:

> Hi,
>
> Point cloud reconstruction could be quite a benefitial project for blender
> if fully/properly done. There're basis stuff for this already present in
> libmv (in terms of reconstruction/bundling), would need to implement
> correspondence matching tho and outlier filtering (which would also be
> needed for automated tracking workflow). But this will also imply quite
> enough of improvements in blender side -- being able to manipulate with
> unordered image collections, point clouds structure (afraid mesh will be
> too much expensive for this), tools to deal with point clouds (surface
> reconstruction/optimization/texturing).. This kind of glues two your
> proposals into one.
>
> Ends up in quite bigger project, but think delivering new tools to artists
> is more priority than adding extra ground for them when speaking of GSoC
> project. So think better thing would be to first aim to surface
> reconstruction tools, texturing, orientation  (bundles could be obtained
> from several cameras) and so on, and then going to "depths" in terms adding
> SfM from unordered image sounds more awesome. :)
>
> Motion capture i'm not sure. This is rather challenging project for nice
> integration into blender/libmv, not sure GSoC's time frame is enough for
> this?
>
>
> On Fri, Apr 26, 2013 at 8:57 PM, Shabbir Marzban <mshabbir66 at gmail.com
> >wrote:
>
> > Hi,
> > I wish to participate in upcoming GSOC. I want to discuss couple of
> ideas.
> > It would be great if I can have some feedback on these :).
> >
> > I have described these briefly below. Kindly let me know if further
> > details/ clarifications are required.
> >
> > 1) Automated rigid structure reconstruction from set of un-ordered
> images:
> > A very common structure from motion (SFM) tool, Bundler[1] can be used to
> > recover 3D point cloud from such images. Its output can
> > further transformed into dense patches[2] on which schematic
> reconstruction
> > can be done[3]. See the video here: <
> > http://www.youtube.com/watch?v=UwSGtqUZx3M>. Developing this pipeline
> into
> > Blender will be a fun project.
> >
> > 2) An interactive tool for 3D modelling: A tool can be made
> > and integrated into blender as indicated in [4] which uses bundler's[1]
> > point cloud. They use plane estimation for modelling which is quite
> > accurate. They have presented a pretty neat and robust way of estimating
> > planes using points clouds and vanishing points. See the video here:
> > http://www.youtube.com/watch?v=RhNNLxHwz6s.
> >
> > 3)Non-rigid reconstruction (motion capture) from set of static
> cameras:This
> > reconstruction is for slightly different domain, mainly motion capture.
> > This is actually one of the methods I have coauthored[5]. It requires
> > labelled track points in all the cameras feeds as input. The salient
> > feature of this method is that it can handle occlusions. This method has
> > been extensively tested on CMU mocap database.
> >
> > Poject website: http://cvlab.lums.edu.pk/MultiviewNRSFM/
> > Video: http://www.youtube.com/watch?v=a1XUPI9L9vA
> >
> > Reconstruction from this method can be used in motion re-targeting like
> in
> > Avator and Pirates of the Caribbean. Including this into blender will be
> a
> > great GSOC project.
> >
> > Kind Regards.
> > Shabbir Marzban,
> >
> > I am first year grad-student at School of Science and Engineering, Lahore
> > University of Management Sciences, Pakistan. My resume and sample of some
> > old projects can be seen here: http://shabbirm.weebly.com/index.html
> >
> >
> > [1]. Snavely, Noah, Steven M. Seitz, and Richard Szeliski. "Modeling the
> > world from internet photo collections." *International Journal of
> Computer
> > Vision* 80.2 (2008): 189-210.
> >
> > [2]Furukawa, Yasutaka, et al. "Towards internet-scale multi-view
> > stereo."*Computer
> > Vision and Pattern Recognition (CVPR), 2010 IEEE Conference on*. IEEE,
> > 2010.
> >
> > [3] Wu, Changchang, et al. "Schematic surface reconstruction." *Computer
> > Vision and Pattern Recognition (CVPR), 2012 IEEE Conference on*. IEEE,
> > 2012.
> >
> > [4] Sinha, Sudipta N., et al. "Interactive 3D architectural modeling from
> > unordered photo collections." *ACM Transactions on Graphics (TOG)*. Vol.
> > 27. No. 5. ACM, 2008.
> >
> > [5] Zaheer, Aamer, et al. "Multiview structure from motion in trajectory
> > space."*Computer Vision (ICCV), 2011 IEEE International Conference on*.
> > IEEE, 2011.
> > _______________________________________________
> > Bf-committers mailing list
> > Bf-committers at blender.org
> > http://lists.blender.org/mailman/listinfo/bf-committers
> >
>
>
>
> --
> With best regards, Sergey Sharybin
> _______________________________________________
> Bf-committers mailing list
> Bf-committers at blender.org
> http://lists.blender.org/mailman/listinfo/bf-committers
>



-- 


--------------------------------------------------------------------------
Jace Priester
Threespace Imaging
jacepriester at threespaceimaging.com
559-284-0904
--------------------------------------------------------------------------