[Bf-committers] GSOC Ideas: 3D Reconstruction

Tue Apr 30 22:28:57 CEST 2013

Custom data layers can be present, or not. The whole point is that they're
optional. This means that the color/size CDLs will only be in use when
colored point cloud data is imported. This means zero change and zero extra
memory usage for all other meshes.

On Tue, Apr 30, 2013 at 1:07 PM, Sergey Sharybin <sergey.vfx at gmail.com>wrote:

> Per-vertex color and size is not something i'll want to be added to general
> vertex. This will lead to much higher memory consumption of every mesh, and
> this could become really an issue for modelers. That'd also lead to wish
> being able to render such vertices as "strand"? And mixing "point cloud"
> vertices with regular modelling.. Well, i'm skeptical about all this.
>
> Perhaps CD layers will solve memory issues, but eeh.. Really not sure.
>
>
> On Wed, May 1, 2013 at 1:53 AM, Jace Priester <
> jacepriester at threespaceimaging.com> wrote:
>
> > Meshes are very sufficient for handling point cloud data as I have
> already
> > discussed. They are a collection of vertices, stored efficiently and
> > rendered to the viewport efficiently. They are simply lacking per-vertex
> > color and size, which is what my code adds. There is no need for a new
> > object type to accommodate point cloud data. That's actually a really
> good
> > thing, because all the existing mesh tools continue to work.
> >
> > I agree this all sounds like too big of a project. If there were a
> library
> > out there that did this type of work well and it could be integrated with
> > Blender, that would be great, but there isn't. Bundler is great but it's
> > only one step in the pipeline. That said, if this does end up getting
> > underway I'm very interested in working on it with whoever is involved.
> And
> > like I said I'm working on putting together a patch for the little bit
> that
> > I have, and I'll get that part out in a couple days here.
> >
> >
> > On Tue, Apr 30, 2013 at 12:41 PM, Sergey Sharybin <sergey.vfx at gmail.com
> > >wrote:
> >
> > > Hi,
> > >
> > > Point cloud reconstruction could be quite a benefitial project for
> > blender
> > > if fully/properly done. There're basis stuff for this already present
> in
> > > libmv (in terms of reconstruction/bundling), would need to implement
> > > correspondence matching tho and outlier filtering (which would also be
> > > needed for automated tracking workflow). But this will also imply quite
> > > enough of improvements in blender side -- being able to manipulate with
> > > unordered image collections, point clouds structure (afraid mesh will
> be
> > > too much expensive for this), tools to deal with point clouds (surface
> > > reconstruction/optimization/texturing).. This kind of glues two your
> > > proposals into one.
> > >
> > > Ends up in quite bigger project, but think delivering new tools to
> > artists
> > > is more priority than adding extra ground for them when speaking of
> GSoC
> > > project. So think better thing would be to first aim to surface
> > > reconstruction tools, texturing, orientation  (bundles could be
> obtained
> > > from several cameras) and so on, and then going to "depths" in terms
> > adding
> > > SfM from unordered image sounds more awesome. :)
> > >
> > > Motion capture i'm not sure. This is rather challenging project for
> nice
> > > integration into blender/libmv, not sure GSoC's time frame is enough
> for
> > > this?
> > >
> > >
> > > On Fri, Apr 26, 2013 at 8:57 PM, Shabbir Marzban <mshabbir66 at gmail.com
> > > >wrote:
> > >
> > > > Hi,
> > > > I wish to participate in upcoming GSOC. I want to discuss couple of
> > > ideas.
> > > > It would be great if I can have some feedback on these :).
> > > >
> > > > I have described these briefly below. Kindly let me know if further
> > > > details/ clarifications are required.
> > > >
> > > > 1) Automated rigid structure reconstruction from set of un-ordered
> > > images:
> > > > A very common structure from motion (SFM) tool, Bundler[1] can be
> used
> > to
> > > > recover 3D point cloud from such images. Its output can
> > > > further transformed into dense patches[2] on which schematic
> > > reconstruction
> > > > can be done[3]. See the video here: <
> > > > http://www.youtube.com/watch?v=UwSGtqUZx3M>. Developing this
> pipeline
> > > into
> > > > Blender will be a fun project.
> > > >
> > > > 2) An interactive tool for 3D modelling: A tool can be made
> > > > and integrated into blender as indicated in [4] which uses
> bundler's[1]
> > > > point cloud. They use plane estimation for modelling which is quite
> > > > accurate. They have presented a pretty neat and robust way of
> > estimating
> > > > planes using points clouds and vanishing points. See the video here:
> > > > http://www.youtube.com/watch?v=RhNNLxHwz6s.
> > > >
> > > > 3)Non-rigid reconstruction (motion capture) from set of static
> > > cameras:This
> > > > reconstruction is for slightly different domain, mainly motion
> capture.
> > > > This is actually one of the methods I have coauthored[5]. It requires
> > > > labelled track points in all the cameras feeds as input. The salient
> > > > feature of this method is that it can handle occlusions. This method
> > has
> > > > been extensively tested on CMU mocap database.
> > > >
> > > > Poject website: http://cvlab.lums.edu.pk/MultiviewNRSFM/
> > > > Video: http://www.youtube.com/watch?v=a1XUPI9L9vA
> > > >
> > > > Reconstruction from this method can be used in motion re-targeting
> like
> > > in
> > > > Avator and Pirates of the Caribbean. Including this into blender will
> > be
> > > a
> > > > great GSOC project.
> > > >
> > > > Kind Regards.
> > > > Shabbir Marzban,
> > > >
> > > > I am first year grad-student at School of Science and Engineering,
> > Lahore
> > > > University of Management Sciences, Pakistan. My resume and sample of
> > some
> > > > old projects can be seen here: http://shabbirm.weebly.com/index.html
> > > >
> > > >
> > > > [1]. Snavely, Noah, Steven M. Seitz, and Richard Szeliski. "Modeling
> > the
> > > > world from internet photo collections." *International Journal of
> > > Computer
> > > > Vision* 80.2 (2008): 189-210.
> > > >
> > > > [2]Furukawa, Yasutaka, et al. "Towards internet-scale multi-view
> > > > stereo."*Computer
> > > > Vision and Pattern Recognition (CVPR), 2010 IEEE Conference on*.
> IEEE,
> > > > 2010.
> > > >
> > > > [3] Wu, Changchang, et al. "Schematic surface reconstruction."
> > *Computer
> > > > Vision and Pattern Recognition (CVPR), 2012 IEEE Conference on*.
> IEEE,
> > > > 2012.
> > > >
> > > > [4] Sinha, Sudipta N., et al. "Interactive 3D architectural modeling
> > from
> > > > unordered photo collections." *ACM Transactions on Graphics (TOG)*.
> > Vol.
> > > > 27. No. 5. ACM, 2008.
> > > >
> > > > [5] Zaheer, Aamer, et al. "Multiview structure from motion in
> > trajectory
> > > > space."*Computer Vision (ICCV), 2011 IEEE International Conference
> on*.
> > > > IEEE, 2011.
> > > > _______________________________________________
> > > > Bf-committers mailing list
> > > > Bf-committers at blender.org
> > > > http://lists.blender.org/mailman/listinfo/bf-committers
> > > >
> > >
> > >
> > >
> > > --
> > > With best regards, Sergey Sharybin
> > > _______________________________________________
> > > Bf-committers mailing list
> > > Bf-committers at blender.org
> > > http://lists.blender.org/mailman/listinfo/bf-committers
> > >
> >
> >
> >
> > --
> >
> >
> >
> --------------------------------------------------------------------------
> > Jace Priester
> > Threespace Imaging
> > jacepriester at threespaceimaging.com
> > 559-284-0904
> >
> --------------------------------------------------------------------------
> > _______________________________________________
> > Bf-committers mailing list
> > Bf-committers at blender.org
> > http://lists.blender.org/mailman/listinfo/bf-committers
> >
>
>
>
> --
> With best regards, Sergey Sharybin
> _______________________________________________
> Bf-committers mailing list
> Bf-committers at blender.org
> http://lists.blender.org/mailman/listinfo/bf-committers
>

-- 

--------------------------------------------------------------------------
Jace Priester
Threespace Imaging
jacepriester at threespaceimaging.com
559-284-0904
--------------------------------------------------------------------------