[Bf-committers] GSOC Ideas: 3D Reconstruction

Tue Apr 30 22:47:25 CEST 2013

If it's done this way, i'm not so much fussed then :)

On Wed, May 1, 2013 at 2:28 AM, Jace Priester <
jacepriester at threespaceimaging.com> wrote:

> Custom data layers can be present, or not. The whole point is that they're
> optional. This means that the color/size CDLs will only be in use when
> colored point cloud data is imported. This means zero change and zero extra
> memory usage for all other meshes.
>
>
>
>
> On Tue, Apr 30, 2013 at 1:07 PM, Sergey Sharybin <sergey.vfx at gmail.com
> >wrote:
>
> > Per-vertex color and size is not something i'll want to be added to
> general
> > vertex. This will lead to much higher memory consumption of every mesh,
> and
> > this could become really an issue for modelers. That'd also lead to wish
> > being able to render such vertices as "strand"? And mixing "point cloud"
> > vertices with regular modelling.. Well, i'm skeptical about all this.
> >
> > Perhaps CD layers will solve memory issues, but eeh.. Really not sure.
> >
> >
> > On Wed, May 1, 2013 at 1:53 AM, Jace Priester <
> > jacepriester at threespaceimaging.com> wrote:
> >
> > > Meshes are very sufficient for handling point cloud data as I have
> > already
> > > discussed. They are a collection of vertices, stored efficiently and
> > > rendered to the viewport efficiently. They are simply lacking
> per-vertex
> > > color and size, which is what my code adds. There is no need for a new
> > > object type to accommodate point cloud data. That's actually a really
> > good
> > > thing, because all the existing mesh tools continue to work.
> > >
> > > I agree this all sounds like too big of a project. If there were a
> > library
> > > out there that did this type of work well and it could be integrated
> with
> > > Blender, that would be great, but there isn't. Bundler is great but
> it's
> > > only one step in the pipeline. That said, if this does end up getting
> > > underway I'm very interested in working on it with whoever is involved.
> > And
> > > like I said I'm working on putting together a patch for the little bit
> > that
> > > I have, and I'll get that part out in a couple days here.
> > >
> > >
> > > On Tue, Apr 30, 2013 at 12:41 PM, Sergey Sharybin <
> sergey.vfx at gmail.com
> > > >wrote:
> > >
> > > > Hi,
> > > >
> > > > Point cloud reconstruction could be quite a benefitial project for
> > > blender
> > > > if fully/properly done. There're basis stuff for this already present
> > in
> > > > libmv (in terms of reconstruction/bundling), would need to implement
> > > > correspondence matching tho and outlier filtering (which would also
> be
> > > > needed for automated tracking workflow). But this will also imply
> quite
> > > > enough of improvements in blender side -- being able to manipulate
> with
> > > > unordered image collections, point clouds structure (afraid mesh will
> > be
> > > > too much expensive for this), tools to deal with point clouds
> (surface
> > > > reconstruction/optimization/texturing).. This kind of glues two your
> > > > proposals into one.
> > > >
> > > > Ends up in quite bigger project, but think delivering new tools to
> > > artists
> > > > is more priority than adding extra ground for them when speaking of
> > GSoC
> > > > project. So think better thing would be to first aim to surface
> > > > reconstruction tools, texturing, orientation  (bundles could be
> > obtained
> > > > from several cameras) and so on, and then going to "depths" in terms
> > > adding
> > > > SfM from unordered image sounds more awesome. :)
> > > >
> > > > Motion capture i'm not sure. This is rather challenging project for
> > nice
> > > > integration into blender/libmv, not sure GSoC's time frame is enough
> > for
> > > > this?
> > > >
> > > >
> > > > On Fri, Apr 26, 2013 at 8:57 PM, Shabbir Marzban <
> mshabbir66 at gmail.com
> > > > >wrote:
> > > >
> > > > > Hi,
> > > > > I wish to participate in upcoming GSOC. I want to discuss couple of
> > > > ideas.
> > > > > It would be great if I can have some feedback on these :).
> > > > >
> > > > > I have described these briefly below. Kindly let me know if further
> > > > > details/ clarifications are required.
> > > > >
> > > > > 1) Automated rigid structure reconstruction from set of un-ordered
> > > > images:
> > > > > A very common structure from motion (SFM) tool, Bundler[1] can be
> > used
> > > to
> > > > > recover 3D point cloud from such images. Its output can
> > > > > further transformed into dense patches[2] on which schematic
> > > > reconstruction
> > > > > can be done[3]. See the video here: <
> > > > > http://www.youtube.com/watch?v=UwSGtqUZx3M>. Developing this
> > pipeline
> > > > into
> > > > > Blender will be a fun project.
> > > > >
> > > > > 2) An interactive tool for 3D modelling: A tool can be made
> > > > > and integrated into blender as indicated in [4] which uses
> > bundler's[1]
> > > > > point cloud. They use plane estimation for modelling which is quite
> > > > > accurate. They have presented a pretty neat and robust way of
> > > estimating
> > > > > planes using points clouds and vanishing points. See the video
> here:
> > > > > http://www.youtube.com/watch?v=RhNNLxHwz6s.
> > > > >
> > > > > 3)Non-rigid reconstruction (motion capture) from set of static
> > > > cameras:This
> > > > > reconstruction is for slightly different domain, mainly motion
> > capture.
> > > > > This is actually one of the methods I have coauthored[5]. It
> requires
> > > > > labelled track points in all the cameras feeds as input. The
> salient
> > > > > feature of this method is that it can handle occlusions. This
> method
> > > has
> > > > > been extensively tested on CMU mocap database.
> > > > >
> > > > > Poject website: http://cvlab.lums.edu.pk/MultiviewNRSFM/
> > > > > Video: http://www.youtube.com/watch?v=a1XUPI9L9vA
> > > > >
> > > > > Reconstruction from this method can be used in motion re-targeting
> > like
> > > > in
> > > > > Avator and Pirates of the Caribbean. Including this into blender
> will
> > > be
> > > > a
> > > > > great GSOC project.
> > > > >
> > > > > Kind Regards.
> > > > > Shabbir Marzban,
> > > > >
> > > > > I am first year grad-student at School of Science and Engineering,
> > > Lahore
> > > > > University of Management Sciences, Pakistan. My resume and sample
> of
> > > some
> > > > > old projects can be seen here:
> http://shabbirm.weebly.com/index.html
> > > > >
> > > > >
> > > > > [1]. Snavely, Noah, Steven M. Seitz, and Richard Szeliski.
> "Modeling
> > > the
> > > > > world from internet photo collections." *International Journal of
> > > > Computer
> > > > > Vision* 80.2 (2008): 189-210.
> > > > >
> > > > > [2]Furukawa, Yasutaka, et al. "Towards internet-scale multi-view
> > > > > stereo."*Computer
> > > > > Vision and Pattern Recognition (CVPR), 2010 IEEE Conference on*.
> > IEEE,
> > > > > 2010.
> > > > >
> > > > > [3] Wu, Changchang, et al. "Schematic surface reconstruction."
> > > *Computer
> > > > > Vision and Pattern Recognition (CVPR), 2012 IEEE Conference on*.
> > IEEE,
> > > > > 2012.
> > > > >
> > > > > [4] Sinha, Sudipta N., et al. "Interactive 3D architectural
> modeling
> > > from
> > > > > unordered photo collections." *ACM Transactions on Graphics (TOG)*.
> > > Vol.
> > > > > 27. No. 5. ACM, 2008.
> > > > >
> > > > > [5] Zaheer, Aamer, et al. "Multiview structure from motion in
> > > trajectory
> > > > > space."*Computer Vision (ICCV), 2011 IEEE International Conference
> > on*.
> > > > > IEEE, 2011.
> > > > > _______________________________________________
> > > > > Bf-committers mailing list
> > > > > Bf-committers at blender.org
> > > > > http://lists.blender.org/mailman/listinfo/bf-committers
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > With best regards, Sergey Sharybin
> > > > _______________________________________________
> > > > Bf-committers mailing list
> > > > Bf-committers at blender.org
> > > > http://lists.blender.org/mailman/listinfo/bf-committers
> > > >
> > >
> > >
> > >
> > > --
> > >
> > >
> > >
> >
> --------------------------------------------------------------------------
> > > Jace Priester
> > > Threespace Imaging
> > > jacepriester at threespaceimaging.com
> > > 559-284-0904
> > >
> >
> --------------------------------------------------------------------------
> > > _______________________________________________
> > > Bf-committers mailing list
> > > Bf-committers at blender.org
> > > http://lists.blender.org/mailman/listinfo/bf-committers
> > >
> >
> >
> >
> > --
> > With best regards, Sergey Sharybin
> > _______________________________________________
> > Bf-committers mailing list
> > Bf-committers at blender.org
> > http://lists.blender.org/mailman/listinfo/bf-committers
> >
>
>
>
> --
>
>
> --------------------------------------------------------------------------
> Jace Priester
> Threespace Imaging
> jacepriester at threespaceimaging.com
> 559-284-0904
> --------------------------------------------------------------------------
> _______________________________________________
> Bf-committers mailing list
> Bf-committers at blender.org
> http://lists.blender.org/mailman/listinfo/bf-committers
>

-- 
With best regards, Sergey Sharybin