[Bf-vfx] Footage for Tracking

Fri Jun 10 02:33:39 CEST 2011

On Thu, Jun 9, 2011 at 5:12 PM, Francesco Callari <fgcallari at gmail.com>wrote:

> On Wed, Jun 8, 2011 at 2:17 PM, Keir Mierle <mierle at gmail.com> wrote:
> > On Wed, Jun 8, 2011 at 1:44 PM, François T. <francoistarlier at gmail.com>
> > wrote:
> >>>
> >>> Regarding file formats: JPEG frames are probably fine; there's little
> >>> value in tracking on 16-bit data. The frames can even get converted to
> >>> grayscale, since the tracking ignores color information.
> >>
> >>
> >> are you not planning on using all the depth on the image for tracking ?
> in
> >> my experience, it can helps a lot sometimes !
> >
> > Maybe eventually but not to start. Libmv takes 32 bit grayscale float
> images
> > to track them. However, I'm skeptical that more than 8 bits of grayscale
> > data would make a difference for tracking. Do you have examples where
> this
> > is the case?
>
> One data point. When I was at ILM nearly all tracking (and, in fact,
> nearly all the imagery pipeline save for final rendering) was done on
> 16bit OpenEXR float images (the "half" pixel format of OepnEXR). Of
> course the tracking code internally used higher precision for the
> arithmetics, and was independent of the image format. Generally
> speaking, OpenEXR is a really nice image library to work with in a
> graphics + vision environment.
>

It sounds like supporting higher bit depth is useful. This shouldn't be hard
since libmv already operates on excessively precise 32 bit floats. It will
just be a matter of using the appropriate conversion to 32 bit floats when
extracting patches to track from the underlying footage.

I believe Blender already supports OpenEXR throughout.

Keir

>
> Franco
>
>
> > Keir
> >
> >>
> >> 2011/6/8 Keir Mierle <mierle at gmail.com>
> >>>
> >>> On Wed, Jun 8, 2011 at 11:53 AM, Colin Levy
> >>> <colin at peerlessproductions.com> wrote:
> >>>>
> >>>> Yeah, rolling shutter is an *incredible* pain.  It can be corrected to
> >>>> an extent with tools like The Foundry's "RollingShutter" plugin for
> Nuke,
> >>>> which I've heard can make previously un-trackable shots trackable, but
> you
> >>>> see it *everywhere* these days -- even in hollywood features.
> >>>
> >>> Yes, we will deal with it but it will take some time. It's
> mathematically
> >>> impossible to fix perfectly, so the best we can do is approximate
> fixes.
> >>> Truly unfortunate.
> >>>
> >>>>
> >>>> I do have some footage I can get you from my Canon XH-A1, which is a
> >>>> 3-CCD HDV camera.  I sold the camera recently (CMOS is all the rage!),
> but I
> >>>> have a ton of footage I can go through.
> >>>
> >>> That would be great! The only constraint is going to be calibration.
> Can
> >>> you get access to the camera for 10 minutes to take some shots of a
> >>> calibration pattern?
> >>> I realize the big tools deal with this automatically, but it's
> difficult
> >>> to do automatically and won't be supported in our first release.
> >>>
> >>>>
> >>>> That said, RED's CMOS sensor is waayy better than the "jellocam"
> sensors
> >>>> in DSLRs like the 5D or T2i.  (RED's new MX sensor I think is even
> better).
> >>>>  So the rolling shutter is not bad -- but it's still noticeable in
> very
> >>>> high-motion shots.
> >>>> I'd say 80% of the RED shots I've got should not pose issues due to
> >>>> rolling shutter.  The tracking tools I've been using do not account
> for
> >>>> rolling shutter, yet I'm able to get very solid tracks with little
> manual
> >>>> intervention.  So perhaps still valuable to test with?
> >>>
> >>> Sounds like the RED shots will work as long as the motion is not too
> >>> extreme.
> >>> Regarding file formats: JPEG frames are probably fine; there's little
> >>> value in tracking on 16-bit data. The frames can even get converted to
> >>> grayscale, since the tracking ignores color information.
> >>>
> >>> Keir
> >>>>
> >>>> --Colin
> >>>>
> >>>> On Jun 8, 2011, at 2:32 PM, Keir Mierle wrote:
> >>>>
> >>>> Thanks Colin!
> >>>> One point though: RED uses a CMOS sensor and is subject to rolling
> >>>> shutter. Do you know how bad it is? Some cameras have very slow sensor
> >>>> readout (and so terrible distortions with movement) and others have a
> faster
> >>>> rolling shutter which still has distortions, but less.
> >>>> For those who don't know: rolling shutter means that each scanline has
> a
> >>>> *different* shutter time! Yes, that means that the image is composed
> of
> >>>> 1000's of images, one scanline taken from each, with the camera moved
> >>>> between images. Extremely annoying, and totally breaks all the camera
> >>>> tracking assumptions.
> >>>> I realize that we have to handle rolling shutter but it's a big job
> and
> >>>> we won't support it in the first release.
> >>>> So I repeat my request: does anyone have a camera that doesn't have a
> >>>> rolling shutter? For example, the Panasonic SD9 has a CCD sensor and
> so
> >>>> takes true pinhole images for each frame.
> >>>> Virtually all modern DSLR's have awful rolling shutter (D90 included),
> >>>> so they are also excluded.
> >>>> Keir
> >>>> On Wed, Jun 8, 2011 at 11:00 AM, Colin Levy
> >>>> <colin at peerlessproductions.com> wrote:
> >>>>>
> >>>>> Hey guys,
> >>>>> I've got several TB worth of 4K material I can offer for tracking.
> >>>>>  However, the shots will represent a more "real-world" scenario
> rather than
> >>>>> the test case shots listed below.
> >>>>> I don't suppose you want the RED .r3d files, so will TIFF sequences
> >>>>> work for you?  (Or do you want both?)  I will go through my drives in
> the
> >>>>> next few days and upload a few shots to my FTP, if you think this
> would be
> >>>>> useful!  Let me know whether you'd like 32-bit images, with all the
> 'RAW'
> >>>>> data intact, or if 8 or 16bit will suffice.
> >>>>> One thing to note regarding resolution: I have noticed that in many
> >>>>> cases, tracking downrezzed shots is frequently more successful than
> trying
> >>>>> to track the raw 4K material.  This may have something to do with the
> >>>>> default values (the default auto feature search range, for example,
> may be
> >>>>> something like 20px rather than 80px, which might be a better setting
> for
> >>>>> 4K).  For my projects I usually downrez to 2K before tracking... also
> >>>>> because it's a lot faster.  :P
> >>>>> Great discussion.  Can't wait to see these tools develop!
> >>>>> --Colin
> >>>>>
> >>>>> On Jun 8, 2011, at 12:59 PM, François T. wrote:
> >>>>>
> >>>>> I'll do all that on saturday with D90.
> >>>>> cheers
> >>>>> F.
> >>>>>
> >>>>> 2011/6/8 Keir Mierle <mierle at gmail.com>
> >>>>>>
> >>>>>> A catalog of the following shots would be useful:
> >>>>>>
> >>>>>> General motion: Moving and rotating camera. Camera is /not/ always
> >>>>>> pointing at one part of the scene.
> >>>>>> Rotation only: camera on a tripod, rotating around a fixed point (if
> >>>>>> possible the camera's "Nodal point", but you need a special tripod
> for that)
> >>>>>> Translation only: walking sideways keeping the camera pointed
> forward
> >>>>>> (e.g. not circling)
> >>>>>> Translation only: walking towards the scene
> >>>>>> General motion when the scene is planar (e.g. camera keeps a big
> >>>>>> billboard in view and only the billboard)
> >>>>>> Circle strafing around an object; camera is fixated on one part of
> the
> >>>>>> scene (intersecting principal ray)
> >>>>>>
> >>>>>> The scenes should have good texture so that tracking is easy; we can
> >>>>>> add challenging scenes later. It's ok if the translation-only scenes
> have
> >>>>>> some hand held wobble; that's fine. Boxes with texture are good.
> Complicated
> >>>>>> objects like bushes or trees are bad. The scene must be static (no
> cars,
> >>>>>> people, birds, godzilla, etc).
> >>>>>> The camera must be calibrated. There is a libmv calibration target
> >>>>>> that you can affix to a really flat object. The calibration target
> must be
> >>>>>> really flat; something that has a bit of bend is unacceptable and
> will give
> >>>>>> poor results.
> >>>>>>
> >>>>>>
> http://libmv.googlecode.com/svn/trunk/extras/calibration/calibration.pdf
> >>>>>> Take several images of the calibration target (or a video in HD).
> Get
> >>>>>> different rotations and fill the frame to get the edges (important
> for
> >>>>>> distortion correction). Then run one of the calibration tools to
> find the
> >>>>>> intrinsic parameters (focal length in pixels, distortion, skew,
> etc).
> >>>>>> Note: Rolling shutter is *horrible* for tracking. We're going to
> have
> >>>>>> to handle it at some point, but I am trying to avoid it as long as
> possible.
> >>>>>> Do you have access to a CCD style camera without a rolling shutter?
> >>>>>> Keir
> >>>>>>
> >>>>>> On Tue, Jun 7, 2011 at 3:27 PM, Ian Hubert
> >>>>>> <ian at projectlondonmovie.com> wrote:
> >>>>>>>
> >>>>>>> Hey all!
> >>>>>>> I've been following the conversations, and wanted to throw out that
> >>>>>>> if anyone needs any sort of footage, I can run out and film stuff.
> I already
> >>>>>>> have a few things things filmed- mostly just walking around the
> city holding
> >>>>>>> the camera as steady as I can- but if anyone wants anything else
> >>>>>>> (underexposed, really shaky, steady smooth,
> panning--but-no-movement,
> >>>>>>> somebody walking across the frame, stuff like that), just say so!
> >>>>>>> At this point all I can quickly provide is SD and HD-
> >>>>>>> I unfortunately don't have access to a 4k camera, but I'll film
> some random
> >>>>>>> stuff if I ever get my hands on one.
> >>>>>>>
> >>>>>>> --
> >>>>>>> Ian Hubert
> >>>>>>>
> >>>>>>> _______________________________________________
> >>>>>>> Bf-vfx mailing list
> >>>>>>> Bf-vfx at blender.org
> >>>>>>> http://lists.blender.org/mailman/listinfo/bf-vfx
> >>>>>>>
> >>>>>>
> >>>>>>
> >>>>>> _______________________________________________
> >>>>>> Bf-vfx mailing list
> >>>>>> Bf-vfx at blender.org
> >>>>>> http://lists.blender.org/mailman/listinfo/bf-vfx
> >>>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>> --
> >>>>> ____________________
> >>>>> François Tarlier
> >>>>> www.francois-tarlier.com
> >>>>> www.linkedin.com/in/francoistarlier
> >>>>> _______________________________________________
> >>>>> Bf-vfx mailing list
> >>>>> Bf-vfx at blender.org
> >>>>> http://lists.blender.org/mailman/listinfo/bf-vfx
> >>>>>
> >>>>>
> >>>>> _______________________________________________
> >>>>> Bf-vfx mailing list
> >>>>> Bf-vfx at blender.org
> >>>>> http://lists.blender.org/mailman/listinfo/bf-vfx
> >>>>>
> >>>>
> >>>> _______________________________________________
> >>>> Bf-vfx mailing list
> >>>> Bf-vfx at blender.org
> >>>> http://lists.blender.org/mailman/listinfo/bf-vfx
> >>>>
> >>>>
> >>>> _______________________________________________
> >>>> Bf-vfx mailing list
> >>>> Bf-vfx at blender.org
> >>>> http://lists.blender.org/mailman/listinfo/bf-vfx
> >>>>
> >>>
> >>>
> >>> _______________________________________________
> >>> Bf-vfx mailing list
> >>> Bf-vfx at blender.org
> >>> http://lists.blender.org/mailman/listinfo/bf-vfx
> >>>
> >>
> >>
> >>
> >> --
> >> ____________________
> >> François Tarlier
> >> www.francois-tarlier.com
> >> www.linkedin.com/in/francoistarlier
> >>
> >> _______________________________________________
> >> Bf-vfx mailing list
> >> Bf-vfx at blender.org
> >> http://lists.blender.org/mailman/listinfo/bf-vfx
> >>
> >
> >
> > _______________________________________________
> > Bf-vfx mailing list
> > Bf-vfx at blender.org
> > http://lists.blender.org/mailman/listinfo/bf-vfx
> >
> >
>
>
>
> --
> Franco Callari <fgcallari at gmail.com>
>
>             EC67 BEBE 62AC 8415 7591  2B12 A6CD D5EE D8CB D0ED
>
> I am not bound to win, but I am bound to be true. I am not bound to
> succeed, but I am bound to live by the light that I have. (Abraham
> Lincoln)
> _______________________________________________
> Bf-vfx mailing list
> Bf-vfx at blender.org
> http://lists.blender.org/mailman/listinfo/bf-vfx
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.blender.org/pipermail/bf-vfx/attachments/20110609/7204e587/attachment-0001.htm 