[Bf-vfx] Footage for Tracking

Keir Mierle mierle at gmail.com
Sun Jun 12 03:04:36 CEST 2011 

Unfortunately these are not good enough. To get really solid calibration,
you need:

   1. A checkerboard (turns out it is easy to very precisely locate the
   corners)
   2. Several views of the calibration grid at different angles; the
   perspective on the checkerboard should be very apparent (in addition to
   front-on views).
   3. Each shot of the checkerboard should try to cover as much of the field
   of view of the camera as possible, while still showing the full checker
   board
   4. The checkerboard and camera should be still (no rolling shutter for
   calibration please!)

#3 is important since the distortion tends to happen towards the edges of
the view, so if the grid only occupies the middle 25% of the view (like in
your examples) it won't produce a good calibration. Even with the full
autotracking that you get in the commercial packages, you can get superior
results if you properly calibrate your camera; especially for difficult
shots where there is ambiguity, etc.

Do you know anyone who might be willing to take a few shots of a
checkerboard?

Keir

On Sat, Jun 11, 2011 at 5:54 PM, Colin Levy
<colin at peerlessproductions.com>wrote:

> Hey Kier,
>
> Unfortunately I no longer have access to the camera package, but while we
> were on set we shot some lens calibration reference.  Don't know if this is
> useful at all, but here's what we shot for the plate I named "LOBBY":
>
> http://www.colinlevy.com/upload/LensCalibration01.jpg
> http://www.colinlevy.com/upload/LensCalibration02.jpg
>
> Unfortunately that's all I seem to have for these three shots, and I don't
> know if they will do the trick.
>
> --Colin
>
> On Jun 11, 2011, at 8:23 PM, Keir Mierle wrote:
>
> Hi Colin,
>
> These look great. Can I convince you to take images of a calibration
> pattern with the lenses? We are not going to support uncalibrated tracking
> in the first release, so this is not as useful without calibration. Here's a
> calibration pattern I made for libmv:
>
> http://libmv.googlecode.com/svn/trunk/extras/calibration/calibration.pdf
>
> Just print it out and carefully affix it to something very, very flat (no
> bends or wrinkles when gluing please).
>
> Thanks,
> Keir
>
> On Sat, Jun 11, 2011 at 3:15 PM, Colin Levy <colin at peerlessproductions.com
> > wrote:
>
>> Hey guys,
>>
>> Wanted to let you know I've uploaded three shots which may hopefully be
>> useful to you.
>>
>> All three shots were shot on RED at 4K resolution for my film "En Route".
>>  All three have similar camera movement: a simple forward dolly push.  I've
>> uploaded JPG sequences for now, but can supply EXRs if requested.
>>
>> You can watch H264 previews of these shots here:
>>
>> http://www.colinlevy.com/upload/LOBBY_preview.mov
>> http://www.colinlevy.com/upload/FIELD_preview.mov
>> http://www.colinlevy.com/upload/WINDOW_preview.mov
>>
>> And you can log into the FTP space I'm using for a temporary project to
>> access the image sequences.  The folder is called "blender-vfx".
>>
>> Username: b173379
>> Password: bleem34
>> Server: hanjin.dreamhost.com
>> (Please keep this info private!)
>>
>> I can supply more varied material if you like as well.  Hope this helps!
>>
>> --Colin
>>
>>
>>
>> On Jun 9, 2011, at 8:33 PM, Keir Mierle wrote:
>>
>> On Thu, Jun 9, 2011 at 5:12 PM, Francesco Callari <fgcallari at gmail.com>wrote:
>>
>>> On Wed, Jun 8, 2011 at 2:17 PM, Keir Mierle <mierle at gmail.com> wrote:
>>> > On Wed, Jun 8, 2011 at 1:44 PM, François T. <francoistarlier at gmail.com
>>> >
>>> > wrote:
>>> >>>
>>> >>> Regarding file formats: JPEG frames are probably fine; there's little
>>> >>> value in tracking on 16-bit data. The frames can even get converted
>>> to
>>> >>> grayscale, since the tracking ignores color information.
>>> >>
>>> >>
>>> >> are you not planning on using all the depth on the image for tracking
>>> ? in
>>> >> my experience, it can helps a lot sometimes !
>>> >
>>> > Maybe eventually but not to start. Libmv takes 32 bit grayscale float
>>> images
>>> > to track them. However, I'm skeptical that more than 8 bits of
>>> grayscale
>>> > data would make a difference for tracking. Do you have examples where
>>> this
>>> > is the case?
>>>
>>> One data point. When I was at ILM nearly all tracking (and, in fact,
>>> nearly all the imagery pipeline save for final rendering) was done on
>>> 16bit OpenEXR float images (the "half" pixel format of OepnEXR). Of
>>> course the tracking code internally used higher precision for the
>>> arithmetics, and was independent of the image format. Generally
>>> speaking, OpenEXR is a really nice image library to work with in a
>>> graphics + vision environment.
>>>
>>
>> It sounds like supporting higher bit depth is useful. This shouldn't be
>> hard since libmv already operates on excessively precise 32 bit floats. It
>> will just be a matter of using the appropriate conversion to 32 bit floats
>> when extracting patches to track from the underlying footage.
>>
>> I believe Blender already supports OpenEXR throughout.
>>
>> Keir
>>
>>
>>>
>>> Franco
>>>
>>>
>>> > Keir
>>> >
>>> >>
>>> >> 2011/6/8 Keir Mierle <mierle at gmail.com>
>>> >>>
>>> >>> On Wed, Jun 8, 2011 at 11:53 AM, Colin Levy
>>> >>> <colin at peerlessproductions.com> wrote:
>>> >>>>
>>> >>>> Yeah, rolling shutter is an *incredible* pain.  It can be corrected
>>> to
>>> >>>> an extent with tools like The Foundry's "RollingShutter" plugin for
>>> Nuke,
>>> >>>> which I've heard can make previously un-trackable shots trackable,
>>> but you
>>> >>>> see it *everywhere* these days -- even in hollywood features.
>>> >>>
>>> >>> Yes, we will deal with it but it will take some time. It's
>>> mathematically
>>> >>> impossible to fix perfectly, so the best we can do is approximate
>>> fixes.
>>> >>> Truly unfortunate.
>>> >>>
>>> >>>>
>>> >>>> I do have some footage I can get you from my Canon XH-A1, which is a
>>> >>>> 3-CCD HDV camera.  I sold the camera recently (CMOS is all the
>>> rage!), but I
>>> >>>> have a ton of footage I can go through.
>>> >>>
>>> >>> That would be great! The only constraint is going to be calibration.
>>> Can
>>> >>> you get access to the camera for 10 minutes to take some shots of a
>>> >>> calibration pattern?
>>> >>> I realize the big tools deal with this automatically, but it's
>>> difficult
>>> >>> to do automatically and won't be supported in our first release.
>>> >>>
>>> >>>>
>>> >>>> That said, RED's CMOS sensor is waayy better than the "jellocam"
>>> sensors
>>> >>>> in DSLRs like the 5D or T2i.  (RED's new MX sensor I think is even
>>> better).
>>> >>>>  So the rolling shutter is not bad -- but it's still noticeable in
>>> very
>>> >>>> high-motion shots.
>>> >>>> I'd say 80% of the RED shots I've got should not pose issues due to
>>> >>>> rolling shutter.  The tracking tools I've been using do not account
>>> for
>>> >>>> rolling shutter, yet I'm able to get very solid tracks with little
>>> manual
>>> >>>> intervention.  So perhaps still valuable to test with?
>>> >>>
>>> >>> Sounds like the RED shots will work as long as the motion is not too
>>> >>> extreme.
>>> >>> Regarding file formats: JPEG frames are probably fine; there's little
>>> >>> value in tracking on 16-bit data. The frames can even get converted
>>> to
>>> >>> grayscale, since the tracking ignores color information.
>>> >>>
>>> >>> Keir
>>> >>>>
>>> >>>> --Colin
>>> >>>>
>>> >>>> On Jun 8, 2011, at 2:32 PM, Keir Mierle wrote:
>>> >>>>
>>> >>>> Thanks Colin!
>>> >>>> One point though: RED uses a CMOS sensor and is subject to rolling
>>> >>>> shutter. Do you know how bad it is? Some cameras have very slow
>>> sensor
>>> >>>> readout (and so terrible distortions with movement) and others have
>>> a faster
>>> >>>> rolling shutter which still has distortions, but less.
>>> >>>> For those who don't know: rolling shutter means that each scanline
>>> has a
>>> >>>> *different* shutter time! Yes, that means that the image is composed
>>> of
>>> >>>> 1000's of images, one scanline taken from each, with the camera
>>> moved
>>> >>>> between images. Extremely annoying, and totally breaks all the
>>> camera
>>> >>>> tracking assumptions.
>>> >>>> I realize that we have to handle rolling shutter but it's a big job
>>> and
>>> >>>> we won't support it in the first release.
>>> >>>> So I repeat my request: does anyone have a camera that doesn't have
>>> a
>>> >>>> rolling shutter? For example, the Panasonic SD9 has a CCD sensor and
>>> so
>>> >>>> takes true pinhole images for each frame.
>>> >>>> Virtually all modern DSLR's have awful rolling shutter (D90
>>> included),
>>> >>>> so they are also excluded.
>>> >>>> Keir
>>> >>>> On Wed, Jun 8, 2011 at 11:00 AM, Colin Levy
>>> >>>> <colin at peerlessproductions.com> wrote:
>>> >>>>>
>>> >>>>> Hey guys,
>>> >>>>> I've got several TB worth of 4K material I can offer for tracking.
>>> >>>>>  However, the shots will represent a more "real-world" scenario
>>> rather than
>>> >>>>> the test case shots listed below.
>>> >>>>> I don't suppose you want the RED .r3d files, so will TIFF sequences
>>> >>>>> work for you?  (Or do you want both?)  I will go through my drives
>>> in the
>>> >>>>> next few days and upload a few shots to my FTP, if you think this
>>> would be
>>> >>>>> useful!  Let me know whether you'd like 32-bit images, with all the
>>> 'RAW'
>>> >>>>> data intact, or if 8 or 16bit will suffice.
>>> >>>>> One thing to note regarding resolution: I have noticed that in many
>>> >>>>> cases, tracking downrezzed shots is frequently more successful than
>>> trying
>>> >>>>> to track the raw 4K material.  This may have something to do with
>>> the
>>> >>>>> default values (the default auto feature search range, for example,
>>> may be
>>> >>>>> something like 20px rather than 80px, which might be a better
>>> setting for
>>> >>>>> 4K).  For my projects I usually downrez to 2K before tracking...
>>> also
>>> >>>>> because it's a lot faster.  :P
>>> >>>>> Great discussion.  Can't wait to see these tools develop!
>>> >>>>> --Colin
>>> >>>>>
>>> >>>>> On Jun 8, 2011, at 12:59 PM, François T. wrote:
>>> >>>>>
>>> >>>>> I'll do all that on saturday with D90.
>>> >>>>> cheers
>>> >>>>> F.
>>> >>>>>
>>> >>>>> 2011/6/8 Keir Mierle <mierle at gmail.com>
>>> >>>>>>
>>> >>>>>> A catalog of the following shots would be useful:
>>> >>>>>>
>>> >>>>>> General motion: Moving and rotating camera. Camera is /not/ always
>>> >>>>>> pointing at one part of the scene.
>>> >>>>>> Rotation only: camera on a tripod, rotating around a fixed point
>>> (if
>>> >>>>>> possible the camera's "Nodal point", but you need a special tripod
>>> for that)
>>> >>>>>> Translation only: walking sideways keeping the camera pointed
>>> forward
>>> >>>>>> (e.g. not circling)
>>> >>>>>> Translation only: walking towards the scene
>>> >>>>>> General motion when the scene is planar (e.g. camera keeps a big
>>> >>>>>> billboard in view and only the billboard)
>>> >>>>>> Circle strafing around an object; camera is fixated on one part of
>>> the
>>> >>>>>> scene (intersecting principal ray)
>>> >>>>>>
>>> >>>>>> The scenes should have good texture so that tracking is easy; we
>>> can
>>> >>>>>> add challenging scenes later. It's ok if the translation-only
>>> scenes have
>>> >>>>>> some hand held wobble; that's fine. Boxes with texture are good.
>>> Complicated
>>> >>>>>> objects like bushes or trees are bad. The scene must be static (no
>>> cars,
>>> >>>>>> people, birds, godzilla, etc).
>>> >>>>>> The camera must be calibrated. There is a libmv calibration target
>>> >>>>>> that you can affix to a really flat object. The calibration target
>>> must be
>>> >>>>>> really flat; something that has a bit of bend is unacceptable and
>>> will give
>>> >>>>>> poor results.
>>> >>>>>>
>>> >>>>>>
>>> http://libmv.googlecode.com/svn/trunk/extras/calibration/calibration.pdf
>>> >>>>>> Take several images of the calibration target (or a video in HD).
>>> Get
>>> >>>>>> different rotations and fill the frame to get the edges (important
>>> for
>>> >>>>>> distortion correction). Then run one of the calibration tools to
>>> find the
>>> >>>>>> intrinsic parameters (focal length in pixels, distortion, skew,
>>> etc).
>>> >>>>>> Note: Rolling shutter is *horrible* for tracking. We're going to
>>> have
>>> >>>>>> to handle it at some point, but I am trying to avoid it as long as
>>> possible.
>>> >>>>>> Do you have access to a CCD style camera without a rolling
>>> shutter?
>>> >>>>>> Keir
>>> >>>>>>
>>> >>>>>> On Tue, Jun 7, 2011 at 3:27 PM, Ian Hubert
>>> >>>>>> <ian at projectlondonmovie.com> wrote:
>>> >>>>>>>
>>> >>>>>>> Hey all!
>>> >>>>>>> I've been following the conversations, and wanted to throw out
>>> that
>>> >>>>>>> if anyone needs any sort of footage, I can run out and film
>>> stuff. I already
>>> >>>>>>> have a few things things filmed- mostly just walking around the
>>> city holding
>>> >>>>>>> the camera as steady as I can- but if anyone wants anything else
>>> >>>>>>> (underexposed, really shaky, steady smooth,
>>> panning--but-no-movement,
>>> >>>>>>> somebody walking across the frame, stuff like that), just say so!
>>> >>>>>>> At this point all I can quickly provide is SD and HD-
>>> >>>>>>> I unfortunately don't have access to a 4k camera, but I'll film
>>> some random
>>> >>>>>>> stuff if I ever get my hands on one.
>>> >>>>>>>
>>> >>>>>>> --
>>> >>>>>>> Ian Hubert
>>> >>>>>>>
>>> >>>>>>> _______________________________________________
>>> >>>>>>> Bf-vfx mailing list
>>> >>>>>>> Bf-vfx at blender.org
>>> >>>>>>> http://lists.blender.org/mailman/listinfo/bf-vfx
>>> >>>>>>>
>>> >>>>>>
>>> >>>>>>
>>> >>>>>> _______________________________________________
>>> >>>>>> Bf-vfx mailing list
>>> >>>>>> Bf-vfx at blender.org
>>> >>>>>> http://lists.blender.org/mailman/listinfo/bf-vfx
>>> >>>>>>
>>> >>>>>
>>> >>>>>
>>> >>>>>
>>> >>>>> --
>>> >>>>> ____________________
>>> >>>>> François Tarlier
>>> >>>>> www.francois-tarlier.com
>>> >>>>> www.linkedin.com/in/francoistarlier
>>> >>>>> _______________________________________________
>>> >>>>> Bf-vfx mailing list
>>> >>>>> Bf-vfx at blender.org
>>> >>>>> http://lists.blender.org/mailman/listinfo/bf-vfx
>>> >>>>>
>>> >>>>>
>>> >>>>> _______________________________________________
>>> >>>>> Bf-vfx mailing list
>>> >>>>> Bf-vfx at blender.org
>>> >>>>> http://lists.blender.org/mailman/listinfo/bf-vfx
>>> >>>>>
>>> >>>>
>>> >>>> _______________________________________________
>>> >>>> Bf-vfx mailing list
>>> >>>> Bf-vfx at blender.org
>>> >>>> http://lists.blender.org/mailman/listinfo/bf-vfx
>>> >>>>
>>> >>>>
>>> >>>> _______________________________________________
>>> >>>> Bf-vfx mailing list
>>> >>>> Bf-vfx at blender.org
>>> >>>> http://lists.blender.org/mailman/listinfo/bf-vfx
>>> >>>>
>>> >>>
>>> >>>
>>> >>> _______________________________________________
>>> >>> Bf-vfx mailing list
>>> >>> Bf-vfx at blender.org
>>> >>> http://lists.blender.org/mailman/listinfo/bf-vfx
>>> >>>
>>> >>
>>> >>
>>> >>
>>> >> --
>>> >> ____________________
>>> >> François Tarlier
>>> >> www.francois-tarlier.com
>>> >> www.linkedin.com/in/francoistarlier
>>> >>
>>> >> _______________________________________________
>>> >> Bf-vfx mailing list
>>> >> Bf-vfx at blender.org
>>> >> http://lists.blender.org/mailman/listinfo/bf-vfx
>>> >>
>>> >
>>> >
>>> > _______________________________________________
>>> > Bf-vfx mailing list
>>> > Bf-vfx at blender.org
>>> > http://lists.blender.org/mailman/listinfo/bf-vfx
>>> >
>>> >
>>>
>>>
>>>
>>> --
>>> Franco Callari <fgcallari at gmail.com>
>>>
>>>             EC67 BEBE 62AC 8415 7591  2B12 A6CD D5EE D8CB D0ED
>>>
>>> I am not bound to win, but I am bound to be true. I am not bound to
>>> succeed, but I am bound to live by the light that I have. (Abraham
>>> Lincoln)
>>> _______________________________________________
>>> Bf-vfx mailing list
>>> Bf-vfx at blender.org
>>> http://lists.blender.org/mailman/listinfo/bf-vfx
>>>
>>
>> _______________________________________________
>> Bf-vfx mailing list
>> Bf-vfx at blender.org
>> http://lists.blender.org/mailman/listinfo/bf-vfx
>>
>>
>>
>> _______________________________________________
>> Bf-vfx mailing list
>> Bf-vfx at blender.org
>> http://lists.blender.org/mailman/listinfo/bf-vfx
>>
>>
> _______________________________________________
> Bf-vfx mailing list
> Bf-vfx at blender.org
> http://lists.blender.org/mailman/listinfo/bf-vfx
>
>
>
> _______________________________________________
> Bf-vfx mailing list
> Bf-vfx at blender.org
> http://lists.blender.org/mailman/listinfo/bf-vfx
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.blender.org/pipermail/bf-vfx/attachments/20110611/fd94be9e/attachment-0001.htm

More information about the Bf-vfx mailing list