[Bf-vfx] Footage for Tracking

Keir Mierle mierle at gmail.com
Sun Jun 12 11:37:15 CEST 2011 

Typical lenses have the following parameters:

   - focal length x
   - focal length y
   - principal point x  (center of projection)
   - principal point y
   - skew
   - radial distortion coefficients which describe barrel, pincushion, or
   tangential distortion; typically ~1-5 numbers.

Keir

On Sun, Jun 12, 2011 at 1:40 AM, François T. <francoistarlier at gmail.com>wrote:

> what is this calibrating ? lens distortion ? or lens millimeter ?
>
>
>
> 2011/6/12 Keir Mierle <mierle at gmail.com>
>
>> Unfortunately these are not good enough. To get really solid calibration,
>> you need:
>>
>>    1. A checkerboard (turns out it is easy to very precisely locate the
>>    corners)
>>    2. Several views of the calibration grid at different angles; the
>>    perspective on the checkerboard should be very apparent (in addition to
>>    front-on views).
>>    3. Each shot of the checkerboard should try to cover as much of the
>>    field of view of the camera as possible, while still showing the full
>>    checker board
>>    4. The checkerboard and camera should be still (no rolling shutter for
>>    calibration please!)
>>
>> #3 is important since the distortion tends to happen towards the edges of
>> the view, so if the grid only occupies the middle 25% of the view (like in
>> your examples) it won't produce a good calibration. Even with the full
>> autotracking that you get in the commercial packages, you can get superior
>> results if you properly calibrate your camera; especially for difficult
>> shots where there is ambiguity, etc.
>>
>> Do you know anyone who might be willing to take a few shots of a
>> checkerboard?
>>
>> Keir
>>
>> On Sat, Jun 11, 2011 at 5:54 PM, Colin Levy <
>> colin at peerlessproductions.com> wrote:
>>
>>> Hey Kier,
>>>
>>> Unfortunately I no longer have access to the camera package, but while we
>>> were on set we shot some lens calibration reference.  Don't know if this is
>>> useful at all, but here's what we shot for the plate I named "LOBBY":
>>>
>>> http://www.colinlevy.com/upload/LensCalibration01.jpg
>>> http://www.colinlevy.com/upload/LensCalibration02.jpg
>>>
>>> Unfortunately that's all I seem to have for these three shots, and I
>>> don't know if they will do the trick.
>>>
>>> --Colin
>>>
>>> On Jun 11, 2011, at 8:23 PM, Keir Mierle wrote:
>>>
>>> Hi Colin,
>>>
>>> These look great. Can I convince you to take images of a calibration
>>> pattern with the lenses? We are not going to support uncalibrated tracking
>>> in the first release, so this is not as useful without calibration. Here's a
>>> calibration pattern I made for libmv:
>>>
>>> http://libmv.googlecode.com/svn/trunk/extras/calibration/calibration.pdf
>>>
>>> Just print it out and carefully affix it to something very, very flat (no
>>> bends or wrinkles when gluing please).
>>>
>>> Thanks,
>>> Keir
>>>
>>> On Sat, Jun 11, 2011 at 3:15 PM, Colin Levy <
>>> colin at peerlessproductions.com> wrote:
>>>
>>>> Hey guys,
>>>>
>>>> Wanted to let you know I've uploaded three shots which may hopefully be
>>>> useful to you.
>>>>
>>>> All three shots were shot on RED at 4K resolution for my film "En
>>>> Route".  All three have similar camera movement: a simple forward dolly
>>>> push.  I've uploaded JPG sequences for now, but can supply EXRs if
>>>> requested.
>>>>
>>>> You can watch H264 previews of these shots here:
>>>>
>>>> http://www.colinlevy.com/upload/LOBBY_preview.mov
>>>> http://www.colinlevy.com/upload/FIELD_preview.mov
>>>> http://www.colinlevy.com/upload/WINDOW_preview.mov
>>>>
>>>> And you can log into the FTP space I'm using for a temporary project to
>>>> access the image sequences.  The folder is called "blender-vfx".
>>>>
>>>> Username: b173379
>>>> Password: bleem34
>>>> Server: hanjin.dreamhost.com
>>>> (Please keep this info private!)
>>>>
>>>> I can supply more varied material if you like as well.  Hope this helps!
>>>>
>>>> --Colin
>>>>
>>>>
>>>>
>>>> On Jun 9, 2011, at 8:33 PM, Keir Mierle wrote:
>>>>
>>>> On Thu, Jun 9, 2011 at 5:12 PM, Francesco Callari <fgcallari at gmail.com>wrote:
>>>>
>>>>> On Wed, Jun 8, 2011 at 2:17 PM, Keir Mierle <mierle at gmail.com> wrote:
>>>>> > On Wed, Jun 8, 2011 at 1:44 PM, François T. <
>>>>> francoistarlier at gmail.com>
>>>>> > wrote:
>>>>> >>>
>>>>> >>> Regarding file formats: JPEG frames are probably fine; there's
>>>>> little
>>>>> >>> value in tracking on 16-bit data. The frames can even get converted
>>>>> to
>>>>> >>> grayscale, since the tracking ignores color information.
>>>>> >>
>>>>> >>
>>>>> >> are you not planning on using all the depth on the image for
>>>>> tracking ? in
>>>>> >> my experience, it can helps a lot sometimes !
>>>>> >
>>>>> > Maybe eventually but not to start. Libmv takes 32 bit grayscale float
>>>>> images
>>>>> > to track them. However, I'm skeptical that more than 8 bits of
>>>>> grayscale
>>>>> > data would make a difference for tracking. Do you have examples where
>>>>> this
>>>>> > is the case?
>>>>>
>>>>> One data point. When I was at ILM nearly all tracking (and, in fact,
>>>>> nearly all the imagery pipeline save for final rendering) was done on
>>>>> 16bit OpenEXR float images (the "half" pixel format of OepnEXR). Of
>>>>> course the tracking code internally used higher precision for the
>>>>> arithmetics, and was independent of the image format. Generally
>>>>> speaking, OpenEXR is a really nice image library to work with in a
>>>>> graphics + vision environment.
>>>>>
>>>>
>>>> It sounds like supporting higher bit depth is useful. This shouldn't be
>>>> hard since libmv already operates on excessively precise 32 bit floats. It
>>>> will just be a matter of using the appropriate conversion to 32 bit floats
>>>> when extracting patches to track from the underlying footage.
>>>>
>>>> I believe Blender already supports OpenEXR throughout.
>>>>
>>>> Keir
>>>>
>>>>
>>>>>
>>>>> Franco
>>>>>
>>>>>
>>>>> > Keir
>>>>> >
>>>>> >>
>>>>> >> 2011/6/8 Keir Mierle <mierle at gmail.com>
>>>>> >>>
>>>>> >>> On Wed, Jun 8, 2011 at 11:53 AM, Colin Levy
>>>>> >>> <colin at peerlessproductions.com> wrote:
>>>>> >>>>
>>>>> >>>> Yeah, rolling shutter is an *incredible* pain.  It can be
>>>>> corrected to
>>>>> >>>> an extent with tools like The Foundry's "RollingShutter" plugin
>>>>> for Nuke,
>>>>> >>>> which I've heard can make previously un-trackable shots trackable,
>>>>> but you
>>>>> >>>> see it *everywhere* these days -- even in hollywood features.
>>>>> >>>
>>>>> >>> Yes, we will deal with it but it will take some time. It's
>>>>> mathematically
>>>>> >>> impossible to fix perfectly, so the best we can do is approximate
>>>>> fixes.
>>>>> >>> Truly unfortunate.
>>>>> >>>
>>>>> >>>>
>>>>> >>>> I do have some footage I can get you from my Canon XH-A1, which is
>>>>> a
>>>>> >>>> 3-CCD HDV camera.  I sold the camera recently (CMOS is all the
>>>>> rage!), but I
>>>>> >>>> have a ton of footage I can go through.
>>>>> >>>
>>>>> >>> That would be great! The only constraint is going to be
>>>>> calibration. Can
>>>>> >>> you get access to the camera for 10 minutes to take some shots of a
>>>>> >>> calibration pattern?
>>>>> >>> I realize the big tools deal with this automatically, but it's
>>>>> difficult
>>>>> >>> to do automatically and won't be supported in our first release.
>>>>> >>>
>>>>> >>>>
>>>>> >>>> That said, RED's CMOS sensor is waayy better than the "jellocam"
>>>>> sensors
>>>>> >>>> in DSLRs like the 5D or T2i.  (RED's new MX sensor I think is even
>>>>> better).
>>>>> >>>>  So the rolling shutter is not bad -- but it's still noticeable in
>>>>> very
>>>>> >>>> high-motion shots.
>>>>> >>>> I'd say 80% of the RED shots I've got should not pose issues due
>>>>> to
>>>>> >>>> rolling shutter.  The tracking tools I've been using do not
>>>>> account for
>>>>> >>>> rolling shutter, yet I'm able to get very solid tracks with little
>>>>> manual
>>>>> >>>> intervention.  So perhaps still valuable to test with?
>>>>> >>>
>>>>> >>> Sounds like the RED shots will work as long as the motion is not
>>>>> too
>>>>> >>> extreme.
>>>>> >>> Regarding file formats: JPEG frames are probably fine; there's
>>>>> little
>>>>> >>> value in tracking on 16-bit data. The frames can even get converted
>>>>> to
>>>>> >>> grayscale, since the tracking ignores color information.
>>>>> >>>
>>>>> >>> Keir
>>>>> >>>>
>>>>> >>>> --Colin
>>>>> >>>>
>>>>> >>>> On Jun 8, 2011, at 2:32 PM, Keir Mierle wrote:
>>>>> >>>>
>>>>> >>>> Thanks Colin!
>>>>> >>>> One point though: RED uses a CMOS sensor and is subject to rolling
>>>>> >>>> shutter. Do you know how bad it is? Some cameras have very slow
>>>>> sensor
>>>>> >>>> readout (and so terrible distortions with movement) and others
>>>>> have a faster
>>>>> >>>> rolling shutter which still has distortions, but less.
>>>>> >>>> For those who don't know: rolling shutter means that each scanline
>>>>> has a
>>>>> >>>> *different* shutter time! Yes, that means that the image is
>>>>> composed of
>>>>> >>>> 1000's of images, one scanline taken from each, with the camera
>>>>> moved
>>>>> >>>> between images. Extremely annoying, and totally breaks all the
>>>>> camera
>>>>> >>>> tracking assumptions.
>>>>> >>>> I realize that we have to handle rolling shutter but it's a big
>>>>> job and
>>>>> >>>> we won't support it in the first release.
>>>>> >>>> So I repeat my request: does anyone have a camera that doesn't
>>>>> have a
>>>>> >>>> rolling shutter? For example, the Panasonic SD9 has a CCD sensor
>>>>> and so
>>>>> >>>> takes true pinhole images for each frame.
>>>>> >>>> Virtually all modern DSLR's have awful rolling shutter (D90
>>>>> included),
>>>>> >>>> so they are also excluded.
>>>>> >>>> Keir
>>>>> >>>> On Wed, Jun 8, 2011 at 11:00 AM, Colin Levy
>>>>> >>>> <colin at peerlessproductions.com> wrote:
>>>>> >>>>>
>>>>> >>>>> Hey guys,
>>>>> >>>>> I've got several TB worth of 4K material I can offer for
>>>>> tracking.
>>>>> >>>>>  However, the shots will represent a more "real-world" scenario
>>>>> rather than
>>>>> >>>>> the test case shots listed below.
>>>>> >>>>> I don't suppose you want the RED .r3d files, so will TIFF
>>>>> sequences
>>>>> >>>>> work for you?  (Or do you want both?)  I will go through my
>>>>> drives in the
>>>>> >>>>> next few days and upload a few shots to my FTP, if you think this
>>>>> would be
>>>>> >>>>> useful!  Let me know whether you'd like 32-bit images, with all
>>>>> the 'RAW'
>>>>> >>>>> data intact, or if 8 or 16bit will suffice.
>>>>> >>>>> One thing to note regarding resolution: I have noticed that in
>>>>> many
>>>>> >>>>> cases, tracking downrezzed shots is frequently more successful
>>>>> than trying
>>>>> >>>>> to track the raw 4K material.  This may have something to do with
>>>>> the
>>>>> >>>>> default values (the default auto feature search range, for
>>>>> example, may be
>>>>> >>>>> something like 20px rather than 80px, which might be a better
>>>>> setting for
>>>>> >>>>> 4K).  For my projects I usually downrez to 2K before tracking...
>>>>> also
>>>>> >>>>> because it's a lot faster.  :P
>>>>> >>>>> Great discussion.  Can't wait to see these tools develop!
>>>>> >>>>> --Colin
>>>>> >>>>>
>>>>> >>>>> On Jun 8, 2011, at 12:59 PM, François T. wrote:
>>>>> >>>>>
>>>>> >>>>> I'll do all that on saturday with D90.
>>>>> >>>>> cheers
>>>>> >>>>> F.
>>>>> >>>>>
>>>>> >>>>> 2011/6/8 Keir Mierle <mierle at gmail.com>
>>>>> >>>>>>
>>>>> >>>>>> A catalog of the following shots would be useful:
>>>>> >>>>>>
>>>>> >>>>>> General motion: Moving and rotating camera. Camera is /not/
>>>>> always
>>>>> >>>>>> pointing at one part of the scene.
>>>>> >>>>>> Rotation only: camera on a tripod, rotating around a fixed point
>>>>> (if
>>>>> >>>>>> possible the camera's "Nodal point", but you need a special
>>>>> tripod for that)
>>>>> >>>>>> Translation only: walking sideways keeping the camera pointed
>>>>> forward
>>>>> >>>>>> (e.g. not circling)
>>>>> >>>>>> Translation only: walking towards the scene
>>>>> >>>>>> General motion when the scene is planar (e.g. camera keeps a big
>>>>> >>>>>> billboard in view and only the billboard)
>>>>> >>>>>> Circle strafing around an object; camera is fixated on one part
>>>>> of the
>>>>> >>>>>> scene (intersecting principal ray)
>>>>> >>>>>>
>>>>> >>>>>> The scenes should have good texture so that tracking is easy; we
>>>>> can
>>>>> >>>>>> add challenging scenes later. It's ok if the translation-only
>>>>> scenes have
>>>>> >>>>>> some hand held wobble; that's fine. Boxes with texture are good.
>>>>> Complicated
>>>>> >>>>>> objects like bushes or trees are bad. The scene must be static
>>>>> (no cars,
>>>>> >>>>>> people, birds, godzilla, etc).
>>>>> >>>>>> The camera must be calibrated. There is a libmv calibration
>>>>> target
>>>>> >>>>>> that you can affix to a really flat object. The calibration
>>>>> target must be
>>>>> >>>>>> really flat; something that has a bit of bend is unacceptable
>>>>> and will give
>>>>> >>>>>> poor results.
>>>>> >>>>>>
>>>>> >>>>>>
>>>>> http://libmv.googlecode.com/svn/trunk/extras/calibration/calibration.pdf
>>>>> >>>>>> Take several images of the calibration target (or a video in
>>>>> HD). Get
>>>>> >>>>>> different rotations and fill the frame to get the edges
>>>>> (important for
>>>>> >>>>>> distortion correction). Then run one of the calibration tools to
>>>>> find the
>>>>> >>>>>> intrinsic parameters (focal length in pixels, distortion, skew,
>>>>> etc).
>>>>> >>>>>> Note: Rolling shutter is *horrible* for tracking. We're going to
>>>>> have
>>>>> >>>>>> to handle it at some point, but I am trying to avoid it as long
>>>>> as possible.
>>>>> >>>>>> Do you have access to a CCD style camera without a rolling
>>>>> shutter?
>>>>> >>>>>> Keir
>>>>> >>>>>>
>>>>> >>>>>> On Tue, Jun 7, 2011 at 3:27 PM, Ian Hubert
>>>>> >>>>>> <ian at projectlondonmovie.com> wrote:
>>>>> >>>>>>>
>>>>> >>>>>>> Hey all!
>>>>> >>>>>>> I've been following the conversations, and wanted to throw out
>>>>> that
>>>>> >>>>>>> if anyone needs any sort of footage, I can run out and film
>>>>> stuff. I already
>>>>> >>>>>>> have a few things things filmed- mostly just walking around the
>>>>> city holding
>>>>> >>>>>>> the camera as steady as I can- but if anyone wants anything
>>>>> else
>>>>> >>>>>>> (underexposed, really shaky, steady smooth,
>>>>> panning--but-no-movement,
>>>>> >>>>>>> somebody walking across the frame, stuff like that), just say
>>>>> so!
>>>>> >>>>>>> At this point all I can quickly provide is SD and HD-
>>>>> >>>>>>> I unfortunately don't have access to a 4k camera, but I'll film
>>>>> some random
>>>>> >>>>>>> stuff if I ever get my hands on one.
>>>>> >>>>>>>
>>>>> >>>>>>> --
>>>>> >>>>>>> Ian Hubert
>>>>> >>>>>>>
>>>>> >>>>>>> _______________________________________________
>>>>> >>>>>>> Bf-vfx mailing list
>>>>> >>>>>>> Bf-vfx at blender.org
>>>>> >>>>>>> http://lists.blender.org/mailman/listinfo/bf-vfx
>>>>> >>>>>>>
>>>>> >>>>>>
>>>>> >>>>>>
>>>>> >>>>>> _______________________________________________
>>>>> >>>>>> Bf-vfx mailing list
>>>>> >>>>>> Bf-vfx at blender.org
>>>>> >>>>>> http://lists.blender.org/mailman/listinfo/bf-vfx
>>>>> >>>>>>
>>>>> >>>>>
>>>>> >>>>>
>>>>> >>>>>
>>>>> >>>>> --
>>>>> >>>>> ____________________
>>>>> >>>>> François Tarlier
>>>>> >>>>> www.francois-tarlier.com
>>>>> >>>>> www.linkedin.com/in/francoistarlier
>>>>> >>>>> _______________________________________________
>>>>> >>>>> Bf-vfx mailing list
>>>>> >>>>> Bf-vfx at blender.org
>>>>> >>>>> http://lists.blender.org/mailman/listinfo/bf-vfx
>>>>> >>>>>
>>>>> >>>>>
>>>>> >>>>> _______________________________________________
>>>>> >>>>> Bf-vfx mailing list
>>>>> >>>>> Bf-vfx at blender.org
>>>>> >>>>> http://lists.blender.org/mailman/listinfo/bf-vfx
>>>>> >>>>>
>>>>> >>>>
>>>>> >>>> _______________________________________________
>>>>> >>>> Bf-vfx mailing list
>>>>> >>>> Bf-vfx at blender.org
>>>>> >>>> http://lists.blender.org/mailman/listinfo/bf-vfx
>>>>> >>>>
>>>>> >>>>
>>>>> >>>> _______________________________________________
>>>>> >>>> Bf-vfx mailing list
>>>>> >>>> Bf-vfx at blender.org
>>>>> >>>> http://lists.blender.org/mailman/listinfo/bf-vfx
>>>>> >>>>
>>>>> >>>
>>>>> >>>
>>>>> >>> _______________________________________________
>>>>> >>> Bf-vfx mailing list
>>>>> >>> Bf-vfx at blender.org
>>>>> >>> http://lists.blender.org/mailman/listinfo/bf-vfx
>>>>> >>>
>>>>> >>
>>>>> >>
>>>>> >>
>>>>> >> --
>>>>> >> ____________________
>>>>> >> François Tarlier
>>>>> >> www.francois-tarlier.com
>>>>> >> www.linkedin.com/in/francoistarlier
>>>>> >>
>>>>> >> _______________________________________________
>>>>> >> Bf-vfx mailing list
>>>>> >> Bf-vfx at blender.org
>>>>> >> http://lists.blender.org/mailman/listinfo/bf-vfx
>>>>> >>
>>>>> >
>>>>> >
>>>>> > _______________________________________________
>>>>> > Bf-vfx mailing list
>>>>> > Bf-vfx at blender.org
>>>>> > http://lists.blender.org/mailman/listinfo/bf-vfx
>>>>> >
>>>>> >
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Franco Callari <fgcallari at gmail.com>
>>>>>
>>>>>             EC67 BEBE 62AC 8415 7591  2B12 A6CD D5EE D8CB D0ED
>>>>>
>>>>> I am not bound to win, but I am bound to be true. I am not bound to
>>>>> succeed, but I am bound to live by the light that I have. (Abraham
>>>>> Lincoln)
>>>>> _______________________________________________
>>>>> Bf-vfx mailing list
>>>>> Bf-vfx at blender.org
>>>>> http://lists.blender.org/mailman/listinfo/bf-vfx
>>>>>
>>>>
>>>> _______________________________________________
>>>> Bf-vfx mailing list
>>>> Bf-vfx at blender.org
>>>> http://lists.blender.org/mailman/listinfo/bf-vfx
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> Bf-vfx mailing list
>>>> Bf-vfx at blender.org
>>>> http://lists.blender.org/mailman/listinfo/bf-vfx
>>>>
>>>>
>>> _______________________________________________
>>> Bf-vfx mailing list
>>> Bf-vfx at blender.org
>>> http://lists.blender.org/mailman/listinfo/bf-vfx
>>>
>>>
>>>
>>> _______________________________________________
>>> Bf-vfx mailing list
>>> Bf-vfx at blender.org
>>> http://lists.blender.org/mailman/listinfo/bf-vfx
>>>
>>>
>>
>> _______________________________________________
>> Bf-vfx mailing list
>> Bf-vfx at blender.org
>> http://lists.blender.org/mailman/listinfo/bf-vfx
>>
>>
>
>
> --
> ____________________
> François Tarlier
> www.francois-tarlier.com
> www.linkedin.com/in/francoistarlier
>
> _______________________________________________
> Bf-vfx mailing list
> Bf-vfx at blender.org
> http://lists.blender.org/mailman/listinfo/bf-vfx
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.blender.org/pipermail/bf-vfx/attachments/20110612/338e27c5/attachment-0001.htm

More information about the Bf-vfx mailing list