[Bf-vfx] Footage for Tracking

Sun Jun 12 10:40:17 CEST 2011

what is this calibrating ? lens distortion ? or lens millimeter ?

2011/6/12 Keir Mierle <mierle at gmail.com>

> Unfortunately these are not good enough. To get really solid calibration,
> you need:
>
>    1. A checkerboard (turns out it is easy to very precisely locate the
>    corners)
>    2. Several views of the calibration grid at different angles; the
>    perspective on the checkerboard should be very apparent (in addition to
>    front-on views).
>    3. Each shot of the checkerboard should try to cover as much of the
>    field of view of the camera as possible, while still showing the full
>    checker board
>    4. The checkerboard and camera should be still (no rolling shutter for
>    calibration please!)
>
> #3 is important since the distortion tends to happen towards the edges of
> the view, so if the grid only occupies the middle 25% of the view (like in
> your examples) it won't produce a good calibration. Even with the full
> autotracking that you get in the commercial packages, you can get superior
> results if you properly calibrate your camera; especially for difficult
> shots where there is ambiguity, etc.
>
> Do you know anyone who might be willing to take a few shots of a
> checkerboard?
>
> Keir
>
> On Sat, Jun 11, 2011 at 5:54 PM, Colin Levy <colin at peerlessproductions.com
> > wrote:
>
>> Hey Kier,
>>
>> Unfortunately I no longer have access to the camera package, but while we
>> were on set we shot some lens calibration reference.  Don't know if this is
>> useful at all, but here's what we shot for the plate I named "LOBBY":
>>
>> http://www.colinlevy.com/upload/LensCalibration01.jpg
>> http://www.colinlevy.com/upload/LensCalibration02.jpg
>>
>> Unfortunately that's all I seem to have for these three shots, and I don't
>> know if they will do the trick.
>>
>> --Colin
>>
>> On Jun 11, 2011, at 8:23 PM, Keir Mierle wrote:
>>
>> Hi Colin,
>>
>> These look great. Can I convince you to take images of a calibration
>> pattern with the lenses? We are not going to support uncalibrated tracking
>> in the first release, so this is not as useful without calibration. Here's a
>> calibration pattern I made for libmv:
>>
>> http://libmv.googlecode.com/svn/trunk/extras/calibration/calibration.pdf
>>
>> Just print it out and carefully affix it to something very, very flat (no
>> bends or wrinkles when gluing please).
>>
>> Thanks,
>> Keir
>>
>> On Sat, Jun 11, 2011 at 3:15 PM, Colin Levy <
>> colin at peerlessproductions.com> wrote:
>>
>>> Hey guys,
>>>
>>> Wanted to let you know I've uploaded three shots which may hopefully be
>>> useful to you.
>>>
>>> All three shots were shot on RED at 4K resolution for my film "En Route".
>>>  All three have similar camera movement: a simple forward dolly push.  I've
>>> uploaded JPG sequences for now, but can supply EXRs if requested.
>>>
>>> You can watch H264 previews of these shots here:
>>>
>>> http://www.colinlevy.com/upload/LOBBY_preview.mov
>>> http://www.colinlevy.com/upload/FIELD_preview.mov
>>> http://www.colinlevy.com/upload/WINDOW_preview.mov
>>>
>>> And you can log into the FTP space I'm using for a temporary project to
>>> access the image sequences.  The folder is called "blender-vfx".
>>>
>>> Username: b173379
>>> Password: bleem34
>>> Server: hanjin.dreamhost.com
>>> (Please keep this info private!)
>>>
>>> I can supply more varied material if you like as well.  Hope this helps!
>>>
>>> --Colin
>>>
>>>
>>>
>>> On Jun 9, 2011, at 8:33 PM, Keir Mierle wrote:
>>>
>>> On Thu, Jun 9, 2011 at 5:12 PM, Francesco Callari <fgcallari at gmail.com>wrote:
>>>
>>>> On Wed, Jun 8, 2011 at 2:17 PM, Keir Mierle <mierle at gmail.com> wrote:
>>>> > On Wed, Jun 8, 2011 at 1:44 PM, François T. <
>>>> francoistarlier at gmail.com>
>>>> > wrote:
>>>> >>>
>>>> >>> Regarding file formats: JPEG frames are probably fine; there's
>>>> little
>>>> >>> value in tracking on 16-bit data. The frames can even get converted
>>>> to
>>>> >>> grayscale, since the tracking ignores color information.
>>>> >>
>>>> >>
>>>> >> are you not planning on using all the depth on the image for tracking
>>>> ? in
>>>> >> my experience, it can helps a lot sometimes !
>>>> >
>>>> > Maybe eventually but not to start. Libmv takes 32 bit grayscale float
>>>> images
>>>> > to track them. However, I'm skeptical that more than 8 bits of
>>>> grayscale
>>>> > data would make a difference for tracking. Do you have examples where
>>>> this
>>>> > is the case?
>>>>
>>>> One data point. When I was at ILM nearly all tracking (and, in fact,
>>>> nearly all the imagery pipeline save for final rendering) was done on
>>>> 16bit OpenEXR float images (the "half" pixel format of OepnEXR). Of
>>>> course the tracking code internally used higher precision for the
>>>> arithmetics, and was independent of the image format. Generally
>>>> speaking, OpenEXR is a really nice image library to work with in a
>>>> graphics + vision environment.
>>>>
>>>
>>> It sounds like supporting higher bit depth is useful. This shouldn't be
>>> hard since libmv already operates on excessively precise 32 bit floats. It
>>> will just be a matter of using the appropriate conversion to 32 bit floats
>>> when extracting patches to track from the underlying footage.
>>>
>>> I believe Blender already supports OpenEXR throughout.
>>>
>>> Keir
>>>
>>>
>>>>
>>>> Franco
>>>>
>>>>
>>>> > Keir
>>>> >
>>>> >>
>>>> >> 2011/6/8 Keir Mierle <mierle at gmail.com>
>>>> >>>
>>>> >>> On Wed, Jun 8, 2011 at 11:53 AM, Colin Levy
>>>> >>> <colin at peerlessproductions.com> wrote:
>>>> >>>>
>>>> >>>> Yeah, rolling shutter is an *incredible* pain.  It can be corrected
>>>> to
>>>> >>>> an extent with tools like The Foundry's "RollingShutter" plugin for
>>>> Nuke,
>>>> >>>> which I've heard can make previously un-trackable shots trackable,
>>>> but you
>>>> >>>> see it *everywhere* these days -- even in hollywood features.
>>>> >>>
>>>> >>> Yes, we will deal with it but it will take some time. It's
>>>> mathematically
>>>> >>> impossible to fix perfectly, so the best we can do is approximate
>>>> fixes.
>>>> >>> Truly unfortunate.
>>>> >>>
>>>> >>>>
>>>> >>>> I do have some footage I can get you from my Canon XH-A1, which is
>>>> a
>>>> >>>> 3-CCD HDV camera.  I sold the camera recently (CMOS is all the
>>>> rage!), but I
>>>> >>>> have a ton of footage I can go through.
>>>> >>>
>>>> >>> That would be great! The only constraint is going to be calibration.
>>>> Can
>>>> >>> you get access to the camera for 10 minutes to take some shots of a
>>>> >>> calibration pattern?
>>>> >>> I realize the big tools deal with this automatically, but it's
>>>> difficult
>>>> >>> to do automatically and won't be supported in our first release.
>>>> >>>
>>>> >>>>
>>>> >>>> That said, RED's CMOS sensor is waayy better than the "jellocam"
>>>> sensors
>>>> >>>> in DSLRs like the 5D or T2i.  (RED's new MX sensor I think is even
>>>> better).
>>>> >>>>  So the rolling shutter is not bad -- but it's still noticeable in
>>>> very
>>>> >>>> high-motion shots.
>>>> >>>> I'd say 80% of the RED shots I've got should not pose issues due to
>>>> >>>> rolling shutter.  The tracking tools I've been using do not account
>>>> for
>>>> >>>> rolling shutter, yet I'm able to get very solid tracks with little
>>>> manual
>>>> >>>> intervention.  So perhaps still valuable to test with?
>>>> >>>
>>>> >>> Sounds like the RED shots will work as long as the motion is not too
>>>> >>> extreme.
>>>> >>> Regarding file formats: JPEG frames are probably fine; there's
>>>> little
>>>> >>> value in tracking on 16-bit data. The frames can even get converted
>>>> to
>>>> >>> grayscale, since the tracking ignores color information.
>>>> >>>
>>>> >>> Keir
>>>> >>>>
>>>> >>>> --Colin
>>>> >>>>
>>>> >>>> On Jun 8, 2011, at 2:32 PM, Keir Mierle wrote:
>>>> >>>>
>>>> >>>> Thanks Colin!
>>>> >>>> One point though: RED uses a CMOS sensor and is subject to rolling
>>>> >>>> shutter. Do you know how bad it is? Some cameras have very slow
>>>> sensor
>>>> >>>> readout (and so terrible distortions with movement) and others have
>>>> a faster
>>>> >>>> rolling shutter which still has distortions, but less.
>>>> >>>> For those who don't know: rolling shutter means that each scanline
>>>> has a
>>>> >>>> *different* shutter time! Yes, that means that the image is
>>>> composed of
>>>> >>>> 1000's of images, one scanline taken from each, with the camera
>>>> moved
>>>> >>>> between images. Extremely annoying, and totally breaks all the
>>>> camera
>>>> >>>> tracking assumptions.
>>>> >>>> I realize that we have to handle rolling shutter but it's a big job
>>>> and
>>>> >>>> we won't support it in the first release.
>>>> >>>> So I repeat my request: does anyone have a camera that doesn't have
>>>> a
>>>> >>>> rolling shutter? For example, the Panasonic SD9 has a CCD sensor
>>>> and so
>>>> >>>> takes true pinhole images for each frame.
>>>> >>>> Virtually all modern DSLR's have awful rolling shutter (D90
>>>> included),
>>>> >>>> so they are also excluded.
>>>> >>>> Keir
>>>> >>>> On Wed, Jun 8, 2011 at 11:00 AM, Colin Levy
>>>> >>>> <colin at peerlessproductions.com> wrote:
>>>> >>>>>
>>>> >>>>> Hey guys,
>>>> >>>>> I've got several TB worth of 4K material I can offer for tracking.
>>>> >>>>>  However, the shots will represent a more "real-world" scenario
>>>> rather than
>>>> >>>>> the test case shots listed below.
>>>> >>>>> I don't suppose you want the RED .r3d files, so will TIFF
>>>> sequences
>>>> >>>>> work for you?  (Or do you want both?)  I will go through my drives
>>>> in the
>>>> >>>>> next few days and upload a few shots to my FTP, if you think this
>>>> would be
>>>> >>>>> useful!  Let me know whether you'd like 32-bit images, with all
>>>> the 'RAW'
>>>> >>>>> data intact, or if 8 or 16bit will suffice.
>>>> >>>>> One thing to note regarding resolution: I have noticed that in
>>>> many
>>>> >>>>> cases, tracking downrezzed shots is frequently more successful
>>>> than trying
>>>> >>>>> to track the raw 4K material.  This may have something to do with
>>>> the
>>>> >>>>> default values (the default auto feature search range, for
>>>> example, may be
>>>> >>>>> something like 20px rather than 80px, which might be a better
>>>> setting for
>>>> >>>>> 4K).  For my projects I usually downrez to 2K before tracking...
>>>> also
>>>> >>>>> because it's a lot faster.  :P
>>>> >>>>> Great discussion.  Can't wait to see these tools develop!
>>>> >>>>> --Colin
>>>> >>>>>
>>>> >>>>> On Jun 8, 2011, at 12:59 PM, François T. wrote:
>>>> >>>>>
>>>> >>>>> I'll do all that on saturday with D90.
>>>> >>>>> cheers
>>>> >>>>> F.
>>>> >>>>>
>>>> >>>>> 2011/6/8 Keir Mierle <mierle at gmail.com>
>>>> >>>>>>
>>>> >>>>>> A catalog of the following shots would be useful:
>>>> >>>>>>
>>>> >>>>>> General motion: Moving and rotating camera. Camera is /not/
>>>> always
>>>> >>>>>> pointing at one part of the scene.
>>>> >>>>>> Rotation only: camera on a tripod, rotating around a fixed point
>>>> (if
>>>> >>>>>> possible the camera's "Nodal point", but you need a special
>>>> tripod for that)
>>>> >>>>>> Translation only: walking sideways keeping the camera pointed
>>>> forward
>>>> >>>>>> (e.g. not circling)
>>>> >>>>>> Translation only: walking towards the scene
>>>> >>>>>> General motion when the scene is planar (e.g. camera keeps a big
>>>> >>>>>> billboard in view and only the billboard)
>>>> >>>>>> Circle strafing around an object; camera is fixated on one part
>>>> of the
>>>> >>>>>> scene (intersecting principal ray)
>>>> >>>>>>
>>>> >>>>>> The scenes should have good texture so that tracking is easy; we
>>>> can
>>>> >>>>>> add challenging scenes later. It's ok if the translation-only
>>>> scenes have
>>>> >>>>>> some hand held wobble; that's fine. Boxes with texture are good.
>>>> Complicated
>>>> >>>>>> objects like bushes or trees are bad. The scene must be static
>>>> (no cars,
>>>> >>>>>> people, birds, godzilla, etc).
>>>> >>>>>> The camera must be calibrated. There is a libmv calibration
>>>> target
>>>> >>>>>> that you can affix to a really flat object. The calibration
>>>> target must be
>>>> >>>>>> really flat; something that has a bit of bend is unacceptable and
>>>> will give
>>>> >>>>>> poor results.
>>>> >>>>>>
>>>> >>>>>>
>>>> http://libmv.googlecode.com/svn/trunk/extras/calibration/calibration.pdf
>>>> >>>>>> Take several images of the calibration target (or a video in HD).
>>>> Get
>>>> >>>>>> different rotations and fill the frame to get the edges
>>>> (important for
>>>> >>>>>> distortion correction). Then run one of the calibration tools to
>>>> find the
>>>> >>>>>> intrinsic parameters (focal length in pixels, distortion, skew,
>>>> etc).
>>>> >>>>>> Note: Rolling shutter is *horrible* for tracking. We're going to
>>>> have
>>>> >>>>>> to handle it at some point, but I am trying to avoid it as long
>>>> as possible.
>>>> >>>>>> Do you have access to a CCD style camera without a rolling
>>>> shutter?
>>>> >>>>>> Keir
>>>> >>>>>>
>>>> >>>>>> On Tue, Jun 7, 2011 at 3:27 PM, Ian Hubert
>>>> >>>>>> <ian at projectlondonmovie.com> wrote:
>>>> >>>>>>>
>>>> >>>>>>> Hey all!
>>>> >>>>>>> I've been following the conversations, and wanted to throw out
>>>> that
>>>> >>>>>>> if anyone needs any sort of footage, I can run out and film
>>>> stuff. I already
>>>> >>>>>>> have a few things things filmed- mostly just walking around the
>>>> city holding
>>>> >>>>>>> the camera as steady as I can- but if anyone wants anything else
>>>> >>>>>>> (underexposed, really shaky, steady smooth,
>>>> panning--but-no-movement,
>>>> >>>>>>> somebody walking across the frame, stuff like that), just say
>>>> so!
>>>> >>>>>>> At this point all I can quickly provide is SD and HD-
>>>> >>>>>>> I unfortunately don't have access to a 4k camera, but I'll film
>>>> some random
>>>> >>>>>>> stuff if I ever get my hands on one.
>>>> >>>>>>>
>>>> >>>>>>> --
>>>> >>>>>>> Ian Hubert
>>>> >>>>>>>
>>>> >>>>>>> _______________________________________________
>>>> >>>>>>> Bf-vfx mailing list
>>>> >>>>>>> Bf-vfx at blender.org
>>>> >>>>>>> http://lists.blender.org/mailman/listinfo/bf-vfx
>>>> >>>>>>>
>>>> >>>>>>
>>>> >>>>>>
>>>> >>>>>> _______________________________________________
>>>> >>>>>> Bf-vfx mailing list
>>>> >>>>>> Bf-vfx at blender.org
>>>> >>>>>> http://lists.blender.org/mailman/listinfo/bf-vfx
>>>> >>>>>>
>>>> >>>>>
>>>> >>>>>
>>>> >>>>>
>>>> >>>>> --
>>>> >>>>> ____________________
>>>> >>>>> François Tarlier
>>>> >>>>> www.francois-tarlier.com
>>>> >>>>> www.linkedin.com/in/francoistarlier
>>>> >>>>> _______________________________________________
>>>> >>>>> Bf-vfx mailing list
>>>> >>>>> Bf-vfx at blender.org
>>>> >>>>> http://lists.blender.org/mailman/listinfo/bf-vfx
>>>> >>>>>
>>>> >>>>>
>>>> >>>>> _______________________________________________
>>>> >>>>> Bf-vfx mailing list
>>>> >>>>> Bf-vfx at blender.org
>>>> >>>>> http://lists.blender.org/mailman/listinfo/bf-vfx
>>>> >>>>>
>>>> >>>>
>>>> >>>> _______________________________________________
>>>> >>>> Bf-vfx mailing list
>>>> >>>> Bf-vfx at blender.org
>>>> >>>> http://lists.blender.org/mailman/listinfo/bf-vfx
>>>> >>>>
>>>> >>>>
>>>> >>>> _______________________________________________
>>>> >>>> Bf-vfx mailing list
>>>> >>>> Bf-vfx at blender.org
>>>> >>>> http://lists.blender.org/mailman/listinfo/bf-vfx
>>>> >>>>
>>>> >>>
>>>> >>>
>>>> >>> _______________________________________________
>>>> >>> Bf-vfx mailing list
>>>> >>> Bf-vfx at blender.org
>>>> >>> http://lists.blender.org/mailman/listinfo/bf-vfx
>>>> >>>
>>>> >>
>>>> >>
>>>> >>
>>>> >> --
>>>> >> ____________________
>>>> >> François Tarlier
>>>> >> www.francois-tarlier.com
>>>> >> www.linkedin.com/in/francoistarlier
>>>> >>
>>>> >> _______________________________________________
>>>> >> Bf-vfx mailing list
>>>> >> Bf-vfx at blender.org
>>>> >> http://lists.blender.org/mailman/listinfo/bf-vfx
>>>> >>
>>>> >
>>>> >
>>>> > _______________________________________________
>>>> > Bf-vfx mailing list
>>>> > Bf-vfx at blender.org
>>>> > http://lists.blender.org/mailman/listinfo/bf-vfx
>>>> >
>>>> >
>>>>
>>>>
>>>>
>>>> --
>>>> Franco Callari <fgcallari at gmail.com>
>>>>
>>>>             EC67 BEBE 62AC 8415 7591  2B12 A6CD D5EE D8CB D0ED
>>>>
>>>> I am not bound to win, but I am bound to be true. I am not bound to
>>>> succeed, but I am bound to live by the light that I have. (Abraham
>>>> Lincoln)
>>>> _______________________________________________
>>>> Bf-vfx mailing list
>>>> Bf-vfx at blender.org
>>>> http://lists.blender.org/mailman/listinfo/bf-vfx
>>>>
>>>
>>> _______________________________________________
>>> Bf-vfx mailing list
>>> Bf-vfx at blender.org
>>> http://lists.blender.org/mailman/listinfo/bf-vfx
>>>
>>>
>>>
>>> _______________________________________________
>>> Bf-vfx mailing list
>>> Bf-vfx at blender.org
>>> http://lists.blender.org/mailman/listinfo/bf-vfx
>>>
>>>
>> _______________________________________________
>> Bf-vfx mailing list
>> Bf-vfx at blender.org
>> http://lists.blender.org/mailman/listinfo/bf-vfx
>>
>>
>>
>> _______________________________________________
>> Bf-vfx mailing list
>> Bf-vfx at blender.org
>> http://lists.blender.org/mailman/listinfo/bf-vfx
>>
>>
>
> _______________________________________________
> Bf-vfx mailing list
> Bf-vfx at blender.org
> http://lists.blender.org/mailman/listinfo/bf-vfx
>
>

-- 
____________________
François Tarlier
www.francois-tarlier.com
www.linkedin.com/in/francoistarlier
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.blender.org/pipermail/bf-vfx/attachments/20110612/96ed89c4/attachment-0001.htm 