[Bf-vfx] Footage for Tracking

Colin Levy colin at peerlessproductions.com
Sun Jun 12 02:54:13 CEST 2011 

Hey Kier,

Unfortunately I no longer have access to the camera package, but while we were on set we shot some lens calibration reference.  Don't know if this is useful at all, but here's what we shot for the plate I named "LOBBY":

http://www.colinlevy.com/upload/LensCalibration01.jpg
http://www.colinlevy.com/upload/LensCalibration02.jpg

Unfortunately that's all I seem to have for these three shots, and I don't know if they will do the trick.

--Colin

On Jun 11, 2011, at 8:23 PM, Keir Mierle wrote:

> Hi Colin,
> 
> These look great. Can I convince you to take images of a calibration pattern with the lenses? We are not going to support uncalibrated tracking in the first release, so this is not as useful without calibration. Here's a calibration pattern I made for libmv:
> 
> http://libmv.googlecode.com/svn/trunk/extras/calibration/calibration.pdf
> 
> Just print it out and carefully affix it to something very, very flat (no bends or wrinkles when gluing please).
> 
> Thanks,
> Keir
> 
> On Sat, Jun 11, 2011 at 3:15 PM, Colin Levy <colin at peerlessproductions.com> wrote:
> Hey guys,
> 
> Wanted to let you know I've uploaded three shots which may hopefully be useful to you.
> 
> All three shots were shot on RED at 4K resolution for my film "En Route".  All three have similar camera movement: a simple forward dolly push.  I've uploaded JPG sequences for now, but can supply EXRs if requested.
> 
> You can watch H264 previews of these shots here:
> 
> http://www.colinlevy.com/upload/LOBBY_preview.mov
> http://www.colinlevy.com/upload/FIELD_preview.mov
> http://www.colinlevy.com/upload/WINDOW_preview.mov
> 
> And you can log into the FTP space I'm using for a temporary project to access the image sequences.  The folder is called "blender-vfx".
> 
> Username: b173379
> Password: bleem34
> Server: hanjin.dreamhost.com
> (Please keep this info private!)
> 
> I can supply more varied material if you like as well.  Hope this helps!
> 
> --Colin
> 
> 
> 
> On Jun 9, 2011, at 8:33 PM, Keir Mierle wrote:
> 
>> On Thu, Jun 9, 2011 at 5:12 PM, Francesco Callari <fgcallari at gmail.com> wrote:
>> On Wed, Jun 8, 2011 at 2:17 PM, Keir Mierle <mierle at gmail.com> wrote:
>> > On Wed, Jun 8, 2011 at 1:44 PM, François T. <francoistarlier at gmail.com>
>> > wrote:
>> >>>
>> >>> Regarding file formats: JPEG frames are probably fine; there's little
>> >>> value in tracking on 16-bit data. The frames can even get converted to
>> >>> grayscale, since the tracking ignores color information.
>> >>
>> >>
>> >> are you not planning on using all the depth on the image for tracking ? in
>> >> my experience, it can helps a lot sometimes !
>> >
>> > Maybe eventually but not to start. Libmv takes 32 bit grayscale float images
>> > to track them. However, I'm skeptical that more than 8 bits of grayscale
>> > data would make a difference for tracking. Do you have examples where this
>> > is the case?
>> 
>> One data point. When I was at ILM nearly all tracking (and, in fact,
>> nearly all the imagery pipeline save for final rendering) was done on
>> 16bit OpenEXR float images (the "half" pixel format of OepnEXR). Of
>> course the tracking code internally used higher precision for the
>> arithmetics, and was independent of the image format. Generally
>> speaking, OpenEXR is a really nice image library to work with in a
>> graphics + vision environment.
>> 
>> It sounds like supporting higher bit depth is useful. This shouldn't be hard since libmv already operates on excessively precise 32 bit floats. It will just be a matter of using the appropriate conversion to 32 bit floats when extracting patches to track from the underlying footage.
>> 
>> I believe Blender already supports OpenEXR throughout.
>> 
>> Keir
>>  
>> 
>> Franco
>> 
>> 
>> > Keir
>> >
>> >>
>> >> 2011/6/8 Keir Mierle <mierle at gmail.com>
>> >>>
>> >>> On Wed, Jun 8, 2011 at 11:53 AM, Colin Levy
>> >>> <colin at peerlessproductions.com> wrote:
>> >>>>
>> >>>> Yeah, rolling shutter is an *incredible* pain.  It can be corrected to
>> >>>> an extent with tools like The Foundry's "RollingShutter" plugin for Nuke,
>> >>>> which I've heard can make previously un-trackable shots trackable, but you
>> >>>> see it *everywhere* these days -- even in hollywood features.
>> >>>
>> >>> Yes, we will deal with it but it will take some time. It's mathematically
>> >>> impossible to fix perfectly, so the best we can do is approximate fixes.
>> >>> Truly unfortunate.
>> >>>
>> >>>>
>> >>>> I do have some footage I can get you from my Canon XH-A1, which is a
>> >>>> 3-CCD HDV camera.  I sold the camera recently (CMOS is all the rage!), but I
>> >>>> have a ton of footage I can go through.
>> >>>
>> >>> That would be great! The only constraint is going to be calibration. Can
>> >>> you get access to the camera for 10 minutes to take some shots of a
>> >>> calibration pattern?
>> >>> I realize the big tools deal with this automatically, but it's difficult
>> >>> to do automatically and won't be supported in our first release.
>> >>>
>> >>>>
>> >>>> That said, RED's CMOS sensor is waayy better than the "jellocam" sensors
>> >>>> in DSLRs like the 5D or T2i.  (RED's new MX sensor I think is even better).
>> >>>>  So the rolling shutter is not bad -- but it's still noticeable in very
>> >>>> high-motion shots.
>> >>>> I'd say 80% of the RED shots I've got should not pose issues due to
>> >>>> rolling shutter.  The tracking tools I've been using do not account for
>> >>>> rolling shutter, yet I'm able to get very solid tracks with little manual
>> >>>> intervention.  So perhaps still valuable to test with?
>> >>>
>> >>> Sounds like the RED shots will work as long as the motion is not too
>> >>> extreme.
>> >>> Regarding file formats: JPEG frames are probably fine; there's little
>> >>> value in tracking on 16-bit data. The frames can even get converted to
>> >>> grayscale, since the tracking ignores color information.
>> >>>
>> >>> Keir
>> >>>>
>> >>>> --Colin
>> >>>>
>> >>>> On Jun 8, 2011, at 2:32 PM, Keir Mierle wrote:
>> >>>>
>> >>>> Thanks Colin!
>> >>>> One point though: RED uses a CMOS sensor and is subject to rolling
>> >>>> shutter. Do you know how bad it is? Some cameras have very slow sensor
>> >>>> readout (and so terrible distortions with movement) and others have a faster
>> >>>> rolling shutter which still has distortions, but less.
>> >>>> For those who don't know: rolling shutter means that each scanline has a
>> >>>> *different* shutter time! Yes, that means that the image is composed of
>> >>>> 1000's of images, one scanline taken from each, with the camera moved
>> >>>> between images. Extremely annoying, and totally breaks all the camera
>> >>>> tracking assumptions.
>> >>>> I realize that we have to handle rolling shutter but it's a big job and
>> >>>> we won't support it in the first release.
>> >>>> So I repeat my request: does anyone have a camera that doesn't have a
>> >>>> rolling shutter? For example, the Panasonic SD9 has a CCD sensor and so
>> >>>> takes true pinhole images for each frame.
>> >>>> Virtually all modern DSLR's have awful rolling shutter (D90 included),
>> >>>> so they are also excluded.
>> >>>> Keir
>> >>>> On Wed, Jun 8, 2011 at 11:00 AM, Colin Levy
>> >>>> <colin at peerlessproductions.com> wrote:
>> >>>>>
>> >>>>> Hey guys,
>> >>>>> I've got several TB worth of 4K material I can offer for tracking.
>> >>>>>  However, the shots will represent a more "real-world" scenario rather than
>> >>>>> the test case shots listed below.
>> >>>>> I don't suppose you want the RED .r3d files, so will TIFF sequences
>> >>>>> work for you?  (Or do you want both?)  I will go through my drives in the
>> >>>>> next few days and upload a few shots to my FTP, if you think this would be
>> >>>>> useful!  Let me know whether you'd like 32-bit images, with all the 'RAW'
>> >>>>> data intact, or if 8 or 16bit will suffice.
>> >>>>> One thing to note regarding resolution: I have noticed that in many
>> >>>>> cases, tracking downrezzed shots is frequently more successful than trying
>> >>>>> to track the raw 4K material.  This may have something to do with the
>> >>>>> default values (the default auto feature search range, for example, may be
>> >>>>> something like 20px rather than 80px, which might be a better setting for
>> >>>>> 4K).  For my projects I usually downrez to 2K before tracking... also
>> >>>>> because it's a lot faster.  :P
>> >>>>> Great discussion.  Can't wait to see these tools develop!
>> >>>>> --Colin
>> >>>>>
>> >>>>> On Jun 8, 2011, at 12:59 PM, François T. wrote:
>> >>>>>
>> >>>>> I'll do all that on saturday with D90.
>> >>>>> cheers
>> >>>>> F.
>> >>>>>
>> >>>>> 2011/6/8 Keir Mierle <mierle at gmail.com>
>> >>>>>>
>> >>>>>> A catalog of the following shots would be useful:
>> >>>>>>
>> >>>>>> General motion: Moving and rotating camera. Camera is /not/ always
>> >>>>>> pointing at one part of the scene.
>> >>>>>> Rotation only: camera on a tripod, rotating around a fixed point (if
>> >>>>>> possible the camera's "Nodal point", but you need a special tripod for that)
>> >>>>>> Translation only: walking sideways keeping the camera pointed forward
>> >>>>>> (e.g. not circling)
>> >>>>>> Translation only: walking towards the scene
>> >>>>>> General motion when the scene is planar (e.g. camera keeps a big
>> >>>>>> billboard in view and only the billboard)
>> >>>>>> Circle strafing around an object; camera is fixated on one part of the
>> >>>>>> scene (intersecting principal ray)
>> >>>>>>
>> >>>>>> The scenes should have good texture so that tracking is easy; we can
>> >>>>>> add challenging scenes later. It's ok if the translation-only scenes have
>> >>>>>> some hand held wobble; that's fine. Boxes with texture are good. Complicated
>> >>>>>> objects like bushes or trees are bad. The scene must be static (no cars,
>> >>>>>> people, birds, godzilla, etc).
>> >>>>>> The camera must be calibrated. There is a libmv calibration target
>> >>>>>> that you can affix to a really flat object. The calibration target must be
>> >>>>>> really flat; something that has a bit of bend is unacceptable and will give
>> >>>>>> poor results.
>> >>>>>>
>> >>>>>> http://libmv.googlecode.com/svn/trunk/extras/calibration/calibration.pdf
>> >>>>>> Take several images of the calibration target (or a video in HD). Get
>> >>>>>> different rotations and fill the frame to get the edges (important for
>> >>>>>> distortion correction). Then run one of the calibration tools to find the
>> >>>>>> intrinsic parameters (focal length in pixels, distortion, skew, etc).
>> >>>>>> Note: Rolling shutter is *horrible* for tracking. We're going to have
>> >>>>>> to handle it at some point, but I am trying to avoid it as long as possible.
>> >>>>>> Do you have access to a CCD style camera without a rolling shutter?
>> >>>>>> Keir
>> >>>>>>
>> >>>>>> On Tue, Jun 7, 2011 at 3:27 PM, Ian Hubert
>> >>>>>> <ian at projectlondonmovie.com> wrote:
>> >>>>>>>
>> >>>>>>> Hey all!
>> >>>>>>> I've been following the conversations, and wanted to throw out that
>> >>>>>>> if anyone needs any sort of footage, I can run out and film stuff. I already
>> >>>>>>> have a few things things filmed- mostly just walking around the city holding
>> >>>>>>> the camera as steady as I can- but if anyone wants anything else
>> >>>>>>> (underexposed, really shaky, steady smooth, panning--but-no-movement,
>> >>>>>>> somebody walking across the frame, stuff like that), just say so!
>> >>>>>>> At this point all I can quickly provide is SD and HD-
>> >>>>>>> I unfortunately don't have access to a 4k camera, but I'll film some random
>> >>>>>>> stuff if I ever get my hands on one.
>> >>>>>>>
>> >>>>>>> --
>> >>>>>>> Ian Hubert
>> >>>>>>>
>> >>>>>>> _______________________________________________
>> >>>>>>> Bf-vfx mailing list
>> >>>>>>> Bf-vfx at blender.org
>> >>>>>>> http://lists.blender.org/mailman/listinfo/bf-vfx
>> >>>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>> _______________________________________________
>> >>>>>> Bf-vfx mailing list
>> >>>>>> Bf-vfx at blender.org
>> >>>>>> http://lists.blender.org/mailman/listinfo/bf-vfx
>> >>>>>>
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>> --
>> >>>>> ____________________
>> >>>>> François Tarlier
>> >>>>> www.francois-tarlier.com
>> >>>>> www.linkedin.com/in/francoistarlier
>> >>>>> _______________________________________________
>> >>>>> Bf-vfx mailing list
>> >>>>> Bf-vfx at blender.org
>> >>>>> http://lists.blender.org/mailman/listinfo/bf-vfx
>> >>>>>
>> >>>>>
>> >>>>> _______________________________________________
>> >>>>> Bf-vfx mailing list
>> >>>>> Bf-vfx at blender.org
>> >>>>> http://lists.blender.org/mailman/listinfo/bf-vfx
>> >>>>>
>> >>>>
>> >>>> _______________________________________________
>> >>>> Bf-vfx mailing list
>> >>>> Bf-vfx at blender.org
>> >>>> http://lists.blender.org/mailman/listinfo/bf-vfx
>> >>>>
>> >>>>
>> >>>> _______________________________________________
>> >>>> Bf-vfx mailing list
>> >>>> Bf-vfx at blender.org
>> >>>> http://lists.blender.org/mailman/listinfo/bf-vfx
>> >>>>
>> >>>
>> >>>
>> >>> _______________________________________________
>> >>> Bf-vfx mailing list
>> >>> Bf-vfx at blender.org
>> >>> http://lists.blender.org/mailman/listinfo/bf-vfx
>> >>>
>> >>
>> >>
>> >>
>> >> --
>> >> ____________________
>> >> François Tarlier
>> >> www.francois-tarlier.com
>> >> www.linkedin.com/in/francoistarlier
>> >>
>> >> _______________________________________________
>> >> Bf-vfx mailing list
>> >> Bf-vfx at blender.org
>> >> http://lists.blender.org/mailman/listinfo/bf-vfx
>> >>
>> >
>> >
>> > _______________________________________________
>> > Bf-vfx mailing list
>> > Bf-vfx at blender.org
>> > http://lists.blender.org/mailman/listinfo/bf-vfx
>> >
>> >
>> 
>> 
>> 
>> --
>> Franco Callari <fgcallari at gmail.com>
>> 
>>             EC67 BEBE 62AC 8415 7591  2B12 A6CD D5EE D8CB D0ED
>> 
>> I am not bound to win, but I am bound to be true. I am not bound to
>> succeed, but I am bound to live by the light that I have. (Abraham
>> Lincoln)
>> _______________________________________________
>> Bf-vfx mailing list
>> Bf-vfx at blender.org
>> http://lists.blender.org/mailman/listinfo/bf-vfx
>> 
>> _______________________________________________
>> Bf-vfx mailing list
>> Bf-vfx at blender.org
>> http://lists.blender.org/mailman/listinfo/bf-vfx
> 
> 
> _______________________________________________
> Bf-vfx mailing list
> Bf-vfx at blender.org
> http://lists.blender.org/mailman/listinfo/bf-vfx
> 
> 
> _______________________________________________
> Bf-vfx mailing list
> Bf-vfx at blender.org
> http://lists.blender.org/mailman/listinfo/bf-vfx

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.blender.org/pipermail/bf-vfx/attachments/20110611/30d61554/attachment-0001.htm

More information about the Bf-vfx mailing list