[Bf-vfx] Footage for Tracking

Sun Jun 12 02:23:39 CEST 2011

Hi Colin,

These look great. Can I convince you to take images of a calibration pattern
with the lenses? We are not going to support uncalibrated tracking in the
first release, so this is not as useful without calibration. Here's a
calibration pattern I made for libmv:

http://libmv.googlecode.com/svn/trunk/extras/calibration/calibration.pdf

Just print it out and carefully affix it to something very, very flat (no
bends or wrinkles when gluing please).

Thanks,
Keir

On Sat, Jun 11, 2011 at 3:15 PM, Colin Levy
<colin at peerlessproductions.com>wrote:

> Hey guys,
>
> Wanted to let you know I've uploaded three shots which may hopefully be
> useful to you.
>
> All three shots were shot on RED at 4K resolution for my film "En Route".
>  All three have similar camera movement: a simple forward dolly push.  I've
> uploaded JPG sequences for now, but can supply EXRs if requested.
>
> You can watch H264 previews of these shots here:
>
> http://www.colinlevy.com/upload/LOBBY_preview.mov
> http://www.colinlevy.com/upload/FIELD_preview.mov
> http://www.colinlevy.com/upload/WINDOW_preview.mov
>
> And you can log into the FTP space I'm using for a temporary project to
> access the image sequences.  The folder is called "blender-vfx".
>
> Username: b173379
> Password: bleem34
> Server: hanjin.dreamhost.com
> (Please keep this info private!)
>
> I can supply more varied material if you like as well.  Hope this helps!
>
> --Colin
>
>
>
> On Jun 9, 2011, at 8:33 PM, Keir Mierle wrote:
>
> On Thu, Jun 9, 2011 at 5:12 PM, Francesco Callari <fgcallari at gmail.com>wrote:
>
>> On Wed, Jun 8, 2011 at 2:17 PM, Keir Mierle <mierle at gmail.com> wrote:
>> > On Wed, Jun 8, 2011 at 1:44 PM, François T. <francoistarlier at gmail.com>
>> > wrote:
>> >>>
>> >>> Regarding file formats: JPEG frames are probably fine; there's little
>> >>> value in tracking on 16-bit data. The frames can even get converted to
>> >>> grayscale, since the tracking ignores color information.
>> >>
>> >>
>> >> are you not planning on using all the depth on the image for tracking ?
>> in
>> >> my experience, it can helps a lot sometimes !
>> >
>> > Maybe eventually but not to start. Libmv takes 32 bit grayscale float
>> images
>> > to track them. However, I'm skeptical that more than 8 bits of grayscale
>> > data would make a difference for tracking. Do you have examples where
>> this
>> > is the case?
>>
>> One data point. When I was at ILM nearly all tracking (and, in fact,
>> nearly all the imagery pipeline save for final rendering) was done on
>> 16bit OpenEXR float images (the "half" pixel format of OepnEXR). Of
>> course the tracking code internally used higher precision for the
>> arithmetics, and was independent of the image format. Generally
>> speaking, OpenEXR is a really nice image library to work with in a
>> graphics + vision environment.
>>
>
> It sounds like supporting higher bit depth is useful. This shouldn't be
> hard since libmv already operates on excessively precise 32 bit floats. It
> will just be a matter of using the appropriate conversion to 32 bit floats
> when extracting patches to track from the underlying footage.
>
> I believe Blender already supports OpenEXR throughout.
>
> Keir
>
>
>>
>> Franco
>>
>>
>> > Keir
>> >
>> >>
>> >> 2011/6/8 Keir Mierle <mierle at gmail.com>
>> >>>
>> >>> On Wed, Jun 8, 2011 at 11:53 AM, Colin Levy
>> >>> <colin at peerlessproductions.com> wrote:
>> >>>>
>> >>>> Yeah, rolling shutter is an *incredible* pain.  It can be corrected
>> to
>> >>>> an extent with tools like The Foundry's "RollingShutter" plugin for
>> Nuke,
>> >>>> which I've heard can make previously un-trackable shots trackable,
>> but you
>> >>>> see it *everywhere* these days -- even in hollywood features.
>> >>>
>> >>> Yes, we will deal with it but it will take some time. It's
>> mathematically
>> >>> impossible to fix perfectly, so the best we can do is approximate
>> fixes.
>> >>> Truly unfortunate.
>> >>>
>> >>>>
>> >>>> I do have some footage I can get you from my Canon XH-A1, which is a
>> >>>> 3-CCD HDV camera.  I sold the camera recently (CMOS is all the
>> rage!), but I
>> >>>> have a ton of footage I can go through.
>> >>>
>> >>> That would be great! The only constraint is going to be calibration.
>> Can
>> >>> you get access to the camera for 10 minutes to take some shots of a
>> >>> calibration pattern?
>> >>> I realize the big tools deal with this automatically, but it's
>> difficult
>> >>> to do automatically and won't be supported in our first release.
>> >>>
>> >>>>
>> >>>> That said, RED's CMOS sensor is waayy better than the "jellocam"
>> sensors
>> >>>> in DSLRs like the 5D or T2i.  (RED's new MX sensor I think is even
>> better).
>> >>>>  So the rolling shutter is not bad -- but it's still noticeable in
>> very
>> >>>> high-motion shots.
>> >>>> I'd say 80% of the RED shots I've got should not pose issues due to
>> >>>> rolling shutter.  The tracking tools I've been using do not account
>> for
>> >>>> rolling shutter, yet I'm able to get very solid tracks with little
>> manual
>> >>>> intervention.  So perhaps still valuable to test with?
>> >>>
>> >>> Sounds like the RED shots will work as long as the motion is not too
>> >>> extreme.
>> >>> Regarding file formats: JPEG frames are probably fine; there's little
>> >>> value in tracking on 16-bit data. The frames can even get converted to
>> >>> grayscale, since the tracking ignores color information.
>> >>>
>> >>> Keir
>> >>>>
>> >>>> --Colin
>> >>>>
>> >>>> On Jun 8, 2011, at 2:32 PM, Keir Mierle wrote:
>> >>>>
>> >>>> Thanks Colin!
>> >>>> One point though: RED uses a CMOS sensor and is subject to rolling
>> >>>> shutter. Do you know how bad it is? Some cameras have very slow
>> sensor
>> >>>> readout (and so terrible distortions with movement) and others have a
>> faster
>> >>>> rolling shutter which still has distortions, but less.
>> >>>> For those who don't know: rolling shutter means that each scanline
>> has a
>> >>>> *different* shutter time! Yes, that means that the image is composed
>> of
>> >>>> 1000's of images, one scanline taken from each, with the camera moved
>> >>>> between images. Extremely annoying, and totally breaks all the camera
>> >>>> tracking assumptions.
>> >>>> I realize that we have to handle rolling shutter but it's a big job
>> and
>> >>>> we won't support it in the first release.
>> >>>> So I repeat my request: does anyone have a camera that doesn't have a
>> >>>> rolling shutter? For example, the Panasonic SD9 has a CCD sensor and
>> so
>> >>>> takes true pinhole images for each frame.
>> >>>> Virtually all modern DSLR's have awful rolling shutter (D90
>> included),
>> >>>> so they are also excluded.
>> >>>> Keir
>> >>>> On Wed, Jun 8, 2011 at 11:00 AM, Colin Levy
>> >>>> <colin at peerlessproductions.com> wrote:
>> >>>>>
>> >>>>> Hey guys,
>> >>>>> I've got several TB worth of 4K material I can offer for tracking.
>> >>>>>  However, the shots will represent a more "real-world" scenario
>> rather than
>> >>>>> the test case shots listed below.
>> >>>>> I don't suppose you want the RED .r3d files, so will TIFF sequences
>> >>>>> work for you?  (Or do you want both?)  I will go through my drives
>> in the
>> >>>>> next few days and upload a few shots to my FTP, if you think this
>> would be
>> >>>>> useful!  Let me know whether you'd like 32-bit images, with all the
>> 'RAW'
>> >>>>> data intact, or if 8 or 16bit will suffice.
>> >>>>> One thing to note regarding resolution: I have noticed that in many
>> >>>>> cases, tracking downrezzed shots is frequently more successful than
>> trying
>> >>>>> to track the raw 4K material.  This may have something to do with
>> the
>> >>>>> default values (the default auto feature search range, for example,
>> may be
>> >>>>> something like 20px rather than 80px, which might be a better
>> setting for
>> >>>>> 4K).  For my projects I usually downrez to 2K before tracking...
>> also
>> >>>>> because it's a lot faster.  :P
>> >>>>> Great discussion.  Can't wait to see these tools develop!
>> >>>>> --Colin
>> >>>>>
>> >>>>> On Jun 8, 2011, at 12:59 PM, François T. wrote:
>> >>>>>
>> >>>>> I'll do all that on saturday with D90.
>> >>>>> cheers
>> >>>>> F.
>> >>>>>
>> >>>>> 2011/6/8 Keir Mierle <mierle at gmail.com>
>> >>>>>>
>> >>>>>> A catalog of the following shots would be useful:
>> >>>>>>
>> >>>>>> General motion: Moving and rotating camera. Camera is /not/ always
>> >>>>>> pointing at one part of the scene.
>> >>>>>> Rotation only: camera on a tripod, rotating around a fixed point
>> (if
>> >>>>>> possible the camera's "Nodal point", but you need a special tripod
>> for that)
>> >>>>>> Translation only: walking sideways keeping the camera pointed
>> forward
>> >>>>>> (e.g. not circling)
>> >>>>>> Translation only: walking towards the scene
>> >>>>>> General motion when the scene is planar (e.g. camera keeps a big
>> >>>>>> billboard in view and only the billboard)
>> >>>>>> Circle strafing around an object; camera is fixated on one part of
>> the
>> >>>>>> scene (intersecting principal ray)
>> >>>>>>
>> >>>>>> The scenes should have good texture so that tracking is easy; we
>> can
>> >>>>>> add challenging scenes later. It's ok if the translation-only
>> scenes have
>> >>>>>> some hand held wobble; that's fine. Boxes with texture are good.
>> Complicated
>> >>>>>> objects like bushes or trees are bad. The scene must be static (no
>> cars,
>> >>>>>> people, birds, godzilla, etc).
>> >>>>>> The camera must be calibrated. There is a libmv calibration target
>> >>>>>> that you can affix to a really flat object. The calibration target
>> must be
>> >>>>>> really flat; something that has a bit of bend is unacceptable and
>> will give
>> >>>>>> poor results.
>> >>>>>>
>> >>>>>>
>> http://libmv.googlecode.com/svn/trunk/extras/calibration/calibration.pdf
>> >>>>>> Take several images of the calibration target (or a video in HD).
>> Get
>> >>>>>> different rotations and fill the frame to get the edges (important
>> for
>> >>>>>> distortion correction). Then run one of the calibration tools to
>> find the
>> >>>>>> intrinsic parameters (focal length in pixels, distortion, skew,
>> etc).
>> >>>>>> Note: Rolling shutter is *horrible* for tracking. We're going to
>> have
>> >>>>>> to handle it at some point, but I am trying to avoid it as long as
>> possible.
>> >>>>>> Do you have access to a CCD style camera without a rolling shutter?
>> >>>>>> Keir
>> >>>>>>
>> >>>>>> On Tue, Jun 7, 2011 at 3:27 PM, Ian Hubert
>> >>>>>> <ian at projectlondonmovie.com> wrote:
>> >>>>>>>
>> >>>>>>> Hey all!
>> >>>>>>> I've been following the conversations, and wanted to throw out
>> that
>> >>>>>>> if anyone needs any sort of footage, I can run out and film stuff.
>> I already
>> >>>>>>> have a few things things filmed- mostly just walking around the
>> city holding
>> >>>>>>> the camera as steady as I can- but if anyone wants anything else
>> >>>>>>> (underexposed, really shaky, steady smooth,
>> panning--but-no-movement,
>> >>>>>>> somebody walking across the frame, stuff like that), just say so!
>> >>>>>>> At this point all I can quickly provide is SD and HD-
>> >>>>>>> I unfortunately don't have access to a 4k camera, but I'll film
>> some random
>> >>>>>>> stuff if I ever get my hands on one.
>> >>>>>>>
>> >>>>>>> --
>> >>>>>>> Ian Hubert
>> >>>>>>>
>> >>>>>>> _______________________________________________
>> >>>>>>> Bf-vfx mailing list
>> >>>>>>> Bf-vfx at blender.org
>> >>>>>>> http://lists.blender.org/mailman/listinfo/bf-vfx
>> >>>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>> _______________________________________________
>> >>>>>> Bf-vfx mailing list
>> >>>>>> Bf-vfx at blender.org
>> >>>>>> http://lists.blender.org/mailman/listinfo/bf-vfx
>> >>>>>>
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>> --
>> >>>>> ____________________
>> >>>>> François Tarlier
>> >>>>> www.francois-tarlier.com
>> >>>>> www.linkedin.com/in/francoistarlier
>> >>>>> _______________________________________________
>> >>>>> Bf-vfx mailing list
>> >>>>> Bf-vfx at blender.org
>> >>>>> http://lists.blender.org/mailman/listinfo/bf-vfx
>> >>>>>
>> >>>>>
>> >>>>> _______________________________________________
>> >>>>> Bf-vfx mailing list
>> >>>>> Bf-vfx at blender.org
>> >>>>> http://lists.blender.org/mailman/listinfo/bf-vfx
>> >>>>>
>> >>>>
>> >>>> _______________________________________________
>> >>>> Bf-vfx mailing list
>> >>>> Bf-vfx at blender.org
>> >>>> http://lists.blender.org/mailman/listinfo/bf-vfx
>> >>>>
>> >>>>
>> >>>> _______________________________________________
>> >>>> Bf-vfx mailing list
>> >>>> Bf-vfx at blender.org
>> >>>> http://lists.blender.org/mailman/listinfo/bf-vfx
>> >>>>
>> >>>
>> >>>
>> >>> _______________________________________________
>> >>> Bf-vfx mailing list
>> >>> Bf-vfx at blender.org
>> >>> http://lists.blender.org/mailman/listinfo/bf-vfx
>> >>>
>> >>
>> >>
>> >>
>> >> --
>> >> ____________________
>> >> François Tarlier
>> >> www.francois-tarlier.com
>> >> www.linkedin.com/in/francoistarlier
>> >>
>> >> _______________________________________________
>> >> Bf-vfx mailing list
>> >> Bf-vfx at blender.org
>> >> http://lists.blender.org/mailman/listinfo/bf-vfx
>> >>
>> >
>> >
>> > _______________________________________________
>> > Bf-vfx mailing list
>> > Bf-vfx at blender.org
>> > http://lists.blender.org/mailman/listinfo/bf-vfx
>> >
>> >
>>
>>
>>
>> --
>> Franco Callari <fgcallari at gmail.com>
>>
>>             EC67 BEBE 62AC 8415 7591  2B12 A6CD D5EE D8CB D0ED
>>
>> I am not bound to win, but I am bound to be true. I am not bound to
>> succeed, but I am bound to live by the light that I have. (Abraham
>> Lincoln)
>> _______________________________________________
>> Bf-vfx mailing list
>> Bf-vfx at blender.org
>> http://lists.blender.org/mailman/listinfo/bf-vfx
>>
>
> _______________________________________________
> Bf-vfx mailing list
> Bf-vfx at blender.org
> http://lists.blender.org/mailman/listinfo/bf-vfx
>
>
>
> _______________________________________________
> Bf-vfx mailing list
> Bf-vfx at blender.org
> http://lists.blender.org/mailman/listinfo/bf-vfx
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.blender.org/pipermail/bf-vfx/attachments/20110611/af6e14d5/attachment-0001.htm 