[Bf-vfx] Blender camera tracking approach

Sun May 8 23:05:20 CEST 2011

Tobias,

here is a 'complete' workflow for tracking, it might be good to
provide details for some of your flow areas off to the side.

1) load image data - video or sequence of images
2a) (optional) correct rolling shutter
2b) (optional) input known camera parameters
2c) (optional) provide hints to camera lens undistortion solver
(identify straight lines in the image) to help with undistorting the
image
2d) (optional) undistort the camera lens using a distortion solver
3) (optional) create mask for data you don't want tracked (or for
'grouping'/seperating of tracking ie tracking the motion of a camera
and tracking the motion of an object)
4) (optional) adjust color/contrast/etc so that feature points to
track have increased contrast and are thus easier for the tracker to
find them (and easier for the tracker to ignore stuff you don't want
tracked - ie uneven lighting on a green screen creating spurious
motion)
5) (optional) select specific features you want tracked (either by
mask or placement of tracking markers)
5a) if you specified specific features to the tracker you may also
want to specify how the points should be tracked (bounding box for the
tracker to look for the point in; whether the object being tracked is
'rigid' or can deform, and knowledge of the types of camera/object
motion - object translation; rotation; camera translation; rotation)
6) send image data, specified track data; camera data; and mask data to tracker
7) tracker library can automatically identify feature points to track
and or use user specified features, ignoring areas that are masked
8) do a 'solve' of the camera motion based on the track points,
including statistics on how 'good' the tracks are in contributing to
the solve
9) return track point data and camera solve to software, including the
statistical analysis of the track points and their 3d projection
10) based on the statistical analysis pick error thresholds for what
track points to automatically delete (automatic threshold picking or
manually adjusted)
11) (optional) manually delete any track points
12) (optional) create a mask to 'hide' unwanted track points
12a) (optional) mask can be assigned to follow a set of track points
to automatically mask a moving object from the tracker/solver
12b) (optional) mask can be manually keyframed so that it moves and
deforms over time to mask a moving object from the tracker/solver
12) (optional) provide a manually created camera curve to 'hint' the
tracker/solver what you expect the actual curve to look like
13) retrack if additional tracker points are now needed
14) pass the tracker points, camera hints, etc. to the camera solver
15) return the solved camera track and the 3d location of the track
points to the software
16) visualize the camera motion and track points in the 3d view
17) define a ground plane reference to the 3d track points and camera
18) define the scene origin relative to the 3d track points and camera
19) define the world scale and orientation relative to the 3d track
points and data
20) add a test object into the 3d view and see if it stays in the
proper location
21) (optional) stabalize the camera view based on the solved camera track
22) (optional) smooth the camera track curve
23) repeat until the error in the solve is good enough

On Sun, May 8, 2011 at 12:33 PM, Tobias Kummer <supertoilet at gmx.net> wrote:
> After some more tinkering, I came up with the following revised proposal:
> http://www.pasteall.org/pic/12084
>
> This takes some of your (Remos) concerns into account.
>
> Regards,
>
> Tobi
>
> On 05/08/2011 10:09 PM, Remo Pini wrote:
>> Hm... I have never noticed the slight distortions to cause issues with
>> the tracker... might be a question of the implementation, but I don't
>> see, why the material would have to be converted twice, even IF
>> "undistorting" is required...
>>
>> (1) take original footage
>> (2) undistort to track (temporary copy)
>> (3) track
>> (4) distort tracking data according to undistort
>> (5) use original footage + track data + lensinfo from source (or
>> undistort function)
>>
>> Syntheyes seems to do that under the hood, I guess, never seen tracking
>> issues due to distortions...
>>
>>
>>> -----Original Message-----
>>> From: bf-vfx-bounces at blender.org [mailto:bf-vfx-bounces at blender.org]
>>> On Behalf Of Tom M
>>> Sent: Sonntag, 8. Mai 2011 8:20
>>> To: Blender motion tracking&  VFX
>>> Subject: Re: [Bf-vfx] Blender camera tracking approach
>>>
>>> On Sun, May 8, 2011 at 10:06 AM, Remo Pini<remo.pini at avexys.com>
>>> wrote:
>>>> I kind of disagree...
>>>>
>>>> If you have lens distortion in you footage, this would most likely
>> be
>>>> bad and you would not want the final result to have that in it
>> anymore,
>>>> sooo....
>>> You are thinking extreme distortion - this is about the very slight
>>> distortion that comes from all camera lenses due to their lense shape
>>> (as well as lense imperfections - but the imperfections are mostly
>>> ignored by most lense solvers).
>>>
>>> Our brains ignore the distortion but it confuses trackers since travel
>>> in a straight line isn't straight with the distortion.
>>>
>>> Undistorting the lense reduces the image quality because it shifts
>>> pixel locations and thus 'blurs' pixels that the undistorted location
>>> of a pixel will not be exactly on the pixel boundary of a previous
>>> pixel.
>>>
>>> Thus undistortion is only used to provide straighter lines to the
>> tracker.
>>> Applying the lens distortion to your rendered image is so that it's
>>> straight lines are curved similarly to that of the original image.
>>> Again this introduces blurring of your render if it is applied as a
>>> post process - so either you need to render larger or do the
>>> distortion at render time.  In practice the amount of bluring from
>>> doing it as a post process is probably not significant enough to be
>>> noticable in most situations.
>>>
>>> LetterRip
>>> _______________________________________________
>>> Bf-vfx mailing list
>>> Bf-vfx at blender.org
>>> http://lists.blender.org/mailman/listinfo/bf-vfx
>> _______________________________________________
>> Bf-vfx mailing list
>> Bf-vfx at blender.org
>> http://lists.blender.org/mailman/listinfo/bf-vfx
>>
> _______________________________________________
> Bf-vfx mailing list
> Bf-vfx at blender.org
> http://lists.blender.org/mailman/listinfo/bf-vfx
>