[Bf-committers] GSoC 2017: Camera breathing support

[Levon]

>
> ----------------------------------------------------
>
> Message: 1
> Date: Fri, 24 Mar 2017 02:26:38 +0800
> From: Tianwei Shen <shentianweipku at gmail.com>
> Subject: [Bf-committers] GSoC 2017: Camera breathing support
> To: bf-blender developers <bf-committers at blender.org>
> Message-ID: <3B4BA086-116B-417D-A288-C8F1CA7A880F at gmail.com>
> Content-Type: text/plain;       charset=us-ascii
>
> Hi Everyone,
>
> Last summer I participated in GSoC 2016 and worked on the multi-view
> camera reconstruction project. Some of my efforts are summarized in this
> blog: http://hlzz.github.io/blender3/ <http://hlzz.github.io/blender3/>.
> And this patch (https://developer.blender.org/D2187 <
> https://developer.blender.org/D2187>) is now being reviewed and revised.
> This year I would like to apply again and work on the camera breathing
> support, which is already requested by some users during the time I worked
> on the motion tracking project. Now I need clarifications for some specific
> problems.
>
> 1. should we automatic detect the changes of focal lengths, or is it
> specified by users as additional inputs (like the focal length for each
> frame)? I know we can read exif tags to get focal lengths for photos. Do we
> have a similar approach for videos?
>
> 2. Is the current UI able to handle camera breathing, if we need
> additional inputs from users?
>
> I think this project also has something to do with my revisions done on
> the motion tracking system last summer. Hopefully I should be able to merge
> the revisions and move towards the goal of automatic tracking.
>
>
> Thanks,
> Tianwei
>


First of all, the work you did on the previous GSOC has been great. i have
been testing it on and off the last few months, and overall it works quite
well. i do have some ideas that could make workflow a bit better, but ill
leave that for another email.

Lens Breathing can be calculated as a change in Lens distortion parameters
and minor change in focal length for a spherical lens, Anamorphic lens's
have some other issues, so ill focus on spherical for now.

on a prime lens, when you change the focal point from closest to infinity,
the lens will have a slight zoom/crop in, in my experience about 2-3mm in
focal length.

The Lens distortion also will change. depending on the optics this can be
as simple as very slight changes to 1st and 2nd degree distortion, though
on some lenses i have seen the lens going from a barrel distortion to a
Moustache type distortion, as well as a slight change in Focal Length.

anamorphic has the same problems as above, but there is also a noticable
squeeze in height when a change in focal distance. there can also be a
trapazoid tilt effect and other rotation/non uniform distortion if the
anamorphic lens is not mounted correctly.

currently Blenders Distortion model is quite good, K1,K2,K3, and lens
center offset. (syntheyes until the last release only had K1 and lens
center offset, and Pftrack has K1,K2 and an Anamorphic Squeeze),
3Dequalizer has a few different models, (the anamorphic having 16
parameters)

however one issue with Blender's distortion is after the distortion is
calculated, the rX and ry are multiplied by the focal length. this means
even if your lens distortion is correct at 20mm, and the solver solves at
24mm, the lens distortion is now no longer accurate.

This gets even harder if using a zoom lens, or stereo rig. (transformers 4
shot on a stereo rig with two anamorphic zoom lenses :O )

Primarily i use 3dequalizer, which has more advanced lens distortion, which
is open source part of the LDPK.(but not under a formal license not sure
how GPL compatible it is.)
models. https://www.3dequalizer.com/?site=tech_docs&id=110216_01

i have actually coded the 3de_Standard 4 degree lens into blender as an
alternative distortion model. it was more for using in the Compositor so i
didnt finish integrating it into the solving engine.
http://pasteall.org/69753/diff

to answer your questions..

1) some cameras do record the focal length and focus distance, but that i
know of, only the Red's and Arri XT's, arri 65 do.

Generally I would get multiple lensgrids at recorded focal Distances, then
solve an animated Focus distance in a way that minimises the error for each
frame, sometimes this works, sometimes it returns illogical results.

2) if we were able to get a solved lens distortion for the length of the
footage, it would be something lie animating the cameras Focal Distance,
and either changing focus in the compositor, or having a lens distortion
model built into Cycles render. i recall there was a branch that had
lens/camera Shaders that would allow for this, but i dont know how far it
got.

i could talk about lens distortion for days, so feel free to email me off
list if you want.

Levon