[Bf-compositor] Image sampling issue

Fri Nov 8 14:02:42 CET 2013

I've made some progress with the EWA (elliptical weighted average) filter,
which is one of the main grievances with tracking+compositor according to
Sebastian.

http://www.pasteall.org/pic/show.php?id=62224
http://www.pasteall.org/pic/show.php?id=62225

The Plane Track Warp operation now calculates actual partial derivatives
from the plane corners (via extended UV resolve function) instead of trying
to estimate them from insufficient data. This allows the EWA filtering to
use accurate elliptical regions in the image space for filtering. Such
derivatives can be calculated in different ways as pointed out in the
previous mail, the important thing is that they are actually available.

The code is not quite ready yet, i'll put it up for review soon. There are
still some details i need to look into, like optimal filter sizes, some
optimizations and safeguards that can be implemented and just a bunch of
debug code i need to clean up.
@Jeroen: please poke me when you are on IRC, i'd like to discuss this whole
sampling issue a bit to get a broader picture.

On Sun, Nov 3, 2013 at 6:31 PM, Lukas Tönne <lukas.toenne at gmail.com> wrote:

> @Jeroen: I've been looking into the EWA sampling. Here are some thoughts,
> please tell me if this would make any sense, just to see if i understand
> you correctly:
>
> 1) Give each operation an explicit coordinate transform function which,
> given x/y coordinates, calculates both the transformed coordinates u/v of
> the source image(s) as well as the partial derivatives du/dx, du/dy, dv/dx,
> dv/dy (jacobian matrix)
>
> 2) For most nodes this transformation is linear, i.e. the derivatives are
> straightforward. A "LinearTransformOperation" base class or so could
> simplify it then by keeping a transform matrix. Only a few nodes such as
> UVMap (?) node currently do non-linear transforms afaik. PlaneTrack could
> perhaps use a full 3D transform.
>
> 3) Remove integrated pixel transform calculations from executePixel
> functions and let the SocketReader handle this by passing x/y through the
> transform functions and giving source-space coordinates to the input node.
> Nodes such as Translate or PlaneTrack would not do any actual color
> manipulation but only coordinate transforms.
>
> 4) For canvas compositing (later): Add a transform matrix to bNode types
> that support it, with all the necessary UI bells and whistles for
> manipulating these in the editor. Then use these matrices to set up the
> associated operations.
>
>
> On Sun, Nov 3, 2013 at 1:09 PM, Francesco Paglia <f.paglia.80 at gmail.com>wrote:
>
>> Hi Jeroen,
>> as I can see every node that involves pixel transformation gives a much
>> more blurred image.
>> the example in the link clearly shows what I'm talking about.
>>
>> http://www.pasteall.org/pic/show.php?id=61835
>>
>>  there's an image input rotated twice in both direction to get back to
>> the original position then I compared the resulting image with the original
>> one using a difference operand.
>> The image "should" result in a completely black image but as you can see
>> there are difference bigger than a pixel using the bicubic filter.
>>
>>  This result shows up 2 consideration:
>> 1. a sub-pixel transformation should be taken into account for all the
>> distortion nodes (maybe matrices should be the right choice?)
>> 2. there's no optimization for two nodes whom do the same operation the
>> so called "concatenation" in other software.
>>
>> To further explain concatenation is an evaluation of all the node that
>> are of the same kind and that are linked one to each other prior to apply
>> the final transformation to the image.
>> here a reference link:
>>
>> http://www.nukepedia.com/written-tutorials/concatenation-of-transforms-in-nuke/
>>
>>
>>
>>
>>
>> 2013/11/3 Sebastian König <koenig.sebastian at gmx.net>
>>
>>> Hi all!
>>>
>>> I have created a new wiki page:
>>> http://wiki.blender.org/index.php/Dev:Ref/Proposals/Compositor
>>> I will collect the proposals and requests there, and also try to merge
>>> it with requests and proposals from here:
>>> http://wiki.blender.org/index.php/Dev:Ref/Requests/Compositing2.6
>>>
>>> I will also start threads in this list mailing list, as Francesco
>>> suggested, so that we can discuss things more clearly and structured.
>>> What I would like to see at some point is some kind of priority list.
>>> Everyone has his main issues and problems with the compositor and also
>>> probably a pretty clear vision what should be changed and how. Would be
>>> cool if we can get agreement on the most important issues.
>>>
>>> Cheers!
>>>
>>> Seb
>>>
>>>
>>>
>>> On 3. November 2013 at 09:21:30, Jeroen Bakker (j.bakker at atmind.nl<//j.bakker at atmind.nl>)
>>> wrote:
>>>
>>>  Hi!
>>>
>>> Just to clarify the issue, the quick solution and an draft-idea of the
>>> possible solution:
>>>
>>> Issue: the compositor can not handle sub-sampling with good quality by
>>> default. Nodes that want to use it should currently implement a style like
>>> UV mapping (EWA sampling). To use EWA sampling the pixel size needs to be
>>> evaluated (DX, DY).
>>>
>>> Quick solution: let the plane tracker node calculate the pixel-size and
>>> EWA filtering methods. This solution will work for most cases, but is not
>>> the ideal solution. It would be better that the pixel size is known by all
>>> nodes.
>>>
>>> Possible solution (draft idea): In stead of evaluating pixels with just
>>> an X, Y coordinate or even X, Y, DX, DY. We should investigate on using 3x3
>>> matrices. This solution should also help to implement canvas compositor,
>>> but it could lead to strange results as matrices are limited by the
>>> optimization structure (Complex vs Simple nodes) of the compositor.
>>>
>>> Jeroen
>>>
>>> On 11/02/2013 04:48 PM, Sebastian König wrote:
>>>
>>>  Hi!
>>>  Here’s a file that illustrates the problem with sampling of plane
>>> track: http://www.pasteall.org/blend/24971
>>>  It’s a UV grid that is being blurred quite badly, even though the plane
>>> track has only straight lines.
>>>  I was talking to Jeroen and Sergey about that, and they said there are
>>> issues with the entire sampling code in compositor, so it seems not to be a
>>> local issue.
>>>  But I don’t know about the technical issues. It just looks crappy. :)
>>>
>>>  Cheers!
>>>
>>>  Seb
>>>
>>>
>>>
>>>
>>> --
>>> Sebastian König
>>> Schenkendorfstrasse 45
>>> 04275 Leipzig
>>> Germany
>>> +49 176 20319318
>>> www.3dzentrale.com
>>> koenig.sebastian at gmx.net
>>>
>>>
>>> On 2. November 2013 at 13:20:56, Lukas Tönne (lukas.toenne at gmail.com<//lukas.toenne at gmail.com>)
>>> wrote:
>>>
>>>  Sebastian: Could you create a few test files to demonstrate the
>>> sampling issues?
>>>
>>>
>>> _______________________________________________
>>> Bf-compositor mailing list
>>> Bf-compositor at blender.org
>>> http://lists.blender.org/mailman/listinfo/bf-compositor
>>>
>>>
>>> _______________________________________________
>>> Bf-compositor mailing list
>>> Bf-compositor at blender.org
>>> http://lists.blender.org/mailman/listinfo/bf-compositor
>>>
>>>
>>
>>
>> --
>> Francesco Paglia
>> Vfx and Production Supervisor
>>
>> mobile  +39 347.82.12.473
>> e-mail   f.paglia.80 at gmail.com
>>
>>
>> _______________________________________________
>> Bf-compositor mailing list
>> Bf-compositor at blender.org
>> http://lists.blender.org/mailman/listinfo/bf-compositor
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.blender.org/pipermail/bf-compositor/attachments/20131108/b27c3f10/attachment-0001.htm 