[Bf-cycles] Feature: Recoloring of lights and materials post-render.

Peter Schmidt-Nielsen schmidtnielsenpeter at gmail.com
Thu Oct 11 18:26:57 CEST 2018

Hi Brecht,

Thanks for the very quick response.

For lights groups, there is a patch here which we should get committed at
> some point:
> https://developer.blender.org/D3607

Very cool to see, I think I haven't been keeping up; I don't think this was
there yet when I last was asking around about this in #blendercoders in
August 2017.

For changing shader colors, personally I'm not entirely convinced it's
> worth the code complexity. The trend is towards interactive previews and
> realtime raytracing, and with a finite amount of development time to me it
> seems more useful to improve that workflow. The other reservation I have
> about this is if it really works in the more complex scenes where almost
> every surface is textured. You might have hair, SSS, volumes, transparency,
> and it just gets harder and harder to do a meaningful approximation. But
> it's the more complex scene where relighting is most useful.

I agree that the code complexity and implementation effort question is
pretty important, especially if the the general trend is towards enabling
an artistic workflow of rapidly tweaking material colors by simply having
very fast previews. In a world where most scenes can render some reasonable
approximation to the viewport nearly instantly this sort of feature is much
less compelling, such as with Eevee, as Zauber points out. This seems like
a very valid criticism, especially given limited development time.

However, I think it is possible to do this sort of shader relighting even
in arbitrarily complex scenes, for the simple algebraic reason that a
path's throughput is a product of inverse PDFs and BsdfEvals that it
accumulates, and therefore if we can write the BsdfEvals as affine
functions of the (unknown at render time) material parameters, then the
final PathRadiance will necessarily be a polynomial in these unknown
parameters, regardless of how complicated any other BSDFs are, or if other
complicated phenomena are modeled (SSS, volumetrics, etc.).

To put it in other words, we are pretty limited in the sorts of material
parameters we can relight (only those that result in an affine response on
the material's BsdfEvals and inverse PDFs). However, we are unconstrained
in what other phenomena occur elsewhere in the scene in *other* materials.
For example, one can't relight the color of a volumetric scatter, or change
a glossy BSDF's roughness, or change an SSS color, because this results in
non-affine changes in the material's properties, and effects on
throughputs. However, one *can* relight an object (whose material admits
relighting) even in the presence of these other fancier materials on *other*
objects. This is not an approximation: if done with LPEs in Iray, the
relighting is exact, even with all these fancier materials elsewhere (not
on the recolored object) in the scene.

My suggested implementation even allows recoloring of textured objects that
use normal maps, etc.; it's all fine so long as the recoloring is applied
multiplicatively (in scene referred terms; see the example complex material
from my write-up PDF). (Another valid criticism of my suggested feature is
that it's unclear if multiplicatively recoloring an object in scene
referred terms is an artistically useful thing to be able to do efficiently

Apologies for the long-winded explanation above. The above may have already
all been obvious to you, but I figured I'd spell it out exactly, because it
was not obvious to me just how powerful this technique is (and that it's
generally exact, and not an approximation, even with complicated phenomena
elsewhere in the scene) until I read the Nvidia paper and thought about it
for a while.

It really depends what this is intended to be used for though. For example
> I can imagine shader relighting being greater for e.g. an interior design
> application where you want to the user to be able to pick a custom color,
> or you want to save render time when rendering many variations (maybe
> that's the kind of thing iRay had in mind?). It's less obvious to me how it
> fits into an artistic workflow, where you are generally changing many
> settings at will, or navigating the viewport, and it's hard to pick a few
> settings in advance that you want to get a quick preview of later. The
> intended use cases should then also information how the UI works.

This also seems like a very valid criticism. My proposed shader relighting
design would make it prohibitively expensive to be able to recolor more
than a couple of different materials, and maybe that just doesn't fit that
well into a typical artistic workflow.

Thanks again for your quick reply. I'll probably poke around a little bit
more, and look at the patch you linked, to see if I could plausibly even
prototype the shader relighting.

-Peter Schmidt-Nielsen

On Thu, Oct 11, 2018 at 4:34 PM Peter Schmidt-Nielsen <
> schmidtnielsenpeter at gmail.com> wrote:
>> Hello folks,
>> I'd like to add a feature to Cycles (and Blender) to support post-render
>> recoloring of lights, and limited recoloring of materials. Post-render
>> recoloring of lights is a feature implemented in Maxwell, which they call
>> "multilights". Limited recoloring of materials can be achieved in Iray
>> using their Light Path Expressions. The idea is to support both kinds of
>> post-render relighting with a simple design, although not support the full
>> generality and flexibility of Iray's LPEs.
>> I think Cycles' users would benefit enormously from having such a
>> feature. The end user experience would be that a user could change color
>> parameters in specially marked lights and materials after the render, and
>> the render would update instantly to a completely accurate version of what
>> the render would have yielded with the different color parameters.
>> I have a description of my plan, which I wrote up about a year ago when I
>> first started poking around Cycles:
>> http://web.mit.edu/snp/Public/polynomialrelighting.pdf
>> However, I'm not very familiar with the Cycles code base beyond the very
>> small amount of poking I did a year ago, and would love some advice and
>> direction on how feasible it is to implement this feature.
>> A TL;DR for my write-up linked above:
>> 1) One can implement Maxwell-style "multilights" by just having one
>> render buffer for each relightable light, and writing path contributions
>> into the appropriate light's buffer.
>> 2) I think one can implement Iray style post-render changes to material
>> colors without implementing a full-on LPE system, if you just track a
>> little extra data with PathRadiances and throughputs, and have a bunch of
>> extra render buffers.
>> Is anyone else interested in working on such a feature, or at the very
>> least giving me some advice on this?
>> Thanks,
>> -Peter Schmidt-Nielsen
>> P.S. I mentioned this about a year ago in #blendercoders, but I got busy
>> and didn't keep working on it. A few months ago someone in #blendercoders
>> recommended that I ask here.
>> _______________________________________________
>> Bf-cycles mailing list
>> Bf-cycles at blender.org
>> https://lists.blender.org/mailman/listinfo/bf-cycles
> _______________________________________________
> Bf-cycles mailing list
> Bf-cycles at blender.org
> https://lists.blender.org/mailman/listinfo/bf-cycles
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.blender.org/pipermail/bf-cycles/attachments/20181011/63814010/attachment.html>

More information about the Bf-cycles mailing list