[Bf-viewport] Render Layers, Object Layers and Viewport

[Brecht Van Lommel]

Hi Julian,

I explained my reasoning in the task, but yes I think the object layer
and render layer unification can't really work. At least not if the
combined layer handles the same use cases that we have object and
render layers for now. What I could imagine working for example is
unifying object layers and groups, or perhaps transferring some
responsibilities from groups to layers, limiting groups to be a
tagging system while layers would become more like USD layers. There's
many possibilities here.

However regardless of the object layers/groups design, the viewport
might work mostly the same? The important layer concept for the
viewport could be the render layer, and each render layer (or the full
scene) would provide some list of visible objects to the viewport and
renderer. If that list is defined by object groups, object layers, or
something else would not be a not a major concern for the viewport
design.


Currently the purpose of render layers is for compositing, either with
compositing nodes in Blender or in an external application. You split
the render up into separate image buffer, and then manipulate and
recompose them, doing things like color correction, integrating with
real footage, NPR effects, etc. The existing goal here is a good
looking final render, not a specific viewport look for workflow
purposes.

Ideally the existing compositing node graphs would work realtime in
the 3D viewport, though I don't know if that would be within the scope
of the viewport project. They would work similar to compositing nodes
now, with the difference that the render layers can be rendered by
either the viewport in addition to a renderer like Cycles, and the
compositing output could be displayed in the 3D viewport with
manipulators on top.


The big unknown for me is to what extent and how you would be able to
customize the viewport for workflow purposes, and which role (render)
layers play in that, if any.

The simplest solution would be for the 3D viewport to provide a number
of settings as it does now, which control the look of the viewport.
These could be made extensible by addons, and there could be ways to
switch between them quickly using presets. It would be interesting to
get a list of uses cases we care about that need control beyond that,
where you need to involve user defined (render) layers.

Some special viewports I can think of:
* Modeller display to show edited / sculpted object with full shading,
and other objects more muted.
* Material overrides or passes used to inspect surface properties like
UVs, texture resolution or tessellation rate.
* Lighting view where all meshes use clay shading, and all helper
objects besides lamps are hidden.

Still that could perhaps be handled better by an addon, as these might
need to take things like modes, selected/active, objects types, ...
into account in a way that is not easily made user configurable
without programming.

Regards,
Brecht.

On Wed, Oct 5, 2016 at 11:52 PM, Julian Eisel <eiseljulian at gmail.com> wrote:
> Hi all,
>
> Yesterday, Dalai, Brecht and myself discussed how the new layer
> management will work together with render layers, object layers and
> the viewport. While designing the new layer manager [1], we left any
> decisions on that pending since we weren't really sure what the best
> approach would be and how to realize it. However, for the ongoing
> viewport design work we probably need to make a decision really soon.
>
> Let's first look on the main user-end requirements the system needs to meet:
> * Apply different drawing styles or compositing on different sets of
> objects (e.g. SSAO only for foreground characters)
> * Be ready for overrides (e.g. override material of all lamps in the scene)
> * Allow to precisely define what each viewport will show
> (Did I forget anything?)
>
> There are two basic approaches to solving all this: Unifying render
> layers and object layers, or keeping them separate (maybe separate
> them further even). Here are my current thoughts on them (a bit biased
> I guess :) )
>
> -- Unifying Render Layers and Object Layers --
> This would mean we get rid of the concept of render layers and just
> integrate them into object layers. It could be an attractive solution,
> but I personally doubt we'll find a design that's going to work. Each
> object layer would need to have a whole bunch of options and there
> would probably be a need for special layer types (so options can be
> attached to multiple layers at once).
> Then there still needs to be some way to store a number of variations
> of the layer tree, mainly concerning visibility and draw options. It
> could be solved by supporting presets but that's basically what render
> layers are - presets. Another idea was to make layer trees
> (un-linkable) data-blocks so each scene could have multiple
> independent object layer trees.
> Whatever solution we'll go with, in the end the user needs to tell the
> render engine (viewport engine is a render engine too) what to render.
> If he doesn't want to render the current main object layer tree (but a
> variation of it), this means either selecting one of the previously
> defined presets, one of the layer tree data-blocks or (locally)
> overriding the main layer tree settings just for this render (e.g. to
> show a different set of objects in different viewports).
> If all these issues are solved, we still have to find a way to nicely
> integrate a UI/workflow for overrides. Compatibility to the old render
> layers might also be a bit hard to accomplish.
>
> -- Keeping Render Layers and Object Layers Separate --
> Keeping the separation solves the issue of allowing layer tree
> variations, because as said, render layers are nothing but presets
> (for layer visibility, with some additional options). Per viewport,
> the user could simply choose to display the main object layer tree or
> a render layer. Render layers could be composited together and have
> different drawing styles.
> We could go even further and allow render layers that don't just
> contain object layers, but also arbitrary sets of objects, even
> defined by a filter (e.g. all objects whose name start with "GEO_").
> And now, if we allow overrides per render layer, users have incredibly
> flexibility.
>
> TBH, I spent lots of time trying to find a way to unify the layer
> systems, but after all I only see it creating more problems than it
> solves. IMHO we would end up creating one overly complex system
> (object layers) out of two pretty simple ones that can work together
> pretty well. After all, user benefit of a unified system is definitely
> questionable.
> Brecht appears to have a similar opinion which he put down here ->
> https://developer.blender.org/T38384#379476
>
> [1] https://wiki.blender.org/index.php/User:Julianeisel/GSoC-2016/UI_Design
>
> -------
>
> Here we go, quite some food for thought, hope you kinda got what I'm
> trying to say ;) We may have to open a developer.blender.org design
> task too, but let's see if we really need lots of discussion.
>
> Cheers,
> - Julian -
> _______________________________________________
> Bf-viewport mailing list
> Bf-viewport at blender.org
> https://lists.blender.org/mailman/listinfo/bf-viewport