<div dir="ltr">Not to press the issue, but the material system wouldn't need to regenerate the script text for loop-back use, and the current python export would be bypassed completely. All the uniform conversion routines would be sitting compiled and ready in the interpreter context, ready to be called via FFI whenever. Performance is likely to not be an issue, especially compared to the current cost of regenerating all the GLSL shaders, plus all the linear algebra is performed in C via FFI. Adding some SSE intrinsics based fast paths for the linear algebra code can probably help diminish the overhead of the FFI barrier on certain platforms too.</div><div class="gmail_extra"><br><div class="gmail_quote">On Wed, Jun 10, 2015 at 2:34 PM, Antony Riakiotakis <span dir="ltr"><<a href="mailto:kalast@gmail.com" target="_blank">kalast@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">I don't think that for blender a live import of the exported code is such a good idea since material shaders get invalidated quite often mostly from user tweaks. Apart from that we would surely be interested in fixes to the gpu codegen module<br></div><div class="HOEnZb"><div class="h5"><div class="gmail_extra"><br><div class="gmail_quote">On 10 June 2015 at 14:24, Neal Alexander <span dir="ltr"><<a href="mailto:nhalxr@gmail.com" target="_blank">nhalxr@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div>Regarding GLSL export - please do not repeat the same mistakes from the current export mechanism. Due to certain basic bugs I've fixed in it, I suspect that few people are actively using it. This is a shame, because blender's code provides a lot of potentially reusable functionality. </div><div><br></div><div>I've noticed people mentioning the need for a better way to swap blender materials. So, for the last month, I've been working on the current GLSL shader export pipeline to deal with several of it's shortcomings. One of the major issues with the old exporter, aside from being unusable "out of the box", was that several of the uniform inputs (for example lamp-energy, lamp-spotsize, lamp-spotblend) are silently modified by the renderer before being bound, and the python GPU/export module, and it's documentation, are always out of sync with whats actually happening in gpu_material.c. </div><div><br></div><div>For GLSL shader export to work effectively in the long term, you need to export your uniform pre-processing code - either as a standalone C library, or as a side script included with the shaders. The most maintainable and bug free scenario is having blender's real-time renderer load materials directly from the same format that it exports, with zero internal conversion. The current method of simply documenting the engines internal conversion routines and forcing the users to roll their own isn't working out. It suffers from the aforementioned documentation de-sync, and also subtle state-update related issues - spending hours debugging visual discrepancies only to find out a vector was being implicitly normalized, in some far away unrelated code block, really sucks.</div><div><br></div><div><br></div><div>I've solved some of these problems, but haven't discussed the project with anyone else, so I'm not sure how likely the changes will be merged. Right now, I have some blender materials with spotlights, buffer shadows, and textures working in OpenSceneGraph (arbitrarily chosen).</div><div><br></div><div>A full specification of the input uniforms and their conversion from blender native values was required, so I extended the current python GPU export script to generate an AST, exporting it directly to s-expr format. </div><div><br></div><div>The homoiconicity of that format allows for a lot of flexibility:</div><div><br></div><div>- If the user wants to treat the text as a plain data format, it's equivalent to a compact instance of other tree based formats such as XML or JSON, yet more straight-forward to parse.</div><div>- Functions for uniform conversion are included directly in the file. This helps as far as maintainability and compatibility are concerned - everything is self contained. </div><div>- Since the format is simultaneously valid Scheme code, the user can setup a restricted environment, define what a few functions mean to their current context (uniform-bind) and execute the script.</div><div>- Since a drastically reduced subset of Scheme is used, a variety of embedded interpreters and implementations are available, most of which are less code than an XML parser, and some of which provide native code compilation. The project currently is using s7-scheme, which is just one C file, though we can use something even smaller.</div><div>- Since s-exprs are isomorphic to abstract syntax trees, source to source translation is made easier for people wanting to convert to another language. Currently, the project's C headers and s7 C-FFI binding code are auto-generated via this method.</div><div>- Having executable code in file formats can be frowned upon, but were already doing it by providing shader code. Anyway, the uniform script is only executable if the user chooses to interpret it as so, and then only in a restricted environment where nothing can be performed but mathematical operations. In a sense, it is not much different than GLSL code.</div><div><br></div><div>The intended outcome is that given an exported blender GLSL material, a user needs only to load the GLSL shaders, define a handful of callbacks, eval() the uniform </div><div>script, and execute the shader program to produce a test render with the default params, looking identical to the result seen in blender's view-port or material preview. Material params and camera info are manipulated by modifying the scheme environment.</div></div><div><div><div class="gmail_extra"><br><div class="gmail_quote">On Wed, Jun 10, 2015 at 10:11 AM, Olivier Parisy <span dir="ltr"><<a href="mailto:olivier.parisy@gmail.com" target="_blank">olivier.parisy@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><p dir="ltr">I was not advocating for nodes to express loops and branching; I feel instead that they would be better expressed with GLSL scripts nodes. </p>
<p dir="ltr">I am just wondering what inputs/outputs of scripts containing such loops would look like. It's probably best to restrict script node to have a standard "main" function and uniforms/variables indeed. That's an easy to understand and well defined way to modularize code.<br>
</p><div><div>
<br><div class="gmail_quote">Le mer. 10 juin 2015 08:30, Daniel Stokes <<a href="mailto:kupomail@gmail.com" target="_blank">kupomail@gmail.com</a>> a écrit :<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div>Lots of discussion here, sorry I am a little slow to respond here. Hopefully I hit most of the things covered so far:<br></div><div><br>Jacob, I am not sure what you are referring to with a core profile with regards to shaders. There is an OpenGL core profile, but there is no point in us restricting ourselves by following (better to use the compatibility profile).<br><br></div><div>I certainly see GLSL node(s) being mixed with existing nodes. They would be a great way to create utility nodes or otherwise allow users to patch gaps in node system.<br><br></div><div>I agree that GLSL export would be a nice to have.<br><br></div><div>I am not sure about adding too many new nodes for things like branching and looping. Mostly I am concerned about adding nodes that a renderer doesn't actually use. I still don't see how that is intended to be handled.<br></div><div><br></div><div>I don't see much need for a wrapper language, we might as well use straight GLSL. The inputs and outputs should be easy to parse, and is something we already have if we are only looking at the in and out variables of functions (as opposed to uniforms and in/out variables). We may add some custom language things via pragmas, but that is likely as far as we would go with any kind of custom language.<br><br>I am still a little confused about when the GLSL
node will be available. Mostly, what render engine needs to be selected
in order to use it, or will that selection even matter?<br><br></div><div>Regards,<br></div><div>Daniel<br></div><div><br></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Tue, Jun 9, 2015 at 4:10 AM, Khalifa Lame <span dir="ltr"><<a href="mailto:khalibloo@gmail.com" target="_blank">khalibloo@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Here's an idea then. Use a "wrapper language" like unity's shaderlab. this wrapper language can simply be a python class written by the user, that defines the input variables for the GLSL shader and their ui layout/representation.<div><br></div><div>also, i think there should be a way to load in shader code from text files via the materials panel (without using nodes).</div><div><br></div><div>as for the shader stages, what if we had "node subtrees"? the user selects the glsl node tree and then he has the option to select a vertex shader subtree or a fragment/tessellation subtree. the outputs from a vertex shader subtree for example, can be fed into the next subtree by simply declaring inputs with similar IDs.</div></div><div class="gmail_extra"><div><div><br><div class="gmail_quote">On Tue, Jun 9, 2015 at 11:07 AM, Antony Riakiotakis <span dir="ltr"><<a href="mailto:kalast@gmail.com" target="_blank">kalast@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div>We already have a custom parser in gpu_codegen.c, see gpu_parse_functions_string. This can be improved upon if needed.<br></div>I don't think loops or conditionals will complicate things but any "out" variables of a shader function will need to be written to, to guarantee there are no garbage values. But this will be up to the author of the shader.<br></div><div><div><div class="gmail_extra"><br><div class="gmail_quote">On 9 June 2015 at 07:58, Olivier Parisy <span dir="ltr"><<a href="mailto:olivier.parisy@gmail.com" target="_blank">olivier.parisy@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><p dir="ltr">Well, didn't want to clutter the list, but since you're asking for a feedback... <img goomoji="263a" style="margin:0 0.2ex;vertical-align:middle;max-height:24px" alt="☺" src="cid:263a@goomoji.gmail"></p>
<p dir="ltr">My understanding is that this would be a productive, modern way to design GLSL shaders, due to an expressive combination of "classical" nodes and code fragments and real-time feedback. Very empowering, and I suppose this could even be used for postprocessings or, as a strech, "demo-like" oddities à la shadertoy.</p>
<p dir="ltr">In this regard, being able to export generated GLSL code, even unoptimized (as is currently possible) would be a must. </p>
<p dir="ltr">One question: do you feel control structures such as branching or loops would complicate matters? I suppose they would be a good use case for GLSL script nodes. Those nodes may also be a way to code shaders in a more modular way, without resorting to tricks such as a preprocessor.</p>
<p dir="ltr">As already stated, automatic creation of typed inputs/outputs for those nodes will require some GLSL parsing capabilities. Does the blender code base already contain such a parser? If not, what is the preferred parsing strategy in blender? Grammars compilers / code generators? Ad-hoc, handcrafted ones?</p>
<p dir="ltr">Regards,<br>
Olivier. </p><div><div>
<br><div class="gmail_quote">Le lun. 8 juin 2015 23:59, Mike Erwin <<a href="mailto:significant.bit@gmail.com" target="_blank">significant.bit@gmail.com</a>> a écrit :<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div>Re: multiple shader stages</div><div><br></div><div>Some of the wireframe shaders I prototyped are multi-stage, with work split between vertex and fragment. So yes we'll need to do *at least* those. Geometry stage in the near future or maybe for the initial release. How do we visually designate stages in the UI, since as Daniel points out most so far could be lumped into the fragment category?</div><div><br></div>Here's what I see in my head:<div>Inputs to the vertex shader node come from geometry data source (as attributes) or from other nodes (as uniforms). Inputs to the fragment shader node come from vertex shader outputs directly or from other nodes (again as uniforms). Use consistent input/output names and wires automatically connect. So in the basic case we have 3 things wired together: data source --> vertex --> fragment. Can't wait to see this on screen!</div><div><br></div><div>Does anyone else here envision mixing of the new GLSL shader nodes and existing nodes?</div><div><br></div><div>Watching the Guilty Gear Xrd presentation now...</div></div><div class="gmail_extra"></div><div class="gmail_extra"><br clear="all"><div><div>Mike Erwin<br>musician, naturalist, pixel pusher, hacker extraordinaire</div></div></div><div class="gmail_extra">
<br><div class="gmail_quote">On Mon, Jun 8, 2015 at 5:00 PM, Antony Riakiotakis <span dir="ltr"><<a href="mailto:kalast@gmail.com" target="_blank">kalast@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hi Daniel,<br>
<br>
The idea is to make a new system that will be powerful enough allow<br>
the game engine to use it, but I expect the game engine to adapt to it<br>
rather than the opposite. The initial plan was to not have blender<br>
internal compatibility at all.<br>
<br>
The node system already has compatibility flags so nodes can set the<br>
engine(s) they are compatible with. I expect many existing nodes will<br>
need little modification to run on new OpenGL. Most of the code that<br>
needs to be changed is the uniform and attribute declarations and this<br>
is handled internally in the gpu_codegen module.<br>
<br>
For the script nodes, initial plan was to make a fragment shader node<br>
at first, but of course we should make it possible to hook more shader<br>
stages, perhaps by using many text data blocks on the node itself. If<br>
there are constraints that would be nice to have now would be the time<br>
to express them I guess. The problem of compatibility is again<br>
bypassed by ignoring it. Any shiny new shader nodes go only to new<br>
viewport. Obviously shader stages can only be executed in a system<br>
that supports them. Might be worth defining alternative node trees for<br>
system without some shader stages but this becomes too technical very<br>
quickly and can get out of hand.<br>
Let's focus on high level functionality first.<br>
<br>
For the material panel it's more of a UI issue. I agree it would be<br>
nice to expose an interface in a more meaningful way. I think node<br>
groups can give us some tools to optimize this workflow somewhat if we<br>
can expose their input interface in the material panel.<br>
<div><div>_______________________________________________<br>
Bf-viewport mailing list<br>
<a href="mailto:Bf-viewport@blender.org" target="_blank">Bf-viewport@blender.org</a><br>
<a href="http://lists.blender.org/mailman/listinfo/bf-viewport" target="_blank">http://lists.blender.org/mailman/listinfo/bf-viewport</a><br>
</div></div></blockquote></div><br></div>
_______________________________________________<br>
Bf-viewport mailing list<br>
<a href="mailto:Bf-viewport@blender.org" target="_blank">Bf-viewport@blender.org</a><br>
<a href="http://lists.blender.org/mailman/listinfo/bf-viewport" target="_blank">http://lists.blender.org/mailman/listinfo/bf-viewport</a><br>
</blockquote></div>
</div></div><br>_______________________________________________<br>
Bf-viewport mailing list<br>
<a href="mailto:Bf-viewport@blender.org" target="_blank">Bf-viewport@blender.org</a><br>
<a href="http://lists.blender.org/mailman/listinfo/bf-viewport" target="_blank">http://lists.blender.org/mailman/listinfo/bf-viewport</a><br>
<br></blockquote></div><br></div>
</div></div><br>_______________________________________________<br>
Bf-viewport mailing list<br>
<a href="mailto:Bf-viewport@blender.org" target="_blank">Bf-viewport@blender.org</a><br>
<a href="http://lists.blender.org/mailman/listinfo/bf-viewport" target="_blank">http://lists.blender.org/mailman/listinfo/bf-viewport</a><br>
<br></blockquote></div><br><br clear="all"><div><br></div></div></div><span><font color="#888888">-- <br><div>khalibloo®<br><br></div>
</font></span></div>
<br>_______________________________________________<br>
Bf-viewport mailing list<br>
<a href="mailto:Bf-viewport@blender.org" target="_blank">Bf-viewport@blender.org</a><br>
<a href="http://lists.blender.org/mailman/listinfo/bf-viewport" target="_blank">http://lists.blender.org/mailman/listinfo/bf-viewport</a><br>
<br></blockquote></div><br></div>
_______________________________________________<br>
Bf-viewport mailing list<br>
<a href="mailto:Bf-viewport@blender.org" target="_blank">Bf-viewport@blender.org</a><br>
<a href="http://lists.blender.org/mailman/listinfo/bf-viewport" rel="noreferrer" target="_blank">http://lists.blender.org/mailman/listinfo/bf-viewport</a><br>
</blockquote></div>
</div></div><br>_______________________________________________<br>
Bf-viewport mailing list<br>
<a href="mailto:Bf-viewport@blender.org" target="_blank">Bf-viewport@blender.org</a><br>
<a href="http://lists.blender.org/mailman/listinfo/bf-viewport" target="_blank">http://lists.blender.org/mailman/listinfo/bf-viewport</a><br>
<br></blockquote></div><br></div>
</div></div><br>_______________________________________________<br>
Bf-viewport mailing list<br>
<a href="mailto:Bf-viewport@blender.org" target="_blank">Bf-viewport@blender.org</a><br>
<a href="http://lists.blender.org/mailman/listinfo/bf-viewport" target="_blank">http://lists.blender.org/mailman/listinfo/bf-viewport</a><br>
<br></blockquote></div><br></div>
</div></div><br>_______________________________________________<br>
Bf-viewport mailing list<br>
<a href="mailto:Bf-viewport@blender.org">Bf-viewport@blender.org</a><br>
<a href="http://lists.blender.org/mailman/listinfo/bf-viewport" target="_blank">http://lists.blender.org/mailman/listinfo/bf-viewport</a><br>
<br></blockquote></div><br></div>