[Bf-cycles] Cycles development newbie

Bob Geller bobgeller401 at gmail.com
Mon Feb 12 04:21:32 CET 2018

Thank you so much for the detailed explanation once again Lukas. I'll have
a play around and email the mailing list again if I run into any issues.
I'd be happy to write a wiki entry for both the code layout, control flow
and shader addition model once I play around and look at the code myself,
unless you want to do it yourself Lukas, since it's all your effort. Thanks
once again.


On 12 February 2018 at 03:10, Troy Sobotka <troy.sobotka at gmail.com> wrote:

> This effort should be formatted for the Wiki.
> With respect,
> On Sun, Feb 11, 2018, 6:40 PM Lukas Stockner <lukas.stockner at freenet.de>
> wrote:
>> Actually, yes, I do have some suggestions for that :)
>> First, you'll want to add a Shader node to Blender itself. The general
>> way to do that is:
>> - Add a file for your node in source/blender/nodes/shader/nodes/ - just
>> copy the file of some other BSDF node that is somewhat similar and adapt
>> it. I'd suggest to ignore the GPU/GLSL stuff for the first test, you can
>> always add that later.
>> - Add your new file in source/blender/nodes/CMakeLists.txt
>> - Call the register_node_* function of your new node from
>> source/blender/blenkernel/intern/node.c
>> - Add its prototype to source/blender/nodes/NOD_shader.h
>> - Allocate a Node ID for it in source/blender/blenkernel/BKE_node.h
>> - If you have some custom element like a checkbox of an enum in addition
>> to the regular node inputs:
>>   * Define the RNA type of your node in source/blender/makesrna/
>> intern/rna_nodetree.c
>>   * Define a drawing function in source/blender/editors/space_
>> node/drawnode.c
>> - Connect everything by adding a line for your node in
>> source/blender/nodes/NOD_static_types.h
>> - Add the node to the Node editor menu in release/scripts/startup/
>> nodeitems_builtins.py
>> Now for the node in Cycles:
>> - Define the node in intern/cycles/nodes.h and nodes.cpp
>> - Sync the node in intern/cycles/blender/blender_shader.cpp
>> I'd recommend to focus on the SVM implementation first and only worry
>> about OSL once that works.
>> And of course, the actual BSDF in the kernel:
>> - Add a closure ID in intern/cycles/kernel/svm/svm_types.h
>> - Add the BSDF implementation in intern/cycles/kernel/closure/ (the
>> shader seems somewhat complex, so a new file probably is best)
>> - Include your new BSDF file in intern/cycles/kernel/closure/bsdf.h
>> - Add the eval and sample functions in the switch statements in
>> intern/cycles/kernel/closure/bsdf.h
>> - Add closure setup in the SVM interpreter in
>> intern/cycles/kernel/svm/svm_closure.h
>> Oh, and three notes on closure implementation:
>> - Closures in Cycles return f(wo, wi)*cos_theta_i, not just f(wo, wi) as
>> it's done in most other renderers I've seen so far (pbrt, Mitsuba etc.)
>> - The I vector in evaluation/sampling is what's usually referred to as wo
>> in other renderers.
>> - For debugging, you can just not return SD_BSDF_HAS_EVAL from the setup,
>> this will cause Cycles to not call your eval function and not do any
>> light-sampling/bsdf-sampling MIS. Of course, that might cause excessive
>> noise with diffuseish materials and sharp lights.
>> - Lukas
>> On 12.02.2018 03:09, Bob Geller wrote:
>> Thank you so much Lukas for the detailed email. It has definitely helped
>> clear up a few things. Using a debugger was going to the next option but I
>> thought I'd ask the experts first. I will do so anyway and pop any
>> questions on this group if I need any more help. I'm looking to add more
>> BSDFs to the Cycles engine as that forms most of the interest like the
>> Irawan & Marschner woven cloth shader that I see is missing. Any
>> suggestions on adding BSDFs other than the normal closure and osl shader
>> additions? Anyway, hopefully I can get a handle of the code-base quickly
>> and get cracking.  Hope you have a great day! Thanks again!
>> Regards,
>> Bob
>> On 11 February 2018 at 19:21, Lukas Stockner <lukas.stockner at freenet.de>
>> wrote:
>>> Hi,
>>> regarding the overall layout, there is a Wiki article (
>>> https://wiki.blender.org/index.php/Dev:Source/Render/Cycles/SourceLayout)
>>> but it doesn't really say much, so here's my attempt on an overview:
>>> Overall, Cycles can be divided into four layers:
>>> - Kernel code - the computations that are executed on the chosen device,
>>> doing the actual rendering
>>> - Device code - handles memory management and is able to execute tasks
>>> on the different compute devices by calling the kernel
>>> - "Host" code - handles data synchronization to the device and control
>>> flow during rendering, exposes an API
>>> - Blender code - Glue logic between Blender and Cycles, synchronizes
>>> data from Blender into the Cycles-specific formats
>>> Of those four, the first three are independent of Blender and also used
>>> in e.g. the standalone renderer.
>>> When you hit F12, Blender internally starts a rendering job (not going
>>> into details here). Effectively, it ends up calling functions in the Cycles
>>> addon (blender/addon/, all paths relative to intern/cycles/). These in turn
>>> call the Cycles Python API (as defined in blender/blender_python.cpp),
>>> which wraps the BlenderSession (in blender/blender_session.cpp).
>>> This Blender session allocates a Cycles Session (render/session.cpp) and
>>> Scene (render/scene.cpp) and then fills the Cycles-internal scene
>>> representation (consisting of Shaders, Meshes, Lights etc., all defined in
>>> render/ and part of the Scene class) from Blender-internal data through the
>>> code in BlenderSync (blender/blender_sync.cpp).
>>> Then, it calls the Session to start the rendering. The Session in turn
>>> calls the internal Scene synchronization, which then loops over all the
>>> components of a Scene and calls their device_update functions, which are
>>> responsible for converting the internal host representation into the format
>>> that the kernel expects. Once that is done, it divides the image into tiles
>>> (render/tile.cpp) and then starts a DeviceTask (device/device_task.cpp) for
>>> rendering.
>>> The device (device/device_{cpu, cuda, opencl, multi}.cpp) then calls the
>>> acquire_tile callback of the task, which makes the Session return tiles to
>>> render. Then, it calls the rendering kernel for that particular device
>>> until the tile is finished, and goes back to acquiring another tile until
>>> they're all rendered.
>>> In the kernel, most of the code is shared between devices, which means
>>> that we can only use the common subset of C++, CUDA and OpenCL (which is
>>> generally the limiting factor). To archieve this, whenever any language
>>> needs a particular keyword, a ccl_ keyword is defined. Then, for each of
>>> the three languages, these ccl_keywords are #defined to the proper keywords
>>> in kernel/kernel_compat_<device>.h. For example, CUDA requires each
>>> function that's supposed to be executed on the GPU to be prefixed by
>>> __device__. Therefore, we have the ccl_device keyword, which gets #defined
>>> to __device__ for CUDA and to nothing for CPU and OpenCL. This is all there
>>> is to the "cycles compute language" - it's essentially just a bunch of
>>> defines that allows to write code that compiles in all three languages.
>>> As a result, for many changes you can completely ignore the fact that
>>> your code will also be compiled on GPUs. The main downside is that it
>>> completely screws up code completion in most IDEs.
>>> Also, here's the wiki article on the kernel language (a bit outdated
>>> unfortunately): https://wiki.blender.org/index.php/Dev:Source/Render/
>>> Cycles/Kernel
>>> Generally, to get an overview of the control flow, I'd recommend looking
>>> at:
>>> - BlenderSession::create_session, BlenderSession::reset_session,
>>> BlenderSession::render
>>> - BlenderSync in general
>>> - Session::run_cpu
>>> - Scene::device_update
>>> - CPUDevice::thread_render
>>> - Session::acquire_tile
>>> - kernel_path_trace
>>> And ultimately, the best approach for figuring out control flow of
>>> course always is searching and a debugger.
>>> Hope this helps :)
>>> - Lukas
>>> On 11.02.2018 13:09, Bob Geller wrote:
>>> Hi,
>>> My name is Bob and I'm trying to get into cycles development to
>>> hopefully contribute in the future. Is there any documentation to
>>> understand the system better and especially understand the flow of control
>>> in the renderer? Also, I'm struggling to find enough info about cycles
>>> kernel development and the cycles compute language. I would appreciate if
>>> someone can point me in the right direction to help understand this better.
>>> Thank you for your time and sorry for mailing the whole group, I wasn't
>>> sure who to contact.
>>> Regards,
>>> Bob
>>> _______________________________________________
>>> Bf-cycles mailing listBf-cycles at blender.orghttps://lists.blender.org/mailman/listinfo/bf-cycles
>>> _______________________________________________
>>> Bf-cycles mailing list
>>> Bf-cycles at blender.org
>>> https://lists.blender.org/mailman/listinfo/bf-cycles
>> _______________________________________________
>> Bf-cycles mailing listBf-cycles at blender.orghttps://lists.blender.org/mailman/listinfo/bf-cycles
>> _______________________________________________
>> Bf-cycles mailing list
>> Bf-cycles at blender.org
>> https://lists.blender.org/mailman/listinfo/bf-cycles
> _______________________________________________
> Bf-cycles mailing list
> Bf-cycles at blender.org
> https://lists.blender.org/mailman/listinfo/bf-cycles
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.blender.org/pipermail/bf-cycles/attachments/20180212/8f83e45b/attachment.html>

More information about the Bf-cycles mailing list