[Bf-cycles] Cycles development newbie

Jaros Milan milan.jaros at vsb.cz
Mon Feb 12 08:42:53 CET 2018


Hi,

This book can help too: https://store.blender.org/product/cycles-encyclopedia/

Best regards
Milan

Dne 12. 2. 2018 4:22 napsal uživatel Bob Geller <bobgeller401 at gmail.com>:
Thank you so much for the detailed explanation once again Lukas. I'll have a play around and email the mailing list again if I run into any issues. I'd be happy to write a wiki entry for both the code layout, control flow and shader addition model once I play around and look at the code myself, unless you want to do it yourself Lukas, since it's all your effort. Thanks once again.

Regards,
Bob

On 12 February 2018 at 03:10, Troy Sobotka <troy.sobotka at gmail.com<mailto:troy.sobotka at gmail.com>> wrote:

This effort should be formatted for the Wiki.

With respect,
TJS

On Sun, Feb 11, 2018, 6:40 PM Lukas Stockner <lukas.stockner at freenet.de<mailto:lukas.stockner at freenet.de>> wrote:

Actually, yes, I do have some suggestions for that :)

First, you'll want to add a Shader node to Blender itself. The general way to do that is:
- Add a file for your node in source/blender/nodes/shader/nodes/ - just copy the file of some other BSDF node that is somewhat similar and adapt it. I'd suggest to ignore the GPU/GLSL stuff for the first test, you can always add that later.
- Add your new file in source/blender/nodes/CMakeLists.txt
- Call the register_node_* function of your new node from source/blender/blenkernel/intern/node.c
- Add its prototype to source/blender/nodes/NOD_shader.h
- Allocate a Node ID for it in source/blender/blenkernel/BKE_node.h
- If you have some custom element like a checkbox of an enum in addition to the regular node inputs:
  * Define the RNA type of your node in source/blender/makesrna/intern/rna_nodetree.c
  * Define a drawing function in source/blender/editors/space_node/drawnode.c
- Connect everything by adding a line for your node in source/blender/nodes/NOD_static_types.h
- Add the node to the Node editor menu in release/scripts/startup/nodeitems_builtins.py

Now for the node in Cycles:
- Define the node in intern/cycles/nodes.h and nodes.cpp
- Sync the node in intern/cycles/blender/blender_shader.cpp

I'd recommend to focus on the SVM implementation first and only worry about OSL once that works.

And of course, the actual BSDF in the kernel:
- Add a closure ID in intern/cycles/kernel/svm/svm_types.h
- Add the BSDF implementation in intern/cycles/kernel/closure/ (the shader seems somewhat complex, so a new file probably is best)
- Include your new BSDF file in intern/cycles/kernel/closure/bsdf.h
- Add the eval and sample functions in the switch statements in intern/cycles/kernel/closure/bsdf.h
- Add closure setup in the SVM interpreter in intern/cycles/kernel/svm/svm_closure.h

Oh, and three notes on closure implementation:
- Closures in Cycles return f(wo, wi)*cos_theta_i, not just f(wo, wi) as it's done in most other renderers I've seen so far (pbrt, Mitsuba etc.)
- The I vector in evaluation/sampling is what's usually referred to as wo in other renderers.
- For debugging, you can just not return SD_BSDF_HAS_EVAL from the setup, this will cause Cycles to not call your eval function and not do any light-sampling/bsdf-sampling MIS. Of course, that might cause excessive noise with diffuseish materials and sharp lights.

- Lukas

On 12.02.2018 03:09, Bob Geller wrote:
Thank you so much Lukas for the detailed email. It has definitely helped clear up a few things. Using a debugger was going to the next option but I thought I'd ask the experts first. I will do so anyway and pop any questions on this group if I need any more help. I'm looking to add more BSDFs to the Cycles engine as that forms most of the interest like the Irawan & Marschner woven cloth shader that I see is missing. Any suggestions on adding BSDFs other than the normal closure and osl shader additions? Anyway, hopefully I can get a handle of the code-base quickly and get cracking.  Hope you have a great day! Thanks again!

Regards,
Bob

On 11 February 2018 at 19:21, Lukas Stockner <lukas.stockner at freenet.de<mailto:lukas.stockner at freenet.de>> wrote:
Hi,

regarding the overall layout, there is a Wiki article (https://wiki.blender.org/index.php/Dev:Source/Render/Cycles/SourceLayout) but it doesn't really say much, so here's my attempt on an overview:

Overall, Cycles can be divided into four layers:
- Kernel code - the computations that are executed on the chosen device, doing the actual rendering
- Device code - handles memory management and is able to execute tasks on the different compute devices by calling the kernel
- "Host" code - handles data synchronization to the device and control flow during rendering, exposes an API
- Blender code - Glue logic between Blender and Cycles, synchronizes data from Blender into the Cycles-specific formats
Of those four, the first three are independent of Blender and also used in e.g. the standalone renderer.

When you hit F12, Blender internally starts a rendering job (not going into details here). Effectively, it ends up calling functions in the Cycles addon (blender/addon/, all paths relative to intern/cycles/). These in turn call the Cycles Python API (as defined in blender/blender_python.cpp), which wraps the BlenderSession (in blender/blender_session.cpp).

This Blender session allocates a Cycles Session (render/session.cpp) and Scene (render/scene.cpp) and then fills the Cycles-internal scene representation (consisting of Shaders, Meshes, Lights etc., all defined in render/ and part of the Scene class) from Blender-internal data through the code in BlenderSync (blender/blender_sync.cpp).

Then, it calls the Session to start the rendering. The Session in turn calls the internal Scene synchronization, which then loops over all the components of a Scene and calls their device_update functions, which are responsible for converting the internal host representation into the format that the kernel expects. Once that is done, it divides the image into tiles (render/tile.cpp) and then starts a DeviceTask (device/device_task.cpp) for rendering.

The device (device/device_{cpu, cuda, opencl, multi}.cpp) then calls the acquire_tile callback of the task, which makes the Session return tiles to render. Then, it calls the rendering kernel for that particular device until the tile is finished, and goes back to acquiring another tile until they're all rendered.

In the kernel, most of the code is shared between devices, which means that we can only use the common subset of C++, CUDA and OpenCL (which is generally the limiting factor). To archieve this, whenever any language needs a particular keyword, a ccl_ keyword is defined. Then, for each of the three languages, these ccl_keywords are #defined to the proper keywords in kernel/kernel_compat_<device>.h. For example, CUDA requires each function that's supposed to be executed on the GPU to be prefixed by __device__. Therefore, we have the ccl_device keyword, which gets #defined to __device__ for CUDA and to nothing for CPU and OpenCL. This is all there is to the "cycles compute language" - it's essentially just a bunch of defines that allows to write code that compiles in all three languages.

As a result, for many changes you can completely ignore the fact that your code will also be compiled on GPUs. The main downside is that it completely screws up code completion in most IDEs.
Also, here's the wiki article on the kernel language (a bit outdated unfortunately): https://wiki.blender.org/index.php/Dev:Source/Render/Cycles/Kernel

Generally, to get an overview of the control flow, I'd recommend looking at:
- BlenderSession::create_session, BlenderSession::reset_session, BlenderSession::render
- BlenderSync in general
- Session::run_cpu
- Scene::device_update
- CPUDevice::thread_render
- Session::acquire_tile
- kernel_path_trace

And ultimately, the best approach for figuring out control flow of course always is searching and a debugger.

Hope this helps :)

- Lukas


On 11.02.2018 13:09, Bob Geller wrote:
Hi,

My name is Bob and I'm trying to get into cycles development to hopefully contribute in the future. Is there any documentation to understand the system better and especially understand the flow of control in the renderer? Also, I'm struggling to find enough info about cycles kernel development and the cycles compute language. I would appreciate if someone can point me in the right direction to help understand this better. Thank you for your time and sorry for mailing the whole group, I wasn't sure who to contact.

Regards,
Bob



_______________________________________________
Bf-cycles mailing list
Bf-cycles at blender.org<mailto:Bf-cycles at blender.org>
https://lists.blender.org/mailman/listinfo/bf-cycles



_______________________________________________
Bf-cycles mailing list
Bf-cycles at blender.org<mailto:Bf-cycles at blender.org>
https://lists.blender.org/mailman/listinfo/bf-cycles





_______________________________________________
Bf-cycles mailing list
Bf-cycles at blender.org<mailto:Bf-cycles at blender.org>
https://lists.blender.org/mailman/listinfo/bf-cycles


_______________________________________________
Bf-cycles mailing list
Bf-cycles at blender.org<mailto:Bf-cycles at blender.org>
https://lists.blender.org/mailman/listinfo/bf-cycles

_______________________________________________
Bf-cycles mailing list
Bf-cycles at blender.org<mailto:Bf-cycles at blender.org>
https://lists.blender.org/mailman/listinfo/bf-cycles



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.blender.org/pipermail/bf-cycles/attachments/20180212/9e59eed9/attachment.html>


More information about the Bf-cycles mailing list