[Bf-cycles] Cycles development newbie

[Lukas Stockner]

Hi,

regarding the overall layout, there is a Wiki article 
(https://wiki.blender.org/index.php/Dev:Source/Render/Cycles/SourceLayout) 
but it doesn't really say much, so here's my attempt on an overview:

Overall, Cycles can be divided into four layers:
- Kernel code - the computations that are executed on the chosen device, 
doing the actual rendering
- Device code - handles memory management and is able to execute tasks 
on the different compute devices by calling the kernel
- "Host" code - handles data synchronization to the device and control 
flow during rendering, exposes an API
- Blender code - Glue logic between Blender and Cycles, synchronizes 
data from Blender into the Cycles-specific formats
Of those four, the first three are independent of Blender and also used 
in e.g. the standalone renderer.

When you hit F12, Blender internally starts a rendering job (not going 
into details here). Effectively, it ends up calling functions in the 
Cycles addon (blender/addon/, all paths relative to intern/cycles/). 
These in turn call the Cycles Python API (as defined in 
blender/blender_python.cpp), which wraps the BlenderSession (in 
blender/blender_session.cpp).

This Blender session allocates a Cycles Session (render/session.cpp) and 
Scene (render/scene.cpp) and then fills the Cycles-internal scene 
representation (consisting of Shaders, Meshes, Lights etc., all defined 
in render/ and part of the Scene class) from Blender-internal data 
through the code in BlenderSync (blender/blender_sync.cpp).

Then, it calls the Session to start the rendering. The Session in turn 
calls the internal Scene synchronization, which then loops over all the 
components of a Scene and calls their device_update functions, which are 
responsible for converting the internal host representation into the 
format that the kernel expects. Once that is done, it divides the image 
into tiles (render/tile.cpp) and then starts a DeviceTask 
(device/device_task.cpp) for rendering.

The device (device/device_{cpu, cuda, opencl, multi}.cpp) then calls the 
acquire_tile callback of the task, which makes the Session return tiles 
to render. Then, it calls the rendering kernel for that particular 
device until the tile is finished, and goes back to acquiring another 
tile until they're all rendered.

In the kernel, most of the code is shared between devices, which means 
that we can only use the common subset of C++, CUDA and OpenCL (which is 
generally the limiting factor). To archieve this, whenever any language 
needs a particular keyword, a ccl_ keyword is defined. Then, for each of 
the three languages, these ccl_keywords are #defined to the proper 
keywords in kernel/kernel_compat_<device>.h. For example, CUDA requires 
each function that's supposed to be executed on the GPU to be prefixed 
by __device__. Therefore, we have the ccl_device keyword, which gets 
#defined to __device__ for CUDA and to nothing for CPU and OpenCL. This 
is all there is to the "cycles compute language" - it's essentially just 
a bunch of defines that allows to write code that compiles in all three 
languages.

As a result, for many changes you can completely ignore the fact that 
your code will also be compiled on GPUs. The main downside is that it 
completely screws up code completion in most IDEs.
Also, here's the wiki article on the kernel language (a bit outdated 
unfortunately): 
https://wiki.blender.org/index.php/Dev:Source/Render/Cycles/Kernel

Generally, to get an overview of the control flow, I'd recommend looking at:
- BlenderSession::create_session, BlenderSession::reset_session, 
BlenderSession::render
- BlenderSync in general
- Session::run_cpu
- Scene::device_update
- CPUDevice::thread_render
- Session::acquire_tile
- kernel_path_trace

And ultimately, the best approach for figuring out control flow of 
course always is searching and a debugger.

Hope this helps :)

- Lukas

On 11.02.2018 13:09, Bob Geller wrote:
> Hi,
>
> My name is Bob and I'm trying to get into cycles development to 
> hopefully contribute in the future. Is there any documentation to 
> understand the system better and especially understand the flow of 
> control in the renderer? Also, I'm struggling to find enough info 
> about cycles kernel development and the cycles compute language. I 
> would appreciate if someone can point me in the right direction to 
> help understand this better. Thank you for your time and sorry for 
> mailing the whole group, I wasn't sure who to contact.
>
> Regards,
> Bob
>
>
> _______________________________________________
> Bf-cycles mailing list
> Bf-cycles at blender.org
> https://lists.blender.org/mailman/listinfo/bf-cycles

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.blender.org/pipermail/bf-cycles/attachments/20180211/4bf40d9d/attachment.html>