[Soc-2018-dev] Weekly Report #1 - Further Development for Cycles' Volume Rendering

[Brecht Van Lommel]

On Tue, May 22, 2018 at 4:17 PM, Geraldine Chua <chua.gsk at gmail.com> wrote:
>
> Since *ImageManager* only stores images as *device_memory*,
> *device_vector<VolumeTile*>* can act as the VolumeGrid class from your
> example while also being compatible with ImageManager. This also minimizes
> the amount of dedicated code needed. I will then write some helper
> functions for it e.g. init, iteration, lookup. Memory handling in theory
> should already be handled by the existing class in so I do not think I will
> need to make any modifications to *device/device_memory.h*.
>

Putting pointers in a device_vector does not work well for the GPU. I think
each sparse grid can be represented as two device_vectors's. One with the
voxel data, and one with tile offsets:

For the voxel data you would allocate an 8x8x(8*number_of_tiles) 3D image.
Basically put one tile after the other to form a long, thin 3D grid. The
advantage of this will be that on the GPU we can then do hardware
accelerated texture interpolation later.

So this would allocate a device_vector<float>, device_vector<float4>,
device_vector<half>, ... just like other images. For some of the code this
could be treated like a regular 3D image.

The tile offsets would be an additional device_vector<int> used by sparse
images. This would be the equivalent of the "SparseVolumeTile **tiles"
array in my pseudocode, but instead of pointers it would have the offset of
the tile in the voxel data array, or -1 to indicate the tile is empty.

Does that make sense? It's a little complicated but should fit well on the
GPU.


> For loading in particular, *ImageManager::file_load_image* in
> *render/image.cpp* will need to be significantly modified to load the
> volume into the device vector, and the conversion function inserted here
> somwhere after the image is loaded. I have begun to make edits and I have
> some more questions about this function (is this the right place to ask?)
>

Yes, you can definitely ask here.


> 1. What are components? Do volumes ever have >4 components? (since there
> is more processing if components > 4)
>

They are image channels. Only 1 and 4 components are supported for images
currently, if there's more we ignore them.


> 2. In the function, after the image is loaded, there seems to be a lot of
> pixel value changing, e.g.
>             for(size_t i = num_pixels-1, pixel = 0; pixel < num_pixels;
> pixel++, i--) {
>                 pixels[i*4+2] = (pixels[i*4+2]*pixels[i*4+3])/255;
>                 pixels[i*4+1] = (pixels[i*4+1]*pixels[i*4+3])/255;
>                 pixels[i*4+0] = (pixels[i*4+0]*pixels[i*4+3])/255;
>                 pixels[i*4+3] = alpha_one;
>             }
> but the kind of changing is conditional on the components? What is the
> purpose of this part of the code? (in the latest master commit, this code
> is in lines 576-651 of *render/image.cpp*)
>

This particular code is for dealing with images using CMYK colors, while
Cycles expects RGB. Generally for 4 components we need them to be red,
green, blue, alpha and for that we do the appropriate conversions. The data
coming from Blender volume grids should already be in the right format.


>
> I will also have a look at these after image loading is functional. Thanks
>> for the tip! Unfortunately, I am not familiar with GPU programming.
>> Generally, what differences will need to be handled? I had heard that GPUs
>> are not very good at handling pointers and double precision arithmetic. So
>> do you think using *device_vector<VolumeTile*>* rather than a dedicated
>> class is alright?
>>
>
See above.


> I have some code written but generally I don't like to make commits unless
> there is a whole feature fully implemented, I am too obsessed with clean
> git histories :P . Would everyone rather I make commits of incomplete code
> to show progress, or would it be alright for me to only commit once a
> feature is mostly complete and functional?
>

We prefer to see commits or patches often enough so we can see how it's
going and give feedback. It doesn't have to be every day, and you can put a
work-in-progress patch on https://developer.blender.org/paste/ to get
feedback or so. But we do generally want so see commits every week. It's
always possible to do a git rebase and squash at the end for a clean git
history.

Thanks,
Brecht.


> Best regards,
> Geraldine
>
> On Mon, May 21, 2018 at 8:23 AM, Brecht Van Lommel <
> brechtvanlommel at gmail.com> wrote:
>
>> On Sun, May 20, 2018 at 7:11 PM, Geraldine Chua <chua.gsk at gmail.com>
>> wrote:
>>
>>> # Texture saving and loading
>>>
>>> For the most part, it seems like hashtables are the most efficient way
>>> to store sparse data sets. In order to optimize volume texture storage, I
>>> traced the top-most relevant code to `ImageManager::file_load_image` in
>>> `render/image.cpp`. It appears that smoke data specifically is retrieved
>>> through the custom functions in `~/blender/makesrna/intern/rna_smoke.c`
>>> rather than using OIIO. Thus, I can implement here a simple conversion of
>>> the volume grid into a map of <coordinates, voxel value> pairs. All parent
>>> functions will also need to be modified to be compatible with a map object
>>> instead of a vector/array.
>>>
>>
>> <coordinates, voxel value> pairs are not an efficient way to store and
>> sample sparse volume data. If you have just a float (or half float) density
>> grid the coordinates can take up 3x more space than the voxel values.
>> Further hash tables will cause cache misses which is not good for rendering
>> performance.
>>
>> The intention for the GSoC project was to use sparse 3D tiles, which are
>> also used in OpenVDB and most other volume rendering or simulation
>> software. What you do is split up the big 3D grid into smaller tiles of
>> 8x8x8 voxels. If a tile contains no data its voxel values are not stored.
>> The code in mesh_volume.c already detects empty 8x8x8 tiles for creating
>> bounds, so some of that code can perhaps be reused.
>>
>> The other part is how you map from the big grid to the tiles. The
>> simplest solution is to use a two level grid, and that's what I suggest to
>> do. To get an idea, this is how such a data structure and associated lookup
>> function would work:
>> https://developer.blender.org/P692
>>
>> In practice it will be a bit different to make it work well for GPUs.
>>
>> # To-do next week
>>>
>>> * Modify the `rna_SmokeModifier_*_grid_get` functions to optionally
>>> recieve a thresholding value and return an array of coordinates + values of
>>> all active voxels.
>>> * Modify callers of `rna_SmokeModifier_*_grid_get` to properly interpret
>>> and manipulate the new array type.
>>>
>>
>> RNA is not the right place to convert a dense grid to a sparse grid, it's
>> going to be more complicated than it needs to be. It will be good if
>> Blender itself supports sparse grids in the future, but when it does there
>> will be no need for thresholding in RNA, rather it would be exposing
>> existing tiled data.
>>
>> Most of the work for Cycles is supporting tiled 3D image storage and
>> sampling. That means changing mostly the following files :
>> render/image.cpp
>> render/image.h
>> device/device_memory.h
>> kernel/kernels/cpu/kernel_cpu_image.h
>> kernel/kernels/cuda/kernel_cuda_image.h
>> kernel/kernels/opencl/kernel_opencl_image.h
>>
>> Lukas or I can help you figure out a design for the data structures,
>> since the whole GPU and device memory aspect is rather complicated. We can
>> start with a CPU implementation first but it should be along the right
>> direction.
>>
>>
>>> * Create a git branch for code changes.
>>> * Make comparison tests on memory performance.
>>> * If have time, move on to importing OpenVDB files.
>>>
>>
>> You can do either OpenVDB import first, or do tiled 3D images first, but
>> both will take at least some weeks to finish fully.
>>
>>
>> --
>> Soc-2018-dev mailing list
>> Soc-2018-dev at blender.org
>> https://lists.blender.org/mailman/listinfo/soc-2018-dev
>>
>>
>
> --
> Soc-2018-dev mailing list
> Soc-2018-dev at blender.org
> https://lists.blender.org/mailman/listinfo/soc-2018-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.blender.org/pipermail/soc-2018-dev/attachments/20180523/dbd1c964/attachment-0001.html>