[Bf-blender-cvs] [e03d53874da] master: Fix T79127: crash on `scene.ray_cast()` with non-viewport view layer

[Sybren A. Stüvel]

Commit: e03d53874dac5fc24120dfad496ef008c5244b84
Author: Sybren A. Stüvel
Date:   Wed Aug 5 18:13:39 2020 +0200
Branches: master
https://developer.blender.org/rBe03d53874dac5fc24120dfad496ef008c5244b84

Fix T79127: crash on `scene.ray_cast()` with non-viewport view layer

The `rna_Scene_ray_cast()` function tried to find the current depsgraph. To
this end, it required the scene, the view layer, and bmain. Scene has a cache
of per-view-layer depsgraphs, to speed up switching between view layers. This
cache does not contain render depsgraphs, and evaluated view layers also don't
have a depsgraph here.

When a suitable depsgraph cannot be found, a new depsgraph is created. However,
this depsgraph is not evaluated, and has an unexpanded scene pointer with a
`NULL` `view_layer`. Using this then crashes Blender. Also, there was no way
for the code to get the render depsgraph.

The solution is to pass the depsgraph to the `ray_cast()` function, instead of
the view layer. This avoids the depsgraph lookup, and also works correctly when
rendering.

Some add-ons also need updating, which I'll do in the `addons`
repository soon.

Reviewed By: Sergey

Differential Revision: https://developer.blender.org/D8475

===================================================================

M	source/blender/makesrna/intern/rna_scene_api.c

===================================================================

diff --git a/source/blender/makesrna/intern/rna_scene_api.c b/source/blender/makesrna/intern/rna_scene_api.c
index a66e20258d2..06c73fbb19c 100644
--- a/source/blender/makesrna/intern/rna_scene_api.c
+++ b/source/blender/makesrna/intern/rna_scene_api.c
@@ -138,8 +138,7 @@ static void rna_SceneRender_get_frame_path(
 }
 
 static void rna_Scene_ray_cast(Scene *scene,
-                               Main *bmain,
-                               ViewLayer *view_layer,
+                               Depsgraph *depsgraph,
                                float origin[3],
                                float direction[3],
                                float ray_dist,
@@ -151,8 +150,6 @@ static void rna_Scene_ray_cast(Scene *scene,
                                float r_obmat[16])
 {
   normalize_v3(direction);
-
-  Depsgraph *depsgraph = BKE_scene_get_depsgraph(bmain, scene, view_layer, true);
   SnapObjectContext *sctx = ED_transform_snap_object_context_create(scene, 0);
 
   bool ret = ED_transform_snap_object_project_ray_ex(sctx,
@@ -292,9 +289,9 @@ void RNA_api_scene(StructRNA *srna)
 
   /* Ray Cast */
   func = RNA_def_function(srna, "ray_cast", "rna_Scene_ray_cast");
-  RNA_def_function_flag(func, FUNC_USE_MAIN);
   RNA_def_function_ui_description(func, "Cast a ray onto in object space");
-  parm = RNA_def_pointer(func, "view_layer", "ViewLayer", "", "Scene Layer");
+
+  parm = RNA_def_pointer(func, "depsgraph", "Depsgraph", "", "The current dependency graph");
   RNA_def_parameter_flags(parm, PROP_NEVER_NULL, PARM_REQUIRED);
   /* ray start and end */
   parm = RNA_def_float_vector(func, "origin", 3, NULL, -FLT_MAX, FLT_MAX, "", "", -1e4, 1e4);