[Bf-blender-cvs] [9336edd0602] blender-v2.93-release: Fix T94635: Sculpt Smooth in Surface mode with Anchored Stroke crash

[Philipp Oeser]

Commit: 9336edd060200755c2160844cbfae144556de6f8
Author: Philipp Oeser
Date:   Thu Jan 6 11:28:02 2022 +0100
Branches: blender-v2.93-release
https://developer.blender.org/rB9336edd060200755c2160844cbfae144556de6f8

Fix T94635: Sculpt Smooth in Surface mode with Anchored Stroke crash

Sculpt Smooth in Surface mode (as opposed to Laplacian) needs a cache
initialized on first time. In anchored stroke mode with spherical falloff
this was skipped though (because this starts of with no PBVH nodes and
an early return checks for this) and `first_time` was set to false before
cache initialization.

Now move the cache initalization to happen earlier (same as the cache
initialization for automasking).

Maniphest Tasks: T94635

Differential Revision: https://developer.blender.org/D13746

===================================================================

M	source/blender/editors/sculpt_paint/sculpt.c
M	source/blender/editors/sculpt_paint/sculpt_smooth.c

===================================================================

diff --git a/source/blender/editors/sculpt_paint/sculpt.c b/source/blender/editors/sculpt_paint/sculpt.c
index 06c300108ec..a1a5001d3c9 100644
--- a/source/blender/editors/sculpt_paint/sculpt.c
+++ b/source/blender/editors/sculpt_paint/sculpt.c
@@ -5964,14 +5964,22 @@ static void do_brush_action(Sculpt *sd, Object *ob, Brush *brush, UnifiedPaintSe
     }
   }
 
-  /* Initialize automasking cache. For anchored brushes with spherical falloff, we start off with
-   * zero radius, thus we have no pbvh nodes on the first brush step. */
+  /* For anchored brushes with spherical falloff, we start off with zero radius, thus we have no
+   * PBVH nodes on the first brush step. */
   if (totnode ||
       ((brush->falloff_shape == PAINT_FALLOFF_SHAPE_SPHERE) && (brush->flag & BRUSH_ANCHORED))) {
     if (SCULPT_stroke_is_first_brush_step(ss->cache)) {
+      /* Initialize auto-masking cache. */
       if (SCULPT_is_automasking_enabled(sd, ss, brush)) {
         ss->cache->automasking = SCULPT_automasking_cache_init(sd, brush, ob);
       }
+      /* Initialize surface smooth cache. */
+      if ((brush->sculpt_tool == SCULPT_TOOL_SMOOTH) &&
+          (brush->smooth_deform_type == BRUSH_SMOOTH_DEFORM_SURFACE)) {
+        BLI_assert(ss->cache->surface_smooth_laplacian_disp == NULL);
+        ss->cache->surface_smooth_laplacian_disp = MEM_callocN(
+            sizeof(float[3]) * SCULPT_vertex_count_get(ss), "HC smooth laplacian b");
+      }
     }
   }
 
diff --git a/source/blender/editors/sculpt_paint/sculpt_smooth.c b/source/blender/editors/sculpt_paint/sculpt_smooth.c
index 3c597fe2b88..6738e31c9d9 100644
--- a/source/blender/editors/sculpt_paint/sculpt_smooth.c
+++ b/source/blender/editors/sculpt_paint/sculpt_smooth.c
@@ -540,13 +540,6 @@ static void SCULPT_do_surface_smooth_brush_displace_task_cb_ex(
 void SCULPT_do_surface_smooth_brush(Sculpt *sd, Object *ob, PBVHNode **nodes, int totnode)
 {
   Brush *brush = BKE_paint_brush(&sd->paint);
-  SculptSession *ss = ob->sculpt;
-
-  if (SCULPT_stroke_is_first_brush_step(ss->cache)) {
-    BLI_assert(ss->cache->surface_smooth_laplacian_disp == NULL);
-    ss->cache->surface_smooth_laplacian_disp = MEM_callocN(
-        sizeof(float[3]) * SCULPT_vertex_count_get(ss), "HC smooth laplacian b");
-  }
 
   /* Threaded loop over nodes. */
   SculptThreadedTaskData data = {