[Bf-blender-cvs] [ea11b4e10ce] blender-v2.93-release: Fix T88230: Properly handle Face Set visibility in Expand Face Sets editing

[Pablo Dobarro]

Commit: ea11b4e10ce06e68ba365c8c7cbdf2ab5bdaf12f
Author: Pablo Dobarro
Date:   Thu May 13 02:40:10 2021 +0200
Branches: blender-v2.93-release
https://developer.blender.org/rBea11b4e10ce06e68ba365c8c7cbdf2ab5bdaf12f

Fix T88230: Properly handle Face Set visibility in Expand Face Sets editing

Expand is not expected to update the visibility state of the PBVH, only
the Face Sets IDs. If visibility updates are made accidentally, PBVH
rendering breaks.

In order for this to work properly, the following fixes are needed:

- Expand should always check for active component before attempting to
 modify a Face Set ID

- Expand should always check the visibility state on original_face_sets, as
it is the array that contains the visiblilty state that corresponds with the
current state used for PBVH rendering. This implies that after any modification
done by Expand, the visibility state of ss->face_sets and
expand_cache->original_face_sets should match (like in any other tool that
does not modify visibility).

- Expand should never modify the Face Set ID of a poly that is hidden in
expand_cache->original_face_sets.

- When deleting an ID, hidden Face Sets should be skipped when picking IDs for
content filling. This avoids introducing hidden IDs back into the visible
geometry even after updating its visibility state.

Reviewed By: JulienKaspar, JacquesLucke

Maniphest Tasks: T88230

Differential Revision: https://developer.blender.org/D11243

===================================================================

M	source/blender/editors/sculpt_paint/sculpt_expand.c

===================================================================

diff --git a/source/blender/editors/sculpt_paint/sculpt_expand.c b/source/blender/editors/sculpt_paint/sculpt_expand.c
index 2de7d5d72cc..40874375772 100644
--- a/source/blender/editors/sculpt_paint/sculpt_expand.c
+++ b/source/blender/editors/sculpt_paint/sculpt_expand.c
@@ -273,7 +273,7 @@ static bool sculpt_expand_state_get(SculptSession *ss, ExpandCache *expand_cache
  */
 static bool sculpt_expand_face_state_get(SculptSession *ss, ExpandCache *expand_cache, const int f)
 {
-  if (ss->face_sets[f] <= 0) {
+  if (expand_cache->original_face_sets[f] <= 0) {
     return false;
   }
 
@@ -1398,6 +1398,13 @@ static void sculpt_expand_face_sets_restore(SculptSession *ss, ExpandCache *expa
 {
   const int totfaces = ss->totfaces;
   for (int i = 0; i < totfaces; i++) {
+    if (expand_cache->original_face_sets[i] <= 0) {
+      /* Do not modify hidden Face Sets, even when restoring the IDs state. */
+      continue;
+    }
+    if (!sculpt_expand_is_face_in_active_component(ss, expand_cache, i)) {
+      continue;
+    }
     ss->face_sets[i] = expand_cache->initial_face_sets[i];
   }
 }
@@ -1930,6 +1937,7 @@ static void sculpt_expand_delete_face_set_id(int *r_face_sets,
   }
 
   while (BLI_LINKSTACK_SIZE(queue)) {
+    bool any_updated = false;
     while (BLI_LINKSTACK_SIZE(queue)) {
       const int f_index = POINTER_AS_INT(BLI_LINKSTACK_POP(queue));
       int other_id = delete_id;
@@ -1940,6 +1948,10 @@ static void sculpt_expand_delete_face_set_id(int *r_face_sets,
         for (int i = 0; i < vert_map->count; i++) {
 
           const int neighbor_face_index = vert_map->indices[i];
+          if (expand_cache->original_face_sets[neighbor_face_index] <= 0) {
+            /* Skip picking IDs from hidden Face Sets. */
+            continue;
+          }
           if (r_face_sets[neighbor_face_index] != delete_id) {
             other_id = r_face_sets[neighbor_face_index];
           }
@@ -1947,18 +1959,36 @@ static void sculpt_expand_delete_face_set_id(int *r_face_sets,
       }
 
       if (other_id != delete_id) {
+        any_updated = true;
         r_face_sets[f_index] = other_id;
       }
       else {
         BLI_LINKSTACK_PUSH(queue_next, POINTER_FROM_INT(f_index));
       }
     }
+    if (!any_updated) {
+      /* No Face Sets where updated in this iteration, which means that no more content to keep
+       * filling the polys of the deleted Face Set was found. Break to avoid entering an infinite
+       * loop trying to search for those polys again. */
+      break;
+    }
 
     BLI_LINKSTACK_SWAP(queue, queue_next);
   }
 
   BLI_LINKSTACK_FREE(queue);
   BLI_LINKSTACK_FREE(queue_next);
+
+  /* Ensure that the visibility state of the modified Face Sets is the same as the original ones.
+   */
+  for (int i = 0; i < totface; i++) {
+    if (expand_cache->original_face_sets[i] >= 0) {
+      r_face_sets[i] = abs(r_face_sets[i]);
+    }
+    else {
+      r_face_sets[i] = -abs(r_face_sets[i]);
+    }
+  }
 }
 
 static void sculpt_expand_cache_initial_config_set(bContext *C,