[Bf-blender-cvs] [158679c] master: Fix T48666: Segfault on boolean operation when exiting edit mode of instanced operand

[Sergey Sharybin]

Commit: 158679c9cc0f024fd54874f48e82cc00072caab4
Author: Sergey Sharybin
Date:   Fri Jul 1 14:19:47 2016 +0200
Branches: master
https://developer.blender.org/rB158679c9cc0f024fd54874f48e82cc00072caab4

Fix T48666: Segfault on boolean operation when exiting edit mode of instanced operand

This is quite tricky situation which combined:

- Boolean modifier which accesses other object's derived mesh
  (in fact, it's nothing to do with boolean modifier, any other
  modifier which uses other's DM will have same bug).

- Dependency cycles in the scene which is rather russian-roulette
  from the cycle solver point of view.

- Multiple instanced objects used as boolean operand.

With all this things combined boolean modifier was accessing operand's derived
mesh which was referencing data from Mesh datablock. The issue is that those
references are becoming invalid after EDBM_mesh_load().

This function already had code to make sure object itself does not end up with
dangling pointers from derived mesh. Make it now so no possible instanced objects
are left with dangling pointers.

And who said it's a good idea to reference something from derived mesh..

===================================================================

M	source/blender/editors/mesh/editmesh_utils.c

===================================================================

diff --git a/source/blender/editors/mesh/editmesh_utils.c b/source/blender/editors/mesh/editmesh_utils.c
index e0e20fa..5101608 100644
--- a/source/blender/editors/mesh/editmesh_utils.c
+++ b/source/blender/editors/mesh/editmesh_utils.c
@@ -401,10 +401,25 @@ void EDBM_mesh_load(Object *ob)
 	BKE_mesh_tessface_calc(me);
 #endif
 
-	/* free derived mesh. usually this would happen through depsgraph but there
+	/* Free derived mesh. usually this would happen through depsgraph but there
 	 * are exceptions like file save that will not cause this, and we want to
-	 * avoid ending up with an invalid derived mesh then */
-	BKE_object_free_derived_caches(ob);
+	 * avoid ending up with an invalid derived mesh then.
+	 *
+	 * Do it for all objects which shares the same mesh datablock, since their
+	 * derived meshes might also be referencing data which was just freed,
+	 *
+	 * Annoying enough, but currently seems most efficient way to avoid access
+	 * of freed data on scene update, especially in cases when there are dependency
+	 * cycles.
+	 */
+	for (Object *other_object = G.main->object.first;
+	     other_object != NULL;
+	     other_object = other_object->id.next)
+	{
+		if (other_object->data == ob->data) {
+			BKE_object_free_derived_caches(other_object);
+		}
+	}
 }
 
 /**