[Bf-blender-cvs] [a41fe949d8c] master: Fix T53398: Surface deform modifier says that convex polygons are concave for big faces

[Sergey Sharybin]

Commit: a41fe949d8c346b246bdd1bff4ae50aee19ca859
Author: Sergey Sharybin
Date:   Mon Dec 18 15:16:22 2017 +0100
Branches: master
https://developer.blender.org/rBa41fe949d8c346b246bdd1bff4ae50aee19ca859

Fix T53398: Surface deform modifier says that convex polygons are concave for big faces

Dot-product for angle check need either to be divided by vectors magnitude or be
calculated for normalized vectors.

===================================================================

M	source/blender/modifiers/intern/MOD_surfacedeform.c

===================================================================

diff --git a/source/blender/modifiers/intern/MOD_surfacedeform.c b/source/blender/modifiers/intern/MOD_surfacedeform.c
index b692137b604..f8bad417f0b 100644
--- a/source/blender/modifiers/intern/MOD_surfacedeform.c
+++ b/source/blender/modifiers/intern/MOD_surfacedeform.c
@@ -331,11 +331,13 @@ BLI_INLINE int isPolyValid(const float coords[][2], const unsigned int nr)
 
 	copy_v2_v2(prev_co, coords[nr - 1]);
 	sub_v2_v2v2(prev_vec, prev_co, coords[nr - 2]);
+	normalize_v2(prev_vec);
 
 	for (int i = 0; i < nr; i++) {
 		sub_v2_v2v2(curr_vec, coords[i], prev_co);
 
-		if (len_squared_v2(curr_vec) < FLT_EPSILON) {
+		const float curr_len = normalize_v2(curr_vec);
+		if (curr_len < FLT_EPSILON) {
 			return MOD_SDEF_BIND_RESULT_OVERLAP_ERR;
 		}