[Bf-blender-cvs] [4c46f693763] blender-v2.79a-release: Fix T53398: Surface deform modifier says that convex polygons are concave for big faces

[Sergey Sharybin]

Commit: 4c46f693763bd09c9b68de5ffd552f140b062eeb
Author: Sergey Sharybin
Date:   Mon Dec 18 15:16:22 2017 +0100
Branches: blender-v2.79a-release
https://developer.blender.org/rB4c46f693763bd09c9b68de5ffd552f140b062eeb

Fix T53398: Surface deform modifier says that convex polygons are concave for big faces

Dot-product for angle check need either to be divided by vectors magnitude or be
calculated for normalized vectors.

===================================================================

M	source/blender/modifiers/intern/MOD_surfacedeform.c

===================================================================

diff --git a/source/blender/modifiers/intern/MOD_surfacedeform.c b/source/blender/modifiers/intern/MOD_surfacedeform.c
index b692137b604..f8bad417f0b 100644
--- a/source/blender/modifiers/intern/MOD_surfacedeform.c
+++ b/source/blender/modifiers/intern/MOD_surfacedeform.c
@@ -331,11 +331,13 @@ BLI_INLINE int isPolyValid(const float coords[][2], const unsigned int nr)
 
 	copy_v2_v2(prev_co, coords[nr - 1]);
 	sub_v2_v2v2(prev_vec, prev_co, coords[nr - 2]);
+	normalize_v2(prev_vec);
 
 	for (int i = 0; i < nr; i++) {
 		sub_v2_v2v2(curr_vec, coords[i], prev_co);
 
-		if (len_squared_v2(curr_vec) < FLT_EPSILON) {
+		const float curr_len = normalize_v2(curr_vec);
+		if (curr_len < FLT_EPSILON) {
 			return MOD_SDEF_BIND_RESULT_OVERLAP_ERR;
 		}