[Bf-blender-cvs] SVN commit: /data/svn/bf-blender [52837] trunk/blender/source/blender/ editors: fix for knife tool when the mouse was moved outside the clipped area the mouse line would reset to 0 /0/0

[Campbell Barton]

Revision: 52837
          http://projects.blender.org/scm/viewvc.php?view=rev&root=bf-blender&revision=52837
Author:   campbellbarton
Date:     2012-12-10 07:53:20 +0000 (Mon, 10 Dec 2012)
Log Message:
-----------
fix for knife tool when the mouse was moved outside the clipped area the mouse line would reset to 0/0/0

a few areas that use ED_view3d_win_to_segment_clip() didnt take into account the case where the segment was filly clipped, some callers even needed the segment not to be clipped.
- added ED_view3d_win_to_segment()
- ED_view3d_win_to_segment_clip() now returns FALSE if the segment is totally clipped, but the start/ends of the line are not zero'd as they were before.

Modified Paths:
--------------
    trunk/blender/source/blender/editors/include/ED_view3d.h
    trunk/blender/source/blender/editors/mesh/editmesh_bvh.c
    trunk/blender/source/blender/editors/sculpt_paint/sculpt.c
    trunk/blender/source/blender/editors/space_view3d/view3d_project.c

Modified: trunk/blender/source/blender/editors/include/ED_view3d.h
===================================================================
--- trunk/blender/source/blender/editors/include/ED_view3d.h	2012-12-10 07:20:52 UTC (rev 52836)
+++ trunk/blender/source/blender/editors/include/ED_view3d.h	2012-12-10 07:53:20 UTC (rev 52837)
@@ -193,7 +193,8 @@
 void ED_view3d_win_to_3d(struct ARegion *ar, const float depth_pt[3], const float mval[2], float out[3]);
 void ED_view3d_win_to_delta(struct ARegion *ar, const float mval[2], float out[3]);
 void ED_view3d_win_to_vector(struct ARegion *ar, const float mval[2], float out[3]);
-void ED_view3d_win_to_segment_clip(struct ARegion *ar, struct View3D *v3d, const float mval[2], float ray_start[3], float ray_end[3]);
+void ED_view3d_win_to_segment(struct ARegion *ar, struct View3D *v3d, const float mval[2], float ray_start[3], float ray_end[3]);
+int  ED_view3d_win_to_segment_clip(struct ARegion *ar, struct View3D *v3d, const float mval[2], float ray_start[3], float ray_end[3]);
 void ED_view3d_ob_project_mat_get(struct RegionView3D *v3d, struct Object *ob, float pmat[4][4]);
 void ED_view3d_unproject(struct bglMats *mats, float out[3], const float x, const float y, const float z);
 

Modified: trunk/blender/source/blender/editors/mesh/editmesh_bvh.c
===================================================================
--- trunk/blender/source/blender/editors/mesh/editmesh_bvh.c	2012-12-10 07:20:52 UTC (rev 52836)
+++ trunk/blender/source/blender/editors/mesh/editmesh_bvh.c	2012-12-10 07:53:20 UTC (rev 52837)
@@ -399,7 +399,7 @@
 	const float mval_f[2] = {ar->winx / 2.0f,
 	                         ar->winy / 2.0f};
 
-	ED_view3d_win_to_segment_clip(ar, v3d, mval_f, origin, end);
+	ED_view3d_win_to_segment(ar, v3d, mval_f, origin, end);
 	
 	invert_m4_m4(invmat, obedit->obmat);
 	mul_m4_v3(invmat, origin);

Modified: trunk/blender/source/blender/editors/sculpt_paint/sculpt.c
===================================================================
--- trunk/blender/source/blender/editors/sculpt_paint/sculpt.c	2012-12-10 07:20:52 UTC (rev 52836)
+++ trunk/blender/source/blender/editors/sculpt_paint/sculpt.c	2012-12-10 07:53:20 UTC (rev 52837)
@@ -3764,6 +3764,7 @@
 	mval[0] = mouse[0] - vc.ar->winrct.xmin;
 	mval[1] = mouse[1] - vc.ar->winrct.ymin;
 
+	/* TODO: what if the segment is totally clipped? (return == 0) */
 	ED_view3d_win_to_segment_clip(vc.ar, vc.v3d, mval, ray_start, ray_end);
 
 	invert_m4_m4(obimat, ob->obmat);

Modified: trunk/blender/source/blender/editors/space_view3d/view3d_project.c
===================================================================
--- trunk/blender/source/blender/editors/space_view3d/view3d_project.c	2012-12-10 07:20:52 UTC (rev 52836)
+++ trunk/blender/source/blender/editors/space_view3d/view3d_project.c	2012-12-10 07:53:20 UTC (rev 52837)
@@ -296,7 +296,7 @@
 {
 	float ray_end[3];
 	
-	ED_view3d_win_to_segment_clip(ar, v3d, mval, ray_start, ray_end);
+	ED_view3d_win_to_segment(ar, v3d, mval, ray_start, ray_end);
 	sub_v3_v3v3(ray_normal, ray_end, ray_start);
 	normalize_v3(ray_normal);
 }
@@ -419,19 +419,7 @@
 	normalize_v3(out);
 }
 
-/**
- * Calculate a 3d segment from 2d window coordinates.
- * This ray_start is located at the viewpoint, ray_end is a far point.
- * ray_start and ray_end are clipped by the view near and far limits
- * so points along this line are always in view.
- * In orthographic view all resulting segments will be parallel.
- * \param ar The region (used for the window width and height).
- * \param v3d The 3d viewport (used for near and far clipping range).
- * \param mval The area relative 2d location (such as event->mval, converted into float[2]).
- * \param ray_start The world-space starting point of the segment.
- * \param ray_end The world-space end point of the segment.
- */
-void ED_view3d_win_to_segment_clip(ARegion *ar, View3D *v3d, const float mval[2], float ray_start[3], float ray_end[3])
+void ED_view3d_win_to_segment(ARegion *ar, View3D *v3d, const float mval[2], float ray_start[3], float ray_end[3])
 {
 	RegionView3D *rv3d = ar->regiondata;
 
@@ -455,14 +443,46 @@
 		madd_v3_v3v3fl(ray_start, vec, rv3d->viewinv[2],  1000.0f);
 		madd_v3_v3v3fl(ray_end, vec, rv3d->viewinv[2], -1000.0f);
 	}
+}
 
+/**
+ * Calculate a 3d segment from 2d window coordinates.
+ * This ray_start is located at the viewpoint, ray_end is a far point.
+ * ray_start and ray_end are clipped by the view near and far limits
+ * so points along this line are always in view.
+ * In orthographic view all resulting segments will be parallel.
+ * \param ar The region (used for the window width and height).
+ * \param v3d The 3d viewport (used for near and far clipping range).
+ * \param mval The area relative 2d location (such as event->mval, converted into float[2]).
+ * \param ray_start The world-space starting point of the segment.
+ * \param ray_end The world-space end point of the segment.
+ * \return success, FALSE if the segment is totally clipped.
+ */
+int ED_view3d_win_to_segment_clip(ARegion *ar, View3D *v3d, const float mval[2], float ray_start[3], float ray_end[3])
+{
+	RegionView3D *rv3d = ar->regiondata;
+	ED_view3d_win_to_segment(ar, v3d, mval, ray_start, ray_end);
+
 	/* clipping */
 	if (rv3d->rflag & RV3D_CLIPPING) {
+		/* if the ray is totally clipped,
+		 * restore the original values but return FALSE
+		 * caller can choose what to do */
+		float tray_start[3] = {UNPACK3(ray_start)};
+		float tray_end[3]   = {UNPACK3(ray_end)};
 		int a;
 		for (a = 0; a < 4; a++) {
-			clip_line_plane(ray_start, ray_end, rv3d->clip[a]);
+			if (clip_line_plane(tray_start, tray_end, rv3d->clip[a]) == FALSE) {
+				return FALSE;
+			}
 		}
+
+		/* copy in clipped values */
+		copy_v3_v3(ray_start, tray_start);
+		copy_v3_v3(ray_end, tray_end);
 	}
+
+	return TRUE;
 }