[Bf-codereview] On-surface brush (issue 5554059)

[kalast at gmail.com]

Did a first review on the OpenGL code with some comments. Looks OK to
me. About the stencil buffer, I don't think it will be problematic since
it's an old feature (GL 1.0), but it never hurts to be careful. I think
most colour formats with 24-bit depth support stencil too to make a nice
32-bit alignment in video memory. So I expect we should be safe there.


http://codereview.appspot.com/5554059/diff/1/source/blender/editors/sculpt_paint/paint_cursor.c
File source/blender/editors/sculpt_paint/paint_cursor.c (right):

http://codereview.appspot.com/5554059/diff/1/source/blender/editors/sculpt_paint/paint_cursor.c#newcode547
source/blender/editors/sculpt_paint/paint_cursor.c:547:
The explanation is more or less as follows:

1) Draw the outer radius sphere and write 3 int the stencil buffer where
the sphere fails the depth test (this will be fragments occluded by the
painted mesh. GL_REPLACE in the second argument of glStencilOp means
replace the stencil value when the depth test fails). The stencil test
is disabled here(GL_ALWAYS in glStenciFunc means it always passes)

http://codereview.appspot.com/5554059/diff/1/source/blender/editors/sculpt_paint/paint_cursor.c#newcode551
source/blender/editors/sculpt_paint/paint_cursor.c:551:
2) If we have an inner radius, draw a second sphere, decrease the value
by one each time the sphere fails the depth test. What happens is that
if just the back faces of the inner sphere are occluded in that pixel we
have 2 in the stencil buffer, but if both sides of the sphere are
occluded, we have 1.

This call to glStencilFunc is extraneous as I think GL_DECR always
decreases by one

http://codereview.appspot.com/5554059/diff/1/source/blender/editors/sculpt_paint/paint_cursor.c#newcode561
source/blender/editors/sculpt_paint/paint_cursor.c:561:
3) The step that remains is to obviously draw the intersection itself.
Now we use the stencil test to only draw on places where stencil equals
1 or 3. These are places where the mesh is inside the outer sphere and
outside the inner sphere. On places where it is inside the inner sphere
the stencil is zero so we the stencil will fail there.

http://codereview.appspot.com/5554059/diff/1/source/blender/editors/sculpt_paint/paint_cursor.c#newcode567
source/blender/editors/sculpt_paint/paint_cursor.c:567:
glStencilOp(GL_KEEP, GL_KEEP, GL_KEEP);
Again, extraneous glStencilOp call.

http://codereview.appspot.com/5554059/