[Bf-committers] "off by one" error in renderdatabase.c?

Martin Dickopp martin at zero-based.org
Wed Mar 2 18:39:48 CET 2005


Hello,

I have a Blender file which crashes Blender (2.36 compiled from official
source and CVS HEAD as of 02-Mar-2005 18:00:06 CET) when a specific
frame is rendered. Although I can reproduce the problem only with 64 bit
GNU/Linux (on an AMD64 machine), but not with 32 bit GNU/Linux (on an
x86 machine), I think that the problem is not 64 bit specific. See my
analysis below.


First of all, how to reproduce it: Download the file

  http://www.zero-based.org/blender-bug-report/crash.blend

and render frame 726 on an AMD64 GNU/Linux system. The following
messages appear

  Memoryblock free: attempt to free illegal pointer
  Memoryblock free: attempt to free illegal pointer
  Memoryblock free: pointer not in memlist
  Memoryblock free: pointer not in memlist
  Memoryblock free: pointer not in memlist
  Memoryblock free: attempt to free illegal pointer

and then Blender crashes with a segmentation fault.


I think I know what is causing this. By looking
how the function RE_freeRotateBlenderScene in
source/blender/renderconverter/intern/convertBlenderScene.c
frees the elements of the arrays R.blove, R.blovl and R.bloha,
I conclude that these arrays are supposed to be terminated by
an element which is a null pointer.

If I now look at source/blender/render/intern/source/renderdatabase.c,
it contains the following code (slightly shortened):

  VlakRen *RE_findOrAddVlak(int nr)
  {
          VlakRen *v, **temp;
          static int rblovllen=TABLEINITSIZE;
          int a;

          /* ... */
          a= nr>>8;
        
          if (a>=rblovllen){
                  /* ... allocate more memory and fill it with zeros. */
          }

          v= R.blovl[a];
          if(v==0) {
                  v= (VlakRen *)MEM_callocN(256*sizeof(VlakRen),"findOrAddVlak");
                  R.blovl[a]= v;
          }
          v+= (nr & 255);
          return v;
  }

Now image that the variable a gets the value rblovllen-1. In this case,
new memory is not yet allocated, but all the rblovllen members of
R.blovl (with indices 0 to rblovllen-1) are non-null pointers.
Therefore, R.blovl is not null terminated until further calls to
RE_findOrAddVlak cause the value of a to become rblovllen.
RE_freeRotateBlenderScene barfs if it is called in this situation.

R.blove and R.bloha have the same problem. I therefore propose to apply
the attached patch. It fixes the problem for me.

Thank you very much,
Martin
-------------- next part --------------
Index: source/blender/render/intern/source/renderdatabase.c
===================================================================
RCS file: /cvsroot/bf-blender/blender/source/blender/render/intern/source/renderdatabase.c,v
retrieving revision 1.9
diff -u -r1.9 renderdatabase.c
--- source/blender/render/intern/source/renderdatabase.c	7 Jan 2005 14:11:00 -0000	1.9
+++ source/blender/render/intern/source/renderdatabase.c	2 Mar 2005 16:33:56 -0000
@@ -85,7 +85,7 @@
 	}
 	a= nr>>8;
 	
-	if (a>=rblovelen){  /* Need to allocate more columns...*/
+	if (a+1>=rblovelen){  /* Need to allocate more columns...*/
 		// printf("Allocating %i more vert groups.  %i total.\n", 
 		//	TABLEINITSIZE, rblovelen+TABLEINITSIZE );
 		temp=R.blove;
@@ -118,7 +118,7 @@
 	}
 	a= nr>>8;
 	
-	if (a>=rblohalen){  /* Need to allocate more columns...*/
+	if (a+1>=rblohalen){  /* Need to allocate more columns...*/
 		//printf("Allocating %i more halo groups.  %i total.\n", 
 		//	TABLEINITSIZE, rblohalen+TABLEINITSIZE );
 		temp=R.bloha;
@@ -152,7 +152,7 @@
 	}
 	a= nr>>8;
 	
-	if (a>=rblovllen){  /* Need to allocate more columns...*/
+	if (a+1>=rblovllen){  /* Need to allocate more columns...*/
 		// printf("Allocating %i more face groups.  %i total.\n", 
 		//	TABLEINITSIZE, rblovllen+TABLEINITSIZE );
 		temp=R.blovl;


More information about the Bf-committers mailing list