[Bf-committers] SDNA do's and don'ts

[Ton Roosendaal]

Hi all,

Everyone who wants to add features to the blender core, will be  
confronted with this weirdo system. I called it 'Struct DNA' or SDNA in  
the code. This is essential information that should have been  
documented long ago... but it's never too late to make docs, er!

The system was divised to support Blender's file format, and internal  
library/database system. It allows Blender to do a runtime checking of  
the contents of the Structure data. This enables a few nice features:
- you can just add or remove variables from structures, or add/remove  
complete new structures, without damaging backward *and* forward  
compatiblity of files
- you can use this for Python or smart buttons in Blender, to check  
run-time which variable or structure is being provided

All of Blender's structures that can be saved to a file, are described  
with the SDNA. This is basically a long string of codes, with the  
structure and variable names and types. The 'makesdna' utility takes  
care of this, it has a built-in list of .h header files, that are read  
and converted to the SDNA file.
When a file is saved, the functions that write structs look up the  
structure code and write that in the chunk header (struct BHead,  
writefile.c). At the end of the file, the full SDNA description file is  
appended to the .blend file, to make the file backward/upward  
compatible.
Each compiled Blender has an SDNA description as well. When a file is  
read, the SDNA of the file is compared with the SDNA of the Blender  
executable. Structures that differ get flagged, and converted while  
read.

Allowed conversions (changes) are:
- if a name is identical, but the type differs it tries to convert.  
Only conversions that make sense happen (e.g. short->int, or  
int->float, etc). Otherwise it is set at zero.
- a new variable is always initialized zero
- dimensions of arrays get fixed (e.g. [3][3] to [4][4])
- pointer sizes corrected (32 bits, 64 bits)
- char to float: divided by 255 automatically
- changes in structures within structures (within structures etc) work  
fine

At the time, I didn't include automatic 'padding' in the system... it's  
the platform dependent habit of compilers to include empty spaces in  
structures, to conform alignment of data. For example, at SGI an 'int'  
should always start at an address that can be divided by 4. When you  
design a struct like this:

struct XYZ {
	char x_flag;
	float x;
	char y_flag;
	float y;
	char z_flag;
	float z;
}

The actual  length of the structure will be 6x4=24 bytes. Even worse, a  
struct like this:

struct ABC {
	char a;
}

is 8 bytes at an Alpha, and 4 bytes at 32 bits OS's.

Therefore, for correct SDNA structures in Blender, you have to design  
the structures padding-less, for all supported systems. These are the  
rules:

- structures are always multiples of 8 bytes in size
- structs within structs start at an 8-dividable location
- pointers start at an 8-dividable location, unless the previous  
variable is a pointer!
- ints/floats start at a 4-dividable location
- shorts start at a 2-dividable location

If you can't manage this, just instert 'pad' variables in structures.  
You'll find them in Blender more, it's meant to be free space that can  
be used for later. Do not use 'pad' variables in Blender!!!

-Ton-

------------------------------------------------------------------------ 
--
Ton Roosendaal  Blender Foundation ton@blender.org