[Bf-committers] 64 bits CPU support

Ton Roosendaal ton at blender.org
Tue Jan 4 14:23:02 CET 2005


Hi,

Thanks sgefant for the kick-off, so let's make 2005 the 64 bits year! :)
For a better understanding of the problems we need to solve, we first  
have to pin down the conventions we're going to use.

---------- Data sizes --------

A good read on the topic can be found here;
http://www.unix.org/version2/whatsnew/login_64bit.html

Or in short; for Blender we've already adopted the - quite standard -  
ILP32 and LP64 data size models, which define the following bit sizes  
for standard variables:

          ILP32   LP64
char        8      8
short      16     16
int        32     32
long       32     64
pointer    32     64
long long  64     64

By default, a 32 bits Blender binary is ILP32, and a 64 bits Blender is  
LP64. Where the "long" serves for both as the only integer to be used  
for pointer arithmetic.

Of course, Microsoft thinks a little bit different about the topic;
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/win64/ 
win64/rules_for_using_pointers.asp

But it wouldn't be too hard to make sure the correct types are defined  
for compiling Blender under 64 bits windows. A topic to look on though,  
especially related to the cumbersome "winstuff.h" that lies around, and  
nobody seems to understand anymore.

------------- Blender SDNA and files -------------

Since we already did 64 bits ports, there's the built-in convention in  
SDNA to map pointers and longs to 64 bits if needed. Structure  
alignment is already tested for; remember that pointers or structs  
inside structs had to be 8-byte aligned!

Blender sneaks in the file header a code to define endianess and cpu  
type. The header of a file on a mac looks like this:

BLENDER_V236

Where the '_' means 32 bits and the 'V' means big endian. A 64 bits  
little endian file would get this:

BLENDER-v236

Where the '-' means 64 bits and the 'v' means little endian.

------------ The SDNA issue -------------------

Weakest point in the conversion code now is how a 64 bits pointer in a  
file gets converted back to a 32 bits one. This conversion is only  
needed for the construction of a lookup table, where the old pointer  
values can be mapped to the new actual ones.
Needless to note is of course that someone saving a 6 Gigabyte file on  
a 64 bits system can't expect it to read OK on a 32 bits system. That  
simply exceeds the available memory address space.
But for smaller files there's a tricky issue; somehow the virtual  
address space of 64 bits has to be mapped. Blender currently does this  
to convert a 64 bits 'olddata' pointer to a 32 bits 'newdata':

long long lval;

lval= *( (long long *)olddata );
*((int *)newdata) = lval>>3;

This has been tested to run fine on Dec Alpha Linux. Main question that  
remains; can the virtual memory mapping used on 64 bits architecture be  
assumed to be in a coherent chunk? Because his conversion only works  
for a memory map not exceeding the 32 Gigabyte range. I am not a CPU  
guru at all, so hopefully someone here knows the answer.

------------ The Blender code issue -------------------

With the move to 32 bits integer indexes in Meshes, abuse of that  
feature has been made to store temporally pointers in here, to remap  
back later on. This happens in the displist.c and subsurf.c code for  
example.

Needless to say; that's not going to work on a 64 bits system. Making  
vertex indices 64 bits too isn't a good solution either. The ugliest  
and simplest hack would be of course some platform specific #define  
that maps a 64 bits pointer to 32 bits int and back (with dangerous  
assumptions of used address space. That's what I need advise on too...  
if that's relatively safe (like with a 32 Gig memory range too) we  
could stick with that temporally. Otherwise we need to recode the  
current index abuse.

------------- The patch -------------------------

Sgefant; you've found quite some of the issues in the patch, but not  
all solutions you provide are correct. Parts of the code you propose to  
make "long" can remain to be int, especially where (color or mesh)  
indices are involved.

I want to go into further detail later on this... first have to find  
out when the right moment is to move to 64 bits Tiger for my system..  
Anyone out there has successful experiences with it?

Thanks,

-Ton-



On 17 Nov, 2004, at 19:46, Stefan Gartner wrote:

> Hi,
> the attached patch should deal with most of the int <-> pointer stuff.
> As I'm not sure whether all those changes are really necessary  
> (especially the
> changes to intern/guardedalloc and the DNA headers), it would be nice  
> if
> someone more familiar with blender code and 64 bit platforms than me  
> could
> look over it. If there are no objections, I will commit my changes  
> after the
> release dust has settled.
>
> The patch also contains some minor changes to the Makefiles, to support
> building on Linux/x86_64.
>
> Stuff that still needs to be done:
> - update scons (not sure if there's anything that needs to be changed)
> - check the 32 bit <-> 64 bit conversions in readfile.c
> - make sure it works on other platforms. So far I have tested it on  
> Linux
>   (i386 and x86_64)
>
> greetings,
> sgefant
>
> On Tuesday, 9. November 2004 12:34, Ton Roosendaal wrote:
>> Hi,
>>
>> Here's some essential info for those interested in testing 64 bits
>> binaries;
>>
>> Back in the nineties I ported & maintained a 64 bits Linux dec alpha
>> version. We dropped this in 2001 or so... but the foundation for  
>> proper
>> 64 bits support in Blender is still there. It is likely though that  
>> new
>> code was added that's not 64 bits compatible (yes, not allowed to put  
>> a
>> pointer in an integer), that's all to clean up.
>>
>> Apple is going to 64 bits in their next upgrade, will definitely check
>> that out. For those interested to already work on it, there's one
>> important coding rule to apply in blender:
>>
>> -> the variable type "long" has been typedeffed to be either 32 or 64
>> bits integer, depending pointer size.
>>
>> So, if you want to do pointer arithmetic, or cast pointers to int, use
>> a "long". Also use "long" in (DNA) structures, that gets mapped OK.  
>> And
>> make sure longs and pointers are properly aligned (8 byte aligned) in
>> structs that get saved in blender files.
>>
>> This rule might have been gone lost in recent code... or not fully
>> covered in makefiles or Scons. All work todo.
>>
>> Oh, and hackish part in Blender needs to be checked; to map pointer
>> values from 64 bits systems back to 32 bits ones (file load,
>> readfile.c) there's a conversion that might need to be updated...  
>> right
>> now it does: pointer_32 = (pointer_64>>3);
>> This assuming that address space on a 64 bits system doesn't start
>> somewhere in the billions.
>>
>> -Ton-
>>
>> ---------------------------------------------------------------------- 
>> --
>> --
>> Ton Roosendaal  Blender Foundation ton at blender.org
>> http://www.blender.org
>>
>> _______________________________________________
>> Bf-committers mailing list
>> Bf-committers at projects.blender.org
>> http://projects.blender.org/mailman/listinfo/bf-committers
> <x86_64.diff.gz>_______________________________________________
> Bf-committers mailing list
> Bf-committers at projects.blender.org
> http://projects.blender.org/mailman/listinfo/bf-committers
>
------------------------------------------------------------------------ 
--
Ton Roosendaal  Blender Foundation ton at blender.org  
http://www.blender.org



More information about the Bf-committers mailing list