[Bf-committers] Hiearchical Build

[Kiernan Holland]

In a previous email  I was struggling with the idea of a  method for
building 
blender that would work on all platforms and also allow the inheritance 
of the core build into the individual specific system builds, without the
hackish 
modifications that usually come with multiplatform builds. 

The structure of the build is simple in idea, hard in implementation: 

I start with the core pieces of blender that don't 
interface with anything system specific, that only do math or logic, but
very 
little system and user interfacing.. 

Then building upon this the pieces that compile,  system specific parts.. 

Then upon the system specific parts, have sub-builds that deal with
specific machine configurations, 
 so that someone with a multiprocessor uber-reality-engine SGI super
computer design, can benefit from 
not having to define the generic and system specific build processes.. 

Thus a Hiearchical compile process..  

I had used the concept of a "lambda function" without revisiting the term
and determined that 
I was really talking about function generators. Functions that generate
functions.. Lambda's are 
temporary inline functions not what I had in mind.. 

 I had the idea of using a function to generate functions, as a basis for
the build process.
Drawing creatively from an idea that the build process could be a series of
functions that 
resolve the build when applied in phases. 

Like the generic build would define blender as compiled for a generic
system with nothing 
on it (perfect for render servers, Verse servers, dynamic modellers, super
computer model/animation 
generators, things that do not require a user interface or anything of a
specific system). 

Another function could resolve the build to a particular operating system
(where sounds, graphics, storage and windowing system specifics are
resolved as 
well as the specifics of a compiler). 

And yet another would resolve the build to work for a specific
configuration 
(substituting sounds, graphics, processing options, possibly even
substituting source includes 
even modifying the source in temporary files and including them in the
compile)

-----------

Now, it needn't be three layers of specification, it could be 300 layers,
but 
the core layers that exist are generic, system specific and configuration
specific. 
Like there may be a layer between the generic and system specific layers
for 
systems that support windowing systems.

-----------

How ever the implementation is carried out, the basis for the idea is to
have the build 
resolved in phases, not as a inline mess of symbold definitions,
conditionals, execution of 
scripts, etc.. As a tree hiearchy placing the basics of the compile at the
root and the 
specifics at the leaf nodes, and having every necessary process to reach
the 
leaves as a step in resolving the basic compile at the root. 

The solution could manifest itself in a number of ways: 

1. A series of functions take a template of a build and resolve the
requirements of the 
template to define a build for a specific or general system configuration. 

2. Using Diff/Patch on Makefiles, in a clever editing system that creates a
hiearchical layering of 
of Makefile modifications based upon one another, starting with the
simplest makefile and working 
down to Makefiles for specific machines, driving downward more to specific
machine configurations. 
(this would be the easiest to implement, think PHP + Diff + Patch +
Hiearchical Directory/Database structure) 

3. Use of a Makefile with a bunch of unresolved symbols, loading and
executing makefiles that 
resolve the symbols. (the way it is done now, but keeping the basic
Makefile pure of any system 
specifics). 

4. Take #3 and instead place the system specific Makefiles in seperate
folders, and 
make configuration specific Makefiles as subfolders. Execute the makefile
from the subfolders, 
such that the configuration specific makefiles execute system specific
makefiles which execute 
the "grand-daddy" generic makefile (this way the generic makefile doesn't
have to 
know to include system specific subfolders). 

There are probably many more, but the developers would know how best to
organize and 
implement this. The important thing to keep in mind is that this method
would have to 
share builds in a way that generic and system specific builds are seperated
but dependent, and 
it would have to distribute the management of the build in a logical way
that reduces 
replication of work.  

I would strongly suggest something like solutions #1 or #2, #2 is my
favorite. The idea being that 
one could toggle between the specifics of the build without corrupting all
layers of the build at 
the same time. 

What's the difference? 
Normally builds are done in sequence, you compile the program, it has a set
of dependencies, 
the dependencies, the dependencies are compiled, and the result is a
linking of the dependencies 
into the compiled program. However to allow for compiles for multiple
platforms, there 
is usually a mess of "if-conditions" that are to determine what to compile
based on 
the characteristics of the particular system configuration available. One
way to organize the 
Makefiles is to put them in every directory where source code is found, and 
the general makefile includes them as dependencies, that they be compiled 
before linking all the objects from the sub-compiles into the grand
compile. 

The problem with this structure is the "if-conditionals", it disperses the
specifics of each 
version of the compile to every makefile and sub-makefile in the source..
To design a compile 
for a specific system, you must determine what needs to be changed and add
if-conditions 
to the Makefiles to account for this new version, without corrupting what
exists (this risk of corruption 
can lead to typos and ultimately, build failures.. Where objects are left
out of the link process). 

What I'm suggesting is re-arranging this so that there is one place to
define a build for a 
particular system for all the Makefiles in the source hiearchy. And how I
propose it be done 
is using software that allows a systematic hierarchical modification of the
basic 
build (beit a single makefile or multiple ones) down to the individual
builds, without using   
conditionals, relying on software to create builds for specific machines,
without 
requiring the developer to be as organized as the build process.

riseofthethorax@earthlink.net
http://www.bl3nder.com/