[Bf-committers] For the record: the Blender Render Deamon!

Diego Hernan Borghetti bdiego at gmail.com
Fri Nov 14 18:33:42 CET 2008


On Tue, 11 Nov 2008, Ton Roosendaal wrote:

> Hi all,
>
> I keep wondering why no good, useful and simple render farms exist...
> apparently such systems are never coded by people who actually use it
> in a (local) studio setup. ;)

  When I was working in plumiferos, i write a small render farm system,
it's base in a client/server model and it's gpl.

  Have 3 program:

1) The "master"
    This is the server that keep the job data base and also managed all
    the slave that connect to it.

2) The "slaves"
    Every slave is a machine that connect to the server and waiting
    for a job.

3) The "manager"
    This is the main program that connect to the master and let add,
    remove and change all the things, like make "farms" (a farm is a group
    of slave), job, users, etc.


  It's not finish and still need a lot of work, the manager only have
a curses ui, this is because the system also work with sql, so most
of the options have web ui.

  Also one of the problem is that the slave don't run blender, it run
a serie of script (pre-*, render-*, post-*, end-* and error-*), bash
script, so in Windows this can be a problem (or not ?)

  For example, the script for plumiferos:
 	pre-svn	-> This script checkout the blend files, textures and
 		   others things using svn.

 	render-all -> This script run blender and perform a little
 		      "check-file" (for example, if we already have
 		      files start from the last frame so don't
 		      over-write renders.)

 	post-checkX -> This have four script (post-check1, 2, 3, 4)
 			every script check the output files, jpg,
 			exr and also move the files to the nfs folder.

 	end-all -> This script cleanup all the tmp directory and also
 		   save a log of all the command.

  Also, for plumiferos I add a small script (using sendmail or any
other mail-client) that send a mail to the "own" of the job when
it finish or stop because a error.

  I am not working on this now, but have some plans for it, so maybe
the code can help or I can start with this again.. .. anyway,
I can cleanup a little (remove spanish comment and other things ;)
and make a tar.gz an upload it to some place.

  The bad things is that I don't write documentation...

>
> Just for the record, the ancient 90ies NeoGeo render deamon system is
> still worth documenting. We used it for 100s of projects, and it worked
> pretty flawless.
>
> The system was designed to run locally, on every system in a studio
> network. At that time we used SGI's, and its design - most likely -
> will only port well in a Linux/Unix environment now. It also worked on
> a very nice background mode, allowing renders to continue while you
> were working without really noticing it.
>
> These were the components:
>
> 1) renderd
> The deamon itself, running in a terminal in background, registering its
> system info and handling messages. Running this was first thing to do
> on your system to add it in the network.
> 2) Render
> The front-end, using fdesign "xforms", which can also run via X11
> remotely. It had buttons to set priority, add/remove .blend files,
> showing status of .blend renders, memory/swap status, status of all
> systems, etc.
> 3) fs
> The "file select", which was a small standalone program (working like
> blender's!) to browse files.
>
> And it worked like this:
>
> - we had NFS running, with one shared path accessible via all
> workstations. (In our case /usr/people/deamon/ was shared). An ENV
> variable was used for it.
> - when you add a .blend file (with Render) it read the file, found the
> REND chunk, which has all scenes in it that needed to be rendered,
> including start/end frames. That's what the 'render deamon' button in
> Blender was for.
> - Render then splits the above info in chunks, default in 10-file
> renders, and writes every chunk as a commandline instruction ("info
> files") in the NFS deamon folder. Those commandlines could be anything
> btw, also for other programs as blender.
> - The renderd deamons have internal (idle) loops checking for such
> "info files". The files had status encoded in it, and could be locked
> and changed when available for execution. For as long there are
> available "info files" it keeps running them. On succesful completion
> the files were removed.
> - The renderd deamons also used sockets/pipes to accept or send
> messages. This allowed the Render front-end to manage them, to get
> status or detect whether a render command failed (crashed).
>
> That's about it. Code can be found in the 'neogeo chest':
> http://download.blender.org/source/chest/neogeo/frank/
>
> Would be interesting for someone to check on a similar system for
> inclusion in Blender? Are sockets and pipes supported in Windows now?
>
> -Ton-
>
> ------------------------------------------------------------------------
> Ton Roosendaal  Blender Foundation   ton at blender.org    www.blender.org
> Blender Institute BV  Entrepotdok 57A  1018AD Amsterdam The Netherlands
>
> _______________________________________________
> Bf-committers mailing list
> Bf-committers at blender.org
> http://lists.blender.org/mailman/listinfo/bf-committers
>

-- 
 			Diego


More information about the Bf-committers mailing list