[Bf-committers] Development Environments

Chad Fraleigh chadf at triularity.org
Thu Dec 13 09:06:17 CET 2012


On Tue, Dec 11, 2012 at 1:58 AM, Brecht Van Lommel
<brechtvanlommel at pandora.be> wrote:

> Further, I could commit your libraries but what we really need is a
> proper plan to clean up of this stuff. There have been complaints that
> our lib/ directory is getting too big, so we should try to split it
> up. Usually you only need libraries for one visual studio version, and
> only a few developers need the debug libraries which tend to be huge.
>
> We should figure out a directory structure to make it possible to
> checkout only the needed subset, before we start adding a new batch of
> libraries for vs2012. Headers can be shared, maybe some (C only?)
> libraries can be as well. And then cmake/scons build systems should
> also be updated to follow this new structure.

Ok, plan #2.. Instead of having all the pre-compiled windows libraries
in the checkout we maintain copies of the needed external libs as
source code (in compressed tarballs). Then have something like what
the linux install_deps.sh script, only it builds all the needed libs
and installs them in a structure like lib/windows has now (only
simpler since there would be no need to have special directories for
various tools).

As a side option there could be a set of per-environment, pre-compiled
"drop-in" binaries that one could download and extract in the same
place the build script would, so the truly lazy people (or ones having
problems building them) could still work. Only these would just be zip
(or tar) files for download and not part of subversion.

I've started creating a MSVC based script (well, nmake file) to do
such a task. So far it batch builds the following libs: boost png jpeg
tiff python zlib. It also only does the 32-bit versions right now (I
wanted to start out with something VS 2008 would use). I also got rid
of that annoying vc9/vc100/vc110 name in the boost lib names using the
--layout= option (so no special cmake/scons link names needed there).
The hardest part is that it is soooooooo tedious to locate the right
site for each package, then find the version blender currently has (or
something close), and figure out how to compile it (where there is no
blender build.bat already).. so it's slow work.

Here is what I have so far (remember it's a work in progress, but
shows the idea). The first section is for compile environment specific
settings, and the second general configuration. Eventually the first
section would be auto-configured via a .bat script or something which
would then run nmake. It also assumes the UnxUtils tools
(http://unxutils.sourceforge.net/) is bundled since I needed a
reliable way to uncompress/untar the code, and xcopy is still giving
me issues that cp doesn't.

 -<>-<>-<>-<>-<>-<>-<>-<>-<>-<>-<>-<>-<>-<>-<>-<>-<>-<>-<>-<>-<>-<>-<>-<>-<>-<>-

#
# @(#) Makefile.nmake
#

PLATFORM=x32
VS_PLATFORM=Win32
VC_VERSION=9.0
#VC_VERSION=10.0
#VC_VERSION=11.0
CMAKE_GENERATOR=Visual Studio 9 2008
#CMAKE_GENERATOR=Visual Studio 11
PYTHON_PCBUILD=PC\VS9.0
#PYTHON_PCBUILD=PCbuild

#
#

MAXCPU_COUNT=2
SOURCE_DIR=..\source
STAGE_DIR=stage-$(PLATFORM)
INSTALL_DIR=$(MAKEDIR)\install-$(PLATFORM)

#
#

BOOST_VERSION=1_52_0
BOOST_TARBALL=boost_$(BOOST_VERSION).tar.gz
BOOST_INSTALL_DIR=$(INSTALL_DIR)\boost
BOOST_STAGE_DIR=$(STAGE_DIR)\boost_$(BOOST_VERSION)
BOOST_DEPS=$(BOOST_INSTALL_DIR)\include\boost\version.h

JPEG_VERSION=8d
JPEG_TARBALL=jpegsrc.v$(JPEG_VERSION).tar.gz
JPEG_INSTALL_DIR=$(INSTALL_DIR)\jpeg
JPEG_STAGE_DIR=$(STAGE_DIR)\jpeg-$(JPEG_VERSION)
JPEG_DEPS=$(JPEG_INSTALL_DIR)\include\jpeglib.h

PNG_VERSION=1.5.13
PNG_TARBALL=libpng-$(PNG_VERSION).tar.gz
PNG_INSTALL_DIR=$(INSTALL_DIR)\png
PNG_STAGE_DIR=$(STAGE_DIR)\libpng-$(PNG_VERSION)
PNG_DEPS=$(PNG_INSTALL_DIR)\include\png.h

PYTHON_VERSION=3.3.0
PYTHON_TARBALL=Python-$(PYTHON_VERSION).tgz
PYTHON_VERSION_LIBNAME=33
PYTHON_INSTALL_DIR=$(INSTALL_DIR)\python
PYTHON_STAGE_DIR=$(STAGE_DIR)\Python-$(PYTHON_VERSION)
PYTHON_DEPS=$(PYTHON_INSTALL_DIR)\include\python.h

TIFF_VERSION=3.9.2
TIFF_TARBALL=tiff-$(TIFF_VERSION).tar.gz
TIFF_INSTALL_DIR=$(INSTALL_DIR)\tiff
TIFF_STAGE_DIR=$(STAGE_DIR)\tiff-$(TIFF_VERSION)
TIFF_DEPS=$(TIFF_INSTALL_DIR)\include\tiff.h

ZLIB_VERSION=1.2.7
ZLIB_TARBALL=zlib-$(ZLIB_VERSION).tar.gz
ZLIB_INSTALL_DIR=$(INSTALL_DIR)\zlib
ZLIB_STAGE_DIR=$(STAGE_DIR)\zlib-$(ZLIB_VERSION)
ZLIB_DEPS=$(ZLIB_INSTALL_DIR)\include\zlib.h

#
#

UNXUTILS=$(MAKEDIR)\UnxUtils

CMAKE=cmake
CP=$(UNXUTILS)\usr\local\wbin\cp -p
CP_R=$(UNXUTILS)\usr\local\wbin\cp -Rp
GZIP=$(UNXUTILS)\usr\local\wbin\gzip
MKDIR=mkdir
#MKDIR=$(UNXUTILS)\usr\local\wbin\mkdir -p
RM=del /f
#RM=$(UNXUTILS)\usr\local\wbin\rm -f
RM_R=rmdir /s /q
#RM_R=$(UNXUTILS)\usr\local\wbin\rm -rf
TAR=$(UNXUTILS)\usr\local\wbin\tar

#
#

all:		boost zlib png jpeg tiff python

clean:		boost-clean jpeg-clean png-clean python-clean tiff-clean zlib-clean

cleanall:	boost-cleanall jpeg-cleanall png-cleanall python-cleanall
tiff-clean zlib-cleanall


#
# Boost
#

boost:		$(BOOST_DEPS)

$(BOOST_DEPS):	boost-extract boost-configure boost-compile
boost-install boost-clean

boost-extract:
	-$(MKDIR) "$(STAGE_DIR)"
	$(RM) "$(BOOST_STAGE_DIR).tar"
	$(GZIP) -d -c "$(SOURCE_DIR)\$(BOOST_TARBALL)" > "$(BOOST_STAGE_DIR).tar"
	$(TAR) -C "$(STAGE_DIR)" -xf "$(BOOST_STAGE_DIR).tar"
	$(RM) "$(BOOST_STAGE_DIR).tar"

boost-configure:
	cd "$(BOOST_STAGE_DIR)" & .\bootstrap.bat

boost-compile:
	cd "$(BOOST_STAGE_DIR)" & \
		bjam -j$(MAXCPU_COUNT) \
			toolset=msvc-$(VC_VERSION) \
			variant=release \
			link=static \
			runtime-link=static \
			threading=multi \
			--layout=tagged \
			--with-date_time \
			--with-filesystem \
			--with-locale \
			--with-regex \
			--with-system \
			--with-thread \
			--with-wave \
			stage
	cd "$(BOOST_STAGE_DIR)" & \
		bjam -j$(MAXCPU_COUNT) \
			toolset=msvc-$(VC_VERSION) \
			variant=debug \
			link=static \
			runtime-link=static \
			threading=multi \
			--layout=tagged \
			--with-date_time \
			--with-filesystem \
			--with-locale \
			--with-regex \
			--with-system \
			--with-thread \
			--with-wave \
			stage

boost-install:
	-$(MKDIR) "$(INSTALL_DIR)"
	-$(MKDIR) "$(BOOST_INSTALL_DIR)"
	-$(MKDIR) "$(BOOST_INSTALL_DIR)\include"
	$(CP_R) "$(BOOST_STAGE_DIR)\stage\lib" "$(BOOST_INSTALL_DIR)\lib"
	$(CP_R) "$(BOOST_STAGE_DIR)\boost" "$(BOOST_INSTALL_DIR)\include\boost"

boost-clean:
	-$(RM) "$(BOOST_STAGE_DIR).tar"
	-$(RM_R) "$(BOOST_STAGE_DIR)"

boost-cleanall:	boost-clean
	-$(RM_R) "$(BOOST_INSTALL_DIR)"


#
# JPEG
#

jpeg:		$(JPEG_DEPS)

$(JPEG_DEPS):	jpeg-extract jpeg-configure jpeg-compile jpeg-install jpeg-clean

jpeg-extract:
	-$(MKDIR) "$(STAGE_DIR)"
	$(RM) "$(JPEG_STAGE_DIR).tar"
	$(GZIP) -d -c "$(SOURCE_DIR)\$(JPEG_TARBALL)" > "$(JPEG_STAGE_DIR).tar"
	$(TAR) -C "$(STAGE_DIR)" -xf "$(JPEG_STAGE_DIR).tar"
	$(RM) "$(JPEG_STAGE_DIR).tar"

jpeg-configure:	jpeg-configure-$(VC_VERSION)

jpeg-configure-9.0:
	cd "$(JPEG_STAGE_DIR)" & nmake /f makefile.vc setup-vc6

jpeg-configure-10.0:
	cd "$(JPEG_STAGE_DIR)" & nmake /f makefile.vc setup-vc10

jpeg-configure-11.0:
	cd "$(JPEG_STAGE_DIR)" & nmake /f makefile.vc setup-vc10

jpeg-compile:	jpeg-compile-$(VC_VERSION)

jpeg-compile-9.0:
	cd "$(JPEG_STAGE_DIR)" & nmake /f makefile.vc libjpeg.lib

jpeg-compile-10.0:
	cd "$(JPEG_STAGE_DIR)" & nmake /f makefile.vc libjpeg.lib

jpeg-compile-11.0:
	cd "$(JPEG_STAGE_DIR)" & nmake /f makefile.vc libjpeg.lib

jpeg-install:
	-$(MKDIR) "$(INSTALL_DIR)"
	-$(MKDIR) "$(JPEG_INSTALL_DIR)"
	-$(MKDIR) "$(JPEG_INSTALL_DIR)\include"
	-$(MKDIR) "$(JPEG_INSTALL_DIR)\lib"
	$(CP) "$(JPEG_STAGE_DIR)\libjpeg.lib" "$(JPEG_INSTALL_DIR)\lib\libjpeg.lib"
	$(CP) "$(JPEG_STAGE_DIR)\jconfig.h" "$(JPEG_INSTALL_DIR)\include\jconfig.h"
	$(CP) "$(JPEG_STAGE_DIR)\jerror.h" "$(JPEG_INSTALL_DIR)\include\jerror.h"
	$(CP) "$(JPEG_STAGE_DIR)\jmorecfg.h" "$(JPEG_INSTALL_DIR)\include\jmorecfg.h"
	$(CP) "$(JPEG_STAGE_DIR)\jpeglib.h" "$(JPEG_INSTALL_DIR)\include\jpeglib.h"

jpeg-clean:
	-$(RM) "$(JPEG_STAGE_DIR).tar"
	-$(RM_R) "$(JPEG_STAGE_DIR)"

jpeg-cleanall:	jpeg-clean
	-$(RM_R) "$(JPEG_INSTALL_DIR)"



#
# PNG
#

png:		$(PNG_DEPS)

$(PNG_DEPS):	png-extract png-configure png-compile png-install png-clean

png-extract:
	-$(MKDIR) "$(STAGE_DIR)"
	$(RM) "$(PNG_STAGE_DIR).tar"
	$(GZIP) -d -c "$(SOURCE_DIR)\$(PNG_TARBALL)" > "$(PNG_STAGE_DIR).tar"
	$(TAR) -C "$(STAGE_DIR)" -xf "$(PNG_STAGE_DIR).tar"
	$(RM) "$(PNG_STAGE_DIR).tar"

png-configure:
	-$(MKDIR) "$(PNG_STAGE_DIR).build"
	cd "$(PNG_STAGE_DIR).build" & \
		$(CMAKE) -G "$(CMAKE_GENERATOR)" \
			-DCMAKE_INSTALL_PREFIX:PATH="$(PNG_INSTALL_DIR)" \
			-DPNG_SHARED:BOOL=OFF \
			-DZLIB_LIBRARY:PATH="$(ZLIB_INSTALL_DIR)\lib\zlib.lib" \
			-DZLIB_INCLUDE_DIR:PATH="$(ZLIB_INSTALL_DIR)\include" \
			..\libpng-$(PNG_VERSION)

png-compile:
	msbuild "$(MAKEDIR)\$(PNG_STAGE_DIR).build\libpng.sln" \
		/target:ALL_BUILD \
		/maxcpucount:$(MAXCPU_COUNT) \
		/verbosity:detailed \
		/property:Configuration=Debug \
		/property:Platform=$(VS_PLATFORM)
	msbuild "$(MAKEDIR)\$(PNG_STAGE_DIR).build\libpng.sln" \
		/target:ALL_BUILD \
		/maxcpucount:$(MAXCPU_COUNT) \
		/verbosity:detailed \
		/property:Configuration=Release \
		/property:Platform=$(VS_PLATFORM)

#
# NOTE: "msbuild /target:INSTALL ... "  broken
#
png-install:
	-$(MKDIR) "$(INSTALL_DIR)"
	-$(MKDIR) "$(PNG_INSTALL_DIR)"
	-$(MKDIR) "$(PNG_INSTALL_DIR)\include"
	-$(MKDIR) "$(PNG_INSTALL_DIR)\lib"
	$(CP) "$(PNG_STAGE_DIR).build\Release\libpng15_static.lib"
"$(PNG_INSTALL_DIR)\lib\libpng.lib"
	$(CP) "$(PNG_STAGE_DIR).build\Debug\libpng15_staticd.lib"
"$(PNG_INSTALL_DIR)\lib\libpng_d.lib"
	$(CP) "$(PNG_STAGE_DIR).build\pnglibconf.h"
"$(PNG_INSTALL_DIR)\include\pnglibconf.h"
	$(CP) "$(PNG_STAGE_DIR)\pngconf.h" "$(PNG_INSTALL_DIR)\include\pngconf.h"
	$(CP) "$(PNG_STAGE_DIR)\png.h" "$(PNG_INSTALL_DIR)\include\png.h"

png-clean:
	-$(RM) "$(PNG_STAGE_DIR).tar"
	-$(RM_R) "$(PNG_STAGE_DIR)"
	-$(RM_R) "$(PNG_STAGE_DIR).build"

png-cleanall:	png-clean
	-$(RM_R) "$(PNG_INSTALL_DIR)"


#
# Python
#

python:		$(PYTHON_DEPS)

$(PYTHON_DEPS):	python-extract python-configure python-compile
python-install python-clean

python-extract:
	-$(MKDIR) "$(STAGE_DIR)"
	$(RM) "$(PYTHON_STAGE_DIR).tar"
	$(GZIP) -d -c "$(SOURCE_DIR)\Python-$(PYTHON_VERSION).tgz" >
"$(PYTHON_STAGE_DIR).tar"
	$(TAR) -C "$(STAGE_DIR)" -xf "$(PYTHON_STAGE_DIR).tar"
	$(RM) "$(PYTHON_STAGE_DIR).tar"

python-configure:

python-compile:
	msbuild "$(MAKEDIR)\$(PYTHON_STAGE_DIR)\$(PYTHON_PCBUILD)\pcbuild.sln" \
		/target:pythoncore \
		/maxcpucount:$(MAXCPU_COUNT) \
		/verbosity:detailed \
		/property:useenv=true \
		/property:Configuration=Debug \
		/property:Platform=Win32
	msbuild "$(MAKEDIR)\$(PYTHON_STAGE_DIR)\$(PYTHON_PCBUILD)\pcbuild.sln" \
		/target:pythoncore \
		/maxcpucount:$(MAXCPU_COUNT) \
		/verbosity:detailed \
		/property:useenv=true \
		/property:Configuration=Release \
		/property:Platform=Win32

python-install:
	-$(MKDIR) "$(INSTALL_DIR)"
	-$(MKDIR) "$(PYTHON_INSTALL_DIR)"
	-$(MKDIR) "$(PYTHON_INSTALL_DIR)\lib"
	$(CP) "$(PYTHON_STAGE_DIR)\$(PYTHON_PCBUILD)\python$(PYTHON_VERSION_LIBNAME).lib"
"$(PYTHON_INSTALL_DIR)\lib\python$(PYTHON_VERSION_LIBNAME).lib"
	$(CP) "$(PYTHON_STAGE_DIR)\$(PYTHON_PCBUILD)\python$(PYTHON_VERSION_LIBNAME).dll"
"$(PYTHON_INSTALL_DIR)\lib\python$(PYTHON_VERSION_LIBNAME).dll"
	$(CP) "$(PYTHON_STAGE_DIR)\$(PYTHON_PCBUILD)\python$(PYTHON_VERSION_LIBNAME)_d.lib"
"$(PYTHON_INSTALL_DIR)\lib\python$(PYTHON_VERSION_LIBNAME)_d.lib"
	$(CP) "$(PYTHON_STAGE_DIR)\$(PYTHON_PCBUILD)\python$(PYTHON_VERSION_LIBNAME)_d.dll"
"$(PYTHON_INSTALL_DIR)\lib\python$(PYTHON_VERSION_LIBNAME)_d.dll"
	$(CP_R) "$(PYTHON_STAGE_DIR)\Include" "$(PYTHON_INSTALL_DIR)\include"

python-clean:
	-$(RM) "$(PYTHON_STAGE_DIR).tar"
	-$(RM_R) "$(PYTHON_STAGE_DIR)"

python-cleanall:	python-clean
	-$(RM_R) "$(PYTHON_INSTALL_DIR)"


#
# TIFF
#

tiff:		$(TIFF_DEPS)

$(TIFF_DEPS):	tiff-extract tiff-configure tiff-compile tiff-install tiff-clean

tiff-extract:
	-$(MKDIR) "$(STAGE_DIR)"
	$(RM) "$(TIFF_STAGE_DIR).tar"
	$(GZIP) -d -c "$(SOURCE_DIR)\$(TIFF_TARBALL)" > "$(TIFF_STAGE_DIR).tar"
	$(TAR) -C "$(STAGE_DIR)" -xf "$(TIFF_STAGE_DIR).tar"
	$(RM) "$(TIFF_STAGE_DIR).tar"

tiff-configure:

tiff-compile:
	cd "$(TIFF_STAGE_DIR)" & nmake /f Makefile.vc lib

tiff-install:
	-$(MKDIR) "$(INSTALL_DIR)"
	-$(MKDIR) "$(TIFF_INSTALL_DIR)"
	-$(MKDIR) "$(TIFF_INSTALL_DIR)\include"
	-$(MKDIR) "$(TIFF_INSTALL_DIR)\lib"
	$(CP) "$(TIFF_STAGE_DIR)\libtiff\libtiff.lib"
"$(TIFF_INSTALL_DIR)\lib\libtiff.lib"
	$(CP) "$(TIFF_STAGE_DIR)\libtiff\tiffconf.h"
"$(TIFF_INSTALL_DIR)\include\tiffconf.h"
	$(CP) "$(TIFF_STAGE_DIR)\libtiff\tiffio.h"
"$(TIFF_INSTALL_DIR)\include\tiffio.h"
	$(CP) "$(TIFF_STAGE_DIR)\libtiff\tiffvers.h"
"$(TIFF_INSTALL_DIR)\include\tiffvers.h"
	$(CP) "$(TIFF_STAGE_DIR)\libtiff\tiff.h" "$(TIFF_INSTALL_DIR)\include\tiff.h"

tiff-clean:
	-$(RM) "$(TIFF_STAGE_DIR).tar"
	-$(RM_R) "$(TIFF_STAGE_DIR)"

tiff-cleanall:	tiff-clean
	-$(RM_R) "$(TIFF_INSTALL_DIR)"



#
# Zlib
#

zlib:		$(ZLIB_DEPS)

$(ZLIB_DEPS):	zlib-extract zlib-configure zlib-compile zlib-install zlib-clean

zlib-extract:
	-$(MKDIR) "$(STAGE_DIR)"
	$(RM) "$(ZLIB_STAGE_DIR).tar"
	$(GZIP) -d -c "$(SOURCE_DIR)\$(ZLIB_TARBALL)" > "$(ZLIB_STAGE_DIR).tar"
	$(TAR) -C "$(STAGE_DIR)" -xf "$(ZLIB_STAGE_DIR).tar"
	$(RM) "$(ZLIB_STAGE_DIR).tar"

zlib-configure:

zlib-compile:
	cd "$(ZLIB_STAGE_DIR)" & \
		nmake /f win32\Makefile.msc zlib.lib zlib1.dll

zlib-install:
	-$(MKDIR) "$(INSTALL_DIR)"
	-$(MKDIR) "$(ZLIB_INSTALL_DIR)"
	-$(MKDIR) "$(ZLIB_INSTALL_DIR)\include"
	-$(MKDIR) "$(ZLIB_INSTALL_DIR)\lib"
	$(CP) "$(ZLIB_STAGE_DIR)\zlib.lib" "$(ZLIB_INSTALL_DIR)\lib\zlib.lib"
	$(CP) "$(ZLIB_STAGE_DIR)\zlib.pdb" "$(ZLIB_INSTALL_DIR)\lib\zlib.pdb"
	$(CP) "$(ZLIB_STAGE_DIR)\zlib1.dll" "$(ZLIB_INSTALL_DIR)\lib\zlib1.dll"
	$(CP) "$(ZLIB_STAGE_DIR)\win32\zlib.def" "$(ZLIB_INSTALL_DIR)\lib\zlib.def"
	$(CP) "$(ZLIB_STAGE_DIR)\zconf.h" "$(ZLIB_INSTALL_DIR)\include\zconf.h"
	$(CP) "$(ZLIB_STAGE_DIR)\zlib.h" "$(ZLIB_INSTALL_DIR)\include\zlib.h"

zlib-clean:
	-$(RM) "$(ZLIB_STAGE_DIR).tar"
	-$(RM_R) "$(ZLIB_STAGE_DIR)"

zlib-cleanall:	zlib-clean
	-$(RM_R) "$(ZLIB_INSTALL_DIR)"


 -<>-<>-<>-<>-<>-<>-<>-<>-<>-<>-<>-<>-<>-<>-<>-<>-<>-<>-<>-<>-<>-<>-<>-<>-<>-<>-

This makefile assumes these files exist:

../source/Python-3.3.0.tgz
../source/boost_1_52_0.tar.gz
../source/jpegsrc.v8d.tar.gz
../source/libpng-1.5.13.tar.gz
../source/tiff-3.9.2.tar.gz
../source/zlib-1.2.7.tar.gz


-Chad


More information about the Bf-committers mailing list