[Bf-blender-cvs] [c73a99ef902] blender-v2.81-release: Initial implementation of code signing routines

Sergey Sharybin noreply at git.blender.org
Wed Nov 13 09:37:54 CET 2019


Commit: c73a99ef902b21cc0ed2b03daffa9f1adfb70412
Author: Sergey Sharybin
Date:   Thu Nov 7 16:50:31 2019 +0100
Branches: blender-v2.81-release
https://developer.blender.org/rBc73a99ef902b21cc0ed2b03daffa9f1adfb70412

Initial implementation of code signing routines

This changes integrates code signing steps into a buildbot worker
process.

The configuration requires having a separate machine running with
a shared folder access between the signing machine and worker machine.

Actual signing is happening as a "POST-INSTALL" script run by CMake,
which allows to sign any binary which ends up in the final bundle.
Additionally, such way allows to avoid signing binaries in the build
folder (if we were signing as a built process, which iwas another
alternative).
Such complexity is needed on platforms which are using CPack to
generate final bundle: CPack runs INSTALL target into its own location,
so it is useless to run signing on a folder which is considered INSTALL
by the buildbot worker.

There is a signing script which can be used as a standalone tool,
making it possible to hook up signing for macOS's bundler.

There is a dummy Linux signer implementation, which can be activated
by returning True from mock_codesign in linux_code_signer.py.
Main purpose of this signer is to give an ability to develop the
scripts on Linux environment, without going to Windows VM.

The code is based on D6036 from Nathan Letwory.

Differential Revision: https://developer.blender.org/D6216

===================================================================

M	.gitignore
M	CMakeLists.txt
A	build_files/buildbot/README.md
A	build_files/buildbot/codesign/absolute_and_relative_filename.py
A	build_files/buildbot/codesign/archive_with_indicator.py
A	build_files/buildbot/codesign/base_code_signer.py
A	build_files/buildbot/codesign/config_builder.py
A	build_files/buildbot/codesign/config_common.py
A	build_files/buildbot/codesign/config_server_template.py
A	build_files/buildbot/codesign/linux_code_signer.py
A	build_files/buildbot/codesign/simple_code_signer.py
A	build_files/buildbot/codesign/util.py
A	build_files/buildbot/codesign/windows_code_signer.py
A	build_files/buildbot/codesign_server_linux.py
A	build_files/buildbot/codesign_server_windows.bat
A	build_files/buildbot/codesign_server_windows.py
A	build_files/buildbot/slave_codesign.cmake
A	build_files/buildbot/slave_codesign.py
M	build_files/buildbot/slave_compile.py
M	build_files/buildbot/slave_pack.py
M	source/creator/CMakeLists.txt

===================================================================

diff --git a/.gitignore b/.gitignore
index ef39eb5796c..a62802c42fb 100644
--- a/.gitignore
+++ b/.gitignore
@@ -40,3 +40,6 @@ Desktop.ini
 
 # in-source lib downloads
 /build_files/build_environment/downloads
+
+# in-source buildbot signing configuration
+/build_files/buildbot/codesign/config_server.py
\ No newline at end of file
diff --git a/CMakeLists.txt b/CMakeLists.txt
index e468ae36906..c5c65f8a371 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -590,6 +590,10 @@ if(UNIX AND NOT APPLE)
   mark_as_advanced(WITH_CXX11_ABI)
 endif()
 
+# Installation process.
+option(POSTINSTALL_SCRIPT "Run given CMake script after installation process" OFF)
+mark_as_advanced(POSTINSTALL_SCRIPT)
+
 # avoid using again
 option_defaults_clear()
 
diff --git a/build_files/buildbot/README.md b/build_files/buildbot/README.md
new file mode 100644
index 00000000000..cf129f83b39
--- /dev/null
+++ b/build_files/buildbot/README.md
@@ -0,0 +1,70 @@
+Blender Buildbot
+================
+
+Code signing
+------------
+
+Code signing is done as part of INSTALL target, which makes it possible to sign
+files which are aimed into a bundle and coming from a non-signed source (such as
+libraries SVN).
+
+This is achieved by specifying `slave_codesign.cmake` as a post-install script
+run by CMake. This CMake script simply involves an utility script written in
+Python which takes care of an actual signing.
+
+### Configuration
+
+Client configuration doesn't need anything special, other than variable
+`SHARED_STORAGE_DIR` pointing to a location which is watched by a server.
+This is done in `config_builder.py` file and is stored in Git (which makes it
+possible to have almost zero-configuration buildbot machines).
+
+Server configuration requires copying `config_server_template.py` under the
+name of `config_server.py` and tweaking values, which are platform-specific.
+
+#### Windows configuration
+
+There are two things which are needed on Windows in order to have code signing
+to work:
+
+- `TIMESTAMP_AUTHORITY_URL` which is most likely set http://timestamp.digicert.com
+- `CERTIFICATE_FILEPATH` which is a full file path to a PKCS #12 key (.pfx).
+
+## Tips
+
+### Self-signed certificate on Windows
+
+It is easiest to test configuration using self-signed certificate.
+
+The certificate manipulation utilities are coming with Windows SDK.
+Unfortunately, they are not added to PATH. Here is an example of how to make
+sure they are easily available:
+
+```
+set PATH=C:\Program Files (x86)\Windows Kits\10\App Certification Kit;%PATH%
+set PATH=C:\Program Files (x86)\Windows Kits\10\bin\10.0.18362.0\x64;%PATH%
+```
+
+Generate CA:
+
+```
+makecert -r -pe -n "CN=Blender Test CA" -ss CA -sr CurrentUser -a sha256 ^
+         -cy authority -sky signature -sv BlenderTestCA.pvk BlenderTestCA.cer
+```
+
+Import the generated CA:
+
+```
+certutil -user -addstore Root BlenderTestCA.cer
+```
+
+Create self-signed certificate and pack it into PKCS #12:
+
+```
+makecert -pe -n "CN=Blender Test SPC" -a sha256 -cy end ^
+         -sky signature ^
+         -ic BlenderTestCA.cer -iv BlenderTestCA.pvk ^
+         -sv BlenderTestSPC.pvk BlenderTestSPC.cer
+
+pvk2pfx -pvk BlenderTestSPC.pvk -spc BlenderTestSPC.cer -pfx BlenderTestSPC.pfx
+```
\ No newline at end of file
diff --git a/build_files/buildbot/codesign/absolute_and_relative_filename.py b/build_files/buildbot/codesign/absolute_and_relative_filename.py
new file mode 100644
index 00000000000..bea9ea7e8d0
--- /dev/null
+++ b/build_files/buildbot/codesign/absolute_and_relative_filename.py
@@ -0,0 +1,77 @@
+# ##### BEGIN GPL LICENSE BLOCK #####
+#
+#  This program is free software; you can redistribute it and/or
+#  modify it under the terms of the GNU General Public License
+#  as published by the Free Software Foundation; either version 2
+#  of the License, or (at your option) any later version.
+#
+#  This program is distributed in the hope that it will be useful,
+#  but WITHOUT ANY WARRANTY; without even the implied warranty of
+#  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+#  GNU General Public License for more details.
+#
+#  You should have received a copy of the GNU General Public License
+#  along with this program; if not, write to the Free Software Foundation,
+#  Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
+#
+# ##### END GPL LICENSE BLOCK #####
+
+# <pep8 compliant>
+
+from dataclasses import dataclass
+from pathlib import Path
+from typing import List
+
+
+ at dataclass
+class AbsoluteAndRelativeFileName:
+    """
+    Helper class which keeps track of absolute file path for a direct access and
+    corresponding relative path against given base.
+
+    The relative part is used to construct a file name within an archive which
+    contains files which are to be signed or which has been signed already
+    (depending on whether the archive is addressed to signing server or back
+    to the buildbot worker).
+    """
+
+    # Base directory which is where relative_filepath is relative to.
+    base_dir: Path
+
+    # Full absolute path of the corresponding file.
+    absolute_filepath: Path
+
+    # Derived from full file path, contains part of the path which is relative
+    # to a desired base path.
+    relative_filepath: Path
+
+    def __init__(self, base_dir: Path, filepath: Path):
+        self.base_dir = base_dir
+        self.absolute_filepath = filepath.resolve()
+        self.relative_filepath = self.absolute_filepath.relative_to(
+            self.base_dir)
+
+    @classmethod
+    def from_path(cls, path: Path) -> 'AbsoluteAndRelativeFileName':
+        assert path.is_absolute()
+        assert path.is_file()
+
+        base_dir = path.parent
+        return AbsoluteAndRelativeFileName(base_dir, path)
+
+    @classmethod
+    def recursively_from_directory(cls, base_dir: Path) \
+            -> List['AbsoluteAndRelativeFileName']:
+        """
+        Create list of AbsoluteAndRelativeFileName for all the files in the
+        given directory.
+        """
+        assert base_dir.is_absolute()
+        assert base_dir.is_dir()
+
+        result = []
+        for filename in base_dir.glob('**/*'):
+            if not filename.is_file():
+                continue
+            result.append(AbsoluteAndRelativeFileName(base_dir, filename))
+        return result
diff --git a/build_files/buildbot/codesign/archive_with_indicator.py b/build_files/buildbot/codesign/archive_with_indicator.py
new file mode 100644
index 00000000000..51bcc28520d
--- /dev/null
+++ b/build_files/buildbot/codesign/archive_with_indicator.py
@@ -0,0 +1,101 @@
+# ##### BEGIN GPL LICENSE BLOCK #####
+#
+#  This program is free software; you can redistribute it and/or
+#  modify it under the terms of the GNU General Public License
+#  as published by the Free Software Foundation; either version 2
+#  of the License, or (at your option) any later version.
+#
+#  This program is distributed in the hope that it will be useful,
+#  but WITHOUT ANY WARRANTY; without even the implied warranty of
+#  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+#  GNU General Public License for more details.
+#
+#  You should have received a copy of the GNU General Public License
+#  along with this program; if not, write to the Free Software Foundation,
+#  Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
+#
+# ##### END GPL LICENSE BLOCK #####
+
+# <pep8 compliant>
+
+from pathlib import Path
+
+from codesign.util import ensure_file_does_not_exist_or_die
+
+
+class ArchiveWithIndicator:
+    """
+    The idea of this class is to wrap around logic which takes care of keeping
+    track of a name of an archive and synchronization routines between buildbot
+    worker and signing server.
+
+    The synchronization is done based on creating a special file after the
+    archive file is knowingly ready for access.
+    """
+
+    # Base directory where the archive is stored (basically, a basename() of
+    # the absolute archive file name).
+    #
+    # For example, 'X:\\TEMP\\'.
+    base_dir: Path
+
+    # Absolute file name of the archive.
+    #
+    # For example, 'X:\\TEMP\\FOO.ZIP'.
+    archive_filepath: Path
+
+    # Absolute name of a file which acts as an indication of the fact that the
+    # archive is ready and is available for access.
+    #
+    # This is how synchronization between buildbot worker and signing server is
+    # done:
+    # - First, the archive is created under archive_filepath name.
+    # - Second, the indication file is created under ready_indicator_filepath
+    #   name.
+    # - Third, the colleague of whoever created the indicator name watches for
+    #   the indication file to appear, and once it's there it access the
+    #   archive.
+    ready_indicator_filepath: Path
+
+    def __init__(
+            self, base_dir: Path, archive_name: str, ready_indicator_name: str):
+        """
+        Construct the object from given base directory and name of the archive
+        file:
+          ArchiveWithIndicator(Path('X:\\TEMP'), 'FOO.ZIP', 'INPUT_READY')
+        """
+
+        self.base_dir = base_dir
+        self.archive_filepath = self.base_dir / archive_name
+        self.ready_indicator_filepath = self.base_dir / ready_indicator_name
+
+    def is_ready(self) -> bool:
+        """Check whether the archive is ready for access."""
+        return self.ready_indicator_filepath.exists()
+
+    def tag_ready(self) -> None:
+        """
+        Tag the archive as ready by creating the corresponding indication file.
+
+        NOTE: It is expected that the archive was never tagged as ready before
+              and that there are no subsequent tags of the same archive.
+              If it is violated, an assert will fail.
+        """
+        assert not self.is_ready()
+        self.ready_indicator_filepath.touch()
+
+    def clean(self) -> None:
+        """
+        Remove both archive and the ready indication file.
+        """
+        ensure_file_does_not_exist_or_die(self.ready_indicator_filepath)
+        ensure_file_does_not_exist_or_die(self.archive_filepath)
+
+    def is_fully_absent(self) -> bool:
+        """
+        Check whether both archive and

@@ Diff output truncated at 10240 characters. @@



More information about the Bf-blender-cvs mailing list