[Bf-blender-cvs] [517870a4a11] blender-v2.82-release: CMake: Refactor external dependencies handling

Sergey Sharybin noreply at git.blender.org
Thu Jan 23 16:59:33 CET 2020


Commit: 517870a4a11f660c71d3901818fbb09798cb2d7d
Author: Sergey Sharybin
Date:   Mon Jan 20 18:36:19 2020 +0100
Branches: blender-v2.82-release
https://developer.blender.org/rB517870a4a11f660c71d3901818fbb09798cb2d7d

CMake: Refactor external dependencies handling

This is a more correct fix to the issue Brecht was fixing in D6600.

While the fix in that patch worked fine for linking it broke ASAN
runtime under some circumstances.
For example, `make full debug developer` would compile, but trying
to start blender will cause assert failure in ASAN (related on check
that ASAN is not running already).

Top-level idea: leave it to CMake to keep track of dependency graph.

The root of the issue comes to the fact that target like "blender" is
configured to use a lot of static libraries coming from Blender sources
and to use external static libraries. There is nothing which ensures
order between blender's and external libraries. Only order of blender
libraries is guaranteed.

It was possible that due to a cycle or other circumstances some of
blender libraries would have been passed to linker after libraries
it uses, causing linker errors.

For example, this order will likely fail:

  libbf_blenfont.a libfreetype6.a libbf_blenfont.a

This change makes it so blender libraries are explicitly provided
their dependencies to an external libraries, which allows CMake to
ensure they are always linked against them.

General rule here: if bf_foo depends on an external library it is
to be provided to LIBS for bf_foo.
For example, if bf_blenkernel depends on opensubdiv then LIBS in
blenkernel's CMakeLists.txt is to include OPENSUBDIB_LIBRARIES.

The change is made based on searching for used include folders
such as OPENSUBDIV_INCLUDE_DIRS and adding corresponding libraries
to LIBS ion that CMakeLists.txt. Transitive dependencies are not
simplified by this approach, but I am not aware of any downside of
this: CMake should be smart enough to simplify them on its side.
And even if not, this shouldn't affect linking time.

Benefit of not relying on transitive dependencies is that build
system is more robust towards future changes. For example, if
bf_intern_opensubiv is no longer depends on OPENSUBDIV_LIBRARIES
and all such code is moved to bf_blenkernel this will not break
linking.

The not-so-trivial part is change to blender_add_lib (and its
version in Cycles). The complexity is caused by libraries being
provided as a single list argument which doesn't allow to use
different release and debug libraries on Windows. The idea is:

- Have every library prefixed as "optimized" or "debug" if
  separation is needed (non-prefixed libraries will be considered
  "generic").

- Loop through libraries passed to function and do simple parsing
  which will look for "optimized" and "debug" words and specify
  following library to corresponding category.

This isn't something particularly great. Alternative would be to
use target_link_libraries() directly, which sounds like more code
but which is more explicit and allows to have more flexibility
and control comparing to wrapper approach.

Tested the following configurations on Linux, macOS and Windows:

- make full debug developer
- make full release developer
- make lite debug developer
- make lite release developer

NOTE: Linux libraries needs to be compiled with D6641 applied,
otherwise, depending on configuration, it's possible to run into
duplicated zlib symbols error.

Differential Revision: https://developer.blender.org/D6642

===================================================================

M	CMakeLists.txt
M	build_files/cmake/macros.cmake
M	build_files/cmake/platform/platform_apple.cmake
M	build_files/cmake/platform/platform_unix.cmake
M	build_files/cmake/platform/platform_win32.cmake
M	extern/ceres/CMakeLists.txt
M	extern/ceres/bundle.sh
M	extern/mantaflow/CMakeLists.txt
M	extern/quadriflow/CMakeLists.txt
M	intern/audaspace/CMakeLists.txt
M	intern/cycles/blender/CMakeLists.txt
M	intern/cycles/bvh/CMakeLists.txt
M	intern/cycles/cmake/macros.cmake
M	intern/cycles/device/CMakeLists.txt
M	intern/cycles/graph/CMakeLists.txt
M	intern/cycles/kernel/osl/CMakeLists.txt
M	intern/cycles/render/CMakeLists.txt
M	intern/ghost/CMakeLists.txt
M	intern/libmv/CMakeLists.txt
M	intern/libmv/bundle.sh
M	intern/locale/CMakeLists.txt
M	intern/mantaflow/CMakeLists.txt
M	intern/opencolorio/CMakeLists.txt
M	intern/opensubdiv/CMakeLists.txt
M	intern/openvdb/CMakeLists.txt
M	intern/quadriflow/CMakeLists.txt
M	intern/rigidbody/CMakeLists.txt
M	source/blender/alembic/CMakeLists.txt
M	source/blender/avi/CMakeLists.txt
M	source/blender/blenfont/CMakeLists.txt
M	source/blender/blenkernel/CMakeLists.txt
M	source/blender/blenlib/CMakeLists.txt
M	source/blender/blentranslation/msgfmt/CMakeLists.txt
M	source/blender/bmesh/CMakeLists.txt
M	source/blender/collada/CMakeLists.txt
M	source/blender/compositor/CMakeLists.txt
M	source/blender/editors/sound/CMakeLists.txt
M	source/blender/editors/space_graph/CMakeLists.txt
M	source/blender/editors/space_sequencer/CMakeLists.txt
M	source/blender/freestyle/CMakeLists.txt
M	source/blender/gpu/CMakeLists.txt
M	source/blender/imbuf/CMakeLists.txt
M	source/blender/imbuf/intern/oiio/CMakeLists.txt
M	source/blender/imbuf/intern/openexr/CMakeLists.txt
M	source/blender/makesrna/intern/CMakeLists.txt
M	source/blender/nodes/CMakeLists.txt
M	source/blender/physics/CMakeLists.txt
M	source/blender/python/bmesh/CMakeLists.txt
M	source/blender/python/generic/CMakeLists.txt
M	source/blender/python/gpu/CMakeLists.txt
M	source/blender/python/intern/CMakeLists.txt
M	source/blender/python/mathutils/CMakeLists.txt
M	source/blender/usd/CMakeLists.txt
M	source/blender/windowmanager/CMakeLists.txt
M	tests/gtests/alembic/CMakeLists.txt
M	tests/gtests/blenlib/CMakeLists.txt
M	tests/gtests/usd/CMakeLists.txt

===================================================================

diff --git a/CMakeLists.txt b/CMakeLists.txt
index 6f8b512b69b..70196d00df6 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -780,9 +780,11 @@ if(NOT CMAKE_BUILD_TYPE MATCHES "Release")
     if(MSVC)
       set(COMPILER_ASAN_LINKER_FLAGS "/FUNCTIONPADMIN:6")
     endif()
-    set(PLATFORM_LINKLIBS "${PLATFORM_LINKLIBS};${COMPILER_ASAN_LIBRARY}")
-    set(PLATFORM_LINKFLAGS "${COMPILER_ASAN_LIBRARY} ${COMPILER_ASAN_LINKER_FLAGS}")
-    set(PLATFORM_LINKFLAGS_DEBUG "${COMPILER_ASAN_LIBRARY} ${COMPILER_ASAN_LINKER_FLAGS}")
+    if(COMPILER_ASAN_LIBRARY)
+      set(PLATFORM_LINKLIBS "${PLATFORM_LINKLIBS};${COMPILER_ASAN_LIBRARY}")
+      set(PLATFORM_LINKFLAGS "${COMPILER_ASAN_LIBRARY} ${COMPILER_ASAN_LINKER_FLAGS}")
+      set(PLATFORM_LINKFLAGS_DEBUG "${COMPILER_ASAN_LIBRARY} ${COMPILER_ASAN_LINKER_FLAGS}")
+    endif()
   endif()
 endif()
 
diff --git a/build_files/cmake/macros.cmake b/build_files/cmake/macros.cmake
index 23ed5b6cbfb..2d13476f09a 100644
--- a/build_files/cmake/macros.cmake
+++ b/build_files/cmake/macros.cmake
@@ -251,8 +251,59 @@ function(blender_add_lib__impl
 
   add_library(${name} ${sources})
 
+  # On Windows certain libraries have two sets of binaries: one for debug builds and one for
+  # release builds. The root of this requirement goes into ABI, I believe, but that's outside
+  # of a scope of this comment.
+  #
+  # CMake have a native way of dealing with this, which is specifying what build type the
+  # libraries are provided for:
+  #
+  #   target_link_libraries(tagret optimized|debug|general <libraries>)
+  #
+  # The build type is to be provided as a separate argument to the function.
+  #
+  # CMake's variables for libraries will contain build type in such cases. For example:
+  #
+  #   set(FOO_LIBRARIES optimized libfoo.lib debug libfoo_d.lib)
+  #
+  # Complications starts with a single argument for library_deps: all the elements are being
+  # put to a list: "${FOO_LIBRARIES}" will become "optimized;libfoo.lib;debug;libfoo_d.lib".
+  # This makes it impossible to pass it as-is to target_link_libraries sine it will treat
+  # this argument as a list of libraries to be linked against, causing missing libraries
+  # for optimized.lib.
+  #
+  # What this code does it traverses library_deps and extracts information about whether
+  # library is to provided as general, debug or optimized. This is a little state machine which
+  # keeps track of whiuch build type library is to provided for:
+  #
+  # - If "debug" or "optimized" word is found, the next element in the list is expected to be
+  #   a library which will be passed to target_link_libraries() under corresponding build type.
+  #
+  # - If there is no "debug" or "optimized" used library is specified for all build types.
+  #
+  # NOTE: If separated libraries for debug and release ar eneeded every library is the list are
+  # to be prefixed explicitly.
+  #
+  #  Use: "optimized libfoo optimized libbar debug libfoo_d debug libbar_d"
+  #  NOT: "optimized libfoo libbar debug libfoo_d libbar_d"
   if(NOT "${library_deps}" STREQUAL "")
-    target_link_libraries(${name} INTERFACE "${library_deps}")
+    set(next_library_mode "")
+    foreach(library ${library_deps})
+      string(TOLOWER "${library}" library_lower)
+      if(("${library_lower}" STREQUAL "optimized") OR
+         ("${library_lower}" STREQUAL "debug"))
+        set(next_library_mode "${library_lower}")
+      else()
+        if("${next_library_mode}" STREQUAL "optimized")
+          target_link_libraries(${name} optimized ${library})
+        elseif("${next_library_mode}" STREQUAL "debug")
+          target_link_libraries(${name} debug ${library})
+        else()
+          target_link_libraries(${name} ${library})
+        endif()
+        set(next_library_mode "")
+      endif()
+    endforeach()
   endif()
 
   # works fine without having the includes
@@ -404,6 +455,11 @@ function(setup_liblinks
   target
   )
 
+  # NOTE: This might look like it affects global scope, accumulating linker flags on every call
+  # to setup_liblinks, but this isn't how CMake works. These flags will only affect current
+  # directory from where the function is called.
+  # This means that setup_liblinks() called for ffmpeg_test will not affect blender, and each
+  # of thsoe targets will have single set of linker flags.
   set(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} ${PLATFORM_LINKFLAGS}" PARENT_SCOPE)
   set(CMAKE_EXE_LINKER_FLAGS_DEBUG "${CMAKE_EXE_LINKER_FLAGS_DEBUG} ${PLATFORM_LINKFLAGS_DEBUG}" PARENT_SCOPE)
   set(CMAKE_EXE_LINKER_FLAGS_RELEASE "${CMAKE_EXE_LINKER_FLAGS_RELEASE} ${PLATFORM_LINKFLAGS_RELEASE}" PARENT_SCOPE)
@@ -416,201 +472,17 @@ function(setup_liblinks
   set(CMAKE_MODULE_LINKER_FLAGS_DEBUG "${CMAKE_MODULE_LINKER_FLAGS_DEBUG} ${PLATFORM_LINKFLAGS_DEBUG}" PARENT_SCOPE)
   set(CMAKE_MODULE_LINKER_FLAGS_RELEASE "${CMAKE_MODULE_LINKER_FLAGS_RELEASE} ${PLATFORM_LINKFLAGS_RELEASE}" PARENT_SCOPE)
 
-  # Work around undefined reference errors when disabling certain libraries.
-  # Finding the right order for all combinations of options is too hard, so
-  # we use --start-group and --end-group so the linker does not discard symbols
-  # too early. This appears to have no significant performance impact.
-  if(UNIX AND NOT APPLE)
-    target_link_libraries(
-      ${target}
-      -Wl,--start-group
-    )
-  endif()
-
   # jemalloc must be early in the list, to be before pthread (see T57998)
   if(WITH_MEM_JEMALLOC)
     target_link_libraries(${target} ${JEMALLOC_LIBRARIES})
   endif()
 
-  target_link_libraries(
-    ${target}
-    ${PNG_LIBRARIES}
-    ${FREETYPE_LIBRARY}
-  )
-
-
-  if(WITH_PYTHON)
-    target_link_libraries(${target} ${PYTHON_LINKFLAGS})
-    target_link_libraries(${target} ${PYTHON_LIBRARIES})
-  endif()
-
-  if(WITH_LZO AND WITH_SYSTEM_LZO)
-    target_link_libraries(${target} ${LZO_LIBRARIES})
-  endif()
-  if(WITH_SYSTEM_GLEW)
-    target_link_libraries(${target} ${BLENDER_GLEW_LIBRARIES})
-  endif()
-  if(WITH_BULLET AND WITH_SYSTEM_BULLET)
-    target_link_libraries(${target} ${BULLET_LIBRARIES})
-  endif()
-  if(WITH_AUDASPACE AND WITH_SYSTEM_AUDASPACE)
-    target_link_libraries(${target} ${AUDASPACE_C_LIBRARIES} ${AUDASPACE_PY_LIBRARIES})
-  endif()
-  if(WITH_OPENAL)
-    target_link_libraries(${target} ${OPENAL_LIBRARY})
-  endif()
-  if(WITH_FFTW3)
-    target_link_libraries(${target} ${FFTW3_LIBRARIES})
-  endif()
-  if(WITH_JACK AND NOT WITH_JACK_DYNLOAD)
-    target_link_libraries(${target} ${JACK_LIBRARIES})
-  endif()
-  if(WITH_CODEC_SNDFILE)
-    target_link_libraries(${target} ${LIBSNDFILE_LIBRARIES})
-  endif()
-  if(WITH_SDL AND NOT WITH_SDL_DYNLOAD)
-    target_link_libraries(${target} ${SDL_LIBRARY})
-  endif()
-  if(WITH_CYCLES_OSL)
-    target_link_libraries(${target} ${OSL_LIBRARIES})
-  endif()
-  if(WITH_OPENVDB)
-    target_link_libraries(${target} ${OPENVDB_LIBRARIES} ${BLOSC_LIBRARIES})
-  endif()
-  if(WITH_USD)
-    # Source: https://github.com/PixarAnimationStudios/USD/blob/master/BUILDING.md#linking-whole-archives
-    if(WIN32)
-      target_link_libraries(${target} ${USD_LIBRARIES})
-      set_property(TARGET ${target} APPEND_STRING PROPERTY LINK_FLAGS_DEBUG " /WHOLEARCHIVE:${USD_DEBUG_LIB}")
-      set_property(TARGET ${target} APPEND_STRING PROPERTY LINK_FLAGS_RELEASE " /WHOLEARCHIVE:${USD_RELEASE_LIB}")
-      set_property(TARGET ${target} APPEND_STRING PROPERTY LINK_FLAGS_RELWITHDEBINFO " /WHOLEARCHIVE:${USD_RELEASE_LIB}")
-      set_property(TARGET ${target} APPEND_STRING PROPERTY LINK_FLAGS_MINSIZEREL " /WHOLEARCHIVE:${USD_RELEASE_LIB}")
-    elseif(CMAKE_COMPILER_IS_GNUCXX)
-      target_link_libraries(${target} -Wl,--whole-archive ${USD_LIBRARIES} -Wl,--no-whole-archive)
-    elseif("${CMAKE_CXX_COMPILER_ID}" MATCHES "Clang")
-      target_link_libraries(${target} -Wl,-force_load ${USD_LIBRARIES})
-    else()
-      message(FATAL_ERROR "Unknown how to link USD with your compiler ${CMAKE_CXX_COMPILER_ID}")
-    endif()
-  endif()
-  if(WITH_OPENIMAGEIO)
-    target_link_libraries(${target} ${OPENIMAGEIO_LIBRARIES})
-  endif()
-  if(WITH_OPENIMAGEDENOISE)
-    target_link_libraries(${target} ${OPENIMAGEDENOISE_LIBRARIES})
-  endif()
-  if(WITH_TBB)
-    target_link_libraries(${target} ${TBB_LIBRARIES})
-  endif()
-  if(WITH_OPENCOLORIO)
-    target_link_libraries(${target} ${OPENCOLORIO_LIBRARIES})
-  endif()
-  if(WITH_OPENSUBDIV)
-      target_link_libraries(${target} ${OPENSUBDIV_LIBRARIES})
-  endif()
-  if(WITH_CYCLES_EMBREE)
-    target_link_libraries(${target} ${EMBREE_LIBRARIES})
-  endif()
-  if(WITH_BOOST)
-    target_link_libraries(${target} ${BOOST_LIBRARIES})
-    if(Boost_USE_STATIC_LIBS AND Boost_USE_ICU)
-      target_link_libraries(${target} ${ICU_LIBRARIES})
-    endif()
-  endif()
-  target_link_libraries(${target} ${JPEG_LIBRARIES})
-  if(WITH_ALEMBIC)
-    target_link_libraries(${target} ${ALEMBIC_LIBRARIES} ${HDF5_LIBRARIES})
-  endif()
-  if(WITH_IMAGE_TIFF)
-    target_link_libraries(${target} ${TIFF_LIBRARY})
-  endif()
-  if(WITH_IMAGE_OPENEXR)
-    target_link_libraries(${target} ${OPENEXR_LIBRARIES})
-  endif()
-  if(WITH_IMAGE_OPENJPEG)
-    target_link_libraries(${target} ${OPENJPEG_LIBRARIES})
-  endif()
-  if(WITH_CODEC_FFMPEG)
-    target_link_libraries(${target} ${FFMPEG_LIBRARIES})
-  endif()
-  if(WITH_OPENCOLLADA)
-    if(WIN32 AND NOT UNIX)
-      file_list_suffix(OPENCOLLADA_LIBRARIES_DEBUG "${OPENCOLLADA_LIBRARIES}" "_d")
-      target_link_libraries_debug(${target} "${OPENCOLLADA_LIBRARIES_DEBUG}")
-      target_link_libraries_optimized(${target} "${OPENCOLLADA_LIBRARIES}")
-      unset(OPENCOLLADA_LIBRARIES_DEBUG)
-
-      file_list_suffix(PCRE_LIBRARIES_DEBUG "${PCRE_LIBRARIES}" "_d")
-      target_link_libraries_debug(${target} "${PCRE_LIBRARIES_DEBUG}")
-      target_link_libraries_optimized(${target} "${PCRE_LIBRARIES}")
-      unset(PCRE_LIBRARIES_DEBUG)
-
-      if(EXPAT_LIB)
-        file_list_suffix(EXPAT_LIB_DEBUG "${EXPAT_LIB}" "_d")
-        target_link_libraries_debug(${target} "${EXPAT_LIB_DEBUG}")
-        target_link_libraries_optimized(${target} "${EXPAT_LIB}")
-        unset(EXPAT_LIB_DEBUG)
-      endif()
-    else()
-      target_link_libraries(
-        ${target}


@@ Diff output truncated at 10240 characters. @@



More information about the Bf-blender-cvs mailing list