[Bf-blender-cvs] [ec62413f803] master: USD: Introducing a simple USD Exporter
Sybren A. Stüvel
noreply at git.blender.org
Fri Dec 13 10:28:12 CET 2019
Commit: ec62413f803ee506633f0e52d1e52b0980c0ed0d
Author: Sybren A. Stüvel
Date: Fri Dec 13 10:27:40 2019 +0100
Branches: master
https://developer.blender.org/rBec62413f803ee506633f0e52d1e52b0980c0ed0d
USD: Introducing a simple USD Exporter
This commit introduces the first version of an exporter to Pixar's
Universal Scene Description (USD) format.
Reviewed By: sergey, LazyDodo
Differential Revision: https://developer.blender.org/D6287
- The USD libraries are built by `make deps`, but not yet built by
install_deps.sh.
- Only experimental support for instancing; by default all duplicated
objects are made real in the USD file. This is fine for exporting a
linked-in posed character, not so much for thousands of pebbles etc.
- The way materials and UV coordinates and Normals are exported is going
to change soon.
- This patch contains LazyDodo's fixes for building on Windows in D5359.
== Meshes ==
USD seems to support neither per-material nor per-face-group
double-sidedness, so we just use the flag from the first non-empty
material slot. If there is no material we default to double-sidedness.
Each UV map is stored on the mesh in a separate primvar. Materials can
refer to these UV maps, but this is not yet exported by Blender. The
primvar name is the same as the UV Map name. This is to allow the
standard name "st" for texture coordinates by naming the UV Map as such,
without having to guess which UV Map is the "standard" one.
Face-varying mesh normals are written to USD. When the mesh has custom
loop normals those are written. Otherwise the poly flag `ME_SMOOTH` is
inspected to determine the normals.
The UV maps and mesh normals take up a significant amount of space, so
exporting them is optional. They're still enabled by default, though.
For comparison: a shot of Spring (03_035_A) is 1.2 GiB when exported
with UVs and normals, and 262 MiB without. We probably have room for
optimisation of written UVs and normals.
The mesh subdivision scheme isn't using the default value 'Catmull
Clark', but uses 'None', indicating we're exporting a polygonal mesh.
This is necessary for USD to understand our normals; otherwise the mesh
is always rendered smooth. In the future we may want to expose this
choice of subdivision scheme to the user, or auto-detect it when we
actually support exporting pre-subdivision meshes.
A possible optimisation could be to inspect whether all polygons are
smooth or flat, and mark the USD mesh as such. This can be added when
needed.
== Animation ==
Mesh and transform animation are now written when passing
`animation=True` to the export operator. There is no inspection of
whether an object is actually animated or not; USD can handle
deduplication of static values for us.
The administration of which timecode to use for the export is left to
the file-format-specific concrete subclasses of
`AbstractHierarchyIterator`; the abstract iterator itself doesn't know
anything about the passage of time. This will allow subclasses for the
frame-based USD format and time-based Alembic format.
== Support for simple preview materials ==
Very simple versions of the materials are now exported, using only the
viewport diffuse RGB, metallic, and roughness.
When there are multiple materials, the mesh faces are stored as geometry
subset and each material is assigned to the appropriate subset. If there
is only one material this is skipped.
The first material if any) is always applied to the mesh itself
(regardless of the existence of geometry subsets), because the Hydra
viewport doesn't support materials on subsets. See
https://github.com/PixarAnimationStudios/USD/issues/542 for more info.
Note that the geometry subsets are not yet time-sampled, so it may break
when an animated mesh changes topology.
Materials are exported as a flat list under a top-level '/_materials'
namespace. This inhibits instancing of the objects using those
materials, so this is subject to change.
== Hair ==
Only the parent strands are exported, and only with a constant colour.
No UV coordinates, no information about the normals.
== Camera ==
Only perspective cameras are supported for now.
== Particles ==
Particles are only written when they are alive, which means that they
are always visible (there is currently no code that deals with marking
them as invisible outside their lifespan).
Particle-system-instanced objects are exported by suffixing the object
name with the particle's persistent ID, giving each particle XForm a
unique name.
== Instancing/referencing ==
This exporter has experimental support for instancing/referencing.
Dupli-object meshes are now written to USD as references to the original
mesh. This is still very limited in correctness, as there are issues
referencing to materials from a referenced mesh.
I am still committing this, as it gives us a place to start when
continuing the quest for proper instancing in USD.
== Lights ==
USD does not directly support spot lights, so those aren't exported yet.
It's possible to add this in the future via the UsdLuxShapingAPI. The
units used for the light intensity are also still a bit of a mystery.
== Fluid vertex velocities ==
Currently only fluid simulations (not meshes in general) have explicit
vertex velocities. This is the most important case for exporting
velocities, though, as the baked mesh changes topology all the time, and
thus computing the velocities at import time in a post-processing step
is hard.
== The Building Process ==
- USD is built as monolithic library, instead of 25 smaller libraries.
We were linking all of them as 'whole archive' anyway, so this doesn't
affect the final file size. It does, however, make life easier with
respect to linking order, and handling upstream changes.
- The JSON files required by USD are installed into datafiles/usd; they
are required on every platform. Set the `PXR_PATH_DEBUG` to any value
to have the USD library print the paths it uses to find those files.
- USD is patched so that it finds the aforementioned JSON files in a path
that we pass to it from Blender.
- USD is patched to have a `PXR_BUILD_USD_TOOLS` CMake option to disable
building the tools in its `bin` directory. This is sent as a pull
request at https://github.com/PixarAnimationStudios/USD/pull/1048
===================================================================
M CMakeLists.txt
M build_files/build_environment/CMakeLists.txt
M build_files/build_environment/cmake/harvest.cmake
A build_files/build_environment/cmake/usd.cmake
M build_files/build_environment/cmake/versions.cmake
A build_files/build_environment/patches/usd.diff
A build_files/cmake/Modules/FindUSD.cmake
M build_files/cmake/config/blender_full.cmake
M build_files/cmake/config/blender_release.cmake
M build_files/cmake/macros.cmake
M build_files/cmake/platform/platform_apple.cmake
M build_files/cmake/platform/platform_unix.cmake
M build_files/cmake/platform/platform_win32.cmake
M release/scripts/startup/bl_ui/space_topbar.py
M release/scripts/startup/bl_ui/space_userpref.py
M source/blender/CMakeLists.txt
M source/blender/editors/io/CMakeLists.txt
M source/blender/editors/io/io_ops.c
A source/blender/editors/io/io_usd.c
A source/blender/editors/io/io_usd.h
M source/blender/editors/space_file/filelist.c
M source/blender/editors/space_file/filesel.c
M source/blender/makesdna/DNA_space_types.h
M source/blender/makesdna/DNA_userdef_types.h
M source/blender/makesrna/intern/rna_userdef.c
M source/blender/python/intern/CMakeLists.txt
M source/blender/python/intern/bpy_app_build_options.c
A source/blender/usd/CMakeLists.txt
A source/blender/usd/intern/abstract_hierarchy_iterator.cc
A source/blender/usd/intern/abstract_hierarchy_iterator.h
A source/blender/usd/intern/usd_capi.cc
A source/blender/usd/intern/usd_exporter_context.h
A source/blender/usd/intern/usd_hierarchy_iterator.cc
A source/blender/usd/intern/usd_hierarchy_iterator.h
A source/blender/usd/intern/usd_writer_abstract.cc
A source/blender/usd/intern/usd_writer_abstract.h
A source/blender/usd/intern/usd_writer_camera.cc
A source/blender/usd/intern/usd_writer_camera.h
A source/blender/usd/intern/usd_writer_hair.cc
A source/blender/usd/intern/usd_writer_hair.h
A source/blender/usd/intern/usd_writer_light.cc
A source/blender/usd/intern/usd_writer_light.h
A source/blender/usd/intern/usd_writer_mesh.cc
A source/blender/usd/intern/usd_writer_mesh.h
A source/blender/usd/intern/usd_writer_transform.cc
A source/blender/usd/intern/usd_writer_transform.h
A source/blender/usd/usd.h
M source/blender/windowmanager/intern/wm_operator_props.c
M source/creator/CMakeLists.txt
M source/creator/creator.c
M tests/gtests/CMakeLists.txt
A tests/gtests/usd/CMakeLists.txt
A tests/gtests/usd/abstract_hierarchy_iterator_test.cc
A tests/gtests/usd/hierarchy_context_order_test.cc
A tests/gtests/usd/usd_stage_creation_test.cc
===================================================================
diff --git a/CMakeLists.txt b/CMakeLists.txt
index 372fa8fa784..e3e6f8c0c66 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -265,6 +265,9 @@ option(WITH_CODEC_SNDFILE "Enable libsndfile Support (http://www.mega-nerd
option(WITH_ALEMBIC "Enable Alembic Support" ON)
option(WITH_ALEMBIC_HDF5 "Enable Legacy Alembic Support (not officially supported)" OFF)
+# Universal Scene Description support
+option(WITH_USD "Enable Universal Scene Description (USD) Support" OFF)
+
# 3D format support
# Disable opencollada when we don't have precompiled libs
option(WITH_OPENCOLLADA "Enable OpenCollada Support (http://www.opencollada.org)" ON)
@@ -1728,6 +1731,7 @@ if(FIRST_RUN)
info_cfg_option(WITH_OPENVDB)
info_cfg_option(WITH_ALEMBIC)
info_cfg_option(WITH_QUADRIFLOW)
+ info_cfg_option(WITH_USD)
info_cfg_text("Compiler Options:")
info_cfg_option(WITH_BUILDINFO)
diff --git a/build_files/build_environment/CMakeLists.txt b/build_files/build_environment/CMakeLists.txt
index cdfa18ff4ff..fb32d2218b8 100644
--- a/build_files/build_environment/CMakeLists.txt
+++ b/build_files/build_environment/CMakeLists.txt
@@ -92,6 +92,7 @@ include(cmake/python.cmake)
include(cmake/python_site_packages.cmake)
include(cmake/package_python.cmake)
include(cmake/numpy.cmake)
+include(cmake/usd.cmake)
if(UNIX AND NOT APPLE)
# Rely on PugiXML compiled with OpenImageIO
else()
diff --git a/build_files/build_environment/cmake/harvest.cmake b/build_files/build_environment/cmake/harvest.cmake
index 89eec7cf72f..cc93db7de64 100644
--- a/build_files/build_environment/cmake/harvest.cmake
+++ b/build_files/build_environment/cmake/harvest.cmake
@@ -197,6 +197,9 @@ harvest(x264/lib ffmpeg/lib "*.a")
harvest(xvidcore/lib ffmpeg/lib "*.a")
harvest(embree/include embree/include "*.h")
harvest(embree/lib embree/lib "*.a")
+harvest(usd/include usd/include "*.h")
+harvest(usd/lib/usd usd/lib/usd "*")
+harvest(usd/plugin usd/plugin "*")
if(UNIX AND NOT APPLE)
harvest(libglu/lib mesa/lib "*.so*")
diff --git a/build_files/build_environment/cmake/usd.cmake b/build_files/build_environment/cmake/usd.cmake
new file mode 100644
index 00000000000..c3594390f80
--- /dev/null
+++ b/build_files/build_environment/cmake/usd.cmake
@@ -0,0 +1,101 @@
+# ***** BEGIN GPL LICENSE BLOCK *****
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License
+# as published by the Free Software Foundation; either version 2
+# of the License, or (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write to the Free Software Foundation,
+# Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
+#
+# ***** END GPL LICENSE BLOCK *****
+
+set(USD_EXTRA_ARGS
+ -DBoost_COMPILER:STRING=${BOOST_COMPILER_STRING}
+ -DBoost_USE_MULTITHREADED=ON
+ -DBoost_USE_STATIC_LIBS=ON
+ -DBoost_USE_STATIC_RUNTIME=OFF
+ -DBOOST_ROOT=${LIBDIR}/boost
+ -DTBB_INCLUDE_DIRS=${LIBDIR}/tbb/include
+ -DTBB_LIBRARIES=${LIBDIR}/tbb/lib/${LIBPREFIX}tbb_static${LIBEXT}
+ -DTbb_TBB_LIBRARY=${LIBDIR}/tbb/lib/${LIBPREFIX}tbb_static${LIBEXT}
+
+ # This is a preventative measure that avoids possible conflicts when add-ons
+ # try to load another USD library into the same process space.
+ -DPXR_SET_INTERNAL_NAMESPACE=usdBlender
+
+ -DPXR_ENABLE_PYTHON_SUPPORT=OFF
+ -DPXR_BUILD_IMAGING=OFF
+ -DPXR_BUILD_TESTS=OFF
+ -DBUILD_SHARED_LIBS=OFF
+ -DPYTHON_EXECUTABLE=${PYTHON_BINARY}
+ -DPXR_BUILD_MONOLITHIC=ON
+
+ # The PXR_BUILD_USD_TOOLS argument is patched-in by usd.diff. An upstream pull request
+ # can be found at https://github.com/PixarAnimationStudios/USD/pull/1048.
+ -DPXR_BUILD_USD_TOOLS=OFF
+
+ -DCMAKE_DEBUG_POSTFIX=_d
+ # USD is hellbound on making a shared lib, unless you point this variable to a valid cmake file
+ # doesn't have to make sense, but as long as it points somewhere valid it will skip the shared lib.
+ -DPXR_MONOLITHIC_IMPORT=${BUILD_DIR}/usd/src/external_usd/cmake/defaults/Version.cmake
+)
+
+ExternalProject_Add(external_usd
+ URL ${USD_URI}
+ DOWNLOAD_DIR ${DOWNLOAD_DIR}
+ URL_HASH MD5=${USD_HASH}
+ PREFIX ${BUILD_DIR}/usd
+ PATCH_COMMAND ${PATCH_CMD} -p 1 -d ${BUILD_DIR}/usd/src/external_usd < ${PATCH_DIR}/usd.diff
+ CMAKE_ARGS -DCMAKE_INSTALL_PREFIX=${LIBDIR}/usd -Wno-dev ${DEFAULT_CMAKE_FLAGS} ${USD_EXTRA_ARGS}
+ INSTALL_DIR ${LIBDIR}/usd
+)
+
+add_dependencies(
+ external_usd
+ external_tbb
+ external_boost
+)
+
+if(WIN32)
+ # USD currently demands python be available at build time
+ # and then proceeds not to use it, but still checks that the
+ # version of the interpreter it is not going to use is atleast 2.7
+ # so we need this dep currently since there is no system python
+ # on windows.
+ add_dependencies(
+ external_usd
+ external_python
+ )
+ if(BUILD_MODE STREQUAL Release)
+ ExternalProject_Add_Step(external_usd after_install
+ COMMAND ${CMAKE_COMMAND} -E copy_directory ${LIBDIR}/usd/ ${HARVEST_TARGET}/usd
+ COMMAND ${CMAKE_COMMAND} -E copy ${BUILD_DIR}/usd/src/external_usd-build/pxr/Release/libusd_m.lib ${HARVEST_TARGET}/usd/lib/libusd_m.lib
+ DEPENDEES install
+ )
+ endif()
+ if(BUILD_MODE STREQUAL Debug)
+ ExternalProject_Add_Step(external_usd after_install
+ COMMAND ${CMAKE_COMMAND} -E copy_directory ${LIBDIR}/usd/lib ${HARVEST_TARGET}/usd/lib
+ COMMAND ${CMAKE_COMMAND} -E copy ${BUILD_DIR}/usd/src/external_usd-build/pxr/Debug/libusd_m_d.lib ${HARVEST_TARGET}/usd/lib/libusd_m_d.lib
+ DEPENDEES install
+ )
+ endif()
+else()
+ # USD has two build options. The default build creates lots of small libraries,
+ # whereas the 'monolithic' build produces only a single library. The latter
+ # makes linking simpler, so that's what we use in Blender. However, running
+ # 'make install' in the USD sources doesn't install the static library in that
+ # case (only the shared library). As a result, we need to grab the `libusd_m.a`
+ # file from the build directory instead of from the install directory.
+ ExternalProject_Add_Step(external_usd after_install
+ COMMAND ${CMAKE_COMMAND} -E copy ${BUILD_DIR}/usd/src/external_usd-build/pxr/libusd_m.a ${HARVEST_TARGET}/usd/lib/libusd_m.a
+ DEPENDEES install
+ )
+endif()
diff --git a/build_files/build_environment/cmake/versions.cmake b/build_files/build_environment/cmake/versions.cmake
index 9cbf104e842..2b08a74c1aa 100644
--- a/build_files/build_environment/cmake/versions.cmake
+++ b/build_files/build_environment/cmake/versions.cmake
@@ -307,6 +307,10 @@ set(EMBREE_VERSION 3.2.4)
set(EMBREE_URI https://github.com/embree/embree/archive/v${EMBREE_VERSION}.zip)
set(EMBREE_HASH 3d4a1147002ff43939d45140aa9d6fb8)
+set(USD_VERSION 19.11)
+set(USD_URI https://github.com/PixarAnimationStudios/USD/archive/v${USD_VERSION}.tar.gz)
+set(USD_HASH 79ff176167b3fe85f4953abd6cc5e0cc)
+
set(OIDN_VERSION 1.0.0)
set(OIDN_URI https://github.com/OpenImageDenoise/oidn/releases/download/v${OIDN_VERSION}/oidn-${OIDN_VERSION}.src.zip)
set(OIDN_HASH 19fe67b0164e8f020ac8a4f520defe60)
diff --git a/build_files/build_environment/patches/usd.diff b/build_files/build_environment/patches/usd.diff
new file mode 100644
index 00000000000..8afd9700e59
--- /dev/null
+++ b/build_files/build_environment/patches/usd.diff
@@ -0,0 +1,111 @@
+diff -x .git -ur usd.orig/cmake/defaults/Options.cmake external_usd/cmake/defaults/Options.cmake
+--- usd.orig/cmake/defaults/Options.cmake 2019-10-24 22:39:53.000000000 +0200
++++ external_usd/cmake/defaults/Options.cmake 2019-11-28 13:00:33.197957712 +0100
+@@ -25,6 +25,7 @@
+ option(PXR_VALIDATE_GENERATED_CODE "Validate script generated code" OFF)
+ option(PXR_HEADLESS_TEST_MODE "Disallow GUI based tests, useful for running under headless CI systems." OFF)
+ option(PXR_BUILD_TESTS "Build tests" ON)
++option(PXR_BUILD_USD_TOOLS "Build commandline tools" ON)
+ option(PXR_BUILD_IMAGING "Build imaging components" ON)
+ option(PXR_BUILD_EMBREE_PLUGIN "Build embree imaging plugin" OFF)
+ option(PXR_BUILD_OPENIMAGEIO_PLUGIN "Build OpenImageIO plugin" OFF)
+diff -x .git -ur usd.orig/cmake/defaults/Packages.cmake external_usd/cmake/defaults/Packages.cmake
+--- usd.orig/cmake/defaults/Packages.cmake 2019-10-24 22:39:53.000000000 +0200
++++ external_usd/cmake/defaults/Packages.cmake 2019-11-28 13:00:33.185957483 +0100
+@@ -64,7 +64,7 @@
+ endif()
+
+ # --TBB
+-find_package(TBB REQUIRED COMPONENTS tbb)
++find_package(TBB)
+ add_definitions(${TBB_DEFINITIONS})
+
+ # --math
+diff -x .git -ur usd.orig/pxr/base/lib/plug/initConfig.cpp external_usd/pxr/base/lib/plug/initConfig.cpp
+--- usd.orig/pxr/base/lib/plug/initConfig.cpp 2019-10-24 22:39:53.000000000 +0200
++++ external_usd/pxr/base/lib/plug/initConfig.cpp 2019-12-11 11:00:37.643323127 +0100
+@@ -69,8 +69,38 @@
+
+ ARCH_CONSTRUCTOR(Plug_InitConfig, 2, void)
+ {
++ /* The contents of this constructor have been moved to usd_initialise_plugin_path(...) */
++}
++
++}; // end of anonymous namespace
++
++/**
++ * The contents of this function used to be in the static constructor Plug_InitConfig.
++ * This static constructor made it impossible for Blender to pass a path to the USD
++ * library at runtime, as the constructor would run before Blender's main() function.
++ *
++ * This function is wrapped in a C function of the same name (defined below),
++ * so that it can be called from Blender's main() function.
++ *
++ * The datafiles_usd_path path is used to point to the USD plugin path when Blender
++ * has been installed. The fallback_usd_path path should point to the build-time
++ * location of the USD plugin files so that Blender can be run on a development machine
++ * without requiring an installation step.
++ */
++void
++usd_initialise_plugin_path(const char *datafiles_usd_path)
++{
+ std::vector<std::string> result;
+
++ // Add Blender-specific pat
@@ Diff output truncated at 10240 characters. @@
More information about the Bf-blender-cvs
mailing list