Smaller changes in the build files to reflect the new Mantaflow macro.
Reviewed By: sergey
Maniphest Tasks: T59995
Differential Revision: https://developer.blender.org/D3853
This commit introduces the first version of an exporter to Pixar's
Universal Scene Description (USD) format.
Reviewed By: sergey, LazyDodo
Differential Revision: https://developer.blender.org/D6287
- The USD libraries are built by `make deps`, but not yet built by
install_deps.sh.
- Only experimental support for instancing; by default all duplicated
objects are made real in the USD file. This is fine for exporting a
linked-in posed character, not so much for thousands of pebbles etc.
- The way materials and UV coordinates and Normals are exported is going
to change soon.
- This patch contains LazyDodo's fixes for building on Windows in D5359.
== Meshes ==
USD seems to support neither per-material nor per-face-group
double-sidedness, so we just use the flag from the first non-empty
material slot. If there is no material we default to double-sidedness.
Each UV map is stored on the mesh in a separate primvar. Materials can
refer to these UV maps, but this is not yet exported by Blender. The
primvar name is the same as the UV Map name. This is to allow the
standard name "st" for texture coordinates by naming the UV Map as such,
without having to guess which UV Map is the "standard" one.
Face-varying mesh normals are written to USD. When the mesh has custom
loop normals those are written. Otherwise the poly flag `ME_SMOOTH` is
inspected to determine the normals.
The UV maps and mesh normals take up a significant amount of space, so
exporting them is optional. They're still enabled by default, though.
For comparison: a shot of Spring (03_035_A) is 1.2 GiB when exported
with UVs and normals, and 262 MiB without. We probably have room for
optimisation of written UVs and normals.
The mesh subdivision scheme isn't using the default value 'Catmull
Clark', but uses 'None', indicating we're exporting a polygonal mesh.
This is necessary for USD to understand our normals; otherwise the mesh
is always rendered smooth. In the future we may want to expose this
choice of subdivision scheme to the user, or auto-detect it when we
actually support exporting pre-subdivision meshes.
A possible optimisation could be to inspect whether all polygons are
smooth or flat, and mark the USD mesh as such. This can be added when
needed.
== Animation ==
Mesh and transform animation are now written when passing
`animation=True` to the export operator. There is no inspection of
whether an object is actually animated or not; USD can handle
deduplication of static values for us.
The administration of which timecode to use for the export is left to
the file-format-specific concrete subclasses of
`AbstractHierarchyIterator`; the abstract iterator itself doesn't know
anything about the passage of time. This will allow subclasses for the
frame-based USD format and time-based Alembic format.
== Support for simple preview materials ==
Very simple versions of the materials are now exported, using only the
viewport diffuse RGB, metallic, and roughness.
When there are multiple materials, the mesh faces are stored as geometry
subset and each material is assigned to the appropriate subset. If there
is only one material this is skipped.
The first material if any) is always applied to the mesh itself
(regardless of the existence of geometry subsets), because the Hydra
viewport doesn't support materials on subsets. See
https://github.com/PixarAnimationStudios/USD/issues/542 for more info.
Note that the geometry subsets are not yet time-sampled, so it may break
when an animated mesh changes topology.
Materials are exported as a flat list under a top-level '/_materials'
namespace. This inhibits instancing of the objects using those
materials, so this is subject to change.
== Hair ==
Only the parent strands are exported, and only with a constant colour.
No UV coordinates, no information about the normals.
== Camera ==
Only perspective cameras are supported for now.
== Particles ==
Particles are only written when they are alive, which means that they
are always visible (there is currently no code that deals with marking
them as invisible outside their lifespan).
Particle-system-instanced objects are exported by suffixing the object
name with the particle's persistent ID, giving each particle XForm a
unique name.
== Instancing/referencing ==
This exporter has experimental support for instancing/referencing.
Dupli-object meshes are now written to USD as references to the original
mesh. This is still very limited in correctness, as there are issues
referencing to materials from a referenced mesh.
I am still committing this, as it gives us a place to start when
continuing the quest for proper instancing in USD.
== Lights ==
USD does not directly support spot lights, so those aren't exported yet.
It's possible to add this in the future via the UsdLuxShapingAPI. The
units used for the light intensity are also still a bit of a mystery.
== Fluid vertex velocities ==
Currently only fluid simulations (not meshes in general) have explicit
vertex velocities. This is the most important case for exporting
velocities, though, as the baked mesh changes topology all the time, and
thus computing the velocities at import time in a post-processing step
is hard.
== The Building Process ==
- USD is built as monolithic library, instead of 25 smaller libraries.
We were linking all of them as 'whole archive' anyway, so this doesn't
affect the final file size. It does, however, make life easier with
respect to linking order, and handling upstream changes.
- The JSON files required by USD are installed into datafiles/usd; they
are required on every platform. Set the `PXR_PATH_DEBUG` to any value
to have the USD library print the paths it uses to find those files.
- USD is patched so that it finds the aforementioned JSON files in a path
that we pass to it from Blender.
- USD is patched to have a `PXR_BUILD_USD_TOOLS` CMake option to disable
building the tools in its `bin` directory. This is sent as a pull
request at https://github.com/PixarAnimationStudios/USD/pull/1048
Tests were missing a manifest, and were importing the
wrong version of Microsoft.Windows.Common-Controls
causing blenloader_test, bmesh_core_test and alembic_test
to fail due a loader error.
In older versions the ms crt was only a few dlls, in recent versions
this jumped to over 40 leading to quite a bit of clutter in our
bin folder.
This change moves the CRT into its own folder.
For developers that generally already have the runtime globaly
available on their machine, there is a new cmake option
(WITH_WINDOWS_BUNDLE_CRT, default ON) that you can use to toggle
installing the runtime to the blender bin folder, and save some
time during the initial build, this option is off by default for
only the developer profile.
Reviewed By: brecht
Differential Revision: https://developer.blender.org/D6132
Previously some important features like OpenSubdiv were disabled by default,
which caused confusion.
The purpose of disabling some of these features was to avoid potentiall build
errors on Linux. But with precompiled libraries, install_deps.sh and better
library availability checking this is hopefully not much of a problem anymore.
This makes "make full" obsolete, but it's kept to not break docs or shell
scripts that people may have, and the .cmake config file remains useful to
modify an existing build folder.
This also changes some option to only be available on platforms where they
are actually supported (WITH_JACK, WITH_TBB_MALLOC_PROXY and X11 options).
Fixes T69742
Differential Revision: https://developer.blender.org/D6306
This new test class minimally sets up Blender so that it can load blend
files and construct a depsgraph without crashing.
Note that it hasn't been tested on very complex blend files, so it may
still crash when the loaded blend file references/requires uninitialised
data structures.
The test will certainly crash with Blend files created with Blender
older than 2.80, as the versioning code requires space types to be
registered. This is normally done by initialising the window manager,
which is not done in this test. The WM requires Python to run, which in
turn requires that Blender finds the release directory in the same
directory that contains the running executable, which is not the case
for GTest tests (they are written to `bin/tests/executablename`.
Reviewed By: sergey, mont29
Differential Revision: https://developer.blender.org/D6246
xz compresses 25% better than bz2, reducing download times and server load.
The numbers:
blender-2.80-linux-glibc217-x86_64.tar.bz2 (release): 134 886 174 bytes
with xz: 96 181 604 bytes (-28.7%)
with xz -9: 93 871 548 bytes (-30.4%)
blender-2.81-7c1fbe24ca33-linux-glibc217-x86_64.tar.bz2 (beta): 173 600 363 bytes
with xz: 133 100 664 bytes (-23.3%)
with xz -9: 129 534 124 bytes (-25.4%)
xz also decompresses more than twice as fast as bz2, however compression needs
four times as long (on my 7-year-old laptop 3-4 minutes instead of <1).
Also xz has become more common than bz2, e.g. Debian/Ubuntu deb packages have
been xz-compressed for years, so the dpkg package manager as well as systemd
and grub all depend on liblzma being present, whereas bz2 is becoming more and
more optional.
Current Linux archives also include the UID/GID of whatever user account
happens to be used for building by the blender.org infrastructure. If someone
then installs these archives as root e.g. to /usr/local/... and doesn't pay
full attention the files remain owned by a regular user, which is a serious
security issue. This patch fixes that by setting the UID/GID to 0.
Differential Revision: https://developer.blender.org/D6138
This was added years ago to prepare for code-signing the executable
but was never used, buildbots use a different mechanism now to sign
so no need to keep this around.
The heap on windows is single threaded causing it to lag behind linux in performance in allocation heavy multithreaded scenarios, BVH building is a prime example.
See https://developer.blender.org/D6218 for benchmark results
for testing with the allocator enabled/disabled you can set the environment variable TBB_MALLOC_DISABLE_REPLACEMENT=1 to disable the TBB allocator.
Reviewed By: @sergey
Differential Revision: https://developer.blender.org/D6218
The goal is to make it able to use pre-compiled CentOS libraries on a
more modern system. Main issue was that it's possible that the compiler
on a newer version is defaulting to different C++11 ABI.
This change makes it so that if there is NO native libraries in the
lib folder and there IS pre-compiled CentOS folder, it will be used and
compiler will be forced to old ABI.
Differential Revision: https://developer.blender.org/D6031
Opus support was enabled in 2ddfd51810. This commit adds the Opus
library and configures FFmpeg to be compiled with Opus support.
NOTE: It may be required to run `cmake -U '*FFMPEG_LIBRARIES*' .` in
your Blender build directory in order to refresh the `FFMPEG_LIBRARIES`
setting and add libopus.
Problem was twofold
1) `GENERATOR_IS_MULTI_CONFIG` is a property not a variable so
the test for it would always be false, unless you set a custom
CMAKE_INSTALL_PREFIX (like the buildbot does) the unit tests
would have a wrong working directory and complain about missing
dlls or blender executable
2) Tests added outside of `/test` (like libmv) would have no working
folder set since the variable would not be visible for them.
consulted @sergey who voiced the opinion that duplicating the code
to the test macro was slightly less evil than moving it to the main
CMakeLists.txt
Use cmake TIMESTAMP for BUILD_DATE+TIME
this simplifies code a lot
and even makes it more portable to other platforms
TIMESTAMP is available since cmake-2.8.11 ; blender already requires
cmake>=3.5 so that is fine.
Note that with CMake>=3.8, if defined, the SOURCE_DATE_EPOCH envvar
will be used by CMake here.
Reviewers: mont29, campbellbarton
Reviewed By: mont29, campbellbarton
Differential Revision: https://developer.blender.org/D5760
This change switches the debug symbol format from /Zi to /ZI for
debug builds of blender, allowing Edit and Continue to work.
This allows limited [1] code changes in the debugger without
having to stop the process and recompile a new binary leading
to improved developer productivity.
All MSVC versions we support support this flag, Clang on
windows does not mind the /ZI flag, but doesn't currently
emit the required information to have this feature work.
[1] https://docs.microsoft.com/en-us/visualstudio/debugger/supported-code-changes-cpp
The installer always upgraded the last version installed and did not allow for two versions to be installed side by side.
The reworked installer will allow side by side installs
install order:
```
2.81 -> 2.81a -> 2.82 : Allowed , will result in both 2.82 and 2.81a being installed
2.82 -> 2.81 -> 2.81a : Allowed , will result in both 2.82 and 2.81a being installed
2.82 -> 2.81a : Allowed , will result in both 2.82 and 2.81a being installed
2.82 -> 2.81a -> 2.81 : Not Allowed, 2.81 will only install if you manually remove 2.81a first.
```
Do note though that this will not apply to any previously issued installers and even for 2.80a this is not something we can fix.
This patch is for landing in 2.81 *only* and should be excluded from any possible 2.80a release.
Second change is a change to the compression level, building the MSI takes 30 minutes, which is crazy, perhaps worth it if the compression actually pays of.
```
MSI - none 1:35 247.0 MB (260,025,634 bytes)
MSI - mszip 2:02 89.6 MB ( 94,022,946 bytes)
MSI - low 2:35 81.6 MB ( 85,646,626 bytes)
MSI - medium 4:11 77.3 MB ( 81,136,930 bytes)
MSI - high 28:01 74.7 MB ( 78,384,418 bytes)
zip 1:32 93.2 MB ( 97,732,293 bytes)
7Z 2:22 65.0 MB ( 68,171,614 bytes)
```
It didn't, so I lowered it to medium, seemed reasonable.
Differential Revision: https://developer.blender.org/D5494
Reviewers: brecht, jesterking
bf_intern_openvdb makes a significant number of template instantiations
causing it go over the maximum number of sections (int16) in a coff file
when doing a debug build.
This change switches the compiler to use the extended coff format which
has this field extended (int32) all linkers post msvc2005 can process
this format so there's no reason not to turn this on globally.
Clang on windows does not need this change since clang switches implicitly
to the extended format when required. [1]
[1] https://reviews.llvm.org/rL217812
This change removes 32 bit support from the helper make.bat scripts
as we are dropping official 32 bit support, you can still build for
32 bit by configuring your build yourself using cmake and pointing
the LIBDIR cmake variable to your own 32 bit library folder.
This has faster builds, error checks and tests. The number of cmake options
for this type of thing has grown over the years and it's convenient to be
able to point new developers to a single target.
Previously the combination of all these options did not work correctly, now
all tests should pass.
The easiest way to use this is with the make wrapper, for example:
make full developer debug
Or set it manually with CMake:
cmake -C ../blender/build_files/cmake/config/blender_developer.cmake .
Differential Revision: https://developer.blender.org/D5149
