This patch adds a new Cycles device with similar functionality to the
existing GPU devices. Kernel compilation and runtime interaction happen
via oneAPI DPC++ compiler and SYCL API.
This implementation is primarly focusing on Intel® Arc™ GPUs and other
future Intel GPUs. The first supported drivers are 101.1660 on Windows
and 22.10.22597 on Linux.
The necessary tools for compilation are:
- A SYCL compiler such as oneAPI DPC++ compiler or
https://github.com/intel/llvm
- Intel® oneAPI Level Zero which is used for low level device queries:
https://github.com/oneapi-src/level-zero
- To optionally generate prebuilt graphics binaries: Intel® Graphics
Compiler All are included in Linux precompiled libraries on svn:
https://svn.blender.org/svnroot/bf-blender/trunk/lib The same goes for
Windows precompiled binaries but for the graphics compiler, available
as "Intel® Graphics Offline Compiler for OpenCL™ Code" from
https://www.intel.com/content/www/us/en/developer/articles/tool/oneapi-standalone-components.html,
for which path can be set as OCLOC_INSTALL_DIR.
Being based on the open SYCL standard, this implementation could also be
extended to run on other compatible non-Intel hardware in the future.
Reviewed By: sergey, brecht
Differential Revision: https://developer.blender.org/D15254
Co-authored-by: Nikita Sirgienko <nikita.sirgienko@intel.com>
Co-authored-by: Stefan Werner <stefan.werner@intel.com>
When building blender as a python module, such as for inclusion in a
wheel, it is not permitted to link against python libraries.
This diff does so by simply unsetting the library when building blender
as a python module, instead of the more heavyweight solution of
switching to the cmake FindPython module.
Reviewed By: LazyDodo, campbellbarton
Ref D15012
Building against the existing 3.1 libraries should continue to work, until
the precompiled libraries are committed for all platforms.
* Enable WebP by default.
* Update Windows for new library file names.
* Automatically clear outdated CMake cache variables when upgrading to new
libraries.
* Fix static library linking order issues on Linux for OpenEXR and OpenVDB.
Implemented by Ray Molenkamp, Sybren Stüvel and Brecht Van Lommel.
Ref T95206
Currently only supports single image frames (no animation possible).
If quality slider is set to 100 then lossless compression will be used,
otherwise lossy compression is used.
Gives about 35% reduction of filesize save when re-saving splash screens with lossless
compression.
Also saves much faster, up to 15x faster than PNG with a better compression ratio as a plus.
Note, this is currently left disabled until we have WebP libs (see T95206)
For testing precompiled libs can be downloaded from Google:
https://storage.googleapis.com/downloads.webmproject.org/releases/webp/index.html
Differential Revision: https://developer.blender.org/D1598
rBc1909770e7f192574ea62449dd14b4254637e604 introduced "PXR_LIB_PREFIX" for building the
dependencies, so only makes sense to use the same name in the Hydra render delegate CMake too
This patch adds a Hydra render delegate to Cycles, allowing Cycles to be used for rendering
in applications that provide a Hydra viewport. The implementation was written from scratch
against Cycles X, for integration into the Blender repository to make it possible to continue
developing it in step with the rest of Cycles. For this purpose it follows the style of the rest of
the Cycles code and can be built with a CMake option
(`WITH_CYCLES_HYDRA_RENDER_DELEGATE=1`) similar to the existing standalone version
of Cycles.
Since Hydra render delegates need to be built against the exact USD version and other
dependencies as the target application is using, this is intended to be built separate from
Blender (`WITH_BLENDER=0` CMake option) and with support for library versions different
from what Blender is using. As such the CMake build scripts for Windows had to be modified
slightly, so that the Cycles Hydra render delegate can e.g. be built with MSVC 2017 again
even though Blender requires MSVC 2019 now, and it's possible to specify custom paths to
the USD SDK etc. The codebase supports building against the latest USD release 22.03 and all
the way back to USD 20.08 (with some limitations).
Reviewed By: brecht, LazyDodo
Differential Revision: https://developer.blender.org/D14398
I noticed that there were a few variables that should not be visible per default.
It seems to me to simply be an oversight, so I went ahead and cleaned them up.
Reviewed By: Sybren, Ray molenkamp
Differential Revision: http://developer.blender.org/D14132
FindOpenImageIO was updated to link to separate OpenImageIO_Util for new
versions, where it is required. For older versions, we can not link to it
because there will be duplicated symbols.
Ref D14128
FindOpenEXR was updated to find new lib names and separate Imath. It's all
added to the list of OpenEXR include dirs and libs.
This keeps it compatible with both version 2 and 3 for now, and doesn't
require changes outside the find module.
Ref D14128
Brotli seems to add a custom postfix to its static libraries by default,
but in Debian at least libraries are just named the same for both shared
and static versions, as usual.
So add standard name after static-specific ones.
Follow-up to rB4c617c06e9cb and rBa000de7c2a4d.
And change install_deps.sh to build shared (instead of static) FFMPEG
libraries, for consistency with other library dependencies and to simplify
the logic. This may require users of install_deps.sh to rebuild FFMPEG.
This is the last step that lets us get rid of LIBPATH variables and
link_directories() entirely, as recommended by the CMake docs.
Some fixes were needed in the find FFMPEG module to make it actually work,
this code was unused up to now.
Followup to D8855.
Differential Revision: https://developer.blender.org/D9177
And change install_deps.sh to build shared (instead of static) FFMPEG
libraries, for consistency with other library dependencies and to simplify
the logic. This may require users of install_deps.sh to rebuild FFMPEG.
This is the last step that lets us get rid of LIBPATH variables and
link_directories() entirely, as recommended by the CMake docs.
Some fixes were needed in the find FFMPEG module to make it actually work,
this code was unused up to now.
Followup to D8855.
Differential Revision: https://developer.blender.org/D9177
This includes much improved GPU rendering performance, viewport interactivity,
new shadow catcher, revamped sampling settings, subsurface scattering anisotropy,
new GPU volume sampling, improved PMJ sampling pattern, and more.
Some features have also been removed or changed, breaking backwards compatibility.
Including the removal of the OpenCL backend, for which alternatives are under
development.
Release notes and code docs:
https://wiki.blender.org/wiki/Reference/Release_Notes/3.0/Cycleshttps://wiki.blender.org/wiki/Source/Render/Cycles
Credits:
* Sergey Sharybin
* Brecht Van Lommel
* Patrick Mours (OptiX backend)
* Christophe Hery (subsurface scattering anisotropy)
* William Leeson (PMJ sampling pattern)
* Alaska (various fixes and tweaks)
* Thomas Dinges (various fixes)
For the full commit history, see the cycles-x branch. This squashes together
all the changes since intermediate changes would often fail building or tests.
Ref T87839, T87837, T87836
Fixes T90734, T89353, T80267, T80267, T77185, T69800
Compressing blendfiles can help save a lot of disk space, but the slowdown
while loading and saving is a major annoyance.
Currently Blender uses Zlib (aka gzip aka Deflate) for compression, but there
are now several more modern algorithms that outperform it in every way.
In this patch, I decided for Zstandard aka Zstd for several reasons:
- It is widely supported, both in other programs and libraries as well as in
general-purpose compression utilities on Unix
- It is extremely flexible - spanning several orders of magnitude of
compression speeds depending on the level setting.
- It is pretty much on the Pareto frontier for all of its configurations
(meaning that no other algorithm is both faster and more efficient).
One downside of course is that older versions of Blender will not be able to
read these files, but one can always just re-save them without compression or
decompress the file manually with an external tool.
The implementation here saves additional metadata into the compressed file in
order to allow for efficient seeking when loading. This is standard-compliant
and will be ignored by other tools that support Zstd.
If the metadata is not present (e.g. because you manually compressed a .blend
file with another tool), Blender will fall back to sequential reading.
Saving is multithreaded to improve performance. Loading is currently not
multithreaded since it's not easy to predict the access patterns of the
loading code when seeking is supported.
In the future, we might want to look into making this more predictable or
disabling seeking for the main .blend file, which would then allow for
multiple background threads that decompress data ahead of time.
The compression level was chosen to get sizes comparable to previous versions
at much higher speeds. In the future, this could be exposed as an option.
Reviewed By: campbellbarton, brecht, mont29
Differential Revision: https://developer.blender.org/D5799
This patch fixes many minor spelling mistakes, all in comments or
console output. Mostly contractions like can't, won't, don't, its/it's,
etc.
Differential Revision: https://developer.blender.org/D11663
Reviewed by Harley Acheson
Tell `FindOSL.cmake` where to find the shader header files. These have
moved from `${LIBDIR}/osl/shaders` to `${LIBDIR}/osl/share/OSL/shaders`
when OSL was upgraded (T85365).
This bumps OSL to 1.11.10.0. OSL Has a new build time
dependency: Clang, and more importantly it expects
clang and llvm to share a library folder, which it
previously for us did not.
This patch changes:
-OSL Update to 1.11.10.0
-refactor the llvm/clang/clang-tools-extra builds into the llvm
build using the llvm-project tarball for building that has all
of the subprojects in it.
-update ispc/openmp builds since clang no longer its own dependency
and they have to depend on the llvm build now.
-Update the windows builder to use the 64 bit host tools since it
ran out of ram linking clang
-Since OSL now needs clang to link successfully a findclang.cmake
has been provided for linux/OSX
Differential Revision: https://developer.blender.org/D10212
Reviewed By: brecht, sebbas, sybren
This required using a fork of Embree, newer LLVM version, unreleased ISPC
version and sse2neon directly from Git. Hopefully over time all the required
changes end up in official releases. For now we deviate from other platforms.
Based on contributions by Apple and Stefan Werner.
Ref D9527, D8237, T78710
Ref T84819
Build System
============
This is an API breaking new version, and the updated code only builds with
OpenColorIO 2.0 and later. Adding backwards compatibility was too complicated.
* Tinyxml was replaced with Expat, adding a new dependency.
* Yaml-cpp is now built as a dependency on Unix, as was already done on Windows.
* Removed currently unused LCMS code.
* Pystring remains built as part of OCIO itself, since it has no good build system.
* Linux and macOS check for the OpenColorIO verison, and disable it if too old.
Ref D10270
Processors and Transforms
=========================
CPU processors now need to be created to do CPU processing. These are cached
internally, but the cache lookup is not fast enough to execute per pixel or
texture sample, so for performance these are now also exposed in the C API.
The C API for transforms will no longer be needed afer all changes, so remove
it to simplify the API and fallback implementation.
Ref D10271
Display Transforms
==================
Needs a bit more manual work constructing the transform. LegacyViewingPipeline
could also have been used, but isn't really any simpler and since it's legacy
we better not rely on it.
We moved more logic into the opencolorio module, to simplify the API. There is
no need to wrap a dozen functions just to be able to do this in C rather than C++.
It's also tightly coupled to the GPU shader logic, and so should be in the same
module.
Ref D10271
GPU Display Shader
==================
To avoid baking exposure and gamma into the GLSL shader and requiring slow
recompiles when tweaking, we manually apply them in the shader. This leads
to some logic duplicaton between the CPU and GPU display processor, but it
seems unavoidable.
Caching was also changed. Previously this was done both on the imbuf and
opencolorio module levels. Now it's all done in the opencolorio module by
simply matching color space names. We no longer use cacheIDs from OpenColorIO
since computing them is expensive, and they are unlikely to match now that
more is baked into the shader code.
Shaders can now use multiple 2D textures, 3D textures and uniforms, rather
than a single 3D texture. So allocating and binding those adds some code.
Color space conversions for blending with overlays is now hardcoded in the
shader. This was using harcoded numbers anyway, if this every becomes a
general OpenColorIO transform it can be changed, but for now there is no
point to add code complexity.
Ref D10273
CIE XYZ
=======
We need standard CIE XYZ values for rendering effects like blackbody emission.
The relation to the scene linear role is based on OpenColorIO configuration.
In OpenColorIO 2.0 configs roles can no longer have the same name as color
spaces, which means our XYZ role and colorspace in the configuration give an
error.
Instead use the new standard aces_interchange role, which relates scene linear
to a known scene referred color space. Compatibility with the old XYZ role is
preserved, if the configuration file has no conflicting names.
Also includes a non-functional change to the configuraton file to use an
XYZ-to-ACES matrix instead of REC709-to-ACES, makes debugging a little easier
since the matrix is the same one we have in the code now and that is also
found easily in the ACES specs.
Ref D10274
Paths and names of system packages-installed libhpdf on Debian (and
probably its derived distributions?) are slightly different than what
was given to CMake finding script.
Don't overwrite environment variables that may
contain options like suppression files, symboliser etc.
It's similar to rBa181b156399a13fa429159112e30c8005d5e8a59
and rBA589d13408a60cbec34a8bc3cc798c586043743ae .
For Blender Add-ons repo, see the equivalent in D9816.
Reviewed By: Blendify
Differential Revision: https://developer.blender.org/D9815
Group all tests of a test suite into a single test command invocation.
This reduces the number of invocations by `ctest` by an order of
magnitude.
Since rB56aa5b0d8c6b663, `bin/tests/blender_test` was run for every
individual test. Having over a 1000 tests made testing slower than
necessary. Individual tests can still be run if desired by invocation of
`bin/tests/blender_test --gtest_filter=suitename.testname`.
NOTE: For this commit to have an immediate effect, it may be necessary
to remove the `tests` and `Testing` directories and some CMake files
from your build directory and rebuild. Run `ctest -N` to see the list of
tests; there should be less than 200.
Reviewed By: sergey, LazyDodo, sebbas
Maniphest Tasks: T83222
Differential Revision: https://developer.blender.org/D9649
This commit expands the Windows-specific code in rBdca9aa0053f7 and Linux-specific code in rB33b7d53df08a.
It also fixes a capitalization issue in FindPugiXML.cmake
For blender we disable VCPKG to prevent it from picking
up the wrong libraries from VCPKG rather than our lib folder
some of the cycles tests needed this to link correctly.
reported by @alef on chat
Stop with an error when the Clang-Tidy executable cannot be found.
Without this check, CMake will happily report "Found Clang-Tidy" but with
the fallback version (0, 0, 0), when `CLANG_TIDY_EXECUTABLE` points to a
non-existing executable.
NanoVDB is a platform-independent sparse volume data structure that makes it possible to
use OpenVDB volumes on the GPU. This patch uses it for volume rendering in Cycles,
replacing the previous usage of dense 3D textures.
Since it has a big impact on memory usage and performance and changes the OpenVDB
branch used for the rest of Blender as well, this is not enabled by default yet, which will
happen only after 2.82 was branched off. To enable it, build both dependencies and Blender
itself with the "WITH_NANOVDB" CMake option.
Reviewed By: brecht
Differential Revision: https://developer.blender.org/D8794
