Commit Graph

121 Commits

Author SHA1 Message Date
0803119725 Cycles: merge of cycles-x branch, a major update to the renderer
This includes much improved GPU rendering performance, viewport interactivity,
new shadow catcher, revamped sampling settings, subsurface scattering anisotropy,
new GPU volume sampling, improved PMJ sampling pattern, and more.

Some features have also been removed or changed, breaking backwards compatibility.
Including the removal of the OpenCL backend, for which alternatives are under
development.

Release notes and code docs:
https://wiki.blender.org/wiki/Reference/Release_Notes/3.0/Cycles
https://wiki.blender.org/wiki/Source/Render/Cycles

Credits:
* Sergey Sharybin
* Brecht Van Lommel
* Patrick Mours (OptiX backend)
* Christophe Hery (subsurface scattering anisotropy)
* William Leeson (PMJ sampling pattern)
* Alaska (various fixes and tweaks)
* Thomas Dinges (various fixes)

For the full commit history, see the cycles-x branch. This squashes together
all the changes since intermediate changes would often fail building or tests.

Ref T87839, T87837, T87836
Fixes T90734, T89353, T80267, T80267, T77185, T69800
2021-09-21 14:55:54 +02:00
e7fc15e2ef Fix T70615: Cycles ignores BSDF inputs when nodes are optimized
When compiling BSDF nodes, we only assing stack space to the normal and
tangent inputs if they are linked. However, it could be that the
ConstantFolder removed the link, so checking if there is a link fails
to take this into account.

To fix this, added a flag to ShaderInput to keep track of whether a
constant was folded into the input, and use it as well to verify that
the socket is linked when assigning stack space.

Reviewed By: brecht

Maniphest Tasks: T70615

Differential Revision: https://developer.blender.org/D11731
2021-06-29 16:24:21 +02:00
3bc44233c0 Cycles: use reference count to detect used shaders
Shaders are only compiled if they are used by some other Node (Geometry, Light, etc.).
This usage detection is done before updating the Scene, however it fails at detecting
Shaders used by Procedurals not known to Cycles (e.g. ones defined by third party
applications), as Procedurals are only updated after the shaders are compiled.

To remedy this, we now use the Node reference counting mechanism to detect whether a
Shader is used and therefore should be compiled.

This removes `ShaderManager::update_shaders_used` as it is not needed anymore, however,
since it would also update the Shader ids, this is now performed in
`ShaderManager::device_update`, and a new virtual `device_update_specific` method was
added to handle device updates for SVM and OSL.

Reviewed By: brecht

Differential Revision: https://developer.blender.org/D10965
2021-05-03 01:21:12 +02:00
bbe6d44928 Cycles: optimize device updates
This optimizes device updates (during user edits or frame changes in
the viewport) by avoiding unnecessary computations. To achieve this,
we use a combination of the sockets' update flags as well as some new
flags passed to the various managers when tagging for an update to tell
exactly what the tagging is for (e.g. shader was modified, object was
removed, etc.).

Besides avoiding recomputations, we also avoid resending to the devices
unmodified data arrays, thus reducing bandwidth usage. For OptiX and
Embree, BVH packing was also multithreaded.

The performance improvements may vary depending on the used device (CPU
or GPU), and the content of the scene. Simple scenes (e.g. with no adaptive
subdivision or volumes) rendered using OptiX will benefit from this work
the most.

On average, for a variety of animated scenes, this gives a 3x speedup.

Reviewed By: #cycles, brecht

Maniphest Tasks: T79174

Differential Revision: https://developer.blender.org/D9555
2021-01-22 16:08:25 +01:00
dad5aded0c Fix T84006: Cycles AOV not written with some mix shader node set ups 2020-12-22 14:25:50 +01:00
31a620b942 Cycles API: encapsulate Node socket members
This encapsulates Node socket members behind a set of specific methods;
as such it is no longer possible to directly access Node class members
from exporters and parts of Cycles.

The methods are defined via the NODE_SOCKET_API macros in `graph/
node.h`, and are for getting or setting a specific socket's value, as
well as querying or modifying the state of its update flag.

The setters will check whether the value has changed and tag the socket
as modified appropriately. This will let us know how a Node has changed
and what to update, which is the first concrete step toward a more
granular scene update system.

Since the setters will tag the Node sockets as modified when passed
different data, this patch also removes the various modified methods
on Nodes in favor of Node::is_modified which checks the sockets'
update flags status.

Reviewed By: brecht

Maniphest Tasks: T79174

Differential Revision: https://developer.blender.org/D8544
2020-11-04 13:03:33 +01:00
30f626fe4c Revert "Cycles API: encapsulate Node socket members"
This reverts commit 527f8b32b3. It is causing
motion blur test failures and crashes in some renders, reverting until this is
fixed.
2020-10-27 11:40:42 +01:00
527f8b32b3 Cycles API: encapsulate Node socket members
This encapsulates Node socket members behind a set of specific methods;
as such it is no longer possible to directly access Node class members
from exporters and parts of Cycles.

The methods are defined via the NODE_SOCKET_API macros in `graph/
node.h`, and are for getting or setting a specific socket's value, as
well as querying or modifying the state of its update flag.

The setters will check whether the value has changed and tag the socket
as modified appropriately. This will let us know how a Node has changed
and what to update, which is the first concrete step toward a more
granular scene update system.

Since the setters will tag the Node sockets as modified when passed
different data, this patch also removes the various `modified` methods
on Nodes in favor of `Node::is_modified` which checks the sockets'
update flags status.

Reviewed By: brecht

Maniphest Tasks: T79174

Differential Revision: https://developer.blender.org/D8544
2020-10-26 23:11:14 +01:00
edd1164575 Cycles: add time statistics to scene update
Gathers information for time spent in the various managers or object (Film, Camera, etc.) being updated in Scene::device_update.

The stats include the total time spent in the device_update methods as well as time spent in subroutines (e.g. bvh build, displacement, etc.).

This does not qualify as a full blown profiler, but is useful to identify potential bottleneck areas.

The stats can be enabled and printed by passing `--cycles-print-stats` on the command line to Cycles, or `-- --cycles-print-stats` to Blender.

Reviewed By: brecht

Maniphest Tasks: T79174

Differential Revision: https://developer.blender.org/D8596
2020-10-01 23:21:11 +02:00
d8c2092b15 Cycles: make TBB a required library dependency, and use in a few places
Now that the rest of Blender also relies on TBB, no point in maintaining custom
code for paraller_for and thread local storage.
2020-06-22 13:06:47 +02:00
16d8a683be Fix T73984: unnecessary Cycles viewport updates with object texture coordinates
Remove old code that added extra updates for shaders that have a dependency on
objects. The dependency graph can now tell Cycles when a material is affected by
an object transform.
2020-05-14 17:39:37 +02:00
2d1cce8331 Cleanup: make format after SortedIncludes change 2020-03-19 09:33:58 +01:00
Brecht Van Lommel
1162ba206d Cycles: change volume step size controls, auto adjust based on voxel size
By default it will now set the step size to the voxel size for smoke and
volume objects, and 1/10th the bounding box for procedural volume shaders.

New settings are:
* Scene render/preview step rate: to globally adjust detail and performance
* Material step rate: multiplied with auto detected per-object step size
* World step size: distance to steo for world shader

Differential Revision: https://developer.blender.org/D1777
2020-03-18 11:23:05 +01:00
ec3eeee46b Cycles: add internal default volume shader, to be used for new volume object
This is mostly straightforward, but required some refactoring to ensure that
the default volume material does not always turn on the volume feature for GPU
rendering.
2020-03-11 20:35:38 +01:00
1613c994b0 Fix Cycles not correctly using Background.shader if specified
Based on patch by Alex Fuller.

Differential Revision: https://developer.blender.org/D6627
2020-01-20 14:03:14 +01:00
c30d6571bb Add support for tiled images and the UDIM naming scheme
This patch contains the work that I did during my week at the Code Quest - adding support for tiled images to Blender.

With this patch, images now contain a list of tiles. By default, this just contains one tile, but if the source type is set to Tiled, the user can add additional tiles. When acquiring an ImBuf, the tile to be loaded is specified in the ImageUser.
Therefore, code that is not yet aware of tiles will just access the default tile as usual.

The filenames of the additional tiles are derived from the original filename according to the UDIM naming scheme - the filename contains an index that is calculated as (1001 + 10*<y coordinate of the tile> + <x coordinate of the tile>), where the x coordinate never goes above 9.
Internally, the various tiles are stored in a cache just like sequences. When acquired for the first time, the code will try to load the corresponding file from disk. Alternatively, a new operator can be used to initialize the tile similar to the New Image operator.

The following features are supported so far:
- Automatic detection and loading of all tiles when opening the first tile (1001)
- Saving all tiles
- Adding and removing tiles
- Filling tiles with generated images
- Drawing all tiles in the Image Editor
- Viewing a tiled grid even if no image is selected
- Rendering tiled images in Eevee
- Rendering tiled images in Cycles (in SVM mode)
- Automatically skipping loading of unused tiles in Cycles
- 2D texture painting (also across tiles)
- 3D texture painting (also across tiles, only limitation: individual faces can not cross tile borders)
- Assigning custom labels to individual tiles (drawn in the Image Editor instead of the ID)
- Different resolutions between tiles

There still are some missing features that will be added later (see T72390):
- Workbench engine support
- Packing/Unpacking support
- Baking support
- Cycles OSL support
- many other Blender features that rely on images

Thanks to Brecht for the review and to all who tested the intermediate versions!

Differential Revision: https://developer.blender.org/D3509
2019-12-12 18:40:37 +01:00
Lukas Stockner
e760972221 Cycles: support for custom shader AOVs
Custom render passes are added in the Shader AOVs panel in the view layer
settings, with a name and data type. In shader nodes, an AOV Output node
is then used to output either a value or color to the pass.

Arbitrary names can be used for these passes, as long as they don't conflict
with built-in passes that are enabled. The AOV Output node can be used in both
material and world shader nodes.

Implemented by Lukas, with tweaks by Brecht.

Differential Revision: https://developer.blender.org/D4837
2019-12-10 20:44:46 +01:00
feed46c4ae Merge per-shader SVM nodes in the main update thread to avoid locking and resizing overhead
In a test file containing 1300 copies of the same shader, this reduces shader update time from 3.1 sec to 0.05 sec.

Thanks to @swerner for noticing this issue.

Reviewers: brecht, sergey, swerner

Subscribers: swerner

Differential Revision: https://developer.blender.org/D5376
2019-07-31 12:03:45 -07:00
c47d669f24 Cleanup: comments (long lines) in cycles 2019-05-01 21:41:07 +10:00
e12c08e8d1 ClangFormat: apply to source, most of intern
Apply clang format as proposed in T53211.

For details on usage and instructions for migrating branches
without conflicts, see:

https://wiki.blender.org/wiki/Tools/ClangFormat
2019-04-17 06:21:24 +02:00
84b02dc54a Cleanup: remove accidentally committed debug code. 2018-12-11 15:18:15 +01:00
ece109dd60 Cycles: Add strict assert when assigning input socket stack offset 2018-12-11 14:34:52 +01:00
b45b082531 Cycles: Remove old nasty workaround
It used to be used for some sort of ignoring automatically
generated bump nodes. But nowadays it causes one of the shaders
in Classroom demo file to be compiled wrong.
2018-12-11 14:34:52 +01:00
39d24ffccb Cycles: Cleanup, indentation 2018-12-11 12:57:57 +01:00
cb4b5e12ab Cycles: Cleanup, spacing after preprocessor
It is supposed to be two spaces before comment stating which if
else/endif statements corresponds to. Was mainly violated in the
header guards.
2018-11-09 11:34:54 +01:00
1daa20ad9f Cleanup: strip trailing space for cycles 2018-07-06 10:17:58 +02:00
b763c34e80 Cycles: Cleanup, silence strict compiler warning
There is one legit place in the code where memcpy was used as an
optimization trick. Was needed for older version of GCC, but now
it should be re-evaluated and checked if it still helps to have
that trick.

In other places it's somewhat lazy programming to zero out all
object members. That is absolutely unsafe, at the moment when
less trivial class is used as a member in that object things
will break.

Other cases were using memcpy into an object which comes from
an external library. We don't control that object, and we can
not guarantee it will always be safe for such memory tricks
and debugging bugs caused by such low level access is far fun.

Ideally we need to use more proper C++, but needs to be done with
big care, including benchmarks of each change, For now do
annoying but simple cast to void*.
2018-06-11 13:02:10 +02:00
48155c210a Cycles: Add Support for IES files as textures for light strength
This patch adds support for IES files, a file format that is commonly used to store the directional intensity distribution of light sources.
The new IES node is supposed to be plugged into the Strength input of the Emission node of the lamp.

Since people generating IES files do not really seem to care about the standard, the parser is flexible enough to accept all test files I have tried.
Some common weirdnesses are distributing values over multiple lines that should go into one line, using commas instead of spaces as delimiters and adding various useless stuff at the end of the file.

The user interface of the node is similar to the script node, the user can either select an internal Text or load a file.
Internally, IES files are handled similar to Image textures: They are stored in slots by the LightManager and each unique IES is assigned to one slot.

The local coordinate system of the lamp is used, so that the direction of the light can be changed. For UI reasons, it's usually best to add an area light,
rotate it and then change its type, since especially the point light does not immediately show its local coordinate system in the viewport.

Reviewers: #cycles, dingto, sergey, brecht

Reviewed By: #cycles, dingto, brecht

Subscribers: OgDEV, crazyrobinhood, secundar, cardboard, pisuke, intrah, swerner, micah_denn, harvester, gottfried, disnel, campbellbarton, duarteframos, Lapineige, brecht, juicyfruit, dingto, marek, rickyblender, bliblubli, lockal, sergey

Differential Revision: https://developer.blender.org/D1543
2018-05-27 01:24:57 +02:00
a963c7d48d Code refactor: improve attribute handling for optional volume attributes.
A volume shader should be able to request attributes, and still be rendered
as homogeneous if no volume attributes are available for the object.
2018-02-23 18:57:58 +01:00
8e1dd7ed81 Cycles: Remove unneeded include statements
Also try to move them from headers to implementation files as much as possible.
2018-01-19 15:19:45 +01:00
070a668d04 Code refactor: move more memory allocation logic into device API.
* Remove tex_* and pixels_* functions, replace by mem_*.
* Add MEM_TEXTURE and MEM_PIXELS as memory types recognized by devices.
* No longer create device_memory and call mem_* directly, always go
  through device_only_memory, device_vector and device_pixels.
2017-10-24 01:25:19 +02:00
7ad9333fad Code refactor: store device/interp/extension/type in each device_memory. 2017-10-24 01:03:59 +02:00
57a0cb797d Code refactor: avoid some unnecessary device memory copying. 2017-10-21 20:58:28 +02:00
b5f8063fb9 Cycles: support baking normals plugged into BSDFs, averaged with closure weight. 2017-08-20 16:51:53 +02:00
0b07c2c8a2 Code cleanup: remove copy of shader graph for bump, no longer needed. 2017-08-20 14:27:51 +02:00
c8e764ccbf Cycles: Fix race condition in shader attribute for real now
Ended up moving lock in the more centralized space since multiple shaders
can access this map.
2017-04-10 16:53:01 +02:00
52029e689c Cycles: Fix race condition in attributes creation during SVM compilation 2017-04-05 14:57:54 +02:00
424901ad7b Cycles: Guard global write access in SVM compilation code 2017-04-05 14:21:49 +02:00
0579eaae1f Cycles: Make all #include statements relative to cycles source directory
The idea is to make include statements more explicit and obvious where the
file is coming from, additionally reducing chance of wrong header being
picked up.

For example, it was not obvious whether bvh.h was refferring to builder
or traversal, whenter node.h is a generic graph node or a shader node
and cases like that.

Surely this might look obvious for the active developers, but after some
time of not touching the code it becomes less obvious where file is coming
from.

This was briefly mentioned in T50824 and seems @brecht is fine with such
explicitness, but need to agree with all active developers before committing
this.

Please note that this patch is lacking changes related on GPU/OpenCL
support. This will be solved if/when we all agree this is a good idea to move
forward.

Reviewers: brecht, lukasstockner97, maiself, nirved, dingto, juicyfruit, swerner

Reviewed By: lukasstockner97, maiself, nirved, dingto

Subscribers: brecht

Differential Revision: https://developer.blender.org/D2586
2017-03-29 13:41:11 +02:00
5aaa643947 Cycles: Optimize shaders earlier to skip unneccessary attributes for noninteractive rendering
Before, Cycles would first sync the shader exactly as shown in the UI, then determine and sync the used attributes and later optimize the shader.
Therefore, even completely unconnected nodes would cause unneccessary attributes to be synced.

The reason for this is to avoid frequent resyncs when editing shaders interactively, but it can still be avoided for noninteractive renders - which is what this commit does.

Reviewed by: sergey

Differential Revision: https://developer.blender.org/D2285
2017-03-27 05:36:49 +02:00
265e5def76 Fix T50104, Race condition in SVMShaderManager::device_update_shader 2016-11-25 12:03:04 -07:00
0de69e56b4 Cycles: Implement threaded SVM nodes compilation
The title says it all actually. From tests with barber shop scene here
gives 2-3x speedup for shader compilation on my oldie i7 machine. The
gain is mainly due to textures metadata query from jpeg files (which
seems to requite de-compression before metadata can be read). But in
theory could give nice improvements for scenes with huge node trees
as well (i'm talking about node trees of complexity of fractal which
we had reports about in the past).

Reviewers: juicyfruit, dingto, lukasstockner97, brecht

Reviewed By: brecht

Subscribers: monio, Blendify

Differential Revision: https://developer.blender.org/D2215
2016-09-13 13:46:49 +02:00
92a2c49aab Cycles: Fix bump mapping to use object space when used with true displacement
Bump mapping was happening in world space while displacement happens in object
space, causing shading errors when displacement type was used with bump mapping.

To fix this the proper transforms are added to bump nodes. This is only done
for automatic bump mapping however, to avoid visual changes from other uses of
bump mapping. It would be nice to do this for all bump mapping to be consistent
but that will have to wait till we can break compatibility.

Reviewed By: brecht

Differential Revision: https://developer.blender.org/D2191
2016-09-11 11:20:21 -04:00
4c1d40d87e Cycles: Add overall timing log to SVNShaderManager 2016-09-09 12:27:27 +02:00
7e7a9d146c Cycles: Fix OpenCL speed regression introduced with the improved bump mapping
The two SVM nodes added with e7ea1ae78c caused a slowdown on AMD cards when rendering with OpenCL, whether displacement was used or not.
In the Barcelona Pavillon scene on a RX480, this would cause a 12% slowdown.

Therefore, this commit adds a additional flag for feature-adaptive compilation so that the new SVM nodes are only enabled when they are needed (Node tree connected to the Displacement output and Displacement type set to Both).

Also, the nodes were also added to shaders when the Displacement Type was set to Bump (the default), which was unneccessary and is fixed now.

Thanks to linda2 on IRC for reporting and testing and to maiself for help with the displacement shader code.

This fix might be relevant for 2.78, but it should be tested further before including it.
2016-09-08 01:33:41 +02:00
e7ea1ae78c Cycles microdisplacement: Improved automatic bump mapping
Object coordinates can now be used in the displacement shader and will give
correct results, where as before bump mapping was calculated from the displace
positions and resulted in incorrect shading.

This works by evaluating the shader in two parts, first bump then surface, and
setting the shader state to match what it would be if the surface was
undisplaced for the bump shader evaluation. Currently only `P` is set as if
undisplaced, but other shader variables could be set as well, such as `I` or
`time`. Since these aren't set to anything meaningful for displacement I left
them out of this patch, we can decide what to do with them separately.

Reviewed By: brecht

Differential Revision: https://developer.blender.org/D2156
2016-09-01 22:45:49 -04:00
9f1c42392e Cycles: remove duplicate shader storage
Storing multiple copies of a shader was needed when the displacement method was
a mesh option and could be different for each mesh. Now that its a shader option
this is unnecessary.

Reviewed By: brecht

Differential Revision: https://developer.blender.org/D2156
2016-09-01 22:44:42 -04:00
9f18e3acd6 Cycles: Fix wrong order of light manager update tag
Shader's has_surface_emission will only be known for sure after compilation.
2016-07-28 12:31:01 +02:00
42aec3b355 Cycles: nodify shader nodes
Differential Revision: https://developer.blender.org/D2038
2016-06-11 20:32:24 +02:00
b94bfe4cd8 Code refactor: make ShaderNode match Node a bit more, reusing types and enums.
Differential Revision: https://developer.blender.org/D2016
2016-05-28 15:49:15 +02:00