Reduces the GLSL to MSL translation stage of shader compilation from 120 ms to 5 ms for complex EEVEE materials. This manifests in faster overall compilations, and faster cache hits for secondary compilations, as the MSL variant is needed as a key.
Startup time is also improved for both first-run and second-run. Note that this change does not affect shader compilation times within the Metal API.
Also disables shader output to disk
Authored by Apple: Michael Parkin-White
Ref T96261
Depends on D16990
Reviewed By: fclem
Maniphest Tasks: T96261
Differential Revision: https://developer.blender.org/D17033
Replace texelFetch calls with a texture point-sample rather than a textureRead call. This increases texture cache utilisation when mixing between sampled calls and reads. Bounds checking can also be removed from these functions, reducing instruction count and branch divergence, as the sampler routine handles range clamping.
Authored by Apple: Michael Parkin-White
Ref T96261
Depends on D16923
Reviewed By: fclem
Maniphest Tasks: T96261
Differential Revision: https://developer.blender.org/D17021
Test results were generated from incorrect code.
Code was fixed, but test results weren't updated.
This patch updates the test results to match the implementation.
- `projection_perspective.w.w == 0.0`
Also minor changes in comments:
- Reference BLENDER_HISTORY_FILE instead of the literal file-name
(simplifies looking up usage).
- Use usernames in tags, as noted in code-style.
The glsl files + create infos of shaders that are only used
during development where still being compiled into blender.
This isn't needed and shouldn't be included. This change will
only include them when WITH_GTEST and WITH_OPENGL_DRAW_TESTS are
enabled. All other cases those files will be skipped.
Resolve an issue where released buffers were returned to the reusable memory pool before GPU work associated with these buffers had been encoded. Usually release of memory pools is dependent on successful completion of GPU work via command buffer callbacks. However, if the pool refresh operation occurs between encoding of work and submission, buffer ref-count is prematurely decremented.
Patch also ensures safe buffer free lists are only flushed once a set number of buffers have been used. This reduces overhead of small and frequent flushes, without raising the memory ceiling significantly.
Authored by Apple: Michael Parkin-White
Ref T96261
Reviewed By: fclem
Maniphest Tasks: T96261
Differential Revision: https://developer.blender.org/D17118
Compute test case still used legacy API to construct
GLSL shaders. This change will migrate it to use the
GPUShaderCreateInfo's.
In preparation to run test-cases against non-opengl
back-ends.
Due to shader global scope emulation via class interface, global constant arrays in shaders are allocated in per-thread shader local memory. To reduce memory pressure, placing these constant arrays inside function scope will ensure they only reside within device constant memory. This results in a tangible 1.5-2x performance uplift for the specific shaders affected.
Authored by Apple: Michael Parkin-White
Ref T96261
Reviewed By: fclem
Maniphest Tasks: T96261
Differential Revision: https://developer.blender.org/D17089
Paths to vulkan libraries, paths and related components were
hardcoded in the platform cmake file. This patch separates
this by using adding CMake modules for Vulkan and ShaderC.
This change has only been applied to the macOs configuration as
that is currently our main platform for development. Other platforms
will be added during the development of the Vulkan back-end.
Metal backend does not support primtiive restart for point primtiives. Hence strip_restart_indices removes restart indices by swapping them to the end of the index buffer and reducing the length.
An edge-case existed where all indices within the index buffer were restarts and no valid swap-index would be found, resulting in a buffer underflow.
Authored by Apple: Michael Parkin-White
Ref T96261
Reviewed By: fclem
Maniphest Tasks: T96261
Differential Revision: https://developer.blender.org/D17088
This patch adds support for compilation and execution of GLSL compute shaders. This, along with a few systematic changes and fixes, enable realtime compositor functionality with the Metal backend on macOS. A number of GLSL source modifications have been made to add the required level of type explicitness, allowing all compilations to succeed.
GLSL Compute shader compilation follows a similar path to Vertex/Fragment translation, with added support for shader atomics, shared memory blocks and barriers.
Texture flags have also been updated to ensure correct read/write specification for textures used within the compositor pipeline. GPU command submission changes have also been made in the high level path, when Metal is used, to address command buffer time-outs caused by certain expensive compute shaders.
Authored by Apple: Michael Parkin-White
Ref T96261
Ref T99210
Reviewed By: fclem
Maniphest Tasks: T99210, T96261
Differential Revision: https://developer.blender.org/D16990
Recent changes in our GLSL libraries didn't compile on Vulkan. This
change reverts a compile directive that was removed, but required
in order to compile using the Vulkan backend.
Rewrite of the Workbench engine using C++ and the new Draw Manager API.
The new engine can be enabled in Blender `Preferences > Experimental > Workbench Next`.
After that, the engine can be selected in `Properties > Scene > Render Engine`.
When `Workbench Next` is the active engine, it also handles the `Solid` viewport mode rendering.
The rewrite aims to be functionally equivalent to the current Workbench engine, but it also includes some small fixes/tweaks:
- `In Front` rendered objects now work correctly with DoF and Shadows.
- The `Sampling > Viewport` setting is actually used when the viewport is in `Render Mode`.
- In `Texture` mode, textured materials also use the material properties. (Previously, only non textured materials would)
To do:
- Sculpt PBVH.
- Volume rendering.
- Hair rendering.
- Use the "no_geom" shader versions for shadow rendering.
- Decide the final API for custom visibility culling (Needed for shadows).
- Profile/optimize.
Known Issues:
- Matcaps are not loaded until they’re shown elsewhere. (e.g. when opening the `Viewort Shading` UI)
- Outlines are drawn between different materials of the same object. (Each material submesh has its own object handle)
Reviewed By: fclem
Maniphest Tasks: T101619
Differential Revision: https://developer.blender.org/D16826
Improve handling for cases where maximum in-flight command buffer count is exceeded. This can occur during light-baking operations. Ensures the application handles this gracefully and also improves workload pipelining by situationally stalling until GPU work has completed, if too much work is queued up.
This may have a tangible benefit for T103742 by ensuring Blender does not queue up too much GPU work.
Authored by Apple: Michael Parkin-White
Ref T96261
Ref T103742
Depends on D17018
Reviewed By: fclem
Maniphest Tasks: T103742, T96261
Differential Revision: https://developer.blender.org/D17019
Affecting render output preview when tone mapping is used, and EEVEE scenes such as Mr Elephant rendering in pink due to missing shaders.
Authored by Apple: Michael Parkin-White
Ref T103635
Ref T96261
Reviewed By: fclem
Maniphest Tasks: T103635, T96261
Differential Revision: https://developer.blender.org/D16923
AMD GPUs do not appear to produce consistent results with other GPUs when using textureGather in the Metal backend. Disabling for now to ensure correct function of outline rendering.
This may require an additional sub-pixel offset in the texture sampling calls, to achieve correct behaviour.
Authored by Apple: Michael Parkin-White
Ref T103412
Ref T96261
Reviewed By: fclem
Maniphest Tasks: T103412, T96261
Differential Revision: https://developer.blender.org/D16934
This patch fixes an issue where Blender 3.5 alpha with the Metal GPU backend enabled on Japanese macOS fails to compile shaders and crashes on startup.
In a Japanese environment, `defaultCStringEncoding` is the legacy MacJapanese encoding, and it erroneously converts backslashes (0x5c) to Yen symbols (¥).
Therefore, Metal shader compile fails with the following log and Blender crashes.
```
2022-12-29 13:50:10.200 Blender[13404:246707] Compile Error - Metal Shader Library (Stage: 0), error Error Domain=MTLLibraryErrorDomain Code=3 "program_source:225:74: error: non-ASCII characters are not allowed outside of literals and identifiers
template<typename T, access A = access::sample> struct STRUCT_NAME { ¥
^
program_source:226:14: error: no template named 'TEX_TYPE'
thread TEX_TYPE<T, A> *texture; ¥
^
program_source:226:39: error: non-ASCII characters are not allowed outside of literals and identifiers
thread TEX_TYPE<T, A> *texture; ¥
^
program_source:227:29: error: non-ASCII characters are not allowed outside of literals and identifiers
thread sampler *samp; ¥
^
...
```
We can use `stringWithUTF8String` instead.
Reviewed By: fclem, MichaelPW
Differential Revision: https://developer.blender.org/D16881
`batch_for_shader` is an utility function that creates the correct
vertex buffer based on the given shader. In the shader interface
the `attr_types_` contains the GPUType for each location in the
vertex buffer.
When using Metal, the `attr_types_` was never updated, resulting
in using incorrect or non-existing data types. This patch fixes
this by updating the `attr_types_` when building the shader
interface.
Reviewed By: fclem
Differential Revision: https://developer.blender.org/D17042
These warnings can reveal errors in logic, so quiet them by checking
if the features are enabled before using variables or by assigning
empty strings in some cases.
- Check CMAKE_THREAD_LIBS_INIT is set before use as CMake docs
note that this may be left unset if it's not needed.
- Remove BOOST/OPENVDB/VULKAN references when disable.
- Define INC_SYS even when empty.
- Remove PNG_INC from freetype (not defined anywhere).
Implementation didn't count the string terminator when allocating
memory to store `msl_patch_default`. The string terminator could
be overwritted by other memory adding some undefined behavior.
These allow the usage of `atomicMin` and `atomicMax` function with float
values as there is no overload for these types in GLSL.
This also allows signed 0 preservation.
This wasn't really a problem since these are set on first bind or creation.
The test `if (enabled_srgb && srgb_) {` was depending on that variable that
in certain case, might not have been initialized (because of lazy init).
Currently you can retrieve a mutable array from a const CustomData.
That makes code unsafe since the compiler can't check for correctness
itself. Fix that by introducing a separate function to retrieve mutable
arrays from CustomData. The new functions have the `_for_write`
suffix that make the code's intention clearer.
Because it makes retrieving write access an explicit step, this change
also makes proper copy-on-write possible for attributes.
Notes:
- The previous "duplicate referenced layer" functions are redundant
with retrieving layers with write access
- The custom data functions that give a specific index only have
`for_write` to simplify the API
Differential Revision: https://developer.blender.org/D14140
OpenGL is deprecated by Apple and triggers a warning when used. The goal
is that OpenGL is replaced by Metal backend, but we are not there yet.
To improve tracability of new warnings we hide deprecation warnings
when the GHOST_ContextCGL.h file is included.
NOTE: This change silences other deprecation warnings as well.
Currently the `MLoopUV` struct stores UV coordinates and flags related
to editing UV maps in the UV editor. This patch changes the coordinates
to use the generic 2D vector type, and moves the flags into three
separate boolean attributes. This follows the design in T95965, with
the ultimate intention of simplifying code and improving performance.
Importantly, the change allows exporters and renderers to use UVs
"touched" by geometry nodes, which only creates generic attributes.
It also allows geometry nodes to create "proper" UV maps from scratch,
though only with the Store Named Attribute node for now.
The new design considers any 2D vector attribute on the corner domain
to be a UV map. In the future, they might be distinguished from regular
2D vectors with attribute metadata, which may be helpful because they
are often interpolated differently.
Most of the code changes deal with passing around UV BMesh custom data
offsets and tracking the boolean "sublayers". The boolean layers are
use the following prefixes for attribute names: vert selection: `.vs.`,
edge selection: `.es.`, pinning: `.pn.`. Currently these are short to
avoid using up the maximum length of attribute names. To accommodate
for these 4 extra characters, the name length limit is enlarged to 68
bytes, while the maximum user settable name length is still 64 bytes.
Unfortunately Python/RNA API access to the UV flag data becomes slower.
Accessing the boolean layers directly is be better for performance in
general.
Like the other mesh SoA refactors, backward and forward compatibility
aren't affected, and won't be changed until 4.0. We pay for that by
making mesh reading and writing more expensive with conversions.
Resolves T85962
Differential Revision: https://developer.blender.org/D14365
**Changes**
As described in T93602, this patch removes all use of the `MVert`
struct, replacing it with a generic named attribute with the name
`"position"`, consistent with other geometry types.
Variable names have been changed from `verts` to `positions`, to align
with the attribute name and the more generic design (positions are not
vertices, they are just an attribute stored on the point domain).
This change is made possible by previous commits that moved all other
data out of `MVert` to runtime data or other generic attributes. What
remains is mostly a simple type change. Though, the type still shows up
859 times, so the patch is quite large.
One compromise is that now `CD_MASK_BAREMESH` now contains
`CD_PROP_FLOAT3`. With the general move towards generic attributes
over custom data types, we are removing use of these type masks anyway.
**Benefits**
The most obvious benefit is reduced memory usage and the benefits
that brings in memory-bound situations. `float3` is only 3 bytes, in
comparison to `MVert` which was 4. When there are millions of vertices
this starts to matter more.
The other benefits come from using a more generic type. Instead of
writing algorithms specifically for `MVert`, code can just use arrays
of vectors. This will allow eliminating many temporary arrays or
wrappers used to extract positions.
Many possible improvements aren't implemented in this patch, though
I did switch simplify or remove the process of creating temporary
position arrays in a few places.
The design clarity that "positions are just another attribute" brings
allows removing explicit copying of vertices in some procedural
operations-- they are just processed like most other attributes.
**Performance**
This touches so many areas that it's hard to benchmark exhaustively,
but I observed some areas as examples.
* The mesh line node with 4 million count was 1.5x (8ms to 12ms) faster.
* The Spring splash screen went from ~4.3 to ~4.5 fps.
* The subdivision surface modifier/node was slightly faster
RNA access through Python may be slightly slower, since now we need
a name lookup instead of just a custom data type lookup for each index.
**Future Improvements**
* Remove uses of "vert_coords" functions:
* `BKE_mesh_vert_coords_alloc`
* `BKE_mesh_vert_coords_get`
* `BKE_mesh_vert_coords_apply{_with_mat4}`
* Remove more hidden copying of positions
* General simplification now possible in many areas
* Convert more code to C++ to use `float3` instead of `float[3]`
* Currently `reinterpret_cast` is used for those C-API functions
Differential Revision: https://developer.blender.org/D15982
- Matrix normalize overloads needs to have the vector normalize redefined.
- double underscore (anywhere in symbol name) are reserved.
- Some operation yield different result due to float imprecision. Increasing
epsilon threshold for the failing tests.
Staging texture update copied over the entire texture, rather than just the region of the texture which had been updated. Also added early-exit for cases where the net texture update extent was zero, as this was causing validation failures.
Authored by Apple: Michael Parkin-White
Ref T103658
Ref T96261
Reviewed By: fclem
Maniphest Tasks: T103658, T96261
Differential Revision: https://developer.blender.org/D16924
First binding of a framebuffer lead to an incorrect SRGB conversion state being applied, as attachments, where presence of SRGB is determined, were processed after the SRGB check rather than before.
This DIFF also cleans up SRGB naming conventions and caching of fallback non-srgb texture view, for use when SRGB mode is disabled.
Authored by Apple: Michael Parkin-White
Ref T103399
Ref T96261
Reviewed By: fclem
Maniphest Tasks: T103399, T96261
Differential Revision: https://developer.blender.org/D16907
Required texture bytesize calculation for compacted data types was incorrectly calculated, resulting in an erroneous format conversion taking place instead of direct data upload.
Metal dummy buffer size also temporarily increased to address problematic cases where the bound buffer was too small for missing UBOs.
Authored by Apple: Michael Parkin-White
Ref T96261
Reviewed By: fclem
Maniphest Tasks: T96261
Differential Revision: https://developer.blender.org/D16904
This implement most of the functions provided by the BLI math library.
This is part of the effort to unify GLSL and C++ syntax. Ref T103026.
This also adds some infrastructure to make it possible to run GLSL shader unit
test.
Some code already present in other libs is being copied to the new libs.
This patch does not make use of the new libs outside of the tests.
Note that the test is still crashing when using metal.
