This PR adds basic support for texture update, read back and clearing
for Vulkan. In Vulkan we need to convert each data type ourselves as
vulkan buffers are untyped. Therefore this change mostly is about data
conversions.
Considerations:
- Use a compute shader to do the conversions:
- Leads to performance regression as compute pipeline can stall
graphics pipeline
- Lead to additional memory usage as two staging buffers are needed
one to hold the CPU data, and one to hold the converted data.
- Do inline conversion when sending the data to Vulkan using `eGPUDataFormat`
- Additional CPU cycles required and not easy to optimize as it the
implementation requires many branches.
- Do inline conversion when sending the data to Vulkan (optimized for CPU)
For this solution it was chosen to implement the 3rd option as it is fast
and doesn't require additional memory what the other options do.
**Use Imath/half.h**
This patch uses `Imath/half.h` (dependency of OpenEXR) similar to
alembic. But this makes vulkan dependent of the availability of
OpenEXR. For now this isn't checked, but when we are closer to
a working Vulkan backend we have to make a decision how to cope with
this dependency.
**Missing Features**
*Framebuffer textures*
This doesn't include all possible data transformations. Some of those
transformation can only be tested after the VKFramebuffer has been
implemented. Some texture types are only available when created for a
framebuffer. These include the depth and stencil variations.
*Component format*
Is more relevant when implementing VKVertexBuffer.
*SRGB textures*
SRGB encoded textures aren't natively supported on all platforms, in
all usages and might require workarounds. This should be done in a
separate PR in a later stage when we are required to use SRGB textures.
**Test cases**
The added test cases gives an overview of the missing bits and pieces of
the patch. When the implementation/direction is accepted more test cases
can be enabled/implemented.
Some of these test cases will skip depending on the actual support of
platform the tests are running on. For example OpenGL/NVidia will skip
the next test as it doesn't support the texture format on OpenGL, although
it does support it on Vulkan.
```
[ RUN ] GPUOpenGLTest.texture_roundtrip__GPU_DATA_2_10_10_10_REV__GPU_RGB10_A2UI
[ SKIPPED ] GPUOpenGLTest.texture_roundtrip__GPU_DATA_2_10_10_10_REV__GPU_RGB10_A2UI [ RUN ] GPUVulkanTest.texture_roundtrip__GPU_DATA_2_10_10_10_REV__GPU_RGB10_A2UI
[ OK ] GPUVulkanTest.texture_roundtrip__GPU_DATA_2_10_10_10_REV__GPU_RGB10_A2UI
```
Pull Request: blender/blender#105762
In Vulkan multiple commands can be in flight simultaneously.
These commands can share resources like descriptor sets or push
constants. When between commands these resources are updated
a new version of the resources should be created.
When a resource is updated it should check the submission id of the
command buffer. If this is different than last known by the resources,
the previous resources should be freed.
If the submission id is the same than previously it has to create a
new version of the resource to not intervene with other commands that
uses the resource before the update.
When the resource wasn't updated between multiple usages in the same
submission id it could reuse the previous resource.
This PR introduces a `ResourceTracker` and a `SubmissionTracker`.
A submission tracker can check if the command buffer is submitted.
In this case all resources of the resource tracker should be freed.
Unmodified resources in the same submission can be shared.
A resource tracker will keep track of all resources that are in
flight. After the resources are used (submission + execution) have
finished the resources can be cleared.
Pull Request: blender/blender#105183
This PR uses renderdoc for frame capturing when enabled.
It enabled an easier workflow for frame capturing.
- Capture GPU API calls from test cases
- Capture GPU API calls from background threads
- Capture GPU API calls from background rendering.
Renderdoc is an important GPU debugger used by the Eevee/
Viewport module. Previously we needed to change code in
order to record background rendering, that could on its own
lead to other side-effects.
The integration with renderdoc can be enabled using
`WITH_RENDERDOC=On` compiler option. `GPU_debug_capture_begin`
and `GPU_debug_capture_end` can be added to the section
of the code you want to debug. When running Blender inside
renderdoc this part will automatically be captured.
All GPU test cases are now guarded by these calls. In order
to capture the test cases you need to start the test cases
from renderdoc and the captured GPU API calls will appear
where each capture is a single test case.
Pull Request: blender/blender#105921
This PR adds support for `GPU_storagebuf_clear` and
`GPU_storagebuf_clear_zero` to the Vulkan backend. It also adds test
cases for all backends.
Pull Request: blender/blender#105487
The _gpu_ prefix is redundant as they are inside a namespace
and run as part of a test suite that already contain the name
gpu. (GPUOpenGLTest)
This patch also moved the texture test cases to its own
compile module.
When a shader is bound it should outlife the pipeline. In one
test case where only the shader data was accessed it also bound
the shader. This isn't needed as the shader data should be
retrieved without binding the shader.
This change fixes the issue by not binding the GPU shader.
Both the shader_builder and existing shader tests eventually
tested the same aspects. shader_builder is more modern and
handles more cases.
The old shader test requires a full backend in order to run
This commit replaces the old tests to just use the
shader builder for validation.
Shader builder can still be run at compile time, this is
just a convenience to have as a test case as well for CI/CD.
Ref: #105482
**What are push constants?**
Push constants is a way to quickly provide a small amount of uniform data to shaders.
It should be much quicker than UBOs but a huge limitation is the size of data - spec
requires 128 bytes to be available for a push constant range.
**What are the challenges with push constants?**
The challenge with push constants is that the limited available size. According to
the Vulkan spec each platform should at least have 128 bytes reserved for push
constants. Current Mesa/AMD drivers supports 256 bytes, but Mesa/Intel is only 128
bytes.
**What is our solution?**
Some shaders of Blender uses more than these boundaries. When more data is needed
push constants will not be used, but the shader will be patched to use an uniform
buffer instead. This mechanism will be part of the Vulkan backend and shader
developers should not see any difference on API level.
**Known limitations**
Current state of the vulkan backend does not track resources that are in the
command queue. This patch includes some test cases that identified this issue as
well. See #104771.
Pull Request #104880
Resolves issue with nearest filtering on UI Icons. Note that as
Metal does not support LOD bias as a parameter on a sampler
object, the original code has been modified to perform LOD
biasing at the shader level.
As GPU_SAMPLER_ICON is not widely used, it is more
efficient to apply directly to the affected shaders, rather
than workaround passing in the sampler LOD bias as a
separate value e.g. uniform or push constant.
Original PR feedback addressed to also refactor ICON
shaders to use consistent style for single and multi
Icon rendering.
Authored by Apple: Michael Parkin-White
Ref #96261
Pull Request #105145
Descriptor set locations are now determined in the
VKShaderInterface. Issues with the previous solution:
- Due to legacy code in GPU module the locations/bindings
must be the same. Using one for something else might
result in undesired lookups, incorrect resource
bindings.
- Images/Textures reuses the same namespace, that didn't
work as expected when looking up the resources via
its binding.
This refactoring is required for adding support for
push constants.
Pull Request #105073
This patch adds initial support for compute shaders to
the vulkan backend. As the development is oriented to the test-
cases we have the implementation is limited to what is used there.
It has been validated that with this patch that the following test
cases are running as expected
- `GPUVulkanTest.gpu_shader_compute_vbo`
- `GPUVulkanTest.gpu_shader_compute_ibo`
- `GPUVulkanTest.gpu_shader_compute_ssbo`
- `GPUVulkanTest.gpu_storage_buffer_create_update_read`
- `GPUVulkanTest.gpu_shader_compute_2d`
This patch includes:
- Allocating VkBuffer on device.
- Uploading data from CPU to VkBuffer.
- Binding VkBuffer as SSBO to a compute shader.
- Execute compute shader and altering VkBuffer.
- Download the VkBuffer to CPU ram.
- Validate that it worked.
- Use device only vertex buffer as SSBO
- Use device only index buffer as SSBO
- Use device only image buffers
GHOST API has been changed as the original design was created before
we even had support for compute shaders in blender. The function
`GHOST_getVulkanBackbuffer` has been separated to retrieve the command
buffer without a backbuffer (`GHOST_getVulkanCommandBuffer`). In order
to do correct command buffer processing we needed access to the queue
owned by GHOST. This is returned as part of the `GHOST_getVulkanHandles`
function.
Open topics (not considered part of this patch)
- Memory barriers & command buffer encoding
- Indirect compute dispatching
- Rest of the test cases
- Data conversions when requested data format is different than on device.
- GPUVulkanTest.gpu_shader_compute_1d is supported on AMD devices.
NVIDIA doesn't seem to support 1d textures.
Pull-request: #104518
This replaces `GPU_SHADER_3D_POINT_FIXED_SIZE_VARYING_COLOR` by
GPU_SHADER_2D_POINT_UNIFORM_SIZE_UNIFORM_COLOR_OUTLINE_AA`.
None of the usage made sense to not use the AA shader.
Scale the point size to account for the rounded shape.
The GPU module has 2 different styles when reading back data from
GPU buffers. The SSBOs used a memcpy to copy the data to a
pre-allocated buffer. IndexBuf/VertBuf gave back a driver/platform
controlled pointer to the memory.
Readback is done for test cases returning mapped pointers is not safe.
For this reason we settled on using the same approach as the SSBO.
Copy the data to a caller pre-allocated buffer.
Reason why this API is currently changed is that the Vulkan API is more
strict on mapping/unmapping buffers that can lead to potential issues
down the road.
Pull Request #104571
Implements virtual shadow mapping for EEVEE-Next primary shadow solution.
This technique aims to deliver really high precision shadowing for many
lights while keeping a relatively low cost.
The technique works by splitting each shadows in tiles that are only
allocated & updated on demand by visible surfaces and volumes.
Local lights use cubemap projection with mipmap level of detail to adapt
the resolution to the receiver distance.
Sun lights use clipmap distribution or cascade distribution (depending on
which is better) for selecting the level of detail with the distance to
the camera.
Current maximum shadow precision for local light is about 1 pixel per 0.01
degrees.
For sun light, the maximum resolution is based on the camera far clip
distance which sets the most coarse clipmap.
## Limitation:
Alpha Blended surfaces might not get correct shadowing in some corner
casses. This is to be fixed in another commit.
While resolution is greatly increase, it is still finite. It is virtually
equivalent to one 8K shadow per shadow cube face and per clipmap level.
There is no filtering present for now.
## Parameters:
Shadow Pool Size: In bytes, amount of GPU memory to dedicate to the
shadow pool (is allocated per viewport).
Shadow Scaling: Scale the shadow resolution. Base resolution should
target subpixel accuracy (within the limitation of the technique).
Related to #93220
Related to #104472
Test case is a smaller step towards supporting Vulkan. Other
test cases rely on SSBOs as well so it is better to first satisfy
this step before handling the others.
Test results were generated from incorrect code.
Code was fixed, but test results weren't updated.
This patch updates the test results to match the implementation.
- `projection_perspective.w.w == 0.0`
The glsl files + create infos of shaders that are only used
during development where still being compiled into blender.
This isn't needed and shouldn't be included. This change will
only include them when WITH_GTEST and WITH_OPENGL_DRAW_TESTS are
enabled. All other cases those files will be skipped.
Compute test case still used legacy API to construct
GLSL shaders. This change will migrate it to use the
GPUShaderCreateInfo's.
In preparation to run test-cases against non-opengl
back-ends.
These allow the usage of `atomicMin` and `atomicMax` function with float
values as there is no overload for these types in GLSL.
This also allows signed 0 preservation.
- Matrix normalize overloads needs to have the vector normalize redefined.
- double underscore (anywhere in symbol name) are reserved.
- Some operation yield different result due to float imprecision. Increasing
epsilon threshold for the failing tests.
This implement most of the functions provided by the BLI math library.
This is part of the effort to unify GLSL and C++ syntax. Ref T103026.
This also adds some infrastructure to make it possible to run GLSL shader unit
test.
Some code already present in other libs is being copied to the new libs.
This patch does not make use of the new libs outside of the tests.
Note that the test is still crashing when using metal.
MTLContext provides functionality for command encoding, binding management and graphics device management. MTLImmediate provides simple draw enablement with dynamically encoded data. These draws utilise temporary scratch buffer memory to provide minimal bandwidth overhead during workload submission.
This patch also contains empty placeholders for MTLBatch and MTLDrawList to enable testing of first pixels on-screen without failure.
The Metal API also requires access to the GHOST_Context to ensure the same pre-initialized Metal GPU device is used by the viewport. Given the explicit nature of Metal, explicit control is also needed over presentation, to ensure correct work scheduling and rendering pipeline state.
Authored by Apple: Michael Parkin-White
Ref T96261
(The diff is based on 043f59cb3b)
Reviewed By: fclem
Differential Revision: https://developer.blender.org/D15953
With libepoxy we can choose between EGL and GLX at runtime, as well as
dynamically open EGL and GLX libraries without linking to them.
This will make it possible to build with Wayland, EGL, GLVND support while
still running on systems that only have X11, GLX and libGL. It also paves
the way for headless rendering through EGL.
libepoxy is a new library dependency, and is included in the precompiled
libraries. GLEW is no longer a dependency, and WITH_SYSTEM_GLEW was removed.
Includes contributions by Brecht Van Lommel, Ray Molenkamp, Campbell Barton
and Sergey Sharybin.
Ref T76428
Differential Revision: https://developer.blender.org/D15291
Backend initialization needs to be delayed until after the OpenGL context
is created. This worked fine in foreground mode because the OpenGL context
already exists for the window at the point GPU_backend_init_once was called,
but not for background mode.
Create the backend just in time in GPU_context_create as before, and
automatically free it when the last context id discarded. But check if any
GPU backend is supported before creating the OpenGL context.
Ref D15463, D15465
This causes an assert with libepoxy, but was wrong already regardless.
Refactor logic to work as follows:
* GPU_exit() deletes backend resources
* Destroy UI GPU resources with the context active
* Call GPU_backend_exit() after deleting the context
Ref D15291
Differential Revision: https://developer.blender.org/D15465
Adds an example python script to the documentation for the 3D_IMAGE shader.
The **use-case** is to draw textures with 3D vertex positions, in XR views as well as non-XR views (in a simpler manner).
**Testing**: I've tested that this compiles and works on my Macbook (with the example python script included in this change). I don't have access to a Windows or Linux machine right now, but this change doesn't look platform-specific and no new glsl shaders have been added or edited by this change. I'll try to get access to a Windows machine, but if someone does have one, I'd be really grateful if they could try this change. Thanks!
**Problem addressed**: The existing 2D_IMAGE shader (exposed in the python API) gets near-clipped when drawn in the
XR view, regardless of the near-clip settings. Additionally, the 2D_IMAGE shader only accepts 2D
positions for the image vertices, which means drawing textures in 3D requires providing
2D coordinates and then pushing a transform-rotate-scale matrix to the GPU, even for
non-XR (i.e. WINDOW) views. The 3D_IMAGE shader is simpler: it accepts 3D vertex positions, and doesn't require
any additional work by the scripter.
**Workaround**: The current workaround is to use custom shaders in the python script.
**Non-intrusive change**: No new glsl shaders were added. This change just bundles two existing shaders: the vertex shader used
by the 3D_IMAGE_MODULATE_ALPHA shader, and the fragment shader used by the 2D_IMAGE shader.
Reviewed By: #eevee_viewport, jbakker
Differential Revision: https://developer.blender.org/D14832
Use a shorter/simpler license convention, stops the header taking so
much space.
Follow the SPDX license specification: https://spdx.org/licenses
- C/C++/objc/objc++
- Python
- Shell Scripts
- CMake, GNUmakefile
While most of the source tree has been included
- `./extern/` was left out.
- `./intern/cycles` & `./intern/atomic` are also excluded because they
use different header conventions.
doc/license/SPDX-license-identifiers.txt has been added to list SPDX all
used identifiers.
See P2788 for the script that automated these edits.
Reviewed By: brecht, mont29, sergey
Ref D14069
