This is a first part of the Shader Create Info system could be.
A shader create info provides a way to define shader structure, resources
and interfaces. This makes for a quick way to provide backend agnostic
binding informations while also making shader variations easy to declare.
- Clear source input (only one file). Cleans up the GPU api since we can create a
shader from one descriptor
- Resources and interfaces are generated by the backend (much simpler than parsing).
- Bindings are explicit from position in the array.
- GPUShaderInterface becomes a trivial translation of enums and string copy.
- No external dependency to third party lib.
- Cleaner code, less fragmentation of resources in several libs.
- Easy to modify / extend at runtime.
- no parser involve, very easy to code.
- Does not hold any data, can be static and kept on disc.
- Could hold precompiled bytecode for static shaders.
This also includes a new global dependency system.
GLSL shaders can include other sources by using #pragma BLENDER_REQUIRE(...).
This patch already migrated several builtin shaders. Other shaders should be migrated
one at a time, and could be done inside master.
There is a new compile directive `WITH_GPU_SHADER_BUILDER` this is an optional
directive for linting shaders to increase turn around time.
What is remaining:
- pyGPU API {T94975}
- Migration of other shaders. This could be a community effort.
Reviewed By: jbakker
Maniphest Tasks: T94975
Differential Revision: https://developer.blender.org/D13360
The crash is due to the fact that GPU subdivision extraction routines
for edit data (including UVs) only worked for BMesh. However, a Mesh
based version is still needed for texture painting. This adds the
missing components. This also ensures all data are properly initialized
(at least the ones revealed by the bug).
This puts the loop over the final subdivision quads outside of the mesh
iteration callback. This can also allow for easier parallel execution in
the future if need be.
As described in T91186, this commit moves mesh vertex normals into a
contiguous array of float vectors in a custom data layer, how face
normals are currently stored.
The main interface is documented in `BKE_mesh.h`. Vertex and face
normals are now calculated on-demand and cached, retrieved with an
"ensure" function. Since the logical state of a mesh is now "has
normals when necessary", they can be retrieved from a `const` mesh.
The goal is to use on-demand calculation for all derived data, but
leave room for eager calculation for performance purposes (modifier
evaluation is threaded, but viewport data generation is not).
**Benefits**
This moves us closer to a SoA approach rather than the current AoS
paradigm. Accessing a contiguous `float3` is much more efficient than
retrieving data from a larger struct. The memory requirements for
accessing only normals or vertex locations are smaller, and at the
cost of more memory usage for just normals, they now don't have to
be converted between float and short, which also simplifies code
In the future, the remaining items can be removed from `MVert`,
leaving only `float3`, which has similar benefits (see T93602).
Removing the combination of derived and original data makes it
conceptually simpler to only calculate normals when necessary.
This is especially important now that we have more opportunities
for temporary meshes in geometry nodes.
**Performance**
In addition to the theoretical future performance improvements by
making `MVert == float3`, I've done some basic performance testing
on this patch directly. The data is fairly rough, but it gives an idea
about where things stand generally.
- Mesh line primitive 4m Verts: 1.16x faster (36 -> 31 ms),
showing that accessing just `MVert` is now more efficient.
- Spring Splash Screen: 1.03-1.06 -> 1.06-1.11 FPS, a very slight
change that at least shows there is no regression.
- Sprite Fright Snail Smoosh: 3.30-3.40 -> 3.42-3.50 FPS, a small
but observable speedup.
- Set Position Node with Scaled Normal: 1.36x faster (53 -> 39 ms),
shows that using normals in geometry nodes is faster.
- Normal Calculation 1.6m Vert Cube: 1.19x faster (25 -> 21 ms),
shows that calculating normals is slightly faster now.
- File Size of 1.6m Vert Cube: 1.03x smaller (214.7 -> 208.4 MB),
Normals are not saved in files, which can help with large meshes.
As for memory usage, it may be slightly more in some cases, but
I didn't observe any difference in the production files I tested.
**Tests**
Some modifiers and cycles test results need to be updated with this
commit, for two reasons:
- The subdivision surface modifier is not responsible for calculating
normals anymore. In master, the modifier creates different normals
than the result of the `Mesh` normal calculation, so this is a bug
fix.
- There are small differences in the results of some modifiers that
use normals because they are not converted to and from `short`
anymore.
**Future improvements**
- Remove `ModifierTypeInfo::dependsOnNormals`. Code in each modifier
already retrieves normals if they are needed anyway.
- Copy normals as part of a better CoW system for attributes.
- Make more areas use lazy instead of eager normal calculation.
- Remove `BKE_mesh_normals_tag_dirty` in more places since that is
now the default state of a new mesh.
- Possibly apply a similar change to derived face corner normals.
Differential Revision: https://developer.blender.org/D12770
This adds wrapper classes that make it easier to use GPU objects in C++.
####Motivations:####
- Easier handling of GPU objects.
- EEVEE rewrite already makes use of similar wrappers.
- There is the ongoing effort to use more C++ in the codebase
and lans to port more engines to it.
- The shader code refactor will make use of many UBOs with shared
struct declaration. This helps managing them.
- Safer handling of `TextureFromPool` which can't be bound as normal
texture (only texture ref) and can be better tracked in the future.
####Considerations:####
- I chose the `blender::draw` namespace because `blender::gpu` already has private classes (i.e: `gpu::Texture`).
- Theses are wrappers that manage a GPU object internally. They might be confused with actual `Texture`. However, the name `TextureWrapper` is a bit too much verbose in my opinion. I'm open to suggestion about better name.
Reviewed By: jbakker
Differential Revision: http://developer.blender.org/D13805
This patch implements the vector types (i.e:`float2`) by making heavy
usage of templating. All vector functions are now outside of the vector
classes (inside the `blender::math` namespace) and are not vector size
dependent for the most part.
In the ongoing effort to make shaders less GL centric, we are aiming
to share more code between GLSL and C++ to avoid code duplication.
####Motivations:
- We are aiming to share UBO and SSBO structures between GLSL and C++.
This means we will use many of the existing vector types and others
we currently don't have (uintX, intX). All these variations were
asking for many more code duplication.
- Deduplicate existing code which is duplicated for each vector size.
- We also want to share small functions. Which means that vector
functions should be static and not in the class namespace.
- Reduce friction to use these types in new projects due to their
incompleteness.
- The current state of the `BLI_(float|double|mpq)(2|3|4).hh` is a
bit of a let down. Most clases are incomplete, out of sync with each
others with different codestyles, and some functions that should be
static are not (i.e: `float3::reflect()`).
####Upsides:
- Still support `.x, .y, .z, .w` for readability.
- Compact, readable and easilly extendable.
- All of the vector functions are available for all the vectors types
and can be restricted to certain types. Also template specialization
let us define exception for special class (like mpq).
- With optimization ON, the compiler unroll the loops and performance
is the same.
####Downsides:
- Might impact debugability. Though I would arge that the bugs are
rarelly caused by the vector class itself (since the operations are
quite trivial) but by the type conversions.
- Might impact compile time. I did not saw a significant impact since
the usage is not really widespread.
- Functions needs to be rewritten to support arbitrary vector length.
For instance, one can't call `len_squared_v3v3` in
`math::length_squared()` and call it a day.
- Type cast does not work with the template version of the `math::`
vector functions. Meaning you need to manually cast `float *` and
`(float *)[3]` to `float3` for the function calls.
i.e: `math::distance_squared(float3(nearest.co), positions[i]);`
- Some parts might loose in readability:
`float3::dot(v1.normalized(), v2.normalized())`
becoming
`math::dot(math::normalize(v1), math::normalize(v2))`
But I propose, when appropriate, to use
`using namespace blender::math;` on function local or file scope to
increase readability.
`dot(normalize(v1), normalize(v2))`
####Consideration:
- Include back `.length()` method. It is quite handy and is more C++
oriented.
- I considered the GLM library as a candidate for replacement. It felt
like too much for what we need and would be difficult to extend / modify
to our needs.
- I used Macros to reduce code in operators declaration and potential
copy paste bugs. This could reduce debugability and could be reverted.
- This touches `delaunay_2d.cc` and the intersection code. I would like
to know @howardt opinion on the matter.
- The `noexcept` on the copy constructor of `mpq(2|3)` is being removed.
But according to @JacquesLucke it is not a real problem for now.
I would like to give a huge thanks to @JacquesLucke who helped during this
and pushed me to reduce the duplication further.
Reviewed By: brecht, sergey, JacquesLucke
Differential Revision: https://developer.blender.org/D13791
This patch implements the vector types (i.e:`float2`) by making heavy
usage of templating. All vector functions are now outside of the vector
classes (inside the `blender::math` namespace) and are not vector size
dependent for the most part.
In the ongoing effort to make shaders less GL centric, we are aiming
to share more code between GLSL and C++ to avoid code duplication.
####Motivations:
- We are aiming to share UBO and SSBO structures between GLSL and C++.
This means we will use many of the existing vector types and others
we currently don't have (uintX, intX). All these variations were
asking for many more code duplication.
- Deduplicate existing code which is duplicated for each vector size.
- We also want to share small functions. Which means that vector
functions should be static and not in the class namespace.
- Reduce friction to use these types in new projects due to their
incompleteness.
- The current state of the `BLI_(float|double|mpq)(2|3|4).hh` is a
bit of a let down. Most clases are incomplete, out of sync with each
others with different codestyles, and some functions that should be
static are not (i.e: `float3::reflect()`).
####Upsides:
- Still support `.x, .y, .z, .w` for readability.
- Compact, readable and easilly extendable.
- All of the vector functions are available for all the vectors types
and can be restricted to certain types. Also template specialization
let us define exception for special class (like mpq).
- With optimization ON, the compiler unroll the loops and performance
is the same.
####Downsides:
- Might impact debugability. Though I would arge that the bugs are
rarelly caused by the vector class itself (since the operations are
quite trivial) but by the type conversions.
- Might impact compile time. I did not saw a significant impact since
the usage is not really widespread.
- Functions needs to be rewritten to support arbitrary vector length.
For instance, one can't call `len_squared_v3v3` in
`math::length_squared()` and call it a day.
- Type cast does not work with the template version of the `math::`
vector functions. Meaning you need to manually cast `float *` and
`(float *)[3]` to `float3` for the function calls.
i.e: `math::distance_squared(float3(nearest.co), positions[i]);`
- Some parts might loose in readability:
`float3::dot(v1.normalized(), v2.normalized())`
becoming
`math::dot(math::normalize(v1), math::normalize(v2))`
But I propose, when appropriate, to use
`using namespace blender::math;` on function local or file scope to
increase readability.
`dot(normalize(v1), normalize(v2))`
####Consideration:
- Include back `.length()` method. It is quite handy and is more C++
oriented.
- I considered the GLM library as a candidate for replacement. It felt
like too much for what we need and would be difficult to extend / modify
to our needs.
- I used Macros to reduce code in operators declaration and potential
copy paste bugs. This could reduce debugability and could be reverted.
- This touches `delaunay_2d.cc` and the intersection code. I would like
to know @howardt opinion on the matter.
- The `noexcept` on the copy constructor of `mpq(2|3)` is being removed.
But according to @JacquesLucke it is not a real problem for now.
I would like to give a huge thanks to @JacquesLucke who helped during this
and pushed me to reduce the duplication further.
Reviewed By: brecht, sergey, JacquesLucke
Differential Revision: https://developer.blender.org/D13791
This patch implements the vector types (i.e:float2) by making heavy
usage of templating. All vector functions are now outside of the vector
classes (inside the blender::math namespace) and are not vector size
dependent for the most part.
In the ongoing effort to make shaders less GL centric, we are aiming
to share more code between GLSL and C++ to avoid code duplication.
Motivations:
- We are aiming to share UBO and SSBO structures between GLSL and C++.
This means we will use many of the existing vector types and others we
currently don't have (uintX, intX). All these variations were asking
for many more code duplication.
- Deduplicate existing code which is duplicated for each vector size.
- We also want to share small functions. Which means that vector functions
should be static and not in the class namespace.
- Reduce friction to use these types in new projects due to their
incompleteness.
- The current state of the BLI_(float|double|mpq)(2|3|4).hh is a bit of a
let down. Most clases are incomplete, out of sync with each others with
different codestyles, and some functions that should be static are not
(i.e: float3::reflect()).
Upsides:
- Still support .x, .y, .z, .w for readability.
- Compact, readable and easilly extendable.
- All of the vector functions are available for all the vectors types and
can be restricted to certain types. Also template specialization let us
define exception for special class (like mpq).
- With optimization ON, the compiler unroll the loops and performance is
the same.
Downsides:
- Might impact debugability. Though I would arge that the bugs are rarelly
caused by the vector class itself (since the operations are quite trivial)
but by the type conversions.
- Might impact compile time. I did not saw a significant impact since the
usage is not really widespread.
- Functions needs to be rewritten to support arbitrary vector length. For
instance, one can't call len_squared_v3v3 in math::length_squared() and
call it a day.
- Type cast does not work with the template version of the math:: vector
functions. Meaning you need to manually cast float * and (float *)[3] to
float3 for the function calls.
i.e: math::distance_squared(float3(nearest.co), positions[i]);
- Some parts might loose in readability:
float3::dot(v1.normalized(), v2.normalized())
becoming
math::dot(math::normalize(v1), math::normalize(v2))
But I propose, when appropriate, to use
using namespace blender::math; on function local or file scope to
increase readability. dot(normalize(v1), normalize(v2))
Consideration:
- Include back .length() method. It is quite handy and is more C++
oriented.
- I considered the GLM library as a candidate for replacement.
It felt like too much for what we need and would be difficult to
extend / modify to our needs.
- I used Macros to reduce code in operators declaration and potential
copy paste bugs. This could reduce debugability and could be reverted.
- This touches delaunay_2d.cc and the intersection code. I would like to
know @Howard Trickey (howardt) opinion on the matter.
- The noexcept on the copy constructor of mpq(2|3) is being removed.
But according to @Jacques Lucke (JacquesLucke) it is not a real problem
for now.
I would like to give a huge thanks to @Jacques Lucke (JacquesLucke) who
helped during this and pushed me to reduce the duplication further.
Reviewed By: brecht, sergey, JacquesLucke
Differential Revision: http://developer.blender.org/D13791
Issue introduced in rB1d49293b80446b89b5b12fa0eeefaf14e5051e48
`drw_manager_init` must be called after `drw_context_state_init` as
`DST.draw_ctx.sh_cfg` (indicating when the view is clipped) must be set
first.
Differential Revision: https://developer.blender.org/D13795
`DEG_OBJECT_ITER_FOR_RENDER_ENGINE_BEGIN` creates temporary objects that
correspond to duplicates or instances.
These temporary objects can share same pointers with the original object
as in the case of Bounding Box.
Bound Box of temporary objects is marked dirty in
`BKE_object_replace_data_on_shallow_copy` since `ob->data` is different.
This causes the original Bounding Box, calculated for the evaluated
geometry, to be lost.
The solution in this commit is to change the boundbox reference of the
temporary objects, so the boundbox of the non-temporary object (with
the data curve) is not marked dirty.
Differential Revision: https://developer.blender.org/D13581
The `lines_adjacency` IBO build in the GPU subdivision case was missing
edges at the boundaries of open meshes. As it is used for the shadow
pass, the shadows were then not clipped properly.
This would also make X-Ray mode render differently in those cases.
To fix this, we can simply reuse the buffer finalization routine from the
non-subdivision case, as such edges are handled there.
When weight painting the bone overlay is extremely intrusive,
effectively requiring either extensive use of hiding individual
bones, or disabling the whole bone overlay between selections.
This addresses the issue by adding a bone opacity slider that
is used for the 'wireframe' armature drawing mode. It directly
controls the uniform opacity as a straightforward option.
Differential Revision: https://developer.blender.org/D11804
This evaluator is used in order to evaluate subdivision at render time, allowing for
faster renders of meshes with a subdivision surface modifier placed at the last
position in the modifier list.
When evaluating the subsurf modifier, we detect whether we can delegate evaluation
to the draw code. If so, the subdivision is first evaluated on the GPU using our own
custom evaluator (only the coarse data needs to be initially sent to the GPU), then,
buffers for the final `MeshBufferCache` are filled on the GPU using a set of
compute shaders. However, some buffers are still filled on the CPU side, if doing so
on the GPU is impractical (e.g. the line adjacency buffer used for x-ray, whose
logic is hardly GPU compatible).
This is done at the mesh buffer extraction level so that the result can be readily used
in the various OpenGL engines, without having to write custom geometry or tesselation
shaders.
We use our own subdivision evaluation shaders, instead of OpenSubDiv's vanilla one, in
order to control the data layout, and interpolation. For example, we store vertex colors
as compressed 16-bit integers, while OpenSubDiv's default evaluator only work for float
types.
In order to still access the modified geometry on the CPU side, for use in modifiers
or transform operators, a dedicated wrapper type is added `MESH_WRAPPER_TYPE_SUBD`.
Subdivision will be lazily evaluated via `BKE_object_get_evaluated_mesh` which will
create such a wrapper if possible. If the final subdivision surface is not needed on
the CPU side, `BKE_object_get_evaluated_mesh_no_subsurf` should be used.
Enabling or disabling GPU subdivision can be done through the user preferences (under
Viewport -> Subdivision).
See patch description for benchmarks.
Reviewed By: campbellbarton, jbakker, fclem, brecht, #eevee_viewport
Differential Revision: https://developer.blender.org/D12406
The issue was caused by rB7e712b2d6a0d257d272ed35622b41d06274af8df
and the fact that `GPUMaterialTexture` contains an `ImageUser *` which
references the `ImageUser` on e.g. `NodeTexImage`.
Since the node tree update refactor, it is possible that the node tree changes
without changing the actual material. Therefore, either the renderer should
check if the node tree has changed or it should not store pointers to data in
node storage. The latter approach is implemented in this patch.
Differential Revision: https://developer.blender.org/D13663
This was visible outside of camera view and was not respecting the
"Depth of Field" checkbox on the Camera properties.
Now return early if DoF should not be visible.
Maniphest Tasks: T94262
Differential Revision: https://developer.blender.org/D13631
Issue is that external engine uses the gpu info. but overwrote the
instance data. The draw manager would then detect instance data and
required the engine type to have a instance free callback.
The solution is to save some space in the engine data to hold an empty
and unused instance_data attribute to comply with `ViewportEngineData`
struct.
In the original design draw engines had to copy with a limitation that
they were not allowed to reuse complex data structures between drawing
calls. Data that could be reused were limited to:
- GPUFramebuffers
- GPUTextures
- Memory that could be removed calling MEM_freeN (storage list)
- DRWPass
This is fine when the storage list contains arrays or structs but when
more complex data types (vectors, maps) etc wasn't possible.
This patch adds instance_data that can be reused between drawing calls.
The instance_data is controlled by the draw engine and doesn't need to
be limited as described above.
When an engines stores instance_data it must implement the
`DrawEngineType.instance_free` callback to free the data.
The patch originates from eevee rewrite. But was added to master as the
image engine rewrite also has a need for it.
Reviewed By: fclem
Differential Revision: https://developer.blender.org/D13425
Wire-only meshes have a special case in the overlay drawing to give
the wire shader a special color (which avoids the lines being dashed,
somehow). The fast path for duplis didn't have that special case.
Differential Revision: https://developer.blender.org/D13196
The problem is that drw_batch_cache_generate_requested_delayed
is called on the object, which uses the original object data type to
choose which data type to get info for. So for curves and text it uses
the incorrect type (not the evaluated mesh like we hardcoded in the
armature overlay code).
To fix this I hardcoded the "delayed" generation to only use the
evaluated mesh. Luckily it wasn't use elsewhere besides this
armature overlay system. That seems like the simplest fix for
3.0. A proper solution should rewrite this whole area anyway.
Differential Revision: https://developer.blender.org/D13439
This part of the drawing code assumes that the bone custom object
has only one evaluated geometry component, and it also uses the
object type to check which data to draw, with the functions like
`DRW_cache_object_surface_get` that just take an object input.
Those functions usually work on evaluated objects, which use the
instancing system to access a temporary object with `object.data`
replaced for data types that don't match the original object.
That assumption used to work, but now curve, point cloud, or volume
objects can have an evaluated mesh which is not accessed with the
same object for render engine drawing.
The "correct" solution for the way this code is structured would be to
loop through all of the geometry components and try to get GPU batches
from every one of them. However, that significantly increases complexity
in an area that should probably be refactored anyway. This patch treats
the mesh as a special case, and only draws the evaluated mesh.
The **best** solution in my opinion might be refactoring this area to
use the instancing system with some sort of viewport-only flag so
the custom shape instances aren't added in the render.
The solution is "partial" because the "Wireframe" option only works
for meshes from mesh objects, even after this fix, and because other
data besides meshes is not displayed at all.
Differential Revision: https://developer.blender.org/D13038
