Vulkan: Initial Compute Shaders support. #104518

Closed
Jeroen Bakker wants to merge 31 commits from Jeroen-Bakker:vulkan-compute-shaders into main

When changing the target branch, be careful to rebase the branch in your fork to match. See documentation.
Member

This patch adds initial support for compute shaders to
the vulkan backend. As the development is oriented to the test-
cases we have the implementation is limited to what is used there.

It has been validated that with this patch that the following test
cases are running as expected

  • GPUVulkanTest.gpu_shader_compute_vbo
  • GPUVulkanTest.gpu_shader_compute_ibo
  • GPUVulkanTest.gpu_shader_compute_ssbo
  • GPUVulkanTest.gpu_storage_buffer_create_update_read
  • GPUVulkanTest.gpu_shader_compute_2d

This patch includes:

  • Allocating VkBuffer on device.
  • Uploading data from CPU to VkBuffer.
  • Binding VkBuffer as SSBO to a compute shader.
  • Execute compute shader and altering VkBuffer.
  • Download the VkBuffer to CPU ram.
  • Validate that it worked.
  • Use device only vertex buffer as SSBO
  • Use device only index buffer as SSBO
  • Use device only image buffers

GHOST API has been changed as the original design was created before
we even had support for compute shaders in blender. The function
GHOST_getVulkanBackbuffer has been separated to retrieve the command
buffer without a backbuffer (GHOST_getVulkanCommandBuffer). In order
to do correct command buffer processing we needed access to the queue
owned by GHOST. This is returned as part of the GHOST_getVulkanHandles
function.

Open topics (not considered part of this patch)

  • Memory barriers & command buffer encoding
  • Indirect compute dispatching
  • Rest of the test cases
  • Data conversions when requested data format is different than on device.
  • GPUVulkanTest.gpu_shader_compute_1d is supported on AMD devices.
    NVIDIA doesn't seem to support 1d textures.
This patch adds initial support for compute shaders to the vulkan backend. As the development is oriented to the test- cases we have the implementation is limited to what is used there. It has been validated that with this patch that the following test cases are running as expected - `GPUVulkanTest.gpu_shader_compute_vbo` - `GPUVulkanTest.gpu_shader_compute_ibo` - `GPUVulkanTest.gpu_shader_compute_ssbo` - `GPUVulkanTest.gpu_storage_buffer_create_update_read` - `GPUVulkanTest.gpu_shader_compute_2d` This patch includes: - Allocating VkBuffer on device. - Uploading data from CPU to VkBuffer. - Binding VkBuffer as SSBO to a compute shader. - Execute compute shader and altering VkBuffer. - Download the VkBuffer to CPU ram. - Validate that it worked. - Use device only vertex buffer as SSBO - Use device only index buffer as SSBO - Use device only image buffers GHOST API has been changed as the original design was created before we even had support for compute shaders in blender. The function `GHOST_getVulkanBackbuffer` has been separated to retrieve the command buffer without a backbuffer (`GHOST_getVulkanCommandBuffer`). In order to do correct command buffer processing we needed access to the queue owned by GHOST. This is returned as part of the `GHOST_getVulkanHandles` function. Open topics (not considered part of this patch) - Memory barriers & command buffer encoding - Indirect compute dispatching - Rest of the test cases - Data conversions when requested data format is different than on device. - GPUVulkanTest.gpu_shader_compute_1d is supported on AMD devices. NVIDIA doesn't seem to support 1d textures.
Jeroen Bakker added this to the EEVEE & Viewport project 2023-02-09 15:18:01 +01:00
Jeroen Bakker requested review from Clément Foucault 2023-02-09 15:18:11 +01:00
Jeroen Bakker force-pushed vulkan-compute-shaders from 583871c678 to c1b85103fe 2023-02-13 09:08:08 +01:00 Compare
Jeroen Bakker added 216 commits 2023-02-13 09:14:53 +01:00
c2c6707919 Fix T102690: Object can be animated with a single keyframe
Remove some assumptions that an FCurve with a single keyframe always is
extrapolated in constant fashion. There's no reason for the Beziér handles
to be ignored in such a case.

FCurve evaluation already worked properly, it was just the drawing in the
graph editor and the selectability of the handles that needed adjustments.
fe5d54d3d0 Gizmo: add new cage2d draw style for circular shapes
`ED_GIZMO_CAGE2D_STYLE_CIRCLE` now draw circles. The previous `ED_GIZMO_CAGE2D_STYLE_CIRCLE`, which drew rectangles, is renamed to `ED_GIZMO_CAGE2D_STYLE_RECTANGLE`. The meaning of `ED_GIZMO_CAGE2D_STYLE_BOX` is now unclear and probably needs to be renamed too.
Ref T104280

Maniphest Tasks: T104280

Differential Revision: https://developer.blender.org/D17174
04aab7d516 Animation: Add "Select Linked Vertices" to Weight Paint Mode
Adds two operators to select linked  vertices in weight paint mode.
Similar to how it works in edit mode.
Press "L" to select vertices under the cursor,
or CTRL + "L" to select anything linked to the current selection.

Reviewed by: Sybren A. Stüvel, Hans Goudey, Marion Stalke
Differential Revision: https://developer.blender.org/D16848
Ref: D16848
2a19810f97 Fix T104296: crash caused by incorrectly initialized group nodes
Versioning code in `do_versions_after_linking_260` inserted new group input
and output nodes. And (reasonably?) expected sockets to exist on those nodes.
However, `nodeAddStaticNode` did not initialize sockets on nodes with that use
`declare_dynamic` yet. This patch changes it so that `declare_dynamic` is used
in more places, which caused issues during file loading when node groups are
updated in somewhat arbitrary order (not in an order that is based on which
groups use which).

Differential Revision: https://developer.blender.org/D17183
7208938707 Fix T104278: incorrect handling of unavailable linked node groups
The declaration of group nodes using unavailable linked groups contains
a `skip_updating_sockets` tag, which indicates that the node shouldn't
change. This information was not used properly further down the line.
fc2a64e21a Win32: Better Error Handling in BLI_file_attributes
After Win32 API call GetFileAttributesW, check for
INVALID_FILE_ATTRIBUTES, which is returned on error,
usually if file not found.

See D17176 for more details.

Differential Revision: https://developer.blender.org/D17176

Reviewed by Ray Molenkamp
fdc81be213 Fix T104223: attribute not propagated to Set Curve Radius node
This declaration change tells the evaluation system that the radius
field is evaluated on the input geometry. Which in turn means that
attributes referenced by the radius field should be propagated to
the geometry.
ca99a59605 Fix T104261: crash when trying to draw viewer overlay for empty curve
`BKE_displist_make_curveTypes` only sets `curve_eval` if the final geometry
of the object actually contains curve data, otherwise it's null.
52de84b0db BLI: Use BLI_math_matrix_type.hh instead of BLI_math_float4x4.hh
Straightforward port. I took the oportunity to remove some C vector
functions (ex: `copy_v2_v2`).

This makes some changes to DRWView to accomodate the alignement
requirements of the float4x4 type.
b5e00a1482 Revert "BLI: Use BLI_math_matrix_type.hh instead of BLI_math_float4x4.hh"
This reverts commit 52de84b0db.

had some build issues on windows i can't quickly resolve, revert for
now while we fix the problems
023277359f Fix T104053: Limit pbvh recursion
The recursion depth was checked for equality with a maximum depth,
allowing leaves with more primitives if a certain depth was reached.

However, a single leaf must always use the same material (including
set_smooth), so if a leaf contained multiple materials it was split
anyway. This meant that in the next recursion step the depth was
larger than the cutoff value and it would go back to recursing until
the number of primitives was small enough, ignoring the recursion
depth for the rest of the process.

In certain edge cases this could lead to a stack overflow.

Even with the check changed from 'equality' to 'larger or equal'
this could still fail in the pathological case where every primitive
has another material. But that can't be helped, and it wouldn't
realistically happen either.

Differential Revsision: https://developer.blender.org/D17188
27e2b32a06 Cleanup: Remove redundant calls to init grids
`BKE_volume_init_grids` gets called in `volume_init_data` that is run
on creating new datablocks so it's unnecessary to run it separately.
c468aeefb5 PyAPI: updates to bl_ui.generic_ui_list which didn't follow conventions
- Replace type annotations with doc-strings, the current conventions is
  not to use type annotations in startup scripts.
- Replace abbreviation "idx" with "index" in public arguments/properties.
- Replace `len(..) > 0` with boolean checks.
- Add `__all__` to list public members.
- Use `arg` instead of `param` for doc-strings.
- Locate the doc-string so it shows as `__doc__`.
9565ea0724 IO: Harmonize UI for selection of axes in OBJ and Collada
Implements T103858: in OBJ importer and exporter, and in Collada
exporter, present axis choices as a dropdown instead of inline button
row.

Reviewed By: Bastien Montagne
Differential Revision: https://developer.blender.org/D17186
a45a881534 Spelling: `familly` -> `family`.
Fix spelling mistake originated from tmp-vulkan branch.
ab3e1809b6 Vulkan: Select graphics queue with compute capabilities.
There might be cases that the graphics queue and compute
queue are separated, but that would be something that we
will solve in the future.
d335db09e0 Fix T90893: Restoring a key in the keymap editor doesn't work if the key is only disabled
Reset `KMI_INACTIVE` flag when restore-item  button is called

Reviewed By: campbellbarton

Differential Revision: https://developer.blender.org/D16773
0f51b5b599 Animation: Add Operator for adding FCurve modifiers to more menus
The operator has the option to add to selected FCurves instead
of only the active, but it was only exposed in the redo panel.
This patch adds the operator to the right-click menu on FCurve channels,
and to the channel menu in the Graph editor.
Both times with configured to add to selected
instead of only the active FCurve

Revied by Reviewed by: Sybren A. Stüvel
Differential Revision: https://developer.blender.org/D17066
Ref: D17066
af0d378177 Cleanup: Move multires reshape header to C++
For continued refactoring of the Mesh data structure. See T103343.
19b63b932d Fix transform gizmo not updating according to state
Whenever a transform operation is activated by gizmo, the gizmo
modal is maintained, but its drawing remains the same even if the
transform mode or constrain is changed.

So update the gizmo according to the mode or constrain set.

NOTE: Currently only 3D view gizmo is affected
ebe8f8ce71 BMesh: Parallelize BMesh to evaluated Mesh conversion
Currently this conversion (which happens when using modifiers in edit
mode, for example) is completely single threaded. It's harder than some
other areas to multithread because BMesh elements don't always know
their indices (and vise versa), and because the dynamic AoS format
used by BMesh makes some typical solutions not helpful.

This patch proposes to split the operation into two steps. The first
updates the indices of BMesh elements and builds tables for easy
iteration later. It also checks if some optional mesh attributes
should be added. The second uses parallel loops over all elements,
copying attribute values and building the Mesh topology.

Both steps process different domains in separate threads (though the
first has to combine faces and loops). Though this isn't proper data
parallelism, it's quite helpful because each domain doesn't affect the
others.

**Timings**
I tested this on a Ryzen 7950x with a 1 million face grid, with no
extra attributes and then with several color attributes and vertex
groups.

| File | Before | After |
| Simple | 101.6 ms | 59.6 ms |
| More Attributes | 149.2 ms | 65.6 ms |

The optimization scales better with more attributes on the BMesh. The
speedup isn't as linear as multithreading other operations, indicating
added overhead. I think this is worth it though, because the user is
usually actively interacting with a mesh in edit mode.

See the differential revision for more timing information.

Differential Revision: https://developer.blender.org/D16249
d204830107 UI: Make uiBut safe for non-trivial construction
No user-visible changes expected.

Essentially, this makes it possible to use C++ types like `std::function`
inside `uiBut`. This has plenty of benefits, for example this should help
significantly reducing unsafe `void *` use (since a `std::function` can hold
arbitrary data while preserving types).

----

I wanted to use a non-trivially-constructible C++ type (`std::function`) inside
`uiBut`. But this would mean we can't use `MEM_cnew()` like allocation anymore.

Rather than writing worse code, allow non-trivial construction for `uiBut`.
Member-initializing all members is annoying since there are so many, but rather
safe than sorry. As we use more C++ types (e.g. convert callbacks to use
`std::function`), this should become less since they initialize properly on
default construction.

Also use proper C++ inheritance for `uiBut` subtypes, the old way to allocate
based on size isn't working anymore.

Differential Revision: https://developer.blender.org/D17164

Reviewed by: Hans Goudey
fcc1166821 GPU: Disable verbose GLSL variable names in debug builds
GpuInput::node can be deallocated in some cases. (See T104265)
This is a temp workaround until a proper solution is implemented.
dc79281223 Nodes: Add modal keymap for Node link drag
This will add a proper modal keymap for the node link drag operator.
It allows the user to customize the keys used to start drag and so on.
Also it gets rid of the custom status bar message.

Differential Revision: https://developer.blender.org/D17190
7f958217ad Curves: Use shared caches for evaluated data
Use the `SharedCache` concept introduced in D16204 to share lazily
calculated evaluated data between original and multiple evaluated
curves data-blocks. Combined with D14139, this should basically remove
most costs associated with copying large curves data-blocks (though
they add slightly higher constant overhead). The caches should
interact well with undo steps, limiting recalculations on undo/redo.

Options for avoiding the new overhead associated with the shared
caches are described in T104327 and can be addressed separately.

Simple situations affected by this change are using any of the following data
on an evaluated curves data-block without first invalidating it:
- Evaluated offsets (size of evaluated curves)
- Evaluated positions
- Evaluated tangents
- Evaluated normals
- Evaluated lengths (spline parameter node)
- Internal Bezier and NURBS caches

In a test with 4m points and 170k curves, using curve normals in a
procedural setup that didn't change positions procedurally gave 5x
faster playback speeds. Avoiding recalculating the offsets on every
update saved about 3 ms for every sculpt update for brushes that don't
change topology.

Differential Revision: https://developer.blender.org/D17134
6590a288b0 Fix number sliders not working
Own mistake in d204830107.

For some buttons the type is changed after construction, which means the button
has to be reconstructed. For example number buttons can be turned into number
slider buttons this way. New code was unintentionally overriding the button
type after reconstruction with the old type again.
b454416927 Cycles: add non-uniform scaling to spot light size
Cycles ignores the size of spot lights, therefore the illuminated area doesn't match the gizmo. This patch resolves this discrepancy.
| Before (Cycles) | After (Cycles) | Eevee
|{F14200605}|{F14200595}|{F14200600}|
This is done by scaling the ray direction by the size of the cone. The implementation of `spot_light_attenuation()` in `spot.h` matches `spot_attenuation()` in `lights_lib.glsl`.
**Test file**:
{F14200728}

Differential Revision: https://developer.blender.org/D17129
0220bdc2d5 Cycles: Tests: Add option to increase SPP for manual comparisons
Useful for validating changes when sampling/noise changes:
- First run with BLENDER_TEST_UPDATE=1 and e.g. CYCLESTEST_SPP_MULTIPLIER=32
- Apply your change
- Run with only CYCLESTEST_SPP_MULTIPLIER=32
- Compare
- Reset the SVN repo

Differential Revision: https://developer.blender.org/D17107
73000c792d Cycles: Reorganize Fresnel handling in Microfacet closures
This is both a cleanup and a preparation for the Principled v2 changes.
Notable changes:
- Clearcoat weight is now folded into the closure weight, there's no reason
  to track this separately.
- There's a general-purpose helper for computing a Closure's albedo, which is
  currently used by the denoising albedo and diffuse/gloss/transmission color
  passes.
- The d/g/t color passes didn't account for closure albedo before, this means
  that e.g. metallic shaders with Principled v2 now have their color texture
  included in the glossy color pass. Also fixes T104041 (sheen albedo).
- Instead of precomputing and storing the albedo during shader setup, compute
  it when needed. This is technically redundant since we still need to compute
  it on shader setup to adjust the sample weight, but the operation is cheap
  enough that freeing up the storage seems worth it.
- Future changes (Principled v2) are easier to integrate since the Fresnel
  handling isn't all over the place anymore.
- Fresnel handling in the Multiscattering GGX code is still ugly, but since
  removing that entirely is the next step, putting effort into cleaning it up
  doesn't seem worth it.
- Apart from the d/g/t color passes, no changes to render results are expected.

Differential Revision: https://developer.blender.org/D17101
90f36fc50e Fix (unreported): snap to object origin not respecting clipping planes
There was an incorrect conversion for local clip planes although the
coordinates used are in world positions.
ab9c34e7e7 Fix T104316: Width of headerless panels on regions with overlap
In some cases panels without headers were drawn too wide in sidebars
with region overlap.

Instead of always using the maximum width the view allows, headerless
panels now also use the width calculated in
`panel_draw_width_from_max_width_get`. The function already returns the
correct width in all cases and also takes care of insetting the panels,
when their backdrop needs to be drawn, which is necessary for headerless
panels, too, when there is region overlap.

Reviewed By: Hans Goudey

Differential Revision: http://developer.blender.org/D17194
789e549dba Geometry Nodes: Tweak menu location of sample nodes
There was an inconsistency between geometry sample nodes and mesh/curve
sample nodes, where the latter didn't have a special "Sample" category,
and we categorized as "Operations", which they were not. Also put the
sample category between "Read" and "Write" since the verb name is
more consistent and sampling is an advanced form of reading.
06305e5ca8 Cleanup/refactor: split Edge and Vert Slide code into more specific functions
This makes the code more readable.

In this commit also the `int curr_side_unclamp` member was moved to
`EdgeSlideParams` as it is a common value for all "Containers".
62fc001979 Cleanup: silence warning
```
warning C4457: declaration of 'fac' hides function parameter
message : see declaration of 'fac'
```
1a13940ef8 Fix T88617: Wrong annotation cursor in Preview of sequencer editor
Allow preview region to change cursor as per the selected tool

Reviewed by: campbellbarton, ISS

Differential Revision: https://developer.blender.org/D16878
b2196ebe28 Win32: Relax Debug Assert in BLI_file_attributes
If GetFileAttributesW returns an error, only debug assert if the reason
is file not found.

See D17204 for more details.

Differential Revision: https://developer.blender.org/D17204

Reviewed by Ray Molenkamp
d6cd7d1138 WM: correct the return flag from wm_handler_fileselect_do
In the unlikely case an area could not be created OPERATOR_CANCELLED
was returned, this has the same value of WM_HANDLER_HANDLED however
break is logical in this situation and both flags work.
3d6ceb737d Geometry Nodes: parallelize part of Duplicate Elements node
This implements two optimizations:
* If the duplication count is constant, the offsets array can be
  filled directly in parallel.
* Otherwise, extracting the counts from the virtual array is parallelized.
  But there is still a serial loop over all elements in the end to compute
  the offsets.
501352ef05 Cleanup: Move PBVH files to C++
For continued refactoring of the Mesh data structure. See T103343.
b642dc7bc7 Fix: Incorrect forward-compatible saving of face sets
There were two errors with the function used to convert face sets
to the legacy mesh format for keeping forward compatibility:
- It was moved before `CustomData_blend_write_prepare` so it
  operated on an empty span.
- It modified the mesh when it's only supposed to change the copy
  of the layers written to the file.

Differential Revision: https://developer.blender.org/D17210
d3949a4fdb Fix GHOST/Wayland thread-unsafe key-repeat timer checks
Resolve a thread safety issue reported by valgrind's helgrind checker,
although I wasn't able to redo the error in practice.

NULL check on the key-repeat timer also needs to lock, otherwise it's
possible the timer is set in another thread before the lock is acquired.

Now all key-repeat timer access which may run from a thread
locks the timer mutex before any checks or timer manipulation.
7de1a4d1d8 Fix GHOST/Wayland thread-unsafe timer-manager manipulation
Mutex locks for manipulating GHOST_System::m_timerManager from
GHOST_SystemWayland relied on WAYLAND being the only user of the
timer-manager.

This isn't the case as timers are fired from
`GHOST_System::dispatchEvents`.

Resolve by using a separate timer-manager for wayland key-repeat timers.
329eeacc66 Cleanup: Cycles: Remove isotropic microfacet closure setup functions
Turns out these are 100% redundant, so get rid of them.
6d297c35c8 Fix T104371: GPencil merge down layer duplicates wrong frame
The merge down operator was sometimes copying the wrong frame, which altered the animation.
While merging the layers, it is sometimes needed to duplicate a keyframe,
when the lowest layer does not have a keyframe but the highest layer does.
Instead of duplicating the previous keyframe of the lowest layer, the code
was actually duplicating the active frame of the layer which was the current frame in the timeline.

This patch fixes the issue by setting the previous keyframe of the layer as its active frame before duplication.

Related issue: T104371.

Differential Revision: https://developer.blender.org/D17214
0a3df611e7 Fix T103393: Cycles: Undefine __LIGHT_TREE__ on Metal/AMD to fix perf
This patch fixes T103393 by undefining `__LIGHT_TREE__` on Metal/AMD as it has an unexpected & major impact on performance even when light trees are not in use.

Patch authored by Prakash Kamliya.

Reviewed By: brecht

Maniphest Tasks: T103393

Differential Revision: https://developer.blender.org/D17167
be0912a402 Cycles: Prevent use of both AMD and Intel Metal devices at same time
This patch removes the option to select both AMD and Intel GPUs on system that have both. Currently both devices will be selected by default which results in crashes and other poorly understood behaviour. This patch adds precedence for using any discrete AMD GPU over an integrated Intel one. This can be overridden with CYCLES_METAL_FORCE_INTEL.

Reviewed By: brecht

Differential Revision: https://developer.blender.org/D17166
654e1e901b Cycles: Use local atomics for faster shader sorting (enabled on Metal)
This patch adds two new kernels: SORT_BUCKET_PASS and SORT_WRITE_PASS. These replace PREFIX_SUM and SORTED_PATHS_ARRAY on supported devices (currently implemented on Metal, but will be trivial to enable on the other backends). The new kernels exploit sort partitioning (see D15331) by sorting each partition separately using local atomics. This can give an overall render speedup of 2-3% depending on architecture. As before, we fall back to the original non-partitioned sorting when the shader count is "too high".

Reviewed By: brecht

Differential Revision: https://developer.blender.org/D16909
7beb487e9a Fix T104353: Crash on opening sculpting template
`t->region` was `NULL`.

It can happen depending on the context.

Caused by rB19b63b932d2b.
4bd3b02984 Python: Suppress BGL deprecation messages after 100 times.
BGL deprecation calls used to be reported on each use. As bgl calls
are typically part of a handler that is triggered at refresh this
could lead to overflow of messages and slowing down systems when
the terminal/console had to be refreshed as well.

This patch only reports the first 100 bgl deprecation calls. This
gives enough feedback to the developer that changes needs to be made
. But still provides good responsiveness to users when they have
such add-on enabled. Only the first frames can have a slowdown.
f2538c7173 Fix T104335: MNEE + OptiX OSL results in illegal address error
The OptiX pipeline created for OSL was missing sufficient continuation
stack to handle the MNEE ray generation program.
deaddbdcff Fix forced snap status being removed when changing transform mode
The `SNAP_FORCED` setting is set to the operation and not the snap
status.

Therefore, this option should not be cleared along with the other
statuses when resetting snapping.

Move then the location of this setting to `TransInfo::modifiers`.
cc623ee7b0 Transform: do not save settings when canceling the operation
If we are canceling, the settings must remain the same as before we
start the operation.
8adebaeb7c Modifiers: measure execution time and provide Python access
The goal is to give technical artists the ability to optimize modifier usage
and/or geometry node groups for performance. In the long term, it
would be useful if Blender could provide its own UI to display profiling
information to users. However, right now, there are too many open
design questions making it infeasible to tackle this in the short term.

This commit uses a simpler approach: Instead of adding new ui for
profiling data, it exposes the execution-time of modifiers in the Python
API. This allows technical artists to access the information and to build
their own UI to display the relevant information. In the long term this
will hopefully also help us to integrate a native ui for this in Blender
by observing how users use this information.

Note: The execution time of a modifier highly depends on what other
things the CPU is doing at the same time. For example, in many more
complex files, many objects and therefore modifiers are evaluated at
the same time by multiple threads which makes the measurement
much less reliable. For best results, make sure that only one object
is evaluated at a time (e.g. by changing it in isolation) and that no
other process on the system keeps the CPU busy.

As shown below, the execution time has to be accessed on the
evaluated object, not the original object.

```lang=python
import bpy
depsgraph = bpy.context.view_layer.depsgraph
ob = bpy.context.active_object
ob_eval = ob.evaluated_get(depsgraph)
modifier_eval = ob_eval.modifiers[0]
print(modifier_eval.execution_time, "s")
```

Differential Revision: https://developer.blender.org/D17185
39e0bbfa55 BKE libquery: Add option to also process `ID.lib` pointer.
This was never needed before, but upcomming work for Brush Asset project
requires it.
9c14039a8f BLI ListBase: Add new 'SplitAfter' given link util.
Allows to easily split a ListBase by moving the part after given link
into a new ListBase.
81f8d74f6d EEVEE: Cleanup: light power / radiance code and document it
All thanks to @weizhen for spoting the inconsistencies.
c7b601c79e Cleanup: Subdiv: Use index instead of pointer
The edge pointer was retrieved from the index and then just
used to calculate the index again. Remove the edge pointer
and only use the index.
85b2bce037 BKE libremap: do not assert on identity pairs.
If identity pairs (i.e. old ID pointer being same as new one) was
forbidden, then this should be asserted against in code defining
remapping, not in code applying it.

But it is actually sometimes usefull to allow/use identity pairs, so
simply early-return on these instead of asserting.
288b13b252 BKE lib remapper: Add new util to overwrite an existing mapping.
Current `BKE_id_remapper_add` would not replace an already existing
mapping rule, now `BKE_id_remapper_add_overwrite` allows that behavior
if necessary.
a3551ed878 BKE main namemap: Add a way to clear all namemaps of a given Main.
Existing `BKE_main_namemap_destroy` is too specific when a entire Main
needs to have its namemaps cleared, since it would not handle the
Library ones.

While in regular situation current code is fine, ID management code that
may also edit linked data needs a wider, simpler clearing tool.
430cc9d7bf Fix T104381: Assert on Circle Select end modal
`em_setup_vivewcontext` cannot be used in this function now as it
expects `obedit` to be a mesh. It also duplicated the viewcontext init.
Instead `BKE_editmesh_from_object` is called only when type is a mesh.
2d994de77c Cycles: MetalRT optimisation for subsurface intersection queries
This patch optimises subsurface intersection queries on MetalRT. Currently intersect_local traverses from the scene root, retrospectively discarding all non-local hits. Using a lookup of bottom level acceleration structures, we can explicitly query only the relevant instance. On M1 Max, with MetalRT selected, this can give a render speedup of 15-20% for scenes like Monster which make heavy use of subsurface scattering.

Patch authored by Marco Giordano.

Reviewed By: brecht

Differential Revision: https://developer.blender.org/D17153
75db4c082b Cleanup: Use BitVector instead of BLI_bitmap for subsurf face dot tags
For better type safety and more automatic memory management.
a38d99e0b2 Fix (unreported): Transform gizmo not restoring when changing mode
When activating a rotation with the Transform gizmo for example, some
gizmos are hidden but they don't reappear when changing the mode.

Make sure the gizmos corresponding to the mode always reappear.
d20f992322 Mesh: Avoid copying positions in object mode modifier stack
Remove the use of a separate contiguous positions array now that
they are stored that way in the first place. This allows removing the
complexity of tracking whether it is allocated and deformed in the
mesh modifier stack.

Instead of deferring the creation of the final mesh until after the
positions have been copied and deformed, create the final mesh
first and then deform its positions.

Since vertex and face normals are calculated lazily, we can rely on
individual modifiers to calculate them as necessary and simplify
the modifier stack. This was hard to change before because of the
separate array of deformed positions.

Differential Revision: https://developer.blender.org/D16971
8be3fcab7e Fix tests after recent commit.
`9c14039a8f4b5f` broke blenlib tests in release builds, due to how
`EXPECT_BLI_ASSERT` works (in release builds it just calls the given
function, so if that crashes then the test fails).

For now remove that check in the test.
b0b9e746fa BLI: Use BLI_math_matrix_type.hh instead of BLI_math_float4x4.hh
Straightforward port. I took the oportunity to remove some C vector
functions (ex: copy_v2_v2).

This makes some changes to DRWView to accomodate the alignement
requirements of the float4x4 type.
6dcfb6df9c Cycles: Abstract host memory fallback for GPU devices
Host memory fallback in CUDA and HIP devices is almost identical.
We remove duplicated code and create a shared generic version that
other devices (oneAPI) will be able to use.

Reviewed By: brecht

Differential Revision: https://developer.blender.org/D17173
d3500c482f Cleanup: Move DRW_pbvh.h header to C++
For continued refactoring of the Mesh data structure. See T103343.
f152159101 Metal: Guard advanced command buffer debugging behind OS version flag.
Authored by Apple: Michael Parkin-White

Ref T96261

Reviewed By: fclem

Maniphest Tasks: T96261

Differential Revision: https://developer.blender.org/D17181
8703db393b Metal: Ensure explicit return after discard to eliminate differences in behaviour between GPUs.
Discard is not always treated as an explicit return and flow control can continue for required derivative calculations. This behaviour is different in Metal vs OpenGL. Adding return after discards ensures consistency in expectation as behaviour is well-defined.

Authored by Apple: Michael Parkin-White

Ref T96261

Reviewed By: fclem

Maniphest Tasks: T96261

Differential Revision: https://developer.blender.org/D17199
af5706c960 Docs: improve doc-string for WM_operator_flag_only_pass_through_on_press
The doc-string didn't provide any context for how the funciton is
intended to be used.
e27c89c7c7 Docs: added missing documentation for `WindowManager` methods
Added missing documentation for `draw_cursor_add` and
`draw_cursor_remove` methods for `WindowManager`.

Differential Revision: https://developer.blender.org/D14860
dbca0cc9d5 Fix crash on exit under Wayland
Order of free error from [0] caused the timer manager
to be freed before the timer.

[0]: 7de1a4d1d8
622cad7073 Cleanup: minor tweak to recent fix for T10438
Minor change to [0], prefer calling em_setup_viewcontext,
even though there is no functional difference at the moment,
if this function ever performs additional operations than assigning
`ViewContext.em`, it would have to be manually in-lined in
`view3d_circle_select_recalc`.

[0]: 430cc9d7bf
7e8153b07d Keymap: support default shortcut to toggle overlays in all space-types
UV Editor, Image Editor & Sequencer didn't have a shortcut for toggling
overlays. Use the same shortcut as the 3D viewport.

Ref D16959
2609ca2b8e Cleanup: tweaks to cycles/metal preferences
- Auto-format.
- Use raw string for regex.
- Remove redundant assignment.
- Remove duplicate arm64 check.
- Break early out of loop.
buildbot/vdev-code-daily-coordinator Build done. Details
349350b304
Fix T104390: Regression: Object selection in viewport is not working
Caused by alignment difference between C and C++. Asan caught the issue
on startup.

Removing the unused view matrix storage copy avoids this problem.
buildbot/vdev-code-daily-coordinator Build done. Details
8d9d16fb53
Fix #104396: Blender crashes when moving Keyframes in Graph Editor
`t->region->gizmo_map` can be `nullptr`.

Caused by 19b63b932d
buildbot/vdev-code-daily-coordinator Build done. Details
f01bf82480
Curves: Add select pick operator
This adds the `select_pick` function for to `Curves` objects.
It is used in the common `view3d_select` operator.

Pull Request #104406
buildbot/vdev-code-daily-coordinator Build done. Details
f5552d759c
Fix compiler error
buildbot/vdev-code-daily-coordinator Build done. Details
41ddd3d732
Fix: Experimental Panel links modified for Gitea
Modifies the links to point to the new developer site.

Pull Request #104425
buildbot/vdev-code-daily-coordinator Build done. Details
e817cff009
Release: support generating LTS release notes from Gitea
Now a single script to generate both links and release notes. It also includes
the issue ID for the LTS releases, so only the release version needs to be
specified.

Pull Request #104402
53b057aa09 Cleanup: Move 18 sculpt files to C++
To allow further mesh data structure refactoring. See #103343

Pull Request #104436
buildbot/vdev-code-daily-coordinator Build done. Details
5c994d7846
Fix #104297: Cycling geometry nodes viewer ignores sockets
Sockets after the geometry socket were ignored when cycling through
the node's output sockets. If there are multiple geometry sockets, the
behavior could still be refined probably, but this should at least make
basic non-geometry socket cycling work.
buildbot/vdev-code-daily-coordinator Build done. Details
6aa1b5d031
Cleanup: format
09eb4fe19a Fix #103913: Triangulate sometimes creates degenerate triangles
The ear clipping method used by polyfill_2d only excluded concave ears
which meant ears exactly co-linear edges created zero area triangles
even when convex ears are available.

While polyfill_2d prioritizes performance over *pretty* results,
there is no need to pick degenerate triangles with other candidates
are available. As noted in code-comments, callers that require higher
quality tessellation should use BLI_polyfill_beautify.
buildbot/vdev-code-daily-coordinator Build done. Details
4d3bfb3f41
Subdivision Surface: fix a serious performance hit when mixing CPU & GPU.
Subdivision surface efficiency relies on caching pre-computed topology
data for evaluation between frames. However, while eed45d2a23
introduced a second GPU subdiv evaluator type, it still only kept
one slot for caching this runtime data per mesh.

The result is that if the mesh is also needed on CPU, for instance
due to a modifier on a different object (e.g. shrinkwrap), the two
evaluators are used at the same time and fight over the single slot.
This causes the topology data to be discarded and recomputed twice
per frame.

Since avoiding duplicate evaluation is a complex task, this fix
simply adds a second separate cache slot for the GPU data, so that
the cost is simply running subdivision twice, not recomputing topology
twice.

To help diagnostics, I also add a message to show when GPU evaluation
is actually used to the modifier panel. Two frame counters are used
to suppress flicker in the UI panel.

Differential Revision: https://developer.blender.org/D17117

Pull Request #104441
buildbot/vdev-code-daily-coordinator Build done. Details
4ed8a360e9
Fix references to the main branch in the .gitmodules
buildbot/vdev-code-daily-coordinator Build done. Details
aab707ab70
Un-ignore modules in .gitmodules configuration
The meaning of the ignore option for submodules did change since our
initial Git setup was done: back then it was affecting both diff and
stage families of Git command. Unfortunately, the actual behavior did
violate what documentation was stating (the documentation was stating
that the option only affects diff family of commands). This got fixed
in Git some time after our initial setup and it was the behavior of the
commands changed, not the documentation. This lead to a situation when
we can no longer see that submodules are modified and staged, and it is
very easy to stage the submodules.

For the clarity: diff and status are both "status" family, show and
diff are "diff" family.

Hence this change: since there is no built-in zero-configuration way
of forbidding Git from staging submodules lets make it visible and
clear what the state of submodules is.

We still need to inform people to not stage submodules, for which
we can offer some configuration tips and scripts but doing so is
outside of the scope of this change at it requires some additional
research. Current goal is simple: make it visible and clear what is
going to be committed to Git.

This is a response to an increased frequency of incidents when the
submodules are getting modified and committed without authors even
noticing this (which is also a bit annoying to recover from).

Differential Revision: https://developer.blender.org/D13001
buildbot/vdev-code-daily-coordinator Build done. Details
43f308f216
Make update: Ignore submodules
The previous change in the .gitmodules made it so the `make update`
rejects to do its thing because it now sees changes in the submodules
and rejected to update, thinking there are unstaged changes.

Ignore the submodule changes, bringing the old behavior closer to
what it was.
buildbot/vdev-code-daily-coordinator Build done. Details
a1282ab015
Fix Cycles debug build error after host falback changes
Introduced in dcfb6df9ce6.

Co-authored-by: Lucas Tadeu Teixeira <lucas@lucastadeu.com>

Pull Request #104454
buildbot/vdev-code-daily-coordinator Build done. Details
0ab3ac7a41
BLI: Math: Fix vector operator * with `MutableMatView`
This was caused by operator priority trying to use
`friend VecBase operator*(const VecBase &a, FactorT b)`.

Adding tests as these were not covered.
buildbot/vdev-code-daily-coordinator Build done. Details
a0f5240089
EEVEE-Next: Virtual Shadow Map initial implementation
Implements virtual shadow mapping for EEVEE-Next primary shadow solution.
This technique aims to deliver really high precision shadowing for many
lights while keeping a relatively low cost.

The technique works by splitting each shadows in tiles that are only
allocated & updated on demand by visible surfaces and volumes.
Local lights use cubemap projection with mipmap level of detail to adapt
the resolution to the receiver distance.
Sun lights use clipmap distribution or cascade distribution (depending on
which is better) for selecting the level of detail with the distance to
the camera.

Current maximum shadow precision for local light is about 1 pixel per 0.01
degrees.
For sun light, the maximum resolution is based on the camera far clip
distance which sets the most coarse clipmap.

## Limitation:
Alpha Blended surfaces might not get correct shadowing in some corner
casses. This is to be fixed in another commit.
While resolution is greatly increase, it is still finite. It is virtually
equivalent to one 8K shadow per shadow cube face and per clipmap level.
There is no filtering present for now.

## Parameters:
Shadow Pool Size: In bytes, amount of GPU memory to dedicate to the
shadow pool (is allocated per viewport).
Shadow Scaling: Scale the shadow resolution. Base resolution should
target subpixel accuracy (within the limitation of the technique).

Related to #93220
Related to #104472
buildbot/vdev-code-daily-coordinator Build done. Details
9c03a1c92f
Fix Cycles link error with debug/asan builds after recent bugfix
Pull Request #104487
buildbot/vdev-code-daily-coordinator Build done. Details
9103978952
EEVEE-Next: Shadow: Fix issue with last merge
The merge with master updated the code to use the new matrix API. This
introduce some regressions.

For sunlights make sure there is enough tilemaps in orthographic mode
to cover the depth range and fix the level offset in perspective.
buildbot/vdev-code-daily-coordinator Build done. Details
94d280fc3f
EEVEE-Next: Shadows: Add global switch
This allow to bypass all cost associated with shadow mapping.

This can be useful in certain situation, such as opening a scene on a
lower end system or just to gain performance in some situation (lookdev).
9fd71d470e PyAPI: minor change to rna_manual_reference loading
- Use bpy.utils.execfile instead of importing then deleting from
  sys.modules.
- Add a note for why keeping this cached in memory isn't necessary.

This has the advantage of not interfering with any scripts that import
`rna_manual_reference` as a module.
buildbot/vdev-code-daily-coordinator Build done. Details
0381fe7bfe
Cleanup: update username in code-comments: campbellbarton -> ideasman42
Gitea migration changed my username, update code-comments.
buildbot/vdev-code-daily-coordinator Build done. Details
3c8f7b1a64
Cleanup: Remove unused/redundant includes from BKE_curves.hh
Avoid including headers that are obviously redundant, and don't
include BLI_task.hh in the header file, since it isn't really related.
buildbot/vdev-code-daily-coordinator Build done. Details
f3d7de709f
Cycles: update Intel Graphics Compiler to 1.0.13064.7 on Linux
Linux side of 8afcecdf1f.

Reviewed by: LazyDodo, sergey, campbellbarton
Ref !104458, 16984
buildbot/vdev-code-daily-coordinator Build done. Details
7effc6ffc4
Cleanup: solve compiler warnings.
Classes were predefined as structs.
buildbot/vdev-code-daily-coordinator Build done. Details
8b35db914e
GPU: Fix assert when using light gizmo.
Blender was reporting that the GPU_TEXTURE_USAGE_HOST_READ wasn't set.
This is used to indicate that the textures needs to be read back to
CPU. Textures that don't need to be read back can be optimized by the
GPU backend.

Found during investigation of #104282.
buildbot/vdev-code-daily-coordinator Build done. Details
f222fe6a3a
Build: enable Python optimizations (PGO & LTO) on Linux
This is used for most Python release builds and has been reported to
give a modest 5-10% speedup (depending on the workload).

This could be enabled on macOS too but needs to be tested.
buildbot/vdev-code-daily-coordinator Build done. Details
0e196bab76
Build: disable LTO for Python builds
LTO compiled libpython3.10.a failed to link with GCC 12.0,
disable since these libraries are intended for developers to link
against.
buildbot/vdev-code-daily-coordinator Build done. Details
ca183993a5
Fix freeing uninitialized pointer in GHOST/Wayland + X11 fallback
Freeing the timer manager didn't account for Wayland being partially
initialized.
buildbot/vdev-code-daily-coordinator Build done. Details
666c2ea012
Refactor: remove yscale from bAnimContext
`bAnimContext` had a float property called `yscale_fac` that was used to define the height of the keyframe channels.

However the property was never set, only read so there really is no need to have it in the struct.

Moreover it complicated getting the channel height because `bAnimContext` had to be passed in.

Speaking of getting the channel height. This was done with macros. I ripped them all out and replaced them with function calls.

Originally it was introduced in this patch: https://developer.blender.org/rB095c8dbe6919857ea322b213a1e240161cd7c843

Co-authored-by: Christoph Lendenfeld <chris.lenden@gmail.com>
Pull Request #104500
22edf04458 I18n: use format strings for Cycles version error messages
The required version numbers for various devices was hardcoded in the
UI messages. The result was that every time one of these versions was
bumped, every language team had to update the message in question.

Instead, the version numbers can be extracted, and injected into the
error messages using string formatting so that translation updates
need happen less frequently.

Pull Request #104488
3bed78ff59 Curves: Add box selection
This adds a `select_box` function for the `Curves` object. It is used in the `view3d_box_select` operator.

It also adds the basic selection tools in the toolbar of Edit Mode.

Authored-by: Falk David <falkdavid@gmx.de>
Pull Request #104411
7ca651d182 Mesh: Remove unnecessary edge draw flag
As described in #95966, replace the `ME_EDGEDRAW` flag with a bit
vector in mesh runtime data. Currently the the flag is only ever set
to false for the "optimal display" feature of the subdivision surface
modifier. When creating an "original" mesh in the main data-base,
the flag is always supposed to be true.

The bit vector is now created by the modifier only as necessary, and
is cleared for topology-changing operations. This fixes incorrect
interpolation of the flag as noted in #104376. Generally it isn't
possible to interpolate it through topology-changing operations.

After this, only the seam status needs to be removed from edges before
we can replace them with the generic `int2` type (or something similar)
and reduce memory usage by 1/3.

Related:
- 10131a6f62
- 145839aa42

In the future `BM_ELEM_DRAW` could be removed as well. Currently it is
used and aliased by other defines in some non-obvious ways though.

Pull Request #104417
b8e15a4a84 Fix: Add missing "-" in logic to get the channel height
This was missed when doing the refactoring in #104500
It didn't seem to have any effect until I worked on clamping the view
buildbot/vdev-code-daily-coordinator Build done. Details
bfa7f9db0e
Assets: Implement viewport drag and drop for geometry nodes
Currently there's no way to assign a geometry node group from the asset
browser to an object as a modifier without first appending/linking it
manually. This patch adds a drag and drop operator that adds a new
modifier and assigns the dragged tree.

Pull Request #104430
buildbot/vdev-code-daily-coordinator Build done. Details
50dfd5f501
Geometry Nodes: Edges to Face Groups Node
Add a new node that groups faces inside of boundary edge regions.
This is the opposite action as the existing "Face Group Boundaries"
node. It's also the same as some of the "Initialize Face Sets"
options in sculpt mode.

Discussion in #102962 has favored "Group" for a name for these
sockets rather than "Set", so that is used here.

Pull Request #104428
buildbot/vdev-code-daily-coordinator Build done. Details
1649921791
Fix: Sequencer "Pan" label using incorrect keyword 'heading_ctxt'
Oversight in db87e2a638

Reviewed By: ISS
Differential Revision: https://archive.blender.org/developer/D17213
buildbot/vdev-code-daily-coordinator Build done. Details
50918d44fb
Cleanup: Fix const correctness warning in recent commit
bc0d3c91b1 Fix #104435: Fix rna_NlaStrip_new add strip logic to be correct boolean expression
Fixed #104435: Use correct conditional logic when testing if a new NLA strip can be added in the rna_NlaStrip_new method
buildbot/vdev-code-daily-coordinator Build done. Details
2cfc4d7644
Fix #104383: don't update declaration for clipboard copy
When nodes are copied to the clipboard, they don't need their declaration.
For nodes with dynamic declaration that might depend on the node tree itself,
the declaration could not be build anyway, because the node-clipboard does
not have a node tree.

Pull Request #104432
buildbot/vdev-code-daily-coordinator Build done. Details
5c8edbd99b
Cleanup: Move 6 sculpt-session-related files and header to C++
To allow further mesh data structure refactoring. See #103343

Pull Request #104540
buildbot/vdev-code-daily-coordinator Build done. Details
7e0e07657c
GPU: Cleanup GPU_batch.h documentation and some of the API for consistency
Documented all functions, adding use case and side effects.

Also replace the use of shortened argument name by more meaningful ones.

Renamed `GPU_batch_instbuf_add_ex` and `GPU_batch_vertbuf_add_ex` to remove
the `ex` suffix as they are the main version used (removed the few usage
of the other version).

Renamed `GPU_batch_draw_instanced` to `GPU_batch_draw_instance_range` and
make it consistent with `GPU_batch_draw_range`.
2ee9c12a23 PyAPI: add bpy.utils.manual_locale_code()
Move the function for getting the language code associated with the
user manual into a utility function (from the generated
rna_manual_reference.py).

This allows other parts of Blender to create a manual URL based on the
current locale preferences and environment.

Ref !104494
8ac3096e24 Fix add-on & manual link in Help menu ignoring the current language
Use bpy.utils.manual_language_code() create manual URL's instead of
assuming English.
48d9363fa7 Cleanup: quiet clang compiler warnings
- undeclared variable warning.
- unreachable-code-return warnings.
- array-parameter, mismatch bound.
- 'requires' is a keyword in C++20, (rename to requires_flag).
buildbot/vdev-code-daily-coordinator Build done. Details
4cbe0bff34
Cleanup: spelling in comments
a8d951abdd Docs: remove malformed patterns for RNA mapping
The generator now skips these with a warning, they will need to be
corrected in the user manual.

This caused tests/python/bl_rna_manual_reference.py to fail looking
up URL's.
buildbot/vdev-code-daily-coordinator Build done. Details
c2c62c3618
RNA: return a dummy language value when WITH_INTERNATIONAL=OFF
Without this, every access to "language" would warn that the enum
value didn't match a value in the enum items.

This made the bl_rna_manual_reference.py test output practically
unusable.
buildbot/vdev-code-daily-coordinator Build done. Details
b77c82e2bb
Tests: minor updates to make bl_rna_manual_reference more useful
- Avoid flooding the output with every match that succeeds.
- Report patterns listed in the manual that don't match anything in
  Blender.
- Disable external URL lookups, this is too slow.
  Instead use a LOCAL_PREFIX (a local build of the manual)
  or skip the test.
buildbot/vdev-code-daily-coordinator Build done. Details
01480229b1
Cycles: Fix MetalRT checkbox not hooked up to device on AMD
(Follow on from D17043)
On AMD Navi2 devices the MetalRT checkbox was not hooked up properly and had no effect. This patch fixes it.

Co-authored-by: Michael Jones <michael_p_jones@apple.com>
Pull Request #104520
buildbot/vdev-code-daily-coordinator Build done. Details
51ceeb506f
Fix #104026: Click-Drag to select graph editor channels no longer working
Box-Selecting channels in the dope sheet with click-drag was no longer possible as of Blender 3.2

Due to the removal of tweak events the box select operator was always shadowed by the click operator.

Original Phabricator discussion here: https://archive.blender.org/developer/D17065

Use `WM_operator_flag_only_pass_through_on_press` on click operator to fix it

Co-authored-by: Christoph Lendenfeld <chris.lenden@gmail.com>
Pull Request #104505
buildbot/vdev-code-daily-coordinator Build done. Details
5d30c3994e
Sequencer: Don't create undo step when click-select does nothing
When the sequencer is empty (i.e., there are no sequences),
we would have the deselect_all variable set to true called
ED_sequencer_deselect_all to select any existing sequences.

Ref !104453
buildbot/vdev-code-daily-coordinator Build done. Details
dc9f7fe64f
Fix #104514: GPencil merge down layer misses some frames
When merging two gpencil layers, if the destination layer had a keyframe
where the source layer did not, strokes of the previous keyframe
in source layer were lost in that frame.

This happened because the merge operator was looping through
frames of the source layer and appending strokes in the
corresponding destination layer, but never completing
other frames than the ones existing in the source layer.

This patch fixes it by first adding in source layer
all frames that are in destination layer.

Co-authored-by: Amelie Fondevilla <amelie.fondevilla@les-fees-speciales.coop>
Pull Request #104558
buildbot/vdev-code-daily-coordinator Build done. Details
88f9c55f7f
Sculpt: Fix Dyntopo Warnings
Because of T95965, some attributes are stored as generic attributes
in Mesh but have special handling for the conversion to BMesh.

Expose a function to tell whether certain attribute names are handled
specially in the conversion, and refactor the error checking process
to use it. Also check for generic attributes on the face domain which
wasn't done before.

Author: Hans Goudey
Reviewed By: Joseph Eagar

Co-authored-by: Joseph Eagar <joeedh@gmail.com>
Pull Request #104567
buildbot/vdev-code-daily-coordinator Build done. Details
bad2c3b9ef
Geometry Nodes: Experimental option for Volumes
Adds an experimental option under "New Features" in preferences,
which enables visibility of the new Volume Nodes.
Right now this option does nothing but will be used during development.
See #103248

Pull Request #104552
buildbot/vdev-code-daily-coordinator Build done. Details
284cdbb6cf
Cleanup: Use lambdas in mesh mapping callback, remove unused arguments
Using callback functions didn't scale well as more arguments are added.
It got very confusing when to pass tehmarguments weren't always used.
Instead use a `FunctionRef` with indices for arguments. Also remove
unused edge arguments to topology mapping functions.
buildbot/vdev-code-daily-coordinator Build done. Details
0ea15a6fbb
Fix: Inaccessible default for node group image sockets
The type was just skipped when drawing defaults for the image sockets.
923152d180 Geometry Nodes: improve parallelization in Delete/Separate Geometry node
This just adds `threading::parallel_for` and `threading::parallel_invoke` in a few
places where it can be added trivially. The run time of the `separate_geometry`
function changes from 830 ms to 413 ms in my test file.

Pull Request #104563
buildbot/vdev-code-daily-coordinator Build done. Details
fae661a1ab
Revert "Un-ignore modules in .gitmodules configuration"
This reverts commit aab707ab70.

A different solution to the submodule problem is being considered in #104573.
Revert to the previous behavior that developers are familiar with for now.
buildbot/vdev-code-daily-coordinator Build done. Details
d411be8a99
Cleanup: Use utility function to find groups in node tree
Add `contains_group` method in python api for `NodeTree` type, cleanup
`ntreeHasTree` function, reuse `ntreeHasTree` in more place in code.
The algorithm has been changed to not recheck trees by using set.

Performance gains from avoiding already checked node trees:
Based on tests, can say that for large files with a huge number
of trees, the response speed of opening the search menu in the
node editor increased by ~200 times (for really large projects
with 16 individual groups in 6 levels of nesting). Group insert
operations are also accelerated, but this is different in some cases.

Pull Request #104465
6f8c441950 Curves: Add select linked
This adds a new `select_linked` function that selects all the points
on a curve if there is at least one point already selected.
This also adds a keymap for the operator.

Co-authored-by: Falk David <falkdavid@gmx.de>
Pull Request #104569
5c4e1ed578 UI: Make text nomenclature and ordering consistent
"Center" -> "Middle" when describing vertical alignment.
"Align X" -> "Horizontal Alignment"
"Align Y" -> "Vertical Alignment"
Vertical alignment options rearranged to be consistently top-most to
bottom-most.

---

Co-authored-by: joshua-maros <60271685+joshua-maros@users.noreply.github.com>
Pull Request #104493
buildbot/vdev-code-daily-coordinator Build done. Details
7351f533e0
Curves: Add lasso and circle select
This adds a `select_lasso` and a `select_circle` function for the Curves object. It is used in the `view3d_lasso_select` and `view3d_circle_select` operator.

Co-authored-by: Falk David <falkdavid@gmx.de>
Pull Request #104560
buildbot/vdev-code-daily-coordinator Build done. Details
8a32d56056
Tests: Fix device list of benchmark script only showing a single GPU
Pull Request #104583
buildbot/vdev-code-daily-coordinator Build done. Details
0e6da74e98
Fix #104282: Resolve Depth read for D24_S8 types in Metal.
Fixes incorrect spotlight gizmo orientation when moving.

Authored by Apple: Michael Parkin-White

Related to #96261

Pull Request #104537
buildbot/vdev-code-daily-coordinator Build done. Details
efabe81c91
Fix #103903: Bump Node performance regression
Avoid computing the non-derivative height twice.
The height is now computed as part of the main function, while the height at x and y offsets are still computed on a separate function.
The differentials are now computed directly at node_bump.

Co-authored-by: Miguel Pozo <pragma37@gmail.com>
Pull Request #104595
buildbot/vdev-code-daily-coordinator Build done. Details
343bb4a5a3
Cleanup: Use const char * for layer names in collada exporter
CustomData layer names should not be written except via the CusomData
api. Therefore use const char * instead of char * when referencing the
layer name.

Pull Request #104585
fefc6a73b3 Fix pep8 checker operating on dot-files
Temporary editor files were included which could make the checker fail.
buildbot/vdev-code-daily-coordinator Build done. Details
6478eb565a
Cleanup: format
buildbot/vdev-code-daily-coordinator Build done. Details
0f708fa2e3
Geometry Nodes: use smooth normals in Distribute Points on Faces node
Previously, the node used the "true" normal of every looptri. Now it uses the
"loop normals" which includes e.g. smooth faces and custom normals. The true
normal can still be used on the points by capturing it before the Distribute node.

We do intend to expose the smooth normals separately in geometry nodes as well,
but this is an important first step.

It's also necessary to generate child hair between guide hair strands that don't
have visible artifacts at face boundaries.

For perfect backward compatibility, the node still has a "Legacy Normal" option
in the side bar. Creating the exact same behavior with existing nodes isn't
really possible unfortunately because of the specifics of how the Distribute
node used to compute the normals using looptris.

Pull Request #104414
buildbot/vdev-code-daily-coordinator Build done. Details
b723a398f3
Curves: initial surface collision for curves sculpt mode
During hair grooming in curves sculpt mode, it is very useful when hair strands
are prevented from intersecting with the surface mesh. Unfortunately, it also
decreases performance significantly so we don't want it to be turned on all the time.

The surface collision is used by the Comb, Pinch and Puff brushes currently.
It can be turned on or off on a per-geometry basis.

The intersection prevention quality of this patch is not perfect yet. This can
be improved over time using a better solver. Overall, perfect collision detection
at the cost of bad performance is not necessary for interactive sculpting,
because the user can fix small mistakes very quickly. Nevertheless, the quality
can probably still be improved significantly without too big slow-downs depending
on the use case. This can be done separately from this patch.

Pull Request #104469
buildbot/vdev-code-daily-coordinator Build done. Details
19ea673260
Cleanup: Remove const keyword in declarations
158f809dcb Geometry Nodes: Add option to hide input in modifier
When building a node group that's meant to be used directly in the
node editor as well as in the modifier, it's useful to be able to have
some inputs that are only meant for the node editor, like inputs that
only make sense when combined with other nodes.

In the future we might have the ability to only display certain assets
in the modifier and the node editor, but until then this simple solution
allows a bit more customization.

Pull Request #104517
buildbot/vdev-code-daily-coordinator Build done. Details
e732580fcc
Nodes: change order of Hide Value and Hide in Modifier
Based on the review comment in #104517.
b9fa32cccd Fix #104587: 'Extrude To Cursor' snapping ignoring 'Target Selection'
Although not a transform operator, `Extrude to Cursor` depends on some
snapping settings.

So it should use the `Target Selection` options as well.
232e02282e Fix circular transform gizmo always displaying Global orientation
The Global orientation comes from the mode's default orientation
(without the constraints).

It's not really exposed.
buildbot/vdev-code-daily-coordinator Build done. Details
085c854b2a
Fix curves selection toggling
buildbot/vdev-code-daily-coordinator Build done. Details
d33960aead
Cleanup: remove whole-archive linking for USD
Since USD is no longer statically linked these linker tricks
are no longer needed.

Co-authored-by: Ray Molenkamp <github@lazydodo.com>
Pull Request #104627
buildbot/vdev-code-daily-coordinator Build done. Details
77aa9e8809
Cleanup: GPU: Remove commented lines without any comments or purpose
These were added during a big refactor. They were supposed to be
uncommented at some point but the new code does not even need a default
world.
10354b043f Fix crash selecting faces in wire-frame mode
Regression in [0] didn't account for the mesh not having
subdivision surface is applied.

[0]: 75db4c082b
buildbot/vdev-code-daily-coordinator Build done. Details
a02fa6c40d
Cleanup: spelling in comments
buildbot/vdev-code-daily-coordinator Build done. Details
91346755ce
Cleanup: use '#' prefix for issues instead of 'T'
Match the convention from Gitea instead of Phabricator's T for tasks.
32149f8d7a Tests: add polyfill2d test to ensure the result has no zero area tris
Add a test to address the issue raised in #103913, where zero area
triangles could be created from polygons that have co-linear edges
but were not degenerate.
buildbot/vdev-code-daily-coordinator Build done. Details
3f40962414
Cleanup: use sized int types for polyfill_2d
Also correct building when USE_CLIP_EVEN is disabled.
buildbot/vdev-code-daily-coordinator Build done. Details
f0669ff8ba
BLI: use larger integer type in BitVector
Using larger integer types allows for more efficient code, because we
can use the hardware better. Instead of working on individual bytes,
the code can now work on 8 bytes at a time. We don't really benefit
from this immediately but I'm planning to implement some more optimized
bit vector operations for #104629.

Pull Request #104658
77963ff778 Fix #104637: EEVEE Displacement regression after #104595
Keep using the 3 evaluations dF_branch method for the Displacement output.
The optimized 2 evaluations method used by node_bump is now on its own macro (dF_branch_incomplete).
displacement_bump modifies the normal that nodetree_exec uses, so even with a refactor it wouldn’t be possible to re-use the computation anyway.
buildbot/vdev-code-daily-coordinator Build done. Details
6ea3fdebc8
Fix: Workbench Next: Extruded frustum binding
buildbot/vdev-code-daily-coordinator Build done. Details
2a7440176e
Fix: Missing const specifier for curve field input
Mistake in 000e722c7d, which probably made the viewer node
auto-domain detection behave differently when the special case was used.
buildbot/vdev-code-daily-coordinator Build done. Details
86b3073c9e
Cleanup: Quiet unused variable warning
Also name another argument for consistency.
f828ecf4ba GPU: Use same read back API as SSBOs
The GPU module has 2 different styles when reading back data from
GPU buffers. The SSBOs used a memcpy to copy the data to a
pre-allocated buffer. IndexBuf/VertBuf gave back a driver/platform
controlled pointer to the memory.

Readback is done for test cases returning mapped pointers is not safe.
For this reason we settled on using the same approach as the SSBO.
Copy the data to a caller pre-allocated buffer.

Reason why this API is currently changed is that the Vulkan API is more
strict on mapping/unmapping buffers that can lead to potential issues
down the road.

Pull Request #104571
buildbot/vdev-code-daily-coordinator Build done. Details
af8941e6a8
Vulkan: Use guardedalloc for driver allocations.
Vulkan has a pluggable memory allocation feature, which allows internal
driver allocations to be done by the client application provided
allocator. Vulkan uses this for more client application allocations
done inside the driver, but can also do it for more internal oriented
allocations.

VK_ALLOCATION_CALLBACKS initializes allocation callbacks for host allocations.
The macro creates a local static variable with the name vk_allocation_callbacks
that can be passed to vulkan API functions that expect
const VkAllocationCallbacks *pAllocator.

When WITH_VULKAN_GUARDEDALLOC=Off the memory allocation implemented
in the vulkan device driver is used for both internal and application
oriented memory operations.

For now this would help during the development of Vulkan backend to
detect hidden memory leaks that are hidden inside the driver part
of the stack. In a later stage we need to measure the overhead and
if this should become the default behavior.

Pull Request #104434
Jeroen Bakker added 1 commit 2023-02-13 09:37:35 +01:00
366404d274 Vulkan: Initial Compute Shaders and SSBO+VBO support.
This patch adds initial support for compute shaders and SSBOs to
the vulkan backend. As the development is oriented to the test-
cases we have the implementation is limited to what is used there.

It has been validated that with this patch 2 test cases are running
as expected
- `GPUVulkanTest.gpu_shader_compute_ssbo`
- `GPUVulkanTest.gpu_storage_buffer_create_update_read`

This patch includes:
- Allocating VkBuffer on device.
- Uploading data from CPU to VkBuffer.
- Binding VkBuffer as SSBO to a compute shader.
- Execute compute shader and altering VkBuffer.
- Download the VkBuffer to CPU ram.
- Validate that it worked.

GHOST API has been changed as the original design was created before
we even had support for compute shaders in blender. The function `GHOST_getVulkanBackbuffer` has been separated to retrieve the command
buffer without a backbuffer (`GHOST_getVulkanCommandBuffer`). In order
to do correct command buffer processing we needed access to the queue
owned by GHOST. This is returned as part of the `GHOST_getVulkanHandles`
function.
Jeroen Bakker changed title from Vulkan: Initial Compute Shaders and SSBO support. to Vulkan: Initial Compute Shaders support. 2023-02-13 09:45:32 +01:00
Jeroen Bakker added 1 commit 2023-02-13 09:45:58 +01:00
Jeroen Bakker added 1 commit 2023-02-13 13:30:02 +01:00
Jeroen Bakker added 4 commits 2023-02-13 16:09:02 +01:00
Jeroen Bakker changed title from Vulkan: Initial Compute Shaders support. to WIP: Vulkan: Initial Compute Shaders support. 2023-02-13 16:10:27 +01:00
Clément Foucault approved these changes 2023-02-14 02:04:47 +01:00
Clément Foucault left a comment
Member

This looks pretty solid!
I have to admit I could not review this in depth.

I couldn't make it to run on moltenvk. I always get Error: VK_LAYER_KHRONOS_validation not supported. Error: VK_KHR_portability_enumeration not found. Error: VK_KHR_get_physical_device_properties2 not found. Vulkan Error : intern/ghost/intern/GHOST_ContextVK.cpp:907 : vkCreateInstance(&create_info, __null, &m_instance) failled with VK_ERROR_INCOMPATIBLE_DRIVER

even after following https://stackoverflow.com/a/72791361

But that might be a problem on my end.

This looks pretty solid! I have to admit I could not review this in depth. I couldn't make it to run on moltenvk. I always get ``` Error: VK_LAYER_KHRONOS_validation not supported. Error: VK_KHR_portability_enumeration not found. Error: VK_KHR_get_physical_device_properties2 not found. Vulkan Error : intern/ghost/intern/GHOST_ContextVK.cpp:907 : vkCreateInstance(&create_info, __null, &m_instance) failled with VK_ERROR_INCOMPATIBLE_DRIVER``` even after following https://stackoverflow.com/a/72791361 But that might be a problem on my end.
@ -0,0 +7,4 @@
#pragma once
#ifdef __APPLE__

Cant we make that part of a header to avoid the duplication?

Cant we make that part of a header to avoid the duplication?
Jeroen-Bakker marked this conversation as resolved
@ -680,3 +677,4 @@
BLI_assert(geometry_module_ == VK_NULL_HANDLE);
BLI_assert(fragment_module_ == VK_NULL_HANDLE);
BLI_assert(compute_module_ != VK_NULL_HANDLE);
compute_pipeline_ = std::move(VKPipeline::create_compute_pipeline(
/Users/clement/Blender/blender/source/blender/gpu/vulkan/vk_shader.cc:680:25: warning: moving a temporary object prevents copy elision [-Wpessimizing-move]
    compute_pipeline_ = std::move(VKPipeline::create_compute_pipeline(
                        ^
``` /Users/clement/Blender/blender/source/blender/gpu/vulkan/vk_shader.cc:680:25: warning: moving a temporary object prevents copy elision [-Wpessimizing-move] compute_pipeline_ = std::move(VKPipeline::create_compute_pipeline( ^ ```
Jeroen-Bakker marked this conversation as resolved
Jeroen Bakker added 2 commits 2023-02-14 12:58:18 +01:00
Jeroen Bakker requested review from Bastien Montagne 2023-02-14 13:04:17 +01:00
Jeroen Bakker added 1 commit 2023-02-14 13:47:03 +01:00
Jeroen Bakker added 1 commit 2023-02-14 14:09:56 +01:00
Jeroen Bakker changed title from WIP: Vulkan: Initial Compute Shaders support. to Vulkan: Initial Compute Shaders support. 2023-02-14 14:28:15 +01:00
Jeroen Bakker added 1 commit 2023-02-14 14:28:40 +01:00
Jeroen Bakker added 1 commit 2023-02-14 14:43:23 +01:00
Jeroen Bakker added this to the 3.6 LTS milestone 2023-02-14 14:45:26 +01:00
Jeroen Bakker added 1 commit 2023-02-14 15:38:17 +01:00

Patch builds for me with WITH_VULKAN_BACKEND enabled.

How do we run these tests? I enabled both draw and render Opengl options in CMake, but these do not seem to run any of the GPU ones mentioned in the PR message (and fail a lot for me btw)...

Patch builds for me with `WITH_VULKAN_BACKEND` enabled. How do we run these tests? I enabled both draw and render Opengl options in CMake, but these do not seem to run any of the GPU ones mentioned in the PR message (and fail a lot for me btw)...
Author
Member

@mont29 I compile with WITH_GTEST=On, WITH_OPENGL_DRAW_TEST=On, WITH_VULKAN_BACKEND=On.
To run the test I normally use the command line
bin/tests/blender_test --gtest_filter=GPUVulkanTest.gpu_shader_compute*

If this doesn't work we can check tomorrow.

@mont29 I compile with `WITH_GTEST=On`, `WITH_OPENGL_DRAW_TEST=On`, `WITH_VULKAN_BACKEND=On`. To run the test I normally use the command line `bin/tests/blender_test --gtest_filter=GPUVulkanTest.gpu_shader_compute*` If this doesn't work we can check tomorrow.

Bah forgot to re-enable vulkan before trying again...

Yes tests are working now with this command, not sure why running it with something like `CTest -R "Vulkan" does not work though...

GPUVulkanTest.gpu_shader_compute_1d is failing for me though

Bah forgot to re-enable vulkan before trying again... Yes tests are working now with this command, not sure why running it with something like `CTest -R "Vulkan" does not work though... `GPUVulkanTest.gpu_shader_compute_1d` is failing for me though
Bastien Montagne reviewed 2023-02-15 14:40:05 +01:00
Bastien Montagne left a comment
Owner

Only checked the parts in GHOST currently. I find the API documentation scattering between GHOST_C-api.h, GHOST_IContext.h, and others, fairly confusing...

Some functions are documented in several places (some being seemingly more detailed than the others, like for getVulkanBackbuffer), others are only documented in the C API header. Would love to see this being harmonized (with e.g. everything in the C API, since I think the C++ headers are only used internally currently?).

Only checked the parts in GHOST currently. I find the API documentation scattering between `GHOST_C-api.h`, `GHOST_IContext.h`, and others, fairly confusing... Some functions are documented in several places (some being seemingly more detailed than the others, like for `getVulkanBackbuffer`), others are only documented in the C API header. Would love to see this being harmonized (with e.g. everything in the C API, since I think the C++ headers are only used internally currently?).
@ -1204,1 +1204,3 @@
uint32_t *r_graphic_queue_family);
uint32_t *r_graphic_queue_family,
void *r_queue);
void GHOST_GetVulkanCommandBuffer(GHOST_ContextHandle context, void *r_command_buffer);

Documentation? ;)

Documentation? ;)
Jeroen-Bakker marked this conversation as resolved
Jeroen Bakker added 3 commits 2023-02-16 10:25:54 +01:00
Author
Member

On NVidia platforms compute 1d test is failing due to:

VUID-VkImageCreateInfo-imageCreateMaxMipLevels-02251(ERROR / SPEC): msgNum: -1094930823 - Validation Error: [ VUID-VkImageCreateInfo-imageCreateMaxMipLevels-02251 ] Object 0: handle = 0x7f06b04f0a90, type = VK_OBJECT_TYPE_DEVICE; | MessageID = 0xbebcae79 | vkCreateImage(): Format VK_FORMAT_R32G32B32A32_SFLOAT is not supported for this combination of parameters and VkGetPhysicalDeviceImageFormatProperties returned back VK_ERROR_FORMAT_NOT_SUPPORTED. The Vulkan spec states: Each of the following values (as described in Image Creation Limits) must not be undefined : imageCreateMaxMipLevels, imageCreateMaxArrayLayers, imageCreateMaxExtent, and imageCreateSampleCounts (https://vulkan.lunarg.com/doc/view/1.2.198.1/linux/1.2-extensions/vkspec.html#VUID-VkImageCreateInfo-imageCreateMaxMipLevels-02251)
On NVidia platforms compute 1d test is failing due to: ``` VUID-VkImageCreateInfo-imageCreateMaxMipLevels-02251(ERROR / SPEC): msgNum: -1094930823 - Validation Error: [ VUID-VkImageCreateInfo-imageCreateMaxMipLevels-02251 ] Object 0: handle = 0x7f06b04f0a90, type = VK_OBJECT_TYPE_DEVICE; | MessageID = 0xbebcae79 | vkCreateImage(): Format VK_FORMAT_R32G32B32A32_SFLOAT is not supported for this combination of parameters and VkGetPhysicalDeviceImageFormatProperties returned back VK_ERROR_FORMAT_NOT_SUPPORTED. The Vulkan spec states: Each of the following values (as described in Image Creation Limits) must not be undefined : imageCreateMaxMipLevels, imageCreateMaxArrayLayers, imageCreateMaxExtent, and imageCreateSampleCounts (https://vulkan.lunarg.com/doc/view/1.2.198.1/linux/1.2-extensions/vkspec.html#VUID-VkImageCreateInfo-imageCreateMaxMipLevels-02251) ```
Jeroen Bakker added 1 commit 2023-02-16 12:13:19 +01:00
Jeroen Bakker added 2 commits 2023-02-17 09:58:56 +01:00
Jeroen Bakker added 3 commits 2023-02-17 11:30:42 +01:00
Jeroen Bakker added a new dependency 2023-02-18 17:52:35 +01:00
Jeroen Bakker added 1 commit 2023-02-21 14:34:34 +01:00
Jeroen Bakker closed this pull request 2023-02-21 14:55:37 +01:00
Jeroen Bakker reopened this pull request 2023-02-21 14:56:53 +01:00
Author
Member

Commit-id: 7fb1f060ff

Commit-id: 7fb1f060ff3111c87ef64ba33eb8cce372cfd270
Jeroen Bakker closed this pull request 2023-02-21 15:06:54 +01:00

Pull request closed

Sign in to join this conversation.
No Label
Interest
Alembic
Interest
Animation & Rigging
Interest
Asset Browser
Interest
Asset Browser Project Overview
Interest
Audio
Interest
Automated Testing
Interest
Blender Asset Bundle
Interest
BlendFile
Interest
Collada
Interest
Compatibility
Interest
Compositing
Interest
Core
Interest
Cycles
Interest
Dependency Graph
Interest
Development Management
Interest
EEVEE
Interest
EEVEE & Viewport
Interest
Freestyle
Interest
Geometry Nodes
Interest
Grease Pencil
Interest
ID Management
Interest
Images & Movies
Interest
Import Export
Interest
Line Art
Interest
Masking
Interest
Metal
Interest
Modeling
Interest
Modifiers
Interest
Motion Tracking
Interest
Nodes & Physics
Interest
OpenGL
Interest
Overlay
Interest
Overrides
Interest
Performance
Interest
Physics
Interest
Pipeline, Assets & IO
Interest
Platforms, Builds & Tests
Interest
Python API
Interest
Render & Cycles
Interest
Render Pipeline
Interest
Sculpt, Paint & Texture
Interest
Text Editor
Interest
Translations
Interest
Triaging
Interest
Undo
Interest
USD
Interest
User Interface
Interest
UV Editing
Interest
VFX & Video
Interest
Video Sequencer
Interest
Virtual Reality
Interest
Vulkan
Interest
Wayland
Interest
Workbench
Interest: X11
Legacy
Blender 2.8 Project
Legacy
Milestone 1: Basic, Local Asset Browser
Legacy
OpenGL Error
Meta
Good First Issue
Meta
Papercut
Meta
Retrospective
Meta
Security
Module
Animation & Rigging
Module
Core
Module
Development Management
Module
EEVEE & Viewport
Module
Grease Pencil
Module
Modeling
Module
Nodes & Physics
Module
Pipeline, Assets & IO
Module
Platforms, Builds & Tests
Module
Python API
Module
Render & Cycles
Module
Sculpt, Paint & Texture
Module
Triaging
Module
User Interface
Module
VFX & Video
Platform
FreeBSD
Platform
Linux
Platform
macOS
Platform
Windows
Priority
High
Priority
Low
Priority
Normal
Priority
Unbreak Now!
Status
Archived
Status
Confirmed
Status
Duplicate
Status
Needs Info from Developers
Status
Needs Information from User
Status
Needs Triage
Status
Resolved
Type
Bug
Type
Design
Type
Known Issue
Type
Patch
Type
Report
Type
To Do
No Milestone
No Assignees
3 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Blocks
Reference: blender/blender#104518
No description provided.