Vulkan: track discarded resource in swap chain #124863

New Issue

Jeroen Bakker · 2024-07-17T11:15:08+02:00

Jeroen Bakker commented

2024-07-17 11:15:08 +02:00

Resources

Lifetime

GPU Resource allocation are triggered from the CPU. Commands can be scheduled to the GPU that uses these resources. The actual moment these commands are executed or finished cannot be detected. GPU resources can only be destroyed after the commands using these resources have finished.

States of a command

stateDiagram
    recorded
    scheduled
    executing
    finished
    CPU: Application Controlled
    GPU: GPU Controlled
    state CPU {
        [*] --> recorded: Add to command buffer
        recorded --> scheduled: Queue command buffer
        scheduled --> [*]
    }
    state GPU {
        [*] --> executing
        executing --> finished
        finished --> [*]
    }
    CPU --> GPU

What happens on the GPU is asynchronously done from the CPU. When the GPU is executing the previous frame the CPU can already scheduling the next frame. When the execution of the frame is done the result can be presented on the display.

This gives quite a challenge when destroying resources. A typically solution is to keep track of the resources in a discard pile. There would be a discard pile per swap chain image. When a swap chain image is reused again we know for sure that the previous added resources to the discard pile can be destroyed.

gantt
    title CPU vs GPU vs Display
    dateFormat _
    axisFormat  
    todayMarker off
    section CPU
        Schedule #1: s1, 0, 10ms
        Schedule #2: s2, after s1, 10ms
        Schedule #3: s3, after s2, 11ms
        Schedule #4: s4, after s3, 12ms
        Schedule #5: s5, after s4, 11ms
    section GPU
        Execute #1: e1, after s1, 5ms
        Execute #2: e2, after s2, 5ms
        Execute #3: e3, after s3, 7ms
        Execute #4: e4, after s4, 6ms
    section Display
        Present #1: d1, after e1, until d2
        Present #2: d2, after e2, until d3
        Present #3: d3, after e3, until d4
        Present #1: d4, after e4, 10ms

NOTE: In the above diagram we assume a swap chain of 3 images. Frame 4 will render on the same swap chain image as frame 1.

Resource pool

Resource pools keep track of the resources for a swap chain image.

In Blender this is a bit more complicated due to the way GPUContext work. A single thread can have multiple contexts. Some of them have a swap chain (GHOST Window) other don't (draw manager). The resource pool should be shared between the contexts running on the same thread.

When opening multiple windows there are also multiple swap chains.

Discard pile

Resource handles that are deleted and stored in the discard pile. When we are sure that these resources are not used on the gpu anymore these are destroyed.

Staging buffers

Staging buffers are used to update GPU resources when they need to be updated. For example when a buffer is used in frame 1 and 2, but have different content. The new content is stored in a staging buffer. The new content will be copied just before the buffer is first used in frame 2.

Keeping a pool of staging buffers per swap chain image allow reusing previous allocated resources.

Reusable resources

There are other resources as well like:

Descriptor sets
Descriptor pools

Resource deletion

From Blender perspective there are several deletion patterns.

Active Context

Most common is that there is an active context. In this case we add the resource to the active discard pile.

Dependency graph update

Dependency graph updates occur in Blenders main loop or when rendering. When the dependency graph is updated cached gpu data can be deleted. The data include material shaders and textures and geometry buffers.

The deletion happens from any thread and there is no active context. There is also no information if the deletion happens for rendering (triggered from background thread) or main loop (triggered from main thread).

During rendering the deps graph update happens after the rendering is finished and is safe to destroy immediately. But as we don't know if a resource is destroyed from the main loop or render we assume we cannot destroy it immediately. The deleted resources will be added to the device level discard pile.

Next time a swap chain context advanced to the next swap chain image the device level discard pile is added to the swap chain image discard pile. When Blender runs in the background it is added to the context first discard pile.

Solution

classDiagram
    class VKDevice {
        thread_data()
        resource_pool_get()
    }
    class VKThreadData {
        swap_chain_index
    }
    class VKResourcePool {
        descriptor_pools
        descriptor_sets
    }
    class VKDiscardPool {
        discard_image(vk_image, allocation)
        discard_buffer(vk_buffer, alloction)
        discard_shader_module(vk_shader_module)
        discard_pipeline_layout(vk_pipeline_layout)
        destroy_discarded_resources()
    }

    VKDevice *--> VKThreadData: thread_data
    VKDevice *--> VKDiscardPool: orphaned_data
    VKThreadData *--> VKResourcePool: active_resource_pool
    VKResourcePool *--> VKDiscardPool

Thread discarded resources are destroyed when the context becomes active with a new swap chain index.

Device discarded resources are moved to the thread discarded resources when:

a context becomes active with a swap chain.
a context becomes active and Blender runs in background mode.

Resources that are discarded outside a context are placed inside the orphaned_data. When a context switches its swap chain the content of orphaned_data is moved to the new swap chain image discard pool.

# Resources ## Lifetime GPU Resource allocation are triggered from the CPU. Commands can be scheduled to the GPU that uses these resources. The actual moment these commands are executed or finished cannot be detected. GPU resources can only be destroyed after the commands using these resources have finished. **States of a command** ```mermaid stateDiagram recorded scheduled executing finished CPU: Application Controlled GPU: GPU Controlled state CPU { [*] --> recorded: Add to command buffer recorded --> scheduled: Queue command buffer scheduled --> [*] } state GPU { [*] --> executing executing --> finished finished --> [*] } CPU --> GPU ``` What happens on the GPU is asynchronously done from the CPU. When the GPU is executing the previous frame the CPU can already scheduling the next frame. When the execution of the frame is done the result can be presented on the display. This gives quite a challenge when destroying resources. A typically solution is to keep track of the resources in a discard pile. There would be a discard pile per swap chain image. When a swap chain image is reused again we know for sure that the previous added resources to the discard pile can be destroyed. ```mermaid gantt title CPU vs GPU vs Display dateFormat _ axisFormat todayMarker off section CPU Schedule #1: s1, 0, 10ms Schedule #2: s2, after s1, 10ms Schedule #3: s3, after s2, 11ms Schedule #4: s4, after s3, 12ms Schedule #5: s5, after s4, 11ms section GPU Execute #1: e1, after s1, 5ms Execute #2: e2, after s2, 5ms Execute #3: e3, after s3, 7ms Execute #4: e4, after s4, 6ms section Display Present #1: d1, after e1, until d2 Present #2: d2, after e2, until d3 Present #3: d3, after e3, until d4 Present #1: d4, after e4, 10ms ``` > NOTE: In the above diagram we assume a swap chain of 3 images. Frame 4 will render on the same swap chain image as frame 1. ## Resource pool Resource pools keep track of the resources for a swap chain image. In Blender this is a bit more complicated due to the way GPUContext work. A single thread can have multiple contexts. Some of them have a swap chain (GHOST Window) other don't (draw manager). The resource pool should be shared between the contexts running on the same thread. When opening multiple windows there are also multiple swap chains. ### Discard pile Resource handles that are deleted and stored in the discard pile. When we are sure that these resources are not used on the gpu anymore these are destroyed. ### Staging buffers Staging buffers are used to update GPU resources when they need to be updated. For example when a buffer is used in frame 1 and 2, but have different content. The new content is stored in a staging buffer. The new content will be copied just before the buffer is first used in frame 2. Keeping a pool of staging buffers per swap chain image allow reusing previous allocated resources. ### Reusable resources There are other resources as well like: - Descriptor sets - Descriptor pools ## Resource deletion From Blender perspective there are several deletion patterns. ### Active Context Most common is that there is an active context. In this case we add the resource to the active discard pile. ### Dependency graph update Dependency graph updates occur in Blenders main loop or when rendering. When the dependency graph is updated cached gpu data can be deleted. The data include material shaders and textures and geometry buffers. The deletion happens from any thread and there is no active context. There is also no information if the deletion happens for rendering (triggered from background thread) or main loop (triggered from main thread). During rendering the deps graph update happens after the rendering is finished and is safe to destroy immediately. But as we don't know if a resource is destroyed from the main loop or render we assume we cannot destroy it immediately. The deleted resources will be added to the device level discard pile. Next time a swap chain context advanced to the next swap chain image the device level discard pile is added to the swap chain image discard pile. When Blender runs in the background it is added to the context first discard pile. ## Solution ```mermaid classDiagram class VKDevice { thread_data() resource_pool_get() } class VKThreadData { swap_chain_index } class VKResourcePool { descriptor_pools descriptor_sets } class VKDiscardPool { discard_image(vk_image, allocation) discard_buffer(vk_buffer, alloction) discard_shader_module(vk_shader_module) discard_pipeline_layout(vk_pipeline_layout) destroy_discarded_resources() } VKDevice *--> VKThreadData: thread_data VKDevice *--> VKDiscardPool: orphaned_data VKThreadData *--> VKResourcePool: active_resource_pool VKResourcePool *--> VKDiscardPool ``` Thread discarded resources are destroyed when the context becomes active with a new swap chain index. Device discarded resources are moved to the thread discarded resources when: - a context becomes active with a swap chain. - a context becomes active and Blender runs in background mode. Resources that are discarded outside a context are placed inside the `orphaned_data`. When a context switches its swap chain the content of `orphaned_data` is moved to the new swap chain image discard pool.

Jeroen Bakker added this to the 4.3 milestone 2024-07-17 11:15:08 +02:00

Jeroen Bakker added the

Download

What's New

Blender Studio

Manual

Developers Blog

Documentation

Benchmark

Blender Conference

Development Fund

One-time Donations

Vulkan: track discarded resource in swap chain #124863