Metal Viewport #96261

Open
opened 2022-03-09 12:44:49 +01:00 by Michael Parkin-White · 103 comments

Status: ? Initial stages of development


Description
Bringing Metal support to the Blender viewport for optimal performance on macOS. This will be implemented as a new GPU Backend within Blender's GPU Module.

GHOST

    • Encapsulate use of OpenGL within GHOST_ContextCGL
    • Add support for Metal Backend to GHOST_ContextCGL
    • EDR (Extended Dynamic Range) support for GHOST swapchain

EEVEE Next - Workbench Next - Overlay Next

Initial bring-up

    • Add support for 2D Texture Atomics via Buffer + Implement metal path
    • Add support for virtual frame buffers (No attachments, but specified dimensions, currently allocating temporary texture for enablement).
    • EEVEE + GPU Unit tests 100% pastorate
    • EEVEE irradiance cache baking - Metal support
    • EEVEE Next shadows - Metal support

Optimisation + Later:

    • Optimize compute kernel workgroup sizes for all passes
    • Investigate kernels wherein threadgroup size requirement exceeds optimal sizing heuristic. Aim to avoid using maxTotalThreadsPerThreadgroup.
    • Ensure resource usage is optimal (compression etc)

GPU Module METAL Backend

    • MTL_backend
    • MTL_batch
    • -> Metal indirect draw support
    • MTL_capabilities
    • MTL_command_buffer
    • MTL_context
    • MTL_debug
    • MTL_drawlist
    • MTL_element
    • MTL_extensions (via backend)
    • MTL_framebuffer
    • MTL_immediate
    • MTL_memory_manager
    • MTL_query
    • MTL_shader
    • MTL_shader_interface
    • MTL_state
    • MTL_storage_buf
    • MTL_texture
    • MTL_uniformbuffer
    • MTL_vertex_buffer
    • Compute support for Metal

More items to be added as necessary.

GPU Feature coverage

A number of features available in Blender require API functionality which is either not directly available in the Metal API, such as geometry shaders, or requires a higher degree of explicitness from the high-level renderer. These tasks represent non-invasive changes required to enable full Metal support:

    • GPU - Geometry shader workflow alternative
    • GPU - Multi-layered rendering support in Vertex Shader
    • GPU - Shading language translation layer from GLSL to MSL (Metal Shading Language).
    • GPU - Shader uniform struct alignment validation for Metal (GPU_uniformbuf_create_from_list)
    • GPU - Shader barycentric support in Metal.
    • GPU - default depth format support checks (Apple Silicon does not support GPU_DEPTH24_STENCIL8)
    • GPU - Lightweight GLSL shader requirements for MSL compatibility.
    • GPU - Extended Dynamic Range (EDR) support for OCIO shaders and GPUViewport.
    • #104025 Offline Metal shader compilation - Startup compilation time and system caching.
    • Metal/DRW - EEVEE/Material PSO (Pipeline State Object) parallel compilation.

Shader libraries compile (Front-end compilation) in parallel while displaying the default EEVEE material. For Metal (and likely Vulkan too), PSO compilation (backend/low-level compilation) happens on demand, depending on the current render pipeline state in combination with a given shader, and this can take time, introducing stalls and stutters, such as when an object is first selected and the select-engine/depth-only permutation is generated for that particular mesh.

Ideally we would want to avoid these one-off PSO stalls, which for OpenGL, are generally hidden behind driver processing. Propose a few options:

  • Option to add a "default/fallback shader to a GPUShader such that if it needs a fresh PSO, and a default shader/fallback is provided, it can instead bind that shader for the draw. In essence, continue using the EEVEE default material, if PSO compilation is needed, not only when the shader library is compiling.
  • Shader PSO cache warming: Generate the render pipeline state configurations/permutations up-front to allow background parallel PSO compilation. This could be done by performing a dry-run of Blender and packaging a data-file containing the serialized input structure for the PSO compilation function, and performing background warming after start-up, using this data. This avoids the need to manually specify the configs.
  • Prepare certain configurations up-front in background by performing dummy renders in different states, and skipping draws which trigger a PSO compilation. This allows the dynamic render state to be set, without performing rendering.
    • Metal viewport/Metal Cycles shader compiler load distribution manager.

As both backends use Metal, there is only a fixed pool of shader compilation threads available. A mechanism should be in place to ensure compilation is not stalled, which could cause the viewport to become unresponsive for synchronous shader compilations. e.g. PSO backend compilations.



Relevant links:

**Status:** `?` Initial stages of development --- **Description** Bringing Metal support to the Blender viewport for optimal performance on macOS. This will be implemented as a new GPU Backend within Blender's GPU Module. **GHOST** * - [x] Encapsulate use of OpenGL within GHOST_ContextCGL * - [x] Add support for Metal Backend to GHOST_ContextCGL * - [x] EDR (Extended Dynamic Range) support for GHOST swapchain **EEVEE Next - Workbench Next - Overlay Next** Initial bring-up * - [x] Add support for 2D Texture Atomics via Buffer + Implement metal path * - [ ] Add support for virtual frame buffers (No attachments, but specified dimensions, currently allocating temporary texture for enablement). * - [ ] EEVEE + GPU Unit tests 100% pastorate * - [x] EEVEE irradiance cache baking - Metal support * - [x] EEVEE Next shadows - Metal support Optimisation + Later: * - [ ] Optimize compute kernel workgroup sizes for all passes * - [ ] Investigate kernels wherein threadgroup size requirement exceeds optimal sizing heuristic. Aim to avoid using maxTotalThreadsPerThreadgroup. * - [ ] Ensure resource usage is optimal (compression etc) **GPU Module METAL Backend** * - [x] MTL_backend * - [x] MTL_batch * - [x] -> Metal indirect draw support * - [x] MTL_capabilities * - [x] MTL_command_buffer * - [x] MTL_context * - [x] MTL_debug * - [x] MTL_drawlist * - [x] MTL_element * - [x] MTL_extensions (via backend) * - [x] MTL_framebuffer * - [x] MTL_immediate * - [x] MTL_memory_manager * - [x] MTL_query * - [x] MTL_shader * - [x] MTL_shader_interface * - [x] MTL_state * - [x] MTL_storage_buf * - [x] MTL_texture * - [x] MTL_uniformbuffer * - [x] MTL_vertex_buffer * - [x] Compute support for Metal More items to be added as necessary. **GPU Feature coverage** A number of features available in Blender require API functionality which is either not directly available in the Metal API, such as geometry shaders, or requires a higher degree of explicitness from the high-level renderer. These tasks represent non-invasive changes required to enable full Metal support: * - [x] GPU - Geometry shader workflow alternative * - [x] GPU - Multi-layered rendering support in Vertex Shader * - [x] GPU - Shading language translation layer from GLSL to MSL (Metal Shading Language). * - [x] GPU - Alternatives for unsupported GPU Primitive types: `GPU_PRIM_TRI_FAN`/`GPU_PRIM_LINE_LOOP` (https://developer.apple.com/documentation/metal/mtlprimitivetype?language=objc) * - [x] GPU - Shader uniform struct alignment validation for Metal (`GPU_uniformbuf_create_from_list`) * - [x] GPU - Shader barycentric support in Metal. * - [x] GPU - default depth format support checks (Apple Silicon does not support `GPU_DEPTH24_STENCIL8`) * - [x] GPU - Lightweight GLSL shader requirements for MSL compatibility. * - [x] GPU - Extended Dynamic Range (EDR) support for OCIO shaders and GPUViewport. * - [x] DRW/GPU/COM - Support for Realtime compositor (https://developer.blender.org/T99210) * - [ ] #104025 Offline Metal shader compilation - Startup compilation time and system caching. * - [ ] Metal/DRW - EEVEE/Material PSO (Pipeline State Object) parallel compilation. > Shader libraries compile (Front-end compilation) in parallel while displaying the default EEVEE material. For Metal (and likely Vulkan too), PSO compilation (backend/low-level compilation) happens on demand, depending on the current render pipeline state in combination with a given shader, and this can take time, introducing stalls and stutters, such as when an object is first selected and the select-engine/depth-only permutation is generated for that particular mesh. > > Ideally we would want to avoid these one-off PSO stalls, which for OpenGL, are generally hidden behind driver processing. Propose a few options: > - Option to add a "default/fallback shader to a **GPUShader** such that if it needs a fresh PSO, and a default shader/fallback is provided, it can instead bind that shader for the draw. In essence, continue using the EEVEE default material, if PSO compilation is needed, not only when the shader library is compiling. > - Shader PSO cache warming: Generate the render pipeline state configurations/permutations up-front to allow background parallel PSO compilation. This could be done by performing a dry-run of Blender and packaging a data-file containing the serialized input structure for the PSO compilation function, and performing background warming after start-up, using this data. This avoids the need to manually specify the configs. > - Prepare certain configurations up-front in background by performing dummy renders in different states, and skipping draws which trigger a PSO compilation. This allows the dynamic render state to be set, without performing rendering. * - [x] Metal viewport/Metal Cycles shader compiler load distribution manager. > As both backends use Metal, there is only a fixed pool of shader compilation threads available. A mechanism should be in place to ensure compilation is not stalled, which could cause the viewport to become unresponsive for synchronous shader compilations. e.g. PSO backend compilations. ``` ``` * - [x] Python -- Use of OpenGL within Python API - Ensure safe fallback (https://developer.blender.org/T103863) * - [x] Cycles - Live viewport display -- Currently uses OpenGL directly --- **Relevant links**: * `-` https://developer.apple.com/metal/Metal-Shading-Language-Specification.pdf * `-` https://developer.apple.com/metal/ * `-` https://developer.apple.com/metal/Metal-Feature-Set-Tables.pdf

Added subscriber: @Michael-Parkin-White-Apple

Added subscriber: @Michael-Parkin-White-Apple
Member

Added subscriber: @lichtwerk

Added subscriber: @lichtwerk
Member

Changed status from 'Needs Triage' to: 'Needs Developer To Reproduce'

Changed status from 'Needs Triage' to: 'Needs Developer To Reproduce'
Member

Guess this can be confirmed (it was still in the Needs Triage status)?

Guess this can be confirmed (it was still in the `Needs Triage` status)?

Changed status from 'Needs Developer To Reproduce' to: 'Confirmed'

Changed status from 'Needs Developer To Reproduce' to: 'Confirmed'
Member

Added subscriber: @Alaska

Added subscriber: @Alaska

Added subscriber: @CharlesWardlaw

Added subscriber: @CharlesWardlaw

Added subscriber: @ThomasDinges

Added subscriber: @ThomasDinges

Added subscriber: @SteveWong-1

Added subscriber: @SteveWong-1

Added subscriber: @Steve-Hanff

Added subscriber: @Steve-Hanff

Added subscriber: @hatchli

Added subscriber: @hatchli

Added subscriber: @Dangry

Added subscriber: @Dangry

Added subscriber: @blenderrocket

Added subscriber: @blenderrocket

Added subscriber: @MN-4

Added subscriber: @MN-4

This issue was referenced by 309ea31485

This issue was referenced by 309ea314858a9b7892ea2c8a6fe55ab2a1028697

This issue was referenced by 19c793af35

This issue was referenced by 19c793af35ea8e694c16995d115d7c9247fee81a

Added subscriber: @ErikT

Added subscriber: @ErikT

Added subscriber: @HectorDeAnda

Added subscriber: @HectorDeAnda

Added subscriber: @PhlixFer

Added subscriber: @PhlixFer

Added subscriber: @CookItOff

Added subscriber: @CookItOff

Added subscriber: @Koen-Kooi

Added subscriber: @Koen-Kooi

This issue was referenced by 7c9e128bbf

This issue was referenced by 7c9e128bbfb37d70799f8867d93992ee3f522242

This issue was referenced by 49bc640b76

This issue was referenced by 49bc640b76c77a19e523cd1843585a0c6b21e82f

Added subscriber: @2046411367

Added subscriber: @2046411367

This issue was referenced by e28f07b5c8

This issue was referenced by e28f07b5c899a3036060e9e3f50a3e878663f02a

This issue was referenced by 8f0e06a0ca

This issue was referenced by 8f0e06a0ca1f4b7f8ea00ec5346a5e55a2620707
Contributor

Added subscriber: @FynnGr

Added subscriber: @FynnGr

This issue was referenced by cdd4354c81

This issue was referenced by cdd4354c81f6f9ae3bc72b7abd5dbcfd74fa548e

This issue was referenced by db622b5a0b

This issue was referenced by db622b5a0bea32cb3a78c0a4c7d4372a9178d27e

Added subscriber: @fuchs_und_baer

Added subscriber: @fuchs_und_baer

Added subscriber: @AledBrown

Added subscriber: @AledBrown

This issue was referenced by 073139e329

This issue was referenced by 073139e32943b863828474ca70bbdfc258520769

Added subscriber: @WilliamReynish

Added subscriber: @WilliamReynish

Added subscriber: @Nurb2Kea

Added subscriber: @Nurb2Kea

This issue was referenced by 9130a60d3d

This issue was referenced by 9130a60d3d833718c932d3f971b746ba253734bc

This issue was referenced by 4527dd1ce4

This issue was referenced by 4527dd1ce4784292cd3b8dd3764b9cd843020f9a

Added subscriber: @pixeltrain3d

Added subscriber: @pixeltrain3d

Added subscriber: @mattli911

Added subscriber: @mattli911
Contributor

Added subscriber: @Brainzman

Added subscriber: @Brainzman

This issue was referenced by 6bba4d864e

This issue was referenced by 6bba4d864e13e37b0c95efbf5dab4edb75d110a3

This issue was referenced by 9835d5e58b

This issue was referenced by 9835d5e58ba28ff69abec74c1cb3f3959ff9451a

Added subscriber: @Cigitia

Added subscriber: @Cigitia

Added subscriber: @skyscapeparadise

Added subscriber: @skyscapeparadise

Added subscriber: @Alex-Claydon

Added subscriber: @Alex-Claydon

Added subscriber: @mycoconut

Added subscriber: @mycoconut

Added subscriber: @OlliPikkarainen

Added subscriber: @OlliPikkarainen

Added subscriber: @GeorgiaPacific

Added subscriber: @GeorgiaPacific

Added subscriber: @Austin-Berenyi

Added subscriber: @Austin-Berenyi

Added subscriber: @Kuutti-Taavitsainen

Added subscriber: @Kuutti-Taavitsainen

This issue was referenced by 5f4409b02e

This issue was referenced by 5f4409b02ef7c54089ff1b491e008d4b86c030f4

This issue was referenced by ac07fb38a1

This issue was referenced by ac07fb38a1b35fa156b2d0901eb35cd65ed73903

This issue was referenced by cc8ea6ac67

This issue was referenced by cc8ea6ac67a108fcb96e4a8373ac02faf9ccea3d

Added subscriber: @winnertakesteve-1

Added subscriber: @winnertakesteve-1

This issue was referenced by bb63b98d1f

This issue was referenced by bb63b98d1ff5acfd24dff9b5e72175f82f5bca26

This issue was referenced by 697b447c20

This issue was referenced by 697b447c2069bbbbaa9929aab0ea1f66ef8bf4d0

This issue was referenced by 1514e1a5b7

This issue was referenced by 1514e1a5b7e15ec0c11cd40c2b9389982bd5d00e

This issue was referenced by 18b45aabf9

This issue was referenced by 18b45aabf91ad0a407e8c7cd16c5503e7a99d6fa

This issue was referenced by e8bcca4bdc

This issue was referenced by e8bcca4bdc947caaca6742cdefc60fb9a4f57e9a

Added subscriber: @Emi_Martinez

Added subscriber: @Emi_Martinez

This issue was referenced by 0fdb9cd650

This issue was referenced by 0fdb9cd6507a28b349abf47122d6dc394ca24dee

Removed subscriber: @Alex-Claydon

Removed subscriber: @Alex-Claydon

Added subscriber: @Alex-Claydon

Added subscriber: @Alex-Claydon

This issue was referenced by aed301704a

This issue was referenced by aed301704aa1cd02b8868396d36ce78bc292850e

Added subscriber: @Alexander-Hartmann

Added subscriber: @Alexander-Hartmann

Removed subscriber: @fuchs_und_baer

Removed subscriber: @fuchs_und_baer

Added subscriber: @makizar

Added subscriber: @makizar

This issue was referenced by b132e3b3ce

This issue was referenced by b132e3b3ce1c5ecc836d22f4c4cba90e0ed2edcc

This issue was referenced by 009f7de619

This issue was referenced by 009f7de619e62b9415fa09f4ee72ecb4b8fbb9b9

Added subscriber: @Caco-Oportot

Added subscriber: @Caco-Oportot

This issue was referenced by 6b8bb26c45

This issue was referenced by 6b8bb26c457446875e0782a1701ec35c78b33308

This issue was referenced by d90a2b0ab7

This issue was referenced by d90a2b0ab7926b524d4c1b2223ca2baf73cd0ee4

This issue was referenced by 2744ee2262

This issue was referenced by 2744ee22624b9923ea1ea537410a0f3ea80c45c7

This issue was referenced by a83f2834c7

This issue was referenced by a83f2834c7e555ba85a6cf74a949ec3234849de5

This issue was referenced by 9ec20f2ede

This issue was referenced by 9ec20f2ede3c894b6c92603002aee11aa6a858a2

This issue was referenced by 359d98423e

This issue was referenced by 359d98423e17eb62b03b65666734fd97ecd04318

This issue was referenced by 2e61c446ac

This issue was referenced by 2e61c446acb189ecfbd5ba4508082a25ec9f4369

Removed subscriber: @Alexander-Hartmann

Removed subscriber: @Alexander-Hartmann

This issue was referenced by 81f425a36f

This issue was referenced by 81f425a36ff6b6d9bbc324a16478955758f9b5ac

This issue was referenced by 2712265598

This issue was referenced by 2712265598a09c3868b9d097488f22d42efbf6eb

This issue was referenced by df1fe18ed7

This issue was referenced by df1fe18ed75812265cf2af186f6b082d8d27d9fe

This issue was referenced by b3464fe152

This issue was referenced by b3464fe152ca8a7b50dddf0774e33d24e38fd6b5

This issue was referenced by dedca2c994

This issue was referenced by dedca2c9948af7a65dd78263e37f77052fc03de7

This issue was referenced by 7ff47f7a94

This issue was referenced by 7ff47f7a944cb3107af127cf0545d3c0f04b3555

This issue was referenced by 3535670ff1

This issue was referenced by 3535670ff1a1dd735475303c885e01a07a5f4b54

This issue was referenced by 51e2ce7df5

This issue was referenced by 51e2ce7df5b566b526babc750a7288eab4ccd4e6

This issue was referenced by d4c085c17d

This issue was referenced by d4c085c17d135df810951d5dff4c2693916ff1d6

This issue was referenced by b1d2ea3e1b

This issue was referenced by b1d2ea3e1b1173122eebecda0c3bd7552f4ee9c1

This issue was referenced by ed8f3dc9c7

This issue was referenced by ed8f3dc9c79de72f3b8dd8bdcdf9b47cecb2a541

This issue was referenced by d3f626b535

This issue was referenced by d3f626b535313af616b0574958b2d064fc88c5d3

Added subscriber: @MatthiasZierau

Added subscriber: @MatthiasZierau

This issue was referenced by cd2926fb05

This issue was referenced by cd2926fb05b0179a405a5406d6b87f5bff36d15a

This issue was referenced by 0ba5954bb2

This issue was referenced by 0ba5954bb27b4fee5d0ed1a14feb91118b6f3fad

This issue was referenced by 1c672f3d1d

This issue was referenced by 1c672f3d1dae4f79d327c0fd38c440f3bf23d35c

This issue was referenced by 139fb38d4f

This issue was referenced by 139fb38d4fb4317a467a0abd28557f1713eb15af

This issue was referenced by 84c25fdcaa

This issue was referenced by 84c25fdcaa42ae1ac7a937d4021095273f08603d

This issue was referenced by 57552f52b2

This issue was referenced by 57552f52b2987adfd1525c3e4109565e3641ef5d

This issue was referenced by 6dde185dc4

This issue was referenced by 6dde185dc470498d3eb2676a96728b5aef5e60ed

This issue was referenced by 596ee79a9f

This issue was referenced by 596ee79a9f590c1d66aa1cb5c19f994ddb0f0272

This issue was referenced by aca9c131fc

This issue was referenced by aca9c131fc812697575288e7ab3e2946ad62bddd

This issue was referenced by f3bd5458a3

This issue was referenced by f3bd5458a39b7ebd397dac10c8555a4c9fbc7409

This issue was referenced by a504058dee

This issue was referenced by a504058dee7684758456aaac24342ae24df3f3a1

This issue was referenced by f152159101

This issue was referenced by f1521591017faac334ff78c291ad66bdaf327661

This issue was referenced by 8703db393b

This issue was referenced by 8703db393bfb5290777c59ef29b43d485bcfb80e
Clément Foucault added this to the EEVEE & Viewport project 2023-02-09 01:08:48 +01:00
Clément Foucault removed the
Interest
EEVEE & Viewport
label 2023-02-09 01:09:04 +01:00
Sign in to join this conversation.
No Label
Interest
Alembic
Interest
Animation & Rigging
Interest
Asset Browser
Interest
Asset Browser Project Overview
Interest
Audio
Interest
Automated Testing
Interest
Blender Asset Bundle
Interest
BlendFile
Interest
Collada
Interest
Compatibility
Interest
Compositing
Interest
Core
Interest
Cycles
Interest
Dependency Graph
Interest
Development Management
Interest
EEVEE
Interest
EEVEE & Viewport
Interest
Freestyle
Interest
Geometry Nodes
Interest
Grease Pencil
Interest
ID Management
Interest
Images & Movies
Interest
Import Export
Interest
Line Art
Interest
Masking
Interest
Metal
Interest
Modeling
Interest
Modifiers
Interest
Motion Tracking
Interest
Nodes & Physics
Interest
OpenGL
Interest
Overlay
Interest
Overrides
Interest
Performance
Interest
Physics
Interest
Pipeline, Assets & IO
Interest
Platforms, Builds & Tests
Interest
Python API
Interest
Render & Cycles
Interest
Render Pipeline
Interest
Sculpt, Paint & Texture
Interest
Text Editor
Interest
Translations
Interest
Triaging
Interest
Undo
Interest
USD
Interest
User Interface
Interest
UV Editing
Interest
VFX & Video
Interest
Video Sequencer
Interest
Virtual Reality
Interest
Vulkan
Interest
Wayland
Interest
Workbench
Interest: X11
Legacy
Blender 2.8 Project
Legacy
Milestone 1: Basic, Local Asset Browser
Legacy
OpenGL Error
Meta
Good First Issue
Meta
Papercut
Meta
Retrospective
Meta
Security
Module
Animation & Rigging
Module
Core
Module
Development Management
Module
EEVEE & Viewport
Module
Grease Pencil
Module
Modeling
Module
Nodes & Physics
Module
Pipeline, Assets & IO
Module
Platforms, Builds & Tests
Module
Python API
Module
Render & Cycles
Module
Sculpt, Paint & Texture
Module
Triaging
Module
User Interface
Module
VFX & Video
Platform
FreeBSD
Platform
Linux
Platform
macOS
Platform
Windows
Priority
High
Priority
Low
Priority
Normal
Priority
Unbreak Now!
Status
Archived
Status
Confirmed
Status
Duplicate
Status
Needs Info from Developers
Status
Needs Information from User
Status
Needs Triage
Status
Resolved
Type
Bug
Type
Design
Type
Known Issue
Type
Patch
Type
Report
Type
To Do
No Milestone
No Assignees
41 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: blender/blender#96261
No description provided.