Blender 3.5 issues migrating code from bgl to gpu on Mac M1 Metal backend #107332

New Issue

Andrej · 2023-04-25T14:35:41+02:00

Andrej commented

2023-04-25 14:35:41 +02:00

I've met some issues migrating shader code from bgl to gpu.

Migrating it on windows and linux worked fine - I either simply replaced all built-in line shaders I was using with "3D_POLYLINE_UNIFORM_COLOR" or rewrote all custom shaders having the same polyline principle in mind (basically creating a two triangles through geo shader for each line to support smoothing).

Migrating on Mac didn't worked that well:

When shader doesn't compile on Mac you do get Exception: Shader Compile Error, see console for more details but can't see any details in console besides that. I got more info providing --debug-gpu command line argument but it gives me too much info. Shouldn't it provide compile errors message even without debug mode?
Running script simple_shader.py (attached below) either gives me this error if I run it without geometry shader part

WARN (gpu.debug.metal): [Metal Viewport Warning] : Unable to compile shader 0x118021008 'pyGPUShader' as no create-info was provided!

or if I run it with geometry shader (uncommenting geocode=polyline_geom_glsl)

ERROR (gpu.debug.metal): [Metal Viewport Error] : MTLShader::geometry_shader_from_glsl - Geometry shaders unsupported!
WARN (gpu.debug.metal): [Metal Viewport Warning] : Unable to compile shader 0x118021008 'pyGPUShader' as no create-info was provided!

Is it expected that I cannot create shader with just gpu.types.GPUShader on Metal or it's a bug?

Since it's saying that it cannot find "create-info" I've tried to write shader with gpu.types.GPUShaderCreateInfo and gpu.shader.create_from_info (attached example create_info.py below) and it actually worked on Mac.
But what do I do about the geometry shader - is there some workaround or gpu geometry shaders are not supported at all? Still need geometry shader to either create custom things in shader or to just perfom line smoothing using polyline method.
Line created with create-info method is still a bit different on Mac from what I've got in Windows - on Mac it appear less thick (tried to change it with gpu.state.line_width_set but it had no effect).

simple_shader.py:

import bpy
import gpu
from gpu_extras.batch import batch_for_shader


polyline_vert_glsl = """
uniform mat4 ModelViewProjectionMatrix;
uniform vec4 color;

in vec3 pos;

void main()
{
  gl_Position = ModelViewProjectionMatrix * vec4(pos, 1.0);
}
"""

polyline_geom_glsl = """

layout(lines) in;
layout (line_strip, max_vertices = 4) out;

void main(void)
{
  gl_Position = gl_in[0].gl_Position;
  EmitVertex();
  
  gl_Position = gl_in[1].gl_Position;
  EmitVertex();

  EndPrimitive();
}
"""

polyline_frag_glsl = """

layout(location = 0) out vec4 fragColor;

#pragma BLENDER_REQUIRE(gpu_shader_colorspace_lib.glsl)

void main()
{
  fragColor = vec4(1.0, 0.0, 0.0, 1.0);
}
"""

from mathutils import Vector
coords = [(1, 1, 1), (-2, 0, 0), (-2, -1, 3), (0, 1, 1)]
offset_vector = Vector([0,0,1])
coords = [Vector(c) - offset_vector for c in coords]

def draw():
    print('DRAW------------------')
    shader = gpu.types.GPUShader(
        polyline_vert_glsl, 
        polyline_frag_glsl, 
        # geocode=polyline_geom_glsl
    )
    batch = batch_for_shader(shader, 'LINES', {"pos": coords})
    shader.bind()

    gpu.state.blend_set("ALPHA")
    batch.draw(shader)


bpy.types.SpaceView3D.draw_handler_add(draw, (), 'WINDOW', 'POST_VIEW')

create_info.py

import bpy
import gpu
from gpu_extras.batch import batch_for_shader


polyline_vert_glsl = """
void main()
{
  gl_Position = ModelViewProjectionMatrix * vec4(pos, 1.0);
}
"""

polyline_frag_glsl = """
void main()
{
  fragColor = vec4(1.0, 1.0, 0.0, 1.0);
}
"""

from mathutils import Vector
coords = [(1, 1, 1), (-2, 0, 0), (-2, -1, 3), (0, 1, 1)]
offset_vector = Vector([0,0,1])
coords = [Vector(c) - offset_vector for c in coords]

def draw():
    print('DRAW------------------')
    shader_info = gpu.types.GPUShaderCreateInfo()
    shader_info.vertex_in(0, 'VEC3', 'pos')
    shader_info.push_constant('MAT4', "ModelViewProjectionMatrix")
    shader_info.fragment_out(0, 'VEC4', 'fragColor')
    shader_info.vertex_source(polyline_vert_glsl)
    shader_info.fragment_source(polyline_frag_glsl)
    shader = gpu.shader.create_from_info(shader_info)

    batch = batch_for_shader(shader, 'LINES', {"pos": coords})
    shader.bind()
    gpu.state.blend_set("ALPHA")
    batch.draw(shader)

bpy.types.SpaceView3D.draw_handler_add(draw, (), 'WINDOW', 'POST_VIEW')

I've met some issues migrating shader code from bgl to gpu. Migrating it on windows and linux worked fine - I either simply replaced all built-in line shaders I was using with "3D_POLYLINE_UNIFORM_COLOR" or rewrote all custom shaders having the same polyline principle in mind (basically creating a two triangles through geo shader for each line to support smoothing). Migrating on Mac didn't worked that well: 1. When shader doesn't compile on Mac you do get `Exception: Shader Compile Error, see console for more details` but can't see any details in console besides that. I got more info providing `--debug-gpu` command line argument but it gives me too much info. Shouldn't it provide compile errors message even without debug mode? 2. Running script `simple_shader.py` (attached below) either gives me this error if I run it without geometry shader part ``` WARN (gpu.debug.metal): [Metal Viewport Warning] : Unable to compile shader 0x118021008 'pyGPUShader' as no create-info was provided! ``` or if I run it with geometry shader (uncommenting `geocode=polyline_geom_glsl`) ``` ERROR (gpu.debug.metal): [Metal Viewport Error] : MTLShader::geometry_shader_from_glsl - Geometry shaders unsupported! WARN (gpu.debug.metal): [Metal Viewport Warning] : Unable to compile shader 0x118021008 'pyGPUShader' as no create-info was provided! ``` Is it expected that I cannot create shader with just `gpu.types.GPUShader` on Metal or it's a bug? 3. Since it's saying that it cannot find "create-info" I've tried to write shader with `gpu.types.GPUShaderCreateInfo` and `gpu.shader.create_from_info` (attached example `create_info.py` below) and it actually worked on Mac. But what do I do about the geometry shader - is there some workaround or gpu geometry shaders are not supported at all? Still need geometry shader to either create custom things in shader or to just perfom line smoothing using polyline method. 4. Line created with create-info method is still a bit different on Mac from what I've got in Windows - on Mac it appear less thick (tried to change it with `gpu.state.line_width_set` but it had no effect). simple_shader.py: ```python import bpy import gpu from gpu_extras.batch import batch_for_shader polyline_vert_glsl = """ uniform mat4 ModelViewProjectionMatrix; uniform vec4 color; in vec3 pos; void main() { gl_Position = ModelViewProjectionMatrix * vec4(pos, 1.0); } """ polyline_geom_glsl = """ layout(lines) in; layout (line_strip, max_vertices = 4) out; void main(void) { gl_Position = gl_in[0].gl_Position; EmitVertex(); gl_Position = gl_in[1].gl_Position; EmitVertex(); EndPrimitive(); } """ polyline_frag_glsl = """ layout(location = 0) out vec4 fragColor; #pragma BLENDER_REQUIRE(gpu_shader_colorspace_lib.glsl) void main() { fragColor = vec4(1.0, 0.0, 0.0, 1.0); } """ from mathutils import Vector coords = [(1, 1, 1), (-2, 0, 0), (-2, -1, 3), (0, 1, 1)] offset_vector = Vector([0,0,1]) coords = [Vector(c) - offset_vector for c in coords] def draw(): print('DRAW------------------') shader = gpu.types.GPUShader( polyline_vert_glsl, polyline_frag_glsl, # geocode=polyline_geom_glsl ) batch = batch_for_shader(shader, 'LINES', {"pos": coords}) shader.bind() gpu.state.blend_set("ALPHA") batch.draw(shader) bpy.types.SpaceView3D.draw_handler_add(draw, (), 'WINDOW', 'POST_VIEW') ``` create_info.py ```python import bpy import gpu from gpu_extras.batch import batch_for_shader polyline_vert_glsl = """ void main() { gl_Position = ModelViewProjectionMatrix * vec4(pos, 1.0); } """ polyline_frag_glsl = """ void main() { fragColor = vec4(1.0, 1.0, 0.0, 1.0); } """ from mathutils import Vector coords = [(1, 1, 1), (-2, 0, 0), (-2, -1, 3), (0, 1, 1)] offset_vector = Vector([0,0,1]) coords = [Vector(c) - offset_vector for c in coords] def draw(): print('DRAW------------------') shader_info = gpu.types.GPUShaderCreateInfo() shader_info.vertex_in(0, 'VEC3', 'pos') shader_info.push_constant('MAT4', "ModelViewProjectionMatrix") shader_info.fragment_out(0, 'VEC4', 'fragColor') shader_info.vertex_source(polyline_vert_glsl) shader_info.fragment_source(polyline_frag_glsl) shader = gpu.shader.create_from_info(shader_info) batch = batch_for_shader(shader, 'LINES', {"pos": coords}) shader.bind() gpu.state.blend_set("ALPHA") batch.draw(shader) bpy.types.SpaceView3D.draw_handler_add(draw, (), 'WINDOW', 'POST_VIEW') ```

createinfo_mac.png

1.1 MiB

createinfo_windows.png

163 KiB

Andrej added the

labels 2023-04-25 14:35:41 +02:00

Germano Cavalcante commented

2023-04-26 16:27:35 +02:00

Thanks for the report, but this is the same issue described in #106304.

Unfortunately we cannot use the same GLSL code (for OpenGL), in the Metal or the future Vulkan backend.

Therefore a new API was implemented. You can see examples of it at:
https://docs.blender.org/api/current/gpu.html#triangle-with-custom-shader

Try adapting your shader to use gpu.shader.create_from_info.

It is planned to improve the documentation and add deprecated messages to avoid further confusion.

Thanks for the report, but this is the same issue described in #106304. Unfortunately we cannot use the same GLSL code (for OpenGL), in the Metal or the future Vulkan backend. Therefore a new API was implemented. You can see examples of it at: https://docs.blender.org/api/current/gpu.html#triangle-with-custom-shader Try adapting your shader to use [gpu.shader.create_from_info](https://docs.blender.org/api/current/gpu.shader.html#gpu.shader.create_from_info). It is planned to improve the documentation and add deprecated messages to avoid further confusion.

Germano Cavalcante closed this issue

2023-04-26 16:27:37 +02:00

Blender Bot added

and removed

labels 2023-04-26 16:27:39 +02:00

Germano Cavalcante referenced this issue

2023-04-26 16:29:43 +02:00

PyGPU: Deprecate shader creation through direct GLSL code input #107374

Andrej commented

2023-04-26 16:37:09 +02:00

@mano-wii but is there any way to implement geometry shaders on Metal now or it is still in development?
At the moment GPUShaderCreateInfo doesn't support geometry shaders
Without geometry shaders there is currently no way to implement smoothed line shaders or something more complex on Metal :\

@mano-wii but is there any way to implement geometry shaders on Metal now or it is still in development? At the moment `GPUShaderCreateInfo` doesn't support geometry shaders Without geometry shaders there is currently no way to implement smoothed line shaders or something more complex on Metal :\

Michael Parkin-White commented

2023-04-26 16:40:13 +02:00

To also comment on Geometry shader support, Metal does not have this.
However, there is an option supported by the Metal backend which allows random-access reading of the vertex shader data, allowing you to use information from other/neighbouring vertices within a vertex shader.

This is planned to be a common API, lifted to the high-level such that it can be used for all APIs, and is not recommend for use at the moment, however, if functionality is essential, a similar approach to source/blender/gpu/shaders/gpu_shader_3D_polyline_vert_no_geom.glsl can be used for wideline rendering.

Adding #pragma USE_SSBO_VERTEX_FETCH(TriangleList, 6) in your vertex shader will inform the backend to issue a differing output primitive type, with a set vertex count per input primitive.

i.e. in this case, if your source data is a line shader, then having the settings as above will issue a vertex shader invocation outputting two triangles (6 vertices using triangleList primitive type) for each input line.

vertex_fetch_attribute(vertex_index, attribute_name, data type) can then be used to read the value.

Again, I do not recommend that you use this, but it is an option if required.

To also comment on Geometry shader support, Metal does not have this. However, there is an option supported by the Metal backend which allows random-access reading of the vertex shader data, allowing you to use information from other/neighbouring vertices within a vertex shader. This is planned to be a common API, lifted to the high-level such that it can be used for all APIs, and is not recommend for use at the moment, however, if functionality is essential, a similar approach to `source/blender/gpu/shaders/gpu_shader_3D_polyline_vert_no_geom.glsl` can be used for wideline rendering. Adding `#pragma USE_SSBO_VERTEX_FETCH(TriangleList, 6)` in your vertex shader will inform the backend to issue a differing output primitive type, with a set vertex count per input primitive. i.e. in this case, if your source data is a line shader, then having the settings as above will issue a vertex shader invocation outputting two triangles (6 vertices using triangleList primitive type) for each input line. `vertex_fetch_attribute(vertex_index, attribute_name, data type)` can then be used to read the value. Again, I do not recommend that you use this, but it is an option if required.

Sign in to join this conversation.

No Label

Download

What's New

Blender Studio

Manual

Developers Blog

Documentation

Benchmark

Blender Conference

Development Fund

One-time Donations

Blender 3.5 issues migrating code from bgl to gpu on Mac M1 Metal backend #107332