GPU: GLSL compilation as C++ for gpu static shaders #128724
No reviewers
Labels
No Label
Interest
Alembic
Interest
Animation & Rigging
Interest
Asset System
Interest
Audio
Interest
Automated Testing
Interest
Blender Asset Bundle
Interest
BlendFile
Interest
Code Documentation
Interest
Collada
Interest
Compatibility
Interest
Compositing
Interest
Core
Interest
Cycles
Interest
Dependency Graph
Interest
Development Management
Interest
EEVEE
Interest
Freestyle
Interest
Geometry Nodes
Interest
Grease Pencil
Interest
ID Management
Interest
Images & Movies
Interest
Import Export
Interest
Line Art
Interest
Masking
Interest
Metal
Interest
Modeling
Interest
Modifiers
Interest
Motion Tracking
Interest
Nodes & Physics
Interest
OpenGL
Interest
Overlay
Interest
Overrides
Interest
Performance
Interest
Physics
Interest
Pipeline, Assets & IO
Interest
Platforms, Builds & Tests
Interest
Python API
Interest
Render & Cycles
Interest
Render Pipeline
Interest
Sculpt, Paint & Texture
Interest
Text Editor
Interest
Translations
Interest
Triaging
Interest
Undo
Interest
USD
Interest
User Interface
Interest
UV Editing
Interest
VFX & Video
Interest
Video Sequencer
Interest
Viewport & EEVEE
Interest
Virtual Reality
Interest
Vulkan
Interest
Wayland
Interest
Workbench
Interest: X11
Legacy
Asset Browser Project
Legacy
Blender 2.8 Project
Legacy
Milestone 1: Basic, Local Asset Browser
Legacy
OpenGL Error
Meta
Good First Issue
Meta
Papercut
Meta
Retrospective
Meta
Security
Module
Animation & Rigging
Module
Core
Module
Development Management
Module
Grease Pencil
Module
Modeling
Module
Nodes & Physics
Module
Pipeline, Assets & IO
Module
Platforms, Builds & Tests
Module
Python API
Module
Render & Cycles
Module
Sculpt, Paint & Texture
Module
Triaging
Module
User Interface
Module
VFX & Video
Module
Viewport & EEVEE
Platform
FreeBSD
Platform
Linux
Platform
macOS
Platform
Windows
Severity
High
Severity
Low
Severity
Normal
Severity
Unbreak Now!
Status
Archived
Status
Confirmed
Status
Duplicate
Status
Needs Info from Developers
Status
Needs Information from User
Status
Needs Triage
Status
Resolved
Type
Bug
Type
Design
Type
Known Issue
Type
Patch
Type
Report
Type
To Do
No Milestone
No project
No Assignees
3 Participants
Notifications
Due Date
No due date set.
Dependencies
No dependencies set.
Reference: blender/blender#128724
Loading…
Reference in New Issue
Block a user
No description provided.
Delete Branch "fclem/blender:glsl-cpp-compilation"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Allow compilation of shaders using C++ for linting and
IDE support.
Related #127983
@blender-bot build
@ -915,0 +922,4 @@
)
cmake_policy(SET CMP0119 NEW)
set_source_files_properties(intern/shaders/draw_debug_print_display_vert.glsl LANGUAGE CXX)
set_source_files_properties(intern/shaders/draw_debug_print_display_vert.glsl PROPERTIES LANGUAGE CXX)
without the
PROPERTIES
keyword, cmake won't propagate it being a C++ file to msvc, however once you do about 40 of these guys show up (at different locations, not 40x this one)1>K:\BlenderGit\blender\source\blender\draw\intern\shaders\draw_debug_info.hh(17,1): error C2466: cannot allocate an array of constant size 0
40cae65002
tod522f048b3
d522f048b3
toec03cb6e28
@ -0,0 +81,4 @@
add_definitions(-DGPU_SHADER)
function(compile_sources_as_cpp
@LazyDodo I couldn't make it to work if I define this in
macro.cmake
.Is it ok to copy this a few time (one time per shader folder)?
@ -875,0 +879,4 @@
option(WITH_GPU_SHADER_CPP_COMPILATION "\
Compiler shaders using C++. \
Allows testing Metal compilation on other platform and enable C++ IDE support for shader code"
ON
TODO: Turn off. Only here for checking on the build bot.
WIP: GPU: GLSL compilation as C++to GPU: GLSL compilation as C++ for gpu static shaders@blender-bot build
@ -96,2 +96,4 @@
endif()
# Allow to specify language per file.
if(POLICY CMP0119)
Is this ok to put here or should it be enabled local?
@blender-bot build
I see nothing wrong with the build system changes, everything else is pretty far out of my normal area and i don't feel qualified to review.
It's not working on my end (VSCode, msvc, default C++ extension pack) :(
It looks like it's not catching the
GPU_SHADER
define, if I define it manually it works:I've tried defining it with
target_compile_definitions
but it didn't do the trick.Aside from that, I had to update the
STORAGE_BUF_FREQ
definition toqualifiers type_name name = {0}
, otherwise compilation fails.I'm not sold on requiring a "fake" library compilation to get Intellisense working, to be honest.
It adds to the compile times and prevents the build from finishing if any of the hacks fail.
Does CMAKE config has the
CMAKE_EXPORT_COMPILE_COMMAND
enables and does the C++ extension picks it up? (you might have to trash the intellisense cache)What does the content of your
${BUILD_FOLDER}/compile_commands.json
looks like for the GLSL C++ compilation? (you can attach it here and I'll look at it).The initial target was intellisense support, but the added benefit of correct compilation are much bigger. Once it is setup, you have static analysis & clang-tidy working and useful warnings.
The hacks failing is because not every cases were tested yet and we are still making it work. I don't think this will have a very high maintenance cost and fixing it is quite easy as you saw.
It's enabled by default, but it looks like it does nothing for msvc:
https://cmake.org/cmake/help/latest/variable/CMAKE_EXPORT_COMPILE_COMMANDS.html
So I don't have a
compile_commands.json
at all.Interesting. So I guess we should add a hint inside this new
WITH_GPU_SHADER_CPP_COMPILATION
that its only compatible with Ninja on windows.for VS Code you should be able to use the ninja generator on windows, it likely be the preferred generator for such a setup.
For what it's worth, gave it a whirl with the normal VS IDE (Not VS Code) it builds (kinda) fine, but no intellisense/code completion is available.
ran into the same issue with
STORAGE_BUF_FREQ
as @pragma37 on top of thatAlso when reviewing this i assumed it was just a single file for testing, but as this is now seems to be coming closer to landing time, it only builds a single file per subproject currently, should there have been more of them?
Passing the files as a string fixes the issue and makes IntelliSense work:
But now I'm drowned in compilation errors. 😫
Please attach them here so I can have a look.
Ok, after updating
STORAGE_BUF_FREQ
toqualifiers type_name name = {type_name()};
I no longer get errors, but I get a gigantic list of warnings (mostly duplicated definitions).I think you should get those too if you fix your
compile_sources_as_cpp
call.BTW, for files that have to be included everywhere I would use precompiled headers:
target_precompile_headers(${executable} PRIVATE "gpu_glsl_cpp_stubs.hh")
historically we've only used that when there was a significant savings to be had (ie shave minutes off a build) i don't see precompiled being useful here, only complicating the build without any tangible benefits
most of the warns are because of type_traits somehow including
math.h
andfma
haivng a different calling convention there, naively sidestepping it insidegpu_glsl_cpp_stubs.hh
like this seems to quiet things down a lot.The tangible benefit is not having to add an
#include "gpu_glsl_cpp_stubs.hh"
to every singleinfo.hh
andlib.glsl
file, and not getting weird compiler errors when someone inevitably forgets to do so when they have theWITH_GPU_SHADER_CPP_COMPILATION
option disabled.What's the downside of using precompiled headers in this case?
The same could be done with
/FI
msvc and-include
gcc where the behaviour is explicitly wanted, rather than a side effect you're exploiting.What about clang?
I am not sure about hiding includes inside the build system itself. It would confuse external devs even more than the current duplicated include.
Moreover, often times, the infos file need more (defines + other info includes) than just this file to be made compatible. So I am not sure how much we would be saving here.
clang will also take
-include
on forced includes vs explicit includes, i have no horse in the race but i could see some code completion implementations get confused with the pch/forced include way of things.I would normally agree, but this is just emulating GLSL built-in functions that are always available.
Having to explicitly include empty stubs because of a custom build-system hack only makes it more confusing, IMO, and certainly less convenient.
Code-completion being confused by it is a good point, but unless we have a specific example it's only a "what if".
Heh, yeah soo... fun story, it's more of a "what if" since code completion (on these files) doesn't work at all in the VS IDE , so it doesn't even get to the point it would work badly :)
one for the docs : it'll work if you set
Tools -> Options -> Text Editor -> File Extension -> Extension: glsl, Editor: Microsoft Visual C++
and close/reopen any open glsl tabs.looks like you'll may have to be careful with the comments you leave though, as they show up in the tooltips.
It is not 100% just emulating GLSL built-in. It is supposed to also have all the definitions inside
mtl/glsl_shader_defines.glsl/msl
. I am also looking into making theses include explicit in the future (except for pyGPU shader of course).Adding the header to all infos can be done using a common header which would look like this:
This can be done once we sort the dependency issues and add
#pragma once
to the info files. That wouldn't remove the need for adding#define
s at the top of each info file. So that could be a follow up task.About having to include gpu_glsl_cpp_stubs.hh to the GLSL lib, it only needs to be included in file with no other lib included. Since it is included in very common lib like
gpu_shader_math_lib.glsl
it is already added to a good portion of the codebase.About if someone forgets to put the include:
I am not sure what you are referring to. If the cpp shader fails to build because
vec2
is not a recognized symbol as well as pretty much everything, I think the error is easy to understand.@blender-bot build