Vulkan: Validation layers potential licensing risks. #102055

New Issue

@LazyDodo mentioned a potential licensing risk when using validation layers with vulkan.
Although they are normally used as debug layer and would be normally disabled, the way how they are designed might have a potential security breach.

We should discuss with Ton to make sure this does not backfire.

Vulkan tries to validate everything during develop time. When released validations are and should be disabled. To configure this vulkan uses validation layers.
In normal usage Blender will request the validation layers that they need via vkCreateInstance.

But it also has an option to enable validation layers from outside blender by an environment variable.
VK_INSTANCE_LAYERS=VK_LAYER_LUNARG_standard_validation ./blender.

So there are 2 ways how non compatible GPL code can be run inside blender.

Hijack an existing vk layer name that blender uses to load hostile code.
Using an environment variable.

What could this hostile code do:
From the looks of it, validation layers can perform report callbacks to blender registered callbacks.

VKAPI_ATTR VkBool32 VKAPI_CALL MyDebugReportCallback(
    VkDebugReportFlagsEXT       flags,
    VkDebugReportObjectTypeEXT  objectType,
    uint64_t                    object,
    size_t                      location,
    int32_t                     messageCode,
    const char*                 pLayerPrefix,
    const char*                 pMessage,
    void*                       pUserData)
{
    std::cerr << pMessage << std::endl;
    return VK_FALSE;
}

So we should only register these callbacks when we actually want to use it and not by default.
The code inside the report callbacks should be to the point and only provide logging.
During normal execution of blender no callbacks should be enabled, and we might want to return VK_TRUE as that should abort the running command.

From https://gpuopen.com/learn/using-the-vulkan-validation-layers/:

The return value of the callback is a Boolean that indicates to the validation layers whether the API call that triggered the debug report callback should be aborted or not. However, developers have to be aware that in case an error is reported by one of the validation layers it’s an indication that something invalid was being attempted by the application thus any operation following the error might result in undefined behavior or even a crash. As such, it’s advised that developers stop at the first error and try to resolve that before making any assumptions about the behavior of subsequent operations. Think about validation errors in the same way like errors reported by compilers: often subsequent errors are just consequences of the first one.```

@LazyDodo mentioned a potential licensing risk when using validation layers with vulkan. Although they are normally used as debug layer and would be normally disabled, the way how they are designed might have a potential security breach. We should discuss with Ton to make sure this does not backfire. Vulkan tries to validate everything during develop time. When released validations are and should be disabled. To configure this vulkan uses validation layers. In normal usage Blender will request the validation layers that they need via `vkCreateInstance`. But it also has an option to enable validation layers from outside blender by an environment variable. `VK_INSTANCE_LAYERS=VK_LAYER_LUNARG_standard_validation ./blender`. So there are 2 ways how non compatible GPL code can be run inside blender. 1. Hijack an existing vk layer name that blender uses to load hostile code. 2. Using an environment variable. What could this hostile code do: From the looks of it, validation layers can perform report callbacks to blender registered callbacks. ``` VKAPI_ATTR VkBool32 VKAPI_CALL MyDebugReportCallback( VkDebugReportFlagsEXT flags, VkDebugReportObjectTypeEXT objectType, uint64_t object, size_t location, int32_t messageCode, const char* pLayerPrefix, const char* pMessage, void* pUserData) { std::cerr << pMessage << std::endl; return VK_FALSE; } ``` So we should only register these callbacks when we actually want to use it and not by default. The code inside the report callbacks should be to the point and only provide logging. During normal execution of blender no callbacks should be enabled, and we might want to return `VK_TRUE` as that should abort the running command. > From https://gpuopen.com/learn/using-the-vulkan-validation-layers/: ``` The return value of the callback is a Boolean that indicates to the validation layers whether the API call that triggered the debug report callback should be aborted or not. However, developers have to be aware that in case an error is reported by one of the validation layers it’s an indication that something invalid was being attempted by the application thus any operation following the error might result in undefined behavior or even a crash. As such, it’s advised that developers stop at the first error and try to resolve that before making any assumptions about the behavior of subsequent operations. Think about validation errors in the same way like errors reported by compilers: often subsequent errors are just consequences of the first one.```

Changed status from 'Needs Triage' to: 'Confirmed'

Added subscribers: @LazyDodo, @Jeroen-Bakker, @fclem, @brecht

I created this task as we should validate and check with Ton. But as it is close to the conference I added a ticket to keep track of it.

Added subscriber: @newin

I haven't tried using them in this way myself as I find more convenient to declare those in the code directly but it seems there is a way to enable validation layers in a non-intrusive way using vkconfig shipped with the LunarG SDK:
https://vulkan.lunarg.com/doc/view/1.3.224.1/windows/vkconfig.html

If it work like I understand it does you don't have to ship anything with validation layers.

I hope it solve your issue.

I haven't tried using them in this way myself as I find more convenient to declare those in the code directly but it seems there is a way to enable validation layers in a non-intrusive way using vkconfig shipped with the LunarG SDK: https://vulkan.lunarg.com/doc/view/1.3.224.1/windows/vkconfig.html If it work like I understand it does you don't have to ship anything with validation layers. I hope it solve your issue.

I don't understand what the issue is here, regarding licensing or security.

Environment variables like LD_PRELOAD can already be used to change which libraries are loaded by an application, Python add-ons can load arbitrary shared libraries, etc. Enforcing the licensing is not something we can or should do at a technical level.

In terms of security, if an attacker can set environment variables the machine is fully compromised already and there's nothing we can do to stop it.

I don't understand what the issue is here, regarding licensing or security. Environment variables like `LD_PRELOAD` can already be used to change which libraries are loaded by an application, Python add-ons can load arbitrary shared libraries, etc. Enforcing the licensing is not something we can or should do at a technical level. In terms of security, if an attacker can set environment variables the machine is fully compromised already and there's nothing we can do to stop it.

Added subscriber: @Ton

Actually i think it's important to add some context here, the last time this came up was when Academy Software Foundation started hosting openFX and @Ton was "less than thrilled" with the idea of loading possibly closed sourced addons. Given OpenFX and Vulkan are identical in this regard (open spec, GPL compatible loader library, possibly closed source addons, also opensource ones) i figured i'd give you a nudge to go talk to ton, to be sure he's OK with this. Wasn't expecting this to play out in public.

I'm with brecht here, i'm not seeing licensing or security issues

Actually i think it's important to add some context here, the last time this came up was when [Academy Software Foundation started hosting openFX](https://www.aswf.io/news/academy-software-foundation-adds-openfx-image-processing-standard-as-newest-hosted-project/) and @Ton was "less than thrilled" with the idea of loading possibly closed sourced addons. Given OpenFX and Vulkan are identical in this regard (open spec, GPL compatible loader library, possibly closed source addons, also opensource ones) i figured i'd give you a nudge to go talk to ton, to be sure he's OK with this. Wasn't expecting this to play out in public. I'm with brecht here, i'm not seeing licensing or security issues

Added subscriber: @Emi_Martinez

Added subscriber: @Raimund58

Added subscriber: @ckohl_art

Added subscriber: @vnapdv

I checked with Ton about this Topic and he didn't see an issue with using vulkan validation layers even when they are closed source.
Validation layers will be used to increase the quality of Blenders source code during development and issue solving and will not add user oriented benefits.

Just to clarify the process in the future, bf-admins would be in the lead for these kind of topics.

I checked with Ton about this Topic and he didn't see an issue with using vulkan validation layers even when they are closed source. Validation layers will be used to increase the quality of Blenders source code during development and issue solving and will not add user oriented benefits. Just to clarify the process in the future, bf-admins would be in the lead for these kind of topics.

Added subscriber: @GeorgiaPacific

Download

What's New

Blender Studio

Manual

Developers Blog

Documentation

Benchmark

Blender Conference

Development Fund

One-time Donations

Vulkan: Validation layers potential licensing risks. #102055