Vulkan: Low Precision Float Conversion #108168

This PR adds conversion template to convert between Low Precision float
formats. These include Binary32 floats and lower. It also adds support
to convert between unsigned and signed float formats and float formats
with different mantissa and exponents.

Additionally overflows (values that don't fit in the target float
format) will be clamped to the maximum value.

Reasoning:
Up to now the Vulkan backend only supported float and half float
formats, but to support workbench 11 and 10 unsigned floats have to be
supported as well. The available libraries that support those float
formats targets scientific applications. Where the final code couldn't
be optimized that well by the compiler.

Data conversion for color pixels have different requirements about
clamping and sign, what could eliminate some clamping code in other
areas in Blender as well. Also could fix some undesired overflow when
using pixels with high intensity that didn't fit in the texture format
leading to artifects in Eevee and slow-down in the image editor.

Future
In the future we might want to move this to the public part of the GPU
module so we can use this as well in other areas (Metal backend), Imbuf clamping
See 3c658d2c2e for a commit that uses
this and improves image editor massively as it doesn't need to reiterate over
the image buffer to clamp the values into a known range.

This PR adds conversion template to convert between Low Precision float formats. These include Binary32 floats and lower. It also adds support to convert between unsigned and signed float formats and float formats with different mantissa and exponents. Additionally overflows (values that don't fit in the target float format) will be clamped to the maximum value. **Reasoning**: Up to now the Vulkan backend only supported float and half float formats, but to support workbench 11 and 10 unsigned floats have to be supported as well. The available libraries that support those float formats targets scientific applications. Where the final code couldn't be optimized that well by the compiler. Data conversion for color pixels have different requirements about clamping and sign, what could eliminate some clamping code in other areas in Blender as well. Also could fix some undesired overflow when using pixels with high intensity that didn't fit in the texture format leading to artifects in Eevee and slow-down in the image editor. **Future** In the future we might want to move this to the public part of the GPU module so we can use this as well in other areas (Metal backend), Imbuf clamping See 3c658d2c2e69e9cf97dfaa7a3c164262aefb9e76 for a commit that uses this and improves image editor massively as it doesn't need to reiterate over the image buffer to clamp the values into a known range.

Code that clang generates includes branches. But as most of them are predictable branches and don't need access to different memory it seems to be ok. Note that optimizations to use vectorization could be added later. The idea to add it is to generate shifts and masks between two float formats and use them inside the conversion routine.

convert_f32_to_f16(unsigned int):                # @convert_f32_to_f16(unsigned int)
        mov     ecx, edi
        and     ecx, 8388607
        mov     eax, edi
        shr     eax, 23
        movzx   edx, al
        mov     eax, edx
        or      eax, ecx
        je      .LBB0_1
        xor     esi, esi
        cmp     edx, 255
        mov     r8d, 1023
        mov     eax, 1023
        cmove   eax, esi
        test    ecx, ecx
        cmovne  eax, r8d
        mov     r8d, 31744
        cmp     edx, 255
        je      .LBB0_7
        mov     eax, 8388607
        cmp     edx, 142
        ja      .LBB0_6
        xor     eax, eax
        cmp     edx, 112
        jb      .LBB0_8
        add     edx, -127
        mov     esi, edx
        mov     eax, ecx
.LBB0_6:
        shr     eax, 13
        shl     esi, 10
        add     esi, 15360
        mov     r8d, esi
.LBB0_7:
        shr     edi, 16
        and     edi, 32768
        or      edi, eax
        or      edi, r8d
        mov     eax, edi
.LBB0_8:
        ret
.LBB0_1:
        xor     eax, eax
        ret

convert_f32_to_f11(unsigned int):                # @convert_f32_to_f11(unsigned int)
        test    edi, edi
        setns   sil
        mov     ecx, edi
        and     ecx, 8388607
        shr     edi, 23
        movzx   edx, dil
        cmp     edx, 255
        sete    dil
        test    ecx, ecx
        setne   r8b
        xor     eax, eax
        mov     r9d, edx
        or      r9d, ecx
        je      .LBB0_8
        and     r8b, dil
        or      sil, r8b
        je      .LBB0_8
        xor     edi, edi
        cmp     edx, 255
        mov     eax, 63
        mov     esi, 63
        cmove   esi, edi
        test    ecx, ecx
        cmovne  esi, eax
        mov     eax, 1984
        cmp     edx, 255
        je      .LBB0_7
        mov     esi, 8388607
        cmp     edx, 142
        ja      .LBB0_6
        xor     eax, eax
        cmp     edx, 112
        jb      .LBB0_8
        add     edx, -127
        mov     edi, edx
        mov     esi, ecx
.LBB0_6:
        shr     esi, 17
        shl     edi, 6
        add     edi, 960
        mov     eax, edi
.LBB0_7:
        or      eax, esi
.LBB0_8:
        ret

Code that clang generates includes branches. But as most of them are predictable branches and don't need access to different memory it seems to be ok. Note that optimizations to use vectorization could be added later. The idea to add it is to generate shifts and masks between two float formats and use them inside the conversion routine. ```asm convert_f32_to_f16(unsigned int): # @convert_f32_to_f16(unsigned int) mov ecx, edi and ecx, 8388607 mov eax, edi shr eax, 23 movzx edx, al mov eax, edx or eax, ecx je .LBB0_1 xor esi, esi cmp edx, 255 mov r8d, 1023 mov eax, 1023 cmove eax, esi test ecx, ecx cmovne eax, r8d mov r8d, 31744 cmp edx, 255 je .LBB0_7 mov eax, 8388607 cmp edx, 142 ja .LBB0_6 xor eax, eax cmp edx, 112 jb .LBB0_8 add edx, -127 mov esi, edx mov eax, ecx .LBB0_6: shr eax, 13 shl esi, 10 add esi, 15360 mov r8d, esi .LBB0_7: shr edi, 16 and edi, 32768 or edi, eax or edi, r8d mov eax, edi .LBB0_8: ret .LBB0_1: xor eax, eax ret ``` ```asm convert_f32_to_f11(unsigned int): # @convert_f32_to_f11(unsigned int) test edi, edi setns sil mov ecx, edi and ecx, 8388607 shr edi, 23 movzx edx, dil cmp edx, 255 sete dil test ecx, ecx setne r8b xor eax, eax mov r9d, edx or r9d, ecx je .LBB0_8 and r8b, dil or sil, r8b je .LBB0_8 xor edi, edi cmp edx, 255 mov eax, 63 mov esi, 63 cmove esi, edi test ecx, ecx cmovne esi, eax mov eax, 1984 cmp edx, 255 je .LBB0_7 mov esi, 8388607 cmp edx, 142 ja .LBB0_6 xor eax, eax cmp edx, 112 jb .LBB0_8 add edx, -127 mov edi, edx mov esi, ecx .LBB0_6: shr esi, 17 shl edi, 6 add edi, 960 mov eax, edi .LBB0_7: or eax, esi .LBB0_8: ret ```

 @ -99,0 +106,4 @@
 template<bool HasSignBit, uint8_t MantissaBitLen, uint8_t ExponentBitLen>
 class FloatingPointFormat {
  public:
   static constexpr bool HasSign = HasSignBit;

Download

What's New

Blender Studio

Manual

Developers Blog

Documentation

Benchmark

Blender Conference

Development Fund

One-time Donations

Vulkan: Low Precision Float Conversion #108168

Reviewers