Design: Changes required to target SSE42 #116592

New Issue

Ray molenkamp · 2023-12-28T01:35:41+01:00

Ray molenkamp commented

2023-12-28 01:35:41 +01:00

Introduction

In the admin meetings from 2023-09-12 were the following notes:

Bumping minimum CPU instruction set for 4.0:

Suggestion is to support SSE 4.2. It is very unlikely that computers that do not support this will have a GPU that supports OpenGL4.3 (Blender 4.x requirement)

Raising the requirement to AVX won’t bring that many immediate benefits and better wait.

Raising the requirement to AVX2 is not acceptable as it leaves behind too many artists

While this sounded very indecisive, it was actually decided to bump the blender requirements in that meeting, this was not communicated as such, nor was the platform module informed about this hence nothing has happened in this area for months.

Implications

Since the notes are bit vague, I'm unsure what was actually decided. The blender requirements page has been updated to mention sse4.2 though, so I assume the new target platform is actually x86-64-v2 this will need to be confirmed by the admins.

Changes Required

CMake

The current TEST_SSE_SUPPORT Macro can be updated to test for SSE42 rather than sse and sse2.

GCC/CLang

For linux for both gcc and clang we'd test for the availability of the -march:x86-64-v2 flag

MSVC

None, MSVC will not generate SSE4.2 code, if we were to use the sse42 intrinsics in our code it will happily generate the opcodes for it, but it will not generate sse42 code itself, the next step up it will support is the AVX platfom there. So the current flags of "no architecture flags" will remain, the remnants of 32 bit support in the TEST_SSE_SUPPORT macro can be cleaned up though.

Cycles

If (and that is a very much undecided if afaik) cycles wants to follow blender in bumping its minimum requirements the following changes are likely needed

intern/cycles/kernel/device/cpu/kernel.cpp can be updated to have SSE42 as the lower bar
intern/cycles/kernel/device/cpu/kernel_sse2.cpp can be removed
intern/cycles/kernel/device/cpu/kernel_sse42.cpp can be removed
tests for CYCLES_CPU_NO_SSE41 and CYCLES_CPU_NO_SSE2 can be removed
Update the python debug UI to remove these kernels
Likely more, these are just the most obvious changes

End user facing changes

Previous changes in the minimum (cough GPU cough) requirements of blender requirements have been met with and endless stream of bug reports of blender crashing at startup, for this change i very much would like to prevent this and have pre-flight check in place to determine if the current CPU is supported or not, and if not politely inform the user about this and gracefully exit the process.

A native way to implement this would be having the check on the first line of main() this however would be a mistake, as any initalizers and initializers from shared libraries will run long before execution even reaches the main function, hence SSe42 instructions will likely be executed before our check could run.

Now on windows blender has a blender-launcher binary to hide the console window from the end user, which would be a great place to put such a check, however, that assumes every user will be using the launcher, some will, some won't, some will just still be running blender.exe from a script since that is what they have always done.

on linux a launcher currently (afaik) does not exist, and even if we introduced one, few people would use it.

Proposed solution

both windows and linux (afaik please correct me if i'm wrong) will load/initalize shared libraries in the same order they were linked, which we can exploit, the following proof of concept code was done on linux since it's shorter (Windows gets kinda messy with its DLL main, but the same mechanism has been proved to work there)

if we were to make a small shared library with the following code, build it without sse42 flags, and it be the first thing blender links against, this should do the trick

/* SPDX-FileCopyrightText: 2023 Blender Authors
 *
 * SPDX-License-Identifier: GPL-2.0-or-later */

/** \file
 * \ingroup creator
 */

#include <cstdio>
#include <cstdlib>

static bool check_sse42() { return false; } // TODO: Implement this

static __attribute__ ((constructor)) void cpu_check(void)
{
  bool supported = check_sse42();
  if(supported)
  {
    printf("sse42 supported!\n");
  }
  else
  {
    printf("sse42 not supported exiting...\n");
    exit(-1);
  }
  return;
}

Ideally a more visual popup will be given rather than a printf but I'll leave this to the imagination of the platform dev for each platform.

# Introduction In the admin meetings from [2023-09-12](https://devtalk.blender.org/t/2023-09-12-blender-admins-meeting/) were the following notes: > Bumping minimum CPU instruction set for 4.0: > > - Suggestion is to support SSE 4.2. It is very unlikely that computers that do not support this will have a GPU that supports OpenGL4.3 (Blender 4.x requirement) > - Raising the requirement to AVX won’t bring that many immediate benefits and better wait. > - Raising the requirement to AVX2 is not acceptable as it leaves behind too many artists While this sounded very indecisive, it was actually decided to bump the blender requirements in that meeting, this was not communicated as such, nor was the platform module informed about this hence nothing has happened in this area for months. # Implications Since the notes are bit vague, I'm unsure what was actually decided. The blender requirements page has been updated to mention sse4.2 though, so I _assume_ the new target platform is actually [x86-64-v2](https://en.wikipedia.org/wiki/X86-64#Microarchitecture_levels) this will need to be confirmed by the admins. # Changes Required ## CMake The current [TEST_SSE_SUPPORT](https://projects.blender.org/blender/blender/src/branch/main/build_files/cmake/macros.cmake#L679) Macro can be updated to test for SSE42 rather than sse and sse2. ### GCC/CLang For linux for both gcc and clang we'd test for the availability of the `-march:x86-64-v2` flag ### MSVC None, MSVC will not generate SSE4.2 code, if we were to use the sse42 intrinsics in our code it will happily generate the opcodes for it, but it will not generate sse42 code itself, the next step up it will support is the AVX platfom there. So the current flags of "no architecture flags" will remain, the remnants of 32 bit support in the `TEST_SSE_SUPPORT` macro can be cleaned up though. ## Cycles _If_ (and that is a very much undecided if afaik) cycles wants to follow blender in bumping its minimum requirements the following changes are likely needed - [intern/cycles/kernel/device/cpu/kernel.cpp](https://projects.blender.org/blender/blender/src/branch/main/intern/cycles/kernel/device/cpu/kernel.cpp#L10) can be updated to have SSE42 as the lower bar - [intern/cycles/kernel/device/cpu/kernel_sse2.cpp](https://projects.blender.org/blender/blender/src/branch/main/intern/cycles/kernel/device/cpu/kernel_sse2.cpp) can be removed - [intern/cycles/kernel/device/cpu/kernel_sse42.cpp](https://projects.blender.org/blender/blender/src/branch/main/intern/cycles/kernel/device/cpu/kernel_sse42.cpp) can be removed - tests for `CYCLES_CPU_NO_SSE41` and `CYCLES_CPU_NO_SSE2` can be removed - Update the python debug UI to remove these kernels - Likely more, these are just the most obvious changes # End user facing changes Previous changes in the minimum (cough GPU cough) requirements of blender requirements have been met with and endless stream of bug reports of blender crashing at startup, for this change i very much would like to prevent this and have pre-flight check in place to determine if the current CPU is supported or not, and if not politely inform the user about this and gracefully exit the process. A native way to implement this would be having the check on the first line of `main()` this however would be a mistake, as any initalizers and initializers from shared libraries will run _long_ before execution even reaches the main function, hence SSe42 instructions will likely be executed before our check could run. Now on windows blender has a blender-launcher binary to hide the console window from the end user, which would be a great place to put such a check, however, that assumes every user will be using the launcher, some will, some won't, some will just still be running blender.exe from a script since that is what they have always done. on linux a launcher currently (afaik) does not exist, and even if we introduced one, few people would use it. ## Proposed solution both windows and linux (afaik please correct me if i'm wrong) will load/initalize shared libraries in the same order they were linked, which we can exploit, the following proof of concept code was done on linux since it's shorter (Windows gets kinda messy with its DLL main, but the same mechanism has been proved to work there) if we were to make a small shared library with the following code, build it without sse42 flags, and it be the _first_ thing blender links against, this _should_ do the trick ``` /* SPDX-FileCopyrightText: 2023 Blender Authors * * SPDX-License-Identifier: GPL-2.0-or-later */ /** \file * \ingroup creator */ #include <cstdio> #include <cstdlib> static bool check_sse42() { return false; } // TODO: Implement this static __attribute__ ((constructor)) void cpu_check(void) { bool supported = check_sse42(); if(supported) { printf("sse42 supported!\n"); } else { printf("sse42 not supported exiting...\n"); exit(-1); } return; } ``` Ideally a more visual popup will be given rather than a printf but I'll leave this to the imagination of the platform dev for each platform.

🎉 2

Ray molenkamp added this to the 4.2 LTS milestone 2023-12-28 01:35:41 +01:00

Ray molenkamp added the

Type

Design

label 2023-12-28 01:35:41 +01:00

Brecht Van Lommel was assigned by Ray molenkamp

2023-12-28 01:36:14 +01:00

Thomas Dinges was assigned by Ray molenkamp

2023-12-28 01:36:14 +01:00

Ray molenkamp self-assigned this 2023-12-28 01:36:15 +01:00

Campbell Barton was assigned by Ray molenkamp

2023-12-28 01:36:30 +01:00

Aras Pranckevicius commented

2023-12-30 18:33:31 +01:00

Things like OpenImageIO, OpenEXR and perhaps some other external libraries could get built with SSE4.2 options too, right?

Ray molenkamp commented

2023-12-30 19:20:09 +01:00

Yes, but that won't be relevant until we start the 4.2 library update, @brecht suggested doing blender itself first hence the plan focused on those changes first.

👍 2

Ray molenkamp commented

2024-01-01 11:46:50 +01:00

I was looking further into a /arch:sse42 switch being missing for msvc, it actually looks like without any /arch flags it does actually generate sse42 code already, it does a run-time check before executing any sse42 instructions and the sse4 codepath is the default non-branched code path. There is an undocumented compiler flag /d2archSSE42 which removes the run-time check and generated sse2 codepath.

live example

So that while that all sounds great, the odds of the compiler choosing this optimization is relatively low, in all of blenkernel i only saw it applying it in BKE_scopes_update.

Not super thrilled about using an undocumented flag though, and even if we do i doubt there will be a meaningful performance difference, so probably best to leave that one alone.

I was looking further into a `/arch:sse42` switch being missing for msvc, it actually looks like without any `/arch` flags it does actually generate sse42 code already, it does a run-time check before executing any sse42 instructions and the sse4 codepath is the default non-branched code path. There is an undocumented compiler flag `/d2archSSE42` which removes the run-time check and generated sse2 codepath. [live example](https://godbolt.org/z/hnbnrdxex) So that while that all sounds great, the odds of the compiler choosing this optimization is relatively low, in all of blenkernel i only saw it applying it in `BKE_scopes_update`. Not super thrilled about using an undocumented flag though, and even if we do i doubt there will be a meaningful performance difference, so probably best to leave that one alone.

🎉 1

Thomas Dinges commented

2024-02-09 12:57:23 +01:00

Now that 4.2 development started, what are the first steps to get going? I assume adding the SSE42 check library that Ray suggested is the first thing?

Afterwards further work can be done on enabling SSE4 in the libraries, cleaning up Blender and Cycles code etc. I am available to help.

Now that 4.2 development started, what are the first steps to get going? I assume adding the SSE42 check library that Ray suggested is the first thing? Afterwards further work can be done on enabling SSE4 in the libraries, cleaning up Blender and Cycles code etc. I am available to help.

Ray molenkamp commented

2024-02-09 17:10:39 +01:00

I'll take an initial stab at the check, shouldn't be too hard.

Ray molenkamp referenced this issue

2024-02-09 18:46:04 +01:00

Creator: Add CPU check on startup #118054

Ray molenkamp referenced this issue from a commit

2024-02-19 18:12:06 +01:00

Creator: Add CPU check on startup

Sign in to join this conversation.