Windows ARM64 Support #119126

New Issue

Brecht Van Lommel · 2024-03-06T16:05:48+01:00

Brecht Van Lommel commented

2024-03-06 16:05:48 +01:00

Libraries

Create lib-windows_arm64 repository
Populate repository
Integrate with build system and submodules

Features

SIMD support in BLI_simd.h once blenlib is converted to C++
OpenPGL support

Buildbot

Provision machines
Integration (infrastructure/blender-devops!3)

Issues

Test failures
- io_wavefront
- constraints
- Various python tests, libs appear to be missing some python dlls (see Sergey's comment below)
TBBmalloc: skip allocation functions replacement in ucrtbase.dll: unknown prologue for function free
- WITH_TBB_MALLOC_PROXY=OFF may work around it by turning off tbbmalloc
The viewport flickers sometimes when zoomed in (clay view, textured view is fine) - difficult to reproduce, but happens from time to time.
There is a shader compiler issue which will have a BSP update out in due course - the effect is noticeable on the 3.3 splash in the viewport, where meshes fail to render in some cases

### Libraries * [x] Create [lib-windows_arm64](https://projects.blender.org/blender/lib-windows_arm64) repository * [x] Populate repository * [x] Integrate with build system and submodules ### Features * [ ] SIMD support in `BLI_simd.h` once blenlib is converted to C++ * [ ] OpenPGL support ### Buildbot * [x] Provision machines * [x] Integration (infrastructure/blender-devops!3) ### Issues * [ ] Test failures * [ ] io_wavefront * [ ] constraints * [x] Various python tests, libs appear to be missing some python dlls (see Sergey's comment below) * [ ] TBBmalloc: skip allocation functions replacement in ucrtbase.dll: unknown prologue for function free * `WITH_TBB_MALLOC_PROXY=OFF` may work around it by turning off tbbmalloc * [ ] The viewport flickers sometimes when zoomed in (clay view, textured view is fine) - difficult to reproduce, but happens from time to time. * [ ] There is a shader compiler issue which will have a BSP update out in due course - the effect is noticeable on the 3.3 splash in the viewport, where meshes fail to render in some cases

Brecht Van Lommel added the

Platform

Windows

Module

Platforms, Builds & Tests

Type

To Do

labels 2024-03-06 16:05:48 +01:00

Ray molenkamp commented

2024-03-06 16:11:15 +01:00

I had io_wavefront fail in #118468 due to __SSE2__ not being defined by accident at one point, could be the same root cause.

I had `io_wavefront` fail in #118468 due to `__SSE2__` not being defined by accident at one point, could be the same root cause.

Brecht Van Lommel referenced this issue

2024-03-06 16:12:29 +01:00

Add Windows ARM64 support #117036

Brecht Van Lommel referenced this issue from a commit

2024-03-06 16:16:52 +01:00

Build: Add Windows ARM64 support for library dependencies

Brecht Van Lommel referenced this issue from a commit

2024-03-06 16:16:52 +01:00

Cycles: Add Windows ARM64 support

Brecht Van Lommel referenced this issue from a commit

2024-03-06 16:16:52 +01:00

Windows: Add ARM64 support

Anthony Roberts commented

2024-03-06 18:38:02 +01:00

@LazyDodo Do you know which assert it was failing on due to that? The line from ctest on the wavefront failure for me is:

BLI_assert failed: C:\WoA\blender-build-arm64\blender\source\blender\io\wavefront_obj\importer\importer_mesh_utils.cc:78, fixup_invalid_face(), at 'idx >= 0 && idx < face_verts.size()'

@LazyDodo Do you know which assert it was failing on due to that? The line from ctest on the wavefront failure for me is: ``` BLI_assert failed: C:\WoA\blender-build-arm64\blender\source\blender\io\wavefront_obj\importer\importer_mesh_utils.cc:78, fixup_invalid_face(), at 'idx >= 0 && idx < face_verts.size()' ```

Ray molenkamp commented

2024-03-06 18:43:13 +01:00

I'm about 99% certain thats the same assert

Anthony Roberts commented

2024-03-07 16:56:07 +01:00

As noted by @aras_p here, SIMD disabled causes at least one of these test failures (wavefront): #118468 (comment)

Given the stl failure is also in a maths library, and the constraints failure is on precision, this could be our culprit for the ARM64 failures, as SIMD is disabled until blenlib becomes C++

As noted by @aras_p here, SIMD disabled causes at least one of these test failures (`wavefront`): https://projects.blender.org/blender/blender/pulls/118468#issuecomment-1140814 Given the `stl` failure is also in a maths library, and the `constraints` failure is on precision, this could be our culprit for the ARM64 failures, as SIMD is disabled until blenlib becomes C++

Anthony Roberts commented

2024-03-07 16:57:54 +01:00

Test failures log attached here

arm64_test_failures.txt

94 KiB

Brecht Van Lommel referenced this issue

2024-03-14 17:23:32 +01:00

Windows: Add ARM64 lib submodule #119459

Brecht Van Lommel referenced this issue from a commit

2024-03-14 18:13:01 +01:00

Windows: Add ARM64 lib submodule

Sergey Sharybin commented

2024-04-19 17:43:18 +02:00

I've added Windows Arm64 builders to buildbot. Some tests are failing, on top of the known ones: https://builder.blender.org/admin/#/builders/244/builds/1

io_curve_svg_* seems to be failing due to missing expat. And, in fact, I did not see expat.pyd in the Python's DLLs folder
io_usd_* seems to be failing due to another missing module
scripts_* is a bit tricky to follow. One of obvious related issues is numpy which misses numpy.core._multiarray_umath

P.S. Dropping a message here so the information is not lost until Anthony is back.

I've added Windows Arm64 builders to buildbot. Some tests are failing, on top of the known ones: https://builder.blender.org/admin/#/builders/244/builds/1 - `io_curve_svg_*` seems to be failing due to missing expat. And, in fact, I did not see `expat.pyd` in the [Python's DLLs folder](https://projects.blender.org/blender/lib-windows_arm64/src/branch/main/python/311/DLLs) - `io_usd_*` seems to be failing due to another missing module - `scripts_*` is a bit tricky to follow. One of obvious related issues is numpy which misses `numpy.core._multiarray_umath` P.S. Dropping a message here so the information is not lost until Anthony is back.

Ray molenkamp commented

2024-04-19 18:04:25 +02:00

numpy.core._multiarray_umath is missing the .pyd files, similar to expat, the pdb's are there though, so it looks like the pyd's got build just didn't make it into the git.

`numpy.core._multiarray_umath` is missing the .pyd files, similar to expat, the pdb's are there though, so it looks like the pyd's got build just didn't make it into the git.

Anthony Roberts commented

2024-04-29 12:53:52 +02:00

Thanks @Sergey and @LazyDodo - I am back now and catching up - I have kicked off a local rebuild of the libs from the tip of main, and will update the prebuilts directory when/if it succesfully completes.

You're right that those files seem to be missing in the GIT LFS repo - I'm not sure why that would be the case, as my "output" folder from the lib build (an old one I pulled out from a few months ago) seems to contain them...

Once the latest build is complete and functional I will diff the file lists against x64 to make sure nothing else got unintentionally missed.

Thanks @Sergey and @LazyDodo - I am back now and catching up - I have kicked off a local rebuild of the libs from the tip of main, and will update the prebuilts directory when/if it succesfully completes. You're right that those files seem to be missing in the GIT LFS repo - I'm not sure why that would be the case, as my "output" folder from the lib build (an old one I pulled out from a few months ago) seems to contain them... Once the latest build is complete and functional I will diff the file lists against x64 to make sure nothing else got unintentionally missed.

Brecht Van Lommel commented

2024-04-29 13:37:58 +02:00

.pyd files were ignored through .gitignore.

I've removed them from there now: blender/lib-windows_arm64@6da81e67a9. We only need to ignore them on Linux and macOS.

`.pyd` files were ignored through `.gitignore`. I've removed them from there now: blender/lib-windows_arm64@6da81e67a9852839c9f52bb7c5e08a613ef94ab7. We only need to ignore them on Linux and macOS.

Anthony Roberts commented

2024-04-29 13:45:40 +02:00

ACK, that would make sense :)

RE the TBBmalloc thing - it's always been an issue, even when running the emulated x64 version of blender on an ARM64 device. Other than the error message being a bit spammy, it seems relatively harmless. Is the better option here to disable it completely?

ACK, that would make sense :) RE the TBBmalloc thing - it's always been an issue, even when running the emulated x64 version of blender on an ARM64 device. Other than the error message being a bit spammy, it seems relatively harmless. Is the better option here to disable it completely?

Ray molenkamp commented

2024-04-29 16:23:19 +02:00

it's not entirely harmless, there is a distinct performance penalty, the standard ms allocator is not very good in multi threaded environments, see D6218 for some benchmarks, do note that around that time, we had our own BVH builder now-days we use embree and performance profile will likely have changed. (the old builder allocated many small memory blocks from many threads, i don't think embree does this)

given tbb isn't able to hook the old allocator, having it enabled really adds nothing, I say we disable WITH_TBB_MALLOC_PROXY in cmake for WOA until we can make it work/move to a version that support it on WOA.

it's not entirely harmless, there is a distinct performance penalty, the standard ms allocator is not very good in multi threaded environments, see [D6218](https://archive.blender.org/developer/D6218) for some benchmarks, do note that around that time, we had our own BVH builder now-days we use embree and performance profile will likely have changed. (the old builder allocated many small memory blocks from many threads, i don't think embree does this) given tbb isn't able to hook the old allocator, having it enabled really adds nothing, I say we disable `WITH_TBB_MALLOC_PROXY` in cmake for WOA until we can make it work/move to a version that support it on WOA.

Anthony Roberts commented

2024-04-29 16:29:36 +02:00

Understood - I think this is a solved problem in newer versions of TBB (ie, not 2020u3 which I had to port to ARM64 by hand) - do we know what timescales are looking like for TBB/USD updates?

Last I heard it was "at some point" - it would also reduce the patch size for TBB dramatically for the ARM64 stuff.

Understood - I think this is a solved problem in newer versions of TBB (ie, not 2020u3 which I had to port to ARM64 by hand) - do we know what timescales are looking like for TBB/USD updates? Last I heard it was "at some point" - it would also reduce the patch size for TBB dramatically for the ARM64 stuff.

Ray molenkamp commented

2024-04-29 16:37:15 +02:00

unknown, we're bound by the vfx platform there, they have been saying "next year" for a few years now and it never happens due to lack of support for newer TBB in USD. The draft for 2025 should be posted in the next month or so, then we'll know more.

unknown, we're bound by the [vfx platform](https://vfxplatform.com/) there, they have been saying "next year" for a few years now and it never happens due to lack of support for newer TBB in USD. The draft for 2025 should be posted in the next month or so, then we'll know more.

Brecht Van Lommel commented

2024-04-29 16:46:08 +02:00

I posted pull requests to USD to support newer TBB last year. Half of them were merged last week, so hopefully the other half will not take too long and make it into the next USD release.

I posted [pull requests to USD](https://github.com/PixarAnimationStudios/OpenUSD/pull/2466) to support newer TBB last year. Half of them were merged last week, so hopefully the other half will not take too long and make it into the next USD release.

Anthony Roberts commented

2024-04-30 16:08:28 +02:00

I have updated the files on the LFS repo, and done a build on top of main using my updated submodule - all seems well with the tests, with only constraints failing now:

99% tests passed, 1 tests failed out of 281

Total Test time (real) = 534.63 sec

The following tests FAILED:
         33 - constraints (Failed)
Errors while running CTest

I will make a PR to the main blender repo in the next hour or two updating the submodule version - things should work properly after that!

I have updated the files on the LFS repo, and done a build on top of main using my updated submodule - all seems well with the tests, with only `constraints` failing now: ``` 99% tests passed, 1 tests failed out of 281 Total Test time (real) = 534.63 sec The following tests FAILED: 33 - constraints (Failed) Errors while running CTest ``` I will make a PR to the main blender repo in the next hour or two updating the submodule version - things should work properly after that!

Anthony Roberts referenced this issue

2024-04-30 16:57:19 +02:00

Windows: Update ARM64 submodule to include .pyd files #121271

Sign in to join this conversation.

No Label

Download

What's New

Blender Studio

Manual

Developers Blog

Documentation

Benchmark

Blender Conference

Development Fund

One-time Donations

Windows ARM64 Support #119126

Libraries

Features

Buildbot

Issues