Manager build error on ARM64 #104191

New Issue

k8ie · 2023-02-28T15:31:46+01:00

k8ie commented

2023-02-28 15:31:46 +01:00

System Information
Operating System(s): Arch Linux ARM (in Docker with QEMU emulation)

Flamenco Version
Is Broken: main branch
Worked OK: probably never

Short description of error
The manager fails to build on aarch64 systems. The worker builds correctly.

Log snippet:

...
go: downloading github.com/go-sourcemap/sourcemap v2.1.3+incompatible
go: downloading golang.org/x/text v0.7.0
go: downloading github.com/dlclark/regexp2 v1.4.1-0.20201116162257-a2a8dda75c91
go: downloading github.com/go-openapi/swag v0.19.5
go: downloading github.com/valyala/bytebufferpool v1.0.0
go: downloading github.com/mailru/easyjson v0.7.0
go: downloading modernc.org/mathutil v1.4.1
go: downloading modernc.org/memory v1.1.1
go: downloading github.com/remyoudompheng/bigfft v0.0.0-20200410134404-eec4a21b6bb0
git.blender.org/flamenco/pkg/shaman/touch
# git.blender.org/flamenco/pkg/shaman/touch
pkg/shaman/touch/touch_linux.go:38:38: undefined: syscall.SYS_UTIMES
make: *** [Makefile:61: flamenco-manager] Error 1

Exact steps for others to reproduce the error

Clone the repository on an ARM system
Run npm i in web/app
Run make flamenco-manager in the root of the git repo

**System Information** Operating System(s): Arch Linux ARM (in Docker with QEMU emulation) **Flamenco Version** Is Broken: main branch Worked OK: probably never **Short description of error** The manager fails to build on `aarch64` systems. The worker builds correctly. Log snippet: ``` ... go: downloading github.com/go-sourcemap/sourcemap v2.1.3+incompatible go: downloading golang.org/x/text v0.7.0 go: downloading github.com/dlclark/regexp2 v1.4.1-0.20201116162257-a2a8dda75c91 go: downloading github.com/go-openapi/swag v0.19.5 go: downloading github.com/valyala/bytebufferpool v1.0.0 go: downloading github.com/mailru/easyjson v0.7.0 go: downloading modernc.org/mathutil v1.4.1 go: downloading modernc.org/memory v1.1.1 go: downloading github.com/remyoudompheng/bigfft v0.0.0-20200410134404-eec4a21b6bb0 git.blender.org/flamenco/pkg/shaman/touch # git.blender.org/flamenco/pkg/shaman/touch pkg/shaman/touch/touch_linux.go:38:38: undefined: syscall.SYS_UTIMES make: *** [Makefile:61: flamenco-manager] Error 1 ``` **Exact steps for others to reproduce the error** 1. Clone the repository on an ARM system 1. Run `npm i` in `web/app` 1. Run `make flamenco-manager` in the root of the git repo

Sybren A. Stüvel commented

2023-05-01 10:28:34 +02:00

Which version of Arch Linux were you using? And which version of Go? Which Linux kernel is used?

Sybren A. Stüvel added the

Status

Needs Information from User

label 2023-05-01 10:28:47 +02:00

k8ie commented

2023-05-01 12:09:09 +02:00

I had to retest since I didn't note these things.

I tested on two different systems. One x86_64 CPU, but with QEMU User Mode emulation in Docker and one actual aarch64 system (Raspberry Pi 4).

The x86_64 system with emulation:

go: go version go1.20.3 linux/arm64
Linux: 6.2.13-zen-1-zen (the kernel is not emulated with user mode emulation, so this is the host kernel)

The Raspberry Pi:

go: go version go1.20.3 linux/arm64
Linux: 6.1.25-1-rpi-ARCH (Raspberry Pi Foundation kernel)

Arch doesn't have versioned releases. Originally I wanted to write this off to the emulation, but it happens on a bare metal ARM machine too.

I had to retest since I didn't note these things. I tested on two different systems. One `x86_64` CPU, but with QEMU User Mode emulation in Docker and one actual `aarch64` system (Raspberry Pi 4). The `x86_64` system with emulation: - go: `go version go1.20.3 linux/arm64` - Linux: `6.2.13-zen-1-zen` (the kernel is not emulated with user mode emulation, so this is the host kernel) The Raspberry Pi: - go: `go version go1.20.3 linux/arm64` - Linux: `6.1.25-1-rpi-ARCH` (Raspberry Pi Foundation kernel) Arch doesn't have versioned releases. Originally I wanted to write this off to the emulation, but it happens on a bare metal ARM machine too.

Sybren A. Stüvel commented

2023-05-01 16:09:55 +02:00

Do you have a link to a step by step of running a QEmu with emulated ARM CPU?

k8ie commented

2023-05-01 16:50:29 +02:00

Sure thing.

If you already have Docker set up on your system, it's as easy as running docker run --privileged --rm tonistiigi/binfmt --install all. (Source)

After that, your system is ready to execute binaries compiled for any of the architectures supported by QEMU. Rebooting the system unregisters the emulators again.

You can choose the architecture to run of any multi-arch container image with --platform.

I maintain my own container images for building Arch packages and I used this command to run my build container:

docker run --platform linux/arm64 -it ghcr.io/a13xie/arch-builder:latest /bin/bash

But like I said, you can try this with any container that supports multiple architectures. As I'm writing this I'm trying to compile Flamenco on Ubuntu.

Sure thing. If you already have Docker set up on your system, it's as easy as running `docker run --privileged --rm tonistiigi/binfmt --install all`. ([Source](https://github.com/tonistiigi/binfmt)) After that, your system is ready to execute binaries compiled for any of the architectures supported by QEMU. Rebooting the system unregisters the emulators again. You can choose the architecture to run of any multi-arch container image with `--platform`. I maintain my own container images for building Arch packages and I used this command to run my build container: ``` docker run --platform linux/arm64 -it ghcr.io/a13xie/arch-builder:latest /bin/bash ``` But like I said, you can try this with any container that supports multiple architectures. As I'm writing this I'm trying to compile Flamenco on Ubuntu.

k8ie commented

2023-05-01 17:07:57 +02:00

Ok, I just finished trying with Ubuntu, it ends the same way :/

This was with Ubuntu 22.04 and Go go version go1.18.1 linux/arm64.

Ok, I just finished trying with Ubuntu, it ends the same way :/ This was with Ubuntu 22.04 and Go `go version go1.18.1 linux/arm64`.

Sybren A. Stüvel referenced this issue from a commit

2023-05-01 17:56:03 +02:00

Fix #104191: Manager build error on ARM64

Sybren A. Stüvel referenced a pull request that will close this issue

2023-05-01 17:56:10 +02:00

Fix #104191: Manager build error on ARM64 #104210

Sybren A. Stüvel commented

2023-05-01 17:56:25 +02:00

Could you give #104210 a try?

k8ie commented

2023-05-01 18:30:04 +02:00

I think there might be an error in some makefile somewhere:

git.blender.org/flamenco/cmd/addon-packer
# git.blender.org/flamenco/cmd/addon-packer
flag provided but not defined: -Wl,-O1,--sort-common,--as-needed,-z,relro,-z,now
usage: link [options] main.o
  -B note
    	add an ELF NT_GNU_BUILD_ID note when using ELF
  -E entry
    	set entry symbol name
  -H type
    	set header type
  -I linker
    	use linker as ELF dynamic linker
  -L directory
    	add specified directory to library path
  -R quantum
    	set address rounding quantum (default -1)
  -T address
    	set text segment address (default -1)
  -V	print version and exit
  -X definition
    	add string value definition of the form importpath.name=value
  -a	no-op (deprecated)
  -asan
    	enable ASan interface
  -aslr
    	enable ASLR for buildmode=c-shared on windows (default true)
  -benchmark string
    	set to 'mem' or 'cpu' to enable phase benchmarking
  -benchmarkprofile base
    	emit phase profiles to base_phase.{cpu,mem}prof
  -buildid id
    	record id as Go toolchain build id
  -buildmode mode
    	set build mode
  -c	dump call graph
  -capturehostobjs string
    	capture host object files loaded during internal linking to specified dir
  -compressdwarf
    	compress DWARF if possible (default true)
  -cpuprofile file
    	write cpu profile to file
  -d	disable dynamic executable
  -debugnosplit
    	dump nosplit call graph
  -debugtextsize int
    	debug text section max size
  -debugtramp int
    	debug trampolines
  -dumpdep
    	dump symbol dependency graph
  -extar string
    	archive program for buildmode=c-archive
  -extld linker
    	use linker when linking in external mode
  -extldflags flags
    	pass flags to external linker
  -f	ignore version mismatch
  -g	disable go package data checks
  -h	halt on error
  -importcfg file
    	read import configuration from file
  -installsuffix suffix
    	set package directory suffix
  -k symbol
    	set field tracking symbol
  -libgcc string
    	compiler support lib for internal linking; use "none" to disable
  -linkmode mode
    	set link mode
  -linkshared
    	link against installed Go shared libraries
  -memprofile file
    	write memory profile to file
  -memprofilerate rate
    	set runtime.MemProfileRate to rate
  -msan
    	enable MSan interface
  -n	dump symbol table
  -o file
    	write output to file
  -pluginpath string
    	full path name for plugin
  -r path
    	set the ELF dynamic linker search path to dir1:dir2:...
  -race
    	enable race detector
  -s	disable symbol table
  -strictdups int
    	sanity check duplicate symbol contents during object file reading (1=warn 2=err).
  -tmpdir directory
    	use directory for temporary files
  -v	print link trace
  -w	disable DWARF generation
make[1]: *** [Makefile:75: addon-packer] Error 1
make[1]: Leaving directory '/home/builder/flamenco/src/flamenco'
make: *** [Makefile:60: flamenco-manager] Error 2

I think there might be an error in some makefile somewhere: ``` git.blender.org/flamenco/cmd/addon-packer # git.blender.org/flamenco/cmd/addon-packer flag provided but not defined: -Wl,-O1,--sort-common,--as-needed,-z,relro,-z,now usage: link [options] main.o -B note add an ELF NT_GNU_BUILD_ID note when using ELF -E entry set entry symbol name -H type set header type -I linker use linker as ELF dynamic linker -L directory add specified directory to library path -R quantum set address rounding quantum (default -1) -T address set text segment address (default -1) -V print version and exit -X definition add string value definition of the form importpath.name=value -a no-op (deprecated) -asan enable ASan interface -aslr enable ASLR for buildmode=c-shared on windows (default true) -benchmark string set to 'mem' or 'cpu' to enable phase benchmarking -benchmarkprofile base emit phase profiles to base_phase.{cpu,mem}prof -buildid id record id as Go toolchain build id -buildmode mode set build mode -c dump call graph -capturehostobjs string capture host object files loaded during internal linking to specified dir -compressdwarf compress DWARF if possible (default true) -cpuprofile file write cpu profile to file -d disable dynamic executable -debugnosplit dump nosplit call graph -debugtextsize int debug text section max size -debugtramp int debug trampolines -dumpdep dump symbol dependency graph -extar string archive program for buildmode=c-archive -extld linker use linker when linking in external mode -extldflags flags pass flags to external linker -f ignore version mismatch -g disable go package data checks -h halt on error -importcfg file read import configuration from file -installsuffix suffix set package directory suffix -k symbol set field tracking symbol -libgcc string compiler support lib for internal linking; use "none" to disable -linkmode mode set link mode -linkshared link against installed Go shared libraries -memprofile file write memory profile to file -memprofilerate rate set runtime.MemProfileRate to rate -msan enable MSan interface -n dump symbol table -o file write output to file -pluginpath string full path name for plugin -r path set the ELF dynamic linker search path to dir1:dir2:... -race enable race detector -s disable symbol table -strictdups int sanity check duplicate symbol contents during object file reading (1=warn 2=err). -tmpdir directory use directory for temporary files -v print link trace -w disable DWARF generation make[1]: *** [Makefile:75: addon-packer] Error 1 make[1]: Leaving directory '/home/builder/flamenco/src/flamenco' make: *** [Makefile:60: flamenco-manager] Error 2 ```

Sybren A. Stüvel commented

2023-05-02 10:48:15 +02:00

I just tried building with:

docker run --platform linux/arm64 -it  -v (pwd):/go/flamenco golang:latest bash
# cd flamenco
# git config --global --add safe.directory /go/flamenco
# make flamenco-manager-without-webapp

This avoids the need to pack the add-on, or use yarn to build the webapp. It's sufficient to test the problem that was reported here, and that the PR fixes (I hope).

I just tried building with: ``` docker run --platform linux/arm64 -it -v (pwd):/go/flamenco golang:latest bash # cd flamenco # git config --global --add safe.directory /go/flamenco # make flamenco-manager-without-webapp ``` This avoids the need to pack the add-on, or use `yarn` to build the webapp. It's sufficient to test the problem that was reported here, and that the PR fixes (I hope).

Sybren A. Stüvel commented

2023-05-02 10:51:12 +02:00

Also, I wasn't thinking. This is enough to build for linux/arm64 when on linux/amd64 (like I am):

make GOARCH=arm64 flamenco-manager-without-webapp

Also, I wasn't thinking. This is enough to build for linux/arm64 when on linux/amd64 (like I am): ```sh make GOARCH=arm64 flamenco-manager-without-webapp ```

k8ie commented

2023-05-02 21:58:31 +02:00

This is enough to build for linux/arm64

That's super cool, I didn't know cross-compiling with Go was that easy!

I tried the build without the web app, but I got the same error as in #104191 (comment)

I also tried this in the same container as you (golang:latest) and that worked, so #104191 (comment) is probably unrelated.

Are there any compromises that come with using sys/unix instead of syscall?

> This is enough to build for linux/arm64 That's super cool, I didn't know cross-compiling with Go was that easy! I tried the build without the web app, but I got the same error as in https://projects.blender.org/studio/flamenco/issues/104191#issuecomment-932495 I also tried this in the same container as you (`golang:latest`) and that worked, so https://projects.blender.org/studio/flamenco/issues/104191#issuecomment-932495 is probably unrelated. Are there any compromises that come with using `sys/unix` instead of `syscall`?

MichaelC commented

2023-05-02 22:55:58 +02:00

Could you give #104210 a try?

This built flamenco-manager for me when I hand patched the touch_linux.go file.

Without this I would get the following error building the arm64 target...

GOOS=linux GOARCH=arm64 make with-deps
...
...
...
#0 53.42 go build -v -ldflags=" -X git.blender.org/flamenco/internal/appinfo.ApplicationVersion=3.3-alpha0 -X git.blender.org/flamenco/internal/appinfo.ApplicationGitHash=v3.2-44-g3c1871cf -X git.blender.org/flamenco/internal/appinfo.ReleaseCycle=alpha" git.blender.org/flamenco/cmd/flamenco-manager
#0 53.61 git.blender.org/flamenco/pkg/shaman/touch
#0 53.63 # git.blender.org/flamenco/pkg/shaman/touch
#0 53.63 pkg/shaman/touch/touch_linux.go:38:38: undefined: syscall.SYS_UTIMES
#0 53.70 make[1]: *** [Makefile:61: flamenco-manager] Error 1
#0 53.70 make[1]: Leaving directory '/code'
#0 53.70 make: *** [Makefile:55: with-deps] Error 2

> Could you give #104210 a try? This built flamenco-manager for me when I hand patched the touch_linux.go file. Without this I would get the following error building the arm64 target... ``` GOOS=linux GOARCH=arm64 make with-deps ... ... ... #0 53.42 go build -v -ldflags=" -X git.blender.org/flamenco/internal/appinfo.ApplicationVersion=3.3-alpha0 -X git.blender.org/flamenco/internal/appinfo.ApplicationGitHash=v3.2-44-g3c1871cf -X git.blender.org/flamenco/internal/appinfo.ReleaseCycle=alpha" git.blender.org/flamenco/cmd/flamenco-manager #0 53.61 git.blender.org/flamenco/pkg/shaman/touch #0 53.63 # git.blender.org/flamenco/pkg/shaman/touch #0 53.63 pkg/shaman/touch/touch_linux.go:38:38: undefined: syscall.SYS_UTIMES #0 53.70 make[1]: *** [Makefile:61: flamenco-manager] Error 1 #0 53.70 make[1]: Leaving directory '/code' #0 53.70 make: *** [Makefile:55: with-deps] Error 2 ```

Sybren A. Stüvel commented

2023-05-04 12:51:55 +02:00

Could you give #104210 a try?

This built flamenco-manager for me when I hand patched the touch_linux.go file.

Without this I would get the following error building the arm64 target...

That error indicates you were still on the main branch? #104210 hasn't landed in main yet, so to test you need to switch branches. But, if you manually applied the patch, that's ok as well, just a bit more work.

Are there any compromises that come with using sys/unix instead of syscall?

Not really, IMO it's even a better approach now.

I also tried this in the same container as you (golang:latest) and that worked, so #104191 (comment) is probably unrelated.

👍 I want to get rid of the Makefile altogether anyway, but that's related to #102633 and tracked in the magefile branch.

> > Could you give #104210 a try? > > This built flamenco-manager for me when I hand patched the touch_linux.go file. > > Without this I would get the following error building the arm64 target... That error indicates you were still on the `main` branch? #104210 hasn't landed in `main` yet, so to test you need to switch branches. But, if you manually applied the patch, that's ok as well, just a bit more work. > Are there any compromises that come with using sys/unix instead of syscall? Not really, IMO it's even a better approach now. > I also tried this in the same container as you (`golang:latest`) and that worked, so https://projects.blender.org/studio/flamenco/issues/104191#issuecomment-932495 is probably unrelated. :+1: I want to get rid of the `Makefile` altogether anyway, but that's related to #102633 and tracked in the [magefile branch](https://projects.blender.org/studio/flamenco/src/branch/magefile).

MichaelC commented

2023-05-04 13:13:04 +02:00

Yes, I hand patched main just to see if it helped while experimenting with arm64 build and reading this thread.

k8ie commented

2023-05-13 16:56:46 +02:00

That error indicates you were still on the main branch?

Indeed, I wrongly configured my PKGBUILD, expecting it to checkout that branch. When manually checking out the branch, it builds as expected. This was an error on my side.

> That error indicates you were still on the main branch? Indeed, I wrongly configured my PKGBUILD, expecting it to checkout that branch. When manually checking out the branch, it builds as expected. This was an error on my side.

Sybren A. Stüvel referenced this issue from a commit

2023-05-15 10:51:18 +02:00

Fix #104191: Manager build error on ARM64

Sybren A. Stüvel closed this issue

2023-05-15 10:51:18 +02:00

Sybren A. Stüvel commented

2023-05-15 11:11:29 +02:00

Thanks for the report!

MichaelC referenced this issue

2023-05-23 15:42:39 +02:00

RPI4 Support #104213

Sign in to join this conversation.