studio/flamenco
studio/flamenco
12
30
Fork 38
Code Issues 63 Pull Requests 5 Releases Wiki Activity

Manager build error on ARM64 #104191

New Issue
Closed
opened 2023-02-28 15:31:46 +01:00 by k8ie · 15 comments
k8ie commented 2023-02-28 15:31:46 +01:00
Copy Link

System Information
Operating System(s): Arch Linux ARM (in Docker with QEMU emulation)

Flamenco Version
Is Broken: main branch
Worked OK: probably never

Short description of error
The manager fails to build on aarch64 systems. The worker builds correctly.

Log snippet:

...
go: downloading github.com/go-sourcemap/sourcemap v2.1.3+incompatible
go: downloading golang.org/x/text v0.7.0
go: downloading github.com/dlclark/regexp2 v1.4.1-0.20201116162257-a2a8dda75c91
go: downloading github.com/go-openapi/swag v0.19.5
go: downloading github.com/valyala/bytebufferpool v1.0.0
go: downloading github.com/mailru/easyjson v0.7.0
go: downloading modernc.org/mathutil v1.4.1
go: downloading modernc.org/memory v1.1.1
go: downloading github.com/remyoudompheng/bigfft v0.0.0-20200410134404-eec4a21b6bb0
git.blender.org/flamenco/pkg/shaman/touch
# git.blender.org/flamenco/pkg/shaman/touch
pkg/shaman/touch/touch_linux.go:38:38: undefined: syscall.SYS_UTIMES
make: *** [Makefile:61: flamenco-manager] Error 1

Exact steps for others to reproduce the error

  1. Clone the repository on an ARM system
  2. Run npm i in web/app
  3. Run make flamenco-manager in the root of the git repo
**System Information** Operating System(s): Arch Linux ARM (in Docker with QEMU emulation) **Flamenco Version** Is Broken: main branch Worked OK: probably never **Short description of error** The manager fails to build on `aarch64` systems. The worker builds correctly. Log snippet: ``` ... go: downloading github.com/go-sourcemap/sourcemap v2.1.3+incompatible go: downloading golang.org/x/text v0.7.0 go: downloading github.com/dlclark/regexp2 v1.4.1-0.20201116162257-a2a8dda75c91 go: downloading github.com/go-openapi/swag v0.19.5 go: downloading github.com/valyala/bytebufferpool v1.0.0 go: downloading github.com/mailru/easyjson v0.7.0 go: downloading modernc.org/mathutil v1.4.1 go: downloading modernc.org/memory v1.1.1 go: downloading github.com/remyoudompheng/bigfft v0.0.0-20200410134404-eec4a21b6bb0 git.blender.org/flamenco/pkg/shaman/touch # git.blender.org/flamenco/pkg/shaman/touch pkg/shaman/touch/touch_linux.go:38:38: undefined: syscall.SYS_UTIMES make: *** [Makefile:61: flamenco-manager] Error 1 ``` **Exact steps for others to reproduce the error** 1. Clone the repository on an ARM system 1. Run `npm i` in `web/app` 1. Run `make flamenco-manager` in the root of the git repo
Sybren A. Stüvel commented 2023-05-01 10:28:34 +02:00
Owner
Copy Link

Which version of Arch Linux were you using? And which version of Go? Which Linux kernel is used?

Which version of Arch Linux were you using? And which version of Go? Which Linux kernel is used?
Sybren A. Stüvel added the
Status
Needs Information from User
 label 2023-05-01 10:28:47 +02:00
k8ie commented 2023-05-01 12:09:09 +02:00
Author
Copy Link

I had to retest since I didn't note these things.

I tested on two different systems. One x86_64 CPU, but with QEMU User Mode emulation in Docker and one actual aarch64 system (Raspberry Pi 4).

The x86_64 system with emulation:

  • go: go version go1.20.3 linux/arm64
  • Linux: 6.2.13-zen-1-zen (the kernel is not emulated with user mode emulation, so this is the host kernel)

The Raspberry Pi:

  • go: go version go1.20.3 linux/arm64
  • Linux: 6.1.25-1-rpi-ARCH (Raspberry Pi Foundation kernel)

Arch doesn't have versioned releases. Originally I wanted to write this off to the emulation, but it happens on a bare metal ARM machine too.

I had to retest since I didn't note these things. I tested on two different systems. One `x86_64` CPU, but with QEMU User Mode emulation in Docker and one actual `aarch64` system (Raspberry Pi 4). The `x86_64` system with emulation: - go: `go version go1.20.3 linux/arm64` - Linux: `6.2.13-zen-1-zen` (the kernel is not emulated with user mode emulation, so this is the host kernel) The Raspberry Pi: - go: `go version go1.20.3 linux/arm64` - Linux: `6.1.25-1-rpi-ARCH` (Raspberry Pi Foundation kernel) Arch doesn't have versioned releases. Originally I wanted to write this off to the emulation, but it happens on a bare metal ARM machine too.
Sybren A. Stüvel commented 2023-05-01 16:09:55 +02:00
Owner
Copy Link

Do you have a link to a step by step of running a QEmu with emulated ARM CPU?

Do you have a link to a step by step of running a QEmu with emulated ARM CPU?
k8ie commented 2023-05-01 16:50:29 +02:00
Author
Copy Link

Sure thing.

If you already have Docker set up on your system, it's as easy as running docker run --privileged --rm tonistiigi/binfmt --install all. (Source)

After that, your system is ready to execute binaries compiled for any of the architectures supported by QEMU. Rebooting the system unregisters the emulators again.

You can choose the architecture to run of any multi-arch container image with --platform.

I maintain my own container images for building Arch packages and I used this command to run my build container:

docker run --platform linux/arm64 -it ghcr.io/a13xie/arch-builder:latest /bin/bash

But like I said, you can try this with any container that supports multiple architectures. As I'm writing this I'm trying to compile Flamenco on Ubuntu.

Sure thing. If you already have Docker set up on your system, it's as easy as running `docker run --privileged --rm tonistiigi/binfmt --install all`. ([Source](https://github.com/tonistiigi/binfmt)) After that, your system is ready to execute binaries compiled for any of the architectures supported by QEMU. Rebooting the system unregisters the emulators again. You can choose the architecture to run of any multi-arch container image with `--platform`. I maintain my own container images for building Arch packages and I used this command to run my build container: ``` docker run --platform linux/arm64 -it ghcr.io/a13xie/arch-builder:latest /bin/bash ``` But like I said, you can try this with any container that supports multiple architectures. As I'm writing this I'm trying to compile Flamenco on Ubuntu.
k8ie commented 2023-05-01 17:07:57 +02:00
Author
Copy Link

Ok, I just finished trying with Ubuntu, it ends the same way :/

This was with Ubuntu 22.04 and Go go version go1.18.1 linux/arm64.

Ok, I just finished trying with Ubuntu, it ends the same way :/ This was with Ubuntu 22.04 and Go `go version go1.18.1 linux/arm64`.
Sybren A. Stüvel commented 2023-05-01 17:56:25 +02:00
Owner
Copy Link

Could you give #104210 a try?

Could you give #104210 a try?
k8ie commented 2023-05-01 18:30:04 +02:00
Author
Copy Link

I think there might be an error in some makefile somewhere:

git.blender.org/flamenco/cmd/addon-packer
# git.blender.org/flamenco/cmd/addon-packer
flag provided but not defined: -Wl,-O1,--sort-common,--as-needed,-z,relro,-z,now
usage: link [options] main.o
  -B note
    	add an ELF NT_GNU_BUILD_ID note when using ELF
  -E entry
    	set entry symbol name
  -H type
    	set header type
  -I linker
    	use linker as ELF dynamic linker
  -L directory
    	add specified directory to library path
  -R quantum
    	set address rounding quantum (default -1)
  -T address
    	set text segment address (default -1)
  -V	print version and exit
  -X definition
    	add string value definition of the form importpath.name=value
  -a	no-op (deprecated)
  -asan
    	enable ASan interface
  -aslr
    	enable ASLR for buildmode=c-shared on windows (default true)
  -benchmark string
    	set to 'mem' or 'cpu' to enable phase benchmarking
  -benchmarkprofile base
    	emit phase profiles to base_phase.{cpu,mem}prof
  -buildid id
    	record id as Go toolchain build id
  -buildmode mode
    	set build mode
  -c	dump call graph
  -capturehostobjs string
    	capture host object files loaded during internal linking to specified dir
  -compressdwarf
    	compress DWARF if possible (default true)
  -cpuprofile file
    	write cpu profile to file
  -d	disable dynamic executable
  -debugnosplit
    	dump nosplit call graph
  -debugtextsize int
    	debug text section max size
  -debugtramp int
    	debug trampolines
  -dumpdep
    	dump symbol dependency graph
  -extar string
    	archive program for buildmode=c-archive
  -extld linker
    	use linker when linking in external mode
  -extldflags flags
    	pass flags to external linker
  -f	ignore version mismatch
  -g	disable go package data checks
  -h	halt on error
  -importcfg file
    	read import configuration from file
  -installsuffix suffix
    	set package directory suffix
  -k symbol
    	set field tracking symbol
  -libgcc string
    	compiler support lib for internal linking; use "none" to disable
  -linkmode mode
    	set link mode
  -linkshared
    	link against installed Go shared libraries
  -memprofile file
    	write memory profile to file
  -memprofilerate rate
    	set runtime.MemProfileRate to rate
  -msan
    	enable MSan interface
  -n	dump symbol table
  -o file
    	write output to file
  -pluginpath string
    	full path name for plugin
  -r path
    	set the ELF dynamic linker search path to dir1:dir2:...
  -race
    	enable race detector
  -s	disable symbol table
  -strictdups int
    	sanity check duplicate symbol contents during object file reading (1=warn 2=err).
  -tmpdir directory
    	use directory for temporary files
  -v	print link trace
  -w	disable DWARF generation
make[1]: *** [Makefile:75: addon-packer] Error 1
make[1]: Leaving directory '/home/builder/flamenco/src/flamenco'
make: *** [Makefile:60: flamenco-manager] Error 2
I think there might be an error in some makefile somewhere: ``` git.blender.org/flamenco/cmd/addon-packer # git.blender.org/flamenco/cmd/addon-packer flag provided but not defined: -Wl,-O1,--sort-common,--as-needed,-z,relro,-z,now usage: link [options] main.o -B note add an ELF NT_GNU_BUILD_ID note when using ELF -E entry set entry symbol name -H type set header type -I linker use linker as ELF dynamic linker -L directory add specified directory to library path -R quantum set address rounding quantum (default -1) -T address set text segment address (default -1) -V print version and exit -X definition add string value definition of the form importpath.name=value -a no-op (deprecated) -asan enable ASan interface -aslr enable ASLR for buildmode=c-shared on windows (default true) -benchmark string set to 'mem' or 'cpu' to enable phase benchmarking -benchmarkprofile base emit phase profiles to base_phase.{cpu,mem}prof -buildid id record id as Go toolchain build id -buildmode mode set build mode -c dump call graph -capturehostobjs string capture host object files loaded during internal linking to specified dir -compressdwarf compress DWARF if possible (default true) -cpuprofile file write cpu profile to file -d disable dynamic executable -debugnosplit dump nosplit call graph -debugtextsize int debug text section max size -debugtramp int debug trampolines -dumpdep dump symbol dependency graph -extar string archive program for buildmode=c-archive -extld linker use linker when linking in external mode -extldflags flags pass flags to external linker -f ignore version mismatch -g disable go package data checks -h halt on error -importcfg file read import configuration from file -installsuffix suffix set package directory suffix -k symbol set field tracking symbol -libgcc string compiler support lib for internal linking; use "none" to disable -linkmode mode set link mode -linkshared link against installed Go shared libraries -memprofile file write memory profile to file -memprofilerate rate set runtime.MemProfileRate to rate -msan enable MSan interface -n dump symbol table -o file write output to file -pluginpath string full path name for plugin -r path set the ELF dynamic linker search path to dir1:dir2:... -race enable race detector -s disable symbol table -strictdups int sanity check duplicate symbol contents during object file reading (1=warn 2=err). -tmpdir directory use directory for temporary files -v print link trace -w disable DWARF generation make[1]: *** [Makefile:75: addon-packer] Error 1 make[1]: Leaving directory '/home/builder/flamenco/src/flamenco' make: *** [Makefile:60: flamenco-manager] Error 2 ```
Sybren A. Stüvel commented 2023-05-02 10:48:15 +02:00
Owner
Copy Link

I just tried building with:

docker run --platform linux/arm64 -it  -v (pwd):/go/flamenco golang:latest bash
# cd flamenco
# git config --global --add safe.directory /go/flamenco
# make flamenco-manager-without-webapp

This avoids the need to pack the add-on, or use yarn to build the webapp. It's sufficient to test the problem that was reported here, and that the PR fixes (I hope).

I just tried building with: ``` docker run --platform linux/arm64 -it -v (pwd):/go/flamenco golang:latest bash # cd flamenco # git config --global --add safe.directory /go/flamenco # make flamenco-manager-without-webapp ``` This avoids the need to pack the add-on, or use `yarn` to build the webapp. It's sufficient to test the problem that was reported here, and that the PR fixes (I hope).
Sybren A. Stüvel commented 2023-05-02 10:51:12 +02:00
Owner
Copy Link

Also, I wasn't thinking. This is enough to build for linux/arm64 when on linux/amd64 (like I am):

make GOARCH=arm64 flamenco-manager-without-webapp
Also, I wasn't thinking. This is enough to build for linux/arm64 when on linux/amd64 (like I am): ```sh make GOARCH=arm64 flamenco-manager-without-webapp ```
k8ie commented 2023-05-02 21:58:31 +02:00
Author
Copy Link

This is enough to build for linux/arm64

That's super cool, I didn't know cross-compiling with Go was that easy!

I tried the build without the web app, but I got the same error as in #104191 (comment)

I also tried this in the same container as you (golang:latest) and that worked, so #104191 (comment) is probably unrelated.

Are there any compromises that come with using sys/unix instead of syscall?

> This is enough to build for linux/arm64 That's super cool, I didn't know cross-compiling with Go was that easy! I tried the build without the web app, but I got the same error as in https://projects.blender.org/studio/flamenco/issues/104191#issuecomment-932495 I also tried this in the same container as you (`golang:latest`) and that worked, so https://projects.blender.org/studio/flamenco/issues/104191#issuecomment-932495 is probably unrelated. Are there any compromises that come with using `sys/unix` instead of `syscall`?
MichaelC commented 2023-05-02 22:55:58 +02:00
Contributor
Copy Link

Could you give #104210 a try?

This built flamenco-manager for me when I hand patched the touch_linux.go file.

Without this I would get the following error building the arm64 target...

GOOS=linux GOARCH=arm64 make with-deps
...
...
...
#0 53.42 go build -v -ldflags=" -X git.blender.org/flamenco/internal/appinfo.ApplicationVersion=3.3-alpha0 -X git.blender.org/flamenco/internal/appinfo.ApplicationGitHash=v3.2-44-g3c1871cf -X git.blender.org/flamenco/internal/appinfo.ReleaseCycle=alpha" git.blender.org/flamenco/cmd/flamenco-manager
#0 53.61 git.blender.org/flamenco/pkg/shaman/touch
#0 53.63 # git.blender.org/flamenco/pkg/shaman/touch
#0 53.63 pkg/shaman/touch/touch_linux.go:38:38: undefined: syscall.SYS_UTIMES
#0 53.70 make[1]: *** [Makefile:61: flamenco-manager] Error 1
#0 53.70 make[1]: Leaving directory '/code'
#0 53.70 make: *** [Makefile:55: with-deps] Error 2
> Could you give #104210 a try? This built flamenco-manager for me when I hand patched the touch_linux.go file. Without this I would get the following error building the arm64 target... ``` GOOS=linux GOARCH=arm64 make with-deps ... ... ... #0 53.42 go build -v -ldflags=" -X git.blender.org/flamenco/internal/appinfo.ApplicationVersion=3.3-alpha0 -X git.blender.org/flamenco/internal/appinfo.ApplicationGitHash=v3.2-44-g3c1871cf -X git.blender.org/flamenco/internal/appinfo.ReleaseCycle=alpha" git.blender.org/flamenco/cmd/flamenco-manager #0 53.61 git.blender.org/flamenco/pkg/shaman/touch #0 53.63 # git.blender.org/flamenco/pkg/shaman/touch #0 53.63 pkg/shaman/touch/touch_linux.go:38:38: undefined: syscall.SYS_UTIMES #0 53.70 make[1]: *** [Makefile:61: flamenco-manager] Error 1 #0 53.70 make[1]: Leaving directory '/code' #0 53.70 make: *** [Makefile:55: with-deps] Error 2 ```
Sybren A. Stüvel commented 2023-05-04 12:51:55 +02:00
Owner
Copy Link

Could you give #104210 a try?

This built flamenco-manager for me when I hand patched the touch_linux.go file.

Without this I would get the following error building the arm64 target...

That error indicates you were still on the main branch? #104210 hasn't landed in main yet, so to test you need to switch branches. But, if you manually applied the patch, that's ok as well, just a bit more work.

Are there any compromises that come with using sys/unix instead of syscall?

Not really, IMO it's even a better approach now.

I also tried this in the same container as you (golang:latest) and that worked, so #104191 (comment) is probably unrelated.

👍 I want to get rid of the Makefile altogether anyway, but that's related to #102633 and tracked in the magefile branch.

> > Could you give #104210 a try? > > This built flamenco-manager for me when I hand patched the touch_linux.go file. > > Without this I would get the following error building the arm64 target... That error indicates you were still on the `main` branch? #104210 hasn't landed in `main` yet, so to test you need to switch branches. But, if you manually applied the patch, that's ok as well, just a bit more work. > Are there any compromises that come with using sys/unix instead of syscall? Not really, IMO it's even a better approach now. > I also tried this in the same container as you (`golang:latest`) and that worked, so https://projects.blender.org/studio/flamenco/issues/104191#issuecomment-932495 is probably unrelated. :+1: I want to get rid of the `Makefile` altogether anyway, but that's related to #102633 and tracked in the [magefile branch](https://projects.blender.org/studio/flamenco/src/branch/magefile).
MichaelC commented 2023-05-04 13:13:04 +02:00
Contributor
Copy Link

Yes, I hand patched main just to see if it helped while experimenting with arm64 build and reading this thread.

Yes, I hand patched main just to see if it helped while experimenting with arm64 build and reading this thread.
k8ie commented 2023-05-13 16:56:46 +02:00
Author
Copy Link

That error indicates you were still on the main branch?

Indeed, I wrongly configured my PKGBUILD, expecting it to checkout that branch. When manually checking out the branch, it builds as expected. This was an error on my side.

> That error indicates you were still on the main branch? Indeed, I wrongly configured my PKGBUILD, expecting it to checkout that branch. When manually checking out the branch, it builds as expected. This was an error on my side.
Sybren A. Stüvel commented 2023-05-15 11:11:29 +02:00
Owner
Copy Link

Thanks for the report!

Thanks for the report!
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels   
Good First Issue
  
Priority
High

  
Priority
Low

  
Priority
Normal

  
Status
Archived

  
Status
Confirmed

  
Status
Needs Info from Developers

  
Status
Needs Information from User

  
Status
Needs Triage

  
Status
Resolved

  
Type
Bug

  
Type
Design

  
Type
Job Type
Custom Flamenco job type

  
Type
Known Issue

  
Type
Patch

  
Type
Report

  
Type
To Do

No Label
Good First Issue
Priority
High
Priority
Low
Priority
Normal
Status
Archived
Status
Confirmed
Status
Needs Info from Developers
Status
Needs Information from User
Status
Needs Triage
Status
Resolved
Type
Bug
Type
Design
Type
Job Type
Type
Known Issue
Type
Patch
Type
Report
Type
To Do
Milestone
Clear milestone
No items
No Milestone
Assignees
Clear assignees
No Assignees
3 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: studio/flamenco#104191
No description provided.
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?