studio/flamenco
studio/flamenco
12
30
Fork 38
Code Issues 63 Pull Requests 5 Releases Wiki Activity

RPI4 Support #104213

New Issue
Closed
opened 2023-05-22 23:53:26 +02:00 by PGSCOM · 12 comments
PGSCOM commented 2023-05-22 23:53:26 +02:00
Copy Link

I have tried several ways to run flamenco on my RPI, using box64 to emulate the made build and also baking the build from the RPI4 (I can't install correctly GO on RPI) and neither of the 2 works.

I have tried several ways to run flamenco on my RPI, using box64 to emulate the made build and also baking the build from the RPI4 (I can't install correctly GO on RPI) and neither of the 2 works.
MichaelC commented 2023-05-23 15:29:31 +02:00
Contributor
Copy Link

Hi @PGSCOM - this to me falls into a wider question for flamenco dev team on what is the plan to support ARM targets in general, like Mac M1, RaspberryPi 32 & 64bit etc. Depending on your development experience level, I can help you generate arm64 flamenco-manager executable from flamenco working-copy that is based on a flamenco release but there is a problem in v3.2 that fails to compile on arm64. This was fixed in the main branch and will be included in v3.3.

Does 'baking the build from RPI4" mean you tried to build Flamenco-Manager on RaspberryPi host?

Hi @PGSCOM - this to me falls into a wider question for flamenco dev team on what is the plan to support ARM targets in general, like Mac M1, RaspberryPi 32 & 64bit etc. Depending on your development experience level, I can help you generate arm64 flamenco-manager executable from flamenco working-copy that is based on a flamenco release but there is a [problem in v3.2 that fails to compile on arm64.](https://projects.blender.org/studio/flamenco/issues/104191) This was fixed in the main branch and will be included in v3.3. Does 'baking the build from RPI4" mean you tried to build Flamenco-Manager on RaspberryPi host?
PGSCOM commented 2023-05-23 15:36:27 +02:00
Author
Copy Link

Yes @michael-2, I tried to build Flamenco-Manager and Flamenco-Worker in Raspbian installing all the dependencies with the tutorial.

Yes @michael-2, I tried to build Flamenco-Manager and Flamenco-Worker in Raspbian installing all the dependencies with the [tutorial](https://flamenco.blender.org/development/getting-started/).
MichaelC commented 2023-05-23 15:39:15 +02:00
Contributor
Copy Link

Yes @michael-2, I tried to build Flamenco-Manager and Flamenco-Worker in Raspbian installing all the dependencies with the tutorial.

What was the outcome/failure? I have not tried a native build on RaspPi host.

> Yes @michael-2, I tried to build Flamenco-Manager and Flamenco-Worker in Raspbian installing all the dependencies with the [tutorial](https://flamenco.blender.org/development/getting-started/). What was the outcome/failure? I have not tried a native build on RaspPi host.
PGSCOM commented 2023-05-23 16:18:16 +02:00
Author
Copy Link

With the last commit (0a2d7d3361)

pablo@rpi4:~/flamenco $ make
yarn --cwd web/app install
yarn install v1.22.19
[1/4] Resolving packages...
success Already up-to-date.
Done in 0.87s.
make webapp-static
make[1]: Entering directory '/home/pablo/flamenco'
go build -v -ldflags=" -X git.blender.org/flamenco/internal/appinfo.ApplicationVersion=3.3-alpha0 -X git.blender.org/flamenco/internal/appinfo.ApplicationGitHash=v3.2-57-g9cbdadcd-dirty -X git.blender.org/flamenco/internal/appinfo.ReleaseCycle=alpha" git.blender.org/flamenco/cmd/addon-packer
cmd/addon-packer/addon-packer.go:11:2: package io/fs is not in GOROOT (/usr/lib/go-1.15/src/io/fs)
make[1]: *** [Makefile:73: addon-packer] Error 1
make[1]: Leaving directory '/home/pablo/flamenco'
make: *** [Makefile:58: flamenco-manager] Error 2
With the last commit (0a2d7d3361) ``` pablo@rpi4:~/flamenco $ make yarn --cwd web/app install yarn install v1.22.19 [1/4] Resolving packages... success Already up-to-date. Done in 0.87s. make webapp-static make[1]: Entering directory '/home/pablo/flamenco' go build -v -ldflags=" -X git.blender.org/flamenco/internal/appinfo.ApplicationVersion=3.3-alpha0 -X git.blender.org/flamenco/internal/appinfo.ApplicationGitHash=v3.2-57-g9cbdadcd-dirty -X git.blender.org/flamenco/internal/appinfo.ReleaseCycle=alpha" git.blender.org/flamenco/cmd/addon-packer cmd/addon-packer/addon-packer.go:11:2: package io/fs is not in GOROOT (/usr/lib/go-1.15/src/io/fs) make[1]: *** [Makefile:73: addon-packer] Error 1 make[1]: Leaving directory '/home/pablo/flamenco' make: *** [Makefile:58: flamenco-manager] Error 2 ```
MichaelC commented 2023-05-24 01:42:11 +02:00
Contributor
Copy Link

The minimum version requirements for Flamenco must be met and I can see from your logs, you are using earlier version of golang 1.15 and minimum is 1.20. Usually, RaspPi/Debian/Ubuntu distros have fairly old golang, nodejs and npm releases.

I dont have a RaspPi 4. I have a Rasp3 with 1GB RAM and while trying to create an example for you, it jams up trying to install go packages during make with-deps at the end... but RaspPi runs out of memory.

You could try this script...

#!/bin/bash
# Assuming Ubuntu/Debian/Raspbian environment
# https://flamenco.blender.org/development/getting-started/

set -e

# Toolchain
echo "Toolchain install..."
sudo apt-get update
sudo apt install -y curl gzip software-properties-common gcc g++ make build-essential
sudo apt-get install -y default-jre-headless git golang
curl -sL https://deb.nodesource.com/setup_16.x | sudo bash -
sudo apt-get update
sudo apt install -y hugo file make nodejs
sudo npm install --global yarn

# Golang - Flamenco supported version >=1.20
GOVERSION="1.20.4"
PLATFORM="arm64"
echo "Replacing go with $GOVERSION for $PLATFORM..."
wget "https://golang.org/dl/go${GOVERSION}.linux-$PLATFORM.tar.gz" -O go${GOVERSION}.linux-$PLATFORM.tar.gz
sudo tar -C /usr/local -xvf "go${GOVERSION}.linux-$PLATFORM.tar.gz"
sudo ln -fs /usr/local/go/bin/go /usr/bin/go

# Check versions
echo "Versions..."
node --version
npm --version
go version

# Clone flamenco (main branch)
echo "Cloning Flamenco..."
git clone https://projects.blender.org/studio/flamenco.git

# Build
echo "Building..."
cd flamenco/
go mod download
GOMEMLIMIT=500MiB GOMAXPROCS=2 make with-deps -j 1

Let me know if this succeeds on a RaspPi4. ;o)

I have created a flamenco-manager arm64 using Docker if you're interested in a more involved build process, not using the RaspPi host, but using Docker buildx using a similar script to that shown above on an amd64 workstation.

The minimum version requirements for Flamenco must be met and I can see from your logs, you are using earlier version of golang 1.15 and minimum is 1.20. Usually, RaspPi/Debian/Ubuntu distros have fairly old golang, nodejs and npm releases. I dont have a RaspPi 4. I have a Rasp3 with 1GB RAM and while trying to create an example for you, it jams up trying to install go packages during `make with-deps` at the end... but RaspPi runs out of memory. You could try this script... ``` #!/bin/bash # Assuming Ubuntu/Debian/Raspbian environment # https://flamenco.blender.org/development/getting-started/ set -e # Toolchain echo "Toolchain install..." sudo apt-get update sudo apt install -y curl gzip software-properties-common gcc g++ make build-essential sudo apt-get install -y default-jre-headless git golang curl -sL https://deb.nodesource.com/setup_16.x | sudo bash - sudo apt-get update sudo apt install -y hugo file make nodejs sudo npm install --global yarn # Golang - Flamenco supported version >=1.20 GOVERSION="1.20.4" PLATFORM="arm64" echo "Replacing go with $GOVERSION for $PLATFORM..." wget "https://golang.org/dl/go${GOVERSION}.linux-$PLATFORM.tar.gz" -O go${GOVERSION}.linux-$PLATFORM.tar.gz sudo tar -C /usr/local -xvf "go${GOVERSION}.linux-$PLATFORM.tar.gz" sudo ln -fs /usr/local/go/bin/go /usr/bin/go # Check versions echo "Versions..." node --version npm --version go version # Clone flamenco (main branch) echo "Cloning Flamenco..." git clone https://projects.blender.org/studio/flamenco.git # Build echo "Building..." cd flamenco/ go mod download GOMEMLIMIT=500MiB GOMAXPROCS=2 make with-deps -j 1 ``` Let me know if this succeeds on a RaspPi4. ;o) I have created a flamenco-manager arm64 using Docker if you're interested in a more involved build process, not using the RaspPi host, but using Docker buildx using a similar script to that shown above on an amd64 workstation.
❤️ 1
PGSCOM commented 2023-05-24 08:24:58 +02:00
Author
Copy Link
[other go paths]
go/test/zerodivide.go
Versions...
v16.20.0
8.19.4
bash: /usr/bin/go: No such file or directory
Cloning Flamenco...
fatal: destination path 'flamenco' already exists and is not an empty directory.
Building...
bash: /usr/bin/go: No such file or directory
go install github.com/deepmap/oapi-codegen/cmd/oapi-codegen@v1.9.0
make: go: No such file or directory
make: *** [Makefile:50: with-deps] Error 127
``` [other go paths] go/test/zerodivide.go Versions... v16.20.0 8.19.4 bash: /usr/bin/go: No such file or directory Cloning Flamenco... fatal: destination path 'flamenco' already exists and is not an empty directory. Building... bash: /usr/bin/go: No such file or directory go install github.com/deepmap/oapi-codegen/cmd/oapi-codegen@v1.9.0 make: go: No such file or directory make: *** [Makefile:50: with-deps] Error 127 ```
PGSCOM commented 2023-05-24 11:38:22 +02:00
Author
Copy Link

There's probably that error because I've done tests before, I'd better do a new installation of raspbian and try it again.

There's probably that error because I've done tests before, I'd better do a new installation of raspbian and try it again.
MichaelC commented 2023-05-24 11:47:13 +02:00
Contributor
Copy Link

There's probably that error because I've done tests before, I'd better do a new installation of raspbian and try it again.

The script says it can't find go, so probably the symlink in the script is broken.

The symlink in the script is set by...

sudo ln -fs /usr/local/go/bin/go /usr/bin/go

Check that the script unpacked go to /usr/local/go/ and what the symlink is pointing to by trying these commands on the RaspPi...

#> which go
/usr/bin/go

#> ls -l /usr/bin/go
lrwxrwxrwx 1 root root 20 May 23 21:55 /usr/bin/go -> /usr/local/go/bin/go

#> go version
go version go1.20.4 linux/arm64
> There's probably that error because I've done tests before, I'd better do a new installation of raspbian and try it again. The script says it can't find go, so probably the symlink in the script is broken. The symlink in the script is set by... ``` sudo ln -fs /usr/local/go/bin/go /usr/bin/go ``` Check that the script unpacked go to /usr/local/go/ and what the symlink is pointing to by trying these commands on the RaspPi... ``` #> which go /usr/bin/go #> ls -l /usr/bin/go lrwxrwxrwx 1 root root 20 May 23 21:55 /usr/bin/go -> /usr/local/go/bin/go #> go version go version go1.20.4 linux/arm64 ```
PGSCOM commented 2023-05-24 16:30:49 +02:00
Author
Copy Link

I don't know why Go didn't install correctly. I install it line by line and now works

pablo@rpi4:~ $ node --version
v16.20.0
pablo@rpi4:~ $ npm --version
8.19.4
pablo@rpi4:~ $ go version
bash: go: command not found
pablo@rpi4:~ $ go version
go version go1.20.4 linux/arm64
pablo@rpi4:~ $ git clone https://projects.blender.org/studio/flamenco.git
Cloning into 'flamenco'...
remote: Enumerating objects: 706, done.
remote: Counting objects: 100% (706/706), done.
remote: Compressing objects: 100% (640/640), done.
remote: Total 32492 (delta 388), reused 61 (delta 42), pack-reused 31786
Receiving objects: 100% (32492/32492), 49.00 MiB | 2.14 MiB/s, done.
Resolving deltas: 100% (21192/21192), done.
pablo@rpi4:~ $ cd flamenco/
pablo@rpi4:~/flamenco $ go mod download
pablo@rpi4:~/flamenco $ GOMEMLIMIT=500MiB GOMAXPROCS=2 make with-deps -j 1
[Downloading packages...]
go: downloading golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1
github.com/deepmap/oapi-codegen/cmd/oapi-codegen: go install github.com/deepmap/oapi-codegen/cmd/oapi-codegen: copying /tmp/go-build2456574364/b001/exe/a.out: open /home/pablo/go/bin/oapi-codegen: permission denied
make: *** [Makefile:50: with-deps] Error 1
pablo@rpi4:~/flamenco $ sudo go install github.com/deepmap/oapi-codegen/cmd/oapi-codegen
[Downloading packages...]
go: downloading gopkg.in/yaml.v3 v3.0.1
make application
make[1]: Entering directory '/home/pablo/flamenco'
yarn --cwd web/app install
yarn install v1.22.19
[1/4] Resolving packages...
[2/4] Fetching packages...
[3/4] Linking dependencies...
[4/4] Building fresh packages...
Done in 40.99s.
make webapp-static
make[2]: Entering directory '/home/pablo/flamenco'
go build -v -ldflags=" -X git.blender.org/flamenco/internal/appinfo.ApplicationVersion=3.3-alpha0 -X git.blender.org/flamenco/internal/appinfo.ApplicationGitHash=v3.2-58-g0a2d7d33 -X git.blender.org/flamenco/internal/appinfo.ReleaseCycle=alpha" git.blender.org/flamenco/cmd/addon-packer
go: downloading github.com/rs/zerolog v1.26.1
go: downloading github.com/adrg/xdg v0.4.0
github.com/adrg/xdg/internal/pathutil
archive/zip
github.com/rs/zerolog/internal/json
github.com/adrg/xdg
git.blender.org/flamenco/internal/appinfo
github.com/rs/zerolog
github.com/rs/zerolog/log
git.blender.org/flamenco/cmd/addon-packer
make clean-webapp-static
make[3]: Entering directory '/home/pablo/flamenco'
rm -rf ./web/static
mkdir -p ./web/static
touch web/static/emptyfile
make[3]: Leaving directory '/home/pablo/flamenco'
yarn --cwd web/app build --outDir ../static --base=/app/ --logLevel warn
yarn run v1.22.19
$ vite build --outDir ../static --base=/app/ --logLevel warn

(!) outDir /home/pablo/flamenco/web/static is not inside project root and will not be emptied.
Use --emptyOutDir to override.

Done in 21.64s.
./addon-packer -filename web/static/flamenco3-addon.zip
2023-05-24T16:23:10+02:00 INF creating ZIP file zipname=/home/pablo/flamenco/web/static/flamenco3-addon.zip
Web app has been installed into web/static
make[2]: Leaving directory '/home/pablo/flamenco'
go build -v -ldflags=" -X git.blender.org/flamenco/internal/appinfo.ApplicationVersion=3.3-alpha0 -X git.blender.org/flamenco/internal/appinfo.ApplicationGitHash=v3.2-58-g0a2d7d33 -X git.blender.org/flamenco/internal/appinfo.ReleaseCycle=alpha" git.blender.org/flamenco/cmd/flamenco-manager
[Downloading packages...]
go: downloading github.com/remyoudompheng/bigfft v0.0.0-20200410134404-eec4a21b6bb0
hash/maphash
[More urls...]
git.blender.org/flamenco/internal/worker/persistence
git.blender.org/flamenco/internal/worker
git.blender.org/flamenco/cmd/flamenco-worker
make[1]: Leaving directory '/home/pablo/flamenco'
pablo@rpi4:~/flamenco $
I don't know why Go didn't install correctly. I install it line by line and now works ``` pablo@rpi4:~ $ node --version v16.20.0 pablo@rpi4:~ $ npm --version 8.19.4 pablo@rpi4:~ $ go version bash: go: command not found pablo@rpi4:~ $ go version go version go1.20.4 linux/arm64 pablo@rpi4:~ $ git clone https://projects.blender.org/studio/flamenco.git Cloning into 'flamenco'... remote: Enumerating objects: 706, done. remote: Counting objects: 100% (706/706), done. remote: Compressing objects: 100% (640/640), done. remote: Total 32492 (delta 388), reused 61 (delta 42), pack-reused 31786 Receiving objects: 100% (32492/32492), 49.00 MiB | 2.14 MiB/s, done. Resolving deltas: 100% (21192/21192), done. pablo@rpi4:~ $ cd flamenco/ pablo@rpi4:~/flamenco $ go mod download pablo@rpi4:~/flamenco $ GOMEMLIMIT=500MiB GOMAXPROCS=2 make with-deps -j 1 [Downloading packages...] go: downloading golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1 github.com/deepmap/oapi-codegen/cmd/oapi-codegen: go install github.com/deepmap/oapi-codegen/cmd/oapi-codegen: copying /tmp/go-build2456574364/b001/exe/a.out: open /home/pablo/go/bin/oapi-codegen: permission denied make: *** [Makefile:50: with-deps] Error 1 pablo@rpi4:~/flamenco $ sudo go install github.com/deepmap/oapi-codegen/cmd/oapi-codegen [Downloading packages...] go: downloading gopkg.in/yaml.v3 v3.0.1 make application make[1]: Entering directory '/home/pablo/flamenco' yarn --cwd web/app install yarn install v1.22.19 [1/4] Resolving packages... [2/4] Fetching packages... [3/4] Linking dependencies... [4/4] Building fresh packages... Done in 40.99s. make webapp-static make[2]: Entering directory '/home/pablo/flamenco' go build -v -ldflags=" -X git.blender.org/flamenco/internal/appinfo.ApplicationVersion=3.3-alpha0 -X git.blender.org/flamenco/internal/appinfo.ApplicationGitHash=v3.2-58-g0a2d7d33 -X git.blender.org/flamenco/internal/appinfo.ReleaseCycle=alpha" git.blender.org/flamenco/cmd/addon-packer go: downloading github.com/rs/zerolog v1.26.1 go: downloading github.com/adrg/xdg v0.4.0 github.com/adrg/xdg/internal/pathutil archive/zip github.com/rs/zerolog/internal/json github.com/adrg/xdg git.blender.org/flamenco/internal/appinfo github.com/rs/zerolog github.com/rs/zerolog/log git.blender.org/flamenco/cmd/addon-packer make clean-webapp-static make[3]: Entering directory '/home/pablo/flamenco' rm -rf ./web/static mkdir -p ./web/static touch web/static/emptyfile make[3]: Leaving directory '/home/pablo/flamenco' yarn --cwd web/app build --outDir ../static --base=/app/ --logLevel warn yarn run v1.22.19 $ vite build --outDir ../static --base=/app/ --logLevel warn (!) outDir /home/pablo/flamenco/web/static is not inside project root and will not be emptied. Use --emptyOutDir to override. Done in 21.64s. ./addon-packer -filename web/static/flamenco3-addon.zip 2023-05-24T16:23:10+02:00 INF creating ZIP file zipname=/home/pablo/flamenco/web/static/flamenco3-addon.zip Web app has been installed into web/static make[2]: Leaving directory '/home/pablo/flamenco' go build -v -ldflags=" -X git.blender.org/flamenco/internal/appinfo.ApplicationVersion=3.3-alpha0 -X git.blender.org/flamenco/internal/appinfo.ApplicationGitHash=v3.2-58-g0a2d7d33 -X git.blender.org/flamenco/internal/appinfo.ReleaseCycle=alpha" git.blender.org/flamenco/cmd/flamenco-manager [Downloading packages...] go: downloading github.com/remyoudompheng/bigfft v0.0.0-20200410134404-eec4a21b6bb0 hash/maphash [More urls...] git.blender.org/flamenco/internal/worker/persistence git.blender.org/flamenco/internal/worker git.blender.org/flamenco/cmd/flamenco-worker make[1]: Leaving directory '/home/pablo/flamenco' pablo@rpi4:~/flamenco $ ```
PGSCOM commented 2023-05-24 16:43:02 +02:00
Author
Copy Link

And it finaly works!!! Thanks @michael-2
image

And it finaly works!!! Thanks @michael-2 ![image](/attachments/082d9688-251f-4f68-8d6e-b91182057c89)
image.png
144 KiB
Flamenco for RPI4.zip
87 MiB
PGSCOM commented 2023-05-24 17:54:01 +02:00
Author
Copy Link

Now I have a problem, I can't enter a Windows path (Because the workers will be Windows), so, the RPI4 will do as NAS and also for Manager server. But I cant put it because the path didn't exsist in Linux.

image

And when I edit the .yml file manualy to Z:\ the path in the plugin is /home/pablo/Z:\ and not Z:\

image

There is any config that can decide what path is used in each platform?

Now I have a problem, I can't enter a Windows path (Because the workers will be Windows), so, the RPI4 will do as NAS and also for Manager server. But I cant put it because the path didn't exsist in Linux. ![image](/attachments/b256b0af-bc60-4480-8c86-e6bf4cafab21) And when I edit the .yml file manualy to Z:\ the path in the plugin is /home/pablo/Z:\ and not Z:\ ![image](/attachments/5507014d-28b8-4a1c-a916-e8a24185a4d0) There is any config that can decide what path is used in each platform?
image.png
63 KiB
image.png
52 KiB
PGSCOM reopened this issue 2023-05-24 17:54:01 +02:00
MichaelC commented 2023-05-24 18:18:52 +02:00
Contributor
Copy Link

@PGSCOM - yes, there is documentation on how to configure shared-storage paths between platform types. RaspPi will be a 'linux' platform type. Flamenco will translate between them. See documentation on flamenco website using shared-storage two-way variables.

If you need functional setup help, I recommend using the blender chat instead of raising git issues. I would keep this specific Issue about creating a RaspPi ARM build.

@PGSCOM - yes, there is documentation on how to configure shared-storage paths between platform types. RaspPi will be a 'linux' platform type. Flamenco will translate between them. See documentation on flamenco website using shared-storage two-way variables. If you need functional setup help, I recommend using the blender chat instead of raising git issues. I would keep this specific Issue about creating a RaspPi ARM build.
👍 1
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels   
Good First Issue
  
Priority
High

  
Priority
Low

  
Priority
Normal

  
Status
Archived

  
Status
Confirmed

  
Status
Needs Info from Developers

  
Status
Needs Information from User

  
Status
Needs Triage

  
Status
Resolved

  
Type
Bug

  
Type
Design

  
Type
Job Type
Custom Flamenco job type

  
Type
Known Issue

  
Type
Patch

  
Type
Report

  
Type
To Do

No Label
Good First Issue
Priority
High
Priority
Low
Priority
Normal
Status
Archived
Status
Confirmed
Status
Needs Info from Developers
Status
Needs Information from User
Status
Needs Triage
Status
Resolved
Type
Bug
Type
Design
Type
Job Type
Type
Known Issue
Type
Patch
Type
Report
Type
To Do
Milestone
Clear milestone
No items
No Milestone
Assignees
Clear assignees
No Assignees
2 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: studio/flamenco#104213
No description provided.
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?