Cannot select Render Output Path but default invalid output path and causes blender to quit #104198

New Issue

MichaelC · 2023-03-14T18:39:36+01:00

MichaelC commented

2023-03-14 18:39:36 +01:00

System Information

MacOS NFS mount to NFS Server on local home network.
Blender 3.4 with Flamenco 3.1 addon.
Blender file is saved on NFS Server.

Flamenco Manager and Worker are both configured using yaml files. No GUI Wizard was used. See attached.

Flamenco Version
Is Broken: 3.1

Short description of error

Running a Flamenco Job, the worker starts to successfully render and when it attempts to write the output it has been instructed to use "/mike-redcubes-longer-timeout/2023-03-14_163903/000077.png".

This path is invalid and blender quits with error. Flamenco re-enqueues a few times and fails the job eventually.

Looking at the blender addon plugin, it has an option to set the Render Output Path but says it is not enabled. I can open a Modal dialogue box and select an NFS path... but this is from the MacOS NFS Mount perspective which the Worker would need a different root path.

The Flamenco Manager configuration has

local_manager_storage_path: /data/flamenco-manager-storage
shared_storage_path: /data

The /data is successfully mounted to the NFS server path. The blender file is successfully found/read by Blender on the worker.

Exact steps for others to reproduce the error

Manager Configuration File

_meta:
  version: 3
manager_name: Flamenco Manager
database: flamenco-manager.sqlite
listen: :8080
autodiscoverable: true
local_manager_storage_path: /data/flamenco-manager-storage
shared_storage_path: /data
shaman:
  enabled: true
  garbageCollect:
    period: 24h0m0s
    maxAge: 744h0m0s
    extraCheckoutPaths: []
task_timeout: 60m0s
worker_timeout: 30m0s
blocklist_threshold: 3
task_fail_after_softfail_count: 3
variables:
  blender:
    values:
      - platform: linux
        value: blender
  blenderArgs:
    values:
      - platform: all
        value: -b -y

Worker Configuration

manager_url: http://flamenco-service.flamenco-manager.svc:8080/
task_types: [blender, ffmpeg, file-management, misc]

NFS Mount from the Worker context shows the NFS Server filestore contents as expected:

ls /data/
file-store  jobs  output  projects

Flamenco Job received by Worker

023-03-14T16:53:06Z WRN error executing task error="command exited abnormally with code 1" task={"commands":[{"name":"blender-render","parameters":{"args":["--render-output","/mike-redcubes-longer-timeout/2023-03-14_163903/######","--render-format","PNG","--render-frame","77"],"argsBefore":[],"blendfile":"/data/jobs/mike-redcubes-longer-timeout-a2gd/mike-redcubes.flamenco.blend","exe":"blender","exeArgs":"-b -y"}}],"job":"78c28c62-2a0f-4ec0-bf52-f3b03f19f3c6","job_priority":50,"job_type":"simple-blender-render","name":"render-77","priority":50,"status":"active","task_type":"blender","uuid":"0c5959e0-0ca9-42cd-9104-454d5571e14f"}

This is what worker executes... and the render-output path does not exist on the worker machine:

blender -b -y /data/jobs/mike-redcubes-longer-timeout-a2gd/mike-redcubes.flamenco.blend --render-output /mike-redcubes-longer-timeout/2023-03-14_163903/###### --render-format PNG --render-frame 77 -d

End of blender execution...

...
Fra:77 Mem:161.87M (Peak 161.88M) | Time:11:44.91 | Remaining:00:22.73 | Mem:151.27M, Peak:151.27M | Scene, ViewLayer | Sample 124/128
Fra:77 Mem:225.16M (Peak 320.08M) | Time:12:20.18 | Mem:151.27M, Peak:151.27M | Scene, ViewLayer | Sample 128/128
Fra:77 Mem:225.16M (Peak 320.08M) | Time:12:20.18 | Mem:151.27M, Peak:151.27M | Scene, ViewLayer | Finished
imb_savepng: Cannot open file for writing: '/mike-redcubes-longer-timeout/2023-03-14_163903/000077.png'
/mike-redcubes-longer-timeout/2023-03-14_163903/000077.png: No such file or directory
Error: Render error (No such file or directory) cannot save: '/mike-redcubes-longer-timeout/2023-03-14_163903/000077.png'
 Time: 12:20.62 (Saving: 00:00.42)

Blender quit

Question:

How does worker know what the output path is set to?
Why on my set up is the Render Output Path seen in the Blender Addon invalid.

**System Information** - MacOS NFS mount to NFS Server on local home network. - Blender 3.4 with Flamenco 3.1 addon. - Blender file is saved on NFS Server. Flamenco Manager and Worker are both configured using yaml files. No GUI Wizard was used. See attached. **Flamenco Version** Is Broken: 3.1 **Short description of error** Running a Flamenco Job, the worker starts to successfully render and when it attempts to write the output it has been instructed to use "/mike-redcubes-longer-timeout/2023-03-14_163903/000077.png". This path is invalid and blender quits with error. Flamenco re-enqueues a few times and fails the job eventually. Looking at the blender addon plugin, it has an option to set the Render Output Path but says it is not enabled. I can open a Modal dialogue box and select an NFS path... but this is from the MacOS NFS Mount perspective which the Worker would need a different root path. The Flamenco Manager configuration has ``` local_manager_storage_path: /data/flamenco-manager-storage shared_storage_path: /data ``` The /data is successfully mounted to the NFS server path. The blender file is successfully found/read by Blender on the worker. **Exact steps for others to reproduce the error** Manager Configuration File ``` _meta: version: 3 manager_name: Flamenco Manager database: flamenco-manager.sqlite listen: :8080 autodiscoverable: true local_manager_storage_path: /data/flamenco-manager-storage shared_storage_path: /data shaman: enabled: true garbageCollect: period: 24h0m0s maxAge: 744h0m0s extraCheckoutPaths: [] task_timeout: 60m0s worker_timeout: 30m0s blocklist_threshold: 3 task_fail_after_softfail_count: 3 variables: blender: values: - platform: linux value: blender blenderArgs: values: - platform: all value: -b -y ``` Worker Configuration ``` manager_url: http://flamenco-service.flamenco-manager.svc:8080/ task_types: [blender, ffmpeg, file-management, misc] ``` NFS Mount from the *Worker* context shows the NFS Server filestore contents as expected: ``` ls /data/ file-store jobs output projects ``` Flamenco Job received by Worker ``` 023-03-14T16:53:06Z WRN error executing task error="command exited abnormally with code 1" task={"commands":[{"name":"blender-render","parameters":{"args":["--render-output","/mike-redcubes-longer-timeout/2023-03-14_163903/######","--render-format","PNG","--render-frame","77"],"argsBefore":[],"blendfile":"/data/jobs/mike-redcubes-longer-timeout-a2gd/mike-redcubes.flamenco.blend","exe":"blender","exeArgs":"-b -y"}}],"job":"78c28c62-2a0f-4ec0-bf52-f3b03f19f3c6","job_priority":50,"job_type":"simple-blender-render","name":"render-77","priority":50,"status":"active","task_type":"blender","uuid":"0c5959e0-0ca9-42cd-9104-454d5571e14f"} ``` This is what worker executes... and the render-output path does not exist on the worker machine: ``` blender -b -y /data/jobs/mike-redcubes-longer-timeout-a2gd/mike-redcubes.flamenco.blend --render-output /mike-redcubes-longer-timeout/2023-03-14_163903/###### --render-format PNG --render-frame 77 -d ``` End of blender execution... ``` ... Fra:77 Mem:161.87M (Peak 161.88M) | Time:11:44.91 | Remaining:00:22.73 | Mem:151.27M, Peak:151.27M | Scene, ViewLayer | Sample 124/128 Fra:77 Mem:225.16M (Peak 320.08M) | Time:12:20.18 | Mem:151.27M, Peak:151.27M | Scene, ViewLayer | Sample 128/128 Fra:77 Mem:225.16M (Peak 320.08M) | Time:12:20.18 | Mem:151.27M, Peak:151.27M | Scene, ViewLayer | Finished imb_savepng: Cannot open file for writing: '/mike-redcubes-longer-timeout/2023-03-14_163903/000077.png' /mike-redcubes-longer-timeout/2023-03-14_163903/000077.png: No such file or directory Error: Render error (No such file or directory) cannot save: '/mike-redcubes-longer-timeout/2023-03-14_163903/000077.png' Time: 12:20.62 (Saving: 00:00.42) Blender quit ``` Question: - How does worker know what the output path is set to? - Why on my set up is the Render Output Path seen in the Blender Addon invalid.

Screenshot 2023-03-14 at 1.23.48 PM.png

32 KiB

MichaelC commented

2023-03-14 19:06:22 +01:00

I noticed nothing is set in the Blender Addon "Render Output Root" which is my mistake. Setting this to the NFS output folder I end up with what looks like a relative path to where I saved the blender file (on the NFS filestore).

//../output/

However, I have managed to somehow update the Render Output Path field despite it saying I cannot edit it and cannot set it back. :/

I noticed nothing is set in the Blender Addon "Render Output Root" which is my mistake. Setting this to the NFS output folder I end up with what looks like a relative path to where I saved the blender file (on the NFS filestore). //../output/ However, I have managed to somehow update the Render Output Path field despite it saying I cannot edit it and cannot set it back. :/

MichaelC commented

2023-03-14 19:12:46 +01:00

Not sure how I was allowed to edit a disabled addon field... but not its stuck with a path that the worker will not have available.

{ "args": [ "--render-output", "/Users/mike.cook/flamenco_nfs/flamenco/output/mike-redcubes-longer-timeout/2023-03-14_180235/######", "--render-format", "PNG", "--render-frame", "77" ], "argsBefore": [], "blendfile": "{jobs}/mike-redcubes-longer-timeout-njdo/mike-redcubes.flamenco.blend", "exe": "{blender}", "exeArgs": "{blenderArgs}" }

Not sure how I was allowed to edit a disabled addon field... but not its stuck with a path that the worker will not have available. ``` { "args": [ "--render-output", "/Users/mike.cook/flamenco_nfs/flamenco/output/mike-redcubes-longer-timeout/2023-03-14_180235/######", "--render-format", "PNG", "--render-frame", "77" ], "argsBefore": [], "blendfile": "{jobs}/mike-redcubes-longer-timeout-njdo/mike-redcubes.flamenco.blend", "exe": "{blender}", "exeArgs": "{blenderArgs}" } ```

Screenshot 2023-03-14 at 2.12.09 PM.png

32 KiB

MichaelC commented

2023-03-14 20:03:31 +01:00

I managed to start-over and set the Root Output Path to what the workers' NFS mount path expects and the job was launched and saved successfully.

So I think I've answered my question that it is expected that the NFS Mounts should be identical on each workstation including the one submitting Flamenco jobs. I can re-configure my NFS setup to this requirement.

I managed to start-over and set the Root Output Path to what the workers' NFS mount path expects and the job was launched and saved successfully. So I think I've answered my question that it is expected that the NFS Mounts should be identical on each workstation including the one submitting Flamenco jobs. I can re-configure my NFS setup to this requirement.

Sybren A. Stüvel commented

2023-03-20 11:39:48 +01:00

So I think I've answered my question that it is expected that the NFS Mounts should be identical on each workstation including the one submitting Flamenco jobs.

That is 90% correct. With Flamenco's default config it's actually 100% correct, but you can use the variable 'audience' to make a mapping between 'paths on the submitter side' vs. 'path on the Workers side'.

For differences between paths on different platforms (like macOS vs. Linux) there's the multi-platform support.

> So I think I've answered my question that it is expected that the NFS Mounts should be identical on each workstation including the one submitting Flamenco jobs. That is 90% correct. With Flamenco's default config it's actually 100% correct, but you can use the [variable 'audience'](https://flamenco.blender.org/usage/variables/#advanced-audience) to make a mapping between 'paths on the submitter side' vs. 'path on the Workers side'. For differences between paths on different platforms (like macOS vs. Linux) there's the [multi-platform support](https://flamenco.blender.org/usage/variables/multi-platform/).

Sybren A. Stüvel closed this issue

2023-03-20 11:39:48 +01:00

Sign in to join this conversation.