studio/flamenco
studio/flamenco
12
30
Fork 38
Code Issues 63 Pull Requests 4 Releases Wiki Activity

Flamenco Manager does not report the storage location correctly when mapping to a bare drive path #104237

New Issue
Closed
opened 2023-07-24 01:39:07 +02:00 by NovaXP · 8 comments
NovaXP commented 2023-07-24 01:39:07 +02:00
Copy Link

System Information
Operating System(s): Windows 10 22H2, Ubuntu 22.04.02 LTS

Flamenco Version
Is Broken: 3.2
Worked OK: 3.1

Short description of error
When using Flamenco 3.2, Linux machines that have tasks sent to them will be given the file path using backslashes instead of forward slashes, resulting in a failure.

Exact steps for others to reproduce the error
In Flamenco 3.1, the variables below in flamenco-manager.yaml would work properly, resulting in the following log:

my_storage:
    is_twoway: true
    values:
    - platform: linux
      value: /mnt
    - platform: windows
      value: N:\

2023-02-25T23:49:35-06:00 Task assigned to worker DESKTOP-660 (8b8221d2-2176-4506-b1f6-9a7c0049e0a0)
2023-02-25T23:49:35-06:00 task changed status queued -> active
going to run: blender ["-b" "-y" "/mnt\Blender Renders\Project\AKM 34 Render Test.flamenco.blend" "--render-output" "/mnt/Blender Renders/Renders/AKM 34 Render Test/2023-02-25_234211/######" "--render-format" "JPEG" "--render-frame" "880"]
pid=9392 > Blender 3.4.1 (hash 55485cb379f7 built 2022-12-20 00:46:45)
pid=9392 > Read prefs: /home/user/.config/blender/3.4/config/userpref.blend
pid=9392 > Read blend: /mnt/Blender Renders/Project/AKM 34 Render Test.flamenco.blend

However, when using the same variables in Flamenco 3.2, the following log appears:

2023-07-23T18:25:10-05:00 Task assigned to worker DESKTOP-660 (7975d150-9825-4b18-a168-3fe7240f8008)
2023-07-23T18:25:10-05:00 task changed status queued -> active
going to run: blender ["-b" "-y" "/mnt\Blender Renders\Project\AKM 34 Render Test.flamenco.blend" "--render-output" "/mnt/Blender Renders/Renders/AKM 34 Render Test/2023-07-23_182505/######" "--render-format" "JPEG" "--render-frame" "863"]
pid=24820 > Blender 3.6.1 (hash 8bda729ef4dc built 2023-07-17 23:31:33)
pid=24820 > Read prefs: "/home/user/.config/blender/3.6/config/userpref.blend"
pid=24820 > Error: Cannot read file "/mnt\Blender Renders\Project\AKM 34 Render Test.flamenco.blend": No such file or directory
pid=24820 >
pid=24820 > Blender quit
Failed: command exited abnormally with code 1

On Windows machines this error doesn't appear and Flamenco 3.2 works normally. For context, I didn't use Shaman in either 3.1 or 3.2.

**System Information** Operating System(s): Windows 10 22H2, Ubuntu 22.04.02 LTS **Flamenco Version** Is Broken: 3.2 Worked OK: 3.1 **Short description of error** When using Flamenco 3.2, Linux machines that have tasks sent to them will be given the file path using backslashes instead of forward slashes, resulting in a failure. **Exact steps for others to reproduce the error** In Flamenco 3.1, the variables below in flamenco-manager.yaml would work properly, resulting in the following log: ```yaml my_storage: is_twoway: true values: - platform: linux value: /mnt - platform: windows value: N:\ ``` > 2023-02-25T23:49:35-06:00 Task assigned to worker DESKTOP-660 (8b8221d2-2176-4506-b1f6-9a7c0049e0a0) 2023-02-25T23:49:35-06:00 task changed status queued -> active going to run: blender ["-b" "-y" "/mnt\\Blender Renders\\Project\\AKM 34 Render Test.flamenco.blend" "--render-output" "/mnt/Blender Renders/Renders/AKM 34 Render Test/2023-02-25_234211/######" "--render-format" "JPEG" "--render-frame" "880"] pid=9392 > Blender 3.4.1 (hash 55485cb379f7 built 2022-12-20 00:46:45) pid=9392 > Read prefs: /home/user/.config/blender/3.4/config/userpref.blend pid=9392 > Read blend: /mnt/Blender Renders/Project/AKM 34 Render Test.flamenco.blend However, when using the same variables in Flamenco 3.2, the following log appears: >2023-07-23T18:25:10-05:00 Task assigned to worker DESKTOP-660 (7975d150-9825-4b18-a168-3fe7240f8008) 2023-07-23T18:25:10-05:00 task changed status queued -> active going to run: blender ["-b" "-y" "/mnt\\Blender Renders\\Project\\AKM 34 Render Test.flamenco.blend" "--render-output" "/mnt/Blender Renders/Renders/AKM 34 Render Test/2023-07-23_182505/######" "--render-format" "JPEG" "--render-frame" "863"] pid=24820 > Blender 3.6.1 (hash 8bda729ef4dc built 2023-07-17 23:31:33) pid=24820 > Read prefs: "/home/user/.config/blender/3.6/config/userpref.blend" pid=24820 > Error: Cannot read file "/mnt\Blender Renders\Project\AKM 34 Render Test.flamenco.blend": No such file or directory pid=24820 > pid=24820 > Blender quit Failed: command exited abnormally with code 1 On Windows machines this error doesn't appear and Flamenco 3.2 works normally. For context, I didn't use Shaman in either 3.1 or 3.2.
Sybren A. Stüvel commented 2023-07-24 13:09:00 +02:00
Owner
Copy Link

Please include steps to reproduce this. I'm fairly sure that it's relevant which OS the Manager is running on, and same for the Workers. This, however, is not specified in the report.

Please include steps to reproduce this. I'm fairly sure that it's relevant which OS the Manager is running on, and same for the Workers. This, however, is not specified in the report.
NovaXP commented 2023-07-24 21:03:29 +02:00
Author
Copy Link

Sorry about that, the manager is running on Windows 10, two other workers (one of them being the network storage) are also running on W10 without issue, but the one having problems is on Ubutnu.

Reproducing it (at least on my end) is pretty straightforward:

  1. Get Flamenco 3.2 from https://flamenco.blender.org/download/.
  2. Extract the files and perform the first-time setup for Flamenco-Manager.exe.
  3. During setup, I set the shared storage location to a folder on my network mapped drive, N:\Blender Renders (the network drive is N:\ on all of the Windows machines I use).
  4. Once the setup is finished, I shut down Flamenco-Manager.exe and add the code below to flamenco-manager.yaml:
 my_storage:
    is_twoway: true
    values:
    - platform: linux
      value: /mnt
    - platform: windows
      value: N:\
  1. I download a copy of the the Linux version of Flamenco 3.2, and put both the Windows and Linux versions of flamenco-worker in this location on the network drive for easy access "N:\Blender Renders\Flamenco 3.2". Then I start running Flamenco-Manager.exe again.
  2. I get on my Ubuntu machine (DESKTOP-660) and use the terminal to mount the network storage drive, the location on it is "/mnt" (if you went to N:\ on a Windows machine or /mnt on the Ubuntu machine, it would lead you to the exact same place).
  3. I use terminal to run flamenco-worker on the Ubuntu machine, and see it listed on the list of workers on the Flamenco Manager web interface.
  4. I go back to my main computer (the one running flamenco-manager.exe) and get the Blender Addon from the web interface, and install it on to Blender 3.6.1. I take the IP address of the main computer (the same one the web UI says to use) and put it in the Manager URL field in the addon preferences, and it says that Flamenco 3.2 was detected.
  5. I pick the file I want to render, pack all of the resources into the .blend, and save a copy of it at the location "N:\Blender Renders\Project".
  6. I open that copy on the main computer in Blender 3.6.1, go to the Flamenco options on the side panel, and retrieve the list of job types. I set the job type to Simple Blender Render, and make sure the output format is an image file type like PNG or JPEG.
  7. I make sure the Render Output Root is set to "N:\Blender Renders\Renders" (and that the "relative path" option isn't selected), and then submit it to Flamenco.
  8. Flamenco Manager gets the job, and will assign tasks to the Ubuntu machine (DESKTOP-660). However, any tasks sent to it will soft fail twice before it stops giving any tasks to DESKTOP-660, and the logs report the error I put in the post above.

When using the same computers and configuration on Flamenco 3.1, both Windows and Ubuntu machines were able to run through all the tasks normally. However, I was also using Blender 3.4 and 3.5 previously. I can test Flamenco 3.2 with those versions to see if Blender is actually the cause when I have some time.

Sorry about that, the manager is running on Windows 10, two other workers (one of them being the network storage) are also running on W10 without issue, but the one having problems is on Ubutnu. Reproducing it (at least on my end) is pretty straightforward: 1. Get Flamenco 3.2 from https://flamenco.blender.org/download/. 2. Extract the files and perform the first-time setup for Flamenco-Manager.exe. 3. During setup, I set the shared storage location to a folder on my network mapped drive, N:\Blender Renders (the network drive is N:\ on all of the Windows machines I use). 4. Once the setup is finished, I shut down Flamenco-Manager.exe and add the code below to flamenco-manager.yaml: ```yaml my_storage: is_twoway: true values: - platform: linux value: /mnt - platform: windows value: N:\ ``` 5. I download a copy of the the Linux version of Flamenco 3.2, and put both the Windows and Linux versions of flamenco-worker in this location on the network drive for easy access "N:\Blender Renders\Flamenco 3.2". Then I start running Flamenco-Manager.exe again. 6. I get on my Ubuntu machine (DESKTOP-660) and use the terminal to mount the network storage drive, the location on it is "/mnt" (if you went to N:\ on a Windows machine or /mnt on the Ubuntu machine, it would lead you to the exact same place). 7. I use terminal to run flamenco-worker on the Ubuntu machine, and see it listed on the list of workers on the Flamenco Manager web interface. 8. I go back to my main computer (the one running flamenco-manager.exe) and get the Blender Addon from the web interface, and install it on to Blender 3.6.1. I take the IP address of the main computer (the same one the web UI says to use) and put it in the Manager URL field in the addon preferences, and it says that Flamenco 3.2 was detected. 9. I pick the file I want to render, pack all of the resources into the .blend, and save a copy of it at the location "N:\Blender Renders\Project". 10. I open that copy on the main computer in Blender 3.6.1, go to the Flamenco options on the side panel, and retrieve the list of job types. I set the job type to Simple Blender Render, and make sure the output format is an image file type like PNG or JPEG. 11. I make sure the Render Output Root is set to "N:\Blender Renders\Renders" (and that the "relative path" option isn't selected), and then submit it to Flamenco. 12. Flamenco Manager gets the job, and will assign tasks to the Ubuntu machine (DESKTOP-660). However, any tasks sent to it will soft fail twice before it stops giving any tasks to DESKTOP-660, and the logs report the error I put in the post above. When using the same computers and configuration on Flamenco 3.1, both Windows and Ubuntu machines were able to run through all the tasks normally. However, I was also using Blender 3.4 and 3.5 previously. I can test Flamenco 3.2 with those versions to see if Blender is actually the cause when I have some time.
Sybren A. Stüvel added the
Type
Bug
 label 2023-07-28 14:17:00 +02:00
Sybren A. Stüvel commented 2023-07-28 14:17:13 +02:00
Owner
Copy Link

Thanks for the detailed info. This certainly looks like a bug to me.

Thanks for the detailed info. This certainly looks like a bug to me.
Sybren A. Stüvel commented 2023-07-31 15:38:02 +02:00
Owner
Copy Link

Yup, that was a bug. Thanks again for the report!

Yup, that was a bug. Thanks again for the report!
Sybren A. Stüvel changed title from Flamenco Manager uses backslashes instead of forward slashes for file paths when assigning a job to Linux workers. to Flamenco Manager does not report the storage location correctly when mapping to a bare drive path 2023-07-31 15:38:55 +02:00
sheaf commented 2023-08-04 11:23:04 +02:00
Copy Link

I am a first time user, so I may well just be doing the install wrong. I have a similar setup to NovaXP, but on slightly different hardware/APIs.

A Windows 10 computer acts both as my flamenco-manager as well as the host for my file sharing service (the root is C:\blends). I am using an OS X Catalina computer as my flamenco-worker, with the shared directory at /Volumes/blends . Both are connected through their respective OS-proprietary SMB services. Both are running the most recent version of Flamenco (3.2) and Blender (3.6.1).

During manager setup on Windows, I set the shared storage location to my shared directory, C:\blends (note, unlike NovaXP, my blender folder is the only thing exposed on the network).

With setup finished, I append the following to flamenco-manager.yaml :

 my_storage:
    is_twoway: true
    values:
    - platform: windows
      value: C:\
    - platform: darwin
      value: /Volumes

I only have one worker (on Catalina named myComputer.local, with username myUser), so i just launch flamenco-worker from /Users/myUser/Downloads folder. C:\blends is networked to /Volumes/blends .

I entered my manager's IP into the manager's Blender add-ons prompt as instructed, as well as routing it to C:\blends . With worker running on Catalina, Echo Sleep Test ran flawlessly, and there are no errors on startup of either the manager or worker processes.

I have a test project located at C:\blends\test with savefile test.blend, featuring 5 frames of default cube.

For Simple Blender Render, I set render to PNG with Render Output Root as C:\blends\renders ('relative path' disabled). Submitting to Flamenco, the log outputs:

2023-08-04T01:41:19-07:00 Task assigned to worker myComputer.local (3d416904-d075-463f-ab31-53ed4195df28)
2023-08-04T01:41:19-07:00 task changed status queued -> active
going to run: blender ["-b" "-y" "C:\blends\test\test.flamenco.blend" "--render-output" "C:/blends/renders/test-1/2023-08-04_014115/######" "--render-format" "PNG" "--render-frame" "2"]
pid=64757 > Blender 3.6.1 (hash 8bda729ef4dc built 2023-07-17 23:32:25)
pid=64757 > Read prefs: "/Users/myUser/Library/Application Support/Blender/3.6/config/userpref.blend"
pid=64757 > Error: Cannot read file "/Users/myUser/C:\blends\test\test.flamenco.blend": No such file or directory
pid=64757 >
pid=64757 > Blender quit
Failed: command exited abnormally with code 1
2023-08-04T01:41:20-07:00 Task failed by worker myComputer.local (3d416904-d075-463f-ab31-53ed4195df28), Manager will fail the entire job as there are no more workers left for tasks of type "blender".
2023-08-04T01:41:20-07:00 task changed status active -> failed

Of course, the issue is the construction/parsing of the path '/Users/myUser/C:\blends\test\test.flamenco.blend', but I am unsure whether it is a replication of NovaXP's bug, or if I'm botching the yaml file somehow or mac user groups need to change for my setup. Apologies if it is the latter.

I am a first time user, so I may well just be doing the install wrong. I have a similar setup to NovaXP, but on slightly different hardware/APIs. A Windows 10 computer acts both as my flamenco-manager as well as the host for my file sharing service (the root is C:\blends). I am using an OS X Catalina computer as my flamenco-worker, with the shared directory at /Volumes/blends . Both are connected through their respective OS-proprietary SMB services. Both are running the most recent version of Flamenco (3.2) and Blender (3.6.1). During manager setup on Windows, I set the shared storage location to my shared directory, C:\blends (note, unlike NovaXP, my blender folder is the only thing exposed on the network). With setup finished, I append the following to flamenco-manager.yaml : ``` my_storage: is_twoway: true values: - platform: windows value: C:\ - platform: darwin value: /Volumes ``` I only have one worker (on Catalina named myComputer.local, with username myUser), so i just launch flamenco-worker from /Users/myUser/Downloads folder. C:\blends is networked to /Volumes/blends . I entered my manager's IP into the manager's Blender add-ons prompt as instructed, as well as routing it to C:\blends . With worker running on Catalina, Echo Sleep Test ran flawlessly, and there are no errors on startup of either the manager or worker processes. I have a test project located at C:\blends\test with savefile test.blend, featuring 5 frames of default cube. For Simple Blender Render, I set render to PNG with Render Output Root as C:\blends\renders ('relative path' disabled). Submitting to Flamenco, the log outputs: > 2023-08-04T01:41:19-07:00 Task assigned to worker myComputer.local (3d416904-d075-463f-ab31-53ed4195df28) > 2023-08-04T01:41:19-07:00 task changed status queued -> active > going to run: blender ["-b" "-y" "C:\\blends\\test\\test.flamenco.blend" "--render-output" "C:/blends/renders/test-1/2023-08-04_014115/######" "--render-format" "PNG" "--render-frame" "2"] > pid=64757 > Blender 3.6.1 (hash 8bda729ef4dc built 2023-07-17 23:32:25) > pid=64757 > Read prefs: "/Users/myUser/Library/Application Support/Blender/3.6/config/userpref.blend" > pid=64757 > Error: Cannot read file "/Users/myUser/C:\blends\test\test.flamenco.blend": No such file or directory > pid=64757 > > pid=64757 > Blender quit > Failed: command exited abnormally with code 1 > 2023-08-04T01:41:20-07:00 Task failed by worker myComputer.local (3d416904-d075-463f-ab31-53ed4195df28), Manager will fail the entire job as there are no more workers left for tasks of type "blender". > 2023-08-04T01:41:20-07:00 task changed status active -> failed Of course, the issue is the construction/parsing of the path '/Users/myUser/C:\blends\test\test.flamenco.blend', but I am unsure whether it is a replication of NovaXP's bug, or if I'm botching the yaml file somehow or mac user groups need to change for my setup. Apologies if it is the latter.
Sybren A. Stüvel commented 2023-08-04 11:55:51 +02:00
Owner
Copy Link

I am a first time user, so I may well just be doing the install wrong.

Given that the fix hasn't been released yet, what are you installing? Did you build the fixed Flamenco yourself?

Both are running the most recent version of Flamenco (3.2) and Blender (3.6.1).

Ah yes, that version still has the bug, so there is no use in re-testing with that particular one. You'll either have to wait for Flamenco 3.3 to be released, build it yourself, or grab the special SIGGRAPH 2023 build. Be aware that the latter is an alpha release, i.e. is not feature-complete and may have bugs.

> I am a first time user, so I may well just be doing the install wrong. Given that the fix hasn't been released yet, what are you installing? Did you build the fixed Flamenco yourself? > Both are running the most recent version of Flamenco (3.2) and Blender (3.6.1). Ah yes, that version still has the bug, so there is no use in re-testing with that particular one. You'll either have to wait for Flamenco 3.3 to be released, [build it yourself](https://flamenco.blender.org/development/getting-started/), or grab the special [SIGGRAPH 2023 build](https://flamenco.blender.org/siggraph2023). Be aware that the latter is an alpha release, i.e. is not feature-complete and may have bugs.
Bryce-Christopher commented 2023-09-15 00:28:18 +02:00
Copy Link

@dr.sybren I think I may be experiencing this same issue in my setup as well. Can you please confirm whether there is an immediate solution in my case or if I need to wait until 3.3 as well?

Flamenco 3.2 manager is installed on a Win10 machine with Shaman enabled. The local worker works great. We tried to add a second worker on a different machine (Win 11) but it keeps giving an error: "Unable to read file "F:\jobs\test\test09-14.blend" Invalid Argument" and it soft-fails the job.

The same drive is mapped to F:\ on both systems. The drive itself is an SMB share hosted by the same computer that is running the Flamenco Manager

I have also tried a different computer with a brand new Windows 10 install and it gets the same issue. I have made sure symlinks are enabled on all relevant machines. I spent several hours on this today and haven't been able to get anywhere with it. What is your suggestion?

Thanks

@dr.sybren I think I may be experiencing this same issue in my setup as well. Can you please confirm whether there is an immediate solution in my case or if I need to wait until 3.3 as well? Flamenco 3.2 manager is installed on a Win10 machine with Shaman enabled. The local worker works great. We tried to add a second worker on a different machine (Win 11) but it keeps giving an error: "Unable to read file "F:\jobs\test\test09-14.blend" Invalid Argument" and it soft-fails the job. The same drive is mapped to F:\ on both systems. The drive itself is an SMB share hosted by the same computer that is running the Flamenco Manager I have also tried a different computer with a brand new Windows 10 install and it gets the same issue. I have made sure symlinks are enabled on all relevant machines. I spent several hours on this today and haven't been able to get anywhere with it. What is your suggestion? Thanks
Sybren A. Stüvel commented 2023-09-26 17:46:00 +02:00
Owner
Copy Link

@dr.sybren I think I may be experiencing this same issue in my setup as well. Can you please confirm whether there is an immediate solution in my case or if I need to wait until 3.3 as well?

The immediate solution would be to grab the sources and build Flamenco yourself. This is documented at https://flamenco.blender.org/development/getting-started/

Symlinks are still notoriously difficult for Windows machines. I would recommend running Linux, at least for the Manager and the Workers, or turning off the Shaman system.

> @dr.sybren I think I may be experiencing this same issue in my setup as well. Can you please confirm whether there is an immediate solution in my case or if I need to wait until 3.3 as well? The immediate solution would be to grab the sources and build Flamenco yourself. This is documented at https://flamenco.blender.org/development/getting-started/ Symlinks are still notoriously difficult for Windows machines. I would recommend running Linux, at least for the Manager and the Workers, or turning off the Shaman system.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels   
Good First Issue
  
Priority
High

  
Priority
Low

  
Priority
Normal

  
Status
Archived

  
Status
Confirmed

  
Status
Needs Info from Developers

  
Status
Needs Information from User

  
Status
Needs Triage

  
Status
Resolved

  
Type
Bug

  
Type
Design

  
Type
Job Type
Custom Flamenco job type

  
Type
Known Issue

  
Type
Patch

  
Type
Report

  
Type
To Do

No Label
Good First Issue
Priority
High
Priority
Low
Priority
Normal
Status
Archived
Status
Confirmed
Status
Needs Info from Developers
Status
Needs Information from User
Status
Needs Triage
Status
Resolved
Type
Bug
Type
Design
Type
Job Type
Type
Known Issue
Type
Patch
Type
Report
Type
To Do
Milestone
Clear milestone
No items
No Milestone
Assignees
Clear assignees
No Assignees
4 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: studio/flamenco#104237
No description provided.
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?