Added ToNative to the blendfile args to resolve workers failing due to windows -> linux pathing #104295

Jamie-Newbon · 2024-04-02T21:29:53+02:00

Jamie-Newbon commented

2024-04-02 21:29:53 +02:00

First-time contributor

This is a fix for #104237 and #104246 where the blend file would use the wrong slash's when using a windows manager and Linux workers

I have not tested this with linux manager and windows workers

This is a fix for #104237 and #104246 where the blend file would use the wrong slash's when using a windows manager and Linux workers I have not tested this with linux manager and windows workers

Jamie-Newbon added 1 commit 2024-04-02 21:29:56 +02:00

Added ToNative to the blendfile args to resolve windows manager to linux worker bug e84d18ec8e

Sybren A. Stüvel commented

2024-04-04 10:29:10 +02:00

Thanks for the PR, and your willingness to help the project :)

The referenced issues were already closed, with one actually being fixed and the other where the original reporter was unresponsive, so I don't know exactly how to deal with this PR. If this issue is still there (in the main branch, that is), do let me know.

Thanks for the PR, and your willingness to help the project :) The referenced issues were already closed, with one actually being fixed and the other where the original reporter was unresponsive, so I don't know exactly how to deal with this PR. If this issue is still there (in the `main` branch, that is), do let me know.

Sybren A. Stüvel added the

Status

Needs Information from User

label 2024-04-04 10:33:50 +02:00

Sybren A. Stüvel commented

2024-04-04 10:48:25 +02:00

PS: Please test with 3.5-beta0, I've just released that to the 'experimental' download section.

PS: Please test with 3.5-beta0, I've just released that to [the 'experimental' download section](https://flamenco.blender.org/download/#go-experimental).

Erik-Hals commented

2024-04-17 09:45:04 +02:00

First-time contributor

In 3.5-beta1
When submitting to manager on Windows and when the file and renderoutput is inside the shared folder, I get this error on my Mac worker:
2024-04-17T08:36:18+02:00 INF going to run: blender -b -y '/Volumes/192.168.0.210/render\Project1\Projectblend.flamenco.blend' --render-output '/Volumes/192.168.0.210/render/Project1/Render/Project1/2024-04-17_084036/######' --render-format PNG --render-frame 3,4 command=blender-render task=84c530af-9bc9-436c-a84e-d1bde891ca02
(...)
2024-04-17T08:36:19+02:00 WRN error executing task error="command exited abnormally with code 1" task={"commands":[{"name":"blender-render","parameters":{"args":["--render-output","/Volumes/192.168.0.210/render/Project1/Render/Project1/2024-04-17_084036/######","--render-format","PNG","--render-frame","3,4"],"argsBefore":[],"blendfile":"/Volumes/192.168.0.210/render\\Project1\\Projectblend.flamenco.blend","exe":"blender","exeArgs":"-b -y"}}],"job":"28c4a99a-8157-40c0-9047-9c06fad80b2f","job_priority":50,"job_type":"simple-blender-render","name":"render-3,4","priority":50,"status":"active","task_type":"blender","uuid":"84c530af-9bc9-436c-a84e-d1bde891ca02"}
When the blender-file is outside the shared folder, everything works. It seems the path to the blender-file gets the wrong slashes still, the output path looks ok. It seems the PR would address this, but I haven't built it myself.

This issue seems related: #100540

In 3.5-beta1 When submitting to manager on Windows and when the file and renderoutput is inside the shared folder, I get this error on my Mac worker: `2024-04-17T08:36:18+02:00 INF going to run: blender -b -y '/Volumes/192.168.0.210/render\Project1\Projectblend.flamenco.blend' --render-output '/Volumes/192.168.0.210/render/Project1/Render/Project1/2024-04-17_084036/######' --render-format PNG --render-frame 3,4 command=blender-render task=84c530af-9bc9-436c-a84e-d1bde891ca02` (...) `2024-04-17T08:36:19+02:00 WRN error executing task error="command exited abnormally with code 1" task={"commands":[{"name":"blender-render","parameters":{"args":["--render-output","/Volumes/192.168.0.210/render/Project1/Render/Project1/2024-04-17_084036/######","--render-format","PNG","--render-frame","3,4"],"argsBefore":[],"blendfile":"/Volumes/192.168.0.210/render\\Project1\\Projectblend.flamenco.blend","exe":"blender","exeArgs":"-b -y"}}],"job":"28c4a99a-8157-40c0-9047-9c06fad80b2f","job_priority":50,"job_type":"simple-blender-render","name":"render-3,4","priority":50,"status":"active","task_type":"blender","uuid":"84c530af-9bc9-436c-a84e-d1bde891ca02"} ` When the blender-file is outside the shared folder, everything works. It seems the path to the blender-file gets the wrong slashes still, the output path looks ok. It seems the PR would address this, but I haven't built it myself. This issue seems related: #100540

Jamie-Newbon commented

2024-04-17 15:17:38 +02:00

First-time contributor

Sorry for the delay, When I submitted the PR I had got the issue fixed, then to confirm it actually did fix the issue I reverted my changes to see the the job would render which it didn't thus confirming to me that the issue was still there in main branch at time of submission.

We have now swapped the Manager over to run on Linux and have noticed that instead of the blend file having the wrong slashes, its the render output instead and only on the Windows machines.

Windows Box

going to run:

    'C:\Program Files\Blender Foundation\Blender 4.1\blender.exe' -b -y 'V:\blenderrender\Projects\Donut\donut2.flamenco.blend' --render-output 'V:\blenderrender\Projects/Donut/donut2/2024-04-17_042428/######' --render-format PNG --render-frame 30

Linux Box (Running my changes)

going to run:

    blender -b -y /mnt/my_windows_stuffs/blenderrender/Projects/Donut/donut2.flamenco.blend --render-output '/mnt/my_windows_stuffs/blenderrender/Projects/Donut/donut2/2024-04-17_042428/######' --render-format PNG --render-frame 18

However as windows seems the handles both type of slashes, the render still works.

Sorry for the delay, When I submitted the PR I had got the issue fixed, then to confirm it actually did fix the issue I reverted my changes to see the the job would render which it didn't thus confirming to me that the issue was still there in main branch at time of submission. We have now swapped the Manager over to run on Linux and have noticed that instead of the blend file having the wrong slashes, its the render output instead and only on the Windows machines. Windows Box ``` going to run: 'C:\Program Files\Blender Foundation\Blender 4.1\blender.exe' -b -y 'V:\blenderrender\Projects\Donut\donut2.flamenco.blend' --render-output 'V:\blenderrender\Projects/Donut/donut2/2024-04-17_042428/######' --render-format PNG --render-frame 30 ``` Linux Box (Running my changes) ``` going to run: blender -b -y /mnt/my_windows_stuffs/blenderrender/Projects/Donut/donut2.flamenco.blend --render-output '/mnt/my_windows_stuffs/blenderrender/Projects/Donut/donut2/2024-04-17_042428/######' --render-format PNG --render-frame 18 ``` However as windows seems the handles both type of slashes, the render still works.

👍 1

Sybren A. Stüvel commented

2024-04-18 13:06:59 +02:00

Thanks for coming back with more info :)

2024-04-17T08:36:19+02:00 WRN error executing task error="command exited abnormally with code 1" task={"commands":[{"name":"blender-render","parameters":{"args":["--render-output","/Volumes/192.168.0.210/render/Project1/Render/Project1/2024-04-17_084036/######","--render-format","PNG","--render-frame","3,4"],"argsBefore":[],"blendfile":"/Volumes/192.168.0.210/render\\Project1\\Projectblend.flamenco.blend","exe":"blender","exeArgs":"-b -y"}}],"job":"28c4a99a-8157-40c0-9047-9c06fad80b2f","job_priority":50,"job_type":"simple-blender-render","name":"render-3,4","priority":50,"status":"active","task_type":"blender","uuid":"84c530af-9bc9-436c-a84e-d1bde891ca02"}

~~This is an interesting one. Where does the double \\ come from, in "/Volumes/192.168.0.210/render\\Project1\\Projectblend.flamenco.blend"?~~ never mind, that's just a regular single backslash, but escaped with another backslash by the logger.

~~On Windows, always use drive letters to point to the shared storage. Flamenco does not support UNC notation.~~

Please attach your flamenco-manager.yaml so that I have some more info about how it's set up. That'll help me to dig in further. It's good to know that this PR fixes this issue, but I'd like it even more if it could be fixed in a more general way (without relying on each command knowing exactly what is a filename and what isn't).

Thanks for coming back with more info :) > `2024-04-17T08:36:19+02:00 WRN error executing task error="command exited abnormally with code 1" task={"commands":[{"name":"blender-render","parameters":{"args":["--render-output","/Volumes/192.168.0.210/render/Project1/Render/Project1/2024-04-17_084036/######","--render-format","PNG","--render-frame","3,4"],"argsBefore":[],"blendfile":"/Volumes/192.168.0.210/render\\Project1\\Projectblend.flamenco.blend","exe":"blender","exeArgs":"-b -y"}}],"job":"28c4a99a-8157-40c0-9047-9c06fad80b2f","job_priority":50,"job_type":"simple-blender-render","name":"render-3,4","priority":50,"status":"active","task_type":"blender","uuid":"84c530af-9bc9-436c-a84e-d1bde891ca02"}` ~~This is an interesting one. Where does the double `\\` come from, in `"/Volumes/192.168.0.210/render\\Project1\\Projectblend.flamenco.blend"`?~~ never mind, that's just a regular single backslash, but escaped with another backslash by the logger. ~~On Windows, always use drive letters to point to the shared storage. Flamenco does not support [UNC notation](https://en.wikipedia.org/wiki/Path_(computing)#Universal_Naming_Convention).~~ Please attach your `flamenco-manager.yaml` so that I have some more info about how it's set up. That'll help me to dig in further. It's good to know that this PR fixes this issue, but I'd like it even more if it could be fixed in a more general way (without relying on each command knowing exactly what is a filename and what isn't).

Erik-Hals commented

2024-04-18 13:39:30 +02:00

First-time contributor

This is the storage setup:

shared_storage_path: F:\render
...
my_storage:
    is_twoway: true
    values:
    - platform: linux
      value: /media/192.168.0.210/render
    - platform: windows
      value: F:\render
    - platform: darwin
      value: /Volumes/192.168.0.210/render

The manager is on the Windows machine, which uses drive letters, I only have a mac worker, no linux.

This is the storage setup: ```yaml shared_storage_path: F:\render ... my_storage: is_twoway: true values: - platform: linux value: /media/192.168.0.210/render - platform: windows value: F:\render - platform: darwin value: /Volumes/192.168.0.210/render ``` The manager is on the Windows machine, which uses drive letters, I only have a mac worker, no linux.

flamenco-manager.yaml

1.3 KiB

Sybren A. Stüvel commented

2024-06-25 11:33:05 +02:00

I've tried a slightly different approach to fix the same issue, in 7ffbd72dce. This should avoid the need to call crosspath.ToNative() on every use of a path, as the two-way variables system should already take care of this.

Could you give the latest Flamenco main branch a test, to see if that also solves your problems?

I've tried a slightly different approach to fix the same issue, in 7ffbd72dce1284ce81ff11949aa7ef50ad7f1d32. This should avoid the need to call `crosspath.ToNative()` on every use of a path, as the two-way variables system should already take care of this. Could you give the latest Flamenco `main` branch a test, to see if that also solves your problems?

Sybren A. Stüvel referenced this pull request

2024-08-01 13:08:59 +02:00

Path separator normalisation replaces too much #104331

Sybren A. Stüvel commented

2024-08-01 13:09:56 +02:00

Nope, my approach wasn't the right one. I think your approach is better, as then the command impementation itself can decide which parameters to adjust and which to keep as-is.

Sybren A. Stüvel commented

2024-08-02 11:49:21 +02:00

As it turns out, my approach was fine apart from the little bug I introduced (#104331). Now that that's fixed, please re-test with the latest revision on the main branch to see if your issue is properly addressed now.

As it turns out, my approach was fine apart from the little bug I introduced (#104331). Now that that's fixed, please re-test with the latest revision on the `main` branch to see if your issue is properly addressed now.

Sybren A. Stüvel commented

2024-11-04 10:38:01 +01:00

Closing due to the inactivity of the PR author.

Sybren A. Stüvel closed this pull request

2024-11-04 10:38:02 +01:00