BAT failes to find resources when parsing a blend file from a different platform #59124

New Issue

Jeroen Bakker · 2018-12-10T15:24:36+01:00

Jeroen Bakker commented

2018-12-10 15:24:36 +01:00

I parsed the classroom.blend from the blender-benchmark. This file is created on windows and uses relative paths. These relative paths on linux are completed using pathlib. in linux a '' is a valid char to use in a filename.

For example:

INFO:blender_asset_tracer.trace.result:Resolving //textures\radiator_AO.png rel to /home/jeroen/blender-benchmark-1.0b2-linux-glibc219-x86_64/scenes/classroom/assets/radiator/radiator.blend → /home/jeroen/blender-benchmark-1.0b2-linux-glibc219-x86_64/scenes/classroom/assets/radiator/textures\radiator_AO.png
WARNING:blender_asset_tracer.pack:Missing file: /home/jeroen/blender-benchmark-1.0b2-linux-glibc219-x86_64/scenes/classroom/assets/radiator/textures\radiator_AO.png

Blender can't find the file as pathlib tries to find a file called textures\radiator_AO.png in folder /home/jeroen/blender-benchmark-1.0b2-linux-glibc219-x86_64/scenes/classroom/assets/radiator/.

According to blender BLI_Path we should convert the OS platform we are running on as slash and backslash are both invalid to be used in file and foldernames.

My proposal is to adapt the BlendPath.to_path method to convert the Windows slashes automatically to posix slashes.

I parsed the classroom.blend from the blender-benchmark. This file is created on windows and uses relative paths. These relative paths on linux are completed using pathlib. in linux a '\' is a valid char to use in a filename. For example: > INFO:blender_asset_tracer.trace.result:Resolving //textures\radiator_AO.png rel to /home/jeroen/blender-benchmark-1.0b2-linux-glibc219-x86_64/scenes/classroom/assets/radiator/radiator.blend → /home/jeroen/blender-benchmark-1.0b2-linux-glibc219-x86_64/scenes/classroom/assets/radiator/textures\radiator_AO.png > WARNING:blender_asset_tracer.pack:Missing file: /home/jeroen/blender-benchmark-1.0b2-linux-glibc219-x86_64/scenes/classroom/assets/radiator/textures\radiator_AO.png Blender can't find the file as pathlib tries to find a file called `textures\radiator_AO.png` in folder `/home/jeroen/blender-benchmark-1.0b2-linux-glibc219-x86_64/scenes/classroom/assets/radiator/`. According to blender BLI_Path we should convert the OS platform we are running on as slash and backslash are both invalid to be used in file and foldernames. My proposal is to adapt the `BlendPath.to_path` method to convert the Windows slashes automatically to posix slashes.

Jeroen Bakker commented

2018-12-10 15:24:36 +01:00

Added subscriber: @Jeroen-Bakker

Jeroen Bakker commented

2018-12-10 15:25:17 +01:00

Added subscriber: @dr.sybren

Jasper van Nieuwenhuizen commented

2018-12-10 16:14:28 +01:00

Added subscriber: @jasperge-2

Sybren A. Stüvel commented

2018-12-11 17:52:55 +01:00

My proposal is to adapt the BlendPath.to_path method to convert the Windows slashes automatically to posix slashes.

Sounds good to me!

> My proposal is to adapt the `BlendPath.to_path` method to convert the Windows slashes automatically to posix slashes. Sounds good to me!

Sybren A. Stüvel commented

2018-12-20 15:25:43 +01:00

I've made a new branch windows-support to track Windows-specific fixes. They should only be merged to the master branch when they have been tested on (and proven compatible with) Linux.

@Jeroen-Bakker commented in IRC:

13:58 atmind> dr_sybren: biggest issue is that normal BlendPath vs Posix vs Windows path. Perhaps we shouldn't make use of PosixPath etc at all in BAT. only BlendPath (as a wrapper around PosixPath) assuming that the backend is most likely to accept posix paths

The separation between BlendPath and pathlib.Path is a very conscious one:

BlendPath represents paths as stored in a blendfile. It consists of bytes, possibly with unknown character encoding and path separators. Only a limited set of path manipulation is possible, and it cannot be used to perform any actual filesystem operations.
pathlib.Path represents an interpreted path that has a clear definition and manipulation operations, and can be used to perform filesystem operations.

I really want to keep this separation.

For better Windows compatibility, it's probably necessary to replace our use of pathlib.Path with pathlib.PurePosixPath so that we always have forward slashes. There are some strange cases, though, as on Windows a path that's supposed to point to a file can all of a sudden get a trailing slash:

  Python 3.5.4 (v3.5.4:3f56838, Aug  8 2017, 02:17:05) [MSC v.1900 64 bit (AMD64)] on win32
  Type "help", "copyright", "credits" or "license" for more information.
  >>> from pathlib import *
  >>> Path('//some/file.blend')
  WindowsPath('//some/file.blend/')
  >>> PurePosixPath('//some/file.blend')
  PurePosixPath('//some/file.blend')

This doesn't happen with PurePosixPath, which is why I want to use it as much as possible in our code. We can do this until real filesystem operations are necessary, such as calling Path.absolute().

I've made a new branch [windows-support](https://developer.blender.org/source/blender-asset-tracer/history/windows-support/) to track Windows-specific fixes. They should only be merged to the master branch when they have been tested on (and proven compatible with) Linux. @Jeroen-Bakker commented in IRC: > 13:58 atmind> dr_sybren: biggest issue is that normal BlendPath vs Posix vs Windows path. Perhaps we shouldn't make use of PosixPath etc at all in BAT. only BlendPath (as a wrapper around PosixPath) assuming that the backend is most likely to accept posix paths The separation between `BlendPath` and `pathlib.Path` is a very conscious one: - `BlendPath` represents paths as stored in a blendfile. It consists of bytes, possibly with unknown character encoding and path separators. Only a limited set of path manipulation is possible, and it cannot be used to perform any actual filesystem operations. - `pathlib.Path` represents an interpreted path that has a clear definition and manipulation operations, and can be used to perform filesystem operations. I really want to keep this separation. For better Windows compatibility, it's probably necessary to replace our use of `pathlib.Path` with `pathlib.PurePosixPath` so that we always have forward slashes. There are some strange cases, though, as on Windows a path that's supposed to point to a file can all of a sudden get a trailing slash: ``` Python 3.5.4 (v3.5.4:3f56838, Aug 8 2017, 02:17:05) [MSC v.1900 64 bit (AMD64)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> from pathlib import * >>> Path('//some/file.blend') WindowsPath('//some/file.blend/') >>> PurePosixPath('//some/file.blend') PurePosixPath('//some/file.blend') ``` This doesn't happen with `PurePosixPath`, which is why I want to use it as much as possible in our code. We can do this until real filesystem operations are necessary, such as calling `Path.absolute()`.

Sybren A. Stüvel commented

2019-01-02 16:38:59 +01:00

Changed status from 'Open' to: 'Resolved'

Sybren A. Stüvel closed this issue

2019-01-02 16:38:59 +01:00

Sybren A. Stüvel self-assigned this 2019-01-02 16:38:59 +01:00

Sybren A. Stüvel commented

2019-01-02 16:38:59 +01:00

In current master (released as 0.8) all unit tests work on Windows as well. The windows-support branch has been deleted.

If there are more Windows-incompatibilities, let me know!

In current master (released as 0.8) all unit tests work on Windows as well. The `windows-support` branch has been deleted. If there are more Windows-incompatibilities, let me know!

Sign in to join this conversation.