Various Addons using filesystem properties (PROP_FILEPATH etc): UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe0 in position 24: invalid continuation byte #74510
Labels
No Label
Interest
Alembic
Interest
Animation & Rigging
Interest
Asset Browser
Interest
Asset Browser Project Overview
Interest
Audio
Interest
Automated Testing
Interest
Blender Asset Bundle
Interest
BlendFile
Interest
Collada
Interest
Compatibility
Interest
Compositing
Interest
Core
Interest
Cycles
Interest
Dependency Graph
Interest
Development Management
Interest
EEVEE
Interest
EEVEE & Viewport
Interest
Freestyle
Interest
Geometry Nodes
Interest
Grease Pencil
Interest
ID Management
Interest
Images & Movies
Interest
Import Export
Interest
Line Art
Interest
Masking
Interest
Metal
Interest
Modeling
Interest
Modifiers
Interest
Motion Tracking
Interest
Nodes & Physics
Interest
OpenGL
Interest
Overlay
Interest
Overrides
Interest
Performance
Interest
Physics
Interest
Pipeline, Assets & IO
Interest
Platforms, Builds & Tests
Interest
Python API
Interest
Render & Cycles
Interest
Render Pipeline
Interest
Sculpt, Paint & Texture
Interest
Text Editor
Interest
Translations
Interest
Triaging
Interest
Undo
Interest
USD
Interest
User Interface
Interest
UV Editing
Interest
VFX & Video
Interest
Video Sequencer
Interest
Virtual Reality
Interest
Vulkan
Interest
Wayland
Interest
Workbench
Interest: X11
Legacy
Blender 2.8 Project
Legacy
Milestone 1: Basic, Local Asset Browser
Legacy
OpenGL Error
Meta
Good First Issue
Meta
Papercut
Meta
Retrospective
Meta
Security
Module
Animation & Rigging
Module
Core
Module
Development Management
Module
EEVEE & Viewport
Module
Grease Pencil
Module
Modeling
Module
Nodes & Physics
Module
Pipeline, Assets & IO
Module
Platforms, Builds & Tests
Module
Python API
Module
Render & Cycles
Module
Sculpt, Paint & Texture
Module
Triaging
Module
User Interface
Module
VFX & Video
Platform
FreeBSD
Platform
Linux
Platform
macOS
Platform
Windows
Priority
High
Priority
Low
Priority
Normal
Priority
Unbreak Now!
Status
Archived
Status
Confirmed
Status
Duplicate
Status
Needs Info from Developers
Status
Needs Information from User
Status
Needs Triage
Status
Resolved
Type
Bug
Type
Design
Type
Known Issue
Type
Patch
Type
Report
Type
To Do
No Milestone
No project
No Assignees
12 Participants
Notifications
Due Date
No due date set.
Dependencies
No dependencies set.
Reference: blender/blender#74510
Loading…
Reference in New Issue
No description provided.
Delete Branch "%!s(<nil>)"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
System Information
Operating system: Window 10
Graphics card: NVIDIA GeForce GT 1030
Blender Version
Broken: 2.80
Short description of error
I wanted to export model to .obj, but i can't its giving me
Any idea how i can fix?
Exact steps for others to reproduce the error
Make a model from plane with 2 picture, 5 material. Sword4(finish1).blend
Added subscriber: @SoulDragoz
#80955 was marked as duplicate of this issue
blender/blender-addons#81300 was marked as duplicate of this issue
#77898 was marked as duplicate of this issue
blender/blender-addons#68515 was marked as duplicate of this issue
I'm new to blender, for now i wanted to export model for my friend to use on program
Added subscriber: @ankitm
Changed status from 'Needs Triage' to: 'Archived'
https://stackoverflow.com/questions/1347791/unicode-error-unicodeescape-codec-cant-decode-bytes-cannot-open-text-file
Can't export model (.obj)to Can't export model (.obj): UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe0 in position 24: invalid continuation byteAdded subscriber: @brecht
Changed status from 'Archived' to: 'Needs Triage'
I don't see how that stackoverflow answer explains that this is not a bug.
Users should not get unicode errors from typical Blender usage.
Changed status from 'Needs Triage' to: 'Confirmed'
Changed status from 'Confirmed' to: 'Needs Triage'
Had seen a similar one recently, in chat. where the
scene.render.filepath = "C:\Users\me\a"
had to be changed toscene.render.filepath = r"C:\Users\me\a"
in a python script.Rushed over it here. Sorry.
Added subscriber: @rjg
@ankitm That's a different cause though, you have to escape backslashes in strings. The
filepath
from abpy.types.BlendData
object should be correctly formatted and encoded by Blender.The error looks like the string is encoded with something other than UTF-8, like Windows-1252, and Blender tries to interpret the bytes as UTF-8.
This comment was removed by @rjg
Added subscriber: @ideasman42
This is a problem we ran into early on when moving to Python3.
File path properties of type:
PROP_FILEPATH, PROP_DIRPATH, PROP_FILENAME
usePyC_UnicodeFromByteAndSize
which usesPyUnicode_DecodeFSDefaultAndSize
as a fallback which is intended to handle this situation without raising an exception.Please run this in the Python console and include the output here:
I'm sorry if i did something wrong i'm new to this form too.
@ideasman42 Its say
sys.getfilesystemencoding()
~ 'utf-8'
sys.getfilesystemencodeerrors()
~ 'surrogatepass'
Added subscriber: @LazyDodo
I can't repro this on windows, what is the full path to the blend file on your system?
This comment was removed by @SoulDragoz
Its was C:\Users\peem6\OneDrive\Blender Model
Right? its where my .blend file stay
also the last comment I deleted it's just I locate wrong folder
Changed status from 'Needs Triage' to: 'Needs User Info'
With the blend file that causes the error, could you show the output of this command in the Python console:
It's say
(this is second model that also got same problem that i tried to export)
What about now?
it's been a while i wanted to finish and export my model.
There is seemingly a sub-folder with a really strange name in your
one drive
folder, if you want to export this model, i'd just make a directory calledc:\temp
copy your blend file there and export it. It still be nice to fix this issue, but if you need to export your model right now, that's how to do it.Changed status from 'Needs User Info' to: 'Confirmed'
The issue is caused by an invalid utf-8 encoding which isn't supported by
surrogatepass
, which is the default error handler on Windows.This raises the question - why can't we use
surrogateescape
as Linux/MacOS do, I didn't see anything written down about why however I assume this is caused by NTFS using UTF-16.Possible solutions:
surrogateescape
.bytes
for paths. Listing for completeness, not really practical to apply this to all file handling code.sys._enablelegacywindowsfsencoding()
. Also not a good long term solution, noting this for completeness.Further, checking Python 3.9 commits, there looks to be changes in this area, so it may require a Python version upgrade to resolve.
Marking this as a TODO.
Added subscribers: @axel72, @EitanSomething, @AdamPreisler, @mont29, @Wektop
Added subscriber: @sk13dust
Can't export model (.obj): UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe0 in position 24: invalid continuation byteto Various Addons using filesystem properties (PROP_FILEPATH etc): UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe0 in position 24: invalid continuation byteAdded subscribers: @solitary111, @lichtwerk, @EAW
Added subscribers: @wisen, @mano-wii
Removed subscriber: @AdamPreisler
While more graceful error handling would be good, how did we end up with an invalid utf-8 file path in the first place? Is there perhaps an error in how we convert Windows utf-16 to utf-8? Maybe a bug in our conversion function? Or something related to codepages that we don't take into account?
There was a report on #blender-coders about this error on a render farm, which may be related:
In this case we are retrieving a user configuration path, not reading it from a .blend file. That error seems like it should be reproducible, running Blender from the command line in a folder with certain special characters and perhaps with a particular system language configuration.
I'm still unable to repro this specific error, however putting blender in a folder called
christmas time 🎅
(that's U+1F385 for the santa, it's a valid char) prevents blender from starting completely due to a long standing issue inBLI_strncpy_wchar_from_utf8
where assumptions are made on the size of whcar_t, may not be the same issue but i would not be surprised if it were real close.Added subscriber: @Raimund58
I took a closer look at
BLI_strncpy_wchar_from_utf8
and cannot phantom why we duplicated the code fromBLI_str_utf8_as_utf32
here, given wchar_t is essentially UTF16 on windows. this function could just beI feel like i'm missing something here, it surely cannot be this easy?! (this fix does allow blender to run from the
christmas time 🎅
folder)what am i missing?!
I think it may well be that easy. Likely just legacy code that never got replaced, or code that was developed and tested only on Linux.
This seems to be indeed a windows only problem. At least after what we tested
Added subscriber: @Harley
I haven't found any reason to think that @LazyDodo's solution wouldn't work. Should work and be harmless. I think we should just commit that change and he and I can deal with any effects if they occur. Its not like any of us are going anywhere for Christmas. LOL
I did set the working directory to be
mäh_Документы_cpu1
This cycles error seems to be Windows only and Blender 2.91 only.
I would hope for a corrective 2.91.1 release.
Ok, if it involves environment variables then that is a different issue than the one being fixed in D9822.
BLI_getenv
needs to be modified to get a utf-16 string and convert that to utf-8.I have a fix for
BLI_getenv
will post a small diff later today, that being said, neither of these two fixes would explain any of the breakage between 2.90 and 2.91 so that slightly concerns meAre we sure it's actually 2.91 only, was the exact same thing tested in 2.90?
I'm going off what @Raimund58 is seeing, don't get me wrong the issues we found are valid and need fixing, they just don't explain terribly well why he is only seeing these issues on 2.91 given it's pretty old code we're mucking about in.
Well, the cycles error is after our logs only Blender 2.91, yes.
I found no logs where Cycles failed in a lower version.
And as far as I know we didn't change the script or the render command.
So yes, I think it is Blender 2.91 only. (And after I got a alpha build from @rjg to test it is also in Blender 2.92)
But we weren't able to reproduce it local. But I also didn't set the environments variables because I don't know how. :/
This issue was referenced by
87b19b3aba
@brecht With the latest build from @rjg and the latest commit from @LazyDodo Cycles is again working for us!
Thank you all so much for your time <3
had some time to bisect this, the troubles start with
6f3a9031f7
but have not looked in depth what could cause the behavior we're seeing with 2.91also given this ticket got filed about 6 months previous to that commit landing, it certainly cannot account for the issue in the opening post. So we have successfully muddied up this ticket with unrelated issues, in the future it is probably best to file a new ticket even if the symptoms seem similar .
There may still be a shared underlying cause though.
BLI_strncpy_wchar_as_utf8
is used in the filebrowser, so it's possible that leads to an incorrect .blend file path as in the original description. D9822 would fix such a problem.Removed subscriber: @axel72
This issue was referenced by
5496d8cd36
Changed status from 'Confirmed' to: 'Resolved'