Crash or error when exporting big Layout Uvs #110524

Closed
opened 2023-07-27 10:17:07 +02:00 by DeoFaustino · 23 comments

System Information
Operating system: Windows-10-10.0.19044-SP0 64 Bits
Intel(R) Xeon(R) W-2225 CPU @ 4.10GHz 4.10 GHz
64GB RAM
Graphics card: NVIDIA RTX A4000/PCIe/SSE2 NVIDIA Corporation 4.5.0 NVIDIA 536.67

Blender Version
Broken: version: 3.4.1, branch: blender-v3.4-release, commit date: 2022-12-19 17:00, hash: rB55485cb379f7
Worked: (newest version of Blender that worked as expected)

I either get this error:

Python: Traceback (most recent call last):
File "C:\Program Files\Blender Foundation\Blender 3.4\3.4\scripts\addons\io_mesh_uv_layout_init_.py", line 137, in execute
export(filepath, polygon_data, different_colors, self.size[0], self.size[1], self.opacity)
File "C:\Program Files\Blender Foundation\Blender 3.4\3.4\scripts\addons\io_mesh_uv_layout\export_uv_png.py", line 20, in export
pixel_data.dimensions = width * height * 4
AttributeError: dimension must be greater than or equal to 1

or blender just crashes.

What I am doing.
I work with dpi of 1270 (Laser machining) so the textures are big.
on this case, 40 000 x 20 000px.

even if I set the maximum of the export to the max preset (32 768 x 16384) it will crash.

Please provide feedback

**System Information** Operating system: Windows-10-10.0.19044-SP0 64 Bits Intel(R) Xeon(R) W-2225 CPU @ 4.10GHz 4.10 GHz 64GB RAM Graphics card: NVIDIA RTX A4000/PCIe/SSE2 NVIDIA Corporation 4.5.0 NVIDIA 536.67 **Blender Version** Broken: version: 3.4.1, branch: blender-v3.4-release, commit date: 2022-12-19 17:00, hash: `rB55485cb379f7` Worked: (newest version of Blender that worked as expected) I either get this error: Python: Traceback (most recent call last): File "C:\Program Files\Blender Foundation\Blender 3.4\3.4\scripts\addons\io_mesh_uv_layout\__init__.py", line 137, in execute export(filepath, polygon_data, different_colors, self.size[0], self.size[1], self.opacity) File "C:\Program Files\Blender Foundation\Blender 3.4\3.4\scripts\addons\io_mesh_uv_layout\export_uv_png.py", line 20, in export pixel_data.dimensions = width * height * 4 AttributeError: dimension must be greater than or equal to 1 or blender just crashes. What I am doing. I work with dpi of 1270 (Laser machining) so the textures are big. on this case, 40 000 x 20 000px. even if I set the maximum of the export to the max preset (32 768 x 16384) it will crash. Please provide feedback
DeoFaustino added the
Type
Report
Status
Needs Triage
Priority
Normal
labels 2023-07-27 10:17:08 +02:00
Member

40000 x 20000 x 4 and 32768 x 16384 x 4 both exceed 32 bits, depending on the implementation, the image/file writer may or may not support that size.

Could you share the crash log of calling export UV on 32768 x 16384 size? Thanks!

`40000 x 20000 x 4` and `32768 x 16384 x 4` both exceed 32 bits, depending on the implementation, the image/file writer may or may not support that size. Could you share the [crash log](https://docs.blender.org/manual/en/latest/troubleshooting/crash.html#windows) of calling export UV on 32768 x 16384 size? Thanks!
Author

40000 x 20000 x 4 and 32768 x 16384 x 4 both exceed 32 bits, depending on the implementation, the image/file writer may or may not support that size.

Could you share the crash log of calling export UV on 32768 x 16384 size? Thanks!

Sorry, what do you mean with x4 on your maths? just so I understand

> `40000 x 20000 x 4` and `32768 x 16384 x 4` both exceed 32 bits, depending on the implementation, the image/file writer may or may not support that size. > > Could you share the [crash log](https://docs.blender.org/manual/en/latest/troubleshooting/crash.html#windows) of calling export UV on 32768 x 16384 size? Thanks! Sorry, what do you mean with x4 on your maths? just so I understand
Member

hi, the x4 is for RGBA 4 channels. It's also evidenent in the error log C:\Program Files\Blender Foundation\Blender 3.4\3.4\scripts\addons\io_mesh_uv_layout\export_uv_png.py", line 20, in export pixel_data.dimensions = width * height * 4 AttributeError:

hi, the `x4` is for `RGBA` 4 channels. It's also evidenent in the error log `C:\Program Files\Blender Foundation\Blender 3.4\3.4\scripts\addons\io_mesh_uv_layout\export_uv_png.py", line 20, in export pixel_data.dimensions = width * height * 4 AttributeError: `
Author

how did I not see that. So if instead RGBA we use greyscale or something, would it be lighter and easier?
(I really just use greyscale info)

how did I not see that. So if instead RGBA we use greyscale or something, would it be lighter and easier? (I really just use greyscale info)
Member

@DeoFaustino Technically it should work with grayscale, but depending on the implementation of the algorithm, it may be hard-coded to 4 channels, and this might need fixing.

@DeoFaustino Technically it should work with grayscale, but depending on the implementation of the algorithm, it may be hard-coded to 4 channels, and this might need fixing.
YimingWu added
Module
Sculpt, Paint & Texture
Status
Confirmed
and removed
Status
Needs Information from User
labels 2023-07-27 17:28:08 +02:00
Member

in gpu_py_offscreen.cc, GPU pixel format is only available in RGBA format. Which means in export_uv_png.py the gpu.types.GPUOffScreen(width, height) can only have those trpes of textures for rendering.

@Sergey @mano-wii Maybe we could add GPU_R8, GPU_RG8, GPU_RGB8 for convenience? then the export uv script could run efficiently since it's just one channel useful (It's drawing black lines on top of transparent clear color).

Maybe I could fiddle with this and come up with a patch...

Also... None of the built-in script seems to use custom format.

static const PyC_StringEnumItems pygpu_framebuffer_color_texture_formats[] = {
    {GPU_RGBA8, "RGBA8"},
    {GPU_RGBA16, "RGBA16"},
    {GPU_RGBA16F, "RGBA16F"},
    {GPU_RGBA32F, "RGBA32F"},
    {0, nullptr},
};
in `gpu_py_offscreen.cc`, GPU pixel format is only available in `RGBA` format. Which means in `export_uv_png.py` the `gpu.types.GPUOffScreen(width, height)` can only have those trpes of textures for rendering. @Sergey @mano-wii Maybe we could add `GPU_R8`, `GPU_RG8`, `GPU_RGB8` for convenience? then the export uv script could run efficiently since it's just one channel useful (It's drawing black lines on top of transparent clear color). Maybe I could fiddle with this and come up with a patch... Also... None of the built-in script seems to use custom format. ``` static const PyC_StringEnumItems pygpu_framebuffer_color_texture_formats[] = { {GPU_RGBA8, "RGBA8"}, {GPU_RGBA16, "RGBA16"}, {GPU_RGBA16F, "RGBA16F"}, {GPU_RGBA32F, "RGBA32F"}, {0, nullptr}, }; ```
Member

OK I made a patch for export_uv_png.py and gpu_py_offscreen.cc (but they are two different repo). Will make PRs for them. Test export seems to be working fine now just using grayscale.

OK I made a patch for `export_uv_png.py` and `gpu_py_offscreen.cc` (but they are two different repo). Will make PRs for them. Test export seems to be working fine now just using grayscale.
Member

#110630 and blender/blender-addons#104800 will fix this issue.

https://projects.blender.org/blender/blender/pulls/110630 and https://projects.blender.org/blender/blender-addons/pulls/104800 will fix this issue.
Member

@DeoFaustino Is is absolutely necessary to export PNGs that big? You could apparently export to SVG and render it into whatever size raster image you want from inkscape.

@DeoFaustino Is is absolutely necessary to export PNGs that big? You could apparently export to SVG and render it into whatever size raster image you want from inkscape.
Author

@DeoFaustino Is is absolutely necessary to export PNGs that big? You could apparently export to SVG and render it into whatever size raster image you want from inkscape.

Hello and sorry the delay.

for me it would not really need to be 1270 (I an rescale some how) but even with the half sometimes its to big (in some cases if I go 8000x6000 it will crash same way or same error.

if I go bellow that then the rescale is not worthy because the lines are too wide and I wont be able to use them

> @DeoFaustino Is is absolutely necessary to export PNGs that big? You could apparently export to SVG and render it into whatever size raster image you want from inkscape. Hello and sorry the delay. for me it would not really need to be 1270 (I an rescale some how) but even with the half sometimes its to big (in some cases if I go 8000x6000 it will crash same way or same error. if I go bellow that then the rescale is not worthy because the lines are too wide and I wont be able to use them
Blender Bot added
Status
Archived
and removed
Status
Confirmed
labels 2023-07-31 09:43:14 +02:00
Author

ups, I think I closed it by accident!!!

back to the topic, should I try the addon?

ups, I think I closed it by accident!!! back to the topic, should I try the addon?
Author

ups, I think I closed it by accident!!!

back to the topic, should I try the addon?

ups, I think I closed it by accident!!! back to the topic, should I try the addon?
Author

ups, I think I closed it by accident!!!

back to the topic, should I try the addon?

ups, I think I closed it by accident!!! back to the topic, should I try the addon?
Author

ups, I think I closed it by accident!!!

back to the topic, should I try the addon?

ups, I think I closed it by accident!!! back to the topic, should I try the addon?
Blender Bot added
Status
Needs Triage
and removed
Status
Archived
labels 2023-07-31 09:44:13 +02:00
Author

ups, I think I closed it by accident!!!

back to the topic, should I try the addon?

Actually, once there is somethign for me to try please give me some instrutions on to apply the add-on and then i will give it a try

> ups, I think I closed it by accident!!! > > back to the topic, should I try the addon? Actually, once there is somethign for me to try please give me some instrutions on to apply the add-on and then i will give it a try

@ChengduLittleA it might be fine (to add extra formats), but to me it is hard to tell if it is something easy to add or if it will end up in a deeper change.

it is also not very clear to me how it will solve the problem of reading pixels of large textures. To me the first problem is int shape_len; in the BPyGPUBuffer. I'd expect it to be size_t.

I'd also imagine that there will be some other limitations like texture size limit on GPU.

@ChengduLittleA it might be fine (to add extra formats), but to me it is hard to tell if it is something easy to add or if it will end up in a deeper change. it is also not very clear to me how it will solve the problem of reading pixels of large textures. To me the first problem is `int shape_len;` in the `BPyGPUBuffer`. I'd expect it to be `size_t`. I'd also imagine that there will be some other limitations like texture size limit on GPU.
Member

Yeah if it's size_t then it can solve the immediate problem of 32 bit integer overflowing, but yes it will also need to comply with OpenGL texture size limit. If GPU does support, then we don't need to do anything extra for this particular issue to work. I can't test such big texture since on my machine GL_MAX_TEXTURE_SIZE is only 16384, A 2D texture of that size with 4 channels will just reach 31 bit... Unless we implement tiled uv export, we probably won't be able to support exporting to that big of a size.

I'll check and see if I can give a patch to fix int shape_len into size_t. Thanks @Sergey

@DeoFaustino You will probably need to export SVG and render it into raster in some way eventually. Since it's really rare to have such a huge texture.

Yeah if it's `size_t` then it can solve the immediate problem of 32 bit integer overflowing, but yes it will also need to comply with OpenGL texture size limit. If GPU does support, then we don't need to do anything extra for this particular issue to work. I can't test such big texture since on my machine `GL_MAX_TEXTURE_SIZE` is only 16384, A 2D texture of that size with 4 channels will just reach 31 bit... Unless we implement tiled uv export, we probably won't be able to support exporting to that big of a size. I'll check and see if I can give a patch to fix `int shape_len` into `size_t`. Thanks @Sergey @DeoFaustino You will probably need to export SVG and render it into raster in some way eventually. Since it's really rare to have such a huge texture.
Author

Ok, but even for SVG its telling the max size is 32768.

If you want I can try to share a model and the workfload so you can try to replicate

Ok, but even for SVG its telling the max size is 32768. If you want I can try to share a model and the workfload so you can try to replicate
Member

@DeoFaustino for SVG it really doesn't matter the size in pixel, it's vector so you should be able to scale it to whatever size you want.

@DeoFaustino for SVG it really doesn't matter the size in pixel, it's vector so you should be able to scale it to whatever size you want.
Author

testing that now, Photoshop does not allow to rasterize the SVG above 640mm or 32000 pixeis for a 1270 px/inch resolution...

testing that now, Photoshop does not allow to rasterize the SVG above 640mm or 32000 pixeis for a 1270 px/inch resolution...
Member

@DeoFaustino Apparently inkscape can write to such big image without issue. Just tested 65536*92687

Will try fix the size_t issue then.

@DeoFaustino Apparently inkscape can write to such big image without issue. Just tested `65536*92687` Will try fix the `size_t` issue then.
Julien Kaspar added
Type
Bug
and removed
Type
Report
labels 2023-07-31 16:07:39 +02:00

I took a look at this bug as well:

  • Yes, Blender's GPU Py interop code has some issues. The minimal change necessary to get past the initial error is below [1]
  • Additionally, OpenImageIO's Py interop also has a bug that will affect this. I submitted and checked in a patch there [2]. So we will need to compile a new version of the library if we want to address the PNG issue entirely.

[1] Quick Blender change -- We were using the wrong python API to read into one of our arrays. There might be some other spots that need fixing.

diff --git a/source/blender/python/gpu/gpu_py_buffer.cc b/source/blender/python/gpu/gpu_py_buffer.cc
index 57c52b3ff4b..ed51c6e2827 100644
--- a/source/blender/python/gpu/gpu_py_buffer.cc
+++ b/source/blender/python/gpu/gpu_py_buffer.cc
@@ -62,11 +62,11 @@ static bool pygpu_buffer_pyobj_as_shape(PyObject *shape_obj,
                                         Py_ssize_t *r_shape_len)
 {
   Py_ssize_t shape_len = 0;
   if (PyLong_Check(shape_obj)) {
     shape_len = 1;
-    if ((r_shape[0] = PyLong_AsLong(shape_obj)) < 1) {
+    if ((r_shape[0] = PyLong_AsSsize_t(shape_obj)) < 1) {
       PyErr_SetString(PyExc_AttributeError, "dimension must be greater than or equal to 1");
       return false;
     }
   }
   else if (PySequence_Check(shape_obj)) {

[2] https://github.com/OpenImageIO/oiio/commit/c97826c144c13781c7dd390bc16f4acfa11a0da8

I took a look at this bug as well: - Yes, Blender's GPU Py interop code has some issues. The minimal change necessary to get past the initial error is below [1] - Additionally, OpenImageIO's Py interop also has a bug that will affect this. I submitted and checked in a patch there [2]. So we will need to compile a new version of the library if we want to address the PNG issue entirely. [1] Quick Blender change -- We were using the wrong python API to read into one of our arrays. There might be some other spots that need fixing. ```diff diff --git a/source/blender/python/gpu/gpu_py_buffer.cc b/source/blender/python/gpu/gpu_py_buffer.cc index 57c52b3ff4b..ed51c6e2827 100644 --- a/source/blender/python/gpu/gpu_py_buffer.cc +++ b/source/blender/python/gpu/gpu_py_buffer.cc @@ -62,11 +62,11 @@ static bool pygpu_buffer_pyobj_as_shape(PyObject *shape_obj, Py_ssize_t *r_shape_len) { Py_ssize_t shape_len = 0; if (PyLong_Check(shape_obj)) { shape_len = 1; - if ((r_shape[0] = PyLong_AsLong(shape_obj)) < 1) { + if ((r_shape[0] = PyLong_AsSsize_t(shape_obj)) < 1) { PyErr_SetString(PyExc_AttributeError, "dimension must be greater than or equal to 1"); return false; } } else if (PySequence_Check(shape_obj)) { ``` [2] [https://github.com/OpenImageIO/oiio/commit/c97826c144c13781c7dd390bc16f4acfa11a0da8](https://github.com/OpenImageIO/oiio/commit/c97826c144c13781c7dd390bc16f4acfa11a0da8)
Iliya Katushenock added
Status
Confirmed
and removed
Status
Needs Triage
labels 2023-08-04 03:35:02 +02:00

The module is setted, the bug is defined as a fact. Confirming, as sorting doesn't seem to be required.

The module is setted, the bug is defined as a fact. Confirming, as sorting doesn't seem to be required.
Blender Bot added
Status
Resolved
and removed
Status
Confirmed
labels 2023-10-12 07:44:26 +02:00
Sign in to join this conversation.
No Label
Interest
Alembic
Interest
Animation & Rigging
Interest
Asset Browser
Interest
Asset Browser Project Overview
Interest
Audio
Interest
Automated Testing
Interest
Blender Asset Bundle
Interest
BlendFile
Interest
Collada
Interest
Compatibility
Interest
Compositing
Interest
Core
Interest
Cycles
Interest
Dependency Graph
Interest
Development Management
Interest
EEVEE
Interest
EEVEE & Viewport
Interest
Freestyle
Interest
Geometry Nodes
Interest
Grease Pencil
Interest
ID Management
Interest
Images & Movies
Interest
Import Export
Interest
Line Art
Interest
Masking
Interest
Metal
Interest
Modeling
Interest
Modifiers
Interest
Motion Tracking
Interest
Nodes & Physics
Interest
OpenGL
Interest
Overlay
Interest
Overrides
Interest
Performance
Interest
Physics
Interest
Pipeline, Assets & IO
Interest
Platforms, Builds & Tests
Interest
Python API
Interest
Render & Cycles
Interest
Render Pipeline
Interest
Sculpt, Paint & Texture
Interest
Text Editor
Interest
Translations
Interest
Triaging
Interest
Undo
Interest
USD
Interest
User Interface
Interest
UV Editing
Interest
VFX & Video
Interest
Video Sequencer
Interest
Virtual Reality
Interest
Vulkan
Interest
Wayland
Interest
Workbench
Interest: X11
Legacy
Blender 2.8 Project
Legacy
Milestone 1: Basic, Local Asset Browser
Legacy
OpenGL Error
Meta
Good First Issue
Meta
Papercut
Meta
Retrospective
Meta
Security
Module
Animation & Rigging
Module
Core
Module
Development Management
Module
EEVEE & Viewport
Module
Grease Pencil
Module
Modeling
Module
Nodes & Physics
Module
Pipeline, Assets & IO
Module
Platforms, Builds & Tests
Module
Python API
Module
Render & Cycles
Module
Sculpt, Paint & Texture
Module
Triaging
Module
User Interface
Module
VFX & Video
Platform
FreeBSD
Platform
Linux
Platform
macOS
Platform
Windows
Priority
High
Priority
Low
Priority
Normal
Priority
Unbreak Now!
Status
Archived
Status
Confirmed
Status
Duplicate
Status
Needs Info from Developers
Status
Needs Information from User
Status
Needs Triage
Status
Resolved
Type
Bug
Type
Design
Type
Known Issue
Type
Patch
Type
Report
Type
To Do
No Milestone
No project
No Assignees
5 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: blender/blender#110524
No description provided.