Exporting GLTF with big texture is super slow #68822

New Issue

Jimmy Gunawan · 2019-08-19T12:54:41+02:00

Jimmy Gunawan commented

2019-08-19 12:54:41 +02:00

System Information
Operating system: MacOS Catalina beta
Graphics card: Radeon Pro 555 2 GB

Blender Version
Broken: 2.81 alpha
Worked: (optional)

Short description of error
Exporting GLTF with big texture is super slow

Exact steps for others to reproduce the error
Based on the default startup or an attached .blend file (as simple as possible).

I tried this many times with Blender 2.80 and also 2.81... exporting GLTF + Texture with large texture like 8000 x 8000 is superslow, not sure why... Maybe time it takes to copy? But it's weird.

**System Information** Operating system: MacOS Catalina beta Graphics card: Radeon Pro 555 2 GB **Blender Version** Broken: 2.81 alpha Worked: (optional) **Short description of error** Exporting GLTF with big texture is super slow **Exact steps for others to reproduce the error** Based on the default startup or an attached .blend file (as simple as possible). I tried this many times with Blender 2.80 and also 2.81... exporting GLTF + Texture with large texture like 8000 x 8000 is superslow, not sure why... Maybe time it takes to copy? But it's weird.

Jimmy Gunawan commented

2019-08-19 12:54:41 +02:00

Added subscriber: @BlenderSushiGuy

Mike Erwin commented

2019-08-20 00:59:05 +02:00

Added subscribers: @UX3D-becher, @MikeErwin

Mike Erwin commented

2019-08-20 00:59:05 +02:00

Is the texture packed into the blend file, or in a separate png / jpeg file on disk?

If packed, Blender has to encode the image to a file. More expensive than a simple file copy.

Maybe @UX3D-becher knows more?

Is the texture packed into the blend file, or in a separate png / jpeg file on disk? If packed, Blender has to encode the image to a file. More expensive than a simple file copy. Maybe @UX3D-becher knows more?

Jimmy Gunawan commented

2019-08-20 05:42:30 +02:00

I tested both with Texture separate from GLTF and with GLTF with texture included as one... but both are still performing really, slow. With smaller textures, the export is super fast.

Tommi Hyppänen commented

2019-08-20 12:14:55 +02:00

Added subscriber: @TommiHyppanen

Tommi Hyppänen commented

2019-08-20 12:14:55 +02:00

How long is superslow exactly? I tried a simple Suzanne head with a generated 8k texture (color grid). It took me 39.5 seconds. The extern_draco.dll was found and used. All of this information can be found on the Blender console output.

The export process also eats all of my computer memory (16GB) very quickly, which might also be why it's so slow. It doesn't seem to take a good advantage of multi-core. Mostly a single core process.

Exporting 4k takes 4x less time, 9.8 seconds, which suggests that texture size is the major component in how fast the export process works. 2k is 4x less time than 4k, 2.5 seconds. Which makes sense.

If you were to try exporting 16k textures with it, it would probably take at least 160 seconds if you had memory and completely kill low memory systems, I'm guessing.

All of these tests were exporting .glb binary.

How long is superslow exactly? I tried a simple Suzanne head with a generated 8k texture (color grid). It took me 39.5 seconds. The extern_draco.dll was found and used. All of this information can be found on the Blender console output. The export process also eats all of my computer memory (16GB) very quickly, which might also be why it's so slow. It doesn't seem to take a good advantage of multi-core. Mostly a single core process. Exporting 4k takes 4x less time, 9.8 seconds, which suggests that texture size is the major component in how fast the export process works. 2k is 4x less time than 4k, 2.5 seconds. Which makes sense. If you were to try exporting 16k textures with it, it would probably take at least 160 seconds if you had memory and completely kill low memory systems, I'm guessing. All of these tests were exporting .glb binary.

Tommi Hyppänen commented

2019-08-20 12:32:54 +02:00

I did some profiling:

         70744 function calls (70311 primitive calls) in 9.679 seconds

   Ordered by: internal time
   List reduced from 344 to 20 due to restriction <20>

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    9.586    9.586    9.586    9.586 {built-in method numpy.array}
        1    0.046    0.046    0.064    0.064 gltf2_blender_extract.py:386(extract_primitives)
        6    0.007    0.001    0.007    0.001 {built-in method builtins.print}
        1    0.005    0.005    0.005    0.005 {method 'close' of '_io.BufferedWriter' objects}

Numpy.array seems to be the culprit. At a close look it's not the array transformation itself, but the really painfully slow way that Blender transfers image data to arrays.

gltf2_blender_gather_image.py, line 146: pixels = np.array(image.pixels)
gltf2_blender_image.py, line 46: img = np.array(blender_image.pixels)

Which has been a known issue for a while. https://devtalk.blender.org/t/bpy-data-images-perf-issues/6459

...which means that here's a simple diff to make it at least twice at fast:

gltf2_blender_gather_image.py, line 146: pixels = np.array(image.pixels[:])
gltf2_blender_image.py, line 46: img = np.array(blender_image.pixels[:])

I did some profiling: ``` 70744 function calls (70311 primitive calls) in 9.679 seconds Ordered by: internal time List reduced from 344 to 20 due to restriction <20> ncalls tottime percall cumtime percall filename:lineno(function) 1 9.586 9.586 9.586 9.586 {built-in method numpy.array} 1 0.046 0.046 0.064 0.064 gltf2_blender_extract.py:386(extract_primitives) 6 0.007 0.001 0.007 0.001 {built-in method builtins.print} 1 0.005 0.005 0.005 0.005 {method 'close' of '_io.BufferedWriter' objects} ``` Numpy.array seems to be the culprit. At a close look it's not the array transformation itself, but the really painfully slow way that Blender transfers image data to arrays. ``` gltf2_blender_gather_image.py, line 146: pixels = np.array(image.pixels) gltf2_blender_image.py, line 46: img = np.array(blender_image.pixels) ``` Which has been a known issue for a while. https://devtalk.blender.org/t/bpy-data-images-perf-issues/6459 ...which means that here's a simple diff to make it at least twice at fast: ``` gltf2_blender_gather_image.py, line 146: pixels = np.array(image.pixels[:]) gltf2_blender_image.py, line 46: img = np.array(blender_image.pixels[:]) ```

Julien Duroure was assigned by Brecht Van Lommel

2019-08-22 12:06:36 +02:00

Julien Duroure commented

2019-08-23 14:06:47 +02:00

Note : I created an issue on the upstream repository : https://github.com/KhronosGroup/glTF-Blender-IO/issues/626