Blend file corrupted during save caused by high Cubemap Size #78529

New Issue

Marcin Twarowski · 2020-07-02T12:35:02+02:00

Marcin Twarowski commented

2020-07-02 12:35:02 +02:00

System Information
Operating system: Windows-10-10.0.19041-SP0 64 Bits
Graphics card: GeForce GTX 1080/PCIe/SSE2 NVIDIA Corporation 4.5.0 NVIDIA 451.48

Blender Version
Broken: version: 2.83.1, branch: master, commit date: 2020-06-25 09:47, hash: 8289fc688b
Also broken in 2.90 52b125a790

Short description of error
If indirect lighting is baked in Eeeve with Cubemap Size 2048 or 4096, the file than gets corrupted during save and cannot be opened again. Despite the file weighting 100mb, when opened all you see is default cube. I used to see error message about DNA missing but now it does not appear. There are no error messages during save neither. When indirect lighting is baked with lower Cubemap Sizes it can be reopened normally. It happens to me on two different projects on two different machines.

Exact steps for others to reproduce the error

Open attached file.
DNA_bug.blend
Bake Indirect Lighting with Cubemap Size set to 2048
Save the file after baking.
Try to open the saved file.
Here's the corrupted blend file:
DNA_bug_baked_2048.blend

**System Information** Operating system: Windows-10-10.0.19041-SP0 64 Bits Graphics card: GeForce GTX 1080/PCIe/SSE2 NVIDIA Corporation 4.5.0 NVIDIA 451.48 **Blender Version** Broken: version: 2.83.1, branch: master, commit date: 2020-06-25 09:47, hash: `8289fc688b` Also broken in 2.90 52b125a790e3 **Short description of error** If indirect lighting is baked in Eeeve with Cubemap Size 2048 or 4096, the file than gets corrupted during save and cannot be opened again. Despite the file weighting 100mb, when opened all you see is default cube. I used to see error message about DNA missing but now it does not appear. There are no error messages during save neither. When indirect lighting is baked with lower Cubemap Sizes it can be reopened normally. It happens to me on two different projects on two different machines. **Exact steps for others to reproduce the error** - Open attached file. [DNA_bug.blend](https://archive.blender.org/developer/F8659453/DNA_bug.blend) - Bake Indirect Lighting with Cubemap Size set to 2048 - Save the file after baking. - Try to open the saved file. Here's the corrupted blend file: [DNA_bug_baked_2048.blend](https://archive.blender.org/developer/F8659459/DNA_bug_baked_2048.blend)

Marcin Twarowski commented

2020-07-02 12:35:02 +02:00

Added subscriber: @MarcinTwarowski

Germano Cavalcante commented

2020-07-02 22:41:51 +02:00

Added subscriber: @mano-wii

Germano Cavalcante commented

2020-07-02 22:41:51 +02:00

Changed status from 'Needs Triage' to: 'Confirmed'

Germano Cavalcante commented

2020-07-02 22:41:51 +02:00

I can confirm.
This message appears when trying to load the files with the saved cubemap textures:

Failed to read blend file 'C:\Users\GERMANO\Downloads\DNA_bug_baked_2048.blend': Missing DNA block

I can confirm. This message appears when trying to load the files with the saved cubemap textures: ``` Failed to read blend file 'C:\Users\GERMANO\Downloads\DNA_bug_baked_2048.blend': Missing DNA block ```

Clément Foucault commented

2020-07-03 02:25:50 +02:00

Added subscriber: @fclem

Clément Foucault self-assigned this 2020-07-03 02:25:50 +02:00

Clément Foucault commented

2020-07-03 02:25:50 +02:00

The total data size is reported to be 3.5GiB so there is something really strange here. I need to dig further.

Clément Foucault commented

2020-08-03 23:17:46 +02:00

Added subscriber: @mont29

Clément Foucault commented

2020-08-03 23:17:47 +02:00

Ok, it is caused by an integer overflow in writefile.c:

line 2507: write_lightcache_texture(writer, &cache->cube_tx);
line 2500: BLO_write_raw(writer, data_size, tex->data);

Here data_size is equal to 2 818 572 288 which is casted to int for BLO_write_raw making it negative.

@mont29 So it seems this system was not supposed to support really large datablocks. What would be the best workaround for this? I can split the texture into multiple datablocks if needed.

Ok, it is caused by an integer overflow in `writefile.c`: ```lang=c line 2507: write_lightcache_texture(writer, &cache->cube_tx); line 2500: BLO_write_raw(writer, data_size, tex->data); ``` Here `data_size` is equal to 2 818 572 288 which is casted to `int` for `BLO_write_raw` making it negative. @mont29 So it seems this system was not supposed to support really large datablocks. What would be the best workaround for this? I can split the texture into multiple datablocks if needed.

Clément Foucault commented

2020-08-04 16:54:01 +02:00

On the other hand it seems these size of texture are not supported by many OpenGL implementation. Here a Cubemap Texture of 1024x1024x258 (43 cubemaps) is not supported even on high end AMD Polaris (but 42 are supported which still exceed the Writer limit).

I should add a safeguard if the texture cannot be allocated.

On the other hand it seems these size of texture are not supported by many OpenGL implementation. Here a Cubemap Texture of 1024x1024x258 (43 cubemaps) is not supported even on high end AMD Polaris (but 42 are supported which still exceed the Writer limit). I should add a safeguard if the texture cannot be allocated.

Clément Foucault commented

2020-08-04 23:06:28 +02:00

After trying to get it right I just have to postpone the real fix for 2.91.

For 2.90 will just have a warning that the cache won't be saved if it exceed the size limit 2.15GB.

This is the start of the fix. https://developer.blender.org/P1564 The issue with this fix is that it doubles the needed memory to load the cache and it is cleary a hack. A better fix would be doable once we have the GPU module refactored (part of the Vulkan port).

After trying to get it right I just have to postpone the real fix for 2.91. For 2.90 will just have a warning that the cache won't be saved if it exceed the size limit 2.15GB. This is the start of the fix. https://developer.blender.org/P1564 The issue with this fix is that it doubles the needed memory to load the cache and it is cleary a hack. A better fix would be doable once we have the GPU module refactored (part of the Vulkan port).

blender-admin commented

2020-08-05 02:25:31 +02:00

This issue was referenced by 220470be15

This issue was referenced by 220470be1515d90ae2c3157648bd9da311925427

blender-admin commented

2020-08-05 02:25:31 +02:00

This issue was referenced by 6390b530d0

This issue was referenced by 6390b530d064330edbcc50be0cdabc839f292e7c

Clément Foucault commented

2020-08-05 02:36:17 +02:00

Changed status from 'Confirmed' to: 'Resolved'

Clément Foucault closed this issue

2020-08-05 02:36:17 +02:00

Sign in to join this conversation.

No Label

Download

What's New

Blender Studio

Manual

Developers Blog

Documentation

Benchmark

Blender Conference

Development Fund

One-time Donations

Blend file corrupted during save caused by high Cubemap Size #78529