Blender Crash with subsurf mod on default cube with huge resolution #44869

New Issue

Daniel Dreschers · 2015-05-28T10:41:28+02:00

Daniel Dreschers commented

2015-05-28 10:41:28 +02:00

System Information
Debian 7 (64 Bit), 512 GB RAM (GB, not MB), NVidia Quadro K5200
system-info.txt

Blender Version
Broken: 2.74 release (Linux 64 Bit)

Short description of error
Blender Crash with Segmentation Fault when using subsurf mod and huge render resolutions

Exact steps for others to reproduce the error

Load Factory Settings
Enter Redering Resolution x: 50000 px ; y: 25000 px ; scale: 100%
add Subdivision Surface Modifier on default cube; set render subdivisions to 3
Hit Render Button
Crash
Crash log: DanD_blender_crash_with_subsurf_mod_on_default_cube_with_huge_resolution.crash.txt

DanD_blender_crash_with_subsurf_mod_on_default_cube_with_huge_resolution.blend
When using the attached .blend file, just hit Render Button

If you set even higher resolutions, like 50000 x 40000, rendering starts, but doesn't produce a picture. Instead you get a lot of errors like
"Calloc returns null: len=18446744073119617024 in imbuf display buffer, total 1356434112"
in the terminal.

**System Information** Debian 7 (64 Bit), 512 GB RAM (GB, not MB), NVidia Quadro K5200 [system-info.txt](https://archive.blender.org/developer/F180699/system-info.txt) **Blender Version** Broken: 2.74 release (Linux 64 Bit) **Short description of error** Blender Crash with Segmentation Fault when using subsurf mod and huge render resolutions **Exact steps for others to reproduce the error** - Load Factory Settings - Enter Redering Resolution x: 50000 px ; y: 25000 px ; scale: 100% - add Subdivision Surface Modifier on default cube; set render subdivisions to 3 - Hit Render Button - > Crash Crash log: [DanD_blender_crash_with_subsurf_mod_on_default_cube_with_huge_resolution.crash.txt](https://archive.blender.org/developer/F180703/DanD_blender_crash_with_subsurf_mod_on_default_cube_with_huge_resolution.crash.txt) [DanD_blender_crash_with_subsurf_mod_on_default_cube_with_huge_resolution.blend](https://archive.blender.org/developer/F180702/DanD_blender_crash_with_subsurf_mod_on_default_cube_with_huge_resolution.blend) When using the attached .blend file, just hit Render Button If you set even higher resolutions, like 50000 x 40000, rendering starts, but doesn't produce a picture. Instead you get a lot of errors like "Calloc returns null: len=18446744073119617024 in imbuf display buffer, total 1356434112" in the terminal.

Daniel Dreschers commented

2015-05-28 10:41:28 +02:00

Changed status to: 'Open'

Daniel Dreschers commented

2015-05-28 10:41:28 +02:00

Added subscriber: @DanD

Antonis Ryakiotakis commented

2015-05-28 11:31:21 +02:00

Added subscriber: @Psy-Fi

Antonis Ryakiotakis commented

2015-05-28 11:31:21 +02:00

Can you try a recent version of blender from buildbot? There have been fixes to support rendering of bigger resolutions.

Daniel Dreschers commented

2015-05-28 11:57:56 +02:00

I downloaded Blender 2.74.5 Hash 3e0c6a8

Unfortunately this version also crashes.
No matter if using the .blend file I attached to the original report, or doing the steps manually.
The only difference is, that the blender window doesn't close this time. Have to use the kill command to get rid of it.

DanD_blender_crash_with_subsurf_mod_on_default_cube_with_huge_resolution.Blender_2.74_3e0c6a8.crash.txt

I downloaded Blender 2.74.5 Hash 3e0c6a8 Unfortunately this version also crashes. No matter if using the .blend file I attached to the original report, or doing the steps manually. The only difference is, that the blender window doesn't close this time. Have to use the kill command to get rid of it. [DanD_blender_crash_with_subsurf_mod_on_default_cube_with_huge_resolution.Blender_2.74_3e0c6a8.crash.txt](https://archive.blender.org/developer/F180729/DanD_blender_crash_with_subsurf_mod_on_default_cube_with_huge_resolution.Blender_2.74_3e0c6a8.crash.txt)

Campbell Barton commented

2015-05-28 11:58:47 +02:00

Added subscriber: @ideasman42

Campbell Barton commented

2015-05-28 11:58:47 +02:00

This seems like a regular out-of-memory situation.

While Blender doesn't normally check for this case, it might make sense to do so allocating large buffers (render_layer_add_pass) then fail rendering with an error.

This seems like a regular out-of-memory situation. While Blender doesn't normally check for this case, it might make sense to do so allocating large buffers (`render_layer_add_pass`) then fail rendering with an error.

Daniel Dreschers commented

2015-05-28 12:11:49 +02:00

Out of memory?
On a machine with 512 Gigabyte of RAM?

Can I somewhere see, how much memory Blender is trying to allocate?

Out of memory? On a machine with 512 Gigabyte of RAM? Can I somewhere see, how much memory Blender is trying to allocate?

Antonis Ryakiotakis commented

2015-05-28 12:38:15 +02:00

Doesn't sound like an out of memory, most likely it's an integer overflow due to using int instead of size_t.

Antonis Ryakiotakis commented

2015-05-28 14:12:52 +02:00

So, I'm investigating this here. Indeed there's a deadlock somewhere but increasing the tile size to 256 makes rendering possible here. It's probably caused by an overflow with the tile size somewhere, but at least you should be able to render if you increase the tile size.

Antonis Ryakiotakis commented

2015-05-28 16:05:33 +02:00

Changed status from 'Open' to: 'Archived'

Antonis Ryakiotakis closed this issue

2015-05-28 16:05:33 +02:00

Antonis Ryakiotakis self-assigned this 2015-05-28 16:05:33 +02:00

Antonis Ryakiotakis commented

2015-05-28 16:05:34 +02:00

Issue is that covering the whole image with 64x64 tiles needs a total of 305762 tiles.

Before rendering we sort those tiles so that the ones closer to the center begin rendering first.

The problem is that sorting those is an 2 * n^2 operation (sort every one of them and for every one of them calculate center of available tiles and find nearest to that center tile). It follows that you are going to wait for 186.980.801.288 * cost of operation per tile time before starting rendering. If we suppose that the cost is one CPU cycle, you still need about one minute for the whole thing to finish (this is not a multithreaded operation). However, this costs quite more than one CPU cycle, and tiles are stored in a linked list, adding multiple cache misses to the cost.

For a tile size of 256, we need roughly 16 times less tiles to cover the grid and 256 times less time before starting rendering. Which is roughly the difference between 2 hours and a minute.

In any case, tested this in our super render farm with 64GB of RAM and after about half an hour or so the file begun rendering.

So, verdict is: this is not a bug, you just need to set tile size higher. I can try to optimize that part a little bit, but best complexity I can do is n * (n + 1) / 2 so don't expect miracles.

Issue is that covering the whole image with 64x64 tiles needs a total of 305762 tiles. Before rendering we sort those tiles so that the ones closer to the center begin rendering first. The problem is that sorting those is an 2 * n^2 operation (sort every one of them and for every one of them calculate center of available tiles and find nearest to that center tile). It follows that you are going to wait for 186.980.801.288 * cost of operation per tile time before starting rendering. If we suppose that the cost is one CPU cycle, you still need about one minute for the whole thing to finish (this is not a multithreaded operation). However, this costs quite more than one CPU cycle, and tiles are stored in a linked list, adding multiple cache misses to the cost. For a tile size of 256, we need roughly 16 times less tiles to cover the grid and 256 times less time before starting rendering. Which is roughly the difference between 2 hours and a minute. In any case, tested this in our super render farm with 64GB of RAM and after about half an hour or so the file begun rendering. So, verdict is: this is not a bug, you just need to set tile size higher. I can try to optimize that part a little bit, but best complexity I can do is n * (n + 1) / 2 so don't expect miracles.

Antonis Ryakiotakis commented

2015-05-28 16:15:28 +02:00

In fact if this is plugged to a quicksort, we may even get n log n complexity. But still, it will be quite expensive.

Daniel Dreschers commented

2015-05-28 16:32:12 +02:00

I tried setting the tile size to 256.
It doesn't work.
Blender still crashes here.
Tried 2.74.5 (Hash_3e0c6a8) on my linux machine and 2.74.5 (Hash_dc15860) on my Windows machine.

What you are saying about the complexity of the sorting algorithm would turn out to be a matter waiting.
I wouldn't mind that. I'd get paid for that. ;)
Unfortunately it's not. Blender crashes. I get a segmentation fault and Blender writes out a crash log.

I tried setting the tile size to 256. It doesn't work. Blender still crashes here. Tried 2.74.5 (Hash_3e0c6a8) on my linux machine and 2.74.5 (Hash_dc15860) on my Windows machine. What you are saying about the complexity of the sorting algorithm would turn out to be a matter waiting. I wouldn't mind that. I'd get paid for that. ;) Unfortunately it's not. Blender crashes. I get a segmentation fault and Blender writes out a crash log.

Antonis Ryakiotakis commented

2015-05-28 16:38:38 +02:00

What you described above does not sound like a crash:

"The only difference is, that the blender window doesn't close this time. Have to use the kill command to get rid of it."

This is not a crash, blender is actually computing stuff. If you get a crash (blender closing on its own) then I'm afraid I can't reproduce your issue here.

What you described above does not sound like a crash: "The only difference is, that the blender window doesn't close this time. Have to use the kill command to get rid of it." This is not a crash, blender is actually computing stuff. If you get a crash (blender closing on its own) then I'm afraid I can't reproduce your issue here.

Ton Roosendaal commented

2015-05-28 17:04:42 +02:00

Added subscriber: @Ton

Ton Roosendaal commented

2015-05-28 17:04:42 +02:00

Dan:
It would be quite a simple task to investigate the limits of Blender on your super computer.
Just start small and increase sizes in sensible steps, Observe how things take longer and/or what takes suspicious amount of times more.

Dan: It would be quite a simple task to investigate the limits of Blender on your super computer. Just start small and increase sizes in sensible steps, Observe how things take longer and/or what takes suspicious amount of times more.

Antonis Ryakiotakis commented

2015-05-28 17:07:42 +02:00

Changed status from 'Archived' to: 'Resolved'

Antonis Ryakiotakis commented

2015-05-28 17:07:43 +02:00

Hi, tested your file with 64 by 64 tile size in our render farm after my commit and the time difference is immense - render starts after a few seconds now. I will mark as resolved, at least on our side. You can test a build on tomorrow's buildbot and see if the issue still persists there.

Germano Cavalcante commented

2015-05-29 04:50:14 +02:00

Added subscriber: @mano-wii

Daniel Dreschers commented

2015-05-29 08:54:18 +02:00

Antony, you are correct. It's my lazyness that's now causing confusion.
I omitted to say that when the blender window didn't close, there was a message in the terminal: "Writing: /tmp/blender.crash.txt". That's why I assumed that blender had crashed and I closed it by using kill.

Tested the linux 64 bit build 3511e2d this morning. Crashed again.

But since you can't reproduce this behaviour, I'll keep shut and will try to find another way to reproduce this.

Antony, you are correct. It's my lazyness that's now causing confusion. I omitted to say that when the blender window didn't close, there was a message in the terminal: "Writing: /tmp/blender.crash.txt". That's why I assumed that blender had crashed and I closed it by using kill. Tested the linux 64 bit build 3511e2d this morning. Crashed again. But since you can't reproduce this behaviour, I'll keep shut and will try to find another way to reproduce this.

perfection cat commented

2015-05-29 10:09:04 +02:00

Added subscriber: @sindra1961

perfection cat commented

2015-05-29 10:09:04 +02:00

Can blender make PNG file of the size(50000x40000) with your PC?
You can confirm it in Image Editor of blender.
I think that you should check the size that an error does not produce.

Can blender make PNG file of the size(50000x40000) with your PC? You can confirm it in Image Editor of blender. I think that you should check the size that an error does not produce.

Antonis Ryakiotakis commented

2015-05-29 10:26:35 +02:00

Can you try with background rendering? (n fact this is the only way I've tried rendering here because our farm server does not have any screens and I suspect no GPUs either.

The options are:

blender -b -f -o <imagefile.png>

Can you try removing subsurf and see if it's still impossible to render?
Can you try with even bigger tile sizes and see if the issue is still there?
Lastly, are you certain you are using the correct version of blender when opening those files (ie not running a version in PATH or so)?

Can you try with background rendering? (n fact this is the only way I've tried rendering here because our farm server does not have any screens and I suspect no GPUs either. The options are: blender -b <blendfile> -f <frame> -o <imagefile.png> Can you try removing subsurf and see if it's still impossible to render? Can you try with even bigger tile sizes and see if the issue is still there? Lastly, are you certain you are using the correct version of blender when opening those files (ie not running a version in PATH or so)?

Daniel Dreschers commented

2015-05-29 14:33:38 +02:00

sindra: Yes, it is possible to create a 50000x25000 picture in the UV Editor and save it to a file.

Antony: You're again correct. Using the background rendering, it works. Even the default scene at a resolution of 50000x40000 renders correct.
But when using the GUI, it's not.
I tried with and without subsurf modifier. I tried tile sizes from default up to 4096x4096. Every time I got a segmentation fault when using the GUI.

And yes, I am certain I'm using the correct version. I start from a terminal and check the hash at the splash screen every time I do something related to this report.

Since it now seems to be a problem with the GUI, I'm playing around with some settings.
For the test running at the moment, I set "User preferences"-"System"-"Images Draw Method" to "GLSL".
With this setting I noticed the following:
a) it starts rendering
b) when I use the scroll wheel to zoom out of the rendered picture while it is rendering, Blender crashes
c) when I start rendering and don't touch Blender, it runs. At least for the first 30 minutes I let it. Unfortunately I can't block this machine over the weekend. I need it to do something else. But I also noticed, rendering with the GUI is horribly slow with these settings. The background render was done within a few minutes.

(All test from this posts were done with Blender 2.74.5 Hash_3511e2d on Linux 64 Bit.)

sindra: Yes, it is possible to create a 50000x25000 picture in the UV Editor and save it to a file. Antony: You're again correct. Using the background rendering, it works. Even the default scene at a resolution of 50000x40000 renders correct. But when using the GUI, it's not. I tried with and without subsurf modifier. I tried tile sizes from default up to 4096x4096. Every time I got a segmentation fault when using the GUI. And yes, I am certain I'm using the correct version. I start from a terminal and check the hash at the splash screen every time I do something related to this report. Since it now seems to be a problem with the GUI, I'm playing around with some settings. For the test running at the moment, I set "User preferences"-"System"-"Images Draw Method" to "GLSL". With this setting I noticed the following: a) it starts rendering b) when I use the scroll wheel to zoom out of the rendered picture while it is rendering, Blender crashes c) when I start rendering and don't touch Blender, it runs. At least for the first 30 minutes I let it. Unfortunately I can't block this machine over the weekend. I need it to do something else. But I also noticed, rendering with the GUI is horribly slow with these settings. The background render was done within a few minutes. (All test from this posts were done with Blender 2.74.5 Hash_3511e2d on Linux 64 Bit.)

blender-admin commented

2015-05-29 15:43:21 +02:00

This issue was referenced by 9876d1f299

This issue was referenced by 9876d1f299541752ceb8c685c7fe962e9151e1c2

Antonis Ryakiotakis commented

2015-05-29 15:50:42 +02:00

Looks like there were indeed some integer overflow cases. Can you check again with buildbot tomorrow?

Daniel Dreschers commented

2015-05-30 12:59:39 +02:00

I don't have access to my big computer over the weekend. I'll have to check on monday.
And I'd like to say thank you for taking the time to look into this problem.

I don't have access to my big computer over the weekend. I'll have to check on monday. And I'd like to say thank you for taking the time to look into this problem.

Daniel Dreschers commented

2015-06-01 15:06:08 +02:00

Tested Blender 2.74.5 hash_b147473 today.

Going through the following steps leads to a blender crash.

Load Factory Settings
Set Render Resolution to X: 50000, Y: 25000, Scale: 100%
Hit Render Button
blender.Blender2.74.5_b147473.crash.txt

When I switch to "Images Draw Method: GLSL" in "User Preferences" - "System" after loading factory settings, it starts rendering and zooming out works as well.
The only thing I noticed was a slight offset between the orange surroundings of an actively rendered tile and the actual tile that's being displayed (see Screenshot).

Tested Blender 2.74.5 hash_b147473 today. Going through the following steps leads to a blender crash. - Load Factory Settings - Set Render Resolution to X: 50000, Y: 25000, Scale: 100% - Hit Render Button [blender.Blender2.74.5_b147473.crash.txt](https://archive.blender.org/developer/F182826/blender.Blender2.74.5_b147473.crash.txt) When I switch to "Images Draw Method: GLSL" in "User Preferences" - "System" after loading factory settings, it starts rendering and zooming out works as well. The only thing I noticed was a slight offset between the orange surroundings of an actively rendered tile and the actual tile that's being displayed (see Screenshot). ![Offset Active Tile while Rendering 2015-06-01.png](https://archive.blender.org/developer/F182829/Offset_Active_Tile_while_Rendering_2015-06-01.png)

blender-admin commented

2015-06-02 05:28:24 +02:00

This issue was referenced by abc4a3d455

This issue was referenced by abc4a3d455f8fe91cd32b0cba8dda7f65ee2b77c

Sign in to join this conversation.

No Label

Download

What's New

Blender Studio

Manual

Developers Blog

Documentation

Benchmark

Blender Conference

Development Fund

One-time Donations

Blender Crash with subsurf mod on default cube with huge resolution #44869