GPU render crash #58311

New Issue

Bohdan Lvov · 2018-11-30T15:06:01+01:00

Bohdan Lvov commented

2018-11-30 15:06:01 +01:00

System Information
Operating system: Windows 10 1803
Graphics card: GTX970
DxDiag.rar

Blender Version
Broken:
blender-2.80-4c31bed6b46-win64

Worked (ish):
2.79.6 - at least it render just stops and I get CUDA error message.

Short description of error

Blender crashes when I render this scene when GPU Compute enabled with CPU and GPU enabled in CUDA Cycles Compute Devices : Assblaster_BaseSceneBend4BM28-2.rar . No crash with CPU Compute.
It maybe related to VRAM limitation, since shrinking texture size helps. Still, there's about 400 mb of free VRAM when Blender crashes without any error messages.

Exact steps for others to reproduce the error

Open attached scene and press F12 - on my config it crashes when about 20% rendered.

**System Information** Operating system: Windows 10 1803 Graphics card: GTX970 [DxDiag.rar](https://archive.blender.org/developer/F5768539/DxDiag.rar) **Blender Version** Broken: blender-2.80-4c31bed6b46-win64 Worked (ish): 2.79.6 - at least it render just stops and I get CUDA error message. **Short description of error** Blender crashes when I render this scene when GPU Compute enabled with CPU and GPU enabled in CUDA Cycles Compute Devices : [Assblaster_BaseSceneBend4BM28-2.rar](https://archive.blender.org/developer/F5768345/Assblaster_BaseSceneBend4BM28-2.rar) . No crash with CPU Compute. It maybe related to VRAM limitation, since shrinking texture size helps. Still, there's about 400 mb of free VRAM when Blender crashes without any error messages. **Exact steps for others to reproduce the error** Open attached scene and press F12 - on my config it crashes when about 20% rendered.

Bohdan Lvov commented

2018-11-30 15:06:01 +01:00

Added subscriber: @ostapblender

Sebastian Parborg commented

2019-01-23 11:36:53 +01:00

Added subscriber: @ZedDB

Sebastian Parborg commented

2019-01-23 11:36:53 +01:00

Is this still an issue with the latest 2.8 beta build?

Bohdan Lvov commented

2019-01-23 11:41:55 +01:00

This comment was removed by @ostapblender

*This comment was removed by @ostapblender*

Bohdan Lvov commented

2019-01-23 11:43:05 +01:00

Yes. Another thing that I discovered is that it won't crash if Denoising is disabled.
Tested on dc3b5024be

Yes. Another thing that I discovered is that it won't crash if Denoising is disabled. Tested on dc3b5024be1a

Brecht Van Lommel commented

2019-01-23 11:47:53 +01:00

Added subscriber: @brecht

Brecht Van Lommel commented

2019-01-23 11:47:53 +01:00

Does 2.80 still actually crash, or only show the CUDA error message?

It's quite possible this is just running out of memory. 400MB being reported as free is not really reliable, there is temporary working memory needed that may not be counted.

Does 2.80 still actually crash, or only show the CUDA error message? It's quite possible this is just running out of memory. 400MB being reported as free is not really reliable, there is temporary working memory needed that may not be counted.

Bohdan Lvov commented

2019-01-23 11:52:04 +01:00

Yes, it's still crashing: Blender window is just disappears with no messages, no nothing. Nvidia Driver 417.71 / Windows 1809 17763.253

Bohdan Lvov commented

2019-01-23 12:16:06 +01:00

UPD: looks like now it crashes only when Blender's window isn't in focus while rendering. If I won't switch to another window, it won't crash.

UPD: *looks like* now it crashes only when Blender's window isn't in focus while rendering. If I won't switch to another window, it won't crash.

Sebastian Parborg commented

2019-01-31 11:53:47 +01:00

Does it crash if you try to render from command line without gui?

There are some example how to do that on this page (using the -b flag);
https://docs.blender.org/manual/en/latest/render/workflows/command_line.html

Does it crash if you try to render from command line without gui? There are some example how to do that on this page (using the `-b` flag); https://docs.blender.org/manual/en/latest/render/workflows/command_line.html

Bohdan Lvov commented

2019-01-31 12:48:03 +01:00

Tried on 8c89790674 build config files were deleted beforehand, just to make sure its not an addon or something. 2 out of 5 times with window out focus it crashed. Same statistics for the render from cmd: 2 out of 5 crashes.
Logs of two crashes: https:*pastebin.com/2YhiZ4vc After that I made a few more runs from command, just to be sure, and got another error, now with the direct CUDA mention https:*pastebin.com/qbvsgYtY Another one after that, even without closing cmd window was successful, attached.
CyclesUncrashed.rar
I hope it's somehow helpful.

Tried on 8c8979067490 build config files were deleted beforehand, just to make sure its not an addon or something. 2 out of 5 times with window out focus it crashed. Same statistics for the render from cmd: 2 out of 5 crashes. Logs of two crashes: https:*pastebin.com/2YhiZ4vc After that I made a few more runs from command, just to be sure, and got another error, now with the direct CUDA mention https:*pastebin.com/qbvsgYtY Another one after that, even without closing cmd window was successful, attached. [CyclesUncrashed.rar](https://archive.blender.org/developer/F6463969/CyclesUncrashed.rar) I hope it's somehow helpful.

Sebastian Parborg commented

2019-01-31 13:11:10 +01:00

Can you run this CUDA memtest program? https://simtk.org/projects/memtest (MemtestG80)

You can also try this and see if you manage to trigger any problems: http://geeks3d.com/furmark/kombustor/

Can you run this CUDA memtest program? https://simtk.org/projects/memtest (MemtestG80) You can also try this and see if you manage to trigger any problems: http://geeks3d.com/furmark/kombustor/

Bohdan Lvov commented

2019-01-31 15:13:34 +01:00

Yup, here's the result: https://pastebin.com/qwiXnJ2a
Although your link requires registration with manual approval by a staff, but from a view of forum it doesn't look too much alive. So I downloaded MemtestG80 compiled by someone, so it maybe a bit out of date, mind you. If confirmation mail from SimTK eventually arrive, I'll redo the test. Also it wasn't able to run it on more that 3072 mb, it kept showing and error: Error: unable to allocate 4094 MiB of GPU memory to test, bailing!

About the Kombustor - it didn't showed any error messages or artifacts and GPU didn't explode.

Yup, here's the result: https://pastebin.com/qwiXnJ2a Although your link requires registration with manual approval by a staff, but from a view of forum it doesn't look too much alive. So I downloaded MemtestG80 compiled by someone, so it maybe a bit out of date, mind you. If confirmation mail from SimTK eventually arrive, I'll redo the test. Also it wasn't able to run it on more that 3072 mb, it kept showing and error: Error: unable to allocate 4094 MiB of GPU memory to test, bailing! About the Kombustor - it didn't showed any error messages or artifacts and GPU didn't explode.

Sebastian Parborg commented

2019-01-31 15:29:06 +01:00

The result does say that there are memory errors.
You can try http://www.programming4beginners.com/gpumemtest too and see if that reports any issue too.

The result does say that there are memory errors. You can try http://www.programming4beginners.com/gpumemtest too and see if that reports any issue too.

Bohdan Lvov commented

2019-01-31 16:12:23 +01:00

It says All tests OK https://pastebin.com/dFyCZyEp

But still, it saying Allocated 3312 MiB total, in 4 chunks which is not 4096 MiB of total memory of that GPU.
Of course last chunk of 512 mb is weird with GTX 970, maybe that's the reason?

I've encountered something similar with Substance Painter recently, with Sparse Virtual Textures in particular. GTX 970 and Quado M2000 is the only GPUs that can work with it due to memory issues on Windows, as I can understand: https://support.allegorithmic.com/documentation/spdoc/sparse-virtual-textures-172823866.html

I tried to recreate this scene as close as possible from scratch (maybe issue was in the file itself since it's migrated trough a lot of version), just copy-pasting objects and values and there's actually a difference: Blender doesn't crash as a whole now, only render process stops.

CUDA error: Illegal address in cuCtxSynchronize(), line 1527{F6465676}

It says All tests OK https://pastebin.com/dFyCZyEp But still, it saying Allocated 3312 MiB total, in 4 chunks which is not 4096 MiB of total memory of that GPU. Of course last chunk of 512 mb is weird with GTX 970, maybe that's the reason? I've encountered something similar with Substance Painter recently, with Sparse Virtual Textures in particular. GTX 970 and Quado M2000 is the only GPUs that can work with it due to memory issues on Windows, as I can understand: https://support.allegorithmic.com/documentation/spdoc/sparse-virtual-textures-172823866.html I tried to recreate this scene as close as possible from scratch (maybe issue was in the file itself since it's migrated trough a lot of version), just copy-pasting objects and values and there's actually a difference: Blender doesn't crash as a whole now, only render process stops. ![image.png](https://archive.blender.org/developer/F6465631/image.png) CUDA error: Illegal address in cuCtxSynchronize(), line 1527{[F6465676](https://archive.blender.org/developer/F6465676/ABRecr.rar)}

Sebastian Parborg commented

2019-01-31 17:28:07 +01:00

Because we can't reproduce this on our end, I'm guessing that it might be indeed the weird memory config of your card.

As you pointed out, the guys at substance painter also ran into problems with your specific card. So I guess that the cuda driver for it is buggy.

Because we can't reproduce this on our end, I'm guessing that it might be indeed the weird memory config of your card. As you pointed out, the guys at substance painter also ran into problems with your specific card. So I guess that the cuda driver for it is buggy.

Bohdan Lvov commented

2019-01-31 17:29:51 +01:00

Changed status from 'Open' to: 'Archived'

Bohdan Lvov closed this issue

2019-01-31 17:29:51 +01:00

Bohdan Lvov self-assigned this 2019-01-31 17:29:51 +01:00

Bohdan Lvov commented

2019-01-31 17:29:51 +01:00

Well then, let's close it if there's nothing we can do at the moment.

Sign in to join this conversation.

No Label

Download

What's New

Blender Studio

Manual

Developers Blog

Documentation

Benchmark

Blender Conference

Development Fund

One-time Donations

GPU render crash #58311