Illegal address in CUDA queue copy_from_device #108202

New Issue

Closed

opened 2023-05-23 18:18:17 +02:00 by Walter Myers · 14 comments

Walter Myers commented

2023-05-23 18:18:17 +02:00

System Information
Operating system: Windows-10-10.0.22621-SP0 64 Bits
Graphics card: NVIDIA GeForce RTX 3060 Ti/PCIe/SSE2 NVIDIA Corporation 4.5.0 NVIDIA 531.79

Blender Version
Broken: version: 3.5.1, branch: blender-v3.5-release, commit date: 2023-04-24 18:11, hash: e1ccd9d4a1d3
Worked: (newest version of Blender that worked as expected)

Short description of error
Rendering an animation, but when it reaches frame 66 it quits with this error. Never seen this issue before. Happens with both Blender 3.5.0 and 3.5.1.

Exact steps for others to reproduce the error
[Please describe the exact steps needed to reproduce the issue]
[Based on the default startup or an attached .blend file (as simple as possible)]

**System Information** Operating system: Windows-10-10.0.22621-SP0 64 Bits Graphics card: NVIDIA GeForce RTX 3060 Ti/PCIe/SSE2 NVIDIA Corporation 4.5.0 NVIDIA 531.79 **Blender Version** Broken: version: 3.5.1, branch: blender-v3.5-release, commit date: 2023-04-24 18:11, hash: `e1ccd9d4a1d3` Worked: (newest version of Blender that worked as expected) **Short description of error** Rendering an animation, but when it reaches frame 66 it quits with this error. Never seen this issue before. Happens with both Blender 3.5.0 and 3.5.1. **Exact steps for others to reproduce the error** [Please describe the exact steps needed to reproduce the issue] [Based on the default startup or an attached .blend file (as simple as possible)]

blender_351_console_window_cuda_error.txt

122 KiB

Walter Myers added the

labels 2023-05-23 18:18:18 +02:00

Iliya Katushenock added the

Render & Cycles

label 2023-05-23 21:38:10 +02:00

Walter Myers commented

2023-05-24 16:15:01 +02:00

Author

Workaround: The problem file uses a World data-block consisting of a 16k 32bit *.HDR. If I swap this for an 18k 8bit *.PNG, the issue no longer appears.

Workaround: The problem file uses a World data-block consisting of a 16k 32bit *.HDR. If I swap this for an 18k 8bit *.PNG, the issue no longer appears.

Seth commented

2023-05-24 18:02:15 +02:00

I believe I've been having the same issue!
I couldn't render more than a few frames without consistently getting this error, I think 5 was my record, Optix or Cuda backends. GPU memory use never got close to maxing out.

Removing my 'World' background immediately resolves the issue, 57 frames rendered and still going as I type this.

I'll attach the HDRI file I'm using.

Tried to output a system into text file, but that crashed Blender while it was rendering... :(
Attached. I'm on a recent alpha build of 3.6.

I believe I've been having the same issue! I couldn't render more than a few frames without consistently getting this error, I think 5 was my record, Optix or Cuda backends. GPU memory use never got close to maxing out. Removing my 'World' background immediately resolves the issue, 57 frames rendered and still going as I type this. I'll attach the HDRI file I'm using. Tried to output a system into text file, but that crashed Blender while it was rendering... :( Attached. I'm on a recent alpha build of 3.6.

system-info.txt

23 KiB

space_HDRI_cc0.png

18 MiB

Richard Antalik commented

2023-05-25 06:50:41 +02:00

Member

Please attach .blend file so we can reproduce the issue.

Please attach .blend file so we can reproduce the issue.

Richard Antalik added

Needs Information from User

and removed

labels 2023-05-25 06:50:48 +02:00

Walter Myers commented

2023-05-25 16:10:58 +02:00

Author

Please attach .blend file so we can reproduce the issue.

I'd love to, but this is a public forum and the files (my project links to multiple Blend files) contain proprietary 3rd-party content. Do you have a secure portal? I'd be happy to zip everything up and send it there (about 1 GB). Thanks.

> Please attach .blend file so we can reproduce the issue. I'd love to, but this is a public forum and the files (my project links to multiple Blend files) contain proprietary 3rd-party content. Do you have a secure portal? I'd be happy to zip everything up and send it there (about 1 GB). Thanks.

Richard Antalik commented

2023-05-26 06:32:20 +02:00

Member

We don't have any special means to share proprietary data. Users should try to simplify file as much as possible and substitute any proprietary data so files can be shared. If some proprietary data can't be substituted, you could share file somewhere and provide link via email/chat.

In any case, please make sure your GPU is not overclocked as that often causes issues like this.

We don't have any special means to share proprietary data. Users should try to simplify file as much as possible and substitute any proprietary data so files can be shared. If some proprietary data can't be substituted, you could share file somewhere and provide link via email/chat. In any case, please make sure your GPU is not overclocked as that often causes issues like this.

Walter Myers commented

2023-05-26 15:43:44 +02:00

Author

Nothing is overclocked on my systems. I have two PCs with GeForce GPUs and the issue occurred with both, however the older system would see the error earlier in the animation render.

I initially thought that the World data-block HDR was the issue, however changing HDRs simply changed where in the animation the problem occurred.

I finally got the project rendering again by selectively removing elements from the scene until a stable render was achieved. For what it's worth, the last element I removed was this:

https://www.blenderkit.com/get-blenderkit/b1523e97-ce8b-455c-9545-bfdbefbd074c/

I don't know whether that's the culprit, or just the last element removed that tipped the scales. I get the impression that this is some sort cumulative issue, like a memory leak maybe?

Nothing is overclocked on my systems. I have two PCs with GeForce GPUs and the issue occurred with both, however the older system would see the error earlier in the animation render. I initially thought that the World data-block HDR was the issue, however changing HDRs simply changed where in the animation the problem occurred. I finally got the project rendering again by selectively removing elements from the scene until a stable render was achieved. For what it's worth, the last element I removed was this: [https://www.blenderkit.com/get-blenderkit/b1523e97-ce8b-455c-9545-bfdbefbd074c/](url) I don't know whether that's the culprit, or just the last element removed that tipped the scales. I get the impression that this is some sort cumulative issue, like a memory leak maybe?

Richard Antalik commented

2023-05-28 14:51:49 +02:00

Member

Can you check if having only mentioned model in .blend file causes issues?

Can you check if having only mentioned model in .blend file causes issues?

Iliya Katushenock referenced this issue

2023-05-29 11:44:20 +02:00

Error: illegal address in CUDA copy_from_device (integrator_shade_surface_intergrator_sorted_paths_array prefix_sum) #108389

Yana-Yapici commented

2023-05-29 18:47:54 +02:00

Having the same issue. Happens to me anytime I try to render using Cycles on GPU with OptiX.
I get the error even when starting Blender using --factory-startup and rendering the default cube.

Most of the times I get the illegal address in CUDA copy_from_device (integrator_shade_surface_intergrator_sorted_paths_array prefix_sum) error and sometimes blender just freezes and I have to kill it.

The PC is not overclocked.

System Information
Operating system: Windows-10-10.0.19045-SP0 64 Bits
Graphics card: NVIDIA GeForce RTX 2060/PCIe/SSE2 NVIDIA Corporation 4.5.0 NVIDIA 531.79

Blender Version
Broken: version: 3.5.1, branch: blender-v3.5-release, commit date: 2023-04-24 18:11, hash: e1ccd9d4a1d3

Having the same issue. Happens to me anytime I try to render using Cycles on GPU with OptiX. I get the error even when starting Blender using `--factory-startup` and rendering the default cube. Most of the times I get the `illegal address in CUDA copy_from_device (integrator_shade_surface_intergrator_sorted_paths_array prefix_sum)` error and sometimes blender just freezes and I have to kill it. The PC is not overclocked. **System Information** Operating system: Windows-10-10.0.19045-SP0 64 Bits Graphics card: NVIDIA GeForce RTX 2060/PCIe/SSE2 NVIDIA Corporation 4.5.0 NVIDIA 531.79 **Blender Version** Broken: version: 3.5.1, branch: blender-v3.5-release, commit date: 2023-04-24 18:11, hash: `e1ccd9d4a1d3`

Walter Myers commented

2023-05-29 20:24:52 +02:00

Author

Can you check if having only mentioned model in .blend file causes issues?

The model by itself doesn't cause the error. I may have unfairly singled it out because as I was removing elements from the scene this is the last element I removed before the CUDA error stopped appearing.

The scene is rendering fine now. I'll let you know if the error appears again.

Another thing to note is that when the error appears, e.g., at frame 76 of an animation, rendering that same frame as a single image still results in the error.

Thanks.

> Can you check if having only mentioned model in .blend file causes issues? The model by itself doesn't cause the error. I may have unfairly singled it out because as I was removing elements from the scene this is the last element I removed before the CUDA error stopped appearing. The scene is rendering fine now. I'll let you know if the error appears again. Another thing to note is that when the error appears, e.g., at frame 76 of an animation, rendering that same frame as a single image still results in the error. Thanks.

Pratik Borhade commented

2023-06-06 08:58:45 +02:00

Member

Hi, how much VRAM is consumed during render or during the appearance of error?
Some users were running out of VRAM when this error appeared.

Reducing texture quality, sample count of render might help. Disable persistent data if it is enabled.

Blend file will help Cycles developers to investigate the root problem.

Hi, how much VRAM is consumed during render or during the appearance of error? Some users were running out of VRAM when this error appeared. Reducing texture quality, sample count of render might help. Disable persistent data if it is enabled. Blend file will help Cycles developers to investigate the root problem.

Walter Myers commented

2023-06-06 16:19:17 +02:00

Author

@PratikPB2123 VRAM is never topped out when I've gotten this error. Regarding texture quality, etc., I've had far more complex projects where this issue never came up. Only one project suffered from this error; no idea why.

@PratikPB2123 VRAM is never topped out when I've gotten this error. Regarding texture quality, etc., I've had far more complex projects where this issue never came up. Only one project suffered from this error; no idea why.

Pratik Borhade commented

2023-06-07 08:15:44 +02:00

Member

Thanks.
Also worth to check in recent 3.6-beta build: https://builder.blender.org/download/daily/
Could be related to #105442

I've had far more complex projects where this issue never came up

That's why blend file is required to investigate the issue ;)

Thanks. Also worth to check in recent 3.6-beta build: https://builder.blender.org/download/daily/ Could be related to #105442 - - - > I've had far more complex projects where this issue never came up That's why blend file is required to investigate the issue ;)

Pratik Borhade commented

2023-07-18 07:49:56 +02:00

Member

Hi, any updates?

Hi, any updates?

Richard Antalik commented

2023-09-02 03:59:44 +02:00

Member

No activity for more than a week. As per the tracker policy we assume the issue is gone and can be closed.

Thanks again for the report. If the problem persists please open a new report with the required information.

No activity for more than a week. As per the tracker policy we assume the issue is gone and can be closed. Thanks again for the report. If the problem persists please open a new report with the required information.

Richard Antalik closed this issue

2023-09-02 03:59:46 +02:00

Blender Bot added

and removed

Needs Information from User

labels 2023-09-02 03:59:48 +02:00

Sign in to join this conversation.

No Label

Animation & Rigging

Asset Browser Project Overview

Automated Testing

Blender Asset Bundle

Dependency Graph

Development Management

EEVEE & Viewport

Images & Movies

Motion Tracking

Nodes & Physics

Pipeline, Assets & IO

Platforms, Builds & Tests

Render & Cycles

Render Pipeline

Sculpt, Paint & Texture

Video Sequencer

Virtual Reality

Blender 2.8 Project

Milestone 1: Basic, Local Asset Browser

Good First Issue

Animation & Rigging

Development Management

EEVEE & Viewport

Nodes & Physics

Pipeline, Assets & IO

Platforms, Builds & Tests

Render & Cycles

Sculpt, Paint & Texture

Needs Info from Developers

Needs Information from User

No Milestone

No project

No Assignees

5 Participants

Notifications

Due Date

The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: blender/blender#108202

No description provided.