Cycles: GPU Denoising breaks the UI and doesn't abort #120167

Open
opened 2024-04-02 10:42:28 +02:00 by Raimund Klink · 7 comments
Contributor

System Information
Operating system: Windows-11-10.0.22631-SP0 64 Bits
Graphics card: NVIDIA RTX 2000 Ada Generation Laptop GPU/PCIe/SSE2 NVIDIA Corporation 4.6.0 NVIDIA 551.86

Blender Version
Broken: version: 4.1.0, branch: blender-v4.1-release, commit date: 2024-03-25 20:42, hash: 40a5e739e270
Worked: (newest version of Blender that worked as expected)

Short description of error
The UI breaks sometimes. And the render doesn't abort if it is clearly broken.

System is out of GPU and shared host memory

Refer to the Cycles GPU rendering documentation for possible solutions:
https://docs.blender.org/manual/en/latest/render/cycles/gpu_rendering.html

Invalid value in CUDA queue copy_to_device ()
E0402 12:12:36.646390  9096 denoiser_gpu.cpp:102] Error preprocessing guiding passes.

Exact steps for others to reproduce the error

  • Open attached .blend file or
    • Enable GPU denoising
    • Samples to the minimum (for fast render).
    • Increase the image resolution about 12 times (1200%) (depends on your hardware)
  • render image.
**System Information** Operating system: Windows-11-10.0.22631-SP0 64 Bits Graphics card: NVIDIA RTX 2000 Ada Generation Laptop GPU/PCIe/SSE2 NVIDIA Corporation 4.6.0 NVIDIA 551.86 **Blender Version** Broken: version: 4.1.0, branch: blender-v4.1-release, commit date: 2024-03-25 20:42, hash: `40a5e739e270` Worked: (newest version of Blender that worked as expected) **Short description of error** The UI breaks sometimes. And the render doesn't abort if it is clearly broken. ``` System is out of GPU and shared host memory Refer to the Cycles GPU rendering documentation for possible solutions: https://docs.blender.org/manual/en/latest/render/cycles/gpu_rendering.html Invalid value in CUDA queue copy_to_device () E0402 12:12:36.646390 9096 denoiser_gpu.cpp:102] Error preprocessing guiding passes. ``` **Exact steps for others to reproduce the error** - Open attached .blend file or - Enable GPU denoising - Samples to the minimum (for fast render). - Increase the image resolution about 12 times (1200%) (depends on your hardware) - render image.
Raimund Klink added the
Priority
Normal
Type
Report
Status
Needs Triage
labels 2024-04-02 10:42:29 +02:00
Author
Contributor

There are some errors in the console:

Read blend: "C:\Users\raimund.klink\Downloads\GPU_denoise.blend"
Illegal address in CUDA queue synchronize (film_convert_float3 filter_guiding_preprocess)

Refer to the Cycles GPU rendering documentation for possible solutions:
https://docs.blender.org/manual/en/latest/render/cycles/gpu_rendering.html

E0402 10:55:06.289744 21544 denoiser_gpu.cpp:372] Error converting denoising passes to RGB buffer.
Illegal address in cuMemcpyDtoH_v2( (char *)mem.host_pointer + offset, (CUdeviceptr)mem.device_pointer + offset, size) (C:\Users\blender\git\blender-v410\blender.git\intern\cycles\device\cuda\device_impl.cpp:618)
Illegal address in cuGraphicsUnregisterResource(cu_graphics_resource_) (C:\Users\blender\git\blender-v410\blender.git\intern\cycles\device\cuda\graphics_interop.cpp:24)
Illegal address in cuMemFree_v2((CUdeviceptr)device_pointer) (C:\Users\blender\git\blender-
Illegal address in cuMemFree_v2((CUdeviceptr)device_pointer) (C:\Users\blender\git\blender-v410\blender.git\intern\cycles\device\cuda\device_impl.cpp:542)
Illegal address in cuModuleUnload(cuModule) (C:\Users\blender\git\blender-v410\blender.git\intern\cycles\device\cuda\device_impl.cpp:139)
There are some errors in the console: ``` Read blend: "C:\Users\raimund.klink\Downloads\GPU_denoise.blend" Illegal address in CUDA queue synchronize (film_convert_float3 filter_guiding_preprocess) Refer to the Cycles GPU rendering documentation for possible solutions: https://docs.blender.org/manual/en/latest/render/cycles/gpu_rendering.html E0402 10:55:06.289744 21544 denoiser_gpu.cpp:372] Error converting denoising passes to RGB buffer. Illegal address in cuMemcpyDtoH_v2( (char *)mem.host_pointer + offset, (CUdeviceptr)mem.device_pointer + offset, size) (C:\Users\blender\git\blender-v410\blender.git\intern\cycles\device\cuda\device_impl.cpp:618) Illegal address in cuGraphicsUnregisterResource(cu_graphics_resource_) (C:\Users\blender\git\blender-v410\blender.git\intern\cycles\device\cuda\graphics_interop.cpp:24) Illegal address in cuMemFree_v2((CUdeviceptr)device_pointer) (C:\Users\blender\git\blender- Illegal address in cuMemFree_v2((CUdeviceptr)device_pointer) (C:\Users\blender\git\blender-v410\blender.git\intern\cycles\device\cuda\device_impl.cpp:542) Illegal address in cuModuleUnload(cuModule) (C:\Users\blender\git\blender-v410\blender.git\intern\cycles\device\cuda\device_impl.cpp:139) ```
Author
Contributor

If I then try to render a second time, I get this error:

OPTIX_ERROR_CUDA_ERROR in optixDeviceContextCreate(cuContext, &options, &context) (C:\Users\blender\git\blender-v410\blender.git\intern\cycles\device\optix\device_impl.cpp:90)

Refer to the Cycles GPU rendering documentation for possible solutions:
https://docs.blender.org/manual/en/latest/render/cycles/gpu_rendering.html

OPTIX_ERROR_INVALID_DEVICE_CONTEXT in optixDeviceContextSetLogCallback( context, options.logCallbackFunction, options.logCallbackData, options.logCallbackLevel) (C:\Users\blender\git\blender-v410\blender.git\intern\cycles\device\optix\device_impl.cpp:93)
System is out of GPU memory
Illegal address in cuStreamCreate(&cuda_stream_, CU_STREAM_NON_BLOCKING) (C:\Users\blender\git\blender-v410\blender.git\intern\cycles\device\cuda\queue.cpp:21)

The VRAM from the previous frame wasn't freed, so I am now out of GPU memory.

If I then try to render a second time, I get this error: ``` OPTIX_ERROR_CUDA_ERROR in optixDeviceContextCreate(cuContext, &options, &context) (C:\Users\blender\git\blender-v410\blender.git\intern\cycles\device\optix\device_impl.cpp:90) Refer to the Cycles GPU rendering documentation for possible solutions: https://docs.blender.org/manual/en/latest/render/cycles/gpu_rendering.html OPTIX_ERROR_INVALID_DEVICE_CONTEXT in optixDeviceContextSetLogCallback( context, options.logCallbackFunction, options.logCallbackData, options.logCallbackLevel) (C:\Users\blender\git\blender-v410\blender.git\intern\cycles\device\optix\device_impl.cpp:93) System is out of GPU memory Illegal address in cuStreamCreate(&cuda_stream_, CU_STREAM_NON_BLOCKING) (C:\Users\blender\git\blender-v410\blender.git\intern\cycles\device\cuda\queue.cpp:21) ``` The VRAM from the previous frame wasn't freed, so I am now out of GPU memory.

I can confirm.
This appears to be a case of Blender running out of memory, but the image resolution doesn't seem to be that extreme for that.

Possibly related to one of these other reports:
#104966: OptiX Denoiser fails with large output resolution (final rendering completely black)
#78505: OptiX Denoiser crashes OPTIX_ERROR_CUDA_ERROR in optixDenoiserComputeMemoryResources
#117112: Cycles crash on CPU denoise with old gpu driver

I can confirm. This appears to be a case of Blender running out of memory, but the image resolution doesn't seem to be that extreme for that. Possibly related to one of these other reports: #104966: OptiX Denoiser fails with large output resolution (final rendering completely black) #78505: OptiX Denoiser crashes OPTIX_ERROR_CUDA_ERROR in optixDenoiserComputeMemoryResources #117112: Cycles crash on CPU denoise with old gpu driver
Germano Cavalcante added
Module
Render & Cycles
Status
Confirmed
Interest
Cycles
and removed
Status
Needs Triage
labels 2024-04-02 17:30:31 +02:00
Author
Contributor

I don't think the issue is Cycles only since the UI breaks as well.

I don't think the issue is Cycles only since the UI breaks as well.

Blender 4.1.0
Ubuntu 23.10 x86_64 + Intel repo drivers
i5-12400F
ARC A750
64GB of ram

I got a little different outcome with Intel ARC A750 on linux

Screenshot is with Denosing checked Vram used 1463M, without it checked Vram used 1200M both times the rendering stops but Blender seems ok.

I did close and open blender with each test

oneAPI kernel "integrator_shade_surface" execution error: got runtime exception "Native API failed. Native API returns: -6 (PI_ERROR_OUT_OF_HOST_MEMORY) -6

I'm going to post this and see what happens with CPU only rendering.

Blender 4.1.0 Ubuntu 23.10 x86_64 + Intel repo drivers i5-12400F ARC A750 64GB of ram I got a little different outcome with Intel ARC A750 on linux Screenshot is with Denosing checked Vram used 1463M, without it checked Vram used 1200M both times the rendering stops but Blender seems ok. I did close and open blender with each test oneAPI kernel "integrator_shade_surface" execution error: got runtime exception "Native API failed. Native API returns: -6 (PI_ERROR_OUT_OF_HOST_MEMORY) -6 I'm going to post this and see what happens with CPU only rendering.

With CPU only rendering my whole computer locked up hard about the same spot, peak ram used was only ~250M on the rendering screen before I had to force a reset.

With CPU only rendering my whole computer locked up hard about the same spot, peak ram used was only ~250M on the rendering screen before I had to force a reset.
Author
Contributor

@aafra this might be an interesting report for you. I tried to go out of VRAM but didn't succeed.

@aafra this might be an interesting report for you. I tried to go out of VRAM but didn't succeed.
Sign in to join this conversation.
No Label
Interest
Alembic
Interest
Animation & Rigging
Interest
Asset Browser
Interest
Asset Browser Project Overview
Interest
Audio
Interest
Automated Testing
Interest
Blender Asset Bundle
Interest
BlendFile
Interest
Collada
Interest
Compatibility
Interest
Compositing
Interest
Core
Interest
Cycles
Interest
Dependency Graph
Interest
Development Management
Interest
EEVEE
Interest
EEVEE & Viewport
Interest
Freestyle
Interest
Geometry Nodes
Interest
Grease Pencil
Interest
ID Management
Interest
Images & Movies
Interest
Import Export
Interest
Line Art
Interest
Masking
Interest
Metal
Interest
Modeling
Interest
Modifiers
Interest
Motion Tracking
Interest
Nodes & Physics
Interest
OpenGL
Interest
Overlay
Interest
Overrides
Interest
Performance
Interest
Physics
Interest
Pipeline, Assets & IO
Interest
Platforms, Builds & Tests
Interest
Python API
Interest
Render & Cycles
Interest
Render Pipeline
Interest
Sculpt, Paint & Texture
Interest
Text Editor
Interest
Translations
Interest
Triaging
Interest
Undo
Interest
USD
Interest
User Interface
Interest
UV Editing
Interest
VFX & Video
Interest
Video Sequencer
Interest
Virtual Reality
Interest
Vulkan
Interest
Wayland
Interest
Workbench
Interest: X11
Legacy
Blender 2.8 Project
Legacy
Milestone 1: Basic, Local Asset Browser
Legacy
OpenGL Error
Meta
Good First Issue
Meta
Papercut
Meta
Retrospective
Meta
Security
Module
Animation & Rigging
Module
Core
Module
Development Management
Module
EEVEE & Viewport
Module
Grease Pencil
Module
Modeling
Module
Nodes & Physics
Module
Pipeline, Assets & IO
Module
Platforms, Builds & Tests
Module
Python API
Module
Render & Cycles
Module
Sculpt, Paint & Texture
Module
Triaging
Module
User Interface
Module
VFX & Video
Platform
FreeBSD
Platform
Linux
Platform
macOS
Platform
Windows
Priority
High
Priority
Low
Priority
Normal
Priority
Unbreak Now!
Status
Archived
Status
Confirmed
Status
Duplicate
Status
Needs Info from Developers
Status
Needs Information from User
Status
Needs Triage
Status
Resolved
Type
Bug
Type
Design
Type
Known Issue
Type
Patch
Type
Report
Type
To Do
No Milestone
No project
No Assignees
3 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: blender/blender#120167
No description provided.