CUDA queue error related to integrator_reset_film_convert_combined_half_rgba #117415

New Issue

Paul Melis · 2024-01-22T17:05:11+01:00

Paul Melis commented

2024-01-22 17:05:11 +01:00

System Information
Operating system: RHEL 8.6
Graphics card: 4x NVIDIA A100-SXM4-40GB (CUDA: 12.3, NVIDIA driver 545.23.08)

Blender Version
Broken: 3.6.5

Short description of error

During interactive rendering I occasionally get an error in the viewport (and prints in console as well): Invalid value in CUDA queue enqueue (integrator_reset_film_convert_combined_half_rgba). Rendering then stops, but the viewport remains active. Switching out of interactive render and back again restarts the render, but it triggers quite some times. I think this might be related to a CUDA update on our side, but I'm not sure. This is a locally compiled Blender, but I can't find any reference online to the specific integrator_... call being a known issue. Nor do I see that function name, or something like it, in the blender source code. Any clue where I could start looking in the code to see what's going on?

Exact steps for others to reproduce the error

It's really hard to provide a simple test case that shows this error. The Blender file I encounter this with is 17.6GB, containing a small number of huge meshes (165.7M triangles in total). It's otherwise not very special or complex. Given the name of the symbol in the error I don't suspect the issue is related to the scene size, but something in realtime viewport compositing.

**System Information** Operating system: RHEL 8.6 Graphics card: 4x NVIDIA A100-SXM4-40GB (CUDA: 12.3, NVIDIA driver 545.23.08) **Blender Version** Broken: 3.6.5 **Short description of error** During interactive rendering I occasionally get an error in the viewport (and prints in console as well): `Invalid value in CUDA queue enqueue (integrator_reset_film_convert_combined_half_rgba)`. Rendering then stops, but the viewport remains active. Switching out of interactive render and back again restarts the render, but it triggers quite some times. I think this might be related to a CUDA update on our side, but I'm not sure. This is a locally compiled Blender, but I can't find any reference online to the specific `integrator_...` call being a known issue. Nor do I see that function name, or something like it, in the blender source code. Any clue where I could start looking in the code to see what's going on? **Exact steps for others to reproduce the error** It's really hard to provide a simple test case that shows this error. The Blender file I encounter this with is 17.6GB, containing a small number of huge meshes (165.7M triangles in total). It's otherwise not very special or complex. Given the name of the symbol in the error I don't suspect the issue is related to the scene size, but something in realtime viewport compositing.

Paul Melis added the

labels 2024-01-22 17:05:11 +01:00

Paul Melis commented

2024-01-22 17:12:11 +01:00

Hmmm, I only now look at the GPU mem stats. Blender reports 38.9 out of 40GB, nvidia-smi 39248MiB out of 40960MiB. That's pretty close to full, so I guess during interactive render, switching in and out and tweaking the scene (but not meshes) a bit it might actually fill up the GPU memory. Would that trigger an explicit memory full error, or might that cause the issue reported above?

Germano Cavalcante commented

2024-01-22 19:37:03 +01:00

Any clue where I could start looking in the code to see what's going on?

I would suggest starting by investigating the CUDA integration code.
You may want to try asking in the community channels as this bug tracker is only for confirmed bugs and errors: https://www.blender.org/community/

Would that trigger an explicit memory full error, or might that cause the issue reported above?

Based on other reports with the same error message, this is most likely a memory issue.
#112843: Error when editing displacement values: Illegal address in CUDA queue copy_from_device
#110175: CUDA misaligned address error when rendering the image

Unfortunately this report can't be triaged as a complete bug report.

For the development team to be able to help, there needs to be reproducible steps.

Normally .blend files can be simplified by removing most objects and disabling settings, until the problem reveals itself more clearly.

> Any clue where I could start looking in the code to see what's going on? I would suggest starting by investigating the CUDA integration code. You may want to try asking in the community channels as this bug tracker is only for confirmed bugs and errors: https://www.blender.org/community/ > Would that trigger an explicit memory full error, or might that cause the issue reported above? Based on other reports with the same error message, this is most likely a memory issue. #112843: Error when editing displacement values: Illegal address in CUDA queue copy_from_device #110175: CUDA misaligned address error when rendering the image --- Unfortunately this report can't be triaged as a complete bug report. For the development team to be able to help, there needs to be reproducible steps. Normally .blend files can be simplified by removing most objects and disabling settings, until the problem reveals itself more clearly.

Germano Cavalcante added

Status

Needs Information from User

and removed

Status

Needs Triage

labels 2024-01-22 19:37:13 +01:00

Germano Cavalcante commented

2024-02-28 15:46:05 +01:00

No activity for more than a week. As per the tracker policy we assume the issue is gone and can be closed.

Thanks for the report. If the problem persists please open a new report with the required information.

No activity for more than a week. As per the tracker policy we assume the issue is gone and can be closed. Thanks for the report. If the problem persists please open a new report with the required information.

Germano Cavalcante closed this issue

2024-02-28 15:46:08 +01:00

Blender Bot added

Status

Archived

and removed

Status

Needs Information from User

labels 2024-02-28 15:46:09 +01:00

Sign in to join this conversation.