Memory access fault by GPU when using HIP Cycles #119993

New Issue

TNT3530 · 2024-03-28T00:19:51+01:00

TNT3530 commented

2024-03-28 00:19:51 +01:00

System Information
Operating system: Ubuntu 22.04
Graphics card: AMD Instinct MI100 (4 cards XGMI GPU bridge)

Blender Version
Broken: 4.1.0 Stable (40a5e739e2)
Worked: Never, I haven't found a version where it works consistently

When attempting to render the pre-made Classroom scene, Blender crashes and core dumps throwing a
Memory access fault by GPU node-X (Agent handle: 0x7f7fdda7b400) on address 0x7f7a7fb95000. Reason: Page not present or supervisor privilege., where X varies depending on the run. It does this when using either 1 or all 4 cards, whether the share memory option is selected or not. It occasionally works as expected when Share Memory is off, but it isn't consistent.

The official Benchmark program similarly crashes, along with an animation .blend I found. The default cube works fine with all settings.

Exact steps for others to reproduce the error

Download Blender 4.1.0 Stable
Open classroom.blend benchmark
Edit -> Properties, select HIP and all/only one GPU
Hit F12/Render and wait about 1 second for the crash

**System Information** Operating system: Ubuntu 22.04 Graphics card: AMD Instinct MI100 (4 cards XGMI GPU bridge) **Blender Version** Broken: 4.1.0 Stable (40a5e739e270) Worked: Never, I haven't found a version where it works consistently When attempting to render the pre-made Classroom scene, Blender crashes and core dumps throwing a `Memory access fault by GPU node-X (Agent handle: 0x7f7fdda7b400) on address 0x7f7a7fb95000. Reason: Page not present or supervisor privilege.`, where X varies depending on the run. It does this when using either 1 or all 4 cards, whether the share memory option is selected or not. It occasionally works as expected when Share Memory is off, but it isn't consistent. The official Benchmark program similarly crashes, along with an animation .blend I found. The default cube works fine with all settings. **Exact steps for others to reproduce the error** - Download Blender 4.1.0 Stable - Open `classroom.blend` benchmark - Edit -> Properties, select HIP and all/only one GPU - Hit F12/Render and wait about 1 second for the crash

TNT3530 added the

labels 2024-03-28 00:19:52 +01:00

Alaska commented

2024-03-28 00:48:02 +01:00

If I remember correctly, this is a issue with certain Linux kernel versions, and it was fixed in one of the 6.7.X releases. Can you try updating your Linux kernel to see if that helps? Please let us know how it goes along with the kernel version you updated too.

Alaska added

Status

Needs Information from User

and removed

Status

Needs Triage

labels 2024-03-28 00:48:12 +01:00

TNT3530 commented

2024-03-28 02:28:06 +01:00

Going from 6.5.0-14-generic to 6.5.0-28-generic did not fix it
Upgrading fully to 6.8.1-060801-generic similarly did not fix it

Going from 6.5.0-14-generic to 6.5.0-28-generic did not fix it Upgrading fully to 6.8.1-060801-generic similarly did not fix it

Alaska commented

2024-03-28 02:35:04 +01:00

Which ROCm version do you have installed?

I also wanted to add that the AMD Instinct MI100 is not officially supported by Cycles (It may work, but it isn't tested, and kernels aren't compiled for it).

Which ROCm version do you have installed? I also wanted to add that the AMD Instinct MI100 is not officially supported by Cycles (It may work, but it isn't tested, and kernels aren't compiled for it).

TNT3530 commented

2024-03-28 02:39:50 +01:00

ROCm 6.0
Edit: 6.7.10-060710-generic and 6.6.22-060622-generic also do not work

On the latter 6.6 version it seems to run classroom more frequently than other versions, as long as memory sharing is off. Still instantly crashes on the animation demo though sadly. If it run all 4 cards plow through classroom in 6.5s, so Ive got high hopes if we can get them working regularly. Though it doesn't seem to de-noise as well as my NVIDIA card does

ROCm 6.0 Edit: 6.7.10-060710-generic and 6.6.22-060622-generic also do not work On the latter 6.6 version it seems to run classroom more frequently than other versions, as long as memory sharing is off. Still instantly crashes on the animation demo though sadly. If it run all 4 cards plow through classroom in 6.5s, so Ive got high hopes if we can get them working regularly. Though it doesn't seem to de-noise as well as my NVIDIA card does

Alaska commented

2024-03-28 03:19:39 +01:00

I am setting this back to Needs Triage. I'm not sure what else to suggest here, hopefully other people can provide better info.

Alaska added

Status

Needs Triage

and removed

Status

Needs Information from User

labels 2024-03-28 03:19:50 +01:00

Alaska commented

2024-03-28 03:30:17 +01:00

CC @salipour you probably know this better, or know the right people to talk too. Can you comment on this report?

Pratik Borhade commented

2024-03-28 05:17:39 +01:00

Hi, think this is same as #112084, closing.

Pratik Borhade closed this issue

2024-03-28 05:17:43 +01:00

Blender Bot added

and removed

labels 2024-03-28 05:17:48 +01:00

Sahar A. Kashi commented

2024-03-28 17:33:16 +01:00

Is this still a valid issue?

Alaska commented

2024-03-29 00:03:45 +01:00

@salipour This is still a valid issue, but it has been "merged" with #112084 due to it having the same error message. So it might be best to continue helping out on #112084 if you have the time.

Pratik Borhade commented

2024-04-02 12:16:34 +02:00

yes, would help if you check #112084
@GZGavinZhao already shared some useful findings there.

yes, would help if you check #112084 @GZGavinZhao already shared some useful findings there.

👍 1

Sign in to join this conversation.

No Label

Download

What's New

Blender Studio

Manual

Developers Blog

Documentation

Benchmark

Blender Conference

Development Fund

One-time Donations

Memory access fault by GPU when using HIP Cycles #119993