Memory (RAM) leak since 4.0.0 in bpy when rendering several frames with Cycles #129384

New Issue

Julien-Philip · 2024-10-24T20:16:49+02:00

Julien-Philip commented

2024-10-24 20:16:49 +02:00

System Information
Operating system: Ubuntu 22.04.4 LTS
Graphics card: A10G (also tested on T4 and A100) - Driver Version: 535.183.01 CUDA Version: 12.2

Blender Version
Broken: bpy 4.0.0 to 4.2.0 (tested both)
Worked: bpy 3.6.0 (also tested)

Short description of error
I implemented a system to render scenes programmatically using bpy in Python.
I create or load a scene, then proceed to render the scene from different viewpoints, changing objects and or lighting.
Since version 4.0.0 of bpy I experience a very aggressive memory leak (accounting for several GB of RAM after a few renders) this eventually fill up all the available memory. I believe this leak might exist in the desktop version of blender as well and might impact animation rendering.

Exact steps for others to reproduce the error
On a linux machine
Create a conda environment:
conda create --name blender_test python=3.10
Activate the environment
conda activate blender_test
Install bpy 4.0.0 (alternatively you'd need python 3.11 with bpy 4.2.0) (or 3.6.0 to verify the working version)
pip install bpy==4.0.0
Run the attached script:
python memory_leak_repro.py

This scripts expect two environment files as .exr files that I got from there. You can use other exr files of course, just give the proper path in the script.
https://polyhaven.com/a/abandoned_church
https://polyhaven.com/a/abandoned_parking

My 2cts
The script creates a simple scene with a sphere and renders it with Cycles using Optix on the GPU.
It sets up the geometry, a camera and loads two hdris.
It then proceeds to render alternatively changing from one hdri to the other.
The line bpy.context.scene.camera = camera is here to trigger the update that happens typically when changing the camera or using a different camera. The bug is not present when commenting this line but this should not be considered a "solution" to the problem, I implemented it this way to make the file as simple as possible.
I did a lot of digging and tried many "tips" to solve this issue but could not. I believe that some memory is allocated either when loading images for cycles and/or loading geometry BVH and it is not released.

With bpy==3.6.0 the above script max out as 1.8GB RAM (as measured by the RES column of top) with an average of 1.6GB which stays stable after a few renderings. After a factory reset (at the end of the script) 1.2GB are left in the process.
With bpy==4.* the memory is increasing steadily from 1.5GB to 4+GB over the 100 renders (and keeps increasing with more renders). After the factory reset 2+GB are left in the process.

Thanks a lot!

**System Information** Operating system: Ubuntu 22.04.4 LTS Graphics card: A10G (also tested on T4 and A100) - Driver Version: 535.183.01 CUDA Version: 12.2 **Blender Version** Broken: bpy 4.0.0 to 4.2.0 (tested both) Worked: bpy 3.6.0 (also tested) **Short description of error** I implemented a system to render scenes programmatically using bpy in Python. I create or load a scene, then proceed to render the scene from different viewpoints, changing objects and or lighting. Since version 4.0.0 of bpy I experience a very aggressive memory leak (accounting for several GB of RAM after a few renders) this eventually fill up all the available memory. I believe this leak might exist in the desktop version of blender as well and might impact animation rendering. **Exact steps for others to reproduce the error** On a linux machine Create a conda environment: `conda create --name blender_test python=3.10` Activate the environment `conda activate blender_test` Install bpy 4.0.0 (alternatively you'd need python 3.11 with bpy 4.2.0) (or 3.6.0 to verify the working version) `pip install bpy==4.0.0` Run the attached script: `python memory_leak_repro.py` This scripts expect two environment files as .exr files that I got from there. You can use other exr files of course, just give the proper path in the script. https://polyhaven.com/a/abandoned_church https://polyhaven.com/a/abandoned_parking **My 2cts** The script creates a simple scene with a sphere and renders it with Cycles using Optix on the GPU. It sets up the geometry, a camera and loads two hdris. It then proceeds to render alternatively changing from one hdri to the other. The line `bpy.context.scene.camera = camera` is here to trigger the update that happens typically when changing the camera or using a different camera. The bug is not present when commenting this line but this should not be considered a "solution" to the problem, I implemented it this way to make the file as simple as possible. I did a lot of digging and tried many "tips" to solve this issue but could not. I believe that some memory is allocated either when loading images for cycles and/or loading geometry BVH and it is not released. With bpy==3.6.0 the above script max out as 1.8GB RAM (as measured by the RES column of top) with an average of 1.6GB which stays stable after a few renderings. After a factory reset (at the end of the script) 1.2GB are left in the process. With bpy==4.* the memory is increasing steadily from 1.5GB to 4+GB over the 100 renders (and keeps increasing with more renders). After the factory reset 2+GB are left in the process. Thanks a lot!

memory_leak_repro.py

2.3 KiB

Julien-Philip added the

labels 2024-10-24 20:16:49 +02:00

Alaska commented

2024-10-25 11:24:12 +02:00

I can confirm the issue on Linux, but not Windows.
I did make some modifications to the script just for my own testing:

I used the CPU for rendering.
I set the sample count to 1
I set the render resolution to 256x256
I set the for loop to 100000 iterations rather than 100 so I can go off and do something, then come back and see that the RAM usage had risen drasitcally.

I will mark this as a render and cycles module issue as it's happening during rendering, but it might be a task for a different module.

System Information
Operating system: Linux-6.8.0-47-generic-x86_64-with-glibc2.39 64 Bits, X11 UI
BPY: 4.2.0

I can confirm the issue on Linux, but not Windows. I did make some modifications to the script just for my own testing: - I used the CPU for rendering. - I set the sample count to 1 - I set the render resolution to 256x256 - I set the for loop to 100000 iterations rather than 100 so I can go off and do something, then come back and see that the RAM usage had risen drasitcally. I will mark this as a render and cycles module issue as it's happening during rendering, but it might be a task for a different module. **System Information** Operating system: Linux-6.8.0-47-generic-x86_64-with-glibc2.39 64 Bits, X11 UI BPY: 4.2.0

Alaska added

Download

What's New

Blender Studio

Manual

Developers Blog

Documentation

Benchmark

Blender Conference

Development Fund

One-time Donations

Memory (RAM) leak since 4.0.0 in bpy when rendering several frames with Cycles #129384