Memory leak if both a shader and an AddonPreferences class are used #71362

New Issue

Simon Wendsche · 2019-11-05T15:13:03+01:00

Simon Wendsche commented

2019-11-05 15:13:03 +01:00

System Information
Operating system: Windows-10-10.0.18362 64 Bits
Graphics card: GeForce RTX 2080/PCIe/SSE2 NVIDIA Corporation 4.5.0 NVIDIA 431.86

Blender Version
Broken: version: Official 2.80 (sub 75), branch: master, commit date: 2019-07-29 14:47, hash: f6cb5f5449

Short description of error
Blender shows the warning Error: Not freed memory blocks: 4, total unfreed memory 0.012421 MB when an addon uses both AddonPreferences with an implementation of the draw() method and an instance of gpu.types.GPUShader.

Exact steps for others to reproduce the error

Install the following demo addon: memleak.zip
Start Blender from console with blender.exe --factory-startup --addons memleak
Exit Blender. It will print Error: Not freed memory blocks: 4, total unfreed memory 0.012421 MB.

If you open the addon in a text editor and comment out line 30 (shader = gpu.types.GPUShader(vertex_shader, fragment_shader)), the error is no longer shown.
Similarly, if you comment out lines 37 and 38 instead (def draw(self, context): pass) and leave line 30 enabled, the error will also not show up.
Of course this problem might also be caused by other conditions, but those two (shader and AddonPreferences class) are the combination where I spotted the problem first.

**System Information** Operating system: Windows-10-10.0.18362 64 Bits Graphics card: GeForce RTX 2080/PCIe/SSE2 NVIDIA Corporation 4.5.0 NVIDIA 431.86 **Blender Version** Broken: version: Official 2.80 (sub 75), branch: master, commit date: 2019-07-29 14:47, hash: `f6cb5f5449` **Short description of error** Blender shows the warning `Error: Not freed memory blocks: 4, total unfreed memory 0.012421 MB` when an addon uses both AddonPreferences with an implementation of the `draw()` method **and** an instance of `gpu.types.GPUShader`. **Exact steps for others to reproduce the error** * Install the following demo addon: [memleak.zip](https://archive.blender.org/developer/F7947836/memleak.zip) * Start Blender from console with `blender.exe --factory-startup --addons memleak` * Exit Blender. It will print `Error: Not freed memory blocks: 4, total unfreed memory 0.012421 MB`. If you open the addon in a text editor and comment out line 30 (`shader = gpu.types.GPUShader(vertex_shader, fragment_shader)`), the error is no longer shown. Similarly, if you comment out lines 37 and 38 instead (`def draw(self, context): pass`) and leave line 30 enabled, the error will also not show up. Of course this problem might also be caused by other conditions, but those two (shader and AddonPreferences class) are the combination where I spotted the problem first.

Simon Wendsche commented

2019-11-05 15:13:03 +01:00

Added subscriber: @BYOB

Germano Cavalcante commented

2019-11-06 15:00:20 +01:00

Added subscriber: @mano-wii

Germano Cavalcante commented

2019-11-06 15:00:20 +01:00

Although it is not recommended to create and keep a reference to an object type Shader in the top level of a module (shaders are slow to create and use a lot of GPU resources),
the fact that an object of type AddonPreferences results in a memory leak indicates that somewhere in the creation of such object has a wrong refcount.

This is something that deserves investigation but doesn't have much priority.

Although it is not recommended to create and keep a reference to an object type Shader in the top level of a module (shaders are slow to create and use a lot of GPU resources), the fact that an object of type `AddonPreferences` results in a memory leak indicates that somewhere in the creation of such object has a wrong refcount. This is something that deserves investigation but doesn't have much priority.

Simon Wendsche commented

2019-11-07 10:07:13 +01:00

it is not recommended to create and maintain a reference to an object type shader in the top level of a module (shaders are slow to create and use a lot of GPU resources)

Why would that be a problem? This code is executed exactly once at Blender startup.
(This is a bit off-topic, but I would appreciate to learn more about this comment)

> it is not recommended to create and maintain a reference to an object type shader in the top level of a module (shaders are slow to create and use a lot of GPU resources) Why would that be a problem? This code is executed exactly once at Blender startup. (This is a bit off-topic, but I would appreciate to learn more about this comment)

Sybren A. Stüvel commented

2020-01-23 12:57:39 +01:00

Added subscriber: @dr.sybren

Sybren A. Stüvel commented

2020-01-23 12:57:39 +01:00

Python doesn't always free everything cleanly when the program quits. The memory leak could be due to having any registerable classes in the module, which could impact the order in which things are freed.
I tested the above, and the problem doesn't occur when replacing the bpy.types.AddonPreferences superclass with a bpy.types.Panel or bpy.types.Operator.

This code is executed exactly once at Blender startup.

It's also re-executed when reloading scripts.

I would recommend allocating things in the register() function instead, and freeing in unregister(). This is what I do in the Blender Cloud add-on, for example: https://developer.blender.org/diffusion/BCA/browse/master/blender_cloud/attract/draw.py. This doesn't resolve this issue, but at least you only create the shader when the add-on is enabled.

Python doesn't always free everything cleanly when the program quits. The memory leak could be due to having any registerable classes in the module, which could impact the order in which things are freed. I tested the above, and the problem doesn't occur when replacing the `bpy.types.AddonPreferences` superclass with a `bpy.types.Panel` or `bpy.types.Operator`. > This code is executed exactly once at Blender startup. It's also re-executed when reloading scripts. I would recommend allocating things in the `register()` function instead, and freeing in `unregister()`. This is what I do in the Blender Cloud add-on, for example: https://developer.blender.org/diffusion/BCA/browse/master/blender_cloud/attract/draw.py. This doesn't resolve this issue, but at least you only create the shader when the add-on is enabled.

Jeroen Bakker commented

2020-05-18 12:22:38 +02:00

Added subscriber: @Jeroen-Bakker

Jeroen Bakker commented

2020-05-18 12:22:38 +02:00

The register/unregister does not fix the issue

P1398: (An Untitled Masterwork)

bl_info = {
    "name": "shader mem leak demo",
    "author": "Simon Wendsche (B.Y.O.B.)",
    "category": "",
    "blender": (2, 80, 0),
}

vertex_shader = '''
    void main()
    {
        gl_Position = vec4(0, 0, 0, 0);
    }
'''

fragment_shader = '''
    void main()
    {
        gl_FragColor = vec4(0, 0, 0, 0);
    }
'''

import gpu
from os.path import basename, dirname
from bpy.utils import register_class, unregister_class
from bpy.types import AddonPreferences

addon_name = basename(dirname(__file__))

# If this line commented out, the leak does not appear

shader = None

class TestAddonPreferences(AddonPreferences):
    bl_idname = addon_name

    # If this function is commented out, the leak does not appear
    def draw(self, context):
        pass


def register():
    global shader
    shader = gpu.types.GPUShader(vertex_shader, fragment_shader)
    register_class(TestAddonPreferences)

def unregister():
    global shader
    unregister_class(TestAddonPreferences)
    del shader

Interesting that commenting out the draw method fixes the issue or using a different base class. I also tried some variations with class variables.

The `register`/`unregister` does not fix the issue [P1398: (An Untitled Masterwork)](https://archive.blender.org/developer/P1398.txt) ``` bl_info = { "name": "shader mem leak demo", "author": "Simon Wendsche (B.Y.O.B.)", "category": "", "blender": (2, 80, 0), } vertex_shader = ''' void main() { gl_Position = vec4(0, 0, 0, 0); } ''' fragment_shader = ''' void main() { gl_FragColor = vec4(0, 0, 0, 0); } ''' import gpu from os.path import basename, dirname from bpy.utils import register_class, unregister_class from bpy.types import AddonPreferences addon_name = basename(dirname(__file__)) # If this line commented out, the leak does not appear shader = None class TestAddonPreferences(AddonPreferences): bl_idname = addon_name # If this function is commented out, the leak does not appear def draw(self, context): pass def register(): global shader shader = gpu.types.GPUShader(vertex_shader, fragment_shader) register_class(TestAddonPreferences) def unregister(): global shader unregister_class(TestAddonPreferences) del shader ``` Interesting that commenting out the draw method fixes the issue or using a different base class. I also tried some variations with class variables.

Campbell Barton commented

2020-10-16 07:06:12 +02:00

Added subscriber: @ideasman42

Campbell Barton commented

2020-10-16 07:06:13 +02:00

This report is it's behavior depends on Python's GC, in general that shouldn't be something we worry about unless it's really a leak,
OTOH, that it causes Blender to report leaks isn't good. That there doesn't seem to be a reliable workaround isn't good either.

On testing P1398, I can't redo the error, it never leaks after unregister is called.

I suspect Python's GC isn't cleaning up after calling unregister in some situations.

Does this patch resolve the issue?

diff --git a/release/scripts/startup/bl_ui/space_userpref.py b/release/scripts/startup/bl_ui/space_userpref.py
index 77b0f17a937..a3130ef4145 100644
--- a/release/scripts/startup/bl_ui/space_userpref.py
+++ b/release/scripts/startup/bl_ui/space_userpref.py
@@ -1971,6 +1971,9 @@ class USERPREF_PT_addons(AddOnPanel, Panel):
                                     traceback.print_exc()
                                     box_prefs.label(text="Error (see console)", icon='ERROR')
                                 del addon_preferences_class.layout
+                                del addon_preferences_class
+                            del draw
+                            del addon_preferences
 
         - Append missing scripts
         - First collect scripts that are used but have no script file.

I've committed a change to call unregister at exit, since it's quiet fast and allows us to avoid false positive resource leaks. fa566157a5.

This report is it's behavior depends on Python's GC, in general that shouldn't be something we worry about unless it's really a leak, OTOH, that it causes Blender to report leaks isn't good. That there doesn't seem to be a reliable workaround isn't good either. On testing [P1398](https://archive.blender.org/developer/P1398.txt), I can't redo the error, it never leaks after unregister is called. I suspect Python's GC isn't cleaning up after calling `unregister` in some situations. Does this patch resolve the issue? ``` diff --git a/release/scripts/startup/bl_ui/space_userpref.py b/release/scripts/startup/bl_ui/space_userpref.py index 77b0f17a937..a3130ef4145 100644 --- a/release/scripts/startup/bl_ui/space_userpref.py +++ b/release/scripts/startup/bl_ui/space_userpref.py @@ -1971,6 +1971,9 @@ class USERPREF_PT_addons(AddOnPanel, Panel): traceback.print_exc() box_prefs.label(text="Error (see console)", icon='ERROR') del addon_preferences_class.layout + del addon_preferences_class + del draw + del addon_preferences - Append missing scripts - First collect scripts that are used but have no script file. ``` I've committed a change to call `unregister` at exit, since it's quiet fast and allows us to avoid false positive resource leaks. fa566157a5.

Germano Cavalcante commented

2020-10-16 15:06:41 +02:00

Changed status from 'Confirmed' to: 'Resolved'

Germano Cavalcante closed this issue

2020-10-16 15:06:41 +02:00

Germano Cavalcante self-assigned this 2020-10-16 15:06:41 +02:00

Germano Cavalcante commented

2020-10-16 15:06:41 +02:00

In #71362#1035419, @ideasman42 wrote:
Does this patch resolve the issue?

diff --git a/release/scripts/startup/bl_ui/space_userpref.py b/release/scripts/startup/bl_ui/space_userpref.py
index 77b0f17a937..a3130ef4145 100644
--- a/release/scripts/startup/bl_ui/space_userpref.py
+++ b/release/scripts/startup/bl_ui/space_userpref.py
@@ -1971,6 +1971,9 @@ class USERPREF_PT_addons(AddOnPanel, Panel):
                                     traceback.print_exc()
                                     box_prefs.label(text="Error (see console)", icon='ERROR')
                                 del addon_preferences_class.layout
+                                del addon_preferences_class
+                            del draw
+                            del addon_preferences
 
         # Append missing scripts
         # First collect scripts that are used but have no script file.

This patch does not solve the issue, but with fa566157a5 I can't reproduce the problem anymore.
So I believe we can consider it resolved by that commit :)

> In #71362#1035419, @ideasman42 wrote: > Does this patch resolve the issue? > > ``` > diff --git a/release/scripts/startup/bl_ui/space_userpref.py b/release/scripts/startup/bl_ui/space_userpref.py > index 77b0f17a937..a3130ef4145 100644 > --- a/release/scripts/startup/bl_ui/space_userpref.py > +++ b/release/scripts/startup/bl_ui/space_userpref.py > @@ -1971,6 +1971,9 @@ class USERPREF_PT_addons(AddOnPanel, Panel): > traceback.print_exc() > box_prefs.label(text="Error (see console)", icon='ERROR') > del addon_preferences_class.layout > + del addon_preferences_class > + del draw > + del addon_preferences > > # Append missing scripts > # First collect scripts that are used but have no script file. > ``` This patch does not solve the issue, but with fa566157a5 I can't reproduce the problem anymore. So I believe we can consider it resolved by that commit :)

Sign in to join this conversation.

No Label

Download

What's New

Blender Studio

Manual

Developers Blog

Documentation

Benchmark

Blender Conference

Development Fund

One-time Donations

Memory leak if both a shader and an AddonPreferences class are used #71362