Memory leak if both a shader and an AddonPreferences class are used #71362

Closed
opened 2019-11-05 15:13:03 +01:00 by Simon Wendsche · 12 comments

System Information
Operating system: Windows-10-10.0.18362 64 Bits
Graphics card: GeForce RTX 2080/PCIe/SSE2 NVIDIA Corporation 4.5.0 NVIDIA 431.86

Blender Version
Broken: version: Official 2.80 (sub 75), branch: master, commit date: 2019-07-29 14:47, hash: f6cb5f5449

Short description of error
Blender shows the warning Error: Not freed memory blocks: 4, total unfreed memory 0.012421 MB when an addon uses both AddonPreferences with an implementation of the draw() method and an instance of gpu.types.GPUShader.

Exact steps for others to reproduce the error

  • Install the following demo addon: memleak.zip
  • Start Blender from console with blender.exe --factory-startup --addons memleak
  • Exit Blender. It will print Error: Not freed memory blocks: 4, total unfreed memory 0.012421 MB.

If you open the addon in a text editor and comment out line 30 (shader = gpu.types.GPUShader(vertex_shader, fragment_shader)), the error is no longer shown.
Similarly, if you comment out lines 37 and 38 instead (def draw(self, context): pass) and leave line 30 enabled, the error will also not show up.
Of course this problem might also be caused by other conditions, but those two (shader and AddonPreferences class) are the combination where I spotted the problem first.

**System Information** Operating system: Windows-10-10.0.18362 64 Bits Graphics card: GeForce RTX 2080/PCIe/SSE2 NVIDIA Corporation 4.5.0 NVIDIA 431.86 **Blender Version** Broken: version: Official 2.80 (sub 75), branch: master, commit date: 2019-07-29 14:47, hash: `f6cb5f5449` **Short description of error** Blender shows the warning `Error: Not freed memory blocks: 4, total unfreed memory 0.012421 MB` when an addon uses both AddonPreferences with an implementation of the `draw()` method **and** an instance of `gpu.types.GPUShader`. **Exact steps for others to reproduce the error** * Install the following demo addon: [memleak.zip](https://archive.blender.org/developer/F7947836/memleak.zip) * Start Blender from console with `blender.exe --factory-startup --addons memleak` * Exit Blender. It will print `Error: Not freed memory blocks: 4, total unfreed memory 0.012421 MB`. If you open the addon in a text editor and comment out line 30 (`shader = gpu.types.GPUShader(vertex_shader, fragment_shader)`), the error is no longer shown. Similarly, if you comment out lines 37 and 38 instead (`def draw(self, context): pass`) and leave line 30 enabled, the error will also not show up. Of course this problem might also be caused by other conditions, but those two (shader and AddonPreferences class) are the combination where I spotted the problem first.

Added subscriber: @BYOB

Added subscriber: @BYOB

Added subscriber: @mano-wii

Added subscriber: @mano-wii

Although it is not recommended to create and keep a reference to an object type Shader in the top level of a module (shaders are slow to create and use a lot of GPU resources),
the fact that an object of type AddonPreferences results in a memory leak indicates that somewhere in the creation of such object has a wrong refcount.

This is something that deserves investigation but doesn't have much priority.

Although it is not recommended to create and keep a reference to an object type Shader in the top level of a module (shaders are slow to create and use a lot of GPU resources), the fact that an object of type `AddonPreferences` results in a memory leak indicates that somewhere in the creation of such object has a wrong refcount. This is something that deserves investigation but doesn't have much priority.

it is not recommended to create and maintain a reference to an object type shader in the top level of a module (shaders are slow to create and use a lot of GPU resources)

Why would that be a problem? This code is executed exactly once at Blender startup.
(This is a bit off-topic, but I would appreciate to learn more about this comment)

> it is not recommended to create and maintain a reference to an object type shader in the top level of a module (shaders are slow to create and use a lot of GPU resources) Why would that be a problem? This code is executed exactly once at Blender startup. (This is a bit off-topic, but I would appreciate to learn more about this comment)

Added subscriber: @dr.sybren

Added subscriber: @dr.sybren

Python doesn't always free everything cleanly when the program quits. The memory leak could be due to having any registerable classes in the module, which could impact the order in which things are freed.
I tested the above, and the problem doesn't occur when replacing the bpy.types.AddonPreferences superclass with a bpy.types.Panel or bpy.types.Operator.

This code is executed exactly once at Blender startup.

It's also re-executed when reloading scripts.

I would recommend allocating things in the register() function instead, and freeing in unregister(). This is what I do in the Blender Cloud add-on, for example: https://developer.blender.org/diffusion/BCA/browse/master/blender_cloud/attract/draw.py. This doesn't resolve this issue, but at least you only create the shader when the add-on is enabled.

Python doesn't always free everything cleanly when the program quits. The memory leak could be due to having any registerable classes in the module, which could impact the order in which things are freed. I tested the above, and the problem doesn't occur when replacing the `bpy.types.AddonPreferences` superclass with a `bpy.types.Panel` or `bpy.types.Operator`. > This code is executed exactly once at Blender startup. It's also re-executed when reloading scripts. I would recommend allocating things in the `register()` function instead, and freeing in `unregister()`. This is what I do in the Blender Cloud add-on, for example: https://developer.blender.org/diffusion/BCA/browse/master/blender_cloud/attract/draw.py. This doesn't resolve this issue, but at least you only create the shader when the add-on is enabled.

Added subscriber: @Jeroen-Bakker

Added subscriber: @Jeroen-Bakker

The register/unregister does not fix the issue

P1398: (An Untitled Masterwork)

bl_info = {
    "name": "shader mem leak demo",
    "author": "Simon Wendsche (B.Y.O.B.)",
    "category": "",
    "blender": (2, 80, 0),
}

vertex_shader = '''
    void main()
    {
        gl_Position = vec4(0, 0, 0, 0);
    }
'''

fragment_shader = '''
    void main()
    {
        gl_FragColor = vec4(0, 0, 0, 0);
    }
'''

import gpu
from os.path import basename, dirname
from bpy.utils import register_class, unregister_class
from bpy.types import AddonPreferences

addon_name = basename(dirname(__file__))

# If this line commented out, the leak does not appear

shader = None

class TestAddonPreferences(AddonPreferences):
    bl_idname = addon_name

    # If this function is commented out, the leak does not appear
    def draw(self, context):
        pass


def register():
    global shader
    shader = gpu.types.GPUShader(vertex_shader, fragment_shader)
    register_class(TestAddonPreferences)

def unregister():
    global shader
    unregister_class(TestAddonPreferences)
    del shader

Interesting that commenting out the draw method fixes the issue or using a different base class. I also tried some variations with class variables.

The `register`/`unregister` does not fix the issue [P1398: (An Untitled Masterwork)](https://archive.blender.org/developer/P1398.txt) ``` bl_info = { "name": "shader mem leak demo", "author": "Simon Wendsche (B.Y.O.B.)", "category": "", "blender": (2, 80, 0), } vertex_shader = ''' void main() { gl_Position = vec4(0, 0, 0, 0); } ''' fragment_shader = ''' void main() { gl_FragColor = vec4(0, 0, 0, 0); } ''' import gpu from os.path import basename, dirname from bpy.utils import register_class, unregister_class from bpy.types import AddonPreferences addon_name = basename(dirname(__file__)) # If this line commented out, the leak does not appear shader = None class TestAddonPreferences(AddonPreferences): bl_idname = addon_name # If this function is commented out, the leak does not appear def draw(self, context): pass def register(): global shader shader = gpu.types.GPUShader(vertex_shader, fragment_shader) register_class(TestAddonPreferences) def unregister(): global shader unregister_class(TestAddonPreferences) del shader ``` Interesting that commenting out the draw method fixes the issue or using a different base class. I also tried some variations with class variables.

Added subscriber: @ideasman42

Added subscriber: @ideasman42

This report is it's behavior depends on Python's GC, in general that shouldn't be something we worry about unless it's really a leak,
OTOH, that it causes Blender to report leaks isn't good. That there doesn't seem to be a reliable workaround isn't good either.

On testing P1398, I can't redo the error, it never leaks after unregister is called.

I suspect Python's GC isn't cleaning up after calling unregister in some situations.

Does this patch resolve the issue?

diff --git a/release/scripts/startup/bl_ui/space_userpref.py b/release/scripts/startup/bl_ui/space_userpref.py
index 77b0f17a937..a3130ef4145 100644
--- a/release/scripts/startup/bl_ui/space_userpref.py
+++ b/release/scripts/startup/bl_ui/space_userpref.py
@@ -1971,6 +1971,9 @@ class USERPREF_PT_addons(AddOnPanel, Panel):
                                     traceback.print_exc()
                                     box_prefs.label(text="Error (see console)", icon='ERROR')
                                 del addon_preferences_class.layout
+                                del addon_preferences_class
+                            del draw
+                            del addon_preferences
 
         - Append missing scripts
         - First collect scripts that are used but have no script file.

I've committed a change to call unregister at exit, since it's quiet fast and allows us to avoid false positive resource leaks. fa566157a5.

This report is it's behavior depends on Python's GC, in general that shouldn't be something we worry about unless it's really a leak, OTOH, that it causes Blender to report leaks isn't good. That there doesn't seem to be a reliable workaround isn't good either. On testing [P1398](https://archive.blender.org/developer/P1398.txt), I can't redo the error, it never leaks after unregister is called. I suspect Python's GC isn't cleaning up after calling `unregister` in some situations. Does this patch resolve the issue? ``` diff --git a/release/scripts/startup/bl_ui/space_userpref.py b/release/scripts/startup/bl_ui/space_userpref.py index 77b0f17a937..a3130ef4145 100644 --- a/release/scripts/startup/bl_ui/space_userpref.py +++ b/release/scripts/startup/bl_ui/space_userpref.py @@ -1971,6 +1971,9 @@ class USERPREF_PT_addons(AddOnPanel, Panel): traceback.print_exc() box_prefs.label(text="Error (see console)", icon='ERROR') del addon_preferences_class.layout + del addon_preferences_class + del draw + del addon_preferences - Append missing scripts - First collect scripts that are used but have no script file. ``` I've committed a change to call `unregister` at exit, since it's quiet fast and allows us to avoid false positive resource leaks. fa566157a5.

Changed status from 'Confirmed' to: 'Resolved'

Changed status from 'Confirmed' to: 'Resolved'
Germano Cavalcante self-assigned this 2020-10-16 15:06:41 +02:00

In #71362#1035419, @ideasman42 wrote:
Does this patch resolve the issue?

diff --git a/release/scripts/startup/bl_ui/space_userpref.py b/release/scripts/startup/bl_ui/space_userpref.py
index 77b0f17a937..a3130ef4145 100644
--- a/release/scripts/startup/bl_ui/space_userpref.py
+++ b/release/scripts/startup/bl_ui/space_userpref.py
@@ -1971,6 +1971,9 @@ class USERPREF_PT_addons(AddOnPanel, Panel):
                                     traceback.print_exc()
                                     box_prefs.label(text="Error (see console)", icon='ERROR')
                                 del addon_preferences_class.layout
+                                del addon_preferences_class
+                            del draw
+                            del addon_preferences
 
         # Append missing scripts
         # First collect scripts that are used but have no script file.

This patch does not solve the issue, but with fa566157a5 I can't reproduce the problem anymore.
So I believe we can consider it resolved by that commit :)

> In #71362#1035419, @ideasman42 wrote: > Does this patch resolve the issue? > > ``` > diff --git a/release/scripts/startup/bl_ui/space_userpref.py b/release/scripts/startup/bl_ui/space_userpref.py > index 77b0f17a937..a3130ef4145 100644 > --- a/release/scripts/startup/bl_ui/space_userpref.py > +++ b/release/scripts/startup/bl_ui/space_userpref.py > @@ -1971,6 +1971,9 @@ class USERPREF_PT_addons(AddOnPanel, Panel): > traceback.print_exc() > box_prefs.label(text="Error (see console)", icon='ERROR') > del addon_preferences_class.layout > + del addon_preferences_class > + del draw > + del addon_preferences > > # Append missing scripts > # First collect scripts that are used but have no script file. > ``` This patch does not solve the issue, but with fa566157a5 I can't reproduce the problem anymore. So I believe we can consider it resolved by that commit :)
Sign in to join this conversation.
No Label
Interest
Alembic
Interest
Animation & Rigging
Interest
Asset Browser
Interest
Asset Browser Project Overview
Interest
Audio
Interest
Automated Testing
Interest
Blender Asset Bundle
Interest
BlendFile
Interest
Collada
Interest
Compatibility
Interest
Compositing
Interest
Core
Interest
Cycles
Interest
Dependency Graph
Interest
Development Management
Interest
EEVEE & Viewport
Interest
Freestyle
Interest
Geometry Nodes
Interest
Grease Pencil
Interest
ID Management
Interest
Images & Movies
Interest
Import Export
Interest
Line Art
Interest
Masking
Interest
Metal
Interest
Modeling
Interest
Modifiers
Interest
Motion Tracking
Interest
Nodes & Physics
Interest
OpenGL
Interest
Overrides
Interest
Performance
Interest
Physics
Interest
Pipeline, Assets & IO
Interest
Platforms, Builds & Tests
Interest
Python API
Interest
Render & Cycles
Interest
Render Pipeline
Interest
Sculpt, Paint & Texture
Interest
Text Editor
Interest
Translations
Interest
Triaging
Interest
Undo
Interest
USD
Interest
User Interface
Interest
UV Editing
Interest
VFX & Video
Interest
Video Sequencer
Interest
Virtual Reality
Interest
Vulkan
Interest: Wayland
Legacy
Blender 2.8 Project
Legacy
Milestone 1: Basic, Local Asset Browser
Legacy
OpenGL Error
Meta
Good First Issue
Meta
Papercut
Meta
Retrospective
Meta
Security
Module
Animation & Rigging
Module
Core
Module
Development Management
Module
EEVEE & Viewport
Module
Grease Pencil
Module
Modeling
Module
Nodes & Physics
Module
Pipeline, Assets & IO
Module
Platforms, Builds & Tests
Module
Python API
Module
Render & Cycles
Module
Sculpt, Paint & Texture
Module
Triaging
Module
User Interface
Module
VFX & Video
Platform
FreeBSD
Platform
Linux
Platform
macOS
Platform
Windows
Priority
High
Priority
Low
Priority
Normal
Priority
Unbreak Now!
Status
Archived
Status
Confirmed
Status
Duplicate
Status
Needs Info from Developers
Status
Needs Information from User
Status
Needs Triage
Status
Resolved
Type
Bug
Type
Design
Type
Known Issue
Type
Patch
Type
Report
Type
To Do
No Milestone
No project
5 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: blender/blender#71362
There is no content yet.