removing orphan meshes is slow (after they were in use) #118787

New Issue

Andrej · 2024-02-27T08:12:57+01:00

Andrej commented

2024-02-27 08:12:57 +01:00

Blender 4.0.2

Was working on a script that were creating some temporary meshes (used temporary meshes to keep original geometry still available and easy to switch back) but what I've noticed is the bottle neck in that workflow is not actually processing thousands of meshes geometry using bmesh but just removing left out mesh data.

At first I thought that removing meshes is slow in general but what I've found that removing orphan meshes that were just created and never used goes very fast but if meshes were assigned to some object and then unassigned then removing them becomes 10x slower.

Example script is below, it can be executed in default Blender session - it has two methods that both print timings in console:

create orphan meshes and remove them (~2 seconds)
create orphan meshes, attach them to objects, deattach them from objects, remove orphan meshes (~24 seconds).

Was wondering if it's a known problem or it is a bug.

Also noticed that orphan meshes were not removed during 2nd method (you accomplish that also by setting skip_removal to True at the end of the script) and then removed manually from UI using bpy.ops.outliner.id_operation(type="DELETE") it works a bit faster. There is also option to run "purge all" in Orphan Data outliner and it's lightning fast but it doesn't allow removing specific meshes.

import bpy
from time import time
import bmesh

N_MESHES = 6500


def remove_meshes():
    i = 0
    for mesh in list(bpy.data.meshes):
        if not mesh.name.startswith("test"):
            continue
        bpy.data.meshes.remove(mesh)
        i += 1
    return i


def get_mesh():
    mesh = bpy.data.meshes.new("test")
    bm = bmesh.new()
    bmesh.ops.create_grid(bm, x_segments=1, y_segments=1, size=1)
    bm.to_mesh(mesh)
    return mesh


def create_remove_meshes():
    print("just create and remove meshes")
    t0 = time()
    meshes = []

    for _ in range(N_MESHES):
        meshes.append(get_mesh())

    print(f"meshes created in {time()-t0}")

    meshes_removed = remove_meshes()
    print(f"{meshes_removed} meshes removed in {time()-t0}")


def create_use_remove_meshes(skip_removal=False):
    print("create, use and remove meshes")
    t0 = time()
    meshes = []

    collection = bpy.context.view_layer.active_layer_collection.collection
    cube_data = bpy.data.meshes["Cube"]
    for _ in range(N_MESHES):
        meshes.append(mesh := get_mesh())
        obj = bpy.data.objects.new("test", mesh)
        collection.objects.link(obj)
        # mesh unassigned right away
        obj.data = cube_data

    print(f"meshes created in {time()-t0}")

    if not skip_removal:
        meshes_removed = remove_meshes()
        print(f"{meshes_removed} meshes removed in {time()-t0}")


# just create and remove meshes
# meshes created in 0.6474745273590088
# meshes removed in 2.770238161087036
create_remove_meshes()

# create, use and remove meshes
# meshes created in 0.8088419437408447
# meshes removed in 23.767765045166016
# NOTE: you can also set skip_removal = True
# and try to remove orphan meshes from outliner manually
# (using `bpy.ops.outliner.id_operation(type="DELETE")`)
# and it also will be much faster
create_use_remove_meshes(skip_removal=False)

Blender 4.0.2 Was working on a script that were creating some temporary meshes (used temporary meshes to keep original geometry still available and easy to switch back) but what I've noticed is the bottle neck in that workflow is not actually processing thousands of meshes geometry using bmesh but just removing left out mesh data. At first I thought that removing meshes is slow in general but what I've found that removing orphan meshes that were just created and never used goes very fast but if meshes were assigned to some object and then unassigned then removing them becomes 10x slower. Example script is below, it can be executed in default Blender session - it has two methods that both print timings in console: 1) create orphan meshes and remove them (~2 seconds) 2) create orphan meshes, attach them to objects, deattach them from objects, remove orphan meshes (~24 seconds). Was wondering if it's a known problem or it is a bug. Also noticed that orphan meshes were not removed during 2nd method (you accomplish that also by setting `skip_removal` to `True` at the end of the script) and then removed manually from UI using `bpy.ops.outliner.id_operation(type="DELETE")` it works a bit faster. There is also option to run "purge all" in Orphan Data outliner and it's lightning fast but it doesn't allow removing specific meshes. ```python import bpy from time import time import bmesh N_MESHES = 6500 def remove_meshes(): i = 0 for mesh in list(bpy.data.meshes): if not mesh.name.startswith("test"): continue bpy.data.meshes.remove(mesh) i += 1 return i def get_mesh(): mesh = bpy.data.meshes.new("test") bm = bmesh.new() bmesh.ops.create_grid(bm, x_segments=1, y_segments=1, size=1) bm.to_mesh(mesh) return mesh def create_remove_meshes(): print("just create and remove meshes") t0 = time() meshes = [] for _ in range(N_MESHES): meshes.append(get_mesh()) print(f"meshes created in {time()-t0}") meshes_removed = remove_meshes() print(f"{meshes_removed} meshes removed in {time()-t0}") def create_use_remove_meshes(skip_removal=False): print("create, use and remove meshes") t0 = time() meshes = [] collection = bpy.context.view_layer.active_layer_collection.collection cube_data = bpy.data.meshes["Cube"] for _ in range(N_MESHES): meshes.append(mesh := get_mesh()) obj = bpy.data.objects.new("test", mesh) collection.objects.link(obj) # mesh unassigned right away obj.data = cube_data print(f"meshes created in {time()-t0}") if not skip_removal: meshes_removed = remove_meshes() print(f"{meshes_removed} meshes removed in {time()-t0}") # just create and remove meshes # meshes created in 0.6474745273590088 # meshes removed in 2.770238161087036 create_remove_meshes() # create, use and remove meshes # meshes created in 0.8088419437408447 # meshes removed in 23.767765045166016 # NOTE: you can also set skip_removal = True # and try to remove orphan meshes from outliner manually # (using `bpy.ops.outliner.id_operation(type="DELETE")`) # and it also will be much faster create_use_remove_meshes(skip_removal=False) ```

Andrej added the

labels 2024-02-27 08:12:57 +01:00

Thomas Barlow commented

2024-02-27 13:05:14 +01:00

The problem here is likely bpy.data.meshes.remove(mesh) because its default arguments result in bpy.data.meshes.remove(mesh, do_unlink=True, do_id_user=True, do_ui_user=True), which presumably checks all 6500 Objects for being users of mesh on each call (plus UI and more).

In this script's case, it's known that number of users of the mesh will be zero, so the mesh can be deleted with bpy.data.meshes.remove(mesh, do_unlink=False, do_id_user=False, do_ui_user=False)

If the number of users is accurate, bpy.data.meshes.remove() could probably be updated check the number of users first and then skip the extra work when the number of users is zero. The following does the check manually:

def remove_meshes():
    i = 0
    for mesh in list(bpy.data.meshes):
        if not mesh.name.startswith("test"):
            continue
        if mesh.users == 0:
            bpy.data.meshes.remove(mesh, do_unlink=False, do_id_user=False, do_ui_user=False)
        else:
            bpy.data.meshes.remove(mesh)
        i += 1
    return i

For removal of multiple IDs at once, there is also bpy.data.batch_remove(iterable), though note that it is considered experimental and less safe:

def remove_meshes():
    meshes = [mesh for mesh in bpy.data.meshes if mesh.name.startswith("test")]
    bpy.data.batch_remove(meshes)
    return len(meshes)

Original	check `mesh.users == 0` as above	bpy.data.batch_remove()

The problem here is likely `bpy.data.meshes.remove(mesh)` because its default arguments result in `bpy.data.meshes.remove(mesh, do_unlink=True, do_id_user=True, do_ui_user=True)`, which presumably checks all 6500 Objects for being users of `mesh` on each call (plus UI and more). In this script's case, it's known that number of users of the mesh will be zero, so the mesh can be deleted with `bpy.data.meshes.remove(mesh, do_unlink=False, do_id_user=False, do_ui_user=False)` If the number of users is accurate, `bpy.data.meshes.remove()` could probably be updated check the number of users first and then skip the extra work when the number of users is zero. The following does the check manually: ```py def remove_meshes(): i = 0 for mesh in list(bpy.data.meshes): if not mesh.name.startswith("test"): continue if mesh.users == 0: bpy.data.meshes.remove(mesh, do_unlink=False, do_id_user=False, do_ui_user=False) else: bpy.data.meshes.remove(mesh) i += 1 return i ``` For removal of multiple IDs at once, there is also `bpy.data.batch_remove(iterable)`, though note that it is considered experimental and less safe: ```py def remove_meshes(): meshes = [mesh for mesh in bpy.data.meshes if mesh.name.startswith("test")] bpy.data.batch_remove(meshes) return len(meshes) ``` | Original | check `mesh.users == 0` as above | bpy.data.batch_remove() | | --- | --- | --- | | ![image](/attachments/77307abc-460b-4e82-a44c-c7725f02a50f) | ![image](/attachments/d21a6a02-5e4a-40c4-bd26-ad17aba13bfb) | ![image](/attachments/ebe81770-eab7-4af1-a291-93daec7999a7) |

image.png

9.4 KiB

image.png

9.4 KiB

image.png

9.4 KiB

Germano Cavalcante commented

2024-02-27 19:33:30 +01:00

@Mysteryem, thank you for the detailed explanation and the suggested solution to improve the performance of @Andrej730 's script.
It will certainly be helpful for anyone encountering similar issues ;)

@Andrej730, while we appreciate your effort in optimizing performance, potential performance improvements are not handled as bug reports. We continuously work on enhancing Blender's performance, but such optimizations are typically addressed through coding best practices and workflow enhancements.

So closing this report as it pertains to a potential performance improvement.

@Mysteryem, thank you for the detailed explanation and the suggested solution to improve the performance of @Andrej730 's script. It will certainly be helpful for anyone encountering similar issues ;) @Andrej730, while we appreciate your effort in optimizing performance, potential performance improvements are not handled as bug reports. We continuously work on enhancing Blender's performance, but such optimizations are typically addressed through coding best practices and workflow enhancements. So closing this report as it pertains to a potential performance improvement.

Germano Cavalcante closed this issue

2024-02-27 19:33:33 +01:00

Blender Bot added

and removed

labels 2024-02-27 19:33:35 +01:00

Andrej commented

2024-02-29 10:11:26 +01:00

Thank you for workaround. Settings do_unlink to False indeed speeds things up. Didn't know about bpy.data.batch_remove - it should even work with different types of IDs.

Didn't planned this as a performance issue report - just thought that removing meshes from same conditions but with very decreased performance may indicate some deeper issue.

Thank you for workaround. Settings `do_unlink` to `False` indeed speeds things up. Didn't know about `bpy.data.batch_remove` - it should even work with different types of IDs. Didn't planned this as a performance issue report - just thought that removing meshes from same conditions but with very decreased performance may indicate some deeper issue.

Sign in to join this conversation.

No Label

Download

What's New

Blender Studio

Manual

Developers Blog

Documentation

Benchmark

Blender Conference

Development Fund

One-time Donations

removing orphan meshes is slow (after they were in use) #118787