Recursive Purge Orphans datablocks on file save #87490

New Issue

Demeter Dzadik · 2021-04-14T12:46:39+02:00

Demeter Dzadik commented

2021-04-14 12:46:39 +02:00

Fully deleting datablocks from a blend file doesn't happen as easily as most users might expect, since "delete" is often used in the UI when "un-link" is what is actually meant. This can cause a lot of confusion and uncertainty during production. Proper deletion of datablocks requires that they are first un-linked, then they can be purged via File->Clean Up->Recursive Unused Datablocks. This works beautifully, and I would like to propose adding an option to have this operation executed after existing deletion operators. For example, after deleting a collection of objects, it would then also delete everything that was used only by those objects, such as their mesh and shape key datablocks, materials, actions, textures, etc.

My previous proposal was to have an option to execute Recursive Purge only on file save, which seemed to make sense to everyone I talked to at the time. (@dr.sybren @mont29 @JulianEisel @SimonThommes )
I still think that solution would be a step forward, but the new proposal above would achieve the same goals in an even more foolproof way.

Use case

Currently, this can very easily happen:

Animator duplicates an overridden character. All its objects get a .001 suffix.
Animator deletes that copy, saves their file, and later re-opens it. Only one "layer" of unused objects has been cleaned, so there can still be orphan object datablocks lying around unseen.
Animator duplicates the character again. Now the number suffix of their objects are all over the place. Some of them have .001 and others have .002.

As they continue working and duplicating characters, the old, unused objects slowly get deleted, and the name suffixes really get all over the place. It's not nice to get your objects names all mixed up like this, as things get really confusing when trying to troubleshoot problems in a production file, since everywhere where one object references another, the mismatching suffixes give the troubleshooter anxiety and uncertainty. The only way to avoid this currently is to tell the entire production team to always run a recursive Purge after deleting anything, which is very error-prone.

Potential issues

~~The current recursive purge purges Text datablocks, which is bad because they don't have the possibility of a "Fake User", so they should be excluded (or better yet, be included in the "fake user" system, and get a fake user by default). ~~ (Fixed by D10983)
Actions don't have fake user enabled by default. The long standing issue of Action datablocks "disappearing" on users and making them lose hours of work if they forget to click Fake User would be exacerbated by this change. This could be trivially fixed by giving Actions a fake user on creation, and forcing users to clean up their unused Actions themselves.

Fully deleting datablocks from a blend file doesn't happen as easily as most users might expect, since "delete" is often used in the UI when "un-link" is what is actually meant. This can cause a lot of confusion and uncertainty during production. Proper deletion of datablocks requires that they are first un-linked, then they can be purged via File->Clean Up->Recursive Unused Datablocks. This works beautifully, and I would like to propose adding an option to have this operation executed after existing deletion operators. For example, after deleting a collection of objects, it would then also delete everything that was used only by those objects, such as their mesh and shape key datablocks, materials, actions, textures, etc. My previous proposal was to have an option to execute Recursive Purge only on file save, which seemed to make sense to everyone I talked to at the time. (@dr.sybren @mont29 @JulianEisel @SimonThommes ) I still think that solution would be a step forward, but the new proposal above would achieve the same goals in an even more foolproof way. ### Use case Currently, this can very easily happen: - Animator duplicates an overridden character. All its objects get a .001 suffix. - Animator deletes that copy, saves their file, and later re-opens it. Only one "layer" of unused objects has been cleaned, so there can still be orphan object datablocks lying around unseen. - Animator duplicates the character again. Now the number suffix of their objects are all over the place. Some of them have .001 and others have .002. As they continue working and duplicating characters, the old, unused objects slowly get deleted, and the name suffixes really get all over the place. It's not nice to get your objects names all mixed up like this, as things get really confusing when trying to troubleshoot problems in a production file, since everywhere where one object references another, the mismatching suffixes give the troubleshooter anxiety and uncertainty. The only way to avoid this currently is to tell the entire production team to always run a recursive Purge after deleting anything, which is very error-prone. ### Potential issues - ~~The current recursive purge purges Text datablocks, which is bad because they don't have the possibility of a "Fake User", so they should be excluded (or better yet, be included in the "fake user" system, and get a fake user by default). ~~ (Fixed by [D10983](https://archive.blender.org/developer/D10983)) - Actions don't have fake user enabled by default. The long standing issue of Action datablocks "disappearing" on users and making them lose hours of work if they forget to click Fake User would be exacerbated by this change. This could be trivially fixed by giving Actions a fake user on creation, and forcing users to clean up their unused Actions themselves.

Demeter Dzadik commented

2021-04-14 12:46:39 +02:00

Added subscribers: @SimonThommes, @JulianEisel, @mont29, @dr.sybren, @Mets

Demeter Dzadik commented

2021-04-14 13:11:16 +02:00

Marking as "Good first issue" on Julian's suggestion, if any contributors would like to pick this up, it's probably(hopefully) simple on the implementation side.

Bastien Montagne commented

2021-04-14 13:29:02 +02:00

Also related to #61209 (Always write unused IDs on save)

Demeter Dzadik commented

2021-04-14 14:52:33 +02:00

Whoops, so much for "Good first issue"! I forgot about that task, thanks for the reminder. Maybe this task is sort of redundant then, since the design proposed in #61209 solves the same issues, and perhaps more elegantly. It's just a shame since that seems like a pretty daunting task that probably won't be tackled for some time, so until then I am left with shoving a "Purge All" button in the animation team's face and hope that they press it often. :/

With that design in mind, I think what I'm asking for here could manifest as a "Purge on Save" user preference, that would bring back the old(current) behaviour except with a recursive purge. (Performance be damned.) Alternatively, it would be interesting to explore more ways to allow datablocks to not have to share a name space. Right now, linked and local datablocks are in separate name spaces, which I think is great. If we could have a similar thing for deleted databloks, I think that could be interesting, and also make sense from user PoV.

@mont29 Do you think it's better to close this task and add my thoughts to the discussion in the other thread, or is it worthwhile to keep this separate?

Whoops, so much for "Good first issue"! I forgot about that task, thanks for the reminder. Maybe this task is sort of redundant then, since the design proposed in #61209 solves the same issues, and perhaps more elegantly. It's just a shame since that seems like a pretty daunting task that probably won't be tackled for some time, so until then I am left with shoving a "Purge All" button in the animation team's face and hope that they press it often. :/ With that design in mind, I think what I'm asking for here could manifest as a "Purge on Save" user preference, that would bring back the old(current) behaviour except with a recursive purge. (Performance be damned.) Alternatively, it would be interesting to explore more ways to allow datablocks to not have to share a name space. Right now, linked and local datablocks are in separate name spaces, which I think is great. If we could have a similar thing for deleted databloks, I think that could be interesting, and also make sense from user PoV. @mont29 Do you think it's better to close this task and add my thoughts to the discussion in the other thread, or is it worthwhile to keep this separate?

Bastien Montagne commented

2021-04-14 17:09:34 +02:00

@Mets would keep both for now, while related they are not entirely the same. We can re-iterate on designs and such once we actually start working on those.

Chris Kohl commented

2022-01-24 06:39:25 +01:00

Added subscriber: @ckohl_art

RedMser commented

2022-01-30 11:15:06 +01:00

Added subscriber: @RedMser

Philipp Oeser commented

2022-08-19 12:36:15 +02:00

Added subscriber: @lichtwerk

Philipp Oeser commented

2022-08-19 12:36:15 +02:00

Changed status from 'Needs Triage' to: 'Confirmed'

Philipp Oeser commented

2022-08-19 12:36:15 +02:00

In #87490#1146406, @mont29 wrote:
@Mets would keep both for now, while related they are not entirely the same. We can re-iterate on designs and such once we actually start working on those.

Would assume this can be confirmed as a status then (to get it out of the triaging queue).

> In #87490#1146406, @mont29 wrote: > @Mets would keep both for now, while related they are not entirely the same. We can re-iterate on designs and such once we actually start working on those. Would assume this can be confirmed as a status then (to get it out of the triaging queue).

Philipp Oeser removed the

Download

What's New

Blender Studio

Manual

Developers Blog

Documentation

Benchmark

Blender Conference

Development Fund

One-time Donations

Recursive Purge Orphans datablocks on file save #87490

Use case

Potential issues