Fix undo overreach in various paint modes for #69760 and related. #117417

Alexander Gavrilov · 2024-01-22T17:32:45+01:00

2024-01-22 17:32:45 +01:00

This tries to fix undo overreach on selection of vertex groups,
shape keys, uv maps, colors and attributes in various modes.

The Vertex Paint mode is now a subtype of Sculpt, so it should
use a sculpt undo step on entering the mode, and property changes
shouldn't write undo steps.
Sculpt undo modes were deliberately over-undoing color attribute
selection changes for some reason.
The Texture Paint mode always writes texture steps, so saving
steps for property changes is simply pointless.
The Weight Paint mode uses memfile, so it should save steps
for property changes, except for those excluded from undo,
e.g. by bc0a6b0400.
If a memfile undo step happens anyway for some reason in sculpt
and vertex paint modes, immediately following property changes
to object and mesh datablocks should also push to avoid over-undo
if the immediately following scuplt step is reverted (unrelated
ones like materials or scene seem safe).

This fix addresses these cases:

Prepare	Change (should push undo)	Change (should not push undo)	Test (should not undo the change)
Enter Weight Paint Mode	Object viewport visibility, Collection viewport visibility, Active vertex group. [1]	Brush settings, Tool settings (e.g. Auto-Normalize)	Paint single stroke, Undo.
Enter Sculpt Mode		Active vertex group, Active shape key, Active vertex color, Object viewport visibility, Collection viewport visibility, Brush/tool settings.	Sculpt single stroke, Undo.[3]
Enter Sculpt Mode, Use a non-sculpt operator (e.g. Mirror Shape Key)	Active vertex group, Active shape key, Active vertex color.[2]	Object viewport visibility, Collection viewport visibility, Brush/tool settings.	Sculpt single stroke, Undo.
Enter Vertex Paint Mode		Active vertex color[4]	Paint single stroke, Undo.[3]
Enter Vertex Paint Mode, Use a non-sculpt operator (e.g. Mirror Shape Key)	Active vertex color.[2]		Paint single stroke, Undo.

Weight paint cases are fixed by changes in ed_undo.cc
Sculpt with operator cases are fixed by changes in interface_handler.cc
Active vertex color in Sculpt and Vertex Paint is fixed by the removal in sculpt_undo.cc
Without changes in paint_vertex.cc, the action of entering Vertex Paint Mode itself counts as a 'non-sculpt operator'.

Note: internally Sculpt and Vertex Paint are essentially the same and only differ in the shown set of tools.

This tries to fix undo overreach on selection of vertex groups, shape keys, uv maps, colors and attributes in various modes. - The Vertex Paint mode is now a subtype of Sculpt, so it should use a sculpt undo step on entering the mode, and property changes shouldn't write undo steps. - Sculpt undo modes were deliberately over-undoing color attribute selection changes for some reason. - The Texture Paint mode always writes texture steps, so saving steps for property changes is simply pointless. - The Weight Paint mode uses memfile, so it should save steps for property changes, except for those excluded from undo, e.g. by bc0a6b0400. - If a memfile undo step happens anyway for some reason in sculpt and vertex paint modes, immediately following property changes to object and mesh datablocks should also push to avoid over-undo if the immediately following scuplt step is reverted (unrelated ones like materials or scene seem safe). ------------------ This fix addresses these cases: | Prepare | Change (should push undo) | Change (should not push undo) | Test (should not undo the change) | | :---: | :---: | :---: | :---: | | Enter Weight Paint Mode | _Object viewport visibility, Collection viewport visibility, Active vertex group. [1]_ | Brush settings, Tool settings (e.g. Auto-Normalize) | Paint single stroke, Undo. | | Enter Sculpt Mode | | Active vertex group, Active shape key, Active vertex color, Object viewport visibility, Collection viewport visibility, Brush/tool settings. | _Sculpt single stroke, Undo.[3]_ | | Enter Sculpt Mode, Use a non-sculpt operator (e.g. Mirror Shape Key) | _Active vertex group, Active shape key, Active vertex color.[2]_ | Object viewport visibility, Collection viewport visibility, Brush/tool settings. | Sculpt single stroke, Undo. | | Enter Vertex Paint Mode | | _Active vertex color[4]_ | _Paint single stroke, Undo.[3]_ | | Enter Vertex Paint Mode, Use a non-sculpt operator (e.g. Mirror Shape Key) | _Active vertex color.[2]_ | | Paint single stroke, Undo. | 1. Weight paint cases are fixed by changes in ed_undo.cc 2. Sculpt with operator cases are fixed by changes in interface_handler.cc 3. Active vertex color in Sculpt and Vertex Paint is fixed by the removal in sculpt_undo.cc 4. Without changes in paint_vertex.cc, the action of entering Vertex Paint Mode itself counts as a 'non-sculpt operator'. Note: internally Sculpt and Vertex Paint are essentially the same and only differ in the shown set of tools.

Alexander Gavrilov added the

 @ -837,2 +837,4 @@
     ED_object_vpaintmode_enter_ex(bmain, depsgraph, scene, ob);
     BKE_paint_toolslots_brush_validate(bmain, &ts->vpaint->paint);
     if (ob->mode & mode_flag) {

 @ -839,0 +839,4 @@
     if (ob->mode & mode_flag) {
       /* Without this the memfile undo step is used,
        * while it works it causes lag when undoing the first undo step, see T71564. */

 @ -462,0 +483,4 @@
          *
          * Most other property change undo pushes will be also suppressed
          * by another check later unless preceeded by a memfile push;
          * however these properties should always be blocked. */

Download

What's New

Blender Studio

Manual

Developers Blog

Documentation

Benchmark

Blender Conference

Development Fund

One-time Donations

Fix undo overreach in various paint modes for #69760 and related. #117417

Checkout