Shape keys get out of sync with Basis shape (when using UNDO in basis modification or manipulating basis via python) #44415
Labels
No Label
Interest
Alembic
Interest
Animation & Rigging
Interest
Asset Browser
Interest
Asset Browser Project Overview
Interest
Audio
Interest
Automated Testing
Interest
Blender Asset Bundle
Interest
BlendFile
Interest
Collada
Interest
Compatibility
Interest
Compositing
Interest
Core
Interest
Cycles
Interest
Dependency Graph
Interest
Development Management
Interest
EEVEE
Interest
EEVEE & Viewport
Interest
Freestyle
Interest
Geometry Nodes
Interest
Grease Pencil
Interest
ID Management
Interest
Images & Movies
Interest
Import Export
Interest
Line Art
Interest
Masking
Interest
Metal
Interest
Modeling
Interest
Modifiers
Interest
Motion Tracking
Interest
Nodes & Physics
Interest
OpenGL
Interest
Overlay
Interest
Overrides
Interest
Performance
Interest
Physics
Interest
Pipeline, Assets & IO
Interest
Platforms, Builds & Tests
Interest
Python API
Interest
Render & Cycles
Interest
Render Pipeline
Interest
Sculpt, Paint & Texture
Interest
Text Editor
Interest
Translations
Interest
Triaging
Interest
Undo
Interest
USD
Interest
User Interface
Interest
UV Editing
Interest
VFX & Video
Interest
Video Sequencer
Interest
Virtual Reality
Interest
Vulkan
Interest
Wayland
Interest
Workbench
Interest: X11
Legacy
Blender 2.8 Project
Legacy
Milestone 1: Basic, Local Asset Browser
Legacy
OpenGL Error
Meta
Good First Issue
Meta
Papercut
Meta
Retrospective
Meta
Security
Module
Animation & Rigging
Module
Core
Module
Development Management
Module
EEVEE & Viewport
Module
Grease Pencil
Module
Modeling
Module
Nodes & Physics
Module
Pipeline, Assets & IO
Module
Platforms, Builds & Tests
Module
Python API
Module
Render & Cycles
Module
Sculpt, Paint & Texture
Module
Triaging
Module
User Interface
Module
VFX & Video
Platform
FreeBSD
Platform
Linux
Platform
macOS
Platform
Windows
Priority
High
Priority
Low
Priority
Normal
Priority
Unbreak Now!
Status
Archived
Status
Confirmed
Status
Duplicate
Status
Needs Info from Developers
Status
Needs Information from User
Status
Needs Triage
Status
Resolved
Type
Bug
Type
Design
Type
Known Issue
Type
Patch
Type
Report
Type
To Do
No Milestone
No project
No Assignees
17 Participants
Notifications
Due Date
No due date set.
Dependencies
No dependencies set.
Reference: blender/blender#44415
Loading…
Reference in New Issue
No description provided.
Delete Branch "%!s(<nil>)"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
System Information
Operating system: Windows-10-10.0.19042-SP0 64 Bits
Graphics card: NVIDIA GeForce RTX 2070/PCIe/SSE2 NVIDIA Corporation 4.5.0 NVIDIA 472.12
Blender Version
Broken: version: 3.0.1 Release Candidate, branch: master, commit date: 2022-01-17 12:07, hash:
9600f36cfcWorked: v2.72
Caused by
dab0bd9de6Short description of error
After editing the basis shapekey in edit mode using ctrl+z, shape keys using this as a basis behave incorrectly.
Exact steps for others to reproduce the error
In the last project, I often edit the mesh that I previously created shape keys for. And there is a very critical bug that just breaks all the progress of my work with the model.
If you edit the "Basic" shape key in edit mode, press ctrl+z and exit edit mode, then all shape keys belonging to the model will be broken.
Blend From Shapeon the basis shapekey is not working after UNDO then, see #95747I recorded two short videos for you and attached them .blend file.
Video "TRUE_CORRECT.mp4" everything works correctly if you do not press ctrl+z. This is exactly what I expect from a blender.
Video "BUG_INCORRECT.mp4" it doesn't work correctly, after exiting edit mode I get broken shape keys.
TRUE_CORRECT.mp4
BUG_INCORRECT.mp4
BUG_200122.blend
Original report (manipulating the basis shapekey via script
basis_shape_bug.blend
Hi! I've attached a file where you run a script that copies the vertex coordinates from a second mesh into the Basis shape key of the object.
The copy operation works well, the thing is that the the existing shapekeys don't take into account the changes of the Basis shape.
You may find that perhaps I'm missing something in the script, but I'm reporting this cause I've seen this same behavior in my everyday modelling workflow, only that I could never recreate the bug. Sometimes when going in and out Edit mode or Sculpt mode, I realize that the existing shapekeys have gone out of sync with the changes I was doing to the Basis shape... It's like if the changes of the Basis shape didn't propagate to the rest of the shapes.
So please, give a little check to the shape keys code, perhaps you'll find out why this could happen.
Changed status to: 'Open'
Added subscriber: @jpbouza-4
#95747 was marked as duplicate of this issue
#88361 was marked as duplicate of this issue
#95079 was marked as duplicate of this issue
#48609 was marked as duplicate of this issue
Added subscriber: @mont29
Grumph… Something strange is going on here… Thing is, in theory, you do not edit Basis (i.e. first relative skey), it’s supposed to be exact same as actual mesh. But all this is a bit flacky :|
Also, got a nice crash while playing with your file, will check that first…
Hi Bastien!
Thanks for looking into this!
Yes, I also thought that what should be edited was the actual mesh and not the basis. Therefore I also did the test to transfer the vertex locations to the mesh vertices (vert.co), but the result was the same regarding the propagation to the other shape keys.
Added subscriber: @ideasman42
Ok… So here is the issue. 'relative' shape keys are actually fake-relative, in that they do not really build on top of their basis. Rather, they all store absolute vertex coordinates. Then, when you edit a skey and leave Edit mode, Blender will search for all keys using edited one as basis, and apply to them the offset resulting from that basis’ edition.
That very simple and robust process (sarcastic) is not done when setting values from RNA - and for now I do not see how it could be, not without a huge overhead at least…
Let’s see what Campbell says here.
Oh, that's interesting!
Well, about my script then, I could just ask Campbell how to do that propagation manually, or perhaps I should copy the vertex locations within edit mode?
The other issue here is that many times I see this exact same problem, but when toggling between Edit mode, Sculpt and Object mode. I can't really tell when it happens, cause it's just a random thing (random for user, hehe). So, my guess is that there must be some kind of flaw or exception in the propagation process, maybe some checks missing or something, cause this really happens.
The other week while modelling the sheep flock for Gooseberry, it happened several times, and I had to throw my changes to trash and begin all over again, cause the changes to the Basis had not propagated to all the other shapes. But really, can't tell when exactly it happens, cause I did some test, doing apparently the same thing, and some times it worked ok, sometimes it didn't.
I must also say that every time this happens, I'm editing an objects that has several modifiers... so perhaps it would be a good idea to check if modifiers could somehow cause the propagation process to not work properly?
OK, I think I've found the bug!
Well, at least for the GUI workflow part, not for the script I submitted.
So, I found the reason why changes to the Basis shape will not always propagate to the rest of the shapes. It's because of UNDO.
Procedure:
_Create a cube
_Add a Basis shape
_Add another shape and set it to 1
_Go to the Basis shape and enter edit mode.
_Move one vertex.
_Move it again and UNDO
_Go to object mode and play with key 1 slider, you'll notice that the vertex movement that was introduced to the Basis shape has not propagated to Key 1. Instead, Key 1 is no bringing back the original shape of the cube.
If yo happen to solve this, perhaps the report could be closed, but then ideasman should give me a hand with the shape propagation in my script!
Added subscriber: @GiantCowFIlms
Added subscribers: @SpectreFirst, @JosephCatrambone
Added subscriber: @Shaggy90
Hi, I just wanted to point out that this bug still exists in 2.79. :( I just lost 30 minutes of work because some edits didn't apply to shape keys while some others did. Apparently using UNDO while editing basis is not safe.
Added subscriber: @AgentA1cr
Just noting that this bug still exists as of 2.93.0-a9e7d503ddcd. I'm still trying to figure out the extent of the damage, or if it's even fixable.
Added subscriber: @georgK
Added subscriber: @KonstantinsVisnevskis
Not sure if related, but I couldn't spot any direct causes, which doesn't satisfy a report of it's own, but Undo is probably involved:
Having 15 Shapekeys driven from animation. When editing them, the changes occasionally get applied quite randomly. For example - edited the Basis and one of OTHER Shapekeys, was Added (as in Blend From Shape -> Add) onto all of the rest of Shape Keys, except the Basis, and got nullified (lost it's own transformations) at the same time.
Another case - I edited the Basis and one of the shapekeys got the inverse transformation of it. Meaning - a single Shapekey (which wasn't even selected at any point) remains the same as it was, despite all of the others now being affected by the Basis change.
These things occur one to multiple times a week - rare enough not to spot the cause, but usually are quite time consuming if possible at all to fix.
Shape keys get out of sync with Basis shapeto Shape keys get out of sync with Basis shape (when using UNDO in basis modification or manipulating basis via python)Added subscribers: @Art4D, @lichtwerk
Added subscribers: @KOBELT, @Reoxur, @iss
This bug is 6 years old, I don't think it will ever be fixed.
Can someone suggest an alternative?
I have a mesh with shape keys, in the base form of which I need to add vertexes, as well as edit the current ones. Everything works until I press ctrl+z out of habit. After clicking, I have to do useless work on correcting morphs for several hours, or roll back the file for several hours.
Any ideas?
@Art4D Same here. Just today I spent over an hour fixing shapekeys. And 2h 2 days ago, and 3h over the last week. Over the month that totals to about a full working day.
The fix for #35170 causes this, see:
dab0bd9de6(if that change is disabled, this bug no longer happens).
Although I checked and this fix is still important as the original bug is triggered if it's reverted.
This needs some deeper investigation.
Added subscriber: @Mysteryem-2
Added subscriber: @AdamJanz
This is an extremely severe bug, and there is no workaround whatsoever. Any application of Undo in the Basis shapekey, even just ONE press of Undo, will render ALL changes made to the Basis shapekey during that editing session UNTRANSFERABLE to existing shapekeys. I just tested several combinations of Vertex > Blend from Shape and Propagate from Shape, and there is NO combination that can transfer your changes in the Basis key correctly. Unfortunately, this bug renders Blender's shapekey system completely unusable.
Added subscriber: @dr.sybren
I also think this is pretty bad and will add #animation_rigging (since riggers are most affected I think) for more opinions.
CC @dr.sybren
@lichtwerk Thank you, Philipp! I am indeed surprised this bug has managed to survive untouched for 7 years, but perhaps it went overlooked because editing the Basis shapekey is thought to be taboo by some? However, it's truly a vital feature during character development to be able to freely iterate and update the Basis shapekey at will. I consider this a high-priority fix because of the potential to literally destroy hours of work (and hundreds of individual actions) with a single lethal button press. Only actions done AFTER pressing Undo will be successfully propagated. If one has unwittingly saved their .blend file, and their Undo buffer is not set large enough, their entire project may be irreversibly damaged. I trust the talented Blender devs will look into this and find a great solution...
Added subscriber: @BClark
I hit this early on learning Blender but didn't know it was a bug, I thought I didn't know how to use Blender enough yet!!!
Added subscriber: @rwman
@AdamJanz to reduce the risk of such losses I would recommend some versioning system (Blender Animation Studio uses SVN, but other systems can be used of course). Putting the weight of having one bug in Blender destroy an entire production on the shoulders of Blender developers is not fair.
I agree that this is something to fix, though.
Does this mean you're looking into it @ideasman42 ?
I'll dare changing it to
Bugthen.Added subscriber: @lictex_1
Added subscriber: @Yerus_Ovelha
Thank you, everyone, for looking into the issue. I do save often with incremental file numbering so that can help should something go wrong. Of course, just being aware of the issue while I'm editing the Basis key is the best deterrent at accidental data loss. However, other users who encounter the issue for the first time would not be so fortunate.
(Perhaps related to this issue is a bug that manifests after creating and editing a shapekey and then moving it above the Basis to become the new "Basis". As a result, any changes to the old Basis shapekey (which is no longer the Basis)will now incorrectly affectALLkeys, including the NEW Basis key. Perhaps the underlying cause is related to Blender storing "absolute vertex coordinates" as @mont29 had mentioned above. Thankfully, this issue is not nearly as dangerous as #44415, and can be avoided entirely by simply duplicating the Basis key and then deleting the original Basis rather than re-ordering the keys. Still, the issue isvery "bug-like" and might surprise new users who don't know it's taboo to reorder a shapekey above the Basis.) I couldn't find this issue already reported, so if you wish, I can open a new one-- but perhaps the eventual fix for #44415 will solve it also? Again, thanks so much to the devs for all your amazing work!
What you're describing is the correct behaviour in my opinion. When you edit the Basis shapekey, it doesn't propagate changes to all the other shapekeys because it is the Basis shapekey, it propagates changes to all the other shapekeys because all the other shapekeys are (usually)
Relative Tothe Basis shapekey. If you were to create a new shapekey and set itsRelative Toto itself or any other shapekey other than the Basis shapekey, making changes to the Basis shapekey would not affect that new shapekey. This behaviour can be confusing however, since noRelative Tofield is shown on the UI for the Basis shapekey, yet it is still affected by what it's set to. #79892 probably covers this issue already.Thanks Mysteryem. #79892 is indeed the exact bug I was referring to. Good find. Your explanation in that thread also makes sense, and I agree that either the new Basis should be forced to be relative to itself or allow the UI to display which key the Basis is relative to. This will help avoid majorly confusing scenarios. Glad the devs are already aware of this issue.
This looks incredible, @ideasman42 !!! Thank you so much! I can't wait to try this fix out once it makes its way into one of the daily builds :-)
Hi @AdamJanz, @jpbouza-4 and others interested in testing, here are builds of D14127, feedback would be appreciated.
Amazing--- downloading the Windows build now, will test immediately and update you. Many thanks, @ideasman42 !!
Hi @ideasman42 , firstly let me say this fantastic and does indeed solve the issue with using Undo on the Basis key!
After stress-testing though, I did discover one caveat: the fix works perfectly fine providing the vertex count has not changed when the Basis key is active. If, however, loop cuts are added to the mesh while the Basis key is active, and multiple changes are made to the Basis and Undo is pressed at least once, then the old behavior resumes and the changes made to the Basis will not propagate to the other keys. Nevertheless, if loop cuts are added while any key OTHER than the Basis is active, then the fix again works perfectly fine. I made a short Unlisted video showing both scenarios: EXAMPLE VIDEO
Example1: I add loop cuts while the Basis key is active, then edit the Basis key twice and Undo the last action (the old behavior resumes, and prevents changes from propagating).
Example2: I add loop cuts while Key1 is active, then edit the Basis key twice and Undo the last action (here your fix allows changes to correctly propagate).
I hope this info helps. You are super, super close to a permanent solution!! Again, thank you for your incredible work on this!
Additional info-- I just discovered a third scenario:
Example 3: I add loop cuts while the Basis key is active, then EXIT Edit mode. I then re-enter Edit mode, edit the Basis key twice and Undo the last action (here your fix allows changes to correctly propagate).
This scenario works the same as Example 2. The only difference is exiting Edit mode is required before any changes can be made to the Basis.
Video of Example 3
Thanks for your feedback @AdamJanz (note, it would be better to have this discussion in D14127).
Firstly, can you confirm the issues you've mentioned are not bugs in functionality that previously worked?
If they are - D14127 may need to be re-thought.
If they are not, the issues you mention can be handled as separate reports since I rather not introduce multiple fixes in a single diff as it complicates review and any problems discovered later are more difficult to track down.
Details: There is a limitation with the current logic that means adding new vertices causes the basis key not to update other keys, the changes from D14127 allow the limitation be removed but I'd need to double check the result (from some initial tests P2803 may be acceptable).
You're welcome @ideasman42 ! Sure, I can switch further discussion over to D14127, but I didn't think non-devs were supposed to comment in there. :-)
Since I discovered the issues during testing I initially assumed they were related, however, I can now confirm that the issues described already existed in Blender and are NOT recent regressions. I tested your 3.2 D14127 build, and the official 3.0.1 and 2.93.2 builds, and even the now-ancient 2.79b, and all 3 Examples I gave previously are reproducible even if the Undo command is NOT invoked.
So that means the issue involving Undo has been solved-- and in the process we happened to stumble across a vintage bug and its 2 workarounds:
If you wish, I can open a separate bug report for this, unless you feel the changes you made in D14127 will allow for a combined fix.
Thanks again for your incredible work!
This issue was referenced by
bfdbc78466Changed status from 'Confirmed' to: 'Resolved'
@AdamJanz thanks for the detailed feedback.
Yes, please report a bug regarding vertices being added causing the basis key not to apply offsets to any other shapes. Even though removing this limitation is on my radar, it would be good to have a test-case and report for reference.
You're welcome, @ideasman42 ! I should be able to file a report within the next few days. Have a great week. :-)