NLA: Fix Dragging Strips #82241

New Issue

Wayde Moss · 2020-10-30T01:26:46+01:00

Wayde Moss commented

2020-10-30 01:26:46 +01:00

NOTE: test the patches for Solution1 https://builder.blender.org/download/temp-D10103-nla_support_strip_overlap_during_transform/temp-D10103-nla_support_strip_overlap_during_transform-blender-2.92.0-5065c0e6d5bc-windows64.zip

This is no longer true, we would rather remove them and improve the interaction with HOLD type Update design since transition strips should be kept.
Todo: Add concrete examples where transitions are useful in the ideal case (where strips animate through the transition).

Report: #82195 (NLA tracks won't pass through actions or locked tracks)

Summary: The ideal solution is to get rid of transition strip types. They're useless. Let strips overlap. When they do, they get blended in order like an upper strip. Then we don't have to restrict movement.

Behavior can be found in: (transform_convert_nla.c) recalcData_nla()
Problem: The core intent of the function is to prevent moved strips from overlapping other strips. The restriction is reasonable since NLA evaluation is undefined in this case. However, the implementation prevents the ability to move a strip to its destination even if it wouldn't result in overlap.

When moving strips:

You cannot move a strip through a locked track, even if not the intended final destination.
You cannot move a strip past another in the same track, even if the intended final destination would not result in overlap.
You cannot move a strip into through or into a track if doing so results in overlap.
Vertically moving strips, when there are no more nla tracks, leads to strips being vertically compressed. This breaks the animation when strip extrapolation is important.

Solution1: (Simpler solution) We only apply the restriction on the final strip location. If the strip cannot be placed without overlap, then we either cancel the move operation (not ideal) or notify the user (show the strip as red?) without cancelling.

D10101: Cleanup: Nla: Refactor nla.c functions (independent patch) Does a little bit of refactoring which D10103 relies on.
D10102: NLA: Strip Post-transform Horizontal Shuffle (independent patch) Adds support for horizontal shuffling so users can horizontally drag strips past eachother.
D10103: NLA: Strip Post-transform Vertical Shuffle and Auto-Grow Track List (depends on prior patches) Adds support for vertical shuffling so users can vertically drag strips into a new track even if it results in overlap or the user moves the strip into a locked track.

Patches have been made for this solution since it's simpler and faster to implement. Overlap support can still be worked on but is a lower priority and still has design problems to figure out.

Solution2: (Ideal solution, but would take longer to implement) We change NLA evaluation to work properly when strips overlap. Then we can completely remove the restriction and let the animator freely move strips.

Patch1: Completely remove transition types
We can simplify NLA evaluation here a bit. NLA transitions are problematic and ineffective. Transition strip types should be completely removed. They do a lerp between the previous strip's last pose and next strip's first pose. Generally in animation, lerping 2 poses is a problem you fix, not something you ever want. Support-wise, transitions are a bit broken anyways (see D9711 and D9713)

Patch2: Implement new behavior when strips overlap
Instead of an explicit transition strip type, we'll let action and meta strips support a new transition behavior. When two strips overlap, we simply blend the next strip as if it was an upper strip. That's the core idea. "Transitioning" strips of the same track would used for convenience and organization. This would simplify the keyframe remapping math. In D8296, I couldn't figure out the math to properly invert through transitions between a Quaternion Combine strip and a Quaternion Replace (or any non-Combine) strip. With the new behavior, keyframing through transitions is simplified to inverting through any other strip.

Patch3: Re-add support for holding poses through the transition
Though, as is, both strips are always animated through the transition. We don't have the ability to hold one strip's boundary pose while the other animates through the transition. I propose a few new strip properties: hold_left_duration (unit: scene frame), hold_right_duration. The influence over these regions can be automatically calculated when strips overlap. Or they can be keyed into shape using the existing influence fcurve. As an example of these new properties, we'll have two strips that only overlap over this held duration.
Note. I'm having second thoughts on the properties. Yes, there should be support for holding poses through the transition. But the problem is how these properties are anchored relative to the strip bounds and what's most convenient. I want to avoid problems like the ones we already have with nla strip frame_start/end being tightly related to the action clips frame_start/end, scale and repeat values. This leads to unintuitive strip frame_start/end changes.

Patch4: Finally loosen strip dragging logic (only relies on patch2)
We can completely remove the dragging restriction and let the animator freely move and overlap strips.

(Pretend the two strips are in the same row/track. I did not draw them that way because I'm not sure how the UI should look. This also isn't a suggestion for how the UI should render)

This results in nearlythe same result as old Transition clips. Only nearly, because the old implementation would first blend the prev strip with the lower NLA stack, then blend the next strip with the lower NLA stack, then lerp the two. The new behavior holds the previous strip's last pose over the light green area. The next strip's first pose is held over the pink area. The influence of the next strip increases linearly.

This holds the previous strip's last pose. The next strip animates as it's blended.

Another possibility.

Problematic UI case: I'm not sure what the proper UI should look like when one strip completely overlaps. The animator can always just move the strip out above as a work around. Logically, it would evaluate fine. It would just be a UI issue. What if N strips overlap? It still all works out. We blend the strips in the order of their start frame. What if all strips are on the same start frame? That case won't be properly supported and considered undefined (as in we won't fix it). The animator can and should fix it themselves.
__

Alternative WorkAround: Add operators that can be used as workarounds.

(RCS)Send strip to the end of the active strip or track, with options to handle overlap properly.

Patches:

NOTE: test the patches for Solution1 https://builder.blender.org/download/temp-[D10103](https://archive.blender.org/developer/D10103)-nla_support_strip_overlap_during_transform/temp-[D10103](https://archive.blender.org/developer/D10103)-nla_support_strip_overlap_during_transform-blender-2.92.0-5065c0e6d5bc-windows64.zip This is no longer true, we would rather remove them and improve the interaction with HOLD type ****Update design since transition strips should be kept.**** Todo: Add concrete examples where transitions are useful in the ideal case (where strips animate through the transition). ___ Report: #82195 (NLA tracks won't pass through actions or locked tracks) **Summary:** The ideal solution is to get rid of transition strip types. They're useless. Let strips overlap. When they do, they get blended in order like an upper strip. Then we don't have to restrict movement. ___ Behavior can be found in: (transform_convert_nla.c) *recalcData_nla()* **Problem**: The core intent of the function is to prevent moved strips from overlapping other strips. The restriction is reasonable since NLA evaluation is undefined in this case. However, the implementation prevents the ability to move a strip to its destination even if it wouldn't result in overlap. When moving strips: * You cannot move a strip through a locked track, even if not the intended final destination. ![locked track.gif](https://archive.blender.org/developer/F9128321/locked_track.gif) * You cannot move a strip past another in the same track, even if the intended final destination would not result in overlap. ![same_track.gif](https://archive.blender.org/developer/F9128316/same_track.gif) * You cannot move a strip into through or into a track if doing so results in overlap. ![corner.gif](https://archive.blender.org/developer/F9128309/corner.gif) * Vertically moving strips, when there are no more nla tracks, leads to strips being vertically compressed. This breaks the animation when strip extrapolation is important. ![D10103_vshuffle_autoGrow_nonpatchEx.gif](https://archive.blender.org/developer/F12959094/D10103_vshuffle_autoGrow_nonpatchEx.gif) ___ **Solution1:** (Simpler solution) We only apply the restriction on the final strip location. If the strip cannot be placed without overlap, then we either cancel the move operation (not ideal) or notify the user (show the strip as red?) without cancelling. [D10101: Cleanup: Nla: Refactor nla.c functions](https://archive.blender.org/developer/D10101) (independent patch) Does a little bit of refactoring which [D10103](https://archive.blender.org/developer/D10103) relies on. [D10102: NLA: Strip Post-transform Horizontal Shuffle](https://archive.blender.org/developer/D10102) (independent patch) Adds support for horizontal shuffling so users can horizontally drag strips past eachother. [D10103: NLA: Strip Post-transform Vertical Shuffle and Auto-Grow Track List](https://archive.blender.org/developer/D10103) (depends on prior patches) Adds support for vertical shuffling so users can vertically drag strips into a new track even if it results in overlap or the user moves the strip into a locked track. Patches have been made for this solution since it's simpler and faster to implement. Overlap support can still be worked on but is a lower priority and still has design problems to figure out. ___ **Solution2:** (Ideal solution, but would take longer to implement) We change NLA evaluation to work properly when strips overlap. Then we can completely remove the restriction and let the animator freely move strips. **Patch1: Completely remove transition types** We can simplify NLA evaluation here a bit. NLA transitions are problematic and ineffective. *Transition strip types should be completely removed.* They do a lerp between the previous strip's last pose and next strip's first pose. Generally in animation, lerping 2 poses is a problem you fix, not something you ever want. Support-wise, transitions are a bit broken anyways (see [D9711](https://archive.blender.org/developer/D9711) and [D9713](https://archive.blender.org/developer/D9713)) **Patch2: Implement new behavior when strips overlap** Instead of an explicit transition strip type, we'll let action and meta strips support a new transition behavior. When two strips overlap, we simply blend the next strip as if it was an upper strip. That's the core idea. "Transitioning" strips of the same track would used for convenience and organization. This would simplify the keyframe remapping math. In [D8296](https://archive.blender.org/developer/D8296), I couldn't figure out the math to properly invert through transitions between a Quaternion Combine strip and a Quaternion Replace (or any non-Combine) strip. With the new behavior, keyframing through transitions is simplified to inverting through any other strip. **Patch3: Re-add support for holding poses through the transition** Though, as is, both strips are always animated through the transition. We don't have the ability to hold one strip's boundary pose while the other animates through the transition. I propose a few new strip properties: `hold_left_duration` (unit: scene frame), `hold_right_duration`. The influence over these regions can be automatically calculated when strips overlap. Or they can be keyed into shape using the existing influence fcurve. As an example of these new properties, we'll have two strips that only overlap over this held duration. *Note. I'm having second thoughts on the properties. Yes, there should be support for holding poses through the transition. But the problem is how these properties are anchored relative to the strip bounds and what's most convenient. I want to avoid problems like the ones we already have with nla strip frame_start/end being tightly related to the action clips frame_start/end, scale and repeat values. This leads to unintuitive strip frame_start/end changes.* **Patch4: Finally loosen strip dragging logic (only relies on patch2)** We can completely remove the dragging restriction and let the animator freely move and overlap strips. ___ ![image.png](https://archive.blender.org/developer/F9129515/image.png) (Pretend the two strips are in the same row/track. I did not draw them that way because I'm not sure how the UI should look. This also isn't a suggestion for how the UI should render) This results in *nearly*the same result as old Transition clips. Only *nearly*, because the old implementation would first blend the prev strip with the lower NLA stack, then blend the next strip with the lower NLA stack, then lerp the two. The new behavior holds the previous strip's last pose over the light green area. The next strip's first pose is held over the pink area. The influence of the next strip increases linearly. ___ ![image.png](https://archive.blender.org/developer/F9129577/image.png) This holds the previous strip's last pose. The next strip animates as it's blended. ___ ![image.png](https://archive.blender.org/developer/F9129638/image.png) Another possibility. ___ ![image.png](https://archive.blender.org/developer/F9129718/image.png) **Problematic UI case**: I'm not sure what the proper UI should look like when one strip completely overlaps. The animator can always just move the strip out above as a work around. Logically, it would evaluate fine. It would just be a UI issue. What if N strips overlap? It still all works out. We blend the strips in the order of their start frame. What if all strips are on the same start frame? That case won't be properly supported and considered undefined (as in we won't fix it). The animator can and should fix it themselves. __ **Alternative WorkAround:** Add operators that can be used as workarounds. * ([RCS](https://blender.community/c/rightclickselect/M9gbbc/))Send strip to the end of the active strip or track, with options to handle overlap properly. ___ Patches: **(Abandoned)Patch set 1: Removal of transitions** *Note: I'm not sure how file versioning should be handled? I haven't removed the strip type NLASTRIP_TYPE_TRANSITION yet. I'm not sure if the enum should remain so it can be used for file versioning to either delete transitions or convert them to the new behavior (after Patch set 3).* [D9702: Nla Transitions Obselete: remove ui icon](https://archive.blender.org/developer/D9702) [D9703: Nla Transitions Obselete: Nla Stack no longer evaluates Transitions](https://archive.blender.org/developer/D9703) [D9704: Nla Transitions Obselete: Remove Operator Add Transition](https://archive.blender.org/developer/D9704) [D9705: Nla Transitions Obselete: Remove themed drawing](https://archive.blender.org/developer/D9705) [D9706: Nla Transitions Obselete: Remove default naming case](https://archive.blender.org/developer/D9706) [D9707: Nla Transitions Obselete: Special case for adjacent strip deletion](https://archive.blender.org/developer/D9707) [D9708: Nla Transitions Obselete: Adjacent strip overlap resolution](https://archive.blender.org/developer/D9708) [D9709: Nla Transitions Obselete: nlastrip_get_frame()](https://archive.blender.org/developer/D9709) [D9710: Nla Transitions Obselete: RNA strip frame_start/end set functions](https://archive.blender.org/developer/D9710) [D9711: Nla Transitions Obselete: Transform, redundant check](https://archive.blender.org/developer/D9711) [D9712: Nla Transitions Obselete: Transform, Cancelling](https://archive.blender.org/developer/D9712) [D9713: Nla Transitions Obselete: Transform, broken strip exceed resolution](https://archive.blender.org/developer/D9713) [D9714: Nla Transitions Obselete: Loose comments](https://archive.blender.org/developer/D9714) **Abandon Reason:** I misunderstood why transitions were limited. Yes, their current implementation only supports single pose transitions. But they would be more useful if the strips animated through the transition. Alexander Gavrilov suggested we keep transitions for their branched evaluation behavior.

Wayde Moss commented

2020-10-30 01:26:46 +01:00

Added subscriber: @wbmoss_dev

blender-admin commented

2020-10-30 01:26:46 +01:00

#85059 was marked as duplicate of this issue

blender-admin commented

2020-10-30 01:26:46 +01:00

#82195 was marked as duplicate of this issue

Nicholas Hawkes commented

2020-10-30 06:18:31 +01:00

Added subscriber: @nicholashawkes

Wayde Moss commented

2020-10-30 21:22:36 +01:00

Changed status from 'Needs Triage' to: 'Confirmed'

Philipp Oeser commented

2020-11-18 16:03:11 +01:00

Added subscribers: @Nickname6498156, @lichtwerk, @mano-wii

RedMser commented

2020-12-02 18:44:09 +01:00

Added subscriber: @RedMser

Wayde Moss changed title from ~~NLA: Fix Dragging Strips and/or Remove Transitions~~ to NLA: Fix Dragging Strips

2021-01-13 07:02:27 +01:00

Brad Clark commented

2021-01-13 19:45:40 +01:00

Added subscriber: @BClark

Brad Clark commented

2021-01-13 19:45:40 +01:00

So excited for this!!!!

Philipp Oeser commented

2021-01-26 12:36:41 +01:00

Added subscriber: @Gehrig

Gehrig commented

2021-01-26 16:21:56 +01:00

@lichtwerk ah wonderful! I'm sorry I didn't see this duplicate! Thank you for subscribing me here :)

Anna Celarek commented

2021-11-04 10:33:52 +01:00

Added subscriber: @AnnaCelarek

Anna Celarek commented

2021-11-04 10:39:49 +01:00

I wanted to test the patch, but the link doesn't work.
I hope this featuer gets implemented into the main branch, it's absolutely necessary for efficient working with the NLA.

I wanted to test the patch, but the link doesn't work. I hope this featuer gets implemented into the main branch, it's absolutely necessary for efficient working with the NLA.

Luciano Muñoz Sessarego commented

2021-11-04 19:05:12 +01:00

Added subscriber: @LucianoMunoz

Luciano Muñoz Sessarego commented

2021-11-04 19:05:12 +01:00

This is one of the things that make the nla such a drag!, so yes please!

Brad Clark self-assigned this 2022-02-23 00:27:10 +01:00

Brad Clark removed their assignment 2022-02-23 16:16:56 +01:00

Andy Cuccaro commented

2022-02-24 18:17:13 +01:00

Added subscriber: @AndyCuccaro

blender-admin commented

2023-01-30 20:39:36 +01:00

This issue was referenced by f31ad5d98b

This issue was referenced by f31ad5d98b3aa0697b5b1d477f7d58791ac19f97

Philipp Oeser removed the

Download

What's New

Blender Studio

Manual

Developers Blog

Documentation

Benchmark

Blender Conference

Development Fund

One-time Donations

NLA: Fix Dragging Strips #82241