VSE dynamic retiming design #100337

New Issue

Richard Antalik · 2022-08-11T02:59:12+02:00

Richard Antalik commented

2022-08-11 02:59:12 +02:00

Goal of this feature is to allow variable strip playback speed, using new retiming system.

Overview

Concept of retiming handle is introduced. Retiming handle represents frame of original content, that can be moved in time. Frames between 2 handles are interpolated so effectively playback speed is changed.
Manipulation is facilitated by dedicated tool with gizmos drawn on strip. Gizmos are drawn on active strip only.
Strip animation should be handled, such that existing keyframes are effectively mapped onto particular frames.
Functionality of speed_factor strip property is redundant and will be replaced by tool functionality.

Tool overview

When tool is activated and strip has no retiming handles, one is created to allow for quick manipulation.
Retiming handle is represented by line which can be dragged to retime the strip.
When retiming handle is moved, all handles to the right of the strip will be moved and strip length will be changed. With key modifier alternative behavior can be defined, for example other handles are not moved and strip length is not changed.
There is always retiming handle present on strip leftmost frame, but it is not drawn and can not be moved.
Retiming handle can be added at current frame by activating gizmo near current frame. They can be deleted by activating gizmo on handle itself.

To be clarified

Special handle types may exist that allow features like frame freeze, smooth playback speed gradient, and similar.
There can be gizmo to remove handle, such that playback speed adjusts to previous or next retimed section.

Goal of this feature is to allow variable strip playback speed, using new retiming system. ### Overview - Concept of retiming handle is introduced. Retiming handle represents frame of original content, that can be moved in time. Frames between 2 handles are interpolated so effectively playback speed is changed. - Manipulation is facilitated by dedicated tool with gizmos drawn on strip. Gizmos are drawn on active strip only. - Strip animation should be handled, such that existing keyframes are effectively mapped onto particular frames. - Functionality of `speed_factor` strip property is redundant and will be replaced by tool functionality. ### Tool overview - When tool is activated and strip has no retiming handles, one is created to allow for quick manipulation. - Retiming handle is represented by line which can be dragged to retime the strip. - When retiming handle is moved, all handles to the right of the strip will be moved and strip length will be changed. With key modifier alternative behavior can be defined, for example other handles are not moved and strip length is not changed. - There is always retiming handle present on strip leftmost frame, but it is not drawn and can not be moved. - Retiming handle can be added at current frame by activating gizmo near current frame. They can be deleted by activating gizmo on handle itself. ### To be clarified - Special handle types may exist that allow features like frame freeze, smooth playback speed gradient, and similar. - There can be gizmo to remove handle, such that playback speed adjusts to previous or next retimed section.

Richard Antalik commented

2022-08-11 02:59:12 +02:00

Changed status from 'Needs Triage' to: 'Confirmed'

Richard Antalik commented

2022-08-11 02:59:12 +02:00

Added subscribers: @fsiddi, @iss

Sergey Sharybin commented

2022-09-09 16:23:40 +02:00

Added subscriber: @Sergey

Sergey Sharybin commented

2022-09-09 16:23:40 +02:00

It seems that the task is mainly raising questions about some implementation specific details, and the actual design is only mentioned briefly. It makes it rather hard to comment on.

Goal of this feature is to allow dynamic, or animated strip playback speed, using new retiming system.

What is the difference between dynamic and animated strip playback speed? If the speed is animated, does it mean that the strip length changes during animation playback?

Manipulation is (mainly) facilitated by dedicated tool.

Using the dedicated tool sounds fine. But what you mean by mainly? What are the other ways of retiming manipulation.

User can add control points to strip, which are mapped to particular frame and move them around. By moving control point forward, playback speed between current and previous point decreases, and up to next point increases.

There are some missing concepts here.

How one adds a control point to a strip? Can't think of what existing concept of a strip this relates to, and the proposal does not mention new concepts.
Are the control points mapped to the footage frame, scene frame?

I am not sure if speed is a good mental model. It is harder to understand intuitively what result changing the speed will lead to. To me the better mental model is more in realms of thinking of control point which corresponds to a frame from the media, and user drags it in timeline to the position where they want it to be in the edit.

This feature will require animation curve, but there are 2 aspects that have multiple options:

Is not really clear where the requirement is coming from.

If the idea is to build a tool based on what we already have then to me it seems that the tool will utilize the "Speed Factor" and its f-curve. But then to me it seems that the ownershop, speed, and speed vs. frame mapping questions are answered implicitly:

f-curve is owned by the animation system as it is now
f-curve is visible in all animation system editors
speed is used (Speed Factor in the interface)

If the tool is good then I do not see a reason for artists to go to animation editor to tweak the curve. And even if they do go into the animation editor, and manage to create a really long strip it is not any different from them doing it already. So not sure this is a huge factor for decision making.

If we ignore the speed factor for a moment, and want to design tool which manipulates its own data structures, using a LUT or curve mapping for frames rather than speed seems more straightforward for understanding. But then it is unclear where the requirement of the animation curve is coming from.
The tricky part here is to figure out how to make this new tool and the speed factor to not conflict with each other. Some conversion code and deprecation of the speed factor property could be a solution.

Caching topics I would not go into until the rest of the design is clear and is working. To me caching is a performance improvement technique which does not dictate the design, but rather follows it.

The sound seeking issue is to me also something more into realms of current code limitation rather than a design issue. Not really familiar with the way Audaspace works, but it doesn't seem to be impossible to solve issues you're bringing up.

For the pitch of the audio strips my question is: is it even the way artists expect to operate on audio? I am not really sure why on a user level we should treat retiming of audio and video strips any different.

It seems that the task is mainly raising questions about some implementation specific details, and the actual design is only mentioned briefly. It makes it rather hard to comment on. > Goal of this feature is to allow dynamic, or animated strip playback speed, using new retiming system. What is the difference between dynamic and animated strip playback speed? If the speed is animated, does it mean that the strip length changes during animation playback? > Manipulation is (mainly) facilitated by dedicated tool. Using the dedicated tool sounds fine. But what you mean by mainly? What are the other ways of retiming manipulation. > User can add control points to strip, which are mapped to particular frame and move them around. By moving control point forward, playback speed between current and previous point decreases, and up to next point increases. There are some missing concepts here. How one adds a control point to a strip? Can't think of what existing concept of a strip this relates to, and the proposal does not mention new concepts. Are the control points mapped to the footage frame, scene frame? I am not sure if speed is a good mental model. It is harder to understand intuitively what result changing the speed will lead to. To me the better mental model is more in realms of thinking of control point which corresponds to a frame from the media, and user drags it in timeline to the position where they want it to be in the edit. > This feature will require animation curve, but there are 2 aspects that have multiple options: Is not really clear where the requirement is coming from. If the idea is to build a tool based on what we already have then to me it seems that the tool will utilize the "Speed Factor" and its f-curve. But then to me it seems that the ownershop, speed, and speed vs. frame mapping questions are answered implicitly: - f-curve is owned by the animation system as it is now - f-curve is visible in all animation system editors - speed is used (Speed Factor in the interface) If the tool is good then I do not see a reason for artists to go to animation editor to tweak the curve. And even if they do go into the animation editor, and manage to create a really long strip it is not any different from them doing it already. So not sure this is a huge factor for decision making. If we ignore the speed factor for a moment, and want to design tool which manipulates its own data structures, using a LUT or curve mapping for frames rather than speed seems more straightforward for understanding. But then it is unclear where the requirement of the animation curve is coming from. The tricky part here is to figure out how to make this new tool and the speed factor to not conflict with each other. Some conversion code and deprecation of the speed factor property could be a solution. Caching topics I would not go into until the rest of the design is clear and is working. To me caching is a performance improvement technique which does not dictate the design, but rather follows it. The sound seeking issue is to me also something more into realms of current code limitation rather than a design issue. Not really familiar with the way Audaspace works, but it doesn't seem to be impossible to solve issues you're bringing up. For the pitch of the audio strips my question is: is it even the way artists expect to operate on audio? I am not really sure why on a user level we should treat retiming of audio and video strips any different.

Richard Antalik commented

2022-09-09 18:47:04 +02:00

Added subscriber: @neXyon

Richard Antalik commented

2022-09-09 18:47:04 +02:00

In #100337#1414658, @Sergey wrote:
It seems that the task is mainly raising questions about some implementation specific details, and the actual design is only mentioned briefly. It makes it rather hard to comment on.

That is correct. From my point of view feature design is quite simple, but implementation has to be discussed. Perhaps will try to use just term "retiming" as it is most general.

Goal of this feature is to allow dynamic, or animated strip playback speed, using new retiming system.

What is the difference between dynamic and animated strip playback speed?

There is none, I just did not want to use term "animated playback speed", because I don't think it's quite appropriate. Term "Dynamic" isn't quite appropriate either probably. More correct description would be: The feature can produce strip which frames do not have linear mapping onto original content.

If the speed is animated, does it mean that the strip length changes during animation playback?

No strip length is constant during playback

Manipulation is (mainly) facilitated by dedicated tool.

Using the dedicated tool sounds fine. But what you mean by mainly? What are the other ways of retiming manipulation.

Tool system uses operators, so these should be available and usable by their own. What I wanted to say, that "focal point" of this feature will be a tool with set of gizmos and operators.

User can add control points to strip, which are mapped to particular frame and move them around. By moving control point forward, playback speed between current and previous point decreases, and up to next point increases.

There are some missing concepts here.

How one adds a control point to a strip? Can't think of what existing concept of a strip this relates to, and the proposal does not mention new concepts.
Are the control points mapped to the footage frame, scene frame?

I was describing tool UX here. This could be implemented as adding keyframes on property. in which case this would not be new concept. But here I propose a new concept of strip owning this dataset. I have proposed to store data in FCurve, which I am not sure it's best idea. It provides most functionality I need, but has also some limitations, that could be hard to work around.

I am not sure if speed is a good mental model. It is harder to understand intuitively what result changing the speed will lead to. To me the better mental model is more in realms of thinking of control point which corresponds to a frame from the media, and user drags it in timeline to the position where they want it to be in the edit.

Agree, but since I was explaining UX, there it makes more sense.

This feature will require animation curve, but there are 2 aspects that have multiple options:

Is not really clear where the requirement is coming from.

If the idea is to build a tool based on what we already have then to me it seems that the tool will utilize the "Speed Factor" and its f-curve. But then to me it seems that the ownershop, speed, and speed vs. frame mapping questions are answered implicitly:

f-curve is owned by the animation system as it is now

f-curve is visible in all animation system editors

speed is used (Speed Factor in the interface)

Correct, though here I propose to not build the tool based on what we already have.

If the tool is good then I do not see a reason for artists to go to animation editor to tweak the curve. And even if they do go into the animation editor, and manage to create a really long strip it is not any different from them doing it already. So not sure this is a huge factor for decision making.

My point was, that our animation system can do things my tool would not be able to handle correctly. For example curve modifiers and drivers. That is my main point against this route - it's too complex to be handled "externally" from another module. Also this complexity is not useful in vast majority of use-cases.

If we ignore the speed factor for a moment, and want to design tool which manipulates its own data structures, using a LUT or curve mapping for frames rather than speed seems more straightforward for understanding. But then it is unclear where the requirement of the animation curve is coming from.

Perhaps I am using term "animation curve" more freely, than I should. What I mean is we have to have dataset that describes mapping of strip frames onto frames of original content.

The tricky part here is to figure out how to make this new tool and the speed factor to not conflict with each other. Some conversion code and deprecation of the speed factor property could be a solution.

This is least of my concern probably - I can convert animated speed to new system. I have talked with fsiddi, that if speed changes at any particular frame, I can add control point in new system that would be linear. This should preserve existing animation well.

I don't (necessarily) want to deprecate speed_factor property - if you want to play whole strip x times faster or slower, Changing property is easiest way to achieve this. But this could be facilitated by tool as well just so all controls are at same place.

Caching topics I would not go into until the rest of the design is clear and is working. To me caching is a performance improvement technique which does not dictate the design, but rather follows it.

Well my main point here was that solution where we use existing speed property for retiming will require cache, while solution I am proposing does not.
Also I wanted outline limitations of caching, but that is not that of a problem.

The sound seeking issue is to me also something more into realms of current code limitation rather than a design issue. Not really familiar with the way Audaspace works, but it doesn't seem to be impossible to solve issues you're bringing up.

Yes I went bit too far ahead here... But from talk in D15652, @neXyon suggests, that Audaspace should copy existing animation from Blender, so this is something to keep in mind.

For the pitch of the audio strips my question is: is it even the way artists expect to operate on audio? I am not really sure why on a user level we should treat retiming of audio and video strips any different.

It depends. For example in Heist movie sound pitch animation was used quite extensively to produce some sound effects. Other use case may be to use constant interpolation when you want to just change speed.
This would be limitation of retiming system I propose, that it wouldn't support these smooth sweeps natively and these would have to be constructed by multiple "retiming points". I think this is still beneficial even if this is done extensively, because changes in speed do happen usually in few seconds, then speed is again constant. So amount of points the system has to care about is still much lower. Retiming system that would use existing animation system and speed_factor property will require "retiming point" per each frame internally.

There is also one thing I forgot to mention in UX section - when retiming portion of strip, I want the tool to preserve existing animation as much as possible
With retiming system I propose, you can calculate 1 factor by which you offset all keyframes in portion of strip between 2 control points. Contrast to that, retiming system using speed_factor property would behave in a way, that if you change speed value of a keyframe, there is nothing really to calculate - you will have to do snapshot of frame mapping before the change, then map each keyframe to original content frame, and then use new frame map to map original content frame to new keyframe frame. That is if such point even exists in the map, if not you will have to interpolate it somehow.

In this task I mainly wanted to discuss implementation of retiming with relation to existing vs new animation system. I mainly wanted to discuss this, because since we have animation system already, it would make sense to use it. So I analyzed both solutions and my conclusion is that using existing animation system would be much more complex (as in performance and also code-wise). Therefore I want to handle frame mapping internally in sequencer, that would be "opaque to user" and mainly visualized via tool system. For extensibility I am happy to provide API alongside operators to handle data though.

> In #100337#1414658, @Sergey wrote: > It seems that the task is mainly raising questions about some implementation specific details, and the actual design is only mentioned briefly. It makes it rather hard to comment on. That is correct. From my point of view feature design is quite simple, but implementation has to be discussed. Perhaps will try to use just term "retiming" as it is most general. >> Goal of this feature is to allow dynamic, or animated strip playback speed, using new retiming system. > > What is the difference between dynamic and animated strip playback speed? There is none, I just did not want to use term "animated playback speed", because I don't think it's quite appropriate. Term "Dynamic" isn't quite appropriate either probably. More correct description would be: The feature can produce strip which frames do not have linear mapping onto original content. > If the speed is animated, does it mean that the strip length changes during animation playback? No strip length is constant during playback >> Manipulation is (mainly) facilitated by dedicated tool. > > Using the dedicated tool sounds fine. But what you mean by mainly? What are the other ways of retiming manipulation. Tool system uses operators, so these should be available and usable by their own. What I wanted to say, that "focal point" of this feature will be a tool with set of gizmos and operators. >> User can add control points to strip, which are mapped to particular frame and move them around. By moving control point forward, playback speed between current and previous point decreases, and up to next point increases. > > There are some missing concepts here. > > How one adds a control point to a strip? Can't think of what existing concept of a strip this relates to, and the proposal does not mention new concepts. > Are the control points mapped to the footage frame, scene frame? I was describing tool UX here. This could be implemented as adding keyframes on property. in which case this would not be new concept. But here I propose a new concept of strip owning this dataset. I have proposed to store data in `FCurve`, which I am not sure it's best idea. It provides most functionality I need, but has also some limitations, that could be hard to work around. > I am not sure if speed is a good mental model. It is harder to understand intuitively what result changing the speed will lead to. To me the better mental model is more in realms of thinking of control point which corresponds to a frame from the media, and user drags it in timeline to the position where they want it to be in the edit. Agree, but since I was explaining UX, there it makes more sense. >> This feature will require animation curve, but there are 2 aspects that have multiple options: > > Is not really clear where the requirement is coming from. > > If the idea is to build a tool based on what we already have then to me it seems that the tool will utilize the "Speed Factor" and its f-curve. But then to me it seems that the ownershop, speed, and speed vs. frame mapping questions are answered implicitly: > > - f-curve is owned by the animation system as it is now > - f-curve is visible in all animation system editors > - speed is used (Speed Factor in the interface) Correct, though here I propose to not build the tool based on what we already have. > If the tool is good then I do not see a reason for artists to go to animation editor to tweak the curve. And even if they do go into the animation editor, and manage to create a really long strip it is not any different from them doing it already. So not sure this is a huge factor for decision making. My point was, that our animation system can do things my tool would not be able to handle correctly. For example curve modifiers and drivers. That is my main point against this route - it's too complex to be handled "externally" from another module. Also this complexity is not useful in vast majority of use-cases. > If we ignore the speed factor for a moment, and want to design tool which manipulates its own data structures, using a LUT or curve mapping for frames rather than speed seems more straightforward for understanding. But then it is unclear where the requirement of the animation curve is coming from. Perhaps I am using term "animation curve" more freely, than I should. What I mean is we have to have dataset that describes mapping of strip frames onto frames of original content. > The tricky part here is to figure out how to make this new tool and the speed factor to not conflict with each other. Some conversion code and deprecation of the speed factor property could be a solution. This is least of my concern probably - I can convert animated speed to new system. I have talked with fsiddi, that if speed changes at any particular frame, I can add control point in new system that would be linear. This should preserve existing animation well. I don't (necessarily) want to deprecate `speed_factor` property - if you want to play whole strip x times faster or slower, Changing property is easiest way to achieve this. But this could be facilitated by tool as well just so all controls are at same place. > Caching topics I would not go into until the rest of the design is clear and is working. To me caching is a performance improvement technique which does not dictate the design, but rather follows it. Well my main point here was that solution where we use existing speed property for retiming will require cache, while solution I am proposing does not. Also I wanted outline limitations of caching, but that is not that of a problem. > The sound seeking issue is to me also something more into realms of current code limitation rather than a design issue. Not really familiar with the way Audaspace works, but it doesn't seem to be impossible to solve issues you're bringing up. Yes I went bit too far ahead here... But from talk in [D15652](https://archive.blender.org/developer/D15652), @neXyon suggests, that Audaspace should copy existing animation from Blender, so this is something to keep in mind. > For the pitch of the audio strips my question is: is it even the way artists expect to operate on audio? I am not really sure why on a user level we should treat retiming of audio and video strips any different. It depends. For example in Heist movie sound pitch animation was used quite extensively to produce some sound effects. Other use case may be to use constant interpolation when you want to just change speed. This would be limitation of retiming system I propose, that it wouldn't support these smooth sweeps natively and these would have to be constructed by multiple "retiming points". I think this is still beneficial even if this is done extensively, because changes in speed do happen usually in few seconds, then speed is again constant. So amount of points the system has to care about is still much lower. Retiming system that would use existing animation system and `speed_factor` property will require "retiming point" per each frame internally. There is also one thing I forgot to mention in UX section - when retiming portion of strip, I want the tool to preserve existing animation as much as possible With retiming system I propose, you can calculate 1 factor by which you offset all keyframes in portion of strip between 2 control points. Contrast to that, retiming system using `speed_factor` property would behave in a way, that if you change speed value of a keyframe, there is nothing really to calculate - you will have to do snapshot of frame mapping before the change, then map each keyframe to original content frame, and then use new frame map to map original content frame to new keyframe frame. That is if such point even exists in the map, if not you will have to interpolate it somehow. In this task I mainly wanted to discuss implementation of retiming with relation to existing vs new animation system. I mainly wanted to discuss this, because since we have animation system already, it would make sense to use it. So I analyzed both solutions and my conclusion is that using existing animation system would be much more complex (as in performance and also code-wise). Therefore I want to handle frame mapping internally in sequencer, that would be "opaque to user" and mainly visualized via tool system. For extensibility I am happy to provide API alongside operators to handle data though.

Richard Antalik commented

2022-09-20 10:38:04 +02:00

Just a note, that implementation I am proposing is in D15945. Patch is pretty much feature complete, there are some bugs and code needs cleanup in general. I am quite confident with the design, so my plan is to use it unless you see obvious holes in it (what I have written here).

Just a note, that implementation I am proposing is in [D15945](https://archive.blender.org/developer/D15945). Patch is pretty much feature complete, there are some bugs and code needs cleanup in general. I am quite confident with the design, so my plan is to use it unless you see obvious holes in it (what I have written here).

Sergey Sharybin commented

2022-09-20 15:12:59 +02:00

That is correct. From my point of view feature design is quite simple, but implementation has to be discussed. Perhaps will try to use just term "retiming" as it is most general.

It is not easily or really productive to think of implementation details when the design is not clearly communicated. There are some aspects which comes naturally from the design, and for other cases it is hard to think of details without knowing what are they really about.

There is none, I just did not want to use term "animated playback speed", because I don't think it's quite appropriate. Term "Dynamic" isn't quite appropriate either probably. More correct description would be: The feature can produce strip which frames do not have linear mapping onto original content.

To me it is still not really clear why there was a decision made to think in terms of speed. To me it does not seems to be the best mental model to think of the re-timing tool on an artistic level. It could be something hidden as an implementation detail, but even then it will have downsides.

I do like the direction your suggestion goes. To me it is indeed easier to think of frame re-mapping rather than speed. I see what you mean there, but strictly speaking reducing duration of a strip (hence, increasing it speed) is still a linear mapping.

I was describing tool UX here. This could be implemented as adding keyframes on property. in which case this would not be new concept. But here I propose a new concept of strip owning this dataset. I have proposed to store data in FCurve, which I am not sure it's best idea. It provides most functionality I need, but has also some limitations, that could be hard to work around.

I see. Not sure the control point is the best name for it. In a 3D world control points is something else.

For the f-curve I am not really convinced either, especially after seeing the WIP patch. Seems that it is not that much functionality is re-used from f-curve. and the f-curve is something much more overkill. If there are no plans to support non-linear interpolation it might be more clear to implement evaluation on an array of points (the rest of operations already seems to be implemented on the sequencer side). Alternatively, there are two possibilities:

Look into re-suing even more code between animation system and sequencer when it comes to manipulation of the points
Maybe curve mapping is a more suitable concept than f-curve for this type of funcitonality

Correct, though here I propose to not build the tool based on what we already have.

This is a bit confusing. The proposed patch re-uses parts of the animation system.

My point was, that our animation system can do things my tool would not be able to handle correctly. For example curve modifiers and drivers. That is my main point against this route - it's too complex to be handled "externally" from another module. Also this complexity is not useful in vast majority of use-cases.

I am getting even more confused. My point was that whether data manipulated by a tool is visible in the animation editors is kind of secondary when it comes to focusing on designing the tool.
But now the sentence reads as if the animation system is more powerful. What does it mean?

I don't (necessarily) want to deprecate speed_factor property - if you want to play whole strip x times faster or slower, Changing property is easiest way to achieve this. But this could be facilitated by tool as well just so all controls are at same place.

Is almost never having two things controlling the same thing ends up in a good UX. Thinking aloud: you can imagine that there could be a context menu with some pre-defined remapping choices: 50%, reset to 100%, 200%, Custom. I do not really see how having both curve mapping and (possibly animated) speed factor makes it easy and understandable from the implementation point of view.

There is also one thing I forgot to mention in UX section - when retiming portion of strip, I want the tool to preserve existing animation as much as possible

This sounds awesome. The rest i can not really follow. It goes in some implementation design with decisions which are not very well explained.

I am not even really convinced going the speed route in implementation is the best way to preserve frame at which the control point is added. Speed will intrinsicly lead to precision loss. If i was working on this feature I'd operate on the frame level: when user inserts a retiming point it has fixed frame number of the input media it corresponds to. Then when strip is rendered there is a linear interpolation between 2 adjacent retiming points to get the frame number to be used from the input. This seems to be the robust solution from preserving frame at the retiming tool. Not sure if such approach was considered, and if so, why it was discarded.

From testing the current patch it does not seem that the frame at a keyframe is preserved well. Shrinking the input media of 250 frames to 100 frames makes the last visible frame from the strip to go from 250 to 247. Inserting retiming point somewhere in the middle of the strip and moving it around could lead to couple of frames off. To me it seems to be easily avoidable side effects if a different implementation is used.

> That is correct. From my point of view feature design is quite simple, but implementation has to be discussed. Perhaps will try to use just term "retiming" as it is most general. It is not easily or really productive to think of implementation details when the design is not clearly communicated. There are some aspects which comes naturally from the design, and for other cases it is hard to think of details without knowing what are they really about. > There is none, I just did not want to use term "animated playback speed", because I don't think it's quite appropriate. Term "Dynamic" isn't quite appropriate either probably. More correct description would be: The feature can produce strip which frames do not have linear mapping onto original content. To me it is still not really clear why there was a decision made to think in terms of speed. To me it does not seems to be the best mental model to think of the re-timing tool on an artistic level. It could be something hidden as an implementation detail, but even then it will have downsides. I do like the direction your suggestion goes. To me it is indeed easier to think of frame re-mapping rather than speed. I see what you mean there, but strictly speaking reducing duration of a strip (hence, increasing it speed) is still a linear mapping. > I was describing tool UX here. This could be implemented as adding keyframes on property. in which case this would not be new concept. But here I propose a new concept of strip owning this dataset. I have proposed to store data in FCurve, which I am not sure it's best idea. It provides most functionality I need, but has also some limitations, that could be hard to work around. I see. Not sure the control point is the best name for it. In a 3D world control points is something else. For the f-curve I am not really convinced either, especially after seeing the WIP patch. Seems that it is not that much functionality is re-used from f-curve. and the f-curve is something much more overkill. If there are no plans to support non-linear interpolation it might be more clear to implement evaluation on an array of points (the rest of operations already seems to be implemented on the sequencer side). Alternatively, there are two possibilities: * Look into re-suing even more code between animation system and sequencer when it comes to manipulation of the points * Maybe curve mapping is a more suitable concept than f-curve for this type of funcitonality > Correct, though here I propose to not build the tool based on what we already have. This is a bit confusing. The proposed patch re-uses parts of the animation system. > My point was, that our animation system can do things my tool would not be able to handle correctly. For example curve modifiers and drivers. That is my main point against this route - it's too complex to be handled "externally" from another module. Also this complexity is not useful in vast majority of use-cases. I am getting even more confused. My point was that whether data manipulated by a tool is visible in the animation editors is kind of secondary when it comes to focusing on designing the tool. But now the sentence reads as if the animation system is more powerful. What does it mean? > I don't (necessarily) want to deprecate `speed_factor` property - if you want to play whole strip x times faster or slower, Changing property is easiest way to achieve this. But this could be facilitated by tool as well just so all controls are at same place. Is almost never having two things controlling the same thing ends up in a good UX. Thinking aloud: you can imagine that there could be a context menu with some pre-defined remapping choices: 50%, reset to 100%, 200%, Custom. I do not really see how having both curve mapping and (possibly animated) speed factor makes it easy and understandable from the implementation point of view. > There is also one thing I forgot to mention in UX section - when retiming portion of strip, I want the tool to preserve existing animation as much as possible This sounds awesome. The rest i can not really follow. It goes in some implementation design with decisions which are not very well explained. I am not even really convinced going the speed route in implementation is the best way to preserve frame at which the control point is added. Speed will intrinsicly lead to precision loss. If i was working on this feature I'd operate on the frame level: when user inserts a retiming point it has fixed frame number of the input media it corresponds to. Then when strip is rendered there is a linear interpolation between 2 adjacent retiming points to get the frame number to be used from the input. This seems to be the robust solution from preserving frame at the retiming tool. Not sure if such approach was considered, and if so, why it was discarded. From testing the current patch it does not seem that the frame at a keyframe is preserved well. Shrinking the input media of 250 frames to 100 frames makes the last visible frame from the strip to go from 250 to 247. Inserting retiming point somewhere in the middle of the strip and moving it around could lead to couple of frames off. To me it seems to be easily avoidable side effects if a different implementation is used.

Richard Antalik commented

2022-09-21 00:41:57 +02:00

It is not easily or really productive to think of implementation details when the design is not clearly communicated. There are some aspects which comes naturally from the design, and for other cases it is hard to think of details without knowing what are they really about.

I understand that. If there is still something unclear in user level design, I am happy to clarify it.

To me it is still not really clear why there was a decision made to think in terms of speed.

There wasn't such decision. Speed is important for users and for Audaspace currently.

Not sure the control point is the best name for it. In a 3D world control points is something else.

So far I have iterated to term "retiming handle". I think that is very obvious and quite clear.

For the f-curve I am not really convinced either, especially after seeing the WIP patch. Seems that it is not that much functionality is re-used from f-curve. and the f-curve is something much more overkill. If there are no plans to support non-linear interpolation it might be more clear to implement evaluation on an array of points (the rest of operations already seems to be implemented on the sequencer side). Alternatively, there are two possibilities:

Look into re-suing even more code between animation system and sequencer when it comes to manipulation of the points

Maybe curve mapping is a more suitable concept than f-curve for this type of funcitonality

I thought that using FCurves is something that can be changed rather easily with versioning if I want to change it later on. I was thinking of refactoring function insert_keyframe() so BKE_fcurve functions are available to more modules, but did not do it before I am firmly decided, that I will actually use FCurves.

My main reasons against using it is for implementation of freeze frames and possibly other features like "speed gradient" to smoothly transition between 2 different playback speeds (it's likely that users would want this). Simplest solution would be to have flag for retiming handle. With FCurves this would be not straightforward to do.

Curve mapping is something I was thinking of as alternative. If I understand you correctly, I would have own implementation of this curve mapping with own type, handle/point/keyframe management and evaluation is that correct?

Correct, though here I propose to not build the tool based on what we already have.

This is a bit confusing. The proposed patch re-uses parts of the animation system.

I am fine with using parts as long as they work for purpose and don't cause aditional issues. What I meant is to not use animation system as a whole.

My point was, that our animation system can do things my tool would not be able to handle correctly. For example curve modifiers and drivers. That is my main point against this route - it's too complex to be handled "externally" from another module. Also this complexity is not useful in vast majority of use-cases.

I am getting even more confused. My point was that whether data manipulated by a tool is visible in the animation editors is kind of secondary when it comes to focusing on designing the tool.
But now the sentence reads as if the animation system is more powerful. What does it mean?

I don't think it is secondary. Retiming data do have constraints which makes them unsuitable to be managed with anizmation system. We could make changes to resolve this, but I argue, that it's not in our interest.

Is almost never having two things controlling the same thing ends up in a good UX.

I think you are correct here, so I guess speed_factor could be removed completely. However I need to fix sound seeking first, then convert sound animation to this system, otherwise the tool would not work with sound at all. Also missing ability to do gradual speed changes would be a regression. That is why I don't really want to touch it at this moment. I am quite confident, that I would be able to implement these missing features fairly quickly though.

I am not even really convinced going the speed route in implementation is the best way to preserve frame at which the control point is added. Speed will intrinsicly lead to precision loss. If i was working on this feature I'd operate on the frame level: when user inserts a retiming point it has fixed frame number of the input media it corresponds to. Then when strip is rendered there is a linear interpolation between 2 adjacent retiming points to get the frame number to be used from the input. This seems to be the robust solution from preserving frame at the retiming tool.

This is exactly what I am proposing, so I am happy that we are in agreement on this point.

Not sure if such approach was considered, and if so, why it was discarded.

But I have implemented this - playback speed isn't used anywhere in code apart from printing text labels on strip.

From testing the current patch it does not seem that the frame at a keyframe is preserved well. Shrinking the input media of 250 frames to 100 frames makes the last visible frame from the strip to go from 250 to 247. Inserting retiming point somewhere in the middle of the strip and moving it around could lead to couple of frames off. To me it seems to be easily avoidable side effects if a different implementation is used.

I have noticed this, there must be 1-off error somewhere. There are many smaller issues like this, will fix them soon.

>It is not easily or really productive to think of implementation details when the design is not clearly communicated. There are some aspects which comes naturally from the design, and for other cases it is hard to think of details without knowing what are they really about. I understand that. If there is still something unclear in user level design, I am happy to clarify it. > To me it is still not really clear why there was a decision made to think in terms of speed. There wasn't such decision. Speed is important for users and for Audaspace currently. > Not sure the control point is the best name for it. In a 3D world control points is something else. So far I have iterated to term "retiming handle". I think that is very obvious and quite clear. >For the f-curve I am not really convinced either, especially after seeing the WIP patch. Seems that it is not that much functionality is re-used from f-curve. and the f-curve is something much more overkill. If there are no plans to support non-linear interpolation it might be more clear to implement evaluation on an array of points (the rest of operations already seems to be implemented on the sequencer side). Alternatively, there are two possibilities: > > - Look into re-suing even more code between animation system and sequencer when it comes to manipulation of the points > - Maybe curve mapping is a more suitable concept than f-curve for this type of funcitonality I thought that using FCurves is something that can be changed rather easily with versioning if I want to change it later on. I was thinking of refactoring function `insert_keyframe()` so `BKE_fcurve` functions are available to more modules, but did not do it before I am firmly decided, that I will actually use FCurves. My main reasons against using it is for implementation of freeze frames and possibly other features like "speed gradient" to smoothly transition between 2 different playback speeds (it's likely that users would want this). Simplest solution would be to have flag for retiming handle. With FCurves this would be not straightforward to do. Curve mapping is something I was thinking of as alternative. If I understand you correctly, I would have own implementation of this curve mapping with own type, handle/point/keyframe management and evaluation is that correct? >>Correct, though here I propose to not build the tool based on what we already have. >This is a bit confusing. The proposed patch re-uses parts of the animation system. I am fine with using parts as long as they work for purpose and don't cause aditional issues. What I meant is to not use animation system as a whole. >>My point was, that our animation system can do things my tool would not be able to handle correctly. For example curve modifiers and drivers. That is my main point against this route - it's too complex to be handled "externally" from another module. Also this complexity is not useful in vast majority of use-cases. >I am getting even more confused. My point was that whether data manipulated by a tool is visible in the animation editors is kind of secondary when it comes to focusing on designing the tool. >But now the sentence reads as if the animation system is more powerful. What does it mean? I don't think it is secondary. Retiming data do have constraints which makes them unsuitable to be managed with anizmation system. We could make changes to resolve this, but I argue, that it's not in our interest. > Is almost never having two things controlling the same thing ends up in a good UX. I think you are correct here, so I guess `speed_factor` could be removed completely. However I need to fix sound seeking first, then convert sound animation to this system, otherwise the tool would not work with sound at all. Also missing ability to do gradual speed changes would be a regression. That is why I don't really want to touch it at this moment. I am quite confident, that I would be able to implement these missing features fairly quickly though. >I am not even really convinced going the speed route in implementation is the best way to preserve frame at which the control point is added. Speed will intrinsicly lead to precision loss. If i was working on this feature I'd operate on the frame level: when user inserts a retiming point it has fixed frame number of the input media it corresponds to. Then when strip is rendered there is a linear interpolation between 2 adjacent retiming points to get the frame number to be used from the input. This seems to be the robust solution from preserving frame at the retiming tool. This is exactly what I am proposing, so I am happy that we are in agreement on this point. >Not sure if such approach was considered, and if so, why it was discarded. But I have implemented this - playback speed isn't used anywhere in code apart from printing text labels on strip. >From testing the current patch it does not seem that the frame at a keyframe is preserved well. Shrinking the input media of 250 frames to 100 frames makes the last visible frame from the strip to go from 250 to 247. Inserting retiming point somewhere in the middle of the strip and moving it around could lead to couple of frames off. To me it seems to be easily avoidable side effects if a different implementation is used. I have noticed this, there must be 1-off error somewhere. There are many smaller issues like this, will fix them soon.

Sergey Sharybin commented

2022-09-22 15:51:27 +02:00

There wasn't such decision.

After reading your reply I can see it now.
From the original description it was not really obvious what is the decision decision and what is, so to speak, a consequence of that.

So far I have iterated to term "retiming handle". I think that is very obvious and quite clear.

Retiming Handle sounds clear to me.

I thought that using FCurves is something that can be changed rather easily with versioning if I want to change it later on. I was thinking of refactoring function insert_keyframe() so BKE_fcurve functions are available to more modules, but did not do it before I am firmly decided, that I will actually use FCurves.

It isn't difficult to implement versioning code for any DNA tructure.

My main reasons against using it is for implementation of freeze frames and possibly other features like "speed gradient" to smoothly transition between 2 different playback speeds (it's likely that users would want this). Simplest solution would be to have flag for retiming handle. With FCurves this would be not straightforward to do.

Not sure how commonly used such feature is, or what is the UI/UX for it is. But intuitively, it can be represented as handles of a bezier control points (as opposite of segmented linear interpolation).
To me there are bigger concerns of using FCurves. Unless it is really intended to be used the same as as animation system uses it, it should not be used. The current patch uses FCurve as just a container of linearly inetrpolated control points. This is rather inefficient and weird design decision.

Curve mapping is something I was thinking of as alternative. If I understand you correctly, I would have own implementation of this curve mapping with own type, handle/point/keyframe management and evaluation is that correct?

Initially i meant just use the CurveMapping as is. It is much better choice than FCurve. But that was based on the design description at that time, and at the code state of that time. There was no concept of the frame freeze mentioned until now.
If there are video-editing specific features planned to be added having a dedicated data structure would make much more sense.

I think it would be a lot helpful to take a step back and write the proper proposal. It is important to have the design description, mental models in place. A lot of questions will be naturally answered by a good design document.
Some quick inspiration from the animation module:

#82359 (Stronger binding of Active and Selected)
#91452 (Make saving of quit.blend on exit optional)

> There wasn't such decision. After reading your reply I can see it now. From the original description it was not really obvious what is the decision decision and what is, so to speak, a consequence of that. > So far I have iterated to term "retiming handle". I think that is very obvious and quite clear. Retiming Handle sounds clear to me. > I thought that using FCurves is something that can be changed rather easily with versioning if I want to change it later on. I was thinking of refactoring function insert_keyframe() so BKE_fcurve functions are available to more modules, but did not do it before I am firmly decided, that I will actually use FCurves. It isn't difficult to implement versioning code for any DNA tructure. > My main reasons against using it is for implementation of freeze frames and possibly other features like "speed gradient" to smoothly transition between 2 different playback speeds (it's likely that users would want this). Simplest solution would be to have flag for retiming handle. With FCurves this would be not straightforward to do. Not sure how commonly used such feature is, or what is the UI/UX for it is. But intuitively, it can be represented as handles of a bezier control points (as opposite of segmented linear interpolation). To me there are bigger concerns of using FCurves. Unless it is really intended to be used the same as as animation system uses it, it should not be used. The current patch uses FCurve as just a container of linearly inetrpolated control points. This is rather inefficient and weird design decision. > Curve mapping is something I was thinking of as alternative. If I understand you correctly, I would have own implementation of this curve mapping with own type, handle/point/keyframe management and evaluation is that correct? Initially i meant just use the `CurveMapping` as is. It is much better choice than `FCurve`. But that was based on the design description at that time, and at the code state of that time. There was no concept of the frame freeze mentioned until now. If there are video-editing specific features planned to be added having a dedicated data structure would make much more sense. --- I think it would be a lot helpful to take a step back and write the proper proposal. It is important to have the design description, mental models in place. A lot of questions will be naturally answered by a good design document. Some quick inspiration from the animation module: - #82359 (Stronger binding of Active and Selected) - #91452 (Make saving of `quit.blend` on exit optional)

Richard Antalik commented

2022-09-26 19:53:15 +02:00

I think it would be a lot helpful to take a step back and write the proper proposal. It is important to have the design description, mental models in place. A lot of questions will be naturally answered by a good design document.

Ok I will update description and describe the tool as best as I can. Usually I start with rough idea and smooth out details in the process, so there are still things I want to discuss.

> I think it would be a lot helpful to take a step back and write the proper proposal. It is important to have the design description, mental models in place. A lot of questions will be naturally answered by a good design document. Ok I will update description and describe the tool as best as I can. Usually I start with rough idea and smooth out details in the process, so there are still things I want to discuss.

Sergey Sharybin commented

2022-09-27 16:52:41 +02:00

Thank for the update! It reads much more clearly to me now.

Special handle types may exist that allow features like frame freeze, smooth playback speed gradient, and similar.

I am not so much sure about the gradient. Having frame freeze is something what I expect will be requested soon.
With the updated state of the patch the DNA seems to be flexible enough to support frame freeze around handle. But I am not sure how the gizmo interaction should look like for it. But also, is it something we need to resolve for an initial tool commit, or something we can look into later? Just to understand priorities, you know.

There can be gizmo to remove handle, such that playback speed adjusts to previous or next retimed section.

Yeah, to be honest, i did not figure out how to remove the handle :(
Adding the control feels a bit suboptimal. Maybe it is the best of what we can do. But would be very nice to have @fsiddi feedback on it, to see what is the most efficient and straightforward UI/UX from an editor perspective.

Functionality of speed_factor strip property is redundant and can be removed.

I think it would be very good longer-term goal. Basically, avoid having multiple properties/tools controlling effectively the same thing. For the initial integration it might be easier to leave it as-is, and do the do-versions and such after the tool is in and tested and such.

Btw, we've been talking about audio before, and now I do not see it in the clarification list. Did you figure it out? Kinda curios what would it mean to have re-timing tool for it.
Keep in mind, a decision like "we keep audio out of the scope of the initial patch" is also fine. Is more matter of having clear timeline and what/when/if is being tackled.

Thank for the update! It reads much more clearly to me now. > Special handle types may exist that allow features like frame freeze, smooth playback speed gradient, and similar. I am not so much sure about the gradient. Having frame freeze is something what I expect will be requested soon. With the updated state of the patch the DNA seems to be flexible enough to support frame freeze around handle. But I am not sure how the gizmo interaction should look like for it. But also, is it something we need to resolve for an initial tool commit, or something we can look into later? Just to understand priorities, you know. > There can be gizmo to remove handle, such that playback speed adjusts to previous or next retimed section. Yeah, to be honest, i did not figure out how to remove the handle :( Adding the control feels a bit suboptimal. Maybe it is the best of what we can do. But would be very nice to have @fsiddi feedback on it, to see what is the most efficient and straightforward UI/UX from an editor perspective. > Functionality of `speed_factor` strip property is redundant and can be removed. I think it would be very good longer-term goal. Basically, avoid having multiple properties/tools controlling effectively the same thing. For the initial integration it might be easier to leave it as-is, and do the do-versions and such after the tool is in and tested and such. Btw, we've been talking about audio before, and now I do not see it in the clarification list. Did you figure it out? Kinda curios what would it mean to have re-timing tool for it. Keep in mind, a decision like "we keep audio out of the scope of the initial patch" is also fine. Is more matter of having clear timeline and what/when/if is being tackled.

Richard Antalik commented

2022-09-27 19:06:52 +02:00

In #100337#1423672, @Sergey wrote:
Thank for the update! It reads much more clearly to me now.

Special handle types may exist that allow features like frame freeze, smooth playback speed gradient, and similar.

I am not so much sure about the gradient. Having frame freeze is something what I expect will be requested soon.
With the updated state of the patch the DNA seems to be flexible enough to support frame freeze around handle. But I am not sure how the gizmo interaction should look like for it. But also, is it something we need to resolve for an initial tool commit, or something we can look into later? Just to understand priorities, you know.

We can talk about this later. Currently it is not implemented and not discussed at all.

There can be gizmo to remove handle, such that playback speed adjusts to previous or next retimed section.

Yeah, to be honest, i did not figure out how to remove the handle :(

It's removed by clicking on bottom part of the handle - the triangle. It should change cursor to eraser...

Adding the control feels a bit suboptimal. Maybe it is the best of what we can do. But would be very nice to have @fsiddi feedback on it, to see what is the most efficient and straightforward UI/UX from an editor perspective.

Yes, it is bit awkward. I wanted to propose adding retiming handles anywhere on strip and drawing image of the frame either in preview or in timeline even, but I wouldn't like either solution. Normally you want to retime whole strip or from particular frame, and that is most likely your current frame.

Functionality of speed_factor strip property is redundant and can be removed.

I think it would be very good longer-term goal. Basically, avoid having multiple properties/tools controlling effectively the same thing. For the initial integration it might be easier to leave it as-is, and do the do-versions and such after the tool is in and tested and such.

I don't have very strong opinion here for existance of dedicated property or versioning. Decision to not touch these was mostly because of time constraint.
My only argument to keep Speed Factor property is for simplicity and ease of use. I know it's bad argument, but I stand by it :)

Btw, we've been talking about audio before, and now I do not see it in the clarification list. Did you figure it out? Kinda curios what would it mean to have re-timing tool for it.
Keep in mind, a decision like "we keep audio out of the scope of the initial patch" is also fine. Is more matter of having clear timeline and what/when/if is being tackled.

Technically there is not much to be clarified from design standpoint - it should work same way as retiming for video.

On implementation level, things are bit more complicated, because audaspace does not seek correctly. So if you retime sound, and play it back from middle of strip, it would play incorrect part of strip with correct pitch.

So what I need to do, after new system is in place or at least agreed on is:

Pitch property for audaspace has to be "emulated"
Fix seeking in audaspace
Do versioning for sound (First offsets need to be recalculated to reflect seek bugfix, then animation can be converted)
Update waveform drawing

So timeline I propose is to:

Review retiming tool
Make separate patch for seek bug in audaspace

Make Separate patch for sound strip support itself (items 1, 3, 4).

Francesco said, that sound support is unconditional, but agreed with this order of events. After this, I am fine with removing Speed Factor property as well.
Ultimately though plan is to discuss this more in detail after conference, due to time constraints. Francesco said, that it would be possible to have this as tool and change design later, but he would prefer to finish all features as much as possible before pushing to master.

> In #100337#1423672, @Sergey wrote: > Thank for the update! It reads much more clearly to me now. > >> Special handle types may exist that allow features like frame freeze, smooth playback speed gradient, and similar. > > I am not so much sure about the gradient. Having frame freeze is something what I expect will be requested soon. > With the updated state of the patch the DNA seems to be flexible enough to support frame freeze around handle. But I am not sure how the gizmo interaction should look like for it. But also, is it something we need to resolve for an initial tool commit, or something we can look into later? Just to understand priorities, you know. We can talk about this later. Currently it is not implemented and not discussed at all. >> There can be gizmo to remove handle, such that playback speed adjusts to previous or next retimed section. > > Yeah, to be honest, i did not figure out how to remove the handle :( It's removed by clicking on bottom part of the handle - the triangle. It should change cursor to eraser... > Adding the control feels a bit suboptimal. Maybe it is the best of what we can do. But would be very nice to have @fsiddi feedback on it, to see what is the most efficient and straightforward UI/UX from an editor perspective. Yes, it is bit awkward. I wanted to propose adding retiming handles anywhere on strip and drawing image of the frame either in preview or in timeline even, but I wouldn't like either solution. Normally you want to retime whole strip or from particular frame, and that is most likely your current frame. >> Functionality of `speed_factor` strip property is redundant and can be removed. > > I think it would be very good longer-term goal. Basically, avoid having multiple properties/tools controlling effectively the same thing. For the initial integration it might be easier to leave it as-is, and do the do-versions and such after the tool is in and tested and such. I don't have very strong opinion here for existance of dedicated property or versioning. Decision to not touch these was mostly because of time constraint. My only argument to keep Speed Factor property is for simplicity and ease of use. I know it's bad argument, but I stand by it :) > Btw, we've been talking about audio before, and now I do not see it in the clarification list. Did you figure it out? Kinda curios what would it mean to have re-timing tool for it. > Keep in mind, a decision like "we keep audio out of the scope of the initial patch" is also fine. Is more matter of having clear timeline and what/when/if is being tackled. Technically there is not much to be clarified from design standpoint - it should work same way as retiming for video. On implementation level, things are bit more complicated, because audaspace does not seek correctly. So if you retime sound, and play it back from middle of strip, it would play incorrect part of strip with correct pitch. So what I need to do, after new system is in place or at least agreed on is: - Pitch property for audaspace has to be "emulated" - Fix seeking in audaspace - Do versioning for sound (First offsets need to be recalculated to reflect seek bugfix, then animation can be converted) - Update waveform drawing So timeline I propose is to: - Review retiming tool - Make separate patch for seek bug in audaspace # Make Separate patch for sound strip support itself (items 1, 3, 4). Francesco said, that sound support is unconditional, but agreed with this order of events. After this, I am fine with removing Speed Factor property as well. Ultimately though plan is to discuss this more in detail after conference, due to time constraints. Francesco said, that it would be possible to have this as tool and change design later, but he would prefer to finish all features as much as possible before pushing to master.

Sergey Sharybin commented

2022-09-28 17:55:07 +02:00

The timeline seems good to me.

Can you elaborate on the meaning of "sound support is unconditional"? I am not sure I understand it correctly.

The timeline seems good to me. Can you elaborate on the meaning of "sound support is unconditional"? I am not sure I understand it correctly.

Richard Antalik commented

2022-09-28 18:15:00 +02:00

In #100337#1424462, @Sergey wrote:
The timeline seems good to me.

Can you elaborate on the meaning of "sound support is unconditional"? I am not sure I understand it correctly.

It means that sound retiming must be supported, otherwise feature won't pass functional review.

> In #100337#1424462, @Sergey wrote: > The timeline seems good to me. > > Can you elaborate on the meaning of "sound support is unconditional"? I am not sure I understand it correctly. It means that sound retiming must be supported, otherwise feature won't pass functional review.

Sergey Sharybin commented

2022-09-29 11:05:27 +02:00

As long as it means that the functional; review to determine the project is done, it is fine.
But we should move towards the final goal as a series of incremental patches. The video and audio code paths are rather isolated in Blender, so I do not see it helps mixing all of the changes into a single patch, and do not see a reason to hold off in commit of video retiming support once that part of the project is done.

As long as it means that the functional; review to determine the project is done, it is fine. But we should move towards the final goal as a series of incremental patches. The video and audio code paths are rather isolated in Blender, so I do not see it helps mixing all of the changes into a single patch, and do not see a reason to hold off in commit of video retiming support once that part of the project is done.

Richard Antalik commented

2022-09-29 19:10:39 +02:00

I understand need for sound support, though I would agree, that if the basic functionality is done, I don't see reason not to include it in release (assuming master is open for such features...). But that is also quite different topic to discuss.

Francesco Siddi commented

2022-09-30 00:27:29 +02:00

Since I brought up the importance of having retiming working for sound, I also agree that having working (and released) retiming for video alone is a good start, especially since audio and video are not linked together.

tintwotin commented

2022-11-12 07:17:23 +01:00

Added subscriber: @tintwotin

tintwotin commented

2022-11-12 07:17:23 +01:00

Does this approach support reverse playing as the existing Speed effect does?

Richard Antalik commented

2022-11-12 20:56:22 +01:00

Design does support that, but it's currently prevented by UI

Francesco Siddi commented

2022-11-23 14:24:00 +01:00

Had a review a few days ago with Richard. Here is the feedback:

don't show the retime gizmo element at the end of the clip
make the "marker" part of the gizmo slightly larger and with a larger heat zone
remove the hover effect from the retime gizmos, change of icon is enough
remove the speed attribute from the properties, and make it accessible as an operator in the strip menu instead (besides the retime tool)

Had a review a few days ago with Richard. Here is the feedback: - don't show the retime gizmo element at the end of the clip - make the "marker" part of the gizmo slightly larger and with a larger heat zone - remove the hover effect from the retime gizmos, change of icon is enough - remove the speed attribute from the properties, and make it accessible as an operator in the strip menu instead (besides the retime tool)

Richard Antalik added this to the Video Sequencer project 2023-02-09 21:00:44 +01:00

Philipp Oeser removed the

Download

What's New

Blender Studio

Manual

Developers Blog

Documentation

Benchmark

Blender Conference

Development Fund

One-time Donations

VSE dynamic retiming design #100337

Overview

Tool overview

To be clarified

Make Separate patch for sound strip support itself (items 1, 3, 4).