UI: Tracking: Migrate to active tools based toolbar #83612

New Issue

Aaron Carlisle · 2020-12-10T04:44:55+01:00

Aaron Carlisle commented

2020-12-10 04:44:55 +01:00

Status: Work in progress

Team
Commissioner: ?
Project leader: `@Blendify

Description
Big picture: The current Movie Clip Editor still uses the old toolbar from 2.7x this task is to migrate that to the active tool based workflow

Blender 2.80 is using a new toolshelf and toolsystem, so we need to update the toolshelf in Blender's clip editor too.
The workflow for tracking in the clip editor used to be split between toolshelf on the left and sidebar on the right.
In the toolshelf you would start setting up your scene length based on clip length, prefetch footage etc.
Then you would set the tracking settings for new markers in the toolshelf.
However, the tracking settings for the active track were in the properties sidebar to the right.
Solving and cleaning were on the left in the toolshelf.
So overall, the clip editor was using the concept of toolshelf and sidebar rather loosely.
Blender 2.80 is having a different, a bit more stricter approach to the toolshelf (now toolbar)
The toolbar is the place for active tools, the toolshelf Tabs are gone.
However, now we have Tabs in the sidebar.
So we need to update the overall design of the clip editor and also think about how we can adapt the tools and operators to the new workflow.

Use cases:

This change add the possibility of new/improved tools for working with motion tracking

Design:

Design is to match the toolbar found in the 3D viewport also see #58085 discussion

Work plan

{icon check-circle color=green} Milestone 1
Move properties in toolbar to sidebar

Time estimate: 2 weeks

D9806: UI: Tracking: Move Object Scale to objects panel
D9805: UI: Tracking: Remove operator options exposed as props
D9781: UI: Update Clip Tracking menus
D9729: Movie Clip: Annotation UI Improvements
D9807: UI: Tracking: Move default tracking settings to sidebar
D9808: UI: Tracking: Solve panel to Sidebar

{icon check-circle color=green} Milestone 2
Add new active tool based toolbar
Time estimate: 2 weeks

The following tools should only be relevant to the Clip View, not for Graph or Dopesheet View. There left-click+drag should only move the time cursor, just like in regular Dopesheet.

Add Marker
The closest thing to a new tool is probably Add Marker.
We used to just Ctrl+Click to add a marker, which is very convenient. Maybe we can keep that too.
However, the new toolsystem would make it possible to move all the settings to the topbar (and hence to the tool settings in properties editor as well).
So you choose the add marker tool. In the topbar you can adjust the settings for new markers. Here's a screenshot of the settings currently in the toolbar.

To adjust the settings of the active marker you can still use the track settings in the sidebar.
With the Add Marker tool active you should still be able to track.
EDIT: The Add Marker tool should not only add the marker, but also move it, as long as you have the mouse clicked. Just like Ctrl-Click now in 2.79.
The Clip Editor has a little playback area at the bottom. So the add marker tool would have to respect that, so that we have the tool active but can still drag the time cursor in the cache/playback area at the bottom of the editor.

Box Select
Another active tool could be box and lasso select.

Annotate
Annotate just as in other editors. It should, however, be still possible to assign the annotation to the mask (Data Source), for track masking.

Ideas for other tracking tools

There are some other tools I could think of, but those are maybe something for later and would require a bit more discussion.
For instance, maybe it would be nice to have an orientation tool.
Right now you first choose 3 tracks, then hit Floor, or Origin etc.. Depending on how that went you might have to choose 3 other markers and do that again. But I could imagine that it could also be a tool:

Activate Orientation tool. By default it would be set to "Floor". Once you picked 3 markers they will automatically be set as the Floor. (Maybe even with thin relationship line between them to indicate they are the floor?
Could have an option where you Ctrl-Click for Set Origin, Shift-Click for X-Axis and Alt-Click for Y-Axis.
Could have a mode to set the scale. Tool Setting would be the scale (default=1.0), tool would work in way that if you connect 2 tracks it will apply the scele to the scene.

Masking

Masking could benefit even more from the active tools concept than Tracking. But even though it is part of the clip editor I would rather open another Task for that.

Branch: none

**Status:** Work in progress --- **Team** **Commissioner:** `?` **Project leader:** `@Blendify **Description** **Big picture:** The current Movie Clip Editor still uses the old toolbar from 2.7x this task is to migrate that to the active tool based workflow Blender 2.80 is using a new toolshelf and toolsystem, so we need to update the toolshelf in Blender's clip editor too. The workflow for tracking in the clip editor used to be split between toolshelf on the left and sidebar on the right. In the toolshelf you would start setting up your scene length based on clip length, prefetch footage etc. Then you would set the tracking settings for new markers in the toolshelf. However, the tracking settings for the active track were in the properties sidebar to the right. Solving and cleaning were on the left in the toolshelf. So overall, the clip editor was using the concept of toolshelf and sidebar rather loosely. Blender 2.80 is having a different, a bit more stricter approach to the toolshelf (now toolbar) The toolbar is the place for active tools, the toolshelf Tabs are gone. However, now we have Tabs in the sidebar. So we need to update the overall design of the clip editor and also think about how we can adapt the tools and operators to the new workflow. **Use cases**: * This change add the possibility of new/improved tools for working with motion tracking **Design:** * Design is to match the toolbar found in the 3D viewport also see #58085 discussion **Work plan** **{icon check-circle color=green} Milestone 1** Move properties in toolbar to sidebar Time estimate: 2 weeks [D9806: UI: Tracking: Move Object Scale to objects panel](https://archive.blender.org/developer/D9806) [D9805: UI: Tracking: Remove operator options exposed as props](https://archive.blender.org/developer/D9805) [D9781: UI: Update Clip Tracking menus](https://archive.blender.org/developer/D9781) [D9729: Movie Clip: Annotation UI Improvements](https://archive.blender.org/developer/D9729) [D9807: UI: Tracking: Move default tracking settings to sidebar](https://archive.blender.org/developer/D9807) [D9808: UI: Tracking: Solve panel to Sidebar](https://archive.blender.org/developer/D9808) **{icon check-circle color=green} Milestone 2** Add new active tool based toolbar Time estimate: `2 weeks` The following tools should only be relevant to the Clip View, not for Graph or Dopesheet View. There left-click+drag should only move the time cursor, just like in regular Dopesheet. **Add Marker** The closest thing to a new tool is probably Add Marker. We used to just Ctrl+Click to add a marker, which is very convenient. Maybe we can keep that too. However, the new toolsystem would make it possible to move all the settings to the topbar (and hence to the tool settings in properties editor as well). So you choose the add marker tool. In the topbar you can adjust the settings for new markers. Here's a screenshot of the settings currently in the toolbar. ![tracking_settings.jpeg](https://archive.blender.org/developer/F5733882/tracking_settings.jpeg) To adjust the settings of the active marker you can still use the track settings in the sidebar. With the Add Marker tool active you should still be able to track. EDIT: The Add Marker tool should not only add the marker, but also move it, as long as you have the mouse clicked. Just like Ctrl-Click now in 2.79. The Clip Editor has a little playback area at the bottom. So the add marker tool would have to respect that, so that we have the tool active but can still drag the time cursor in the cache/playback area at the bottom of the editor. ![playback.jpg](https://archive.blender.org/developer/F5733944/playback.jpg) **Box Select** Another active tool could be box and lasso select. **Annotate** Annotate just as in other editors. It should, however, be still possible to assign the annotation to the mask (Data Source), for track masking. # Ideas for other tracking tools There are some other tools I could think of, but those are maybe something for later and would require a bit more discussion. For instance, maybe it would be nice to have an orientation tool. Right now you first choose 3 tracks, then hit Floor, or Origin etc.. Depending on how that went you might have to choose 3 other markers and do that again. But I could imagine that it could also be a tool: - Activate Orientation tool. By default it would be set to "Floor". Once you picked 3 markers they will automatically be set as the Floor. (Maybe even with thin relationship line between them to indicate they are the floor? - Could have an option where you Ctrl-Click for Set Origin, Shift-Click for X-Axis and Alt-Click for Y-Axis. - Could have a mode to set the scale. Tool Setting would be the scale (default=1.0), tool would work in way that if you connect 2 tracks it will apply the scele to the scene. ## Masking Masking could benefit even more from the active tools concept than Tracking. But even though it is part of the clip editor I would rather open another Task for that. **Branch**: none

Aaron Carlisle self-assigned this 2020-12-10 04:44:55 +01:00

Aaron Carlisle commented

2020-12-10 04:44:55 +01:00

Changed status from 'Needs Triage' to: 'Confirmed'

Aaron Carlisle commented

2020-12-10 04:44:55 +01:00

Added subscriber: @Blendify

blender-admin commented

2020-12-10 04:44:55 +01:00

#58085 was marked as duplicate of this issue

Aaron Carlisle commented

2020-12-10 05:08:04 +01:00

Added subscribers: @sebastian_k, @ideasman42, @hlorus, @AntonioBuch, @WilliamReynish, @SeanKennedy, @Sergey

Evan Wilson commented

2020-12-10 05:37:07 +01:00

Added subscriber: @EAW

Aaron Carlisle commented

2020-12-10 06:29:22 +01:00

For Milestone 2, the notes on Add Marker do not take into account the detect features operator that also allows adding markers.
Should this operator have its own options to configure markers or should it still use some centralized global settings for both?
The same question needs to be decided for the ctrl+click operator.

For Milestone 2, the notes on Add Marker do not take into account the detect features operator that also allows adding markers. Should this operator have its own options to configure markers or should it still use some centralized global settings for both? The same question needs to be decided for the ctrl+click operator.

Sean Kennedy commented

2020-12-10 23:22:31 +01:00

it would be nice to have an orientation tool

This is such a great idea! Something that you could adjust in realtime , watching those relationship lines, and press Enter to commit once it's all correct.

Regarding having a mode to set the scale, a very useful feature would be having a very basic human model appear, maybe 170 cm (6 ft) tall. Even if it was just an overlay at the origin point, like the outline of a human, it would be extremely useful. Scaling is something I have to guess at a lot when matchmoving VFX shots at work.

> it would be nice to have an orientation tool This is such a great idea! Something that you could adjust in realtime , watching those relationship lines, and press Enter to commit once it's all correct. Regarding having a mode to set the scale, a very useful feature would be having a very basic human model appear, maybe 170 cm (6 ft) tall. Even if it was just an overlay at the origin point, like the outline of a human, it would be extremely useful. Scaling is something I have to guess at a lot when matchmoving VFX shots at work.

Aaron Carlisle commented

2020-12-11 06:26:51 +01:00

Workflow for a set scale tool could be with the tool, you select two makers if you select a third marker the first becomes unselected. The scale is thwn set by the tool when pressing enter. This same workflow can work with other orientation based tools.

A human overlay feature would indeed be useful, although un related to this task.

Workflow for a set scale tool could be with the tool, you select two makers if you select a third marker the first becomes unselected. The scale is thwn set by the tool when pressing enter. This same workflow can work with other orientation based tools. A human overlay feature would indeed be useful, although un related to this task.

Aaron Carlisle commented

2020-12-12 05:40:37 +01:00

For those who would like to help review patches D9781 is going to be the biggest priority and the least controversial

For those who would like to help review patches [D9781](https://archive.blender.org/developer/D9781) is going to be the biggest priority and the least controversial

Sebastian Koenig commented

2021-01-15 16:13:19 +01:00

I have made a really crappy mockup of how I think in which direction the new MCE interface could go.

Here's how I think it could work:
The default tool would be Add Marker.
It replaces the Default Tracking Settings menu. The settings you enter in the Add Marker tool settings will be used as default for every new marker, including the ones for Detect Features.
Ctrl+T still works to add a new marker, even when a different tool is active. Ctrl+T will use the same settings as the Add Marker tool.
Maybe there could be variations of the Add Marker tool, such as Add Zero Weighted Marker tool, Add Reference Marker Tool etc.

Other tools can be the selection tools, like box select, lasso select.

The header toolbar should be enabled by default, so that the marker settings are accessible.

The solve buttons will also be in the header toolbar, on the right side. To solve a shot you can press the solve button in the header toolbar. I used the play icon for that in the mockup, which is also being used in the text editor to run a script. Of course we might need a different icon. Next to the solve button are the solve settings, such as tripod, keyframes, refine etc.
The Clean Up buttons could be in the header as well as a popover, or we put them into the sidebar.

Most other tools, such as Orientation, Geometry etc. can be accessed via Menu, Pie Menu, Context Menu etc.
Others can be put into the sidebar as well.

I would love to try some things out in python, but before I can do that, the Clip Editor needs to have a toolbar in the header. If someone could setup a branch for that with the basic tool system in place it would be great!

I have made a really crappy mockup of how I think in which direction the new MCE interface could go. ![mce_mockup.png](https://archive.blender.org/developer/F9580499/mce_mockup.png) Here's how I think it could work: The default tool would be Add Marker. It replaces the Default Tracking Settings menu. The settings you enter in the Add Marker tool settings will be used as default for every new marker, including the ones for Detect Features. Ctrl+T still works to add a new marker, even when a different tool is active. Ctrl+T will use the same settings as the Add Marker tool. Maybe there could be variations of the Add Marker tool, such as Add Zero Weighted Marker tool, Add Reference Marker Tool etc. Other tools can be the selection tools, like box select, lasso select. The header toolbar should be enabled by default, so that the marker settings are accessible. The solve buttons will also be in the header toolbar, on the right side. To solve a shot you can press the solve button in the header toolbar. I used the play icon for that in the mockup, which is also being used in the text editor to run a script. Of course we might need a different icon. Next to the solve button are the solve settings, such as tripod, keyframes, refine etc. The Clean Up buttons could be in the header as well as a popover, or we put them into the sidebar. Most other tools, such as Orientation, Geometry etc. can be accessed via Menu, Pie Menu, Context Menu etc. Others can be put into the sidebar as well. I would love to try some things out in python, but before I can do that, the Clip Editor needs to have a toolbar in the header. If someone could setup a branch for that with the basic tool system in place it would be great!

Aaron Carlisle commented

2021-01-15 16:44:26 +01:00

One thing that would have to be discussed with the UI team about your proposal is currently, tool settings only affect the current tool and not operators, so this would have to be discussed.

I like what you did with the solve pop-over but I think it belongs in the regular header and not the tool header because it is not tool-specific.

The Clean Up buttons could be in the header as well as a popover, or we put them into the sidebar.

Currently, we want to keep operators out of property regions (sidebar and properties editor) so I would say the pop-over idea is the best option

I would love to try some things out in python, but before I can do that, the Clip Editor needs to have a toolbar in the header. If someone could setup a branch for that with the basic tool system in place it would be great!

I started on this https://developer.blender.org/D10011 but it's not yet functional, I can try to finish that when I get a chance.

One thing that would have to be discussed with the UI team about your proposal is currently, tool settings only affect the current tool and not operators, so this would have to be discussed. I like what you did with the solve pop-over but I think it belongs in the regular header and not the tool header because it is not tool-specific. > The Clean Up buttons could be in the header as well as a popover, or we put them into the sidebar. Currently, we want to keep operators out of property regions (sidebar and properties editor) so I would say the pop-over idea is the best option > I would love to try some things out in python, but before I can do that, the Clip Editor needs to have a toolbar in the header. If someone could setup a branch for that with the basic tool system in place it would be great! I started on this https://developer.blender.org/D10011 but it's not yet functional, I can try to finish that when I get a chance.

Sebastian Koenig commented

2021-01-15 17:06:28 +01:00

In #83612#1094264, @Blendify wrote:
One thing that would have to be discussed with the UI team about your proposal is currently, tool settings only affect the current tool and not operators, so this would have to be discussed.

Ok, I am open for discussion. I would like to avoid though to have an extra Tracking Settings panel just for that reason in the sidebar. Personally I would not see a problem with having the settings from Add Marker tool be equivalent to the default tracking settings.

I like what you did with the solve pop-over but I think it belongs in the regular header and not the tool header because it is not tool-specific.

Ok, fair enough. I wanted to bring up the Dyntopo toggle button as an example against it, but yeah, that kind of affects the active tool as well, sooo, you are probably right :)

Currently, we want to keep operators out of property regions (sidebar and properties editor) so I would say the pop-over idea is the best option

Right. But I want to mention the Remesh button in sculpt mode as counter example, which is quite similar in scope and options as the clean up buttons. ;)

I started on this https://developer.blender.org/D10011 but it's not yet functional, I can try to finish that when I get a chance.

Awesome! I will check that out!

> In #83612#1094264, @Blendify wrote: > One thing that would have to be discussed with the UI team about your proposal is currently, tool settings only affect the current tool and not operators, so this would have to be discussed. Ok, I am open for discussion. I would like to avoid though to have an extra Tracking Settings panel just for that reason in the sidebar. Personally I would not see a problem with having the settings from Add Marker tool be equivalent to the default tracking settings. > I like what you did with the solve pop-over but I think it belongs in the regular header and not the tool header because it is not tool-specific. Ok, fair enough. I wanted to bring up the Dyntopo toggle button as an example against it, but yeah, that kind of affects the active tool as well, sooo, you are probably right :) > Currently, we want to keep operators out of property regions (sidebar and properties editor) so I would say the pop-over idea is the best option Right. But I want to mention the Remesh button in sculpt mode as counter example, which is quite similar in scope and options as the clean up buttons. ;) > I started on this https://developer.blender.org/D10011 but it's not yet functional, I can try to finish that when I get a chance. Awesome! I will check that out!

Hans Goudey commented

2021-01-15 17:31:24 +01:00

Added subscriber: @HooglyBoogly

Hans Goudey commented

2021-01-15 17:31:24 +01:00

In #83612#1094264, @Blendify wrote:
Currently, we want to keep operators out of property regions (sidebar and properties editor) so I would say the pop-over idea is the best option

Do you have a link about this? I agree that it's a good guideline in principle, but I've never heard it phrased like "keep operators out of property regions" before.
For example, looking through the property editor now, I see operators in almost every tab.

> In #83612#1094264, @Blendify wrote: > Currently, we want to keep operators out of property regions (sidebar and properties editor) so I would say the pop-over idea is the best option Do you have a link about this? I agree that it's a good guideline in principle, but I've never heard it phrased like "keep operators out of property regions" before. For example, looking through the property editor now, I see operators in almost every tab.

Aaron Carlisle commented

2021-01-15 20:35:55 +01:00

I do not know if it was written somewhere but it was the reason the render buttons were removed for example; 1cf17b257d

It also makes sense with the operator search which searches menus.

I do not know if it was written somewhere but it was the reason the render buttons were removed for example; 1cf17b257d It also makes sense with the operator search which searches menus.

Sergey Sharybin commented

2021-01-18 10:12:27 +01:00

On the tool settings and default tracking settings.

From my understanding, tools settings are not saved to .blend file. This means that if default tracking settings are replaced with tool settings, you would need to change tool settings every time you re-open a shot (re-opening Blender) and need to add new markers.

Not sure whether Sebastian suggests to to remove default tracking settings completely. If it is so, please first make it so tool settings are stored in .blend file.

I do not know if it was written somewhere

This is the very first thing to address: all the changes needs to be based on a written down design.

On the tool settings and default tracking settings. From my understanding, tools settings are not saved to .blend file. This means that if default tracking settings are replaced with tool settings, you would need to change tool settings every time you re-open a shot (re-opening Blender) and need to add new markers. Not sure whether Sebastian suggests to to remove default tracking settings completely. If it is so, please first make it so tool settings are stored in .blend file. > I do not know if it was written somewhere This is the very first thing to address: all the changes needs to be based on a written down design.

Sebastian Koenig commented

2021-01-18 10:49:52 +01:00

In #83612#1095302, @Sergey wrote:

From my understanding, tools settings are not saved to .blend file. This means that if default tracking settings are replaced with tool settings, you would need to change tool settings every time you re-open a shot (re-opening Blender) and need to add new markers.

That would be bad of course. Because yes, I was thinking to replace default tracking settings with the tool settings.
However, I just tested it in Sculptmode. I tweaked some brush parameters, such as radius, spacing etc. I saved two different files with different settings, and all of the settings were the same after I reopened Blender and opened those files.
Same with vertex paint.
So I guess it does work?

> In #83612#1095302, @Sergey wrote: > From my understanding, tools settings are not saved to .blend file. This means that if default tracking settings are replaced with tool settings, you would need to change tool settings every time you re-open a shot (re-opening Blender) and need to add new markers. That would be bad of course. Because yes, I was thinking to replace default tracking settings with the tool settings. However, I just tested it in Sculptmode. I tweaked some brush parameters, such as radius, spacing etc. I saved two different files with different settings, and all of the settings were the same after I reopened Blender and opened those files. Same with vertex paint. So I guess it does work?

Sergey Sharybin commented

2021-01-18 11:40:53 +01:00

Well, brush settings are stored in the brush.

But now you made me curios and I've tested with Inset and Extrude tools in Edit Mode. They do appear to save settings in .blend. Hopefully, this is by design :)

So nevermind then the comment about persistence.

Well, brush settings are stored in the brush. But now you made me curios and I've tested with Inset and Extrude tools in Edit Mode. They do appear to save settings in .blend. Hopefully, this is by design :) So nevermind then the comment about persistence.

Sebastian Koenig commented

2021-01-25 11:22:02 +01:00

@Sergey I managed to get the tool header to work in MCE! Now I will see if I manage to get a tool to work as well. I kind of doubt it though... :D https://pasteall.org/cDoN

@Sergey I managed to get the tool header to work in MCE! Now I will see if I manage to get a tool to work as well. I kind of doubt it though... :D ![mce_tools.jpeg](https://archive.blender.org/developer/F9596825/mce_tools.jpeg) https://pasteall.org/cDoN

RedMser commented

2021-10-10 13:32:35 +02:00

Added subscriber: @RedMser

Brecht Van Lommel commented

2022-02-28 17:31:27 +01:00

Added subscriber: @brecht

Brecht Van Lommel commented

2022-02-28 17:31:27 +01:00

Moving to next release since 3.1 is bug fixing only at this point.

Leon Lee commented

2022-12-06 12:48:58 +01:00

Added subscriber: @lileo

Philipp Oeser removed the

Download

What's New

Blender Studio

Manual

Developers Blog

Documentation

Benchmark

Blender Conference

Development Fund

One-time Donations

UI: Tracking: Migrate to active tools based toolbar #83612

Ideas for other tracking tools

Masking