Thoughts on Area Activation via Interaction #115158

New Issue

Harley Acheson · 2023-11-20T01:17:02+01:00

Harley Acheson commented

2023-11-20 01:17:02 +01:00

This is more of a thought experiment, or discussion starter, than a design.

Our user interface consists of multiple editors in screen areas that allow us to do vastly different operations within a single application. Only one area can be "active" at a time, and it is indicated with a slightly highlighted header color. Our use of "active" for areas is mostly to designate which part of the program gets keyboard focus, and is therefore the recipient of keyboard events like text input and operator shortcuts. Since users usually have only a single keyboard it makes sense that we need to designate a single area as having keyboard focus.

However there are problems now - and more coming later - by our use of mouse position to indicate which area has focus. An example we all have noticed is when using the Text Editor. It only accepts keyboard entry while the mouse pointer is positioned within that area. If your mouse creeps out of the area your typing suddenly stops.

This is more than a simple annoyance with one editor, but a general problem of tying input focus to the position of one particular pointing device type. There are many types of pointing devices, and many do not have a "tracking" state where it is moved around and is always persistent. When a graphics tablet stylus is lifted it is out of range and does not have a persistent position. Similarly a touch surface also does not keep a position when you pull your fingers away from it. If you imagine using an eye-tracking device you probably don't think of pushing a mouse cursor around so that you can leave it parked in one of our areas.

We need to find an area activation model that works with any number of multiple pointer devices, including the situation of having no mouse at all like when using a touch surface.

Area Activation via Interaction

The current definition of the active area is that it is the one that "contains the mouse" . We could change that to "the area being interacted with". Specifically this means that we no longer set active area using mouse hover, but instead do so with all non-tracking pointer events. Basically all mouse-like events that are not hover. So pressing, dragging and similar events associated with user action, not just movement. An inactive area would become active the first instant that you press down with a mouse button, stylus, or finger.

In the Text Editor example you could continue to type even if your mouse strays from the area. Or if you have no mouse at all. Add a flashing text caret and it would seem quite civilized.

Active Area Indication

We currently show which area is active in a very subtle way. This is entirely intentional as we don't want anything having large visual changes just by moving your mouse from place to place. The best example is moving your mouse from Properties to the Top Bar, we don't want to see big changes to the 3DViewport as the mouse moves over it. Or see edges flashing as you cross them.

However, with this change to area activation, areas don't become active with just a mouse travelling over it. That means that activation has become more intentional and with that we can add stronger indication. In fact it might require stronger indication. We might want to show the borders of the active area in a different (or brighter) color. Or perhaps show a line in the active color (blue by default) along its top edge.

Default Active Area

On Window creation we would need to set one area (the largest) as active immediately.

Unambiguous Area Context

When you currently move your mouse to the TopBar it becomes the active area. Therefore it can't contain operators that work on other areas. But with this change we could set TopBar to never become active. This would require moving the Scene and View Layer selectors elsewhere, which has been discussed as possible.

This means that if a Node Editor is active and you move your mouse to the Top Bar, it continues to remain active. We could choose to add global toolbars that act on specific areas if active, showing different ones depending on the active area. It is conceivable that menus could be added and removed as different areas are active.

It opens up a lot of possibilities when we have areas that remain active until intentional user action.

This is more of a thought experiment, or discussion starter, than a design. Our user interface consists of multiple editors in screen areas that allow us to do vastly different operations within a single application. Only one area can be "active" at a time, and it is indicated with a slightly highlighted header color. Our use of "active" for areas is _mostly_ to designate which part of the program gets keyboard focus, and is therefore the recipient of keyboard events like text input and operator shortcuts. Since users usually have only a single keyboard it makes sense that we need to designate a single area as having keyboard focus. However there are problems now - and more coming later - by our use of _mouse position_ to indicate which area has focus. An example we all have noticed is when using the Text Editor. It only accepts keyboard entry while the mouse pointer is positioned within that area. If your mouse creeps out of the area your typing suddenly stops. This is more than a simple annoyance with one editor, but a general problem of tying input focus to the position of one particular pointing device type. There are many types of pointing devices, and many do not have a "tracking" state where it is moved around and is always persistent. When a graphics tablet stylus is lifted it is out of range and does not have a persistent position. Similarly a touch surface also does not keep a position when you pull your fingers away from it. If you imagine using an eye-tracking device you probably don't think of pushing a mouse cursor around so that you can leave it parked in one of our areas. We need to find an area activation model that works with any number of multiple pointer devices, including the situation of having no mouse at all like when using a touch surface. ### Area Activation via Interaction The current definition of the active area is that it is the one that "**contains the mouse**" . We could change that to "**the area being interacted with**". Specifically this means that we no longer set active area using mouse hover, but instead do so with all non-tracking pointer events. Basically all mouse-like events that are **not** hover. So pressing, dragging and similar events associated with user action, not just movement. An inactive area would become active the first instant that you press down with a mouse button, stylus, or finger. In the Text Editor example you could continue to type even if your mouse strays from the area. Or if you have no mouse at all. Add a flashing text caret and it would seem quite civilized. ### Active Area Indication We currently show which area is active in a very subtle way. This is entirely intentional as we don't want anything having large visual changes just by moving your mouse from place to place. The best example is moving your mouse from Properties to the Top Bar, we don't want to see big changes to the 3DViewport as the mouse moves over it. Or see edges flashing as you cross them. However, with this change to area activation, areas don't become active with just a mouse travelling over it. That means that activation has become more intentional and with that we can add stronger indication. In fact it might require stronger indication. We might want to show the borders of the active area in a different (or brighter) color. Or perhaps show a line in the active color (blue by default) along its top edge. ### Default Active Area On Window creation we would need to set one area (the largest) as active immediately. ### Unambiguous Area Context When you currently move your mouse to the TopBar it becomes the active area. Therefore it can't contain operators that work on other areas. But with this change we could set TopBar to never become active. This would require moving the Scene and View Layer selectors elsewhere, which has been discussed as possible. This means that if a Node Editor is active and you move your mouse to the Top Bar, it continues to remain active. We could choose to add global toolbars that act on specific areas if active, showing different ones depending on the active area. It is conceivable that menus could be added and removed as different areas are active. It opens up a lot of possibilities when we have areas that remain active until intentional user action.

👀 3 🎉 2

Harley Acheson added the

Type

Design

label 2023-11-20 01:17:02 +01:00

Harley Acheson added this to the User Interface project 2023-11-20 01:17:04 +01:00

RedMser commented

2023-11-20 16:39:56 +01:00

we no longer set active area using mouse hover, but instead do so with all non-tracking pointer events. Basically all mouse-like events that are not hover. So pressing, dragging and similar events associated with user action, not just movement.

This specific part would complicate certain workflows. What comes to mind:

Accidentially changing selection/3d cursor position in 3d view as you try to give it focus
Animation in 3d requires a lot of back and forth between editors (dopesheet, NLA, graph, 3dview). Shortcuts like I to insert key work very differently between these views, so you must stay aware in what kind of editor your focus is currently in.
Same goes for UV editing and texture painting: rotating your view would focus the 3d view, but shortcuts in the uv/image editor would not, so you'd need to remember to re-focus it to use shortcuts there.

However I do agree that for the text inputs (editing a text datablock, using the text editor, or the python console), mouse focus vis hover is not desired.

Would this new interaction paradigm need to be consistent through all input devices, and for the entirety of the software? Since it might be feasible to only follow it for devices that have no other means of indicating a cursor position, like the original goal of this task.

Or only apply it to those places where text input is the primary action. Shortcuts in Blender are often as powerful as actions with the mouse, so surely pointer interactions should not be the only way to indicate focus? If I hover the 3D view and press shift+A, my intention is to add a mesh to the scene, nothing else.

> we no longer set active area using mouse hover, but instead do so with all non-tracking pointer events. Basically all mouse-like events that are not hover. So pressing, dragging and similar events associated with user action, not just movement. This specific part would complicate certain workflows. What comes to mind: - Accidentially changing selection/3d cursor position in 3d view as you try to give it focus - Animation in 3d requires a lot of back and forth between editors (dopesheet, NLA, graph, 3dview). Shortcuts like I to insert key work very differently between these views, so you must stay aware in what kind of editor your focus is currently in. - Same goes for UV editing and texture painting: rotating your view would focus the 3d view, but shortcuts in the uv/image editor would not, so you'd need to remember to re-focus it to use shortcuts there. However I do agree that for the text inputs (editing a text datablock, using the text editor, or the python console), mouse focus vis hover is not desired. Would this new interaction paradigm need to be consistent through all input devices, and for the entirety of the software? Since it might be feasible to only follow it for devices that have no other means of indicating a cursor position, like the original goal of this task. Or only apply it to those places where text input is the primary action. Shortcuts in Blender are often as powerful as actions with the mouse, so surely pointer interactions should not be the only way to indicate focus? If I hover the 3D view and press shift+A, my intention is to add a mesh to the scene, nothing else.

Harley Acheson commented

2023-11-20 17:52:16 +01:00

Thanks for thinking about this.

How it differs from now can be a bit subtle. But my hope is that there would be very few times you would need to explicitly select an area (by clicking the header for example) where it would be done by mouse hover now, and that would be offset enough by having less times that you need to select an area because it moved accidentally. And the ability to have stronger indication might reduce the burden too. Who knows. The reason for this is to think about it, not really propose the change.

Would this new interaction paradigm need to be consistent through all input devices

I'm not sure. It might be possible to do this and then backtrack a bit and make only "mouse enter" and "mouse leave" have special behavior. That way people with mice can work as now but those with only pens or touch can explicitly select areas. But that might also rob us of some utility this gives us. For example we couldn't then have a toolbar with items that work on the active area. There could be in-between solutions where one editor in a workspace is treated specially.

Thanks for thinking about this. How it differs from now can be a bit subtle. But my hope is that there would be very few times you would need to explicitly select an area (by clicking the header for example) where it would be done by mouse hover now, and that would be offset enough by having less times that you need to select an area because it moved accidentally. And the ability to have stronger indication might reduce the burden too. Who knows. The reason for this is to think about it, not really propose the change. > Would this new interaction paradigm need to be consistent through all input devices I'm not sure. It might be possible to do this and then backtrack a bit and make only "mouse enter" and "mouse leave" have special behavior. That way people with mice can work as now but those with only pens or touch can explicitly select areas. But that might also rob us of some utility this gives us. For example we couldn't then have a toolbar with items that work on the active area. There could be in-between solutions where one editor in a workspace is treated specially.

👍 4

Sign in to join this conversation.

No Label

Download

What's New

Blender Studio

Manual

Developers Blog

Documentation

Benchmark

Blender Conference

Development Fund

One-time Donations

Thoughts on Area Activation via Interaction #115158

Area Activation via Interaction

Active Area Indication

Default Active Area

Unambiguous Area Context