Mesh-based armature interaction ("Mesh Gizmos", "Face Maps") #92218
Open
opened 2021-10-14 16:38:30 +02:00 by Demeter Dzadik
·
58 comments
No Branch/Tag Specified
main
blender-v4.0-release
temp-sculpt-brush-channel
temp-sculpt-dyntopo
blender-v3.6-release
universal-scene-description
blender-v3.3-release
asset-browser-frontend-split
brush-assets-project
asset-shelf
anim/armature-drawing-refactor-3
temp-sculpt-dyntopo-hive-alloc
tmp-usd-python-mtl
tmp-usd-3.6
blender-v3.5-release
blender-projects-basics
blender-v2.93-release
temp-sculpt-attr-api
realtime-clock
sculpt-dev
gpencil-next
bevelv2
microfacet_hair
xr-dev
principled-v2
v3.6.4
v3.6.3
v3.3.11
v3.6.2
v3.3.10
v3.6.1
v3.3.9
v3.6.0
v3.3.8
v3.3.7
v2.93.18
v3.5.1
v3.3.6
v2.93.17
v3.5.0
v2.93.16
v3.3.5
v3.3.4
v2.93.15
v2.93.14
v3.3.3
v2.93.13
v2.93.12
v3.4.1
v3.3.2
v3.4.0
v3.3.1
v2.93.11
v3.3.0
v3.2.2
v2.93.10
v3.2.1
v3.2.0
v2.83.20
v2.93.9
v3.1.2
v3.1.1
v3.1.0
v2.83.19
v2.93.8
v3.0.1
v2.93.7
v3.0.0
v2.93.6
v2.93.5
v2.83.18
v2.93.4
v2.93.3
v2.83.17
v2.93.2
v2.93.1
v2.83.16
v2.93.0
v2.83.15
v2.83.14
v2.83.13
v2.92.0
v2.83.12
v2.91.2
v2.83.10
v2.91.0
v2.83.9
v2.83.8
v2.83.7
v2.90.1
v2.83.6.1
v2.83.6
v2.90.0
v2.83.5
v2.83.4
v2.83.3
v2.83.2
v2.83.1
v2.83
v2.82a
v2.82
v2.81a
v2.81
v2.80
v2.80-rc3
v2.80-rc2
v2.80-rc1
v2.79b
v2.79a
v2.79
v2.79-rc2
v2.79-rc1
v2.78c
v2.78b
v2.78a
v2.78
v2.78-rc2
v2.78-rc1
v2.77a
v2.77
v2.77-rc2
v2.77-rc1
v2.76b
v2.76a
v2.76
v2.76-rc3
v2.76-rc2
v2.76-rc1
v2.75a
v2.75
v2.75-rc2
v2.75-rc1
v2.74
v2.74-rc4
v2.74-rc3
v2.74-rc2
v2.74-rc1
v2.73a
v2.73
v2.73-rc1
v2.72b
2.72b
v2.72a
v2.72
v2.72-rc1
v2.71
v2.71-rc2
v2.71-rc1
v2.70a
v2.70
v2.70-rc2
v2.70-rc
v2.69
v2.68a
v2.68
v2.67b
v2.67a
v2.67
v2.66a
v2.66
v2.65a
v2.65
v2.64a
v2.64
v2.63a
v2.63
v2.61
v2.60a
v2.60
v2.59
v2.58a
v2.58
v2.57b
v2.57a
v2.57
v2.56a
v2.56
v2.55
v2.54
v2.53
v2.52
v2.51
v2.50
v2.49b
v2.49a
v2.49
v2.48a
v2.48
v2.47
v2.46
v2.45
v2.44
v2.43
v2.42a
v2.42
v2.41
v2.40
v2.37a
v2.37
v2.36
v2.35a
v2.35
v2.34
v2.33a
v2.33
v2.32
v2.31a
v2.31
v2.30
v2.28c
v2.28a
v2.28
v2.27
v2.26
v2.25
Labels
Clear labels
This issue affects/is about backward or forward compatibility
Issues relating to security: https://wiki.blender.org/wiki/Process/Vulnerability_Reports
Apply labels
Interest
Alembic
Interest
Animation & Rigging
Interest
Asset Browser
Interest
Asset Browser Project Overview
Interest
Audio
Interest
Automated Testing
Interest
Blender Asset Bundle
Interest
BlendFile
Interest
Collada
Interest
Compatibility
This issue affects/is about backward or forward compatibility
Interest
Compositing
Interest
Core
Interest
Cycles
Interest
Dependency Graph
Interest
Development Management
Interest
EEVEE & Viewport
Interest
Freestyle
Interest
Geometry Nodes
Interest
Grease Pencil
Interest
ID Management
Interest
Images & Movies
Interest
Import Export
Interest
Line Art
Interest
Masking
Interest
Metal
Interest
Modeling
Interest
Modifiers
Interest
Motion Tracking
Interest
Nodes & Physics
Interest
OpenGL
Interest
Overrides
Interest
Performance
Interest
Physics
Interest
Pipeline, Assets & IO
Interest
Platforms, Builds & Tests
Interest
Python API
Interest
Render & Cycles
Interest
Render Pipeline
Interest
Sculpt, Paint & Texture
Interest
Text Editor
Interest
Translations
Interest
Triaging
Interest
Undo
Interest
USD
Interest
User Interface
Interest
UV Editing
Interest
VFX & Video
Interest
Video Sequencer
Interest
Virtual Reality
Interest
Vulkan
Interest: Wayland
Legacy
Blender 2.8 Project
Legacy
Milestone 1: Basic, Local Asset Browser
Legacy
OpenGL Error
Meta
Good First Issue
Meta
Papercut
Meta
Retrospective
Meta
Security
Issues relating to security: https://wiki.blender.org/wiki/Process/Vulnerability_Reports
Module
Animation & Rigging
Module
Core
Module
Development Management
Module
EEVEE & Viewport
Module
Grease Pencil
Module
Modeling
Module
Nodes & Physics
Module
Pipeline, Assets & IO
Module
Platforms, Builds & Tests
Module
Python API
Module
Render & Cycles
Module
Sculpt, Paint & Texture
Module
Triaging
Module
User Interface
Module
VFX & Video
Platform
FreeBSD
Platform
Linux
Platform
macOS
Platform
Windows
Priority
High
Priority
Low
Priority
Normal
Priority
Unbreak Now!
Status
Archived
Status
Confirmed
Status
Duplicate
Status
Needs Info from Developers
Status
Needs Information from User
Status
Needs Triage
Status
Resolved
Type
Bug
Type
Design
Type
Known Issue
Type
Patch
Type
Report
Type
To Do
No Label
Interest
Alembic
Interest
Animation & Rigging
Interest
Asset Browser
Interest
Asset Browser Project Overview
Interest
Audio
Interest
Automated Testing
Interest
Blender Asset Bundle
Interest
BlendFile
Interest
Collada
Interest
Compatibility
Interest
Compositing
Interest
Core
Interest
Cycles
Interest
Dependency Graph
Interest
Development Management
Interest
EEVEE & Viewport
Interest
Freestyle
Interest
Geometry Nodes
Interest
Grease Pencil
Interest
ID Management
Interest
Images & Movies
Interest
Import Export
Interest
Line Art
Interest
Masking
Interest
Metal
Interest
Modeling
Interest
Modifiers
Interest
Motion Tracking
Interest
Nodes & Physics
Interest
OpenGL
Interest
Overrides
Interest
Performance
Interest
Physics
Interest
Pipeline, Assets & IO
Interest
Platforms, Builds & Tests
Interest
Python API
Interest
Render & Cycles
Interest
Render Pipeline
Interest
Sculpt, Paint & Texture
Interest
Text Editor
Interest
Translations
Interest
Triaging
Interest
Undo
Interest
USD
Interest
User Interface
Interest
UV Editing
Interest
VFX & Video
Interest
Video Sequencer
Interest
Virtual Reality
Interest
Vulkan
Interest: Wayland
Legacy
Blender 2.8 Project
Legacy
Milestone 1: Basic, Local Asset Browser
Legacy
OpenGL Error
Meta
Good First Issue
Meta
Papercut
Meta
Retrospective
Meta
Security
Module
Animation & Rigging
Module
Core
Module
Development Management
Module
EEVEE & Viewport
Module
Grease Pencil
Module
Modeling
Module
Nodes & Physics
Module
Pipeline, Assets & IO
Module
Platforms, Builds & Tests
Module
Python API
Module
Render & Cycles
Module
Sculpt, Paint & Texture
Module
Triaging
Module
User Interface
Module
VFX & Video
Platform
FreeBSD
Platform
Linux
Platform
macOS
Platform
Windows
Priority
High
Priority
Low
Priority
Normal
Priority
Unbreak Now!
Status
Archived
Status
Confirmed
Status
Duplicate
Status
Needs Info from Developers
Status
Needs Information from User
Status
Needs Triage
Status
Resolved
Type
Bug
Type
Design
Type
Known Issue
Type
Patch
Type
Report
Type
To Do
Milestone
Set milestone
Clear milestone
No items
No Milestone
Projects
Set Project
Clear projects
No project
Assignees
Assign users
Clear assignees
No Assignees
31 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.
No due date set.
Dependencies
No dependencies set.
Reference: blender/blender#92218
Reference in New Issue
There is no content yet.
Delete Branch "%!s(<nil>)"
Deleting a branch is permanent. Although the deleted branch may exist for a short time before cleaning up, in most cases it CANNOT be undone. Continue?
Preface
The current way of interacting with bones in a production environment is based on Custom Shape Objects, which can disappear inside the mesh, be at a completely different place from the area they control, and in general require a fair bit of design from the rigger and learning from the animator to efficiently use.
Using the Python Gizmo API, I created a prototype of an alternative system, where we can click on geometry on the deformed mesh itself to select bones and control said mesh.
The goals of this task are:
The Prototype: "Bone Gizmos" add-on
Video: Setting up a single bone gizmo
Video: The add-on in use
You can download the addon as part of the Blender Studio Pipeline. You have to download the whole repository from the repo's home page, then extract the
bone_gizmos
folder into your addons folder.Features
A list of proposed features, partially working in the prototype, although there is plenty of space for UX discussions.
Design Questions:
While making the prototype, I ran into some design questions where the answers aren't so simple:
There might be no perfect answer to all of these, but they are things to keep in mind and mitigate.
Prototype Limitations
These issues are listed just to clarify that these behaviours are known, and would most likely be easy to fix/avoid in the final implementation, but can't really be addressed in the prototype:
Feedback and discussion welcome! Also about how to structure this document.
Added subscriber: @Mets
Changed status from 'Needs Triage' to: 'Confirmed'
Added subscriber: @LucianoMunoz
Added subscriber: @AndyCuccaro
Added subscriber: @GeorgiaPacific
Added subscriber: @Phigon
Added subscriber: @MaybeLamia
Added subscriber: @PaoloAcampora
Added subscriber: @L0Lock
Added subscriber: @RomboutVersluijs
This looks like tha initial FaceMap idea, what happened to that?
I believe the initial setup was made to "be left to others".
Or rather, specifically the Face attribute/python-property was for that, and the selector thing (like here) was meant to just be a proof of concept, which either succeeded or was abandoned. If the goal was to just show it was possible/an example, it succeeded; if the goal was to have a basic practical/functional version, it was abandoned.
Added subscriber: @HooglyBoogly
I wasn't around when face maps were added, but everything I've seen makes me think that they should just be a generic integer attribute rather than a special thing with yet another list in mesh properties and its own operators, etc.
To make that work, I think we would just have to support assigning values to attributes in edit mode, which is something we have planned anyway.
Just looked at face maps again and that earlier proposal seems to work out of the box by itself. This approach needs a user to set it up manual, if i understood the other one correct. Both seem to have the same goal.
The way I read your post is: "The first addon already works as-is, and this just does the same thing but with more steps".
So I was wondering if maybe the addon was updated since I last tried it, and/or I was just completely using it wrong and misremembering it.
I wasn't. I forgot the exact problems but remembered correctly that it's not practical for usage......and crashes Blender. #51675 is that development page.
My problems started with it is there not being an automated way to transfer bone-vertex groups to face maps, and before I would bother creating a script/addon to do that, I would need to verify that I would use it.
Then I find out it only works in object mode, it doesn't register an undo, and the only method of interaction is click-drag, set to Translation and only does Rotation/Scale if the Location then Rotation is locked on the bone.
Then after getting tripped up with the undo and entering the rig to clear the transforms, soon Blender crashes.
It looks great to finally be able to pose bones by just selecting the mesh itself, and without using the slow Mask Modifier method but too instable and impractical to be used right now.
That's my current thoughts and it's my same thoughts from back in 2.80.
Relooking into it, apparently there were design decisions being discussed about making this possible, and then the dev decided to just make the addon as a proof-of-concept (as stated in the addon's warning), and that was the end of it; and that this was a little over 3 years ago. There was later another page #54989 about this stuff but supposedly it's aim was slightly different, and also 3 years ago. I didn't hear about that until today, and last info on that was it would also be scrapped, last year.
I have high hopes that this time it will get done, all the way, one way or another.
The downsides of vertex groups are:
So I agree, the justification for the existence of Face Maps doesn't look too good.
I didn't know about those threads, so thanks for digging them up! Indeed, both seem fairly abandoned, so you can consider this a revival of those. A lot of the ideas and the conclusions that people came to in those threads seem to align with what I did in the prototype addon, so that's lucky!
Apparently Ton wanted things to work in Object mode. The LilyGizmos addon does a fantastic job of this already, but I would consider this a separate system:
Gizmos for sliding floats != Gizmos for interacting with bones.
Added subscriber: @ErickNyanduKabongo
Added subscriber: @Capitanblue89
Added subscriber: @Mamoun88
Removed subscriber: @Mamoun88
Added subscriber: @rwman
Added subscriber: @Jeroen-Bakker
Added subscriber: @g-lul
Added subscriber: @jeremybep
Added subscriber: @wevon-2
Just a couple of ideas.

If the preselection were gradual as the cursor approaches the Face Maps, this could be done by progressively illuminating the contour of the Face Maps until the cursor is over the face group and the faces are already illuminated, in this way I think that The Christmas tree effect would be avoided.
Second, Vertex Maps could also be used to animate details. In the same way, when approaching the cursor to the vertex sets they could gain opacity.
I have made a video with a non-real face, to show the effect that could be produced when moving the cursor closer to certain vertices or edges. It is less blinking than with faces, although in a way everything is complementary.
kk300.mp4
Added subscriber: @pauanyu_blender
Added subscriber: @ArtemBataev
Added subscriber: @JohanTriHandoyo
Love the idea. I think this will avoid cluttered in rig and keep only the nearest and necessary controller visible at the same time +1
Added subscriber: @RedMser
Added subscriber: @Commander-Mustard
Added subscriber: @Richard-Lee
Added subscriber: @zNight
I've made some tweaks to the add-on, added a new video about how to set up a single bone gizmo, and re-structured the task description. I think it's clear to me now that this task shouldn't be about improving the Python Gizmo API (although that is still welcome). Instead, mesh gizmos NEED to be a core Blender feature to have a chance to shine, especially due to performance. So the task has been re-focused to present the add-on purely as a prototype/testing ground of a potential future system.
I would like to bring this up in a module meeting soon, and try to figure out what could be the next step from here and when that would be done and by whom. I think my main question will be this:
Do we do more in-depth design discussions and testing with this, or is this already good enough to throw a developer at it to come up with a technical roadmap or just really go ham and implement a better prototype in C/C++, and iterate from there?
Added subscriber: @Nika-Kutsniashvili
Added subscriber: @leriaz
Yo @Mets How this project is going? You had some questions that it seems like there is no answer?
Added subscriber: @Zophiekat
i think it would be cool if we had more control over visibility, like having per bone options for disabling highlight blinking, per bone xray visiblity, and gizmo outline
Do you plan to work on this add-on until this becomes Blender feature (if ever)? It is amazing but has some bugs. And could there be a way to automatically select corresponding vertex group for all bones, and hence display it on all bones at once?
I'm currently not getting any interest from either my team of animators or any developers, to work on this further, either as a usable add-on or as a prototype for a core Blender feature.
I think as a prototype it already works okay, but until there is a developer who wants to and is able to and is available to implement it in core Blender, this will continue collecting dust in my backlog.
But still, if you find bugs that aren't listed under the "Prototype Limitations" paragraph in the post, feel free to describe them, I would be curious and might get around to fixing them sometime.
That is an absolute shame. This is one of the most anticipated feature in Blender. Literally every blender video on YouTube about future of animation includes your video. People are hyped and its a shame if we'll be left disappointed. Shows the gap in priorities between developers and artists in the community.
I would advise you to upload the addon on GitHub, GumRoad or someplace else and spread the word about it a little, post it on blenderartists. Very few know that they can already have this feature. If developers don't appreciate it community will.
As for the bugs, two I noticed is that parts of the mesh (in forms of triangles) are sometimes visible through highlight. I guess offset option could be used to fix that.
And second is that updating colors in the panel doesn't update automatically in the viewport. I need to change line thickness in the add-on settings for it to update.
Added subscriber: @BClark
Well I wouldn't say it isn't getting any interest, it is part of the main animation push, improving selection and mesh based selection possibly. I didn't know it was already this far along and so I just shared it back and will remind people in the Animation Module this exists. No timeline or promises but feedback on a working prototype would be great for moving it into core of Blender.
Added subscriber: @JasonSchleifer
I agree with Brad - it's definitely got interest! I'm super interested in it! Let's set up a session to talk about it & give you feedback!
I still cannot figure out how to install it. I have a folder on my computer with the 7 .py files and placed it in my add on folders. That did not work. The directions "just extract the files into your Blender add ons folder" does not explain how to get the bone_gizmos folder from github! PLEASE HELP!!! What am I doing wrong? I wish I could just download a zip and put it in my add ons in preferences like all other add ons. Can someone please show me how to download the file ? Any help is super appreciated. Thanks.
Hi, I have am adding bone_gizmos to Expy Kit, my add-on for quick character setup. If you want to try it, please download and install the attached add-on: it's an early test but it seems to work:
Expy-Kit-expy_gizmos_0122.zip
https://github.com/pKrime/Expy-Kit/tree/expy_gizmos
If you are using Rigify, then in Pose Mode:
Here's an instant setup on SF-girl, a CC-by model by Stan that I downloaded from sketchfab
https://sketchfab.com/3d-models/sf-girl-a219070284e64ddd8a51f69c207cc81b
https://sketchfab.com/Stas_SayHallo
Expy_Pizmo_Example.mp4
Expy Kit supports other humanoid rigs, like Unreal Mannequin and Mixamo. If you are using a non supported rig, you can still set up your own, more info in my videos (sorry, no manual yet)
https://youtu.be/auTkEEK6Dmo
if you have installed @Mets's original add-on from the Blender Studio Tools it must be disabled or removed: Expy Kit contains a copy of it and there is no compatibility at the moment.
I have also noticed a weird offset in blender 3.2, not sure where it comes from.
enjoy!
Added subscriber: @AdamEarle
I know I'm giving away my trade secrets here, so here it is. If you expose the vertex matrix so it can be keyed framed without the AnimAll addon it makes for really great use being able to finalize your polish pass.
The idea here once you have your animation down. you can then come back in over the top and use the sculpt brush to keyframe the verts for cleaning. This is a much faster process than creating a shape key. The only downside is you can't keyframe mesh data in the pose library.
I mention this because I'm not sure how this is going affect the current system that is being implemented, and it's something that should be considered.
What do you mean by vertex matrix?
just bumping this thread. Vertex matrix means being able to animate the vertex directly like the animall addon allows.
I found the Expy kit, it is a bit sluggish on my system. I still cannot find the bone gizmo add on though, does anyone have the link?
Updated links in the post.
Hi @Richard-Lee, did you manage to find the Gizmos branch? It's very close to inclusion. Also, I have kind of made the UI faster for the next release, until then, please hide the Expy tab during playbacks.
cheers!
p.
Expy Kit Gizmos is working, and yes, closing the gizmos panel helped speed things up. Thank you.