Mesh Edit: implement an operator to smooth shape key deformation. #110275

Open
Alexander Gavrilov wants to merge 1 commits from angavrilov:pr-shapekey-smoothing into main

When changing the target branch, be careful to rebase the branch in your fork to match. See documentation.

Similar to the difference between Corrective Smooth and simple Smooth
modifiers, when editing shape keys it sometimes makes sense to smooth
the deformation defined by the key rather than the final shape.

This implements a simple shape key smoothing operator, accessible
via the Vertex menu next to Blend From Shape. The operator applies
a simple iterative smoothing process (identical to the one implemented
by the Smooth tool) to the offsets of vertices from the relative basis
key rather than their positions.

In order to provide more flexibility for dealing with shape keys
intended to be used in conjunction with other ones, the operator
provides an option to modify the relative basis used for smoothing
via selecting another key. The following modes are supported:

  • Only Active Key
    The default mode smoothes the deformation of the key itself.
    The additional reference key is not used.

  • Include Key
    Includes the deformation of the reference key into the smoothing.

    This can be used when the key currently being edited is intended to
    be applied on top of another one to smooth their combined deformation.

  • Exclude Key
    This excludes the deformation of the reference key from smoothing.

    Intended for the opposite case of smoothing the difference between
    two keys being intended to be used alternatively to each other.

  • Relative To Key
    This effectively overrides the Relative To setting of the active key,
    simply replacing the basis key for the purpose of this operator only.

    If both keys have the same Relative To basis, this mode behaves
    the same as Exclude Key.

Like ordinary Smooth, the operator supports locking smoothing of certain
axis directions. In addition, it supports restricting direction based on
the vertex normal to allow separately smoothing volume and topology.

Similar to the difference between Corrective Smooth and simple Smooth modifiers, when editing shape keys it sometimes makes sense to smooth the deformation defined by the key rather than the final shape. This implements a simple shape key smoothing operator, accessible via the Vertex menu next to Blend From Shape. The operator applies a simple iterative smoothing process (identical to the one implemented by the Smooth tool) to the offsets of vertices from the relative basis key rather than their positions. In order to provide more flexibility for dealing with shape keys intended to be used in conjunction with other ones, the operator provides an option to modify the relative basis used for smoothing via selecting another key. The following modes are supported: * **Only Active Key** The default mode smoothes the deformation of the key itself. The additional reference key is not used. * **Include Key** Includes the deformation of the reference key into the smoothing. This can be used when the key currently being edited is intended to be applied on top of another one to smooth their combined deformation. * **Exclude Key** This excludes the deformation of the reference key from smoothing. Intended for the opposite case of smoothing the difference between two keys being intended to be used alternatively to each other. * **Relative To Key** This effectively overrides the Relative To setting of the active key, simply replacing the basis key for the purpose of this operator only. If both keys have the same Relative To basis, this mode behaves the same as Exclude Key. Like ordinary Smooth, the operator supports locking smoothing of certain axis directions. In addition, it supports restricting direction based on the vertex normal to allow separately smoothing volume and topology.
Alexander Gavrilov added this to the Modeling project 2023-07-19 21:46:24 +02:00
Alexander Gavrilov added the
Interest
Animation & Rigging
label 2023-07-19 21:46:35 +02:00
Author
Member

@blender-bot package

@blender-bot package
Member

Package build started. Download here when ready.

Package build started. [Download here](https://builder.blender.org/download/patch/PR110275) when ready.
Author
Member

@blender-bot package

@blender-bot package
Member

Package build started. Download here when ready.

Package build started. [Download here](https://builder.blender.org/download/patch/PR110275) when ready.
Member

I tried sculpting some shape keys on a cylinder, on Sintel's face, and on a subdivided plane, but I can't quite get a solid idea of what scenario some of these features would be useful in, but here's a couple things that I found neat:


In addition, it supports restricting direction based on
the vertex normal to allow separately smoothing volume and topology.

This is a big winner. Volume preserving smoothing is currently only possible with a sculpt brush, which is fine, but having it in an operator like this is also very handy, since it's easier to apply a uniform amount of smoothing to a selected area. (I'm sure that's also possible in sculpt mode, but probably requires quite a lot of steps.)


Include Key: Making one shape key play nicely with another

Here I used an "up" and "wide" example, where it's assumed that the "up" deformation activates first, and then the "wide". The resulting "wide" shape is useless and non-sensical on its own; Because it was smoothed relative to "up", it is only expected to give a desirable shape when "up" is already activated.

This made me realize that this could be handy not for combining entirely different shapes, but rather, for pushing an already existing shape to a further extreme. So, a better example would be a "LipsUp" shape key, and then a "LipsUpExtreme" shape key. Sculpting a more extreme shape is always harder, but the ability to smooth based on the original LipsUp shape would make that a lot easier.


So I'm sold on those things, but I'm still not sure about what would be a good use case for Exclude Key, and Relative To. Could you maybe propose a concrete or semi-concrete use case for those?


Also, is Only Active Key the same as a laplacian smooth?


A proposal: What about the ability to smooth relative not to another shape key, but to the combined shape of all currently active shape keys except the current one? I think this would handy for corrective shapes, which is what I was trying to achieve earlier; Let's say I have a LipsWide and a LipsUp shape, and I want to create the corrective that activates when both of those are activated at the same time. I would want to smooth relative to the shape that is the combined result of those two shapes. I hope that makes sense. But of course you don't have to implement that if you don't want to, I just thought I'd propose it since you're already doing something vaguely similar.

I tried sculpting some shape keys on a cylinder, on Sintel's face, and on a subdivided plane, but I can't quite get a solid idea of what scenario some of these features would be useful in, but here's a couple things that I found neat: ----------------------------------------------------- > In addition, it supports restricting direction based on the vertex normal to allow separately smoothing volume and topology. <video src="/attachments/4593c8e2-8be2-4d99-a283-55040a3efebd" title="2023-07-20 15-56-32.mp4" controls></video> This is a big winner. Volume preserving smoothing is currently only possible with a sculpt brush, which is fine, but having it in an operator like this is also very handy, since it's easier to apply a uniform amount of smoothing to a selected area. (I'm sure that's also possible in sculpt mode, but probably requires quite a lot of steps.) ------------------------------------------------------- **Include Key**: Making one shape key play nicely with another <video src="/attachments/bbe3a8e0-de07-4432-9947-a688f6f32f1f" title="2023-07-20 16-05-59.mp4" controls></video> Here I used an "up" and "wide" example, where it's assumed that the "up" deformation activates first, and then the "wide". The resulting "wide" shape is useless and non-sensical on its own; Because it was smoothed relative to "up", it is only expected to give a desirable shape when "up" is already activated. This made me realize that this could be handy not for combining entirely different shapes, but rather, for pushing an already existing shape to a further extreme. So, a better example would be a "LipsUp" shape key, and then a "LipsUpExtreme" shape key. Sculpting a more extreme shape is always harder, but the ability to smooth based on the original LipsUp shape would make that a lot easier. --------------------------------------------------------- So I'm sold on those things, but I'm still not sure about what would be a good use case for **Exclude Key**, and **Relative To**. Could you maybe propose a concrete or semi-concrete use case for those? --------------------------------------------------------- Also, is **Only Active Key** the same as a laplacian smooth? --------------------------------------------------------- A proposal: What about the ability to smooth relative not to another shape key, but to the combined shape of all currently active shape keys except the current one? I think this would handy for corrective shapes, which is what I was trying to achieve earlier; Let's say I have a LipsWide and a LipsUp shape, and I want to create the corrective that activates when both of those are activated at the same time. I would want to smooth relative to the shape that is the combined result of those two shapes. I hope that makes sense. But of course you don't have to implement that if you don't want to, I just thought I'd propose it since you're already doing something vaguely similar.
Author
Member

This is a big winner. Volume preserving smoothing is currently only possible with a sculpt brush, which is fine, but having it in an operator like this is also very handy, since it's easier to apply a uniform amount of smoothing to a selected area.

I wonder how the sculpt brush does it. My operator simply clamps the movement of vertices relative to the vertex normal (and these normals are updated every smoothing step, so the intensity vs number of steps choice matters).

So I'm sold on those things, but I'm still not sure about what would be a good use case for Exclude Key, and Relative To. Could you maybe propose a concrete or semi-concrete use case for those?

Repeating my answers from chat for the record, expanding a bit on some of it:

  • I actually listed a reason for Exclude - minimizing differences between mutually exclusive keys with a continuous blending transition. The idea is maybe you want to remove noise in the transition between the mutually exclusive shapes - just a low intensity smoothing pass or two.

    Plus, Exclude is basically the same as Include internally - the only difference is whether one math operation is addition or subtraction, so it's there for completeness in a way.

    In general, for this operator I was thinking of the MakeHuman body configuration shapes where you have sliders for fat/slim, muscular, breast size etc and it blends lots of shapes together.

  • Regarding Relative To, I have no specific use but thought it could be useful.

    To understand it you need to understand how shape keys really work: basically, they are supposed to represent a difference in shapes (offset), but are actually stored as shape snapshots, and the difference is obtained by subtracting a basis shape - and, importantly, while I bet that almost nobody uses this, technically you can have a different basis specified for each shape key, and even organize keys into a tree of parent-child keys with the main basis as the root. This option basically allows you to replace that basis for your key.

    One trick you can do using this obscure feature is turn mutually exclusive keys into morphs: e.g. you have keys Caucasian and Asian based on Basis (i.e. regular setup with one basis) and mutually exclusive. If you then change the property on Asian to be Relative To Caucasian, then suddenly the Asian key becomes a morph from Caucasian to Asian. It's convenient for observing the transition in the viewport while only moving one slider.

Also, is Only Active Key the same as a laplacian smooth?

I'm not sure what you mean, but the whole point of this is that it is smoothing not the shape, but the difference in shapes. For Only Active Key it is the difference between the key and its basis (i.e. its own deformation). Unless the basis shape is all vertices gathered into one point, regular smooth can't be the same as this operator.

Btw, think of this: if you use regular non-volume preserving smooth an infinite amount of times, it will crush the mesh into a single point. If you do the same with this operator, it will turn the shape key into a uniform Grab offset of all vertices by the same amount.

A proposal: What about the ability to smooth relative not to another shape key, but to the combined shape of all currently active shape keys except the current one?

You can always make a temporary New Shape From Mix, use that in the options of the smoothing operator, then delete the temporary once you're done. That is the reason I refrained from adding more complex math or multiple shape key selection fields.

> This is a big winner. Volume preserving smoothing is currently only possible with a sculpt brush, which is fine, but having it in an operator like this is also very handy, since it's easier to apply a uniform amount of smoothing to a selected area. I wonder how the sculpt brush does it. My operator simply clamps the movement of vertices relative to the vertex normal (and these normals are updated every smoothing step, so the intensity vs number of steps choice matters). > So I'm sold on those things, but I'm still not sure about what would be a good use case for **Exclude Key**, and **Relative To**. Could you maybe propose a concrete or semi-concrete use case for those? Repeating my answers from chat for the record, expanding a bit on some of it: - I actually listed a reason for Exclude - minimizing differences between mutually exclusive keys with a continuous blending transition. The idea is maybe you want to remove noise in the transition between the mutually exclusive shapes - just a low intensity smoothing pass or two. Plus, Exclude is basically the same as Include internally - the only difference is whether one math operation is addition or subtraction, so it's there for completeness in a way. In general, for this operator I was thinking of the MakeHuman body configuration shapes where you have sliders for fat/slim, muscular, breast size etc and it blends lots of shapes together. - Regarding Relative To, I have no specific use but thought it could be useful. To understand it you need to understand how shape keys really work: basically, they are supposed to represent a difference in shapes (offset), but are actually stored as shape snapshots, and the difference is obtained by subtracting a basis shape - and, importantly, while I bet that almost nobody uses this, technically you can have a different basis specified for each shape key, and even organize keys into a tree of parent-child keys with the main basis as the root. This option basically allows you to replace that basis for your key. One trick you can do using this obscure feature is turn mutually exclusive keys into morphs: e.g. you have keys Caucasian and Asian based on Basis (i.e. regular setup with one basis) and mutually exclusive. If you then change the property on Asian to be Relative To Caucasian, then suddenly the Asian key becomes a morph from Caucasian to Asian. It's convenient for observing the transition in the viewport while only moving one slider. > Also, is **Only Active Key** the same as a laplacian smooth? I'm not sure what you mean, but the whole point of this is that it is smoothing not the shape, but the difference in shapes. For Only Active Key it is the difference between the key and its basis (i.e. its own deformation). Unless the basis shape is all vertices gathered into one point, regular smooth can't be the same as this operator. Btw, think of this: if you use regular non-volume preserving smooth an infinite amount of times, it will crush the mesh into a single point. If you do the same with this operator, it will turn the shape key into a uniform Grab offset of all vertices by the same amount. > A proposal: What about the ability to smooth relative not to another shape key, but to the combined shape of all currently active shape keys except the current one? You can always make a temporary New Shape From Mix, use that in the options of the smoothing operator, then delete the temporary once you're done. That is the reason I refrained from adding more complex math or multiple shape key selection fields.
Alexander Gavrilov force-pushed pr-shapekey-smoothing from 0fe0843849 to 7bd8e5bc51 2023-07-28 13:23:54 +02:00 Compare
Hans Goudey reviewed 2023-07-31 14:38:43 +02:00
Hans Goudey left a comment
Member

Just looked at code style mostly. I know very little about shape keys, so this really isn't a review of the overall operator.

Just looked at code style mostly. I know very little about shape keys, so this really isn't a review of the overall operator.
@ -4010,0 +4011,4 @@
/** \name Smooth Shape Operator
* \{ */
namespace {
Member

I don't see anonymous namespaces much (at all?) in Blender code. What's the goal here? Maybe that would be achieved more clearly by putting the whole operator implementation in a blender::ed::mesh::smooth_shape_key namespace?

I don't see anonymous namespaces much (at all?) in Blender code. What's the goal here? Maybe that would be achieved more clearly by putting the whole operator implementation in a `blender::ed::mesh::smooth_shape_key` namespace?
Author
Member

It's to make this enum and struct local to the file. The style guide says that static is preferred for functions, but for classes there is no other choice so a namespace can be used.

It's to make this enum and struct local to the file. The style guide says that `static` is preferred for functions, but for classes there is no other choice so a namespace can be used.
@ -4010,0 +4028,4 @@
} // namespace
static bool edbm_shape_relative_info_validate(const BMesh *bm,
Member

Pass pointers as references if you expect them to be non-null (same below, etc)

Pass pointers as references if you expect them to be non-null (same below, etc)
Author
Member

Turned these functions into methods of the struct.

Turned these functions into methods of the struct.
angavrilov marked this conversation as resolved
@ -4010,0 +4060,4 @@
static void edbm_shape_get_relative_offset(const BMesh *bm,
const BMVert *eve,
const ShapeKeyRelativeInfo *info,
float *r_offset)
Member

float *r_offset -> float3 &r_offset, and you can use the C++ math:: functions in this function

`float *r_offset` -> `float3 &r_offset`, and you can use the C++ `math::` functions in this function
angavrilov marked this conversation as resolved
@ -4010,0 +4069,4 @@
if (info->mode == MESH_SHAPE_RELATIVE_BASIS) {
sco_basis = static_cast<const float *>(
CustomData_bmesh_get_layer_n(&bm->vdata, eve->head.data, base + info->ref_key));
Member

Could info contain the custom data offset directly, so you could just use BM_ELEM_CD_GET_FLOAT? That should be faster too, by avoiding the custom data API overhead

Could `info` contain the custom data offset directly, so you could just use `BM_ELEM_CD_GET_FLOAT`? That should be faster too, by avoiding the custom data API overhead
angavrilov marked this conversation as resolved
@ -4010,0 +4108,4 @@
int verts_mask_count = 0; /* Number of elements enabled in `verts_mask`. */
BM_ITER_MESH_INDEX (vert, &iter, bm, BM_VERTS_OF_MESH, vert_idx) {
if (!BM_elem_flag_test(vert, BM_ELEM_SELECT) || BM_elem_flag_test(vert, BM_ELEM_HIDDEN)) {
Member

I think all selected vertices are expected not to be hidden. I've seen that assumption described (and assumed) elsewhere in the the code. Seems like it would simplify things a bit.

I think all selected vertices are expected not to be hidden. I've seen that assumption described (and assumed) elsewhere in the the code. Seems like it would simplify things a bit.
Author
Member

This bit is copied from Blend From Shape code.

This bit is copied from Blend From Shape code.
@ -4010,0 +4137,4 @@
int vert_idx;
/* Perform blending on selected vertices. */
float(*offsets)[3] = static_cast<float(*)[3]>(
Member

Use Array<float3>

Use `Array<float3>`
angavrilov marked this conversation as resolved
@ -4010,0 +4167,4 @@
continue;
}
mul_v3_fl(avg_offset, 1.0f / float(count));
Member

Yes, C++ float3 math functions will look a lot nicer here :)

Yes, C++ `float3` math functions will look a lot nicer here :)
angavrilov marked this conversation as resolved
@ -4010,0 +4275,4 @@
}
const KeyBlock *kb_active = BKE_keyblock_from_key(key, em->bm->shapenr - 1);
Member

IMO the blank line between the retrieval of the key block and the null check make the code a bit harder to read

IMO the blank line between the retrieval of the key block and the null check make the code a bit harder to read
Author
Member

Added more comments to structure the flow.

Added more comments to structure the flow.
@ -4010,0 +4437,4 @@
/* api callbacks */
ot->exec = edbm_smooth_shape_exec;
/* disable because search popup closes too easily */
Member

Should this be resolved before the PR lands maybe? It's not clear to me what "closes too easily" means

Should this be resolved before the PR lands maybe? It's not clear to me what "closes too easily" means
Author
Member

I have no idea either - this is copy & paste from Blend From Shape code... So I just removed that bit.

I have no idea either - this is copy & paste from Blend From Shape code... So I just removed that bit.
Alexander Gavrilov force-pushed pr-shapekey-smoothing from 7bd8e5bc51 to 040d845907 2023-07-31 18:51:16 +02:00 Compare
Alexander Gavrilov force-pushed pr-shapekey-smoothing from 040d845907 to bb94b43564 2023-08-01 14:48:49 +02:00 Compare
Alexander Gavrilov force-pushed pr-shapekey-smoothing from bb94b43564 to 56a9bb8f16 2023-08-01 16:35:44 +02:00 Compare
Alexander Gavrilov force-pushed pr-shapekey-smoothing from 56a9bb8f16 to bdd7341bcf 2023-08-01 16:57:48 +02:00 Compare
Alexander Gavrilov force-pushed pr-shapekey-smoothing from bdd7341bcf to 3172b9abab 2023-08-06 18:58:41 +02:00 Compare
Alexander Gavrilov force-pushed pr-shapekey-smoothing from 3172b9abab to 9a5ad2446d 2023-11-19 22:03:53 +01:00 Compare
Alexander Gavrilov force-pushed pr-shapekey-smoothing from 9a5ad2446d to 45ce7d4e29 2023-12-25 21:35:55 +01:00 Compare
Alexander Gavrilov added the
Module
Modeling
label 2024-01-09 12:28:17 +01:00
Alexander Gavrilov force-pushed pr-shapekey-smoothing from 45ce7d4e29 to d1db9b82eb 2024-01-22 13:30:39 +01:00 Compare
This pull request has changes conflicting with the target branch.
  • source/blender/editors/mesh/mesh_intern.h

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u pr-shapekey-smoothing:angavrilov-pr-shapekey-smoothing
git checkout angavrilov-pr-shapekey-smoothing
Sign in to join this conversation.
No reviewers
No Label
Interest
Alembic
Interest
Animation & Rigging
Interest
Asset Browser
Interest
Asset Browser Project Overview
Interest
Audio
Interest
Automated Testing
Interest
Blender Asset Bundle
Interest
BlendFile
Interest
Collada
Interest
Compatibility
Interest
Compositing
Interest
Core
Interest
Cycles
Interest
Dependency Graph
Interest
Development Management
Interest
EEVEE
Interest
EEVEE & Viewport
Interest
Freestyle
Interest
Geometry Nodes
Interest
Grease Pencil
Interest
ID Management
Interest
Images & Movies
Interest
Import Export
Interest
Line Art
Interest
Masking
Interest
Metal
Interest
Modeling
Interest
Modifiers
Interest
Motion Tracking
Interest
Nodes & Physics
Interest
OpenGL
Interest
Overlay
Interest
Overrides
Interest
Performance
Interest
Physics
Interest
Pipeline, Assets & IO
Interest
Platforms, Builds & Tests
Interest
Python API
Interest
Render & Cycles
Interest
Render Pipeline
Interest
Sculpt, Paint & Texture
Interest
Text Editor
Interest
Translations
Interest
Triaging
Interest
Undo
Interest
USD
Interest
User Interface
Interest
UV Editing
Interest
VFX & Video
Interest
Video Sequencer
Interest
Virtual Reality
Interest
Vulkan
Interest
Wayland
Interest
Workbench
Interest: X11
Legacy
Blender 2.8 Project
Legacy
Milestone 1: Basic, Local Asset Browser
Legacy
OpenGL Error
Meta
Good First Issue
Meta
Papercut
Meta
Retrospective
Meta
Security
Module
Animation & Rigging
Module
Core
Module
Development Management
Module
EEVEE & Viewport
Module
Grease Pencil
Module
Modeling
Module
Nodes & Physics
Module
Pipeline, Assets & IO
Module
Platforms, Builds & Tests
Module
Python API
Module
Render & Cycles
Module
Sculpt, Paint & Texture
Module
Triaging
Module
User Interface
Module
VFX & Video
Platform
FreeBSD
Platform
Linux
Platform
macOS
Platform
Windows
Priority
High
Priority
Low
Priority
Normal
Priority
Unbreak Now!
Status
Archived
Status
Confirmed
Status
Duplicate
Status
Needs Info from Developers
Status
Needs Information from User
Status
Needs Triage
Status
Resolved
Type
Bug
Type
Design
Type
Known Issue
Type
Patch
Type
Report
Type
To Do
No Milestone
No project
No Assignees
4 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: blender/blender#110275
No description provided.