Sculpt Mode Cavity Masking #98383
To make the Paint Mode implementation feature complete we need to include an a way to mask the surface based on Cavity.
A first implementation of this has already been started in the sculpt-dev branch and can be further worked on.
In that case this feature will become a new automasking option in sculpt mode and and paint mode as well.
The way we want to handle this is via the auto-masking options in sculpt mode. Since paint mode is based on the same code, the feature can be used for both modes.
- Can be toggled on the fly
- The auto-mask can be combined with other auto-masking methods without changing the painted mask.
- Masked areas can be changed non-destructively
- The auto-mask will update to the latest surface curvature
There are various controls that are needed to get the cavity mask users want.
- Falloff curve (Full control over which areas of the mesh curvature will be auto-masked)
- Inverted Cavity (An easy toggle to flip the effect of the auto-mask)
- Smoothing Steps
- Currently called propagation steps. We could rename the setting to smoothing and keep calling it propagation in the tooltip and code
- This setting must be exclusively tied to the cavity auto-masking, instead of using the regular propagation steps in the brush settings.
There are other design aspects to consider.
On high polycounts or when using a high amount of propagation steps it is likely to cause lag on each brush stroke.
@JosephEagar pointed out that D14272 and an API for temporary attributes would fix this.
Another boost could be to check if any important changes happened to the mesh in between strokes.
If vert positions haven't changed then the cavity auto-masking doesn't need to be recalculated on repeated strokes. So it would often only lag once.
The same could then be done for topology and face set auto-masking if the cursor is still targeting the same area.
Boundary auto-masking would also benefit from this.
This needs to be discussed and tested further.
Visualising the Cavity Mask
There needs to be some visual feedback on what is currently being auto-masked. Especially when we add fine control over propagation steps and what curvatures are masked via a falloff curve.
Since the automask is only generated under the brush cursor when doing a stroke, this becomes hard to implement.
If no solution is found, then a better way would be use an operator to convert the cavity automask into a regular mask attribute direectly.
Persistent Cavity Mask
There needs to be an option to keep the cavity mask persistent between strokes.
This would be easiest to implement with an operator to create a mask attribute based on the current cavity masking settings.
An operator like this could have multiple purposes:
- Use a persistent mask for detailed sculpting adjustments
- Visualise and edit more complex cavity masks
- Bake or export cavity masks for further workflows
We already have an operator like this called "Dirt Mask". This operator could be updated to use the same settings as the cavity automask.
For clarity it could then be renamed to "Mask by Cavity", with a tooltip that states that the Cavity automask settings are used for this operator.
Additional adjust last operation options should be made available too.
Changed status from 'Needs Triage' to: 'Confirmed'
For persistent mode, we could have a "bake to mask" option, and maybe enable multiple mask attribute layers in the code (shouldn't be hard).
That way we can support multires seamlessly (which has hardcoded support for masks).
Hi, idk if it is considered but a pointiness mask option, maybe inside this mask or as its own thing, could be really useful too, it is quite common to use that as base when you are texturing hard surface objects.
Notes from previous module meeting:
We then further discussed how to expose inverted cavity for this patch.
One approach is to expose it as 3 options that are mutually exclusive to the user:
- Enabled Inverted
This would be the most convenient and should be the way to interact with cavity auto masking via the pie menu.
The proposal is to still show the “Invert Cavity” toggle as a sub-setting of the cavity auto-masking toggle in the options panel.
But in the pie menu they would be exposed as equal options “Cavity” and “Inverted Cavity”.
Choosing Cavity will enable the cavity auto masking and disable inverted cavity.
Choosing Inverted Cavity will enable cavity auto masking and enable inverted cavity.
Only one of the two can be enabled (highlighted) at once in the pie menu.
Choosing any of them them again will disable both.
This would be an unusual solution but potentially the best for the user.
We also discussed the operator to create a cavity mask. As of right now this is implemented as a new tab to bake the current auto masks. That is not ideal. We need to make these distinct operators.
For cavity masks we can make a replacement for the “Dirty Mask” operator in the mask menu simply called “Cavity Mask”. The same could then be done for a “Normal Mask” operator.
These operators would then use the auto mask settings by default when used, but the result can be tweaked further with adjust last operation settings.
This issue was referenced by
Deleting a branch is permanent. Although the deleted branch may exist for a short time before cleaning up, in most cases it CANNOT be undone. Continue?