Sculpt Mode Multires Design #73047

The purpose of this task is to describe the main uses cases and functionality that should be provided by a Multires sculpting system, as well as to discuss a plan on how to implement that in Blender.

The main use case of Multires

The problem Multires solves

When you are working on any art medium (this also applies for painting), you should always try to work from the general shapes to the details. In case of digital sculpting, this means that you should always try to work with the minimum number of vertices that represent the level of detail you are trying to add to the model.
This is also true for the current workflow we have implemented (The Voxel remesher). By default, it provides a low poly mesh (about 10K vertices), that you should deform trying to get main shape of the object you want. After that, you increase the resolution of the remesher until you have enough vertices to represent the next level of detail you want to work on, and you continue from there.
The reasons you want to work this way are:

Performance: It is faster (more responsive, without lag) to deform a mesh with 10K vertices than a mesh with 25M vertices. If you use a big brush size needed to modify the object proportions on 25M vertices at the same time, it is going to lag. (Unrelated note, but we should not try to optimize Sculpt Mode to support these kind of operations as they don’t make any sense with a Multires system in place).
(Most important) Noise: If you sculpt with more vertices than needed for the size of details and shapes you are trying to get, you are going to end with a lot of uncontrolled shape noise produced by the tools. This is not a specific problem with Blender brushes and tools, this happens in any software and in every sculpting system. If you use the Draw brush to add some volume on a big area directly onto a 25M vertices mesh, you are going to need need to clean manually all small uncontrolled imperfections that big stroke stroke produced if you want that to be the final surface. If you start sculpting from a sphere with 1M vertices, you are going to spend more time cleaning shape noise than sculpting or the sculpt is just going to look bad. This is why nobody works this way. You don’t generate unintended shape noise if there aren’t enough vertices to represent it.

That is the main problem Multires sculpting solves. With Multires, you are able to choose at any moment the level of detail that is appropriate for the change you want to make in you model, solving both the performance and the noise issues. This gives an artist a huge amount of freedom as you know you can always go back to tweak low frequency details (proportions, general volumes) after sculpting mid/high frequency details (wrinkles, skin alphas...) if you consider that they don’t look right, without loosing any work.

Without this, the rest of the sculpting workflows (the Voxel remesher, Dyntopo) are the same as sculpting with real clay or marble, you can only sculpt bigger first shapes, then mid details and them the small details. If it does not look right, there is not way to fix it, you need to delete what looks wrong and start again. This is why those workflows are often used for concepting general shapes, and you want two switch to Multires as soon as you can. Sculpting a complex model without using Multires, on any software, is extremely (and unnecessarily) hard and it requires an insane amount of skill from the artist to workaround the limitation. It is like sculpting without using undo. It is too risky to create a high poly sculpt with projected alphas without this feature.

In order to solve this problem, the main thing Multires needs to guarantee is that when you sculpt shapes on your model in the appropriate resolution level, all the other levels are updated accordingly to reflect the changes you did. This is what people often refer as ¨detail propagation¨.

Multires Design

When designing a sculpting Multires system that provides the previously mentioned functionality, We need to take the following points into consideration:

All Multires subdivision levels contain valid information (sculpted shapes, mask, colors...) that should never be lost or damaged.
The user wants to modify any subdivision level, at any time, without restrictions (there should not be limits of how much the mesh should be deformed or “not allowed modifications”).
The user wants to delete and rebuild higher/lower subdivision levels in he/she considers that the shape information that level contains is not longer relevant.

For example, if the user wants to do a high poly sculpt, he/she switches to the higher level subdivision possible that the computer allows to create the high frequency detail projecting alphas. When that is done, the user may want to switch to a lower level to tweak proportions, or to a mid level to tweak small volumes with a soft brush. All the work the user put into applying the alphas should be propagated with the lower subdivision level changes with the Multires functionality and the model should look fine.

If, for some reason (a bug or an design issue in the implementation that did not consider point 1), switching and modifying lower levels may introduce mesh artifacts in the higher ones (the ones that contains the alpha projections and high frequency details), it will destroy all the work done previously by the user in that levels.

In that case, the user is again limited to not going back to select the mesh resolution needed for the modifications he/she wants to make in the mesh. A Multires system like this is not solving the intended problem and there is no reason to use it. It provides the same functionality as any of the other sculpting workflows (Voxel, Dyntopo), but even more limited as the user can’t modify the base topology to adapt it to a new shape. It does not have any reason to exist.

If you project requires you to use a fixed predefined topology in your model (for animation, for example), subdividing the mesh manually and deforming it with a cage or a datatransfer provides the same functionality. You can also get a lower subdivision level by duplicating the model and using unsubdivide if you need it for texture baking or animation. This is not the main problem Multires should be trying to solve.

Any unwanted mesh artifact in the detail propagation, no matter how small it is, invalidates all Multires sculpting features and makes it not functional for its intended use case, as it leads to irreparable data corruption.

Current Multires Modifier

Considering the previously design and use case description as valid Multires sculpting system:

Multires in 2.81 does not support switching levels. This invalidates the feature completely and makes it not functional, but prevents the users to corrupt the mesh data if they use it. This is like not having Multires sculpting at all.
Multires in 2.80 creates artifacts and vertex explosions when modifying any level, creating irreparable mesh data corruption. Releasing Blender with Multires in this state and without any visible UI warning could have destroyed the projects of many users in an irreparable way. Luckily, most users were not using it to create their models in the previous versions because it was also not working as intended.
Multires in 2.79 and before creates a visible grid pattern on higher levels when modifying lower ones. This introduces unwanted shape noise and effectively destroys valid sculpted information, making it not functional for its intended use case. Users are limited to work considering of how much they can deform lower levels by the amount of noise they are predicting Multires will produce in the higher ones. This is not something that should be happening in a production ready implementation of the feature.

The most advances tools Blender has right now that were added in the last releases (Quadriflow, Elastic deform, IK Pose brush...) are designed to work with a Multires system:

The main purpose of Quadriflow is to produce subdividable geometry for a Multires. When you are done sculpting with Dyntopo/Voxel remesher, where you can’t have the freedom to choose the level of detail you want to work on, you can quickly generate a base mesh, subdivide it and reproject the details back. That way you can tweak again general proportions and low frequency detail.
These brush tools are designed to tweak proportions/big shapes of models, which is something you most likely want to do after sculpting more detail to have an idea of how your model looks like. Multires solves that problem.
These brushes can’t be used directly on high poly meshes. These tools needs to have disabled some sculpt mode optimizations in order to work, so they are much slower than regular brushes. In fact, if you try to use the IK pose brush directly on a high poly mesh, most likely you are going to run out of memory. If you don’t, it is going to create terrible artifacts between each segment. In order for these brushes to work as intended on high poly meshes, you need to use Multires to switch to a lower level, use the brush, and then switch to a higher level again to get the intended result.

Multires implementation reference

SculptGL contains a valid and functional Multires sculpting implementation. You can try it by sculpting different details at different levels and switching between them, as well as creating and deleting subdivision levels:

https://stephaneginier.com/sculptgl/

This implementation of Multires provides:

Detail propagation that works across all levels. You can choose the desired level you want to sculpt on and that deformation is propagated correctly to all subdivision levels. There are not visible artifacts of any kind.
You can both create and delete levels in both directions (higher levels and lower levels) with subdivide/delete higher and reverse/delete lower.

These features are the minimum and required a Multires implementation in Blender should provide, as they are pretty much the definition of a Multires sculpting system. Any other Multires related feature or optimization (being part of the modifier stack, baking, shape keys, layers, exporting, reprojection, tool compatibility, Quadriflow...) should always come later. There should not be a design decisions that prioritize any other Multires feature or optimization without having those minimums perfectly working, bug free and functional.

Proposal

Given the fact that, as described here, Multires was never a functional feature for its intended purpose and:

The code is extremely hard to read and maintain. I still can’t understand what it is doing in some places or what was the reasoning to make it work that way in the first place.
New features are planned that require Multires support (masking tools, vertex colors, visibility operators). Adding Multires support for those features should not be a project bigger than the feature itself. This was the case with most of the 2.81 tools, and probably with the future planned features (Sculpt vertex colors or visibility management, for example)
We already have and sculpt tools abstraction layer in place. Making changes on the underlying data structures of sculpt mode should be easier than before
The original Multires comes from before 2.5. The basics of how it should works were never correct and a lot of features and modifications were added during the years. We are now trying to build state of the art sculpting features on top of a broken foundation that has more than 10 years.

Some options are:

- Replace Multires modifier with a simple implementation based on the SculptGL code (something we know it works) and start again from there. We can keep the current Multires modifier for compatibility, but we need to evaluate if supporting the features and fixes described in this task on the current code (and make it work exactly as the SculptGL Multires) is going to take more time than implementing something based on the SculptGL code from scratch. We also need to look that the new features that are planned and evaluate how hard they are going to be to add to the current Multires code (colors, per face data...) knowing that the SculptGL implementation already supports them.

- Try to add this functionality to the current Multires modifier. In that case, the priority should always be supporting the functionality described in this task first, and the bring all the rest of the features back. In that case, it would be needed better documentation and code organization to be able to integrate new features into it easily.

The purpose of this task is to describe the main uses cases and functionality that should be provided by a Multires sculpting system, as well as to discuss a plan on how to implement that in Blender. ## The main use case of Multires ### The problem Multires solves When you are working on any art medium (this also applies for painting), you should always try to work from the general shapes to the details. In case of digital sculpting, this means that you should always try to work with the minimum number of vertices that represent the level of detail you are trying to add to the model. This is also true for the current workflow we have implemented (The Voxel remesher). By default, it provides a low poly mesh (about 10K vertices), that you should deform trying to get main shape of the object you want. After that, you increase the resolution of the remesher until you have enough vertices to represent the next level of detail you want to work on, and you continue from there. The reasons you want to work this way are: - Performance: It is faster (more responsive, without lag) to deform a mesh with 10K vertices than a mesh with 25M vertices. If you use a big brush size needed to modify the object proportions on 25M vertices at the same time, it is going to lag. (Unrelated note, but we should not try to optimize Sculpt Mode to support these kind of operations as they don’t make any sense with a Multires system in place). - (Most important) Noise: If you sculpt with more vertices than needed for the size of details and shapes you are trying to get, you are going to end with a lot of uncontrolled shape noise produced by the tools. This is not a specific problem with Blender brushes and tools, this happens in any software and in every sculpting system. If you use the Draw brush to add some volume on a big area directly onto a 25M vertices mesh, you are going to need need to clean manually all small uncontrolled imperfections that big stroke stroke produced if you want that to be the final surface. If you start sculpting from a sphere with 1M vertices, you are going to spend more time cleaning shape noise than sculpting or the sculpt is just going to look bad. This is why nobody works this way. You don’t generate unintended shape noise if there aren’t enough vertices to represent it. That is the main problem Multires sculpting solves. With Multires, you are able to choose at any moment the level of detail that is appropriate for the change you want to make in you model, solving both the performance and the noise issues. This gives an artist a huge amount of freedom as you know you can always go back to tweak low frequency details (proportions, general volumes) after sculpting mid/high frequency details (wrinkles, skin alphas...) if you consider that they don’t look right, without loosing any work. Without this, the rest of the sculpting workflows (the Voxel remesher, Dyntopo) are the same as sculpting with real clay or marble, you can only sculpt bigger first shapes, then mid details and them the small details. If it does not look right, there is not way to fix it, you need to delete what looks wrong and start again. This is why those workflows are often used for concepting general shapes, and you want two switch to Multires as soon as you can. Sculpting a complex model without using Multires, on any software, is extremely (and unnecessarily) hard and it requires an insane amount of skill from the artist to workaround the limitation. It is like sculpting without using undo. It is too risky to create a high poly sculpt with projected alphas without this feature. In order to solve this problem, the main thing Multires needs to guarantee is that when you sculpt shapes on your model in the appropriate resolution level, all the other levels are updated accordingly to reflect the changes you did. This is what people often refer as ¨detail propagation¨. ### Multires Design When designing a sculpting Multires system that provides the previously mentioned functionality, We need to take the following points into consideration: 1. All Multires subdivision levels contain valid information (sculpted shapes, mask, colors...) that should never be lost or damaged. 2. The user wants to modify any subdivision level, at any time, without restrictions (there should not be limits of how much the mesh should be deformed or “not allowed modifications”). 3. The user wants to delete and rebuild higher/lower subdivision levels in he/she considers that the shape information that level contains is not longer relevant. For example, if the user wants to do a high poly sculpt, he/she switches to the higher level subdivision possible that the computer allows to create the high frequency detail projecting alphas. When that is done, the user may want to switch to a lower level to tweak proportions, or to a mid level to tweak small volumes with a soft brush. All the work the user put into applying the alphas should be propagated with the lower subdivision level changes with the Multires functionality and the model should look fine. If, for some reason (a bug or an design issue in the implementation that did not consider point 1), switching and modifying lower levels may introduce mesh artifacts in the higher ones (the ones that contains the alpha projections and high frequency details), it will destroy all the work done previously by the user in that levels. In that case, the user is again limited to not going back to select the mesh resolution needed for the modifications he/she wants to make in the mesh. A Multires system like this is not solving the intended problem and there is no reason to use it. It provides the same functionality as any of the other sculpting workflows (Voxel, Dyntopo), but even more limited as the user can’t modify the base topology to adapt it to a new shape. It does not have any reason to exist. If you project requires you to use a fixed predefined topology in your model (for animation, for example), subdividing the mesh manually and deforming it with a cage or a datatransfer provides the same functionality. You can also get a lower subdivision level by duplicating the model and using unsubdivide if you need it for texture baking or animation. This is not the main problem Multires should be trying to solve. **Any unwanted mesh artifact in the detail propagation, no matter how small it is, invalidates all Multires sculpting features and makes it not functional for its intended use case, as it leads to irreparable data corruption.** ### Current Multires Modifier Considering the previously design and use case description as valid Multires sculpting system: 1. Multires in 2.81 does not support switching levels. This invalidates the feature completely and makes it not functional, but prevents the users to corrupt the mesh data if they use it. This is like not having Multires sculpting at all. 2. Multires in 2.80 creates artifacts and vertex explosions when modifying any level, creating irreparable mesh data corruption. Releasing Blender with Multires in this state and without any visible UI warning could have destroyed the projects of many users in an irreparable way. Luckily, most users were not using it to create their models in the previous versions because it was also not working as intended. 3. Multires in 2.79 and before creates a visible grid pattern on higher levels when modifying lower ones. This introduces unwanted shape noise and effectively destroys valid sculpted information, making it not functional for its intended use case. Users are limited to work considering of how much they can deform lower levels by the amount of noise they are predicting Multires will produce in the higher ones. This is not something that should be happening in a production ready implementation of the feature. The most advances tools Blender has right now that were added in the last releases (Quadriflow, Elastic deform, IK Pose brush...) are designed to work with a Multires system: - The main purpose of Quadriflow is to produce subdividable geometry for a Multires. When you are done sculpting with Dyntopo/Voxel remesher, where you can’t have the freedom to choose the level of detail you want to work on, you can quickly generate a base mesh, subdivide it and reproject the details back. That way you can tweak again general proportions and low frequency detail. - These brush tools are designed to tweak proportions/big shapes of models, which is something you most likely want to do after sculpting more detail to have an idea of how your model looks like. Multires solves that problem. - These brushes can’t be used directly on high poly meshes. These tools needs to have disabled some sculpt mode optimizations in order to work, so they are much slower than regular brushes. In fact, if you try to use the IK pose brush directly on a high poly mesh, most likely you are going to run out of memory. If you don’t, it is going to create terrible artifacts between each segment. In order for these brushes to work as intended on high poly meshes, you need to use Multires to switch to a lower level, use the brush, and then switch to a higher level again to get the intended result. ## Multires implementation reference SculptGL contains a valid and functional Multires sculpting implementation. You can try it by sculpting different details at different levels and switching between them, as well as creating and deleting subdivision levels: https://stephaneginier.com/sculptgl/ This implementation of Multires provides: - Detail propagation that works across all levels. You can choose the desired level you want to sculpt on and that deformation is propagated correctly to all subdivision levels. There are not visible artifacts of any kind. - You can both create and delete levels in both directions (higher levels and lower levels) with subdivide/delete higher and reverse/delete lower. These features are the minimum and required a Multires implementation in Blender should provide, as they are pretty much the definition of a Multires sculpting system. Any other Multires related feature or optimization (being part of the modifier stack, baking, shape keys, layers, exporting, reprojection, tool compatibility, Quadriflow...) should always come later. There should not be a design decisions that prioritize any other Multires feature or optimization without having those minimums perfectly working, bug free and functional. ## Proposal Given the fact that, as described here, Multires was never a functional feature for its intended purpose and: - The code is extremely hard to read and maintain. I still can’t understand what it is doing in some places or what was the reasoning to make it work that way in the first place. - New features are planned that require Multires support (masking tools, vertex colors, visibility operators). Adding Multires support for those features should not be a project bigger than the feature itself. This was the case with most of the 2.81 tools, and probably with the future planned features (Sculpt vertex colors or visibility management, for example) - We already have and sculpt tools abstraction layer in place. Making changes on the underlying data structures of sculpt mode should be easier than before - The original Multires comes from before 2.5. The basics of how it should works were never correct and a lot of features and modifications were added during the years. We are now trying to build state of the art sculpting features on top of a broken foundation that has more than 10 years. Some options are: - Replace Multires modifier with a simple implementation based on the SculptGL code (something we know it works) and start again from there. We can keep the current Multires modifier for compatibility, but we need to evaluate if supporting the features and fixes described in this task on the current code (and make it work exactly as the SculptGL Multires) is going to take more time than implementing something based on the SculptGL code from scratch. We also need to look that the new features that are planned and evaluate how hard they are going to be to add to the current Multires code (colors, per face data...) knowing that the SculptGL implementation already supports them. - Try to add this functionality to the current Multires modifier. In that case, the priority should always be supporting the functionality described in this task first, and the bring all the rest of the features back. In that case, it would be needed better documentation and code organization to be able to integrate new features into it easily.

Added subscriber: @PabloDobarro

Added subscriber: @WilliamReynish

Thanks for making this. A few comments:

1: IMO one of the main original use-cases for Multires, was as an alternative to using Sub-Ds, but with hierarchical detail. So you can animate, rig and play back simpler geometry, but render at a higher level. That is one of the core use-cases, which I don't think you mention here.

2: I'm not sure it even makes sense to keep the 'old' Multires modifier around. It barely works at all right now, so there isn't much value in keeping the code - and we have a potential upside in getting rid of the old cruft and simplifying the tools code that way.

Added subscriber: @Loner-2

Honestly, it's better to replace something that doesn't work than clutter up something else on this. All the more so because never worked she on 100%. So I'll say for replacement of the current multires with SculpGl if its author gives permission. A lot of users understand that multires 100% never worked. Those who have also seen the code also don't understand what works there or how it works. It's better to live with good code than to clutter up something else on a foundation that doesn't work.

In #73047#846601, @WilliamReynish wrote:
Thanks for making this. A few comments:

1: IMO one of the main original use-cases for Multires, was as an alternative to using Sub-Ds, but with hierarchical detail. So you can animate, rig and play back simpler geometry, but render at a higher level. That is one of the core use-cases, which I don't think you mention here.

2: I'm not sure it even makes sense to keep the 'old' Multires modifier around. It barely works at all right now, so there isn't much value in keeping the code - and we have a potential upside in getting rid of the old cruft and simplifying the tools code that way.

I agree that it's better to delete all the old multires code than to hold it.

Added subscriber: @Stan_Pancakes

Added subscriber: @Frozen_Death_Knight

If SculptGL/a new system is the way to go, then you should delete the old MultiRes system. However, until the new system is in place, you can't delete the old one. There are tons of projects that have used it over the years and people need time to transition over their projects from MultiRes to MultiRes2 with Shrinkwraps and more. You need at least one official release where the legacy system and reworked system are available for this to happen.

I don't know about SculptGL, but one thing is true: if something doesn't work, it doesn't need to exist. Supporting legacy is impractical. It's infectious and counter-productive. If a scrap and rewrite is more sensible than maintenance, do it. Just do it!

@WilliamReynish I understand the use case of storing displacement information at different levels of subdivision, but that is not what a Multires Sculpting feature should do. In any case, Multires was added back then when Blender didn't have UDIMs, vector displacement and adaptive subdivision in the renderer. This is now the intended way to rig an animate VFX assets, that is compatible with most studio pipelines and with importing/exporting data to other texturing, sculpting softwares and render engines. I don't see a particular reason to keep it just for animation, but maybe there is still some use case for it that I'm not aware of (I have no idea about animation).
If we want to stop supporting that (which makes total sense), I think we should evaluate how much of the current code can be reused to implement a sculpting Multires (maybe the PBVH_GRIDS code and the iterators can be reused) before removing it completely. Maybe it is less work than it seems if we go with the standard workflow for animation.

Removed subscriber: @Frozen_Death_Knight

Added subscriber: @rubens9119

Added subscriber: @tux21b

Added subscriber: @Andruxa696

@WilliamReynish
I agreed with Pablo, more important to have properly working multires for sculpting , than for animation. Most studios use displacement for this task . We can make multires as an object data property(as vertex colors and shape keys) . In this case we only need proper baking displacement from multires , so later we can use it for animation.
Or we can add later support multires for animation, after base features of new multires will work.

Added subscriber: @JulianPerez

Added subscriber: @riceart

Added subscriber: @ErickNyanduKabongo

In #73047#846635, @PabloDobarro wrote:
@WilliamReynish I understand the use case of storing displacement information at different levels of subdivision, but that is not what a Multires Sculpting feature should do. In any case, Multires was added back then when Blender didn't have UDIMs, vector displacement and adaptive subdivision in the renderer. This is now the intended way to rig an animate VFX assets, that is compatible with most studio pipelines and with importing/exporting data to other texturing, sculpting softwares and render engines. I don't see a particular reason to keep it just for animation, but maybe there is still some use case for it that I'm not aware of (I have no idea about animation).
If we want to stop supporting that (which makes total sense), I think we should evaluate how much of the current code can be reused to implement a sculpting Multires (maybe the PBVH_GRIDS code and the iterators can be reused) before removing it completely. Maybe it is less work than it seems if we go with the standard workflow for animation.

I m with @WilliamReynish in this and since you have no idea about animation it is wiser not to have a strong opinion on it. Having the ability to rig the lower level of the highly sculpted model is a time saver or being able to rig a model and add multires. on top of it, it is a real time saver. If decide to let this feature go then we need a very robust baking system, since the actual baking system is quiet confusing and has many hit and miss. The good baking system will help to bake all those high res. details sculpted with multires to a lower one.

I m pretty happy to read this thread and see that finally Multiresolution modifier problems is going to be tackle and as old Multiresolution modifier user and Multiresolution modifier advocate you have my benedictions to throw multiresolution modifier to the canal. May the force be with you.

It's true that this use-case also has some overlap with adaptive subdivision and baked displacement. It may probably be ok to make the scope of Multires narrower - as long as we do it knowingly.

In any case, at least initially it would be a major benefit to get to a point where it works reliably for a subset of the use-cases.

Added subscriber: @mankysee

Added subscriber: @smramsay

Added subscriber: @AlbertoVelazquez

It is a pleasure to read this proposal.

Personally, I don't know if the new system will work with rigging or not. I would certainly prefer to keep it. Now, if that's a reason to stop, delay or impede the development of a new multires... you have my blessing as a user to burn that feature.

I would certainly like to raise... is it really necessary that the multires be a modifier? Given that having more than one modifier of this type is a nonsense and that it is always forced to be the first modifier... wouldn't it be more logical that it simply be a property of the mesh like the vertex groups? Maybe I'm wrong, but I don't see any sense in it being a modifier. Also future sculpt layers could be placed in same mesh tab, like vertex groups or shapes, where have more sense than inside a modifier.

I agree with Alberto. Heavily delaying development just for having the new system support animation is really not an option. There are other solutions to the animation problem out there that are tried and tested. You can try and see if you can do it, but forcing a new MultiRes system to accomodate animation needs is honestly silly. Animators don't need MultiRes when the SubDiv modifier and displacement maps could handle pretty much all that work for them (Blender needs a way to create displacement maps from your sculpts that the animators can use like ZBrush does it.). Sculptors however just can't work without a MultiRes system, so their needs should be taken priority here.

Heck, why not just rename the old MultiRes modifier to Animation Resolution modifier or something and rework it so it has more features for animation while the sculptors get the new system? Isn't the current MultiRes already coded to accomodate their needs?

Added subscriber: @ThinkingPolygons

Replace Multires modifier with a simple implementation based on the SculptGL code (something we know it works) and start again from there. ✔️

This is it, nuff said.

Added subscriber: @Traediras

Having played around with SculptGL, I feel that having a similar implementation in Blender would be hugely beneficial, as long as the ability to maintain fine levels of detail remains. My current workflow generally revolves around a high-poly sculpt, retopology, then capturing the finer details with a multires modifier from a subdiv'd/shrunkwrapped mesh. I like that the multires modifier basically encapsulates subdivision and displacement in a singular modifier, and I've found it to be particularly useful with armature deforming affecting the base mesh, though I do absolutely understand the challenges around the current implementation of multires, particularly when additional geometry needs to be added to the base mesh.

The current issue I have with voxel remeshing and other auto-retopology tools is that finer details can be lost from remeshing, such as spines, scale plating, membranes, and smaller concave regions like earholes, eyesockets, along with geometry that are situated close together such as digits and maw interiors. It's important that the new multires modifier is able to assist remeshing algorithms to preserve some finer details. Of course, some issues can't be resolved with basic remeshing and subdividing, eventually it's necessary for one to create a base mesh with proper topology to achieve the required result from the high-poly sculpt. If it's also possible with new multires to work with any addition or removal of geometry to the base mesh, then it'd essentially be perfect, but I understand that would be far easier said than done.

It's important at the very least that the chief feature (afaik) of multires - being able to have finer details for a mesh without needing a texture map - is kept. I'm not against a complete rewrite of the multires system as long as it brings improvements, though I do understand if it ends up breaking projects that use the old system, but as long as a workflow is accommodated for artists to be able to transfer projects from the old/legacy multires to the new multires, it shouldn't be an issue.

On an additional note, I feel that it would be highly beneficial for users that may not pay attention to developer forums or social media to have a tutorial or guide of the workflow that is intended with the updates to sculpting tools for Blender. Having basic documentation might not necessarily be enough guidance, particularly for a beginner.

Having played around with SculptGL, I feel that having a similar implementation in Blender would be hugely beneficial, as long as the ability to maintain fine levels of detail remains. My current workflow generally revolves around a high-poly sculpt, retopology, then capturing the finer details with a multires modifier from a subdiv'd/shrunkwrapped mesh. I like that the multires modifier basically encapsulates subdivision and displacement in a singular modifier, and I've found it to be particularly useful with armature deforming affecting the base mesh, though I do absolutely understand the challenges around the current implementation of multires, particularly when additional geometry needs to be added to the base mesh. The current issue I have with voxel remeshing and other auto-retopology tools is that finer details can be lost from remeshing, such as spines, scale plating, membranes, and smaller concave regions like earholes, eyesockets, along with geometry that are situated close together such as digits and maw interiors. It's important that the new multires modifier is able to assist remeshing algorithms to preserve some finer details. Of course, some issues can't be resolved with basic remeshing and subdividing, eventually it's necessary for one to create a base mesh with proper topology to achieve the required result from the high-poly sculpt. If it's also possible with new multires to work with any addition or removal of geometry to the base mesh, then it'd essentially be perfect, but I understand that would be far easier said than done. It's important at the very least that the chief feature (afaik) of multires - being able to have finer details for a mesh without needing a texture map - is kept. I'm not against a complete rewrite of the multires system as long as it brings improvements, though I do understand if it ends up breaking projects that use the old system, but as long as a workflow is accommodated for artists to be able to transfer projects from the old/legacy multires to the new multires, it shouldn't be an issue. On an additional note, I feel that it would be highly beneficial for users that may not pay attention to developer forums or social media to have a tutorial or guide of the workflow that is intended with the updates to sculpting tools for Blender. Having basic documentation might not necessarily be enough guidance, particularly for a beginner.

Added subscriber: @ReinhardK

Added subscriber: @MassimilianoPuliero

I'm with the guys that would love a totally new multires modifier instead of try to fix the current one.
cheers and goodluck!

Added subscriber: @moisessalvador

As a way of storing data for animated models, Multires modifier has advantages over displacement textures. With displacement textures, you first have to UV map, you have to bake (which there is no support for baking vector displacement maps currently) you have to make sure the texture resolution is proper, sometimes you get artifacts if the mesh vertices changes or don't align, or other kinds of artifacts due to the model deforming and CatmullClark subdivision, then you have to worry about using the modifier vs using the shader node in case you need to see the detail in the viewport, etc etc. Some of these are fixable by the user by learning new practices but some inevitably make things slower.

Multires provides an easier and faster way of storing and using that data, per vertex which is all you need in a lot of cases

Now this looks like two distinct cases, so maybe it's worth separating both, and just create an easy way of transfering data from one to the other. Or maybe not, and the new multires ends up working good for animations out of the box. Maybe it could be a switch, where multires gets locked from being sculpted to be used in animation, and unlocked for the opposite. Maybe multires for sculpting should be a native mesh data thing, while multires for animation should be a modifier, where you choose a special file format that contains multires data per vertex index, plus being able to open different multires files

Whatever you guys end up deciding, don't do it at the cost of workflows that people have developed over the years, or without a plan on how to cover them. Put the burden of it on whoever wants to make potentially breaking changes

Added subscriber: @JulienKaspar

Added subscriber: @SecuoyaEx

One thing you can do with the current Multiresolution is modify the base mesh without losing or corrupting the already scupted parts in higher levels, you can add extra edgeloops, extrusions... I hope you guys don't break that

What one considers a priority is very subjective, it would be a shame to lose uniquely blender ways of working, losing flexibility, just to impose one vision of what is important.

Removed subscriber: @SecuoyaEx

Added subscriber: @RodDavis

Added subscriber: @hadrien

In #73047#846737, @AlbertoVelazquez wrote:
I would certainly like to raise... is it really necessary that the multires be a modifier? Given that having more than one modifier of this type is a nonsense and that it is always forced to be the first modifier... wouldn't it be more logical that it simply be a property of the mesh like the vertex groups? Maybe I'm wrong, but I don't see any sense in it being a modifier. Also future sculpt layers could be placed in same mesh tab, like vertex groups or shapes, where have more sense than inside a modifier.

Multires as part of the mesh data sounds good to me - I also see a future where shapekeys can exist at different multires levels, like sculpt layers in other programs - they're really the same thing after all.

And there is another important aspect that I think went unsaid : in rigging, we either use blendshapes or displacement maps (baked from Zbrush multires) to correct deformations or add medium-frequency detail such as wrinkles : if it's a big structural modification/correction, then we're going to use blendshapes ; if it's smaller, superficial/cosmetic change then we're going to use displacement maps. Some corrections however, to be optimal, need a higher resolution than what the basemesh allows - a workflow with multires layers/shapekeys has the potential to completely streamline the creation of both corrective shapes and expressions, as well as everything in-between - so I think it's really essential to consider.
In that case we can imagine that only static, very high frequency detail would be left for rendertime displacement.

From there, it's not a big leap to imagine accessing different multires levels in edit mode as well, although I can see the conversions between edit mesh and sculpt bvh quickly become unmanageable.

Added subscriber: @dlc17

Added subscriber: @T.R.O.Nunes

Added subscriber: @theluthier

Added subscriber: @YegorSmirnov

Added subscriber: @DanieleViagi

Added subscriber: @TheCharacterhero-4

Added subscriber: @0o00o0oo

Added subscriber: @PetterLundh

I am agree, if multires will have shapekeys support in future, then we need to keep the ability to move the low level of mesh by bones and keep all details of multires on high levels in new bone pose. It will give more possibilities for realistic face rigging for example.
Of course it not should be in the first version of new implantation , but need to keep it in mind at the beginning.

Added subscriber: @BartekMoniewski

Added subscriber: @siebeneicher

An important aspect of Blender I rarely find someone speaking about. Its about how to push complexity of your project to its limit without freezing your PC.

Some people mentioned idea, that Multires should not be a modifier, but integrated directly into mesh objects.

On smaller projects, we might not bother about space efficiency. But, what happens, if we push a scene to its limits? Imagine a scene with many thousands of mesh objects.

We could apply same way of thinking on question, wether we use (multires) or (subdiv&displace) in our animation projects. A single multires sculpt could eat away 10G memory. Again, for smaller projects you wont need to bother. But if you have a more complex setup, with many objects (10G each), you freeze your machine by depleting its memory.

If we bake a high res model to displacement, it helps us to reduce memory footprint of our scenes.

@PabloDobarro
Overall this all sounds pretty good. One thing that concerns me right now is backwards compatibility.
Like @WilliamReynish said there is an important use-case for multires at the moment:

IMO one of the main original use-cases for Multires, was as an alternative to using Sub-Ds, but with hierarchical detail. So you can animate, rig and play >back simpler geometry, but render at a higher level. That is one of the core use-cases, which I don't think you mention here.

This is mainly because there is no alternative to this workflow right now since displacement baking is not there yet. But once it is and the old multires has no reason to stay, how will old files, even project files from the Blender Open Movies like Spring react if you open them in a new version of Blender?

Added subscriber: @MatthewHeimlich

I think we need to be careful in this case about being slavish to backwards compatibility. As a developer, I recognize that it's not always smart or sometimes even possible to force yourself to be beholden to the sins of past bad design choices. Sometimes you need to tear things down and start over without being concerned about what effect it's going to have on older work. Do we really have people storming the gate over the fact that you can really only open Sintel in a special Blender branch? I don't think so. Make a breaking change sometimes. It's freeing. The beauty of Blender is that old versions aren't going anywhere for people who want to open legacy data.

Added subscriber: @seaside98-4

Added subscriber: @MadMinstrel

Multires for sculpting is good, and baking vector displacement is also good, but one vital use case that is important in Blender (and not important in other pure sculpting apps) is sculpting on a rigged, posed, animated mesh. This is important for animating muscles and such. Blender has no alternative solution for this such as a muscle system or a robust system for rigging physics-simulated objects.

Granted, the old Multires system is bust anyway, so once you get rid of it, you might as well start small. But you should keep this important functionality in mind for the future.

Added subscriber: @luissantander

Added subscriber: @brecht

What I get from this proposal is mainly the requirement that propagation must be really reliable. So to be clear, the current code is just broken. There are bugs and incomplete code, so that even the intended algorithms are not working, and those algorithms might not be ideal. For me the interesting discussion is really a technical one, getting down to the actual code and algorithms and figuring out the solutions. And then if it turns out rewriting big parts is needed, so be it.

I looked at SculptGL, and found relatively simple code in these two files. However from reading this task I feel like I'm missing something, because to me it seems these algorithms are rather limited:
https://github.com/stephomi/sculptgl/blob/master/src/editing/Reversion.js
https://github.com/stephomi/sculptgl/blob/master/src/editing/Subdivision.js

For example this basic test:

Clear Scene
Add Cube
Change Resolution from 4 to 1
Activate transform tool and rotate 90°
Change Resolution from 1 to 4
Note corners are now distorted.
Undo some steps back to try to get the original cube
Note sure what happened there, but I guess undo works by doing reverse operations and the algorithms are not strictly reversible?

Without proper handling of rotations, for example IK posing at lower levels will not work. For animation it's important to handle this kind of rotation case well of course. The implementation can be simpler like SculptGL if it's not important. But from reading the proposal, probably we need something more advanced.

The classic paper about multiresolution mesh editing is this one, it can be a good starting point for those wanting to understand the problem:
https://mrl.nyu.edu/publications/multires-mesh/

What I get from this proposal is mainly the requirement that propagation must be really reliable. So to be clear, the current code is just broken. There are bugs and incomplete code, so that even the intended algorithms are not working, and those algorithms might not be ideal. For me the interesting discussion is really a technical one, getting down to the actual code and algorithms and figuring out the solutions. And then if it turns out rewriting big parts is needed, so be it. I looked at SculptGL, and found relatively simple code in these two files. However from reading this task I feel like I'm missing something, because to me it seems these algorithms are rather limited: https://github.com/stephomi/sculptgl/blob/master/src/editing/Reversion.js https://github.com/stephomi/sculptgl/blob/master/src/editing/Subdivision.js For example this basic test: * Clear Scene * Add Cube ![sculptgl_before.png](https://archive.blender.org/developer/F8281745/sculptgl_before.png) * Change Resolution from 4 to 1 * Activate transform tool and rotate 90° * Change Resolution from 1 to 4 ![sculptgl_after.png](https://archive.blender.org/developer/F8281750/sculptgl_after.png) * Note corners are now distorted. * Undo some steps back to try to get the original cube ![sculptgl_after_undo.png](https://archive.blender.org/developer/F8281753/sculptgl_after_undo.png) * Note sure what happened there, but I guess undo works by doing reverse operations and the algorithms are not strictly reversible? Without proper handling of rotations, for example IK posing at lower levels will not work. For animation it's important to handle this kind of rotation case well of course. The implementation can be simpler like SculptGL if it's not important. But from reading the proposal, probably we need something more advanced. The classic paper about multiresolution mesh editing is this one, it can be a good starting point for those wanting to understand the problem: https://mrl.nyu.edu/publications/multires-mesh/

@brecht Right, the most important thing is a reliable detail propagation. I mentioned the SculptGL implementation because it has the expected design and behavior for a sculpting multires feature (no need for reshape/apply base, it creates and deletes levels in both directions), and it does not fail when working with brushes. I looked at the code and it also seemed to me that it was simple, so I assumed that it may have limitations like the artifact you mentioned with rotations.

Does the "fixed" version of the current multires modifier have a detail propagation without the SculptGL limitations? Also, some of the mentioned features (like delete lower) are considered as a TODO in the code, so I assume that they are possible to integrate into the current modifier, but I'm not sure if rebuilding lower levels will conflict with some of the existing assumptions (like apply base).

I think that before choosing a solution, we should know how much work different options are going to take to develop, how reliable the result will be and how easy it would be to maintain and integrate with existing and new features (layers, colors, baking, animation...).

I want to remember, that actual blender Multires baking is not reliable, because the bake is incorrect in at least the normal map baking.

I would like to chime in about the implementation of multires in terms of technical things. I haven't read the paper Brecht provided and haven't look into the code of the current multires system but from my user point of view implementation of something in vein of ptex would be the next best thing since it's fairly simple to implement for a quad-based mesh(its just a ptex with vertex displacement data instead of colored pixels), it supports animation pipelines and allow a user to go up and down in resolution at any given time.

the downside of this that the modification of the mesh after ptex multires created upon it can be difficult although it's something that needs to be investigated.

@mankysee I'd avoid ptxes by any means. Long story short, any scenario that I've experienced throughout my career with ptxes was a huge pain.

Why bother with an image-based format, especially one with such widely known performance and implementation limitations as PTex? There's no reason multires data can't be encoded directly into mesh data structures.

Added subscriber: @Teds

In #73047#852027, @MatthewHeimlich wrote:
Why bother with an image-based format, especially one with such widely known performance and implementation limitations as PTex? There's no reason multires data can't be encoded directly into mesh data structures.

I think the important part is not PTex as such, but merely the separation of the vector displacement data from the tessellation scheme. An image format of some sort is good because it's well understood by users and relatively easy to edit, import and export. Additionally, with images you can easily visualize and debug the displacement data, and manipulate it with texture nodes in the future. Also makes layers simple.

A plain old UV-based image can be used as well as long as the initial automatic unwrap can be made quick, reliable and not too wasteful. That places some limitations on the extent of basemesh editing that can be done after starting to sculpt, but these limitation are again well understood thanks to users' experience with uvs in texturing, and can be mitigated with solid baking tools.

In #73047#857352, @MadMinstrel wrote:

In #73047#852027, @MatthewHeimlich wrote:
Why bother with an image-based format, especially one with such widely known performance and implementation limitations as PTex? There's no reason multires data can't be encoded directly into mesh data structures.

I think the important part is not PTex as such, but merely the separation of the vector displacement data from the tessellation scheme. An image format of some sort is good because it's well understood by users and relatively easy to edit, import and export. Additionally, with images you can easily visualize and debug the displacement data, and manipulate it with texture nodes in the future. Also makes layers simple.

Yes, that's what I meant more or less. Ptex is a good example not in an actual implementation but of the principle of generating textures without the need of additional work required to map such images.

In terms of an actual implementation I can imagine a system generating uvs for each subdivided quad and applying sparse textures on top of those. Something not unlike Zbrush is doing(under the hood it's using subdivided meshes and displacement textures of some kind containing various information like depth, orientation and so forth).

Added subscriber: @Josephbburg

It's interesting that @AlbertoVelazquez should mention multi-res baking. Having done some poking around in the baking code myself, it seems multi-res baking is a major branch in the bake code that complicates things tremendously. I think it's one of the few parts of Blender that still uses DerivedMesh. Removing this branch in the bake code by redesigning multi-res may have the benefit of making it easier to design features like AOV baking, bake denoise, etc.

I'm of the opinion that there's no reason to maintain compatibility if the feature was broken to begin with. If people want to bring multi-res meshes in from older versions of Blender, they will probably still be able to do so by saving the various levels of subdivision and applying the shapes to the new multi-res modifier (perhaps this could even be automated somehow).

Added subscriber: @kfir

Added subscriber: @SpectreFirst

Added subscriber: @pawel.palenica

Added subscriber: @VladimirKunyansky

Added subscriber: @ckohl_art

Changed status from 'Needs Triage' to: 'Confirmed'

It sounds to me that the old multires modifier is built on a weak foundation and that maintaining it is increasingly difficult and more effort than is worth doing.

The path forward seems obvious: build a new multires from scratch on top of a strong foundation. Whether that means using SculptGL as a base or something else entirely is up to the implementer.

Quickly getting a working multitres into Blender for sculpting is a high priority.
Having the strong foundation in place will then allow additional feature development on top of the new multires to come after.
However, the old multires should also exist for backwards compatibility. Over time we might rebuild old features/workflow into the new multires on top of the strong foundation, at which point the old multires will become redundant and can be deprecated.

Added subscriber: @martinium

Added subscriber: @ostry

Added subscriber: @AlRedd-3

Hello, any idea when work on this is going to start?

Added subscriber: @AnityEx

Added subscriber: @slumber

Added subscriber: @EvanB

Added subscriber: @Munkle_Mark

Added subscriber: @DirSurya

Added subscriber: @Ape_in_outer_space

Added subscriber: @Isfuelo

This comment was removed by @Isfuelo

Removed subscriber: @moisessalvador

I support the first option. Starting clean with SculptGL base implementation and then add more features with time. It is very often the case with other older software that rely on very old code, always dragging its limitations and issues forward instead of starting clean with fast code...

Removed subscriber: @AnityEx

Added subscriber: @pauanyu_blender

Added subscriber: @JohanTriHandoyo

Added subscriber: @skarkkai

Added subscriber: @Kvendy

Added subscriber: @aoates

Download

What's New

Blender Studio

Manual

Developers Blog

Documentation

Benchmark

Blender Conference

Development Fund

One-time Donations