Fix #99887: Multires Unsubdivide freezes Blender with hidden geometry #121086
No reviewers
Labels
No Label
Interest
Alembic
Interest
Animation & Rigging
Interest
Asset Browser
Interest
Asset Browser Project Overview
Interest
Audio
Interest
Automated Testing
Interest
Blender Asset Bundle
Interest
BlendFile
Interest
Collada
Interest
Compatibility
Interest
Compositing
Interest
Core
Interest
Cycles
Interest
Dependency Graph
Interest
Development Management
Interest
EEVEE
Interest
EEVEE & Viewport
Interest
Freestyle
Interest
Geometry Nodes
Interest
Grease Pencil
Interest
ID Management
Interest
Images & Movies
Interest
Import Export
Interest
Line Art
Interest
Masking
Interest
Metal
Interest
Modeling
Interest
Modifiers
Interest
Motion Tracking
Interest
Nodes & Physics
Interest
OpenGL
Interest
Overlay
Interest
Overrides
Interest
Performance
Interest
Physics
Interest
Pipeline, Assets & IO
Interest
Platforms, Builds & Tests
Interest
Python API
Interest
Render & Cycles
Interest
Render Pipeline
Interest
Sculpt, Paint & Texture
Interest
Text Editor
Interest
Translations
Interest
Triaging
Interest
Undo
Interest
USD
Interest
User Interface
Interest
UV Editing
Interest
VFX & Video
Interest
Video Sequencer
Interest
Virtual Reality
Interest
Vulkan
Interest
Wayland
Interest
Workbench
Interest: X11
Legacy
Blender 2.8 Project
Legacy
Milestone 1: Basic, Local Asset Browser
Legacy
OpenGL Error
Meta
Good First Issue
Meta
Papercut
Meta
Retrospective
Meta
Security
Module
Animation & Rigging
Module
Core
Module
Development Management
Module
EEVEE & Viewport
Module
Grease Pencil
Module
Modeling
Module
Nodes & Physics
Module
Pipeline, Assets & IO
Module
Platforms, Builds & Tests
Module
Python API
Module
Render & Cycles
Module
Sculpt, Paint & Texture
Module
Triaging
Module
User Interface
Module
VFX & Video
Platform
FreeBSD
Platform
Linux
Platform
macOS
Platform
Windows
Priority
High
Priority
Low
Priority
Normal
Priority
Unbreak Now!
Status
Archived
Status
Confirmed
Status
Duplicate
Status
Needs Info from Developers
Status
Needs Information from User
Status
Needs Triage
Status
Resolved
Type
Bug
Type
Design
Type
Known Issue
Type
Patch
Type
Report
Type
To Do
No Milestone
No project
No Assignees
3 Participants
Notifications
Due Date
No due date set.
Dependencies
No dependencies set.
Reference: blender/blender#121086
Loading…
Reference in New Issue
No description provided.
Delete Branch "farsthary/blender:multires_unsubdivide_freezes"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Previously in
With hidden faces this code enters in an infinite loop by returning the same null
edge in edge_step() and the same current_vertex_y causing Blender to crash.
The current fix solves the infinite loop and provides a solution when faces are hidden.
This causes the hidden faces to be merged into one for unsubdivided per original dev's implementation.
At minimum the crash is identified and a solution proposed.
Attached is a video of the fix
The description was a bit confusing because the block of code isn't the previous state, it's the new state in the PR.
But the fix makes sense. To make things clear though, maybe it's best to remove the null check from
edge_step
. If I'm reading things right, the argument should always be non-null now.Thanks! The null check is necessary in the while loop and after since the function edge_step() will can return null and make the edge_y or edge_x null.
I've tried what you suggest and it crash later because of that. As is now works correctly.
I know it can return null, but there is a null check on the next iteration of the while loop that covers this. Unless I'm really missing something, It just seems wrong/weird to check for null on the condition of the while loop, then check again inside the loop itself. Where did it crash later on?
What I meant is that on the second null check,
only if edge_* is not null we should modify the returning values of r_corner_*
The while loop can run and modify internally the edge_* and make it null at the end of the steps
(that's why it was hanging on forever)
The first null check is to break the loop, and the second is to ensure no r_corner_x or r_corner_y is modified in such case.
@HooglyBoogly without the second null check it crashes on Debug only (on Release the BLI_abort() seems to be successfully ignored but still is a crash) it results in a vertex index to be -1 or something like that.
Adding the null check run successfully in both debug and release.
No worth saving a null check.
I am a bit confused about the null-pointer discussion. Just to be sure we're talking about the same thing, this is the null-pointer check we are talking about, right?
Is a bit hard to predict all consequences, as it is used from outside of the loops changed here. But i can not reproduce the crash reported by Raul when he removed the check.
Surely for the final state of code having this block removed is probably clear, but for getting there with a bug fixed, I suggest doing such cleanup as a separate commit after actual functional changes are done. It will make review and bisecting (if something goes wrong) easier.
For this PR it is much more important to tackle the
multires_unsubdivide_get_grid_corners_on_base_mesh
itself. With the proposed change it is possible that it leavesr_corner_{x, y}
uninitialized, and the code which calls it expects both of the corners to be initialized:This actually might be the reason of the crash Raul is getting? It kind of aligns with the code later
const int corner_x_index = orig_to_base_vmap[BM_elem_index_get(corner_x)];
which will crash if corner is not initialized.I am not fully sure sure what is expected to happen in the case
multires_unsubdivide_get_grid_corners_on_base_mesh
did not find one of the vertices. Let me know if you want extra eyes for figuring this out!@Sergey I can 100% repro the Debug build crash with this file.
Steps:
1-compile blender Debug
2-open the file and press unsubdivide
Xcode will trigger a break at
@farsthary Did you check if it's due to the unintiialized
corner_x
/corner_y
after the callmultires_unsubdivide_get_grid_corners_on_base_mesh()
?With your proposed changes, removing the null check inside the function
static BMEdge *edge_step(BMVert *v, BMEdge *edge, BMVert **r_next_vertex),
we get a worse crash down the road becauseBM_edge_other_vert(edge, v)
do not expect NULL edges.I still vote for the original PR solution which is simple and gets the job done, 4 LOC.
Or better yet, either
Unsubdivide completely ignores hidden face state and process every face as visible,
(because creating random N-gons will mess up the modeling pipeline later)
or Unsubdivide throws a warning message about hidden faces and do not proceed.
But I think those should be subject of a separated PR, this one is better to keep it short and small about fixing the infinite loop.
@farsthary My suggestion was indeed to leave the null-pointer check removal to the separate PR. However, I think there is an use-of-uninitialized variable with this PR.
There is this code in the
multires_unsubdivide_extract_grids
:It clearly expects
corner_x
andcorner_y
to be initialized bymultires_unsubdivide_get_grid_corners_on_base_mesh()
. But with the proposed patch it is not guaranteed.I am not really sure why ASAB builds do not catch any issues. This could be due to the fact that all allocations are done via memory pools, and maybe we do not poison it for Apple Clang or something. Or, maybe, it is because of the code being used from a loop, and those variables are always initialized to their previous value the issue does not manifest itself that much.
In any case, we should not rely on such behavior of uninitialized variables and handle it in a more explicit manner.
Fix #99887: Multires Unsubdivide freezes Blender with hidden geoemtryto WIP: Fix #99887: Multires Unsubdivide freezes Blender with hidden geoemtryWIP: Fix #99887: Multires Unsubdivide freezes Blender with hidden geoemtryto WIP: Fix #99887: Multires Unsubdivide freezes Blender with hidden geometry@ -1043,6 +1053,8 @@ static void multires_unsubdivide_extract_grids(MultiresUnsubdivideContext *conte
* base mesh of the face of grid that is going to be extracted. */
BMVert *corner_x, *corner_y;
multires_unsubdivide_get_grid_corners_on_base_mesh(l->f, l->e, &corner_x, &corner_y);
if (!corner_x || !corner_y)
The style guide mentions always using braces for nested scopes: https://developer.blender.org/docs/handbook/guidelines/c_cpp/#braces
wow that's a fast review 😅
Typically "WIP" in the PR title means it isn't ready for review. I'm guessing that's not he case here?
I'll remove the WIP in the next push
WIP: Fix #99887: Multires Unsubdivide freezes Blender with hidden geometryto Fix #99887: Multires Unsubdivide freezes Blender with hidden geometryReady for a final review, hopefully we can get this infinite loop hang crossed over since that's a severe bug out in the open for users.
The code seems to make sense to me now, thanks for the update :)
Did you ever hear back from Julien or Daniel what the expected behavior is?
Thanks @Sergey
I haven't heard from them yet. Worst case if the expected behavior is to ignore hidden faces or halt the unsubdivide operator with a warning we can make it a new PR.
After 1 or 2 approvals I can proceed to merge this.
Checkout
From your project repository, check out a new branch and test the changes.