It is kind of confusing what the user is trying to accomplish here. Since the Bevel2 and Bevel3 edge weights share some edges, you will get a bevel of a bevel, which usually doesn't look…
Just a couple more comments. Sorry for the slowness, I was busy for the last bunch of days. Am getting back to this now.
The lifetime management of extrudedata is kind of messy and perhaps could lead to mistakes in the future. It seems that if the previous call worked, then extrudedata is freed but if it didn't work, then it isn't. Maybe it should either always or never free extrudedata?
This function does the opposite of what its name says. I think it should be called is_point_outside_bound.
Is this documented anywhere (alt-pressing)? I don't recall seeing it in the draft manual PR for this function.
Explaining what will happen to the geometry and its flags after this next block of code would also be nice.
I find "Which boolean operation to apply" pretty undescriptive here about what this flag does. Can you be more precise? Also, kind of confusing that the external-facing code calls this flag 'invert' while the internal code calls it 'do_subtract'.
Feels cleaner to me to have extrude_boolean_data_create() return a completely populated structure rather than have some fields uninitialized and then populate them here. I'd add two more arguments to extrude_boolean_data_create and pass the do_subtract and remove_coplanar flags in as arguments to that.
Would be nice to comment what change to the BMesh is going to be done by the next bit of code (up to the extrude operator): i.e., what geometry is getting duplicated and how the element tags are being set or unset.
Rather than waiting until I've looked at all the files, I'll add some comments here as I go along. If you like, you can wait until all are accumulated.
A comment explaining the purpose of this function would be nice here.
As I am reviewing the code, you might want to start on a companion PR to update the Blender User Manual to document this feature. Both because we want to be better at committing the doc changes…
I will be trying out the code on various cases as part of my review, but others are welcome to do this too, using the build for patch 130172 at https://builder.blender.org/download/patch/
There are a bunch of warnings when compiling on Linux. Please address these:
[2327/4693] Building CXX object source...mesh.dir/editmesh_extrude_boolean.cc.o .../blender-git/blender/source/blen…