Blender's triangulator has been rescued :)
This commit fixes errors with concave holes inside polygons.
Simple explanation:
Blender "ScanFill" works by sorting vertices from top-left to bottom-right, and connecting
these vertices with a sorted list of edges they have.
The inner loop then goes over every vertex, its edges, and tries to make triangles by
checking vertices that are next in the list.
- if the triangle has points inside: it creates an edge to this vertex, and continues
- else: add new triangle.
Very simple, fast and efficient. But it needed one more check for the first step: it should
check every vertex inside the triangle, and pick the best vertex for an edge based on forming
the sharpest angle with the tested edge. That solves the case for concave holes.
Blender ScanFill was coded 20 years ago, and is an own invention. I wanted a triangulator that
just fills any collection of polygons, including with holes.
No idea if this was ever published in a paper!
scanfill remove-doubles pass assumes ordered edges (as with curves), otherwise it can hang.
workaround this problem by skipping removing-doubles for mesh ngons, since this isnt such a common case as it is with curves and we can just not support it.
The cause for this is bmesh faces which have zero area have their normals set to a fake value (Z-Up), this would break scanfill, possible fixes are to calculate the faces normal each time or tag as invalid but its easiest to avoid the eternal loop with an extra test.
Bring back some old scanfill code to handle tesselating polys with some zero-length edges. This code used to cause #4544 but I haven't been able to recreate that now, so I'm going to revive the old code and if #4544 does come back, we'll have to fix it some other way I guess (or come up with a better solution for zero-length edges in polys).
- replace "size= 4*( (size+3)/4 );" with "size = (size + 3 ) & ~3;"
- split new_mem_element into 2 functions, one to reset, another to allocate so allocating isnt checking to reset each time.
