isect_point_tri_v2 and isect_point_quad_v2 are handling tris/quads
in either clockwise or counter-clockwise vertex orderings. However,
for clockwise order it was considering points that lie on the edges
or vertices as "inside", whereas for counter-clockwise it was treating
them as "outside".
Visibly affected place is VSE: it has an optimization that checks
whether a fully opaque strip image fully covers the rendered area.
When the strip was scaled up to *exactly* cover the rendered area,
the check was failing since isect_point_quad_v2 was saying that a
point is outside the rect.
As far as I can tell, the functions have been "slightly wrong" in
this way for at least 15 years; harder to see through earlier
history in git.
Added a bunch of unit tests to cover this. Without the fix, "edge"
and "corner" cases against "cw" tri/quad were failing.
Performance (checked on clang15 on M1 Max):
- isect_point_tri_v2 is pretty much the same performance (assembly
several instructions shorter),
- isect_point_quad_v2 is about three times *faster* (assembly 2x
shorter), seemingly the compiler is able to use some SIMD now.
Aras Pranckevicius