Added option to choose triangulation method on fbx export #104694

Andrej · 2023-06-15T10:03:53+02:00

Andrej commented

2023-06-15 10:03:53 +02:00

Same methods as users can choose from bpy.ops.mesh.quads_convert_to_tris, intergration should work seamlessly, kept the same current default values BEAUTY/BEAUTY.

Same methods as users can choose from `bpy.ops.mesh.quads_convert_to_tris`, intergration should work seamlessly, kept the same current default values BEAUTY/BEAUTY.

👍 1 ❤️ 1

Andrej added 1 commit 2023-06-15 10:03:56 +02:00

36ef660763 Added option to choose triangulation method on fbx export

Andrej requested review from Bastien Montagne 2023-06-15 10:14:59 +02:00

Bastien Montagne commented

2023-06-21 16:07:07 +02:00

While the change itself looks good code-wise, would love to get feedback from experienced users whether this is really a needed one or not, since it adds yet another parameter to the exporter.

maybe @BClark or @Mysteryem can chime in here e.g.?

While the change itself looks good code-wise, would love to get feedback from experienced users whether this is really a needed one or not, since it adds yet another parameter to the exporter. maybe @BClark or @Mysteryem can chime in here e.g.?

Thomas Barlow commented

2023-06-21 22:08:07 +02:00

I almost exclusively model with quads such that if the triangulation ends up being important then it means I've messed up my topology and need to redo some of it, so I'm not really sure in what cases the different triangulation options are needed in general. I guess the n-gon triangulation options could be more important because they result in some of the biggest differences in topology, but I would usually avoid using n-gons in a finished model.

Occasionally I might have a very thin, rigged mesh where I need to ensure that the triangulation of both sides matches to prevent clipping when the mesh deforms, but I can usually do that by putting a Triangulate modifier on the meshes that need it. I suppose it would be more problematic if the meshes needing specific triangulation had shape keys though, since the Apply Modifiers option of the exporter prevents exporting shape keys.

I almost exclusively model with quads such that if the triangulation ends up being important then it means I've messed up my topology and need to redo some of it, so I'm not really sure in what cases the different triangulation options are needed in general. I guess the n-gon triangulation options could be more important because they result in some of the biggest differences in topology, but I would usually avoid using n-gons in a finished model. Occasionally I might have a very thin, rigged mesh where I need to ensure that the triangulation of both sides matches to prevent clipping when the mesh deforms, but I can usually do that by putting a Triangulate modifier on the meshes that need it. I suppose it would be more problematic if the meshes needing specific triangulation had shape keys though, since the `Apply Modifiers` option of the exporter prevents exporting shape keys.

Andrej commented

2023-06-21 22:16:39 +02:00

I mostly work in cinematics and we keep everything in quads until we move mesh outside Blender.
Mostly we need fixed alternate triangulation mod to make sure vertex order will stay intact even if there will be slight variations between the exports.

I mostly work in cinematics and we keep everything in quads until we move mesh outside Blender. Mostly we need fixed alternate triangulation mod to make sure vertex order will stay intact even if there will be slight variations between the exports.

Bob White commented

2023-06-21 22:52:02 +02:00

First-time contributor

The game dev tools that I was maintaining last year still required every mesh to be triangulated before processing, which meant that we needed to incorporate it into every export pipeline (or make our artists do it manually). Streamlining that out of the box seems like a nice QoL improvement for folks in a similar situation.

Thomas Barlow commented

2023-06-21 23:03:30 +02:00

In the .fbx itself, the vertex order is consistent regardless of triangulation (it matches Blender's vertex order). I would guess the software you're importing into is reordering the vertices according to their first use in a polygon or something along those lines. I know Unity reorders materials according to the order they're used by polygons, and it wouldn't surprise if it also does some re-ordering of vertices as part of its mesh optimisations, I can't say I've checked.

Olaf Haag commented

2023-06-22 02:00:47 +02:00

First-time contributor

We're also facing the problem that if you choose Apply Modifiers for triangulation, the shape keys won't get exported. If you have a triangulations modifier, the FBX export without Apply Modifiers just ignores the triangulation modifier and that leads to undesired results. Getting rid of N-gons is important for calculating the tangents, so they stay consistent when we import to other software for baking normal maps, and applying them to the mesh in a game engine.

If I had both, a triangulation modifier and the triangulation setting in the exporter, which takes precedence in this PR when Apply Modifiers is enabled?
If the exporter can't respect the triangulation modifier's settings without Apply Modifiers, I'd rather choose the options being available in the exporter, as long as it keeps the shape keys.

We're also facing the problem that if you choose **Apply Modifiers** for triangulation, the shape keys won't get exported. If you have a triangulations modifier, the FBX export without **Apply Modifiers** just ignores the triangulation modifier and that leads to undesired results. Getting rid of N-gons is important for calculating the tangents, so they stay consistent when we import to other software for baking normal maps, and applying them to the mesh in a game engine. If I had both, a triangulation modifier and the triangulation setting in the exporter, which takes precedence in this PR when **Apply Modifiers** is enabled? If the exporter can't respect the triangulation modifier's settings without **Apply Modifiers**, I'd rather choose the options being available in the exporter, as long as it keeps the shape keys.

Andrej commented

2023-06-22 19:48:17 +02:00

I would guess the software you're importing into is reordering the vertices according to their first use in a polygon or something along those lines.

this problem I've met was happening particularly because of the BEAUTY triangulation method it produced different vertex order after we sculpted mesh a bit (sculpted without creating new geometry) and it caused some problems later on.

If I had both, a triangulation modifier and the triangulation setting in the exporter, which takes precedence in this PR when Apply Modifiers is enabled?

well, it's not really this PR related since this PR is just exposing triangulation settings. But apply modifiers happens first by getting object from depsgraph and only after we triangulate it with bmesh. But just tested it and it seems to loose shape keys even with triangulation without modfiiers. Why is it loosing triangulation though at all - with or without modifiers?

1af41f9617/io_scene_fbx/export_fbx_bin.py (L2573)

ob_to_convert = ob.evaluated_get(depsgraph) if settings.use_mesh_modifiers else ob

1af41f9617/io_scene_fbx/export_fbx_bin.py (L2580)

	    if settings.use_triangles:
                    import bmesh
                    bm = bmesh.new()
                    bm.from_mesh(tmp_me)
                    bmesh.ops.triangulate(bm, faces=bm.faces)
                    bm.to_mesh(tmp_me)
                    bm.free()

>I would guess the software you're importing into is reordering the vertices according to their first use in a polygon or something along those lines. this problem I've met was happening particularly because of the BEAUTY triangulation method it produced different vertex order after we sculpted mesh a bit (sculpted without creating new geometry) and it caused some problems later on. > If I had both, a triangulation modifier and the triangulation setting in the exporter, which takes precedence in this PR when Apply Modifiers is enabled? well, it's not really this PR related since this PR is just exposing triangulation settings. But apply modifiers happens first by getting object from depsgraph and only after we triangulate it with bmesh. But just tested it and it seems to loose shape keys even with triangulation without modfiiers. Why is it loosing triangulation though at all - with or without modifiers? https://projects.blender.org/blender/blender-addons/src/commit/1af41f96171bfa881227670b87d9dcf39ff4746b/io_scene_fbx/export_fbx_bin.py#L2573 ```python ob_to_convert = ob.evaluated_get(depsgraph) if settings.use_mesh_modifiers else ob ```` https://projects.blender.org/blender/blender-addons/src/commit/1af41f96171bfa881227670b87d9dcf39ff4746b/io_scene_fbx/export_fbx_bin.py#L2580 ```python if settings.use_triangles: import bmesh bm = bmesh.new() bm.from_mesh(tmp_me) bmesh.ops.triangulate(bm, faces=bm.faces) bm.to_mesh(tmp_me) bm.free() ```

Bastien Montagne commented

2023-06-23 10:58:58 +02:00

I would guess the software you're importing into is reordering the vertices according to their first use in a polygon or something along those lines.

this problem I've met was happening particularly because of the BEAUTY triangulation method it produced different vertex order after we sculpted mesh a bit (sculpted without creating new geometry) and it caused some problems later on.

If the problem comes from changes in vertices order due to changes in triangulated topology when when original topology remains identical, wouldn't it be a better idea to change the default triangulation method to a topology-stable one instead? That way we avoid adding yet another option...

But just tested it and it seems to loose shape keys even with triangulation without modfiiers. Why is it loosing triangulation though at all - with or without modifiers?

Loosing shapekeys when applying modifiers comes from the fact that you cannot ensure the amount of vertices remain valid after modifiers are applied. Further more, the current mix of shapekeys is applied as a 'virtual' first modifier when evaluating the stack.

Am not sure why shapekeys are used when not applying modifiers though, even with triangulation they should remain valid then.

> >I would guess the software you're importing into is reordering the vertices according to their first use in a polygon or something along those lines. > > this problem I've met was happening particularly because of the BEAUTY triangulation method it produced different vertex order after we sculpted mesh a bit (sculpted without creating new geometry) and it caused some problems later on. If the problem comes from changes in vertices order due to changes in triangulated topology when when original topology remains identical, wouldn't it be a better idea to change the default triangulation method to a topology-stable one instead? That way we avoid adding yet another option... > But just tested it and it seems to loose shape keys even with triangulation without modfiiers. Why is it loosing triangulation though at all - with or without modifiers? Loosing shapekeys when applying modifiers comes from the fact that you cannot ensure the amount of vertices remain valid after modifiers are applied. Further more, the current mix of shapekeys is applied as a 'virtual' first modifier when evaluating the stack. Am not sure why shapekeys are used when not applying modifiers though, even with triangulation they should remain valid then.

Brad Clark commented

2023-06-23 16:25:14 +02:00

I am passing along some feedback from another user that doesn't have an account here. For me, I don't do much with triangle export and care/focus more on animation /rigging data moving back and forth through FBX and because in Maya we never had control over it...but Max users and other FBX users or tools that are working to make very optimized games might find this very valuable.

Feedback from where I posted this thread.

Arvīds Kokins
Thinking back, it's rare that I care about how quads were triangulated (usually they're either flat or misbehaving) but when it does matter (e.g. how vertex colors are interpolated), I just manually triangulate to be able to also preview before exporting
also, curious about a few things myself:

Since the original ngon setting used the slowest, highest quality option, it would effectively be an optimization to use the other ear clip method? if so, curious about performance with bigger meshes
is there a way to review and rotate vertex order in blender for quads? otherwise the fixed/alternate options seem like they'd simply be inconsistent and uncontrollable
Are there situations where the longest diagonal triangulation for quads would be useful? seems like that could only make diagonally stretched quads into less mathematically stable triangles (e.g. worse for normals calculated on the fly that pick an arbitrary base vertex etc.), and I don't even see blender itself offering it when doing triangulation in edit mode (is it a new thing?

I am passing along some feedback from another user that doesn't have an account here. For me, I don't do much with triangle export and care/focus more on animation /rigging data moving back and forth through FBX and because in Maya we never had control over it...but Max users and other FBX users or tools that are working to make very optimized games might find this very valuable. Feedback from where I posted this thread. Arvīds Kokins Thinking back, it's rare that I care about how quads were triangulated (usually they're either flat or misbehaving) but when it does matter (e.g. how vertex colors are interpolated), I just manually triangulate to be able to also preview before exporting also, curious about a few things myself: 1. Since the original ngon setting used the slowest, highest quality option, it would effectively be an optimization to use the other ear clip method? if so, curious about performance with bigger meshes 2. is there a way to review and rotate vertex order in blender for quads? otherwise the fixed/alternate options seem like they'd simply be inconsistent and uncontrollable 3. Are there situations where the longest diagonal triangulation for quads would be useful? seems like that could only make diagonally stretched quads into less mathematically stable triangles (e.g. worse for normals calculated on the fly that pick an arbitrary base vertex etc.), and I don't even see blender itself offering it when doing triangulation in edit mode (is it a new thing?

Thomas Barlow commented

2023-06-23 23:55:16 +02:00

I made an issue to track Triangulate Faces removing shape keys from the export #104714

I made an issue to track Triangulate Faces removing shape keys from the export https://projects.blender.org/blender/blender-addons/issues/104714

👍 1

Andrej commented

2023-07-11 06:40:13 +02:00

If the problem comes from changes in vertices order due to changes in triangulated topology when when original topology remains identical, wouldn't it be a better idea to change the default triangulation method to a topology-stable one instead? That way we avoid adding yet another option...

that would work for my workflow personally but some people probably want to have option to use "BEAUTY" triangulation if they don't worry about keeping the vertex order stable and want better shading

> If the problem comes from changes in vertices order due to changes in triangulated topology when when original topology remains identical, wouldn't it be a better idea to change the default triangulation method to a topology-stable one instead? That way we avoid adding yet another option... that would work for my workflow personally but some people probably want to have option to use "BEAUTY" triangulation if they don't worry about keeping the vertex order stable and want better shading

This pull request has changes conflicting with the target branch.