Speed up FBX export of polygon indices and edges with numpy #104451

Thomas Barlow · 2023-02-27T23:43:25+01:00

Thomas Barlow commented

2023-02-27 23:43:25 +01:00

Use buffers matching the C types of the data in foreach_get to avoid having to iterate and cast every single element in the C foreach_getset function.

Get the edge index for each exported edge key and use this to map per-edge data, such as creases, to the exported edges.

~9-17 times faster for a minimal export (no loose edges, smoothing or creases exported)
~16-23 times faster for a maximal export with FACE smoothing (loose edges, face smoothing and creases exported)
~32-40 times faster for a maximal export with EDGE smoothing (loose edges, edge smoothing and creases exported)

Isolated parts:
~5-7 times faster loose edges processing
~2-4 times faster face smoothing export
~73-96 times faster edge smoothing export
~62-75 times faster creases export

In the rare case where there exist multiple edges with the same edge-key, this patch can result in a different export because the single edge exported from those multiple edges will have the per-edge data of the first found edge in loops. Before this patch, the per-edge data would be from whichever edge with the same edge-key was iterated last since it would overwrite the previously set per-edge data of the exported edge.

Polygon indices and per-edge data are interconnected so had to be part of one patch. This covers the export of polygons and edges, loose edges, edge/face smoothing and edge creases for subdivision.

The t_lvi array used in this patch could be re-used in the export of UVs and/or Vertex Colors since the patches for both of those areas require the same data albeit without any extra elements that have been added for loose edges. As a note on doing this t_pvi array may currently be a view of t_lvi, but for t_lvi to be re-used for UVs/Vertex Colors, t_pvi would have to always be a created as a copy of t_lvi so that modifying t_pvi does not also modify t_lvi. This can be achieved by replacing t_pvi = astype_view_signedness(t_lvi, pvi_fbx_dtype) with t_pvi = t_lvi.astype(pvi_fbx_dtype)
See #104453 and #104454

This patch depends on
#104447
I wasn't sure if I should include its commit in this PR, I have included it for now.

Use buffers matching the C types of the data in foreach_get to avoid having to iterate and cast every single element in the C foreach_getset function. Get the edge index for each exported edge key and use this to map per-edge data, such as creases, to the exported edges. ~9-17 times faster for a minimal export (no loose edges, smoothing or creases exported) ~16-23 times faster for a maximal export with FACE smoothing (loose edges, face smoothing and creases exported) ~32-40 times faster for a maximal export with EDGE smoothing (loose edges, edge smoothing and creases exported) Isolated parts: ~5-7 times faster loose edges processing ~2-4 times faster face smoothing export ~73-96 times faster edge smoothing export ~62-75 times faster creases export In the rare case where there exist multiple edges with the same edge-key, this patch can result in a different export because the single edge exported from those multiple edges will have the per-edge data of the first found edge in loops. Before this patch, the per-edge data would be from whichever edge with the same edge-key was iterated last since it would overwrite the previously set per-edge data of the exported edge. ----- Polygon indices and per-edge data are interconnected so had to be part of one patch. This covers the export of polygons and edges, loose edges, edge/face smoothing and edge creases for subdivision. The `t_lvi` array used in this patch could be re-used in the export of UVs and/or Vertex Colors since the patches for both of those areas require the same data albeit without any extra elements that have been added for loose edges. As a note on doing this `t_pvi` array may currently be a view of `t_lvi`, but for `t_lvi` to be re-used for UVs/Vertex Colors, `t_pvi` would have to always be a created as a copy of `t_lvi` so that modifying `t_pvi` does not also modify `t_lvi`. This can be achieved by replacing `t_pvi = astype_view_signedness(t_lvi, pvi_fbx_dtype)` with `t_pvi = t_lvi.astype(pvi_fbx_dtype)` See https://projects.blender.org/blender/blender-addons/pulls/104453 and https://projects.blender.org/blender/blender-addons/pulls/104454 This patch depends on https://projects.blender.org/blender/blender-addons/pulls/104447 I wasn't sure if I should include its commit in this PR, I have included it for now.

Thomas Barlow referenced this pull request

2023-02-28 01:19:29 +01:00

Speed up FBX export of UVs with numpy #104453

Thomas Barlow referenced this pull request

2023-02-28 01:20:11 +01:00

Speed up FBX export of vertex colors with numpy #104454

Thomas Barlow requested review from Bastien Montagne 2023-02-28 01:21:39 +01:00

Bastien Montagne force-pushed fbx_numpy_polygon_indices_and_edges_pr from b2db1885e0 to dabe504e28

2023-02-28 18:07:05 +01:00

Compare

Bastien Montagne commented

2023-03-06 14:43:18 +01:00

In the rare case where there exist multiple edges with the same edge-key, this patch can result in a different export because the single edge exported from those multiple edges will have the per-edge data of the first found edge in loops. Before this patch, the per-edge data would be from whichever edge with the same edge-key was iterated last since it would overwrite the previously set per-edge data of the exported edge.

Such case is actually an invalid mesh in Blender, so modified behavior here is not an issue.

> In the rare case where there exist multiple edges with the same edge-key, this patch can result in a different export because the single edge exported from those multiple edges will have the per-edge data of the first found edge in loops. Before this patch, the per-edge data would be from whichever edge with the same edge-key was iterated last since it would overwrite the previously set per-edge data of the exported edge. Such case is actually an invalid mesh in Blender, so modified behavior here is not an issue.

Bastien Montagne added 1 commit 2023-03-06 15:11:13 +01:00

Merge branch 'main' into fbx_numpy_polygon_indices_and_edges_pr 23e3fad929

Bastien Montagne merged commit d155650711 into main

2023-03-06 15:17:07 +01:00

Bastien Montagne referenced this issue from a commit

2023-03-06 15:17:07 +01:00

Speed up FBX export of polygon indices and edges with numpy

Bastien Montagne deleted branch fbx_numpy_polygon_indices_and_edges_pr

2023-03-06 15:17:08 +01:00

Bastien Montagne referenced this issue from a commit

2023-03-06 15:28:19 +01:00

Speed up FBX export of UVs with numpy