Speed up FBX export of vertex colors with numpy #104454

Thomas Barlow · 2023-02-28T01:17:11+01:00

Thomas Barlow commented

2023-02-28 01:17:11 +01:00

Use buffer matching the single precision float type of the vertex color data in foreach_get to avoid having to iterate and cast every element in the C foreach_getset function when single precision float is not 64 bits (it's usually 32 bits). Numpy can do the casting to the 64 bit fbx type much faster with .astype when needed. The largest relative speedup from this occurs with getting the 'color' property of Color attributes because the 'color_srgb' property and Byte Color attributes appear to have additional costs, which are presumably the conversion to sRGB and/or the conversion from byte to single precision float.

Find unique colors and their indices using numpy.

Use numpy to expand 'POINT' domain vertex colors to "ByPolygonVertex" using the loop vertex indices.

'POINT' domain vertex colors at about 1000+ vertices of a subdivided default cube:
~12-14 times faster for Byte Color
~16-22 times faster for Color (~12-19 for 'color_srgb')

'CORNER' domain vertex colors at about 1000+ loops of a subdivided default cube:
~5-7 times faster for Byte Color
~7-9 times faster for Color (~6-7 for 'color_srgb')

This does change the exported fbx because the vertex colors are sorted (albeit not in order because they are viewed as a different type for sorting performance), causing the "Colors" array to be in a different order and the "ColorIndex" array to match the new ordering, but the full array of colors reconstructed by indexing the "Colors" array with each "ColorIndex" in sequence does remain the same, so while the exported file may be different, the result of importing the different file will still be the same.

The t_lvi array is also data that is used by the export of polygon indices and edges/UVs, so the same array used by those patches could be re-used for exporting vertex colors. Any extra loops added to t_lvi for loose edges should be excluded, so the expansion of 'POINT' domain vertex colors would become col_indices = col_indices[t_lvi[:len(me.loops)]].
See #104451 and #104453

This patch benefits the most from using the fast_first_axis_unique helper function over np.unique(..., axis=0) because np.unique would have to sort the array 4 times since each color has 4 components.

This patch depends on
#104447
I wasn't sure if I should include its commit in this PR, I have included it for now.

Use buffer matching the single precision float type of the vertex color data in foreach_get to avoid having to iterate and cast every element in the C foreach_getset function when single precision float is not 64 bits (it's usually 32 bits). Numpy can do the casting to the 64 bit fbx type much faster with .astype when needed. The largest relative speedup from this occurs with getting the 'color' property of Color attributes because the 'color_srgb' property and Byte Color attributes appear to have additional costs, which are presumably the conversion to sRGB and/or the conversion from byte to single precision float. Find unique colors and their indices using numpy. Use numpy to expand 'POINT' domain vertex colors to "ByPolygonVertex" using the loop vertex indices. 'POINT' domain vertex colors at about 1000+ vertices of a subdivided default cube: ~12-14 times faster for Byte Color ~16-22 times faster for Color (~12-19 for 'color_srgb') 'CORNER' domain vertex colors at about 1000+ loops of a subdivided default cube: ~5-7 times faster for Byte Color ~7-9 times faster for Color (~6-7 for 'color_srgb') This does change the exported fbx because the vertex colors are sorted (albeit not in order because they are viewed as a different type for sorting performance), causing the "Colors" array to be in a different order and the "ColorIndex" array to match the new ordering, but the full array of colors reconstructed by indexing the "Colors" array with each "ColorIndex" in sequence does remain the same, so while the exported file may be different, the result of importing the different file will still be the same. ----- The `t_lvi` array is also data that is used by the export of polygon indices and edges/UVs, so the same array used by those patches could be re-used for exporting vertex colors. Any extra loops added to `t_lvi` for loose edges should be excluded, so the expansion of 'POINT' domain vertex colors would become `col_indices = col_indices[t_lvi[:len(me.loops)]]`. See https://projects.blender.org/blender/blender-addons/pulls/104451 and https://projects.blender.org/blender/blender-addons/pulls/104453 This patch benefits the most from using the `fast_first_axis_unique` helper function over `np.unique(..., axis=0)` because np.unique would have to sort the array 4 times since each color has 4 components. This patch depends on https://projects.blender.org/blender/blender-addons/pulls/104447 I wasn't sure if I should include its commit in this PR, I have included it for now.

Thomas Barlow referenced this pull request

2023-02-28 01:18:44 +01:00

Speed up FBX export of polygon indices and edges with numpy #104451

Thomas Barlow referenced this pull request

2023-02-28 01:19:29 +01:00

Speed up FBX export of UVs with numpy #104453

Thomas Barlow requested review from Bastien Montagne 2023-02-28 01:22:25 +01:00

Bastien Montagne force-pushed fbx_numpy_vertex_colors_pr_standalone from 0b6a002392 to a07e1bc6ea

2023-02-28 18:08:26 +01:00

Compare

Bastien Montagne referenced this issue from a commit

2023-03-06 15:17:08 +01:00