FBX: Export shape key normals #126491

sap-1 · 2024-08-19T11:51:14+02:00

sap-1 commented

2024-08-19 11:51:14 +02:00

First-time contributor

This adds support for exporting normal deltas from shape keys in FBX, and an export setting to toggle normal deduplication, which appears to be necessary to disable for imported blend shape normals to work in Unity.

The current behavior is to export all zeros as the delta, which causes incorrect normals to appear in some game engines.

Attached (shape-normal-test.blend) is a file you can test shape key normals with. It's a subdivided cube with smooth normals, and it has a shape key that turns it into a sphere. The correct behavior should be looking like a normal cube with rounded edge normals when the shape is set to 0.0, and looking like a normal sphere when the shape is set to 1.0.

This fixes blender/blender-addons#46019. The added setting also fixes #123088, but there is currently another PR #123186 that takes a different approach to fixing it (but that one doesn't fix an additional issue with blend shape normals in Unity described below)

Godot

Examples in Godot 4.3 using the new ufbx importer. If you're testing this, make sure you turn off "Generate LODs" in the Godot mesh import settings as it causes some issues.

Unity (tested using 2022.3.22f1)

In Unity, the common way to deal with blend shape normals was to either not deal with them (legacy blend shape normals), or to recalculate them instead of importing. Recalculating blend shape normals but importing regular normals would usually cause weird issues where enabling more blend shapes would progressively deteriorate the model's normals. Discarding original normals and instead letting Unity recalculate all normals fixes this, but it ignores custom normals and reduces artistic control. With these changes you can now import blend shape normals from Blender.

I initially experienced the issue described in #123088, where the test file wouldn't even import with custom normals (Unity would force recalculate)

I randomly decided to try forcing ByPolygonVertex for the export and Unity finally accepted the custom normals, but the blend shape normals were a disaster, as pictured below. After much debugging this seemed related to using IndexToDirect in general. This is something #123186 doesn't fix (because it only affects ByVertice), the only real way to work around it appears to be to disable deduplicating normals in general. For this I added an "Optimize normals" toggle which avoids deduplicating normals when disabled. I'm not sure if it should be enabled or disabled by default, I kept it enabled for now (i.e. normals are deduplicated by default, as is the current behavior)

Leftmost is the one where custom normals were discarded by Unity and recalculated due to the ImportFbx error (notice the cube has sharp edges). Middle is exported with optimize normals disabled but with the old behavior of not exporting shape key normals, for demonstration purposes. Rightmost is exported with optimize normals disabled, and with the new shape key normals export.

For clarity, these are the import settings used to obtain the rightmost model

This adds support for exporting normal deltas from shape keys in FBX, and an export setting to toggle normal deduplication, which appears to be necessary to disable for imported blend shape normals to work in Unity. The current behavior is to export all zeros as the delta, which causes incorrect normals to appear in some game engines. Attached (shape-normal-test.blend) is a file you can test shape key normals with. It's a subdivided cube with smooth normals, and it has a shape key that turns it into a sphere. The correct behavior should be looking like a normal cube with rounded edge normals when the shape is set to 0.0, and looking like a normal sphere when the shape is set to 1.0. This fixes https://projects.blender.org/blender/blender-addons/issues/46019. The added setting also fixes https://projects.blender.org/blender/blender/issues/123088, but there is currently another PR https://projects.blender.org/blender/blender/pulls/123186 that takes a different approach to fixing it (but that one doesn't fix an additional issue with blend shape normals in Unity described below) ## Godot Examples in Godot 4.3 using the new ufbx importer. If you're testing this, make sure you turn off "Generate LODs" in the Godot mesh import settings as it causes some issues. <img width="792" alt="Screenshot_20240819_113551" src="attachments/610474c8-1781-4461-b532-8081b1c17a4c"> ## Unity (tested using 2022.3.22f1) In Unity, the common way to deal with blend shape normals was to either not deal with them (legacy blend shape normals), or to recalculate them instead of importing. Recalculating blend shape normals but importing regular normals would usually cause weird issues where enabling more blend shapes would progressively deteriorate the model's normals. Discarding original normals and instead letting Unity recalculate all normals fixes this, but it ignores custom normals and reduces artistic control. With these changes you can now import blend shape normals from Blender. I initially experienced the issue described in https://projects.blender.org/blender/blender/issues/123088, where the test file wouldn't even import with custom normals (Unity would force recalculate) <img width="566" alt="image" src="attachments/702d0f09-1ffc-4520-8ae1-36b6a825287e"> I randomly decided to try forcing `ByPolygonVertex` for the export and Unity finally accepted the custom normals, but the blend shape normals were a disaster, as pictured below. After much debugging this seemed related to using `IndexToDirect` in general. This is something https://projects.blender.org/blender/blender/pulls/123186 doesn't fix (because it only affects ByVertice), the only real way to work around it appears to be to disable deduplicating normals in general. For this I added an "Optimize normals" toggle which avoids deduplicating normals when disabled. I'm not sure if it should be enabled or disabled by default, I kept it enabled for now (i.e. normals are deduplicated by default, as is the current behavior) ![image](/attachments/3cd9bdbc-9128-4d82-b95e-fe4aa95d85c4) Leftmost is the one where custom normals were discarded by Unity and recalculated due to the ImportFbx error (notice the cube has sharp edges). Middle is exported with optimize normals disabled but with the old behavior of not exporting shape key normals, for demonstration purposes. Rightmost is exported with optimize normals disabled, and with the new shape key normals export. ![image](/attachments/86459a55-2afc-46d3-97b1-0f1aa5c25fae) For clarity, these are the import settings used to obtain the rightmost model <img width="296" alt="Screenshot_20240819_121442" src="attachments/39d410c1-6b10-4538-b898-53be9d2644fd">

Screenshot_20240819_121442.png

23 KiB

image.png

13 KiB

shape-normal-test.blend

673 KiB

Screenshot_20240819_113551.png

417 KiB

image.png

178 KiB

image.png

123 KiB

❤️ 2

sap-1 added 2 commits 2024-08-19 11:51:26 +02:00

FBX exporter: Export shape key normals 69a03f3774

FBX Exporter: Add setting to disable normal deduplication aff199a2cc

Iliya Katushenock added this to the Pipeline, Assets & IO project 2024-08-19 20:43:22 +02:00

Phoenix Fox commented

2024-08-28 04:23:24 +02:00

First-time contributor

Thanks for this patch @sap-1, we're finding it useful in Blender to Unity workflows among vrchat content creator communities, running custom builds for now but would love to see it merged!

Merge conflict checking is in progress. Try again in few moments.

View command line instructions.

Checkout

From your project repository, check out a new branch and test the changes.

git fetch -u fbx-blend-shapes:sap-1-fbx-blend-shapes

git checkout sap-1-fbx-blend-shapes

Sign in to join this conversation.

No reviewers

No Label

Download

What's New

Blender Studio

Manual

Developers Blog

Documentation

Benchmark

Blender Conference

Development Fund

One-time Donations

FBX: Export shape key normals #126491

Godot

Unity (tested using 2022.3.22f1)

Checkout