FBX: Export shape key normals #126491

Open
sap-1 wants to merge 3 commits from sap-1/blender:fbx-blend-shapes into main

When changing the target branch, be careful to rebase the branch in your fork to match. See documentation.
First-time contributor

This adds support for exporting normal deltas from shape keys in FBX, and an export setting to toggle normal deduplication, which appears to be necessary to disable for imported blend shape normals to work in Unity.

The current behavior is to export all zeros as the delta, which causes incorrect normals to appear in some game engines.

Attached (shape-normal-test.blend) is a file you can test shape key normals with. It's a subdivided cube with smooth normals, and it has a shape key that turns it into a sphere. The correct behavior should be looking like a normal cube with rounded edge normals when the shape is set to 0.0, and looking like a normal sphere when the shape is set to 1.0.

This fixes blender/blender-addons#46019. The added setting also fixes #123088, but there is currently another PR #123186 that takes a different approach to fixing it (but that one doesn't fix an additional issue with blend shape normals in Unity described below)

Godot

Examples in Godot 4.3 using the new ufbx importer. If you're testing this, make sure you turn off "Generate LODs" in the Godot mesh import settings as it causes some issues.
Screenshot_20240819_113551

Unity (tested using 2022.3.22f1)

In Unity, the common way to deal with blend shape normals was to either not deal with them (legacy blend shape normals), or to recalculate them instead of importing. Recalculating blend shape normals but importing regular normals would usually cause weird issues where enabling more blend shapes would progressively deteriorate the model's normals. Discarding original normals and instead letting Unity recalculate all normals fixes this, but it ignores custom normals and reduces artistic control. With these changes you can now import blend shape normals from Blender.

I initially experienced the issue described in #123088, where the test file wouldn't even import with custom normals (Unity would force recalculate)
image

I randomly decided to try forcing ByPolygonVertex for the export and Unity finally accepted the custom normals, but the blend shape normals were a disaster, as pictured below. After much debugging this seemed related to using IndexToDirect in general. This is something #123186 doesn't fix (because it only affects ByVertice), the only real way to work around it appears to be to disable deduplicating normals in general. For this I added an "Optimize normals" toggle which avoids deduplicating normals when disabled. I'm not sure if it should be enabled or disabled by default, I kept it enabled for now (i.e. normals are deduplicated by default, as is the current behavior)
image

Leftmost is the one where custom normals were discarded by Unity and recalculated due to the ImportFbx error (notice the cube has sharp edges). Middle is exported with optimize normals disabled but with the old behavior of not exporting shape key normals, for demonstration purposes. Rightmost is exported with optimize normals disabled, and with the new shape key normals export.
image

For clarity, these are the import settings used to obtain the rightmost model
Screenshot_20240819_121442

This adds support for exporting normal deltas from shape keys in FBX, and an export setting to toggle normal deduplication, which appears to be necessary to disable for imported blend shape normals to work in Unity. The current behavior is to export all zeros as the delta, which causes incorrect normals to appear in some game engines. Attached (shape-normal-test.blend) is a file you can test shape key normals with. It's a subdivided cube with smooth normals, and it has a shape key that turns it into a sphere. The correct behavior should be looking like a normal cube with rounded edge normals when the shape is set to 0.0, and looking like a normal sphere when the shape is set to 1.0. This fixes https://projects.blender.org/blender/blender-addons/issues/46019. The added setting also fixes https://projects.blender.org/blender/blender/issues/123088, but there is currently another PR https://projects.blender.org/blender/blender/pulls/123186 that takes a different approach to fixing it (but that one doesn't fix an additional issue with blend shape normals in Unity described below) ## Godot Examples in Godot 4.3 using the new ufbx importer. If you're testing this, make sure you turn off "Generate LODs" in the Godot mesh import settings as it causes some issues. <img width="792" alt="Screenshot_20240819_113551" src="attachments/610474c8-1781-4461-b532-8081b1c17a4c"> ## Unity (tested using 2022.3.22f1) In Unity, the common way to deal with blend shape normals was to either not deal with them (legacy blend shape normals), or to recalculate them instead of importing. Recalculating blend shape normals but importing regular normals would usually cause weird issues where enabling more blend shapes would progressively deteriorate the model's normals. Discarding original normals and instead letting Unity recalculate all normals fixes this, but it ignores custom normals and reduces artistic control. With these changes you can now import blend shape normals from Blender. I initially experienced the issue described in https://projects.blender.org/blender/blender/issues/123088, where the test file wouldn't even import with custom normals (Unity would force recalculate) <img width="566" alt="image" src="attachments/702d0f09-1ffc-4520-8ae1-36b6a825287e"> I randomly decided to try forcing `ByPolygonVertex` for the export and Unity finally accepted the custom normals, but the blend shape normals were a disaster, as pictured below. After much debugging this seemed related to using `IndexToDirect` in general. This is something https://projects.blender.org/blender/blender/pulls/123186 doesn't fix (because it only affects ByVertice), the only real way to work around it appears to be to disable deduplicating normals in general. For this I added an "Optimize normals" toggle which avoids deduplicating normals when disabled. I'm not sure if it should be enabled or disabled by default, I kept it enabled for now (i.e. normals are deduplicated by default, as is the current behavior) ![image](/attachments/3cd9bdbc-9128-4d82-b95e-fe4aa95d85c4) Leftmost is the one where custom normals were discarded by Unity and recalculated due to the ImportFbx error (notice the cube has sharp edges). Middle is exported with optimize normals disabled but with the old behavior of not exporting shape key normals, for demonstration purposes. Rightmost is exported with optimize normals disabled, and with the new shape key normals export. ![image](/attachments/86459a55-2afc-46d3-97b1-0f1aa5c25fae) For clarity, these are the import settings used to obtain the rightmost model <img width="296" alt="Screenshot_20240819_121442" src="attachments/39d410c1-6b10-4538-b898-53be9d2644fd">
sap-1 added 2 commits 2024-08-19 11:51:26 +02:00
Iliya Katushenock added this to the Module: Pipeline & IO project 2024-08-19 20:43:22 +02:00
First-time contributor

Thanks for this patch @sap-1, we're finding it useful in Blender to Unity workflows among vrchat content creator communities, running custom builds for now but would love to see it merged!

Thanks for this patch @sap-1, we're finding it useful in Blender to Unity workflows among vrchat content creator communities, running custom builds for now but would love to see it merged!
First-time contributor

shape_difference_exclude_similar has a problem -- it doesn't return a normals array if it decides there are no differences. After fixing that, this works great!

`shape_difference_exclude_similar` has a problem -- it doesn't return a normals array if it decides there are no differences. After fixing that, this works great!
sap-1 added 1 commit 2024-11-15 17:27:38 +01:00
Author
First-time contributor

Fixed the problem, thanks

Fixed the problem, thanks
This pull request has changes conflicting with the target branch.
  • scripts/addons_core/io_scene_fbx/export_fbx_bin.py

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u fbx-blend-shapes:sap-1-fbx-blend-shapes
git checkout sap-1-fbx-blend-shapes
Sign in to join this conversation.
No reviewers
No Label
Interest
Alembic
Interest
Animation & Rigging
Interest
Asset System
Interest
Audio
Interest
Automated Testing
Interest
Blender Asset Bundle
Interest
BlendFile
Interest
Code Documentation
Interest
Collada
Interest
Compatibility
Interest
Compositing
Interest
Core
Interest
Cycles
Interest
Dependency Graph
Interest
Development Management
Interest
EEVEE
Interest
FBX
Interest
Freestyle
Interest
Geometry Nodes
Interest
glTF
Interest
Grease Pencil
Interest
ID Management
Interest
Images & Movies
Interest
Import Export
Interest
Line Art
Interest
Masking
Interest
Metal
Interest
Modeling
Interest
Modifiers
Interest
Motion Tracking
Interest
Nodes & Physics
Interest
OpenGL
Interest
Overlay
Interest
Overrides
Interest
Performance
Interest
Physics
Interest
Pipeline & IO
Interest
Platforms, Builds & Tests
Interest
Python API
Interest
Render & Cycles
Interest
Render Pipeline
Interest
Sculpt, Paint & Texture
Interest
Text Editor
Interest
Translations
Interest
Triaging
Interest
Undo
Interest
USD
Interest
User Interface
Interest
UV Editing
Interest
VFX & Video
Interest
Video Sequencer
Interest
Viewport & EEVEE
Interest
Virtual Reality
Interest
Vulkan
Interest
Wayland
Interest
Workbench
Interest: X11
Legacy
Asset Browser Project
Legacy
Blender 2.8 Project
Legacy
Milestone 1: Basic, Local Asset Browser
Legacy
OpenGL Error
Meta
Good First Issue
Meta
Papercut
Meta
Retrospective
Meta
Security
Module
Animation & Rigging
Module
Asset System
Module
Core
Module
Development Management
Module
Grease Pencil
Module
Modeling
Module
Nodes & Physics
Module
Pipeline & IO
Module
Platforms, Builds & Tests
Module
Python API
Module
Render & Cycles
Module
Sculpt, Paint & Texture
Module
Triaging
Module
User Interface
Module
VFX & Video
Module
Viewport & EEVEE
Platform
FreeBSD
Platform
Linux
Platform
macOS
Platform
Windows
Severity
High
Severity
Low
Severity
Normal
Severity
Unbreak Now!
Status
Archived
Status
Confirmed
Status
Duplicate
Status
Needs Info from Developers
Status
Needs Information from User
Status
Needs Triage
Status
Resolved
Type
Bug
Type
Design
Type
Known Issue
Type
Patch
Type
Report
Type
To Do
No Milestone
No Assignees
3 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: blender/blender#126491
No description provided.