Collada importer removes all bones after internally setting their length to zero #101177

New Issue

Perv · 2022-09-18T23:09:46+02:00

Perv commented

2022-09-18 23:09:46 +02:00

Simplified Report

Import the following file.
The importer removes them with the following log output.
This seems to be caused by the fix_leaf_bone_hierarchy setting the length of all bones to a value close to zero.

blender sample broken armature (7 bones) base.dae

Warning: removed zero sized bone: Bip01
Warning: removed zero sized bone: Bip01_Pelvis
Warning: removed zero sized bone: Bip01_Spine
Warning: removed zero sized bone: Bip01_Spine1
Warning: removed zero sized bone: Bip01_Spine2
Warning: removed zero sized bone: Bip01_Neck
Warning: removed zero sized bone: Bip01_Neck01

Original Report

System Information
Operating system: Windows-10-10.0.19044-SP0 64 Bits
Graphics card: NVIDIA GeForce RTX 2080 Ti/PCIe/SSE2 NVIDIA Corporation 4.5.0 NVIDIA 516.94

Blender Version
Broken: At least as far back as 2.78, and up to 3.3.0 (branch: master, commit date: 2022-09-06 15:39, hash: 0759f671ce)
Worked: N/A

Short description of error
Collada import of a mesh with skeleton may destructively import "successfully" (no console errors) if there's a child bone with "length 0" which has it's own child bones but is not weighted to anything. The mesh imports just fine but the skeleton does not, in my test case this causes most of the bones (238) to disappear and only about 10 bones survive.

Exact steps for others to reproduce the error
Import a Collada mesh with a skeleton that has a child bone with length 0, which has it's own child bones and is not referenced by the skin controller, such as the sample file below (bone "Bip01_Neck01"):

pew_00_nude_0001_noalpha_mesh00_lod0 broken.dae

The go-to workaround for the problematic file above for years has been to use "Find Bone Chains" (and usually also "Fix Leaf Bones") during import, this does recover most bones, however there are still a few missing (including the "Bip01" root bone), and they're no longer aligned with the skeleton in use by the game, resulting clipping issues, twisted wrists, etc.

New workarounds as of yesterday include adjusting the bone values so that the length won't be "0", in the file below I've copied values from "Bip01_Neck" (parent) to "Bip01_Neck01" (child):

pew_00_nude_0001_noalpha_mesh00_lod0 broken workaround.dae

Another workaround is to include the bone in the skin controller, where the other files only listed the bones in use by the mesh, this one lists all of them, which seems to prevent the bone from being destructively removed during import:

pew_00_nude_0001_noalpha_mesh00_lod0 refAllBones.dae

## Simplified Report - Import the following file. - The importer removes them with the following log output. - This seems to be caused by the `fix_leaf_bone_hierarchy` setting the length of all bones to a value close to zero. [blender sample broken armature (7 bones) base.dae](https://archive.blender.org/developer/F13562326/blender_sample_broken_armature__7_bones__base.dae) ``` Warning: removed zero sized bone: Bip01 Warning: removed zero sized bone: Bip01_Pelvis Warning: removed zero sized bone: Bip01_Spine Warning: removed zero sized bone: Bip01_Spine1 Warning: removed zero sized bone: Bip01_Spine2 Warning: removed zero sized bone: Bip01_Neck Warning: removed zero sized bone: Bip01_Neck01 ``` ## Original Report **System Information** Operating system: Windows-10-10.0.19044-SP0 64 Bits Graphics card: NVIDIA GeForce RTX 2080 Ti/PCIe/SSE2 NVIDIA Corporation 4.5.0 NVIDIA 516.94 **Blender Version** Broken: At least as far back as 2.78, and up to 3.3.0 (branch: master, commit date: 2022-09-06 15:39, hash: `0759f671ce`) Worked: N/A **Short description of error** Collada import of a mesh with skeleton may destructively import "successfully" (no console errors) if there's a child bone with "length 0" which has it's own child bones but is not weighted to anything. The mesh imports just fine but the skeleton does not, in my test case this causes most of the bones (238) to disappear and only about 10 bones survive. **Exact steps for others to reproduce the error** Import a Collada mesh with a skeleton that has a child bone with length 0, which has it's own child bones and is *not* referenced by the skin controller, such as the sample file below (bone "Bip01_Neck01"): [pew_00_nude_0001_noalpha_mesh00_lod0 broken.dae](https://archive.blender.org/developer/F13507641/pew_00_nude_0001_noalpha_mesh00_lod0_broken.dae) The go-to workaround for the problematic file above for years has been to use "Find Bone Chains" (and usually also "Fix Leaf Bones") during import, this does recover most bones, however there are still a few missing (including the "Bip01" root bone), and they're no longer aligned with the skeleton in use by the game, resulting clipping issues, twisted wrists, etc. New workarounds as of yesterday include adjusting the bone values so that the length won't be "0", in the file below I've copied values from "Bip01_Neck" (parent) to "Bip01_Neck01" (child): [pew_00_nude_0001_noalpha_mesh00_lod0 broken workaround.dae](https://archive.blender.org/developer/F13507640/pew_00_nude_0001_noalpha_mesh00_lod0_broken_workaround.dae) Another workaround is to include the bone in the skin controller, where the other files only listed the bones in use by the mesh, this one lists all of them, which seems to prevent the bone from being destructively removed during import: [pew_00_nude_0001_noalpha_mesh00_lod0 refAllBones.dae](https://archive.blender.org/developer/F13508277/pew_00_nude_0001_noalpha_mesh00_lod0_refAllBones.dae)

Perv commented

2022-09-18 23:09:46 +02:00

Added subscriber: @Perv

Germano Cavalcante commented

2022-09-23 02:03:14 +02:00

Added subscriber: @mano-wii

Germano Cavalcante commented

2022-09-23 02:03:14 +02:00

Changed status from 'Needs Triage' to: 'Archived'

Germano Cavalcante closed this issue

2022-09-23 02:03:14 +02:00

Germano Cavalcante commented

2022-09-23 02:03:14 +02:00

Blender does not allow zero-sized bones. See {https://developer.blender.org/diffusion/B/browse/master/source/blender/editors/armature/armature_utils.c$675-688}.

This is an old behavior (before 2.5) and not easy to change. So it's not surprising the bones with length 0 disappear when imported into Blender.

Thanks for the report, but the issue reported here is a request for modified/improved behavior and not a bug in current behavior. Closing as this bug tracker is only for bugs and errors.

For user requests and feedback, please use other channels: https://wiki.blender.org/wiki/Communication/Contact#User_Feedback_and_Requests

For more information on why this isn't considered a bug, visit: https://wiki.blender.org/wiki/Reference/Not_a_bug

Blender does not allow zero-sized bones. See {https://developer.blender.org/diffusion/B/browse/master/source/blender/editors/armature/armature_utils.c$675-688}. This is an old behavior (before 2.5) and not easy to change. So it's not surprising the bones with length 0 disappear when imported into Blender. Thanks for the report, but the issue reported here is a request for modified/improved behavior and not a bug in current behavior. Closing as this bug tracker is only for bugs and errors. For user requests and feedback, please use other channels: https://wiki.blender.org/wiki/Communication/Contact#User_Feedback_and_Requests For more information on why this isn't considered a bug, visit: https://wiki.blender.org/wiki/Reference/Not_a_bug

Perv commented

2022-09-23 06:44:03 +02:00

Again, the bones do NOT get removed if the bones are "referenced" (no weights) in the joints array, so the behavior to preserve "zero-sized" bones already exists as it can be achieved by referencing all the bones, it could use some tweaking to also apply to all bones that have their own child bones.

Regardless of whether you think zero-sized bones should exist or not, this IS a bug, because again, the entire armature is destroyed, including unrelated bones, that should NOT be intended behavior, in the sample files I posted you can compare the final armature that is imported between the "broken" and "refAllBones" files, it'd be fine if it simply removed all the "zero-sized" bones while leaving a functional armature (child bones being moved one node up, where the parent "zero" bone was) and otherwise unchanged, but it doesn't, it makes the armature unusable.

Again, the bones do NOT get removed if the bones are "referenced" (no weights) in the joints array, so the behavior to preserve "zero-sized" bones already exists as it can be achieved by referencing all the bones, it could use some tweaking to also apply to all bones that have their own child bones. Regardless of whether you think zero-sized bones should exist or not, this IS a bug, because again, the entire armature is destroyed, including unrelated bones, that should NOT be intended behavior, in the sample files I posted you can compare the final armature that is imported between the "broken" and "refAllBones" files, it'd be fine if it simply removed all the "zero-sized" bones while leaving a functional armature (child bones being moved one node up, where the parent "zero" bone was) and otherwise unchanged, but it doesn't, it makes the armature unusable.

Germano Cavalcante commented

2022-09-23 15:09:53 +02:00

Changed status from 'Archived' to: 'Needs Triage'

Germano Cavalcante reopened this issue

2022-09-23 15:09:53 +02:00

Germano Cavalcante commented

2022-09-23 15:09:53 +02:00

The sample files are very complex. It would be preferable to simplify to just the armature with as few bones as possible.

To demonstrate the bug in Collada, it is recommended to use a simplified .blend file that can be exported and re-imported.

I'm not familiar with how zero-sized bones are preserved if they are referenced.

In my opinion this report falls into the category Too complex file report, where user could not simplify enough on the Triaging_Playbook .

The sample files are very complex. It would be preferable to simplify to just the armature with as few bones as possible. To demonstrate the bug in Collada, it is recommended to use a simplified .blend file that can be exported and re-imported. I'm not familiar with how zero-sized bones are preserved if they are referenced. In my opinion this report falls into the category **Too complex file report, where user could not simplify enough** on the [Triaging_Playbook ](https://wiki.blender.org/wiki/Process/Bug_Reports/Triaging_Playbook#Should_Close_Task_Scenarios).

Perv commented

2022-09-24 07:35:04 +02:00

The issue is on import, the Collada file is typically generated by another tool, not Blender, so I don't know how (or if) there's a way to generate a broken armature via Blender, importing with a mesh that references a "zero" bone and then exporting just the armature does not seem to result a skeleton that breaks on import.

I have however simplified the sample, the issue is reproducible with just 7 bones (6 parent nodes to the problematic bone and the problematic bone itself), in that scenario no bones survive the import, an empty armature is added to the scene, I've first also tried with 24 bones, keeping some child bones of the problematic bones and other child bones to the different parent bones, only one bone survives, the other child of the root bone, I've then also tried 8 bones, the same 7 I explained above and the second child of the root bone, it was the only bone to survive again.
I've created "non-zero" variations of the samples above where the values of the parent to the problematic bone were copied, and another variation where the problematic bone is removed, in both cases all bones get imported correctly.

blender sample broken armature (7 bones) base.dae

blender sample broken armature (7 bones) non-zero.dae

blender sample broken armature (7 bones) removed.dae

blender sample broken armature (8 bones) base.dae

blender sample broken armature (8 bones) non-zero.dae

blender sample broken armature (8 bones) removed.dae

blender sample broken armature (24 bones) base.dae

blender sample broken armature (24 bones) non-zero.dae

blender sample broken armature (24 bones) removed.dae

The issue is on import, the Collada file is typically generated by another tool, not Blender, so I don't know how (or if) there's a way to generate a broken armature via Blender, importing with a mesh that references a "zero" bone and then exporting just the armature does not seem to result a skeleton that breaks on import. I have however simplified the sample, the issue is reproducible with just 7 bones (6 parent nodes to the problematic bone and the problematic bone itself), in that scenario no bones survive the import, an empty armature is added to the scene, I've first also tried with 24 bones, keeping some child bones of the problematic bones and other child bones to the different parent bones, only one bone survives, the other child of the root bone, I've then also tried 8 bones, the same 7 I explained above and the second child of the root bone, it was the only bone to survive again. I've created "non-zero" variations of the samples above where the values of the parent to the problematic bone were copied, and another variation where the problematic bone is removed, in both cases all bones get imported correctly. [blender sample broken armature (7 bones) base.dae](https://archive.blender.org/developer/F13562326/blender_sample_broken_armature__7_bones__base.dae) [blender sample broken armature (7 bones) non-zero.dae](https://archive.blender.org/developer/F13562327/blender_sample_broken_armature__7_bones__non-zero.dae) [blender sample broken armature (7 bones) removed.dae](https://archive.blender.org/developer/F13562328/blender_sample_broken_armature__7_bones__removed.dae) [blender sample broken armature (8 bones) base.dae](https://archive.blender.org/developer/F13562329/blender_sample_broken_armature__8_bones__base.dae) [blender sample broken armature (8 bones) non-zero.dae](https://archive.blender.org/developer/F13562330/blender_sample_broken_armature__8_bones__non-zero.dae) [blender sample broken armature (8 bones) removed.dae](https://archive.blender.org/developer/F13562331/blender_sample_broken_armature__8_bones__removed.dae) [blender sample broken armature (24 bones) base.dae](https://archive.blender.org/developer/F13562332/blender_sample_broken_armature__24_bones__base.dae) [blender sample broken armature (24 bones) non-zero.dae](https://archive.blender.org/developer/F13562333/blender_sample_broken_armature__24_bones__non-zero.dae) [blender sample broken armature (24 bones) removed.dae](https://archive.blender.org/developer/F13562334/blender_sample_broken_armature__24_bones__removed.dae)

Scurest commented

2022-09-25 17:08:39 +02:00

Added subscriber: @scurest

Scurest commented

2022-09-25 17:08:39 +02:00

Looking at the first file (7 bones), the matrix for the leaf bone is

<matrix sid="matrix">
  1 0 0 0
  0 1 0 0
  0 0 1 0.000002
  0 0 0 1
</matrix>

Some observations:

If the 0.000_002 is decreased to 0.000_001 or below, the bones don't get deleted.
If the 0.000_002 is increased only some of the bones get deleted. The crossover where it starts fully working is somewhere between 0.000_007 and 0.000_008.
The head of the leaf bone (when it is correctly created) is (x,y,z) and the tail is (x,y+L,z), where y=170.795 and L is the bone length.
The minimum bone length the Collada importer uses is 0.000_001.

I think I can combine these into a tentative theory about what happens:

As you know floats have more precision around the origin and less as you move away from the origin. Near y=170.795 the gap between floats is 1/2^16 or about 0.000_0153. Adding something less than half of this to y is the same as adding zero, since it gets rounded back down to y. The halfway point is 0.000_00763, which notably lies in the range noted above to contain the crossover from "broken" to "working".

So if the code does something like this, where the length is checked before calculating the tail

// in collada importer...
if bone_length < min_length {
  // ensure bone_length > min_length
}
tail = head + bone_length * direction

// later, when exiting edit mode...
if too_small((tail - head).length) {
  // delete bone
}

it won't work, since the "mathematical" assumption that (tail-head).length = bone_length > min_length doesn't hold for floats.

Looking at the first file (7 bones), the matrix for the leaf bone is ``` <matrix sid="matrix"> 1 0 0 0 0 1 0 0 0 0 1 0.000002 0 0 0 1 </matrix> ``` Some observations: * If the 0.000_002 is decreased to 0.000_001 or below, the bones don't get deleted. * If the 0.000_002 is increased only some of the bones get deleted. The crossover where it starts fully working is somewhere between 0.000_007 and 0.000_008. * The head of the leaf bone (when it is correctly created) is (x,y,z) and the tail is (x,y+L,z), where y=170.795 and L is the bone length. * The minimum bone length the Collada importer uses [is 0.000_001](https://github.com/blender/blender/blob/594f47ecd2d5367ca936cf6fc6ec8168c2b360d0/source/blender/io/collada/ArmatureImporter.h#L51). I think I can combine these into a tentative theory about what happens: As you know floats have more precision around the origin and less as you move away from the origin. Near y=170.795 the gap between floats is 1/2^16 or about 0.000_0153. Adding something less than half of this to y is the same as adding zero, since it gets rounded back down to y. The halfway point is 0.000_00763, which notably lies in the range noted above to contain the crossover from "broken" to "working". So if the code does something like this, where the length is checked before calculating the tail ``` // in collada importer... if bone_length < min_length { // ensure bone_length > min_length } tail = head + bone_length * direction // later, when exiting edit mode... if too_small((tail - head).length) { // delete bone } ``` it won't work, since the "mathematical" assumption that (tail-head).length = bone_length > min_length doesn't hold for floats.

Omar Emara commented

2022-09-26 11:22:14 +02:00

Added subscriber: @OmarEmaraDev

Omar Emara commented

2022-09-26 11:22:14 +02:00

Changed status from 'Needs Triage' to: 'Confirmed'

Omar Emara commented

2022-09-26 11:22:14 +02:00

This seems like a bug indeed. But I am not really sure if it is related to your investigation on float precision.
The issues seems to be caused by the fix_leaf_bone_hierarchy function, which seems to consider all bones to be leaf bones in bc_is_leaf_bone and then it adjust the length of the bones to be close to zero which is then removed in the next call to ED_armature_from_edit. This seems to be the unexpected part.

This seems like a bug indeed. But I am not really sure if it is related to your investigation on float precision. The issues seems to be caused by the `fix_leaf_bone_hierarchy` function, which seems to consider all bones to be leaf bones in `bc_is_leaf_bone` and then it adjust the length of the bones to be close to zero which is then removed in the next call to `ED_armature_from_edit`. This seems to be the unexpected part.

Omar Emara changed title from ~~Collada mesh import is potentially destructive to the included skeleton~~ to Collada importer removes all bones after internally setting their length to zero

2022-09-26 11:27:53 +02:00

Perv commented

2022-09-26 13:06:39 +02:00

With the original report I've stated that there's a workaround, where referencing the problematic bone in the joints array of the mesh (even without assigning any weights to it) causes it to import correctly, could it be that one of the leaf bone checks fails due to that reference?

Also it might be worth mentioning that similar behavior was experienced with 3ds Max (skeleton does not import correctly unless the "zero sized" bone is referenced in the joints array of a mesh), I thought the issue might be with OpenCollada code that both Blender and 3ds share.

With the original report I've stated that there's a workaround, where referencing the problematic bone in the joints array of the mesh (even without assigning any weights to it) causes it to import correctly, could it be that one of the leaf bone checks fails due to that reference? Also it might be worth mentioning that similar behavior was experienced with 3ds Max (skeleton does not import correctly unless the "zero sized" bone is referenced in the joints array of a mesh), I thought the issue might be with OpenCollada code that both Blender and 3ds share.

Bastien Montagne added this to the Pipeline, Assets & IO project 2023-02-09 15:39:30 +01:00

Sign in to join this conversation.

No Label

Download

What's New

Blender Studio

Manual

Developers Blog

Documentation

Benchmark

Blender Conference

Development Fund

One-time Donations

Collada importer removes all bones after internally setting their length to zero #101177

Simplified Report

Original Report