USD: Skeleton and blend shape import #110912

Merged
Michael Kowalski merged 38 commits from makowalski/blender:usdskel_import into main 2023-08-17 20:11:58 +02:00

Added support for UsdSkel animation import.

This addresses #110076.

For a description of the UsdSkel API, please see: https://openusd.org/23.05/api/usd_skel_page_front.html

  • New USDSkeletonReader class which imports UsdSkelSkeleton primitives as armatures.
  • Extended USDMeshReader to import UsdSkelBlendShape as shape keys.
  • Extended USDMeshReader to import USD skinning data as as deform groups and an armature modifier on the mesh object.
  • Added USDMeshReader::get_local_usd_xform() to override the transform computation to account for the binding transformation for skinned meshes.

Limitations

  • Imported bones do not align as one would expect in Blender. This doesn't appear to affect binding and animation, but looks odd. I believe the FBX importer has similar issues and provides an option to rotate the bones. That feels destructive to me, but we could discuss this as an option for the USD importer as well. Alternatively, perhaps this could be documented as a known limitation for the time being, since it doesn't appear to affect functionality. In other DCCs the UsdSkel bone hierarchy is displayed as relationship lines between parent and child bones. Maybe if some day there is such a display option in Blender as well, that could help in this case.
  • Importing blend shape in-betweens is not yet supported.
  • I ran into one case where the USD contains bind transforms with negative determinants and the resulting Blender bone animations have the wrong rotations. This appears to be similar to issue #82930, which seems to suggest that such matrices can't be properly converted to Blender's axis/roll bone representation. But I haven't researched this in depth and perhaps there is a solution I'm not aware of. For now, if the code encounters a bone matrix with a negative determinant, it issues a warning and does not import the animation. The USD with negative determinants may be found here: https://openusd.org/release/dl_usdskel_examples.html

USD Import Options UI Changes

image

Skeletal Animation Test Files

https://developer.apple.com/augmented-reality/quick-look/models/drummertoy/toy_drummer_idle.usdz

https://developer.apple.com/augmented-reality/quick-look/models/vintagerobot2k/robot_walk_idle.usdz

https://developer.apple.com/augmented-reality/quick-look/models/biplane/toy_biplane_idle.usdz

https://github.com/usd-wg/assets/tree/main/full_assets/ElephantWithMonochord
https://github.com/usd-wg/assets/blob/main/full_assets/ElephantWithMonochord/SoC-ElephantWithMonochord.usdz

https://github.com/PixarAnimationStudios/OpenUSD/blob/release/pxr/usdImaging/bin/testusdview/testenv/testUsdviewSkinning/arm.usda (also attached as a test suite file)

Blend Shape Animation Test File

See attached bs_face_anim_example.usd and usd_blend_shape_test.usda (test suite file).

Added support for `UsdSkel` animation import. This addresses #110076. For a description of the `UsdSkel` API, please see: https://openusd.org/23.05/api/usd_skel_page_front.html - New `USDSkeletonReader` class which imports `UsdSkelSkeleton` primitives as armatures. - Extended `USDMeshReader` to import `UsdSkelBlendShape` as shape keys. - Extended `USDMeshReader` to import USD skinning data as as deform groups and an armature modifier on the mesh object. - Added `USDMeshReader::get_local_usd_xform()` to override the transform computation to account for the binding transformation for skinned meshes. ### Limitations - Imported bones do not align as one would expect in Blender. This doesn't appear to affect binding and animation, but looks odd. I believe the FBX importer has similar issues and provides an option to rotate the bones. That feels destructive to me, but we could discuss this as an option for the USD importer as well. Alternatively, perhaps this could be documented as a known limitation for the time being, since it doesn't appear to affect functionality. In other DCCs the `UsdSkel` bone hierarchy is displayed as relationship lines between parent and child bones. Maybe if some day there is such a display option in Blender as well, that could help in this case. - Importing blend shape in-betweens is not yet supported. - I ran into one case where the USD contains bind transforms with negative determinants and the resulting Blender bone animations have the wrong rotations. This appears to be similar to issue https://projects.blender.org/blender/blender/issues/82930, which seems to suggest that such matrices can't be properly converted to Blender's axis/roll bone representation. But I haven't researched this in depth and perhaps there is a solution I'm not aware of. For now, if the code encounters a bone matrix with a negative determinant, it issues a warning and does not import the animation. The USD with negative determinants may be found here: https://openusd.org/release/dl_usdskel_examples.html ### USD Import Options UI Changes ![image](/attachments/08362f21-96e3-410b-a289-7eacc52aa8c9) ### Skeletal Animation Test Files https://developer.apple.com/augmented-reality/quick-look/models/drummertoy/toy_drummer_idle.usdz https://developer.apple.com/augmented-reality/quick-look/models/vintagerobot2k/robot_walk_idle.usdz https://developer.apple.com/augmented-reality/quick-look/models/biplane/toy_biplane_idle.usdz https://github.com/usd-wg/assets/tree/main/full_assets/ElephantWithMonochord https://github.com/usd-wg/assets/blob/main/full_assets/ElephantWithMonochord/SoC-ElephantWithMonochord.usdz https://github.com/PixarAnimationStudios/OpenUSD/blob/release/pxr/usdImaging/bin/testusdview/testenv/testUsdviewSkinning/arm.usda (also attached as a test suite file) ### Blend Shape Animation Test File See attached `bs_face_anim_example.usd` and `usd_blend_shape_test.usda` (test suite file).
Michael Kowalski added 15 commits 2023-08-08 05:21:17 +02:00
0bf5c465b2 USD: new import_skeleton() function.
Moved skeleton conversion code from the USDSkeletonReader
class to a new import_skeleton() utility function.
b6cee3cdcd USD skel import: comments and cleanup.
Replaced cout statements with Blender warnings and
general cleanup.
83ef176502 USD skel import: optional curve import, comments.
Added flag to control whether to import animation
cuves.  Also added comments and made naming more
descriptive.
3acd02441d USD: refactor bone curve creation.
Made import_skeleton_curves() local to the file
and simplified rest-relative transform computation.
Added comments.
e89681955e USD: cleanup skel import.
Added comments, reporting warnings and
reorganized code to make it easier to
follow.
6a56a8547f USD: Add comments for import_mesh_skel_bindings.
Also reporting warnings rather than printing to std out.
buildbot/vexp-code-patch-coordinator Build done. Details
4dddd6a96d
Format fixes.
Author
Member

@blender-bot build

@blender-bot build
Michael Kowalski added the
Interest
USD
Interest
Pipeline, Assets & IO
labels 2023-08-08 05:23:17 +02:00
Michael Kowalski added this to the USD project 2023-08-08 05:23:36 +02:00
Author
Member

@blender-bot package

@blender-bot package
Member

Package build started. Download here when ready.

Package build started. [Download here](https://builder.blender.org/download/patch/PR110912) when ready.
Author
Member

@blender-bot package

@blender-bot package
Member

Package build started. Download here when ready.

Package build started. [Download here](https://builder.blender.org/download/patch/PR110912) when ready.
Bastien Montagne reviewed 2023-08-08 18:34:54 +02:00
Bastien Montagne left a comment
Owner

From a quick look at the code generally looks fine to me, only have a few minor comments below.

Will complete this pass of review tomorrow, but might be good to get also someone from the Animation team to have a look on it, @dr.sybren ? @nathanvegdahl ?

From a quick look at the code generally looks fine to me, only have a few minor comments below. Will complete this pass of review tomorrow, but might be good to get also someone from the Animation team to have a look on it, @dr.sybren ? @nathanvegdahl ?
@ -984,0 +1023,4 @@
if (!import_params_.import_skeletons || prim_.IsInstanceProxy()) {
/* Use the standard transform computation, since we are ignoring
* skinning data. Note that applying theUsdSkelBindingAPI to an

theUsdSkelBindingAPI missing space ;)

`theUsdSkelBindingAPI` missing space ;)
makowalski marked this conversation as resolved
@ -0,0 +47,4 @@
namespace {
/* Utility: create curve at the given array index. */
FCurve *create_fcurve(int array_index, const char *rna_path)

const char *rna_path could probably still be const std::string &rna_path instead?

And then call BLI_strdup(rna_path.c_str()) below.

`const char *rna_path` could probably still be `const std::string &rna_path` instead? And then call `BLI_strdup(rna_path.c_str())` below.
makowalski marked this conversation as resolved
@ -0,0 +51,4 @@
{
FCurve *fcu = BKE_fcurve_create();
fcu->flag = (FCURVE_VISIBLE | FCURVE_SELECTED);
fcu->rna_path = BLI_strdupn(rna_path, strlen(rna_path));

Could simply call BLI_strdup() here, it's doing exactly that.

Could simply call `BLI_strdup()` here, it's doing exactly that.
makowalski marked this conversation as resolved
@ -0,0 +59,4 @@
/* Utility: create curve at the given array index and
* adds it as a channel to a group. */
FCurve *create_chan_fcurve(
bAction *act, bActionGroup *grp, int array_index, const char *rna_path, int totvert)

const char *rna_path could probably still be const std::string &rna_path instead? would also makes call to this function less verbose below.

`const char *rna_path` could probably still be `const std::string &rna_path` instead? would also makes call to this function less verbose below.
makowalski marked this conversation as resolved
@ -0,0 +78,4 @@
bez.f1 = bez.f2 = bez.f3 = SELECT;
bez.h1 = bez.h2 = HD_AUTO;
insert_bezt_fcurve(fcu, &bez, INSERTKEY_NOFLAGS);
BKE_fcurve_handles_recalc(fcu);

This is a fairly expansive process, I don't think it should be called on every 'bezt' insertion.

Would rather call that once on each curve at the end of import_skeleton_curves() (and import_blendshapes()).

This is a fairly expansive process, I don't think it should be called on every 'bezt' insertion. Would rather call that once on each curve at the end of `import_skeleton_curves()` (and `import_blendshapes()`).
makowalski marked this conversation as resolved
@ -0,0 +238,4 @@
/* Set the curve samples. */
for (double frame : samples) {

picky empty line ;)

*picky* empty line ;)
makowalski marked this conversation as resolved
@ -0,0 +253,4 @@
}
for (int i = 0; i < joint_local_xforms.size(); ++i) {

picky empty line

*picky* empty line
makowalski marked this conversation as resolved
@ -0,0 +406,4 @@
std::set<pxr::TfToken> shapekey_names;
for (int i = 0; i < targets.size(); ++i) {

picky empty line

*picky* empty line
makowalski marked this conversation as resolved
@ -0,0 +559,4 @@
std::vector<FCurve *> curves;
for (auto blendshape_name : blendshapes) {

picky empty line

*picky* empty line
makowalski marked this conversation as resolved
@ -0,0 +685,4 @@
return;
}
/* Check if any bone natrices have negative determinants,

Type: bone natrices have

Type: `bone natrices have`
makowalski marked this conversation as resolved
@ -0,0 +689,4 @@
* indicating negative scales, possibly due to mirroring
* operations. Such matrices can't be propery converted
* to Blender's axis/roll bone representation (see
* https://developer.blender.org/T82930). If we detect

Updated URL: https://projects.blender.org/blender/blender/issues/82930

Updated URL: `https://projects.blender.org/blender/blender/issues/82930`
makowalski marked this conversation as resolved
Michael Kowalski added 3 commits 2023-08-08 21:36:52 +02:00
Michael Kowalski added 1 commit 2023-08-08 22:54:41 +02:00
buildbot/vexp-code-patch-coordinator Build done. Details
4c91b30530
Merge branch 'main' into usdskel_import
Author
Member

Thanks for the review, @mont29! I've made the requested changes.

Thanks for the review, @mont29! I've made the requested changes.
Author
Member

@blender-bot package

@blender-bot package
Member

Package build started. Download here when ready.

Package build started. [Download here](https://builder.blender.org/download/patch/PR110912) when ready.

Thanks, patch LGTM now.

Am getting a crash though in debug build on linux when trying to import robot_walk_idle.usdz and the others, but this is also happening in main, will open a separate bug report for that.

Am still on the fence regarding the bones orientation handling, I have the feeling users won't be very happy the have orthogonal bones if the want to edit their pose or animation... @dfelinto @dr.sybren @nathanvegdahl thoughts on this topic?

Also, @dr.sybren and @nathanvegdahl , adding you as reviewers, please resign if you do not have time for this patch. ;)

Thanks, patch LGTM now. Am getting a crash though in debug build on linux when trying to import `robot_walk_idle.usdz` and the others, but this is also happening in main, will open a separate bug report for that. Am still on the fence regarding the bones orientation handling, I have the feeling users won't be very happy the have orthogonal bones if the want to edit their pose or animation... @dfelinto @dr.sybren @nathanvegdahl thoughts on this topic? Also, @dr.sybren and @nathanvegdahl , adding you as reviewers, please resign if you do not have time for this patch. ;)
Bastien Montagne requested review from Nathan Vegdahl 2023-08-09 12:16:56 +02:00
Bastien Montagne requested review from Sybren A. Stüvel 2023-08-09 12:16:56 +02:00
Bastien Montagne requested review from Bastien Montagne 2023-08-09 12:16:56 +02:00
Bastien Montagne added the
Module
Pipeline, Assets & IO
label 2023-08-09 12:17:17 +02:00
Bastien Montagne added this to the 4.0 milestone 2023-08-09 12:17:23 +02:00
Nathan Vegdahl requested changes 2023-08-10 14:07:20 +02:00
Nathan Vegdahl left a comment
Member

Generally looks good to me. Just some nits. But full disclosure that I only have a cursory knowledge of USD, and am definitely not familiar with its APIs. So this is mainly a Blender-side review.

Generally looks good to me. Just some nits. But full disclosure that I only have a cursory knowledge of USD, and am definitely not familiar with its APIs. So this is mainly a Blender-side review.
@ -0,0 +228,4 @@
pxr::VtMatrix4dArray joint_local_bind_xforms(bind_xforms.size());
for (int i = 0; i < bind_xforms.size(); ++i) {
int parent_id = skel_topology.GetParent(i);
Member

I think this can be const.

I think this can be const.
makowalski marked this conversation as resolved
@ -0,0 +269,4 @@
}
float re = qrot.GetReal();
pxr::GfVec3f im = qrot.GetImaginary();
Member

I think these can be const...?

I think these can be const...?
makowalski marked this conversation as resolved
@ -0,0 +272,4 @@
pxr::GfVec3f im = qrot.GetImaginary();
for (int j = 0; j < 3; ++j) {
int k = 3 * i + j;
Member

I think this can be const.

I think this can be const.
makowalski marked this conversation as resolved
@ -0,0 +283,4 @@
}
for (int j = 0; j < 4; ++j) {
int k = 4 * i + j;
Member

I think this can be const.

I think this can be const.
makowalski marked this conversation as resolved
@ -0,0 +299,4 @@
}
for (int j = 0; j < 3; ++j) {
int k = 3 * i + j;
Member

I think this can be const.

I think this can be const.
makowalski marked this conversation as resolved
@ -0,0 +560,4 @@
return;
}
size_t num_samples = times.size();
Member

I think this can be const.

I think this can be const.
makowalski marked this conversation as resolved
@ -0,0 +669,4 @@
}
/* Sanity check: we should have created a bone for each joint. */
size_t num_joints = skel_topology.GetNumJoints();
Member

I think this can be const.

I think this can be const.
makowalski marked this conversation as resolved
@ -0,0 +703,4 @@
* https://projects.blender.org/blender/blender/issues/82930).
* If we detect such matrices, we will flag an error and won't
* try to import the animation, since the rotations would
* be incorrect in such cases. Unfortunately, the Pixar
Member

This is probably better as future work, but for sampled animation data it should be possible to just convert the animation data to be equivalent for the skeleton we actually end up with. On the other hand, it might just not be worth it, and maybe there are round-tripping considerations I'm not aware of here.

This is probably better as future work, but for sampled animation data it should be possible to just convert the animation data to be equivalent for the skeleton we actually end up with. On the other hand, it might just not be worth it, and maybe there are round-tripping considerations I'm not aware of here.
nathanvegdahl marked this conversation as resolved
@ -0,0 +755,4 @@
std::vector<std::vector<int>> child_bones(num_joints);
for (size_t i = 0; i < num_joints; ++i) {
int parent_idx = skel_topology.GetParent(i);
Member

I think this can be const.

I think this can be const.
makowalski marked this conversation as resolved
@ -0,0 +802,4 @@
pxr::GfVec3f parent_head(parent->head);
pxr::GfVec3f parent_tail(parent->tail);
float new_len = (avg_child_head - parent_head).GetLength();
Member

I think this can be const.

I think this can be const.
makowalski marked this conversation as resolved
@ -0,0 +805,4 @@
float new_len = (avg_child_head - parent_head).GetLength();
/* Be sure not to scale by zero. */
if (new_len > .00001) {
Member

This is probably an unlikely corner case, so maybe not worth the effort. But if you want to take the time:

Rather than a fixed epsilon, it would probably be better to use an epsilon relative to the magnitude of the largest component of parent_head. Something like:

if (new_len > .00001 * fabs(max_magnitude_component(parent_head))) {
    ...

Fixed epsilons can produce strange/unexpected results when the numbers involved are large. (Relative epsilons can also break with e.g. denormal numbers, but I think that's extremely unlikely in this case.) In this case, with a fixed epsilon, there's a risk of the direction of the bone significantly changing during the re-scaling due to floating point rounding error when the head of the parent is close to its children but far away from the origin.

This is probably an unlikely corner case, so maybe not worth the effort. But if you want to take the time: Rather than a fixed epsilon, it would probably be better to use an epsilon relative to the magnitude of the largest component of `parent_head`. Something like: ``` if (new_len > .00001 * fabs(max_magnitude_component(parent_head))) { ... ``` Fixed epsilons can produce strange/unexpected results when the numbers involved are large. (Relative epsilons can also break with e.g. denormal numbers, but I think that's extremely unlikely in this case.) In this case, with a fixed epsilon, there's a risk of the direction of the bone significantly changing during the re-scaling due to floating point rounding error when the head of the parent is close to its children but far away from the origin.
Author
Member

I made the change you suggested. Thanks for pointing out this issue!

It occurs to me in the edge case when parent_head is the origin, the comparison becomes if (new_len > 0), but I suspect this is fine.

I made the change you suggested. Thanks for pointing out this issue! It occurs to me in the edge case when `parent_head` is the origin, the comparison becomes `if (new_len > 0)`, but I suspect this is fine.
Member

Yeah, I was thinking of that and similar edge cases when I mentioned denormal numbers. But indeed, the bones involved would have to be extremely close to the origin (on the order of 1.0e-34 ish units, I think) for that to start to be an issue, which is extremely unlikely in any practical situation.

For comparison, protons are about 1.0e-15 meters in size, according to wikipedia. So we're talking modeling sub-sub-sub-atomic things in real world units. And then rigging them with armatures, ha ha.

(Edit: the Planck length is about 1.0-e35 meters, apparently. So if people are putting bones that close to both each other and the origin without them being literally equal positions, they must be working on a hell of a project!)

Yeah, I was thinking of that and similar edge cases when I mentioned denormal numbers. But indeed, the bones involved would have to be *extremely* close to the origin (on the order of 1.0e-34 ish units, I think) for that to start to be an issue, which is extremely unlikely in any practical situation. For comparison, protons are about 1.0e-15 meters in size, according to wikipedia. So we're talking modeling sub-sub-sub-atomic things in real world units. And then rigging them with armatures, ha ha. (Edit: the [Planck length](https://en.wikipedia.org/wiki/Planck_units) is about 1.0-e35 meters, apparently. So if people are putting bones that close to both each other and the origin without them being literally equal positions, they must be working on a hell of a project!)
nathanvegdahl marked this conversation as resolved
@ -0,0 +815,4 @@
/* Scale terminal bones by the average length scale. */
avg_len_scale /= num_joints;
if (avg_len_scale > .00001) {
Member

Same comment here, regarding fixed epsilons. If you decide this is worth addressing, you'll need to move the check into the loop and compute it per bone.

Same comment here, regarding fixed epsilons. If you decide this is worth addressing, you'll need to move the check into the loop and compute it per bone.
makowalski marked this conversation as resolved
@ -0,0 +1011,4 @@
for (int j = 0; j < joint_weights_elem_size; ++j) {
int k = offset + j;
float w = joint_weights[k];
if (w < .00001) {
Member

I think k, w, and joint_idx (further below) can all be const.

I think k, w, and joint_idx (further below) can all be const.
makowalski marked this conversation as resolved
@ -0,0 +44,4 @@
bool import_anim = true);
/**
* Import the given USD sekeleton as an armature object. Optionally, if the
Member

Typo: sekeleton > skeleton

Typo: sekeleton > skeleton
makowalski marked this conversation as resolved
@ -0,0 +59,4 @@
const pxr::UsdSkelSkeleton &skel,
bool import_anim = true);
/**
* Import skinning data from a source USD prim as deform groups and an armature
Member

I might be misunderstanding the comment, but I think by deform groups you mean vertex groups? That's the Blender terminology.

I might be misunderstanding the comment, but I think by deform groups you mean vertex groups? That's the Blender terminology.
Member

@mont29 informed me that in fact "deform group" is also used in some areas of Blender's code. So never mind! (Might be good to unify the terminology in the code base at some point, but that's obviously not for this PR!)

@mont29 informed me that in fact "deform group" is also used in some areas of Blender's code. So never mind! (Might be good to unify the terminology in the code base at some point, but that's obviously not for this PR!)
nathanvegdahl marked this conversation as resolved
Member

Am still on the fence regarding the bones orientation handling, I have the feeling users won't be very happy the have orthogonal bones if the want to edit their pose or animation... @dfelinto @dr.sybren @nathanvegdahl thoughts on this topic?

If I'm understanding the concern right, then I think it depends somewhat on the primary intended use case for this. If it's interop with USD, then leaving the orientations alone probably makes sense. But if it's purely import, then adapting the orientations to make sense in Blender is probably better. And if it's both, then I have no idea.

> Am still on the fence regarding the bones orientation handling, I have the feeling users won't be very happy the have orthogonal bones if the want to edit their pose or animation... @dfelinto @dr.sybren @nathanvegdahl thoughts on this topic? If I'm understanding the concern right, then I think it depends somewhat on the primary intended use case for this. If it's interop with USD, then leaving the orientations alone probably makes sense. But if it's purely import, then adapting the orientations to make sense in Blender is probably better. And if it's both, then I have no idea.

Am still on the fence regarding the bones orientation handling, I have the feeling users won't be very happy the have orthogonal bones if the want to edit their pose or animation... @dfelinto @dr.sybren @nathanvegdahl thoughts on this topic?

If I'm understanding the concern right, then I think it depends somewhat on the primary intended use case for this. If it's interop with USD, then leaving the orientations alone probably makes sense. But if it's purely import, then adapting the orientations to make sense in Blender is probably better. And if it's both, then I have no idea.

In FBX importer, ensuring 'Blender-like' orientation for bones are two options (force connect, and automatic bone orientation), which are OFF by default...

> > Am still on the fence regarding the bones orientation handling, I have the feeling users won't be very happy the have orthogonal bones if the want to edit their pose or animation... @dfelinto @dr.sybren @nathanvegdahl thoughts on this topic? > > If I'm understanding the concern right, then I think it depends somewhat on the primary intended use case for this. If it's interop with USD, then leaving the orientations alone probably makes sense. But if it's purely import, then adapting the orientations to make sense in Blender is probably better. And if it's both, then I have no idea. In FBX importer, ensuring 'Blender-like' orientation for bones are two options (force connect, and automatic bone orientation), which are OFF by default...
Member

In that case, maybe an option would indeed be good. Although I don't feel strongly about that being part of this PR. It's the kind of thing that can be added later, I think.

In that case, maybe an option would indeed be good. Although I don't feel strongly about that being part of this PR. It's the kind of thing that can be added later, I think.
Author
Member

Thanks so much for the review! I will implement the requested changes.

Regarding the bone orientation. I wonder if another option could be to provide another display option in Blender more suitable to representing joints. Here is how toy_drummer_idle.usdz displays in Omniverse Composer:

image

The joints are represented as spheres with connections between parent and child joints. That way the USD transforms could be kept intact, but they would make more sense visually. I don't know how feasible this would be in Blender. If this seems like a realistic option, we would be happy to work on this on the NVIDIA side, if that makes sense. @CharlesWardlaw for visibility.

Thanks so much for the review! I will implement the requested changes. Regarding the bone orientation. I wonder if another option could be to provide another display option in Blender more suitable to representing joints. Here is how `toy_drummer_idle.usdz` displays in Omniverse Composer: ![image](/attachments/800382c0-1dbc-4bb6-bd04-c9bb3aaadfaf) The joints are represented as spheres with connections between parent and child joints. That way the USD transforms could be kept intact, but they would make more sense visually. I don't know how feasible this would be in Blender. If this seems like a realistic option, we would be happy to work on this on the NVIDIA side, if that makes sense. @CharlesWardlaw for visibility.
101 KiB
Michael Kowalski added 8 commits 2023-08-14 03:11:34 +02:00
Author
Member

@blender-bot build

@blender-bot build
Michael Kowalski added 1 commit 2023-08-14 03:24:24 +02:00
Author
Member

@nathanvegdahl Thanks, again, for the review! At this point, I made most of the changes you requested, I believe, except for a couple you indicated might be addressed in future tasks.

@mont29 @nathanvegdahl I will certainly investigate implementing bone orientation fixes similar to what is done in the FBX importer. But I do wonder if this might be better to do as a future task, given that the current pull request is already fairly complex. However, I will follow your lead on this.

@nathanvegdahl Thanks, again, for the review! At this point, I made most of the changes you requested, I believe, except for a couple you indicated might be addressed in future tasks. @mont29 @nathanvegdahl I will certainly investigate implementing bone orientation fixes similar to what is done in the FBX importer. But I do wonder if this might be better to do as a future task, given that the current pull request is already fairly complex. However, I will follow your lead on this.

At this I think the bone orientation can be a future task yes.

At this I think the bone orientation can be a future task yes.

BTW, I think fixing #110950 should be high priority here, since it basically makes testing this PR impossible for me (and likely anybody using linux builds?)

BTW, I think fixing #110950 should be high priority here, since it basically makes testing this PR impossible for me (and likely anybody using linux builds?)
Author
Member

BTW, I think fixing #110950 should be high priority here, since it basically makes testing this PR impossible for me (and likely anybody using linux builds?)

Indeed! I believe I have a fix for that crash and will create a pull request for you to try today.

> BTW, I think fixing #110950 should be high priority here, since it basically makes testing this PR impossible for me (and likely anybody using linux builds?) Indeed! I believe I have a fix for that crash and will create a pull request for you to try today.
Michael Kowalski added 1 commit 2023-08-14 22:30:22 +02:00
buildbot/vexp-code-patch-coordinator Build done. Details
9091cc7dc9
Merge branch 'main' into usdskel_import
Author
Member

@mont29 FYI, I just merged the latest from main into this pull request, including the fix to the USDZ import crash on linux.

@mont29 FYI, I just merged the latest from main into this pull request, including the fix to the USDZ import crash on linux.
Bastien Montagne approved these changes 2023-08-15 11:00:58 +02:00
Bastien Montagne left a comment
Owner

Can confirm all proposed test files now import perfectly on linux as well.

For me this patch is ready for master.

Would be good to add one or two test cases though to our unittests suit for USD ?

Can confirm all proposed test files now import perfectly on linux as well. For me this patch is ready for master. Would be good to add one or two test cases though to our unittests suit for USD ?

@blender-bot build

@blender-bot build
Nathan Vegdahl approved these changes 2023-08-15 13:44:56 +02:00
Nathan Vegdahl left a comment
Member

Looks good to me as well!

Looks good to me as well!
Sybren A. Stüvel added the
Interest
Animation & Rigging
label 2023-08-15 15:26:18 +02:00
  • Imported bones do not align as one would expect in Blender. This doesn't appear to affect binding and animation, but looks odd. I believe the FBX importer has similar issues and provides an option to rotate the bones. That feels destructive to me, but we could discuss this as an option for the USD importer as well. Alternatively, perhaps this could be documented as a known limitation for the time being, since it doesn't appear to affect functionality.

This can be noted down as a known limitation for now. This has come up with pretty much every importer that can import armatures. Next Animation & Rigging module meeting will actually discuss putting a solution for this on the agenda. It might not get the highest priority, but as a module we do want to have an official plan for this.

As an intermediary solution, there's also the idea of having bones in pose/object mode just displayed as spheres (#106230), which would at least remove the porcupine look.

In other DCCs the UsdSkel bone hierarchy is displayed as relationship lines between parent and child bones. Maybe if some day there is such a display option in Blender as well, that could help in this case.

Yup, that's pretty much what the spheres mode would be.

  • I ran into one case where the USD contains bind transforms with negative determinants and the resulting Blender bone animations have the wrong rotations. This appears to be similar to issue #82930, which seems to suggest that such matrices can't be properly converted to Blender's axis/roll bone representation. But I haven't researched this in depth and perhaps there is a solution I'm not aware of. For now, if the code encounters a bone matrix with a negative determinant, it issues a warning and does not import the animation. The USD with negative determinants may be found here: https://openusd.org/release/dl_usdskel_examples.html

Is that warning displayed in the UI? Or only on the console?

https://developer.apple.com/augmented-reality/quick-look/models/drummertoy/toy_drummer_idle.usdz

When I import this file I have lots of these in the terminal while importing:

Warning: Unsupported type matrix4d for mesh data

If I then switch the viewport to Material mode, Blender crashes. I've posted more info in Blender Chat, as that crash happens at startup in a main branch build as well, so it's unrelated to this PR. It does block me from testing this properly though.

> - Imported bones do not align as one would expect in Blender. This doesn't appear to affect binding and animation, but looks odd. I believe the FBX importer has similar issues and provides an option to rotate the bones. That feels destructive to me, but we could discuss this as an option for the USD importer as well. Alternatively, perhaps this could be documented as a known limitation for the time being, since it doesn't appear to affect functionality. This can be noted down as a known limitation for now. This has come up with pretty much every importer that can import armatures. Next Animation & Rigging module meeting will actually discuss putting a solution for this on the agenda. It might not get the highest priority, but as a module we do want to have an official plan for this. As an intermediary solution, there's also the idea of having bones in pose/object mode just displayed as spheres (#106230), which would at least remove the porcupine look. > In other DCCs the `UsdSkel` bone hierarchy is displayed as relationship lines between parent and child bones. Maybe if some day there is such a display option in Blender as well, that could help in this case. Yup, that's pretty much what the spheres mode would be. > - I ran into one case where the USD contains bind transforms with negative determinants and the resulting Blender bone animations have the wrong rotations. This appears to be similar to issue https://projects.blender.org/blender/blender/issues/82930, which seems to suggest that such matrices can't be properly converted to Blender's axis/roll bone representation. But I haven't researched this in depth and perhaps there is a solution I'm not aware of. For now, if the code encounters a bone matrix with a negative determinant, it issues a warning and does not import the animation. The USD with negative determinants may be found here: https://openusd.org/release/dl_usdskel_examples.html Is that warning displayed in the UI? Or only on the console? > https://developer.apple.com/augmented-reality/quick-look/models/drummertoy/toy_drummer_idle.usdz When I import this file I have lots of these in the terminal while importing: ``` Warning: Unsupported type matrix4d for mesh data ``` If I then switch the viewport to Material mode, Blender crashes. I've posted more info in Blender Chat, as that crash happens at startup in a `main` branch build as well, so it's unrelated to this PR. It does block me from testing this properly though.
Author
Member

Would be good to add one or two test cases though to our unittests suit for USD ?

Yes, I will be adding the test cases. Thanks!

> Would be good to add one or two test cases though to our unittests suit for USD ? Yes, I will be adding the test cases. Thanks!
Sybren A. Stüvel requested changes 2023-08-15 16:58:13 +02:00
Sybren A. Stüvel left a comment
Member

The code is fortunately straight-forward, that's nice! Just one nag about using C-style optional return parameters instead of C++ std::optional<>.

The code is fortunately straight-forward, that's nice! Just one nag about using C-style optional return parameters instead of C++ `std::optional<>`.
@ -45,6 +45,11 @@ class USDStageReader {
void collect_readers(struct Main *bmain);
/* Complete setting up the armature modifiers that

Docstrings should be Doxygen style, so start with /**

Docstrings should be Doxygen style, so start with `/**`
makowalski marked this conversation as resolved
@ -171,0 +155,4 @@
pxr::GfMatrix4d *r_xform,
bool *r_is_constant) const
{
if (!r_xform) {

r_xform is not documented as being optional, and this function has no meaning when it is nullptr. This means that having r_xform == nullptr is a programming error, rather than an expected result of some input. Use BLI_assert() to check such errors, instead of silently ignoring them.

It might even be better to circumvent the whole situation, and make this class of errors impossible. This can be done by changing the return type to std::optional<pxr::GfMatrix4d>. If r_is_constant is optional, document it as such. Otherwise it might be useful to just return an optional tuple (matrix, is_constant).

`r_xform` is not documented as being optional, and this function has no meaning when it is `nullptr`. This means that having `r_xform == nullptr` is a programming error, rather than an expected result of some input. Use `BLI_assert()` to check such errors, instead of silently ignoring them. It might even be better to circumvent the whole situation, and make this class of errors impossible. This can be done by changing the return type to `std::optional<pxr::GfMatrix4d>`. If `r_is_constant` is optional, document it as such. Otherwise it might be useful to just return an optional tuple `(matrix, is_constant)`.
Author
Member

I refactored the function to return an optional tuple, as you recommended. I think this is an elegant solution. Thanks for the suggestion!

I refactored the function to return an optional tuple, as you recommended. I think this is an elegant solution. Thanks for the suggestion!
makowalski marked this conversation as resolved
@ -53,0 +59,4 @@
* \param r_is_constant: Return value flag, set to false if the transform is
* time varying
*/
virtual bool get_local_usd_xform(const float time,

This should document what the returned boolean means.

This should document what the returned boolean means.
makowalski marked this conversation as resolved
Author
Member

@dr.sybren Thank you so much for the comments and testing!

As an intermediary solution, there's also the idea of having bones in pose/object mode just displayed as spheres (#106230), which would at least remove the porcupine look.

That sphere display option looks great and would be a very nice option to have!

Is that warning displayed in the UI? Or only on the console?

That warning is displayed in the UI.

https://developer.apple.com/augmented-reality/quick-look/models/drummertoy/toy_drummer_idle.usdz

When I import this file I have lots of these in the terminal while importing:

Warning: Unsupported type matrix4d for mesh data

Yes, this is coming from the new generic mesh attribute import code. Some USD attribute types aren't currently handled, which is to be expected. But maybe the warnings should be made less verbose.

If I then switch the viewport to Material mode, Blender crashes. I've posted more info in Blender Chat, as that crash happens at startup in a main branch build as well, so it's unrelated to this PR. It does block me from testing this properly though.

I'm not set up to run linux at the moment to help diagnose, unfortunately. I will try to set myself up for linux building/testing going forward.

@dr.sybren Thank you so much for the comments and testing! > As an intermediary solution, there's also the idea of having bones in pose/object mode just displayed as spheres (#106230), which would at least remove the porcupine look. That sphere display option looks great and would be a very nice option to have! > Is that warning displayed in the UI? Or only on the console? That warning is displayed in the UI. > > https://developer.apple.com/augmented-reality/quick-look/models/drummertoy/toy_drummer_idle.usdz > > When I import this file I have lots of these in the terminal while importing: > ``` > Warning: Unsupported type matrix4d for mesh data > ``` Yes, this is coming from the new generic mesh attribute import code. Some USD attribute types aren't currently handled, which is to be expected. But maybe the warnings should be made less verbose. > If I then switch the viewport to Material mode, Blender crashes. I've posted more info in Blender Chat, as that crash happens at startup in a `main` branch build as well, so it's unrelated to this PR. It does block me from testing this properly though. I'm not set up to run linux at the moment to help diagnose, unfortunately. I will try to set myself up for linux building/testing going forward.
Michael Kowalski added 6 commits 2023-08-16 20:50:46 +02:00
Author
Member

I added test cases for skeleton and blend shape import. These require the attached usd_blend_shape_test.usda and arm.usda files to be copied to the tests/usd/ directory.

I believe at this point requested changes have been addressed.

I added test cases for skeleton and blend shape import. These require the attached `usd_blend_shape_test.usda` and `arm.usda` files to be copied to the `tests/usd/` directory. I believe at this point requested changes have been addressed.
Author
Member

@blender-bot build

@blender-bot build

If I then switch the viewport to Material mode, Blender crashes. I've posted more info in Blender Chat, as that crash happens at startup in a main branch build as well, so it's unrelated to this PR. It does block me from testing this properly though.

This was caused by an issue in the color management code, it's been fixed in #111144.

> If I then switch the viewport to Material mode, Blender crashes. I've posted more info in Blender Chat, as that crash happens at startup in a `main` branch build as well, so it's unrelated to this PR. It does block me from testing this properly though. This was caused by an issue in the color management code, it's been fixed in #111144.
Sybren A. Stüvel approved these changes 2023-08-17 10:36:02 +02:00
Sybren A. Stüvel left a comment
Member

Just a few smaller remarks, these can be addressed when landing the PR.

Just a few smaller remarks, these can be addressed when landing the PR.
@ -97,0 +110,4 @@
* Override transform computation to account for the binding
* transformation for skinned meshes.
*/
std::optional<XformResult> get_local_usd_xform(const float time) const override;

Remove the const from const float time, in the declaration it has no meaning. See the C/C++ style guide for more info.

Remove the `const` from `const float time`, in the declaration it has no meaning. See the C/C++ style guide for more info.
makowalski marked this conversation as resolved
@ -321,0 +350,4 @@
if (!reader->object()) {
continue;
}
if (const USDMeshReader *mesh_reader = dynamic_cast<const USDMeshReader *>(reader)) {

Swap the condition and continue here as well. That wayall the precondition checks use the same logic, and the main path of the for-body can be followed vertically.

Swap the condition and `continue` here as well. That wayall the precondition checks use the same logic, and the main path of the for-body can be followed vertically.
makowalski marked this conversation as resolved
@ -171,0 +169,4 @@
return std::nullopt;
}
return XformResult(pxr::GfMatrix4f(xform), is_constant);

As xform is already a pxr::GfMatrix4f, and that's the expected type for a XformResult, why is this cast necessary? Might be worth a comment.

As `xform` is already a `pxr::GfMatrix4f`, and that's the expected type for a `XformResult`, why is this cast necessary? Might be worth a comment.
Author
Member

That's a good point, and I've added a comment to explain this. xform is a matrix for doubles (GfMatrix4d), but Blender expects a matrix of floats, so I explicitly convert to a GfMatrix4f for the return value.

That's a good point, and I've added a comment to explain this. `xform` is a matrix for doubles (`GfMatrix4d`), but Blender expects a matrix of floats, so I explicitly convert to a `GfMatrix4f` for the return value.
makowalski marked this conversation as resolved
@ -12,6 +12,8 @@
namespace blender::io::usd {
using XformResult = std::tuple<pxr::GfMatrix4f, bool>;

Document what the boolean means.

Document what the boolean means.
makowalski marked this conversation as resolved
@ -53,0 +58,4 @@
*
* \param time: Time code for evaluating the transform.
*
* \return: Optional tuple with the following elements:

An explanation fo the type should be done at the type declaration itself, not at one specific use of that type.

An explanation fo the type should be done at the type declaration itself, not at one specific use of that type.
makowalski marked this conversation as resolved
Michael Kowalski added 2 commits 2023-08-17 18:44:05 +02:00
buildbot/vexp-code-patch-coordinator Build done. Details
8d67a3a9b4
USD skel import: misc cleanup.
Fixes based on review comments by Sybren.
Author
Member

@blender-bot build

@blender-bot build
Michael Kowalski added 1 commit 2023-08-17 19:20:16 +02:00
buildbot/vexp-code-patch-coordinator Build done. Details
85ad6b89a7
USD skel import: added comment.
Author
Member

@blender-bot build

@blender-bot build
Michael Kowalski merged commit 6f4cb9bc56 into main 2023-08-17 20:11:58 +02:00
Michael Kowalski deleted branch usdskel_import 2023-08-17 20:12:01 +02:00
Bastien Montagne removed this from the USD project 2023-08-25 23:53:27 +02:00
Michael Kowalski added this to the USD project 2023-11-07 16:30:25 +01:00
Sign in to join this conversation.
No Label
Interest
Alembic
Interest
Animation & Rigging
Interest
Asset Browser
Interest
Asset Browser Project Overview
Interest
Audio
Interest
Automated Testing
Interest
Blender Asset Bundle
Interest
BlendFile
Interest
Collada
Interest
Compatibility
Interest
Compositing
Interest
Core
Interest
Cycles
Interest
Dependency Graph
Interest
Development Management
Interest
EEVEE
Interest
EEVEE & Viewport
Interest
Freestyle
Interest
Geometry Nodes
Interest
Grease Pencil
Interest
ID Management
Interest
Images & Movies
Interest
Import Export
Interest
Line Art
Interest
Masking
Interest
Metal
Interest
Modeling
Interest
Modifiers
Interest
Motion Tracking
Interest
Nodes & Physics
Interest
OpenGL
Interest
Overlay
Interest
Overrides
Interest
Performance
Interest
Physics
Interest
Pipeline, Assets & IO
Interest
Platforms, Builds & Tests
Interest
Python API
Interest
Render & Cycles
Interest
Render Pipeline
Interest
Sculpt, Paint & Texture
Interest
Text Editor
Interest
Translations
Interest
Triaging
Interest
Undo
Interest
USD
Interest
User Interface
Interest
UV Editing
Interest
VFX & Video
Interest
Video Sequencer
Interest
Virtual Reality
Interest
Vulkan
Interest
Wayland
Interest
Workbench
Interest: X11
Legacy
Blender 2.8 Project
Legacy
Milestone 1: Basic, Local Asset Browser
Legacy
OpenGL Error
Meta
Good First Issue
Meta
Papercut
Meta
Retrospective
Meta
Security
Module
Animation & Rigging
Module
Core
Module
Development Management
Module
EEVEE & Viewport
Module
Grease Pencil
Module
Modeling
Module
Nodes & Physics
Module
Pipeline, Assets & IO
Module
Platforms, Builds & Tests
Module
Python API
Module
Render & Cycles
Module
Sculpt, Paint & Texture
Module
Triaging
Module
User Interface
Module
VFX & Video
Platform
FreeBSD
Platform
Linux
Platform
macOS
Platform
Windows
Priority
High
Priority
Low
Priority
Normal
Priority
Unbreak Now!
Status
Archived
Status
Confirmed
Status
Duplicate
Status
Needs Info from Developers
Status
Needs Information from User
Status
Needs Triage
Status
Resolved
Type
Bug
Type
Design
Type
Known Issue
Type
Patch
Type
Report
Type
To Do
No Milestone
No project
No Assignees
5 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: blender/blender#110912
No description provided.