USD: Support Python hooks and chasers to extend IO functionality #108825

New Issue

Michael Kowalski · 2023-06-09T22:39:35+02:00

Michael Kowalski commented

2023-06-09 22:39:35 +02:00

Overview

This task is meant to initiate a discussion of design strategies for supporting Python hooks and callbacks (also known as chasers) for extending USD import and export functionality. Although the current discussion will focus on USD IO in particular, the intent is to design an approach that can be generalized to support other formats as well.

As context, some DCCs allow extending the application's USD IO functionality through custom plugins. For example, Solaris in Houdini supports authoring custom material shader translators as Python plugins . Maya’s USD plugin supports the concept of chasers for import and export.

There have been previous discussions in the pipeline-assets-io-module chat for supporting Python customization of Blender’s USD IO, as described here. In addition, a limited prototype was implemented in the universal-scene-description branch for converting USD materials through Python callbacks. This prototype is currently being refactored and will be described in a separate design task in the near future.

Such hooks can be very useful for adding missing or experimental USD conversion functionality and for handling custom schemas and attributes. The set of supported hooks might include the following:

On Import/Export. Called immediately after import and before export finalizes, respectively.
On Prim Import/Export. Customize per-prim conversion.
Material Conversion. Handle converting custom material types. On export, multiple hooks could be used to support different shading contexts for the same material.

Arguments for such callbacks would typically include a pointer to the USD stage and a mapping between USD prims and Blender objects.

Implementation

The implementation strategy for this feature in Blender requires further discussion. As a starting point, it may be useful to describe a simple and direct approach for invoking such callbacks in user-specified modules:

Users implement Python add-ons that define the required chaser functions.
The chaser module names are provided as a parameter to the USD IO operators.
The USD IO code calls the chaser functions in the given modules using the boost::python bindings provided by the USD libraries.

An advantage of this strategy is that it is straightforward to implement and requires no changes to Blender code outside the USD IO module. However, providing module names as a parameter requires an extra step in the workflow and is error prone.

An improved design would be to allow registering new types of classes for implementing USD IO hooks, using the RNA framework. This has the advantages of allowing automatic registration of hooks and potentially better integration with Blender's UI.

I've provided a simple example implementations of a new bpy.types.USDHook class type which defines the callbacks as member functions and which can be sub classed and registered in add-ons. A prototype implementation of this strategy is in pull request #108823.

I welcome feedback and further discussion of these ideas.

### Overview This task is meant to initiate a discussion of design strategies for supporting Python hooks and callbacks (also known as `chasers`) for extending USD import and export functionality. Although the current discussion will focus on USD IO in particular, the intent is to design an approach that can be generalized to support other formats as well. As context, some DCCs allow extending the application's USD IO functionality through custom plugins. For example, `Solaris` in Houdini supports authoring custom material shader [translators](https://www.sidefx.com/docs/houdini/solaris/shader_framework.html#writing-a-custom-shader-translator) as Python plugins . Maya’s USD plugin supports the concept of `chasers` for [import](https://github.com/Autodesk/maya-usd/blob/dev/lib/mayaUsd/commands/Readme.md#import-chasers) and [export](https://github.com/Autodesk/maya-usd/blob/dev/lib/mayaUsd/commands/Readme.md#export-chasers-advancedl). There have been previous discussions in the `pipeline-assets-io-module` chat for supporting Python customization of Blender’s USD IO, as described [here](https://projects.blender.org/blender/blender/issues/101283). In addition, a limited prototype was implemented in the `universal-scene-description` branch for converting USD materials through Python callbacks. This prototype is currently being refactored and will be described in a separate design task in the near future. Such hooks can be very useful for adding missing or experimental USD conversion functionality and for handling custom schemas and attributes. The set of supported hooks might include the following: - **On Import/Export.** Called immediately after import and before export finalizes, respectively. - **On Prim Import/Export.** Customize per-prim conversion. - **Material Conversion.** Handle converting custom material types. On export, multiple hooks could be used to support different shading contexts for the same material. Arguments for such callbacks would typically include a pointer to the USD stage and a mapping between USD prims and Blender objects. ### Implementation The implementation strategy for this feature in Blender requires further discussion. As a starting point, it may be useful to describe a simple and direct approach for invoking such callbacks in user-specified modules: 1. Users implement Python add-ons that define the required chaser functions. 2. The chaser module names are provided as a parameter to the USD IO operators. 3. The USD IO code calls the chaser functions in the given modules using the `boost::python` bindings provided by the USD libraries. An advantage of this strategy is that it is straightforward to implement and requires no changes to Blender code outside the USD IO module. However, providing module names as a parameter requires an extra step in the workflow and is error prone. An improved design would be to allow registering new types of classes for implementing USD IO hooks, using the RNA framework. This has the advantages of allowing automatic registration of hooks and potentially better integration with Blender's UI. I've provided a simple example implementations of a new `bpy.types.USDHook` class type which defines the callbacks as member functions and which can be sub classed and registered in add-ons. A prototype implementation of this strategy is in pull request https://projects.blender.org/blender/blender/pulls/108823. I welcome feedback and further discussion of these ideas.

👍 1 ❤️ 2

Michael Kowalski added the

Type

Design

label 2023-06-09 22:39:36 +02:00

Michael Kowalski referenced this issue

2023-06-10 02:18:30 +02:00

USD export: prototype invoking Python chasers #108823

Michael Kowalski added the