Asset Types & Traits #105807

New Issue

Julian Eisel · 2023-03-15T19:22:09+01:00

Julian Eisel commented

2023-03-15 19:22:09 +01:00

This is more of a design document than a task. It should move to the Wiki soon.

Resolving a definite type for an asset is difficult. Instead, the following proposes a way to define type related characteristics (traits) of an asset, which can then be used to decide how an asset should behave.

Traits would be keywords that are stored in the asset metadata, and provide information about the type of an asset. For example {"ID", "Node Tree", "Geometry Nodes"}, or {"ID", "Brush", "Sculpt Mode"}. Unlike tags, they are managed by Blender or add-ons, not by the user.

What is the Type of an Asset?

The type of an asset is a vague concept. Some considerations:

Is it the ID type? If so, we wouldn't be able to tell apart a compositor node tree from a geometry node tree. Or a mesh object from a curve object. So just ID type often isn't granular enough. Nor does that cover assets that are not IDs (files, USD assets, presets, etc.).
How much should it matter to the user if an image is already a Blender data-block or just an image on the hard drive? Or if an object is already native Blender data or will go through an importer after dragging in. What matters most is the object itself, the format it happens to be stored in is irrelevant to a degree.
An image could be a reference image, a bump map, a stencil texture, ... The fact that an asset is an image may not matter much to the user, what the image represents and thus what it can be used for does.

So the proposal is to avoid the concept of a single static type. Instead, differentiate assets using type related characteristics, so called traits.

Asset Traits

Asset traits are keywords that specify certain type characteristics of an asset. They are loose (in the sense of having no knowledge about each other) but well defined.

With every trait added, the asset's type characteristics get defined more precisely. So traits are additive and increase specificity. Or put differently: Each trait removes ambiguity.

Traits look a lot like the already supported asset tags. And they are similar, but traits are more focused on type-relevant characteristics of the asset, have well-defined meaning, and are managed by software. Tags are managed by the user (more specifically, an asset creator) to help browsing assets, and can contain arbitrary bits of information with or without well-defined meaning.

Design Choices

Redundancy allowed: There may be some redundancy in the traits of an asset. For example Stencil pretty much implies that it's also an image and a texture. This makes it easy for code to check high-level characteristics, while keeping the flexibility to do more granular checks as needed.
Implicit hierarchy: There's also often a semantic hierarchy between the traits. For example the traits of a mesh-object could also be specified in a path-like manner: "ID/Object/Mesh". There doesn't seem to be a need for making the hierarchy explicit though, so it doesn't seem worth the overhead of defining a sensible path system. A loose set of strings seems simple and flexible. Plus, assets may have multiple independent traits, so just a single path-like trait wouldn't be sufficient anyway.
Style conventions: Traits are written using capitalized words and spaces as needed. This way they can be nicely exposed to the user if wanted, without additional "translation" from a coder-y style like ObjectData. For example, we may choose to display some traits as read-only tags to the user, since this information is useful for their browsing experience too.

Trait Glossary

Since traits are supposed to be well-defined, we should keep a glossary of the traits Blender uses natively. Here are some example definitions:

ID: The asset represents and needs to be imported as native ID data-block (using link/append).
Object: The asset represents and should be imported as Object. Note: This may also be set for assets that are not a native format and use an importer, e.g. an Object defined by a GLTF file. Native Blender objects have both the Object- and the ID-trait set.
Modifier: The asset may be imported as modifier. For example the traits Node Tree, Geometry Nodes, and Mesh Modifier would together indicate a geometry node-group that can not only be used as regular node group, but also be dropped into the modifier stack of a mesh object.

Custom Traits

Traits don't have to be defined by Blender natively. An add-on may introduce new traits (which it would have to manage itself). For example an add-on adding USD asset support, or support for using Inkscape brushes in Blender. The add-ons can provide own traits to indicate these assets.

Well-defined just means there is a reasonably clear definition, which is not necessarily provided by Blender itself.

Where do Traits Come From?

The stork brings them 👶 Seriously though, how exactly traits are defined unfortunately depends on the type of asset too. Whatever operation turns data into an asset (as per the asset system's understanding) has to set the initial traits. For example, marking a brush as asset could do the following:

Generic Mark as Asset code sets the ID and Brush traits.
From there a new IDTypeInfo::refine_asset_traits() could add further traits, like Sculpt Mode, to indicate a sculpt brush.

TODO

How do we keep traits updated? E.g. a texture may switch from being a stencil to a regular texture and vice versa. Updating traits on save may be a good start, but since traits drive operations like preview rendering, they also need to be updated before any such operation. Could be just a call to IDTypeInfo::refine_asset_traits()?

Forward Compatibility

While this is a forward compatible system in itself, there is the issue that we can't possibly predict all the possible traits that might be useful in future. For example, will we need image format details as traits in the future? Versioning code could of course add new traits as needed, but then that code may need the fully loaded asset data, not just the lightweight representation of an asset we load for the asset browser/system.

Note that this is a general problem with assets, even if we store type information differently. You just don't know what information you'll need from a the representation of an asset in future.

OpenAssetIO

OpenAssetIO also uses the concept of traits. It's similar to what is proposed here, although it goes a bit further to support communicating more general metadata this way too (which Blender handles differently). Point is, we can easily map Blender's asset traits to the one of OpenAssetIO. There may have to be a small translation layer though, since our trait terminology would be quite Blender-focused. An ID trait could be translated to a blender-ID OpenAssetIO trait. In general prefixing Blender specific traits with blender- seems like a good idea. Further OpenAssetIO traits can be added as needed then. E.g. the Blender Image trait may be expanded into a image and raster trait for OpenAssetIO (indicating an image stored in a pixel format).

This is more of a design document than a task. It should move to the Wiki soon. --- *Resolving a definite type for an asset is difficult. Instead, the following proposes a way to define type related characteristics (traits) of an asset, which can then be used to decide how an asset should behave.* *Traits would be keywords that are stored in the asset metadata, and provide information about the type of an asset. For example `{"ID", "Node Tree", "Geometry Nodes"}`, or `{"ID", "Brush", "Sculpt Mode"}`. Unlike tags, they are managed by Blender or add-ons, not by the user.* ## What is the Type of an Asset? The type of an asset is a vague concept. Some considerations: - Is it the ID type? If so, we wouldn't be able to tell apart a compositor node tree from a geometry node tree. Or a mesh object from a curve object. So just ID type often isn't granular enough. Nor does that cover assets that are not IDs (files, USD assets, presets, etc.). - How much should it matter to the user if an image is already a Blender data-block or just an image on the hard drive? Or if an object is already native Blender data or will go through an importer after dragging in. What matters most is the object itself, the format it happens to be stored in is irrelevant *to a degree*. - An image could be a reference image, a bump map, a stencil texture, ... The fact that an asset is an image may not matter much to the user, what the image represents and thus what it can be used for does. So the proposal is to avoid the concept of a single static type. Instead, differentiate assets using type related characteristics, so called *traits*. ## Asset Traits Asset traits are keywords that specify certain type characteristics of an asset. They are loose (in the sense of having no knowledge about each other) but well defined. ![](https://i.imgur.com/SOmDYnj.png) With every trait added, the asset's type characteristics get defined more precisely. So traits are additive and increase specificity. Or put differently: Each trait removes ambiguity. Traits look a lot like the already supported asset tags. And they are similar, but traits are more focused on type-relevant characteristics of the asset, have well-defined meaning, and are managed by software. Tags are managed by the user (more specifically, an asset creator) to help browsing assets, and can contain arbitrary bits of information with or without well-defined meaning. **Design Choices** - Redundancy allowed: There may be some redundancy in the traits of an asset. For example `Stencil` pretty much implies that it's also an image and a texture. This makes it easy for code to check high-level characteristics, while keeping the flexibility to do more granular checks as needed. - Implicit hierarchy: There's also often a semantic hierarchy between the traits. For example the traits of a mesh-object could also be specified in a path-like manner: `"ID/Object/Mesh"`. There doesn't seem to be a need for making the hierarchy explicit though, so it doesn't seem worth the overhead of defining a sensible path system. A loose set of strings seems simple and flexible. Plus, assets may have multiple independent traits, so just a single path-like trait wouldn't be sufficient anyway. - Style conventions: Traits are written using capitalized words and spaces as needed. This way they can be nicely exposed to the user if wanted, without additional "translation" from a coder-y style like `ObjectData`. For example, we may choose to display some traits as read-only tags to the user, since this information is useful for their browsing experience too. ----- ### Trait Glossary Since traits are supposed to be well-defined, we should keep a glossary of the traits Blender uses natively. Here are some example definitions: `ID` : The asset represents and needs to be imported as native ID data-block (using link/append). `Object` : The asset represents and should be imported as Object. Note: This may also be set for assets that are not a native format and use an importer, e.g. an Object defined by a GLTF file. Native Blender objects have both the `Object`- and the `ID`-trait set. `Modifier` : The asset may be imported as modifier. For example the traits `Node Tree`, `Geometry Nodes`, and `Mesh Modifier` would together indicate a geometry node-group that can not only be used as regular node group, but also be dropped into the modifier stack of a mesh object. ### Custom Traits Traits don't have to be defined by Blender natively. An add-on may introduce new traits (which it would have to manage itself). For example an add-on adding USD asset support, or support for using Inkscape brushes in Blender. The add-ons can provide own traits to indicate these assets. *Well-defined* just means there is a reasonably clear definition, which is not necessarily provided by Blender itself. ### Where do Traits Come From? The stork brings them :baby: Seriously though, how exactly traits are defined unfortunately depends on the type of asset too. Whatever operation turns data into an asset (as per the asset system's understanding) has to set the initial traits. For example, marking a brush as asset could do the following: - Generic *Mark as Asset* code sets the `ID` and `Brush` traits. - From there a new `IDTypeInfo::refine_asset_traits()` could add further traits, like `Sculpt Mode`, to indicate a sculpt brush. **TODO** How do we keep traits updated? E.g. a texture may switch from being a stencil to a regular texture and vice versa. Updating traits on save may be a good start, but since traits drive operations like preview rendering, they also need to be updated before any such operation. Could be just a call to `IDTypeInfo::refine_asset_traits()`? --- ### Forward Compatibility While this is a forward compatible system in itself, there is the issue that we can't possibly predict all the possible traits that might be useful in future. For example, will we need image format details as traits in the future? Versioning code could of course add new traits as needed, but then that code may need the fully loaded asset data, not just the lightweight representation of an asset we load for the asset browser/system. Note that this is a general problem with assets, even if we store type information differently. You just don't know what information you'll need from a the representation of an asset in future. ### OpenAssetIO OpenAssetIO also uses the concept of traits. It's similar to what is proposed here, although it goes a bit further to support communicating more general metadata this way too (which Blender handles differently). Point is, we can easily map Blender's asset traits to the one of OpenAssetIO. There may have to be a small translation layer though, since our trait terminology would be quite Blender-focused. An `ID` trait could be translated to a `blender-ID` OpenAssetIO trait. In general prefixing Blender specific traits with `blender-` seems like a good idea. Further OpenAssetIO traits can be added as needed then. E.g. the Blender `Image` trait may be expanded into a `image` and `raster` trait for OpenAssetIO (indicating an image stored in a pixel format).

🚀 1

Julian Eisel added the

Type

Design

label 2023-03-15 19:22:09 +01:00

Julian Eisel referenced this issue

2023-03-15 19:41:19 +01:00

Implement Asset Traits #105808

Harley Acheson referenced this issue

2023-03-16 20:10:41 +01:00

Assets: Implement Traits #105841

Brecht Van Lommel commented

2023-05-02 14:30:33 +02:00

To me it seems these traits are describing two things about the asset:

What the data type or interface is
More loose information about where and how to use the data

For (2) I can see how this design works, how UI and tools can adjust behavior based on traits like "is stencil" or "use node group as modifier".

However (1) is not clear to me. Such traits need to map to a well defined data type or interface, so that an operator or other code can actually do something with the data. And what I'm missing in this design is how that will work exactly.

The asset system currently understand a few datablock types. Will this be fully replaced by traits? If not, how will the redundancy be handled?

How would for example the brush UI deal with traits? Would it get a list of brushes by filtering purely by traits? And then when a user selects an asset from that list, what does the asset API for that look like? Is validating that the data type is in fact a brush datablock (and how does this work with arbitrary non-ID data)? Or does the asset API understand a number of built-in traits and associated data types, and the operator can just use an API call that is guaranteed to return a brush datablock? How does the operator handle cases where the traits do not match what was expected?

What's also unclear to me is how this works with e.g. and add-on that makes a glTF file available as a collection datablock. Because now you need multiple to work together. You have the trait on the asset, an add-on that can take that glTF file and turn it into a collection datablock, and a tool in Blender that will take that collection datablock and e.g. place it in the scene. But when and how is this trait added to the asset then, and how can the add-on register itself to handle it? And I guess this means you can't distribute this asset without the associated add-on?

To me it seems these traits are describing two things about the asset: 1. What the data type or interface is 2. More loose information about where and how to use the data For (2) I can see how this design works, how UI and tools can adjust behavior based on traits like "is stencil" or "use node group as modifier". However (1) is not clear to me. Such traits need to map to a well defined data type or interface, so that an operator or other code can actually do something with the data. And what I'm missing in this design is how that will work exactly. The asset system currently understand a few datablock types. Will this be fully replaced by traits? If not, how will the redundancy be handled? How would for example the brush UI deal with traits? Would it get a list of brushes by filtering purely by traits? And then when a user selects an asset from that list, what does the asset API for that look like? Is validating that the data type is in fact a brush datablock (and how does this work with arbitrary non-ID data)? Or does the asset API understand a number of built-in traits and associated data types, and the operator can just use an API call that is guaranteed to return a brush datablock? How does the operator handle cases where the traits do not match what was expected? What's also unclear to me is how this works with e.g. and add-on that makes a glTF file available as a collection datablock. Because now you need multiple to work together. You have the trait on the asset, an add-on that can take that glTF file and turn it into a collection datablock, and a tool in Blender that will take that collection datablock and e.g. place it in the scene. But when and how is this trait added to the asset then, and how can the add-on register itself to handle it? And I guess this means you can't distribute this asset without the associated add-on?

Brecht Van Lommel referenced this issue

2023-05-02 15:58:28 +02:00

Assets: Implement Traits #105841

Julian Eisel commented

2023-07-13 16:57:51 +02:00

Sorry for the late reply. I think I misunderstood the comment a bit at first, and the patch wasn't too high priority for the moment, so I let it stall a bit.

To me it seems these traits are describing two things about the asset:

What the data type or interface is

More loose information about where and how to use the data

Basically for now I'd just look at traits as something that we use for the UI mostly, so only (2). I'm not sure what makes you think that (1) would be covered by traits, but I guess it's a reasonable assumption.

Traits could provide useful info for the importing, but I don't think it should rely on them. You would still need some add-on (or builtin functionality) that manages an asset library and knows how to import the asset. I guess there needs to be a standardized interface for the importing indeed (so the assets can be imported through the asset system) but in my mind traits don't play a big role in that.

How would for example the brush UI deal with traits? Would it get a list of brushes by filtering purely by traits?

This is what the asset shelf would do for example, yes. The asset browser shows all assets by default.

And then when a user selects an asset from that list, what does the asset API for that look like? Is validating that the data type is in fact a brush datablock (and how does this work with arbitrary non-ID data)? Or does the asset API understand a number of built-in traits and associated data types, and the operator can just use an API call that is guaranteed to return a brush datablock?

Basically yes, this is what I have in mind for now. The operator can request the asset to be imported which for now will result in a datablock. With non-ID assets we need a more flexible import process, and I still have to figure out how that looks like. It would be nice if there could be just some import() callback on the asset type or the asset library, but importing may mean different things based on context (like mode and editor) and user desires (like link vs append, or collection instancing). So we may still have to provide more flexibility like custom drag & drop for add-ons.

How does the operator handle cases where the traits do not match what was expected?

I'd say the UI should only display assets compatible with the operator for the most part. On failure the operator can just error out as usual.

Note that the add and search menus in the geometry nodes editors already work roughly like this. The UI provides assets filtered by type characteristics, and passes them to an operator that does the import. The asset shelf will work like that too, they can register an operator to use for applying/dragging.

Sorry for the late reply. I think I misunderstood the comment a bit at first, and the patch wasn't too high priority for the moment, so I let it stall a bit. > To me it seems these traits are describing two things about the asset: > 1. What the data type or interface is > 2. More loose information about where and how to use the data Basically for now I'd just look at traits as something that we use for the UI mostly, so only (2). I'm not sure what makes you think that (1) would be covered by traits, but I guess it's a reasonable assumption. Traits could provide useful info for the importing, but I don't think it should rely on them. You would still need some add-on (or builtin functionality) that manages an asset library and knows how to import the asset. I guess there needs to be a standardized interface for the importing indeed (so the assets can be imported through the asset system) but in my mind traits don't play a big role in that. > How would for example the brush UI deal with traits? Would it get a list of brushes by filtering purely by traits? This is what the asset shelf would do for example, yes. The asset browser shows all assets by default. > And then when a user selects an asset from that list, what does the asset API for that look like? Is validating that the data type is in fact a brush datablock (and how does this work with arbitrary non-ID data)? Or does the asset API understand a number of built-in traits and associated data types, and the operator can just use an API call that is guaranteed to return a brush datablock? Basically yes, this is what I have in mind for now. The operator can request the asset to be imported which for now will result in a datablock. With non-ID assets we need a more flexible import process, and I still have to figure out how that looks like. It would be nice if there could be just some `import()` callback on the asset type or the asset library, but importing may mean different things based on context (like mode and editor) and user desires (like link vs append, or collection instancing). So we may still have to provide more flexibility like custom drag & drop for add-ons. > How does the operator handle cases where the traits do not match what was expected? I'd say the UI should only display assets compatible with the operator for the most part. On failure the operator can just error out as usual. ---- Note that the add and search menus in the geometry nodes editors already work roughly like this. The UI provides assets filtered by type characteristics, and passes them to an operator that does the import. The asset shelf will work like that too, they can register an operator to use for applying/dragging.

Brecht Van Lommel commented

2023-07-13 17:47:33 +02:00

What the data type or interface is

More loose information about where and how to use the data

Basically for now I'd just look at traits as something that we use for the UI mostly, so only (2). I'm not sure what makes you think that (1) would be covered by traits, but I guess it's a reasonable assumption.

For me both (1) and (2) are important information for the UI. A brush asset shelf should only show things that it is certain can be actually used as a brush. There's no reason for that to be fuzzy and potentially unreliable for the user.

There's something conceptually that bothers me about this approach. Maybe it's because it reminds me of notifiers. That's a also a loose coupling between data and UI that started with good intentions and then deteriorated into something where it's very hard to tell what's what exactly, and the abstraction gets in the way more than it helps.

If as I suggested in #105841 a trait maps directly to a datablock property at least that is unambiguous.

> > 1. What the data type or interface is > > 2. More loose information about where and how to use the data > > Basically for now I'd just look at traits as something that we use for the UI mostly, so only (2). I'm not sure what makes you think that (1) would be covered by traits, but I guess it's a reasonable assumption. For me both (1) and (2) are important information for the UI. A brush asset shelf should only show things that it is certain can be actually used as a brush. There's no reason for that to be fuzzy and potentially unreliable for the user. There's something conceptually that bothers me about this approach. Maybe it's because it reminds me of notifiers. That's a also a loose coupling between data and UI that started with good intentions and then deteriorated into something where it's very hard to tell what's what exactly, and the abstraction gets in the way more than it helps. If as I suggested in #105841 a trait maps directly to a datablock property at least that is unambiguous.

Brecht Van Lommel commented

2023-07-13 19:55:59 +02:00

To be more clear what I was thinking of, imagine you have traits like this:

type:Brush
type:Mesh
property:Object.type=MESH
property:Brush.mode=SCULPT
property:NodeTree.asset.is_modifier

To be more clear what I was thinking of, imagine you have traits like this: ``` type:Brush type:Mesh property:Object.type=MESH property:Brush.mode=SCULPT property:NodeTree.asset.is_modifier ```

Sign in to join this conversation.

No Label

Download

What's New

Blender Studio

Manual

Developers Blog

Documentation

Benchmark

Blender Conference

Development Fund

One-time Donations

Asset Types & Traits #105807

What is the Type of an Asset?

Asset Traits

Trait Glossary

Custom Traits

Where do Traits Come From?

Forward Compatibility

OpenAssetIO