Asset Types & Traits #105807

Open
opened 2023-03-15 19:22:09 +01:00 by Julian Eisel · 4 comments
Member

This is more of a design document than a task. It should move to the Wiki soon.


Resolving a definite type for an asset is difficult. Instead, the following proposes a way to define type related characteristics (traits) of an asset, which can then be used to decide how an asset should behave.

Traits would be keywords that are stored in the asset metadata, and provide information about the type of an asset. For example {"ID", "Node Tree", "Geometry Nodes"}, or {"ID", "Brush", "Sculpt Mode"}. Unlike tags, they are managed by Blender or add-ons, not by the user.

What is the Type of an Asset?

The type of an asset is a vague concept. Some considerations:

  • Is it the ID type? If so, we wouldn't be able to tell apart a compositor node tree from a geometry node tree. Or a mesh object from a curve object. So just ID type often isn't granular enough. Nor does that cover assets that are not IDs (files, USD assets, presets, etc.).
  • How much should it matter to the user if an image is already a Blender data-block or just an image on the hard drive? Or if an object is already native Blender data or will go through an importer after dragging in. What matters most is the object itself, the format it happens to be stored in is irrelevant to a degree.
  • An image could be a reference image, a bump map, a stencil texture, ... The fact that an asset is an image may not matter much to the user, what the image represents and thus what it can be used for does.

So the proposal is to avoid the concept of a single static type. Instead, differentiate assets using type related characteristics, so called traits.

Asset Traits

Asset traits are keywords that specify certain type characteristics of an asset. They are loose (in the sense of having no knowledge about each other) but well defined.

With every trait added, the asset's type characteristics get defined more precisely. So traits are additive and increase specificity. Or put differently: Each trait removes ambiguity.

Traits look a lot like the already supported asset tags. And they are similar, but traits are more focused on type-relevant characteristics of the asset, have well-defined meaning, and are managed by software. Tags are managed by the user (more specifically, an asset creator) to help browsing assets, and can contain arbitrary bits of information with or without well-defined meaning.

Design Choices

  • Redundancy allowed: There may be some redundancy in the traits of an asset. For example Stencil pretty much implies that it's also an image and a texture. This makes it easy for code to check high-level characteristics, while keeping the flexibility to do more granular checks as needed.

  • Implicit hierarchy: There's also often a semantic hierarchy between the traits. For example the traits of a mesh-object could also be specified in a path-like manner: "ID/Object/Mesh". There doesn't seem to be a need for making the hierarchy explicit though, so it doesn't seem worth the overhead of defining a sensible path system. A loose set of strings seems simple and flexible. Plus, assets may have multiple independent traits, so just a single path-like trait wouldn't be sufficient anyway.

  • Style conventions: Traits are written using capitalized words and spaces as needed. This way they can be nicely exposed to the user if wanted, without additional "translation" from a coder-y style like ObjectData. For example, we may choose to display some traits as read-only tags to the user, since this information is useful for their browsing experience too.


Trait Glossary

Since traits are supposed to be well-defined, we should keep a glossary of the traits Blender uses natively. Here are some example definitions:

ID
The asset represents and needs to be imported as native ID data-block (using link/append).
Object
The asset represents and should be imported as Object. Note: This may also be set for assets that are not a native format and use an importer, e.g. an Object defined by a GLTF file. Native Blender objects have both the Object- and the ID-trait set.
Modifier
The asset may be imported as modifier. For example the traits Node Tree, Geometry Nodes, and Mesh Modifier would together indicate a geometry node-group that can not only be used as regular node group, but also be dropped into the modifier stack of a mesh object.

Custom Traits

Traits don't have to be defined by Blender natively. An add-on may introduce new traits (which it would have to manage itself). For example an add-on adding USD asset support, or support for using Inkscape brushes in Blender. The add-ons can provide own traits to indicate these assets.

Well-defined just means there is a reasonably clear definition, which is not necessarily provided by Blender itself.

Where do Traits Come From?

The stork brings them 👶 Seriously though, how exactly traits are defined unfortunately depends on the type of asset too. Whatever operation turns data into an asset (as per the asset system's understanding) has to set the initial traits. For example, marking a brush as asset could do the following:

  • Generic Mark as Asset code sets the ID and Brush traits.
  • From there a new IDTypeInfo::refine_asset_traits() could add further traits, like Sculpt Mode, to indicate a sculpt brush.

TODO

How do we keep traits updated? E.g. a texture may switch from being a stencil to a regular texture and vice versa. Updating traits on save may be a good start, but since traits drive operations like preview rendering, they also need to be updated before any such operation. Could be just a call to IDTypeInfo::refine_asset_traits()?


Forward Compatibility

While this is a forward compatible system in itself, there is the issue that we can't possibly predict all the possible traits that might be useful in future. For example, will we need image format details as traits in the future? Versioning code could of course add new traits as needed, but then that code may need the fully loaded asset data, not just the lightweight representation of an asset we load for the asset browser/system.

Note that this is a general problem with assets, even if we store type information differently. You just don't know what information you'll need from a the representation of an asset in future.

OpenAssetIO

OpenAssetIO also uses the concept of traits. It's similar to what is proposed here, although it goes a bit further to support communicating more general metadata this way too (which Blender handles differently). Point is, we can easily map Blender's asset traits to the one of OpenAssetIO. There may have to be a small translation layer though, since our trait terminology would be quite Blender-focused. An ID trait could be translated to a blender-ID OpenAssetIO trait. In general prefixing Blender specific traits with blender- seems like a good idea. Further OpenAssetIO traits can be added as needed then. E.g. the Blender Image trait may be expanded into a image and raster trait for OpenAssetIO (indicating an image stored in a pixel format).

This is more of a design document than a task. It should move to the Wiki soon. --- *Resolving a definite type for an asset is difficult. Instead, the following proposes a way to define type related characteristics (traits) of an asset, which can then be used to decide how an asset should behave.* *Traits would be keywords that are stored in the asset metadata, and provide information about the type of an asset. For example `{"ID", "Node Tree", "Geometry Nodes"}`, or `{"ID", "Brush", "Sculpt Mode"}`. Unlike tags, they are managed by Blender or add-ons, not by the user.* ## What is the Type of an Asset? The type of an asset is a vague concept. Some considerations: - Is it the ID type? If so, we wouldn't be able to tell apart a compositor node tree from a geometry node tree. Or a mesh object from a curve object. So just ID type often isn't granular enough. Nor does that cover assets that are not IDs (files, USD assets, presets, etc.). - How much should it matter to the user if an image is already a Blender data-block or just an image on the hard drive? Or if an object is already native Blender data or will go through an importer after dragging in. What matters most is the object itself, the format it happens to be stored in is irrelevant *to a degree*. - An image could be a reference image, a bump map, a stencil texture, ... The fact that an asset is an image may not matter much to the user, what the image represents and thus what it can be used for does. So the proposal is to avoid the concept of a single static type. Instead, differentiate assets using type related characteristics, so called *traits*. ## Asset Traits Asset traits are keywords that specify certain type characteristics of an asset. They are loose (in the sense of having no knowledge about each other) but well defined. ![](https://i.imgur.com/SOmDYnj.png) With every trait added, the asset's type characteristics get defined more precisely. So traits are additive and increase specificity. Or put differently: Each trait removes ambiguity. Traits look a lot like the already supported asset tags. And they are similar, but traits are more focused on type-relevant characteristics of the asset, have well-defined meaning, and are managed by software. Tags are managed by the user (more specifically, an asset creator) to help browsing assets, and can contain arbitrary bits of information with or without well-defined meaning. **Design Choices** - Redundancy allowed: There may be some redundancy in the traits of an asset. For example `Stencil` pretty much implies that it's also an image and a texture. This makes it easy for code to check high-level characteristics, while keeping the flexibility to do more granular checks as needed. - Implicit hierarchy: There's also often a semantic hierarchy between the traits. For example the traits of a mesh-object could also be specified in a path-like manner: `"ID/Object/Mesh"`. There doesn't seem to be a need for making the hierarchy explicit though, so it doesn't seem worth the overhead of defining a sensible path system. A loose set of strings seems simple and flexible. Plus, assets may have multiple independent traits, so just a single path-like trait wouldn't be sufficient anyway. - Style conventions: Traits are written using capitalized words and spaces as needed. This way they can be nicely exposed to the user if wanted, without additional "translation" from a coder-y style like `ObjectData`. For example, we may choose to display some traits as read-only tags to the user, since this information is useful for their browsing experience too. ----- ### Trait Glossary Since traits are supposed to be well-defined, we should keep a glossary of the traits Blender uses natively. Here are some example definitions: `ID` : The asset represents and needs to be imported as native ID data-block (using link/append). `Object` : The asset represents and should be imported as Object. Note: This may also be set for assets that are not a native format and use an importer, e.g. an Object defined by a GLTF file. Native Blender objects have both the `Object`- and the `ID`-trait set. `Modifier` : The asset may be imported as modifier. For example the traits `Node Tree`, `Geometry Nodes`, and `Mesh Modifier` would together indicate a geometry node-group that can not only be used as regular node group, but also be dropped into the modifier stack of a mesh object. ### Custom Traits Traits don't have to be defined by Blender natively. An add-on may introduce new traits (which it would have to manage itself). For example an add-on adding USD asset support, or support for using Inkscape brushes in Blender. The add-ons can provide own traits to indicate these assets. *Well-defined* just means there is a reasonably clear definition, which is not necessarily provided by Blender itself. ### Where do Traits Come From? The stork brings them :baby: Seriously though, how exactly traits are defined unfortunately depends on the type of asset too. Whatever operation turns data into an asset (as per the asset system's understanding) has to set the initial traits. For example, marking a brush as asset could do the following: - Generic *Mark as Asset* code sets the `ID` and `Brush` traits. - From there a new `IDTypeInfo::refine_asset_traits()` could add further traits, like `Sculpt Mode`, to indicate a sculpt brush. **TODO** How do we keep traits updated? E.g. a texture may switch from being a stencil to a regular texture and vice versa. Updating traits on save may be a good start, but since traits drive operations like preview rendering, they also need to be updated before any such operation. Could be just a call to `IDTypeInfo::refine_asset_traits()`? --- ### Forward Compatibility While this is a forward compatible system in itself, there is the issue that we can't possibly predict all the possible traits that might be useful in future. For example, will we need image format details as traits in the future? Versioning code could of course add new traits as needed, but then that code may need the fully loaded asset data, not just the lightweight representation of an asset we load for the asset browser/system. Note that this is a general problem with assets, even if we store type information differently. You just don't know what information you'll need from a the representation of an asset in future. ### OpenAssetIO OpenAssetIO also uses the concept of traits. It's similar to what is proposed here, although it goes a bit further to support communicating more general metadata this way too (which Blender handles differently). Point is, we can easily map Blender's asset traits to the one of OpenAssetIO. There may have to be a small translation layer though, since our trait terminology would be quite Blender-focused. An `ID` trait could be translated to a `blender-ID` OpenAssetIO trait. In general prefixing Blender specific traits with `blender-` seems like a good idea. Further OpenAssetIO traits can be added as needed then. E.g. the Blender `Image` trait may be expanded into a `image` and `raster` trait for OpenAssetIO (indicating an image stored in a pixel format).
Julian Eisel added the
Type
Design
label 2023-03-15 19:22:09 +01:00

To me it seems these traits are describing two things about the asset:

  1. What the data type or interface is
  2. More loose information about where and how to use the data

For (2) I can see how this design works, how UI and tools can adjust behavior based on traits like "is stencil" or "use node group as modifier".

However (1) is not clear to me. Such traits need to map to a well defined data type or interface, so that an operator or other code can actually do something with the data. And what I'm missing in this design is how that will work exactly.

The asset system currently understand a few datablock types. Will this be fully replaced by traits? If not, how will the redundancy be handled?

How would for example the brush UI deal with traits? Would it get a list of brushes by filtering purely by traits? And then when a user selects an asset from that list, what does the asset API for that look like? Is validating that the data type is in fact a brush datablock (and how does this work with arbitrary non-ID data)? Or does the asset API understand a number of built-in traits and associated data types, and the operator can just use an API call that is guaranteed to return a brush datablock? How does the operator handle cases where the traits do not match what was expected?

What's also unclear to me is how this works with e.g. and add-on that makes a glTF file available as a collection datablock. Because now you need multiple to work together. You have the trait on the asset, an add-on that can take that glTF file and turn it into a collection datablock, and a tool in Blender that will take that collection datablock and e.g. place it in the scene. But when and how is this trait added to the asset then, and how can the add-on register itself to handle it? And I guess this means you can't distribute this asset without the associated add-on?

To me it seems these traits are describing two things about the asset: 1. What the data type or interface is 2. More loose information about where and how to use the data For (2) I can see how this design works, how UI and tools can adjust behavior based on traits like "is stencil" or "use node group as modifier". However (1) is not clear to me. Such traits need to map to a well defined data type or interface, so that an operator or other code can actually do something with the data. And what I'm missing in this design is how that will work exactly. The asset system currently understand a few datablock types. Will this be fully replaced by traits? If not, how will the redundancy be handled? How would for example the brush UI deal with traits? Would it get a list of brushes by filtering purely by traits? And then when a user selects an asset from that list, what does the asset API for that look like? Is validating that the data type is in fact a brush datablock (and how does this work with arbitrary non-ID data)? Or does the asset API understand a number of built-in traits and associated data types, and the operator can just use an API call that is guaranteed to return a brush datablock? How does the operator handle cases where the traits do not match what was expected? What's also unclear to me is how this works with e.g. and add-on that makes a glTF file available as a collection datablock. Because now you need multiple to work together. You have the trait on the asset, an add-on that can take that glTF file and turn it into a collection datablock, and a tool in Blender that will take that collection datablock and e.g. place it in the scene. But when and how is this trait added to the asset then, and how can the add-on register itself to handle it? And I guess this means you can't distribute this asset without the associated add-on?
Author
Member

Sorry for the late reply. I think I misunderstood the comment a bit at first, and the patch wasn't too high priority for the moment, so I let it stall a bit.

To me it seems these traits are describing two things about the asset:

  1. What the data type or interface is
  2. More loose information about where and how to use the data

Basically for now I'd just look at traits as something that we use for the UI mostly, so only (2). I'm not sure what makes you think that (1) would be covered by traits, but I guess it's a reasonable assumption.

Traits could provide useful info for the importing, but I don't think it should rely on them. You would still need some add-on (or builtin functionality) that manages an asset library and knows how to import the asset. I guess there needs to be a standardized interface for the importing indeed (so the assets can be imported through the asset system) but in my mind traits don't play a big role in that.

How would for example the brush UI deal with traits? Would it get a list of brushes by filtering purely by traits?

This is what the asset shelf would do for example, yes. The asset browser shows all assets by default.

And then when a user selects an asset from that list, what does the asset API for that look like? Is validating that the data type is in fact a brush datablock (and how does this work with arbitrary non-ID data)? Or does the asset API understand a number of built-in traits and associated data types, and the operator can just use an API call that is guaranteed to return a brush datablock?

Basically yes, this is what I have in mind for now. The operator can request the asset to be imported which for now will result in a datablock. With non-ID assets we need a more flexible import process, and I still have to figure out how that looks like. It would be nice if there could be just some import() callback on the asset type or the asset library, but importing may mean different things based on context (like mode and editor) and user desires (like link vs append, or collection instancing). So we may still have to provide more flexibility like custom drag & drop for add-ons.

How does the operator handle cases where the traits do not match what was expected?

I'd say the UI should only display assets compatible with the operator for the most part. On failure the operator can just error out as usual.


Note that the add and search menus in the geometry nodes editors already work roughly like this. The UI provides assets filtered by type characteristics, and passes them to an operator that does the import. The asset shelf will work like that too, they can register an operator to use for applying/dragging.

Sorry for the late reply. I think I misunderstood the comment a bit at first, and the patch wasn't too high priority for the moment, so I let it stall a bit. > To me it seems these traits are describing two things about the asset: > 1. What the data type or interface is > 2. More loose information about where and how to use the data Basically for now I'd just look at traits as something that we use for the UI mostly, so only (2). I'm not sure what makes you think that (1) would be covered by traits, but I guess it's a reasonable assumption. Traits could provide useful info for the importing, but I don't think it should rely on them. You would still need some add-on (or builtin functionality) that manages an asset library and knows how to import the asset. I guess there needs to be a standardized interface for the importing indeed (so the assets can be imported through the asset system) but in my mind traits don't play a big role in that. > How would for example the brush UI deal with traits? Would it get a list of brushes by filtering purely by traits? This is what the asset shelf would do for example, yes. The asset browser shows all assets by default. > And then when a user selects an asset from that list, what does the asset API for that look like? Is validating that the data type is in fact a brush datablock (and how does this work with arbitrary non-ID data)? Or does the asset API understand a number of built-in traits and associated data types, and the operator can just use an API call that is guaranteed to return a brush datablock? Basically yes, this is what I have in mind for now. The operator can request the asset to be imported which for now will result in a datablock. With non-ID assets we need a more flexible import process, and I still have to figure out how that looks like. It would be nice if there could be just some `import()` callback on the asset type or the asset library, but importing may mean different things based on context (like mode and editor) and user desires (like link vs append, or collection instancing). So we may still have to provide more flexibility like custom drag & drop for add-ons. > How does the operator handle cases where the traits do not match what was expected? I'd say the UI should only display assets compatible with the operator for the most part. On failure the operator can just error out as usual. ---- Note that the add and search menus in the geometry nodes editors already work roughly like this. The UI provides assets filtered by type characteristics, and passes them to an operator that does the import. The asset shelf will work like that too, they can register an operator to use for applying/dragging.
  1. What the data type or interface is
  2. More loose information about where and how to use the data

Basically for now I'd just look at traits as something that we use for the UI mostly, so only (2). I'm not sure what makes you think that (1) would be covered by traits, but I guess it's a reasonable assumption.

For me both (1) and (2) are important information for the UI. A brush asset shelf should only show things that it is certain can be actually used as a brush. There's no reason for that to be fuzzy and potentially unreliable for the user.

There's something conceptually that bothers me about this approach. Maybe it's because it reminds me of notifiers. That's a also a loose coupling between data and UI that started with good intentions and then deteriorated into something where it's very hard to tell what's what exactly, and the abstraction gets in the way more than it helps.

If as I suggested in #105841 a trait maps directly to a datablock property at least that is unambiguous.

> > 1. What the data type or interface is > > 2. More loose information about where and how to use the data > > Basically for now I'd just look at traits as something that we use for the UI mostly, so only (2). I'm not sure what makes you think that (1) would be covered by traits, but I guess it's a reasonable assumption. For me both (1) and (2) are important information for the UI. A brush asset shelf should only show things that it is certain can be actually used as a brush. There's no reason for that to be fuzzy and potentially unreliable for the user. There's something conceptually that bothers me about this approach. Maybe it's because it reminds me of notifiers. That's a also a loose coupling between data and UI that started with good intentions and then deteriorated into something where it's very hard to tell what's what exactly, and the abstraction gets in the way more than it helps. If as I suggested in #105841 a trait maps directly to a datablock property at least that is unambiguous.

To be more clear what I was thinking of, imagine you have traits like this:

type:Brush
type:Mesh
property:Object.type=MESH
property:Brush.mode=SCULPT
property:NodeTree.asset.is_modifier
To be more clear what I was thinking of, imagine you have traits like this: ``` type:Brush type:Mesh property:Object.type=MESH property:Brush.mode=SCULPT property:NodeTree.asset.is_modifier ```
Sign in to join this conversation.
No Label
Interest
Alembic
Interest
Animation & Rigging
Interest
Asset Browser
Interest
Asset Browser Project Overview
Interest
Audio
Interest
Automated Testing
Interest
Blender Asset Bundle
Interest
BlendFile
Interest
Collada
Interest
Compatibility
Interest
Compositing
Interest
Core
Interest
Cycles
Interest
Dependency Graph
Interest
Development Management
Interest
EEVEE
Interest
EEVEE & Viewport
Interest
Freestyle
Interest
Geometry Nodes
Interest
Grease Pencil
Interest
ID Management
Interest
Images & Movies
Interest
Import Export
Interest
Line Art
Interest
Masking
Interest
Metal
Interest
Modeling
Interest
Modifiers
Interest
Motion Tracking
Interest
Nodes & Physics
Interest
OpenGL
Interest
Overlay
Interest
Overrides
Interest
Performance
Interest
Physics
Interest
Pipeline, Assets & IO
Interest
Platforms, Builds & Tests
Interest
Python API
Interest
Render & Cycles
Interest
Render Pipeline
Interest
Sculpt, Paint & Texture
Interest
Text Editor
Interest
Translations
Interest
Triaging
Interest
Undo
Interest
USD
Interest
User Interface
Interest
UV Editing
Interest
VFX & Video
Interest
Video Sequencer
Interest
Virtual Reality
Interest
Vulkan
Interest
Wayland
Interest
Workbench
Legacy
Blender 2.8 Project
Legacy
Milestone 1: Basic, Local Asset Browser
Legacy
OpenGL Error
Meta
Good First Issue
Meta
Papercut
Meta
Retrospective
Meta
Security
Module
Animation & Rigging
Module
Core
Module
Development Management
Module
EEVEE & Viewport
Module
Grease Pencil
Module
Modeling
Module
Nodes & Physics
Module
Pipeline, Assets & IO
Module
Platforms, Builds & Tests
Module
Python API
Module
Render & Cycles
Module
Sculpt, Paint & Texture
Module
Triaging
Module
User Interface
Module
VFX & Video
Platform
FreeBSD
Platform
Linux
Platform
macOS
Platform
Windows
Priority
High
Priority
Low
Priority
Normal
Priority
Unbreak Now!
Status
Archived
Status
Confirmed
Status
Duplicate
Status
Needs Info from Developers
Status
Needs Information from User
Status
Needs Triage
Status
Resolved
Type
Bug
Type
Design
Type
Known Issue
Type
Patch
Type
Report
Type
To Do
No Milestone
No project
No Assignees
2 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: blender/blender#105807
No description provided.