blender/blender
148
473
Fork 696
Code Issues 6.7k Pull Requests 890 Projects 18 Wiki Activity

Brush Asset - Technical Core-level Design #101908

New Issue
Closed
opened 2022-10-18 19:34:09 +02:00 by Bastien Montagne · 8 comments
Bastien Montagne commented 2022-10-18 19:34:09 +02:00
Owner
Copy Link

This task is to design and keep track of how handling brushes will be done in the new Brush Asset system, from the Core module perspective mainly.

Changes in Brush Handling

Brushes are now (in most cases) purely run-time data.

    • They do exist in the Main data-base.
    • They have a new ID tag to mark them as not writable (LIB_TAG_RUNTIME? Also related to #88555 (ID Management: Sanitize and clarify our ID tags) to some extent). b93025db59
    • RUNTIME IDs do not make linked data 'directly linked'.
    • RUNTIME IDs are written in global undo steps.
    • Brushes are not covered by Undo, they always remain in their state when applying undo/redo steps.
    • Brushes are not cleared when loading a new .blend file within a same Blender session. However:
    • They do get cleared when reloading startup or factory startup files.
    • Brushes in newly loaded .blend file are also kept, and replace potential existing ones in old blendfile in case of conflict. This is to allow opening an asset library .blend, otherwise local brushes there would always be removed on opening.
    • This implies a conditional mechanism to re-use brushes from old Main and copy them into new Main on file load (similar to other UI-related IDs like WindowManager e.g.).
    • A weak reference to the active brush must be stored in the toolsettings when writing a .blend file, such that it can be reloaded automatically on file opening when there is no active brushes in the current editing session.
    • Several brushes with the same name (coming from different asset repositories/libraries) must be supported.
    • Displaying them in the asset browser/asset shelf should not be an issue? (Check with @JulianEisel).
    • Local runtime overrides however need a unique name. One idea could be to hide their .xxx number suffix in the UI?
    • It is assumed that only Brush data itself needs to be editable, its dependencies (textures, etc.) can remain linked and readonly.

Typically, real local brush data should only exist in files part of an asset repository.

Brushes are reset when closing and re-opening Blender, however they are kept across file opening within and editing session. So they are handled in a similar way to UI data (WindowManager etc. IDs) when 'load UI' is disabled, and moved from old to new Main on file loading.

Selecting a Brush

When selecting the asset:

    • Brush is linked from asset .blend file (path given by asset system).
    • Existing local runtime liboverride of linked brush is searched for. If none is found, a new one is created.
    • That local runtime liboverride is assigned as active brush.

Undo & File Loading

No undo support should be needed for brush usage nor editing. Brushes can therefore be fully ignored by undo system, and existing brushes should just be re-used as-is.
Brushes are also kept across file loading within a same Blender session, unless the (factory) startup file is loaded, or when loading to/from a brush asset library file.

Technically this has several complex implications, especially in setup_app_data part of the file loading process, mainly due to the fact that brushes can use other data-blocks. Currently:

  • Dependencies are not fully guaranteed to be handled by the new behavior:
    • In undo case, other IDs are handled like any other data-block, i.e. changes to them are undone or redone (so editing an image or texture of a brush will be undoable). This is not ideal, but extremely hard to avoid in current situation.
    • In file loading case, other dependencies can be brought over from the old bmain into the new one. This implies that e.g. is a locale texture is assigned to a runtime only liboverride of a brush while editing file_a.blend, and then user open file_b.blend, a copy of that locale texture will become part of file_b.blend.
  • Opening another .blend file, in general:
    • All fully local, non-asset brushes are preserved (necessary for old brushes compatibility).
    • All fully local, asset brushes are discarded.
    • All local overrides of asset brushes are preserved (besides special cases noted below).
    • All linked brushes are preserved (besides special cases noted below).
  • Opening a file that is a library of current file (typical case: opening a Brush Asset library file):
    • Reset related brush assets to their status in the library (linked brushes and their local overrides are not preserved).
    • Have clear UI visibility of local assets having 'pending changes' in drafts.
    • Warn user that by opening the library file, they will lose their current non-drafted changes in these assets.
      • Requires detection of these cases.
  • Opening a .blend file that is using as library the current blend file.
    • Local asset brushes must be discarded in that case, this is covered by the general case described above.

do_version

For modes moved to the new BrushAssets system, existing local brushes in older .blend files are often redundant and should be cleaned up.

  • Local non-asset brushes which names and values match the current default 'embedded' brushes should be removed.
    • 'Matching' here is used as 'essentially equal', in the sense that some properties (like e.g. size) should not be considered in the comparison.
    • This implies keeping (or making) old brushes available for do_version code.
  • The others (non-matching) will be kept as local, non-asset brushes. Up to the user to clean them up or transform them into assets.

Local brushes that are not defined as assets will not be shown to the user in the UI (they will only be reachable through the Outliner editor, python API, etc.).

To Be Investigated

Toolslot

The 'toolslots' system is used at least by the 'smooth mode' toggle option of stroke code, which does a weird 'hot-swap' between current brush and a smooth brush, searched first in the relevant entry of the toolslot. This needs to be investigated further.

Note


One idea to fix the 'smooth mode' issue is that this toggle would not anymore search for an actual existing smooth brush, but would rather generate it's own private temp smooth brush, with default settings.

This task is to design and keep track of how handling brushes will be done in the new Brush Asset system, from the Core module perspective mainly. ## Changes in Brush Handling Brushes are now (in most cases) purely run-time data. * - [x] They do exist in the Main data-base. * - [x] They have a new ID tag to mark them as not writable (`LIB_TAG_RUNTIME`? Also related to #88555 (ID Management: Sanitize and clarify our ID tags) to some extent). b93025db59 * `RUNTIME` IDs **do not** make linked data 'directly linked'. * `RUNTIME` IDs **are** written in global undo steps. * - [x] Brushes are **not** covered by Undo, they always remain in their state when applying undo/redo steps. * - [x] Brushes are **not** cleared when loading a new .blend file within a same Blender session. However: * They do get cleared when reloading startup or factory startup files. * Brushes in newly loaded .blend file are also kept, and replace potential existing ones in old blendfile in case of conflict. *This is to allow opening an asset library .blend, otherwise local brushes there would always be removed on opening.* * This implies a conditional mechanism to re-use brushes from old Main and copy them into new Main on file load (similar to other UI-related IDs like WindowManager e.g.). * - [x] A weak reference to the active brush must be stored in the toolsettings when writing a .blend file, such that it can be reloaded automatically on file opening when there is no active brushes in the current editing session. * - [ ] Several brushes with the same name (coming from different asset repositories/libraries) must be supported. * Displaying them in the asset browser/asset shelf should not be an issue? (*Check with @JulianEisel*). * Local runtime overrides however need a unique name. One idea could be to hide their `.xxx` number suffix in the UI? * - [x] It is assumed that only Brush data itself needs to be editable, its dependencies (textures, etc.) can remain linked and readonly. Typically, real local brush data should only exist in files part of an asset repository. Brushes are reset when closing and re-opening Blender, however they are kept across file opening within and editing session. So they are handled in a similar way to UI data (WindowManager etc. IDs) when 'load UI' is disabled, and moved from old to new Main on file loading. ## Selecting a Brush When selecting the asset: * - [x] Brush is linked from asset .blend file (path given by asset system). * - [x] Existing local runtime liboverride of linked brush is searched for. If none is found, a new one is created. * - [x] That local runtime liboverride is assigned as active brush. ## Undo & File Loading No undo support should be needed for brush usage nor editing. Brushes can therefore be fully ignored by undo system, and existing brushes should just be re-used as-is. Brushes are also kept across file loading within a same Blender session, unless the (factory) startup file is loaded, or when loading to/from a brush asset library file. Technically this has several complex implications, especially in `setup_app_data` part of the file loading process, mainly due to the fact that brushes can use other data-blocks. Currently: * Dependencies are not fully guaranteed to be handled by the new behavior: * [x] In **undo** case, other IDs are handled like any other data-block, i.e. changes to them are undone or redone (so editing an image or texture of a brush will be undoable). This is not ideal, but extremely hard to avoid in current situation. * [x] In **file loading** case, other dependencies can be brought over from the old bmain into the new one. This implies that e.g. is a locale texture is assigned to a runtime only liboverride of a brush while editing `file_a.blend`, and then user open `file_b.blend`, a copy of that locale texture will become part of `file_b.blend`. * [x] Opening another .blend file, in general: * All fully local, non-asset brushes are preserved *(necessary for old brushes compatibility)*. * All fully local, asset brushes are discarded. * All local overrides of asset brushes are preserved (besides special cases noted below). * All linked brushes are preserved (besides special cases noted below). * [x] Opening a file that is a library of current file (typical case: opening a Brush Asset library file): * Reset related brush assets to their status in the library (linked brushes and their local overrides are not preserved). * Have clear UI visibility of local assets having 'pending changes' in drafts. * Warn user that by opening the library file, they will lose their current non-drafted changes in these assets. * Requires detection of these cases. * [x] Opening a .blend file that is using as library the current blend file. * *Local asset brushes must be discarded in that case, this is covered by the general case described above*. ## do_version For modes moved to the new BrushAssets system, existing local brushes in older .blend files are often redundant and should be cleaned up. * [ ] Local non-asset brushes which names and values match the current default 'embedded' brushes should be removed. * *'Matching' here is used as 'essentially equal', in the sense that some properties (like e.g. size) should not be considered in the comparison.* * This implies keeping (or making) old brushes available for do_version code. * [ ] The others (non-matching) will be kept as local, non-asset brushes. Up to the user to clean them up or transform them into assets. Local brushes that are not defined as assets will not be shown to the user in the UI (they will only be reachable through the Outliner editor, python API, etc.). ## To Be Investigated ### Toolslot The 'toolslots' system is used at least by the 'smooth mode' toggle option of stroke code, which does a weird 'hot-swap' between current brush and a smooth brush, searched first in the relevant entry of the toolslot. This needs to be investigated further. > **Note** > One idea to fix the 'smooth mode' issue is that this toggle would not anymore search for an actual existing smooth brush, but would rather generate it's own private temp smooth brush, with default settings.
Bastien Montagne commented 2022-10-18 19:34:09 +02:00
Author
Owner
Copy Link

Changed status from 'Needs Triage' to: 'Confirmed'

Changed status from 'Needs Triage' to: 'Confirmed'
Bastien Montagne commented 2022-10-18 19:34:09 +02:00
Author
Owner
Copy Link

Added subscribers: @mont29, @JulienKaspar, @JulianEisel, @dfelinto

Added subscribers: @mont29, @JulienKaspar, @JulianEisel, @dfelinto
Paul Larson commented 2022-10-24 17:27:32 +02:00
Copy Link

Added subscriber: @GeorgiaPacific

Added subscriber: @GeorgiaPacific
Julien Kaspar added this to the Sculpt, Paint & Texture project 2023-02-08 10:20:48 +01:00
Bastien Montagne modified the project from Sculpt, Paint & Texture to Core 2023-02-09 15:42:54 +01:00
Bastien Montagne removed this from the Core project 2023-02-09 18:20:29 +01:00
Philipp Oeser removed the
Interest
Sculpt, Paint & Texture
 label 2023-02-10 09:10:59 +01:00
Bastien Montagne added the
Interest
Asset System
Interest
Pipeline, Assets & IO
 labels 2023-02-10 16:08:13 +01:00
Julian Eisel added this to the Brush Assets & Asset Shelf project 2023-02-20 21:01:06 +01:00
Brecht Van Lommel commented 2023-02-28 13:26:20 +01:00
Owner
Copy Link

Is there a plan for how to handle versioning, how we will transitions from brushes saved in .blend files to assets?

I see that local brushes will be preserved. If asset brushes come bundled with Blender and are also loaded, for existing .blend files the user will be presented with duplicated brushes?

Should we automatically remove duplicated brushes? Or put local ones in a somewhat hidden state, with some way for users to add specific ones as draft and clear the remaining ones?

Also related to this, when you edit a brush asset .blend file, I guess it avoids loading the same brushes a second time through the asset system? In this case local brushes are not legacy data.

Is there a plan for how to handle versioning, how we will transitions from brushes saved in .blend files to assets? I see that local brushes will be preserved. If asset brushes come bundled with Blender and are also loaded, for existing .blend files the user will be presented with duplicated brushes? Should we automatically remove duplicated brushes? Or put local ones in a somewhat hidden state, with some way for users to add specific ones as draft and clear the remaining ones? Also related to this, when you edit a brush asset .blend file, I guess it avoids loading the same brushes a second time through the asset system? In this case local brushes are not legacy data.
Bastien Montagne commented 2023-02-28 15:31:01 +01:00
Author
Owner
Copy Link

Regarding versioning, I thought removing local brushes which names and values match exactly the current default 'embedded' brushes would do the trick? The others would be kept as local brushes, up to the user to clean them up or transform them into assets.

Local brushes that are not defined as assets will not be shown to the user in the UI (they will only be reachable through the Outliner editor).

Also related to this, when you edit a brush asset .blend file, I guess it avoids loading the same brushes a second time through the asset system? In this case local brushes are not legacy data.

This is handled by the asset system itself, local brushes are shown as local assets, and not directly exposed as local IDs (besides, once again, from the Outliner view).

Regarding versioning, I thought removing local brushes which names and values match exactly the current default 'embedded' brushes would do the trick? The others would be kept as local brushes, up to the user to clean them up or transform them into assets. Local brushes that are not defined as assets will not be shown to the user in the UI (they will only be reachable through the Outliner editor). > Also related to this, when you edit a brush asset .blend file, I guess it avoids loading the same brushes a second time through the asset system? In this case local brushes are not legacy data. This is handled by the asset system itself, local brushes are shown as local assets, and not directly exposed as local IDs (besides, once again, from the Outliner view).
Brecht Van Lommel commented 2023-02-28 15:42:08 +01:00
Owner
Copy Link

Ah right, if the brush selection UI for the various modes only lists asset brushes then I agree they are hidden well enough.

Removing duplicates by comparing with the default ones seems good too (maybe should be a to do item in this task?). Probably that comparison can be made a bit fuzzy by allowing some things to be different, e.g. a brush with just a different size does not need to be preserved. But not that important.

Ah right, if the brush selection UI for the various modes only lists asset brushes then I agree they are hidden well enough. Removing duplicates by comparing with the default ones seems good too (maybe should be a to do item in this task?). Probably that comparison can be made a bit fuzzy by allowing some things to be different, e.g. a brush with just a different size does not need to be preserved. But not that important.
Bastien Montagne commented 2023-02-28 17:50:47 +01:00
Author
Owner
Copy Link

maybe should be a to do item in this task

Good points indeed, will do.

> maybe should be a to do item in this task Good points indeed, will do.
Brecht Van Lommel commented 2024-05-28 18:42:31 +02:00
Owner
Copy Link

I think this can be closed now, the remaining concrete tasks are in #116337.

I think this can be closed now, the remaining concrete tasks are in #116337.
Blender Bot added
Status
Archived
 and removed
Status
Confirmed
 labels 2024-05-28 18:43:59 +02:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels   
Interest
Alembic

  
Interest
Animation & Rigging

  
Interest
Asset System
Asset datablocks, libraries, browser and shelf

  
Interest
Audio

  
Interest
Automated Testing

  
Interest
Blender Asset Bundle

  
Interest
BlendFile

  
Interest
Collada

  
Interest
Compatibility
Backward and forward compatibility

  
Interest
Compositing

  
Interest
Core

  
Interest
Cycles

  
Interest
Dependency Graph

  
Interest
Development Management

  
Interest
EEVEE

  
Interest
Freestyle

  
Interest
Geometry Nodes

  
Interest
Grease Pencil

  
Interest
ID Management

  
Interest
Images & Movies

  
Interest
Import Export

  
Interest
Line Art

  
Interest
Masking

  
Interest
Metal

  
Interest
Modeling

  
Interest
Modifiers

  
Interest
Motion Tracking

  
Interest
Nodes & Physics

  
Interest
OpenGL

  
Interest
Overlay

  
Interest
Overrides

  
Interest
Performance

  
Interest
Physics

  
Interest
Pipeline, Assets & IO

  
Interest
Platforms, Builds & Tests

  
Interest
Python API

  
Interest
Render & Cycles

  
Interest
Render Pipeline

  
Interest
Sculpt, Paint & Texture

  
Interest
Text Editor

  
Interest
Translations

  
Interest
Triaging

  
Interest
Undo

  
Interest
USD

  
Interest
User Interface

  
Interest
UV Editing

  
Interest
VFX & Video

  
Interest
Video Sequencer

  
Interest
Viewport & EEVEE

  
Interest
Virtual Reality

  
Interest
Vulkan

  
Interest
Wayland
Wayland windowing on Unix

  
Interest
Workbench

  
Interest: X11

Xorg/X11 windowing

  
Legacy
Asset Browser Project

Archived

  
Legacy
Blender 2.8 Project

Archived

  
Legacy
Milestone 1: Basic, Local Asset Browser

Archived

  
Legacy
OpenGL Error

Archived

  
Meta
Good First Issue

  
Meta
Papercut

  
Meta
Retrospective

  
Meta
Security
Related to security, https://wiki.blender.org/wiki/Process/Vulnerability_Reports

  
Module
Animation & Rigging

  
Module
Core

  
Module
Development Management

  
Module
Grease Pencil

  
Module
Modeling

  
Module
Nodes & Physics

  
Module
Pipeline, Assets & IO

  
Module
Platforms, Builds & Tests

  
Module
Python API

  
Module
Render & Cycles

  
Module
Sculpt, Paint & Texture

  
Module
Triaging

  
Module
User Interface

  
Module
VFX & Video

  
Module
Viewport & EEVEE

  
Platform
FreeBSD

  
Platform
Linux

  
Platform
macOS

  
Platform
Windows

  
Severity
High

  
Severity
Low

  
Severity
Normal

  
Severity
Unbreak Now!

  
Status
Archived

  
Status
Confirmed

  
Status
Duplicate

  
Status
Needs Info from Developers

  
Status
Needs Information from User

  
Status
Needs Triage

  
Status
Resolved

  
Type
Bug

  
Type
Design

  
Type
Known Issue

  
Type
Patch

Archived

  
Type
Report

Archived

  
Type
To Do

No Label
Interest
Alembic
Interest
Animation & Rigging
Interest
Asset System
Interest
Audio
Interest
Automated Testing
Interest
Blender Asset Bundle
Interest
BlendFile
Interest
Collada
Interest
Compatibility
Interest
Compositing
Interest
Core
Interest
Cycles
Interest
Dependency Graph
Interest
Development Management
Interest
EEVEE
Interest
Freestyle
Interest
Geometry Nodes
Interest
Grease Pencil
Interest
ID Management
Interest
Images & Movies
Interest
Import Export
Interest
Line Art
Interest
Masking
Interest
Metal
Interest
Modeling
Interest
Modifiers
Interest
Motion Tracking
Interest
Nodes & Physics
Interest
OpenGL
Interest
Overlay
Interest
Overrides
Interest
Performance
Interest
Physics
Interest
Pipeline, Assets & IO
Interest
Platforms, Builds & Tests
Interest
Python API
Interest
Render & Cycles
Interest
Render Pipeline
Interest
Sculpt, Paint & Texture
Interest
Text Editor
Interest
Translations
Interest
Triaging
Interest
Undo
Interest
USD
Interest
User Interface
Interest
UV Editing
Interest
VFX & Video
Interest
Video Sequencer
Interest
Viewport & EEVEE
Interest
Virtual Reality
Interest
Vulkan
Interest
Wayland
Interest
Workbench
Interest: X11
Legacy
Asset Browser Project
Legacy
Blender 2.8 Project
Legacy
Milestone 1: Basic, Local Asset Browser
Legacy
OpenGL Error
Meta
Good First Issue
Meta
Papercut
Meta
Retrospective
Meta
Security
Module
Animation & Rigging
Module
Core
Module
Development Management
Module
Grease Pencil
Module
Modeling
Module
Nodes & Physics
Module
Pipeline, Assets & IO
Module
Platforms, Builds & Tests
Module
Python API
Module
Render & Cycles
Module
Sculpt, Paint & Texture
Module
Triaging
Module
User Interface
Module
VFX & Video
Module
Viewport & EEVEE
Platform
FreeBSD
Platform
Linux
Platform
macOS
Platform
Windows
Severity
High
Severity
Low
Severity
Normal
Severity
Unbreak Now!
Status
Archived
Status
Confirmed
Status
Duplicate
Status
Needs Info from Developers
Status
Needs Information from User
Status
Needs Triage
Status
Resolved
Type
Bug
Type
Design
Type
Known Issue
Type
Patch
Type
Report
Type
To Do
Milestone
Clear milestone
No items
No Milestone
Projects
Clear projects
No project
Brush Assets & Asset Shelf
Assignees
Clear assignees
No Assignees
3 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: blender/blender#101908
No description provided.
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?