LibOverrides - Refactor how diffing of RNA collections is handled, and extend diffing possibilities. #82160
Labels
No Label
Interest
Alembic
Interest
Animation & Rigging
Interest
Asset Browser
Interest
Asset Browser Project Overview
Interest
Audio
Interest
Automated Testing
Interest
Blender Asset Bundle
Interest
BlendFile
Interest
Collada
Interest
Compatibility
Interest
Compositing
Interest
Core
Interest
Cycles
Interest
Dependency Graph
Interest
Development Management
Interest
EEVEE
Interest
EEVEE & Viewport
Interest
Freestyle
Interest
Geometry Nodes
Interest
Grease Pencil
Interest
ID Management
Interest
Images & Movies
Interest
Import Export
Interest
Line Art
Interest
Masking
Interest
Metal
Interest
Modeling
Interest
Modifiers
Interest
Motion Tracking
Interest
Nodes & Physics
Interest
OpenGL
Interest
Overlay
Interest
Overrides
Interest
Performance
Interest
Physics
Interest
Pipeline, Assets & IO
Interest
Platforms, Builds & Tests
Interest
Python API
Interest
Render & Cycles
Interest
Render Pipeline
Interest
Sculpt, Paint & Texture
Interest
Text Editor
Interest
Translations
Interest
Triaging
Interest
Undo
Interest
USD
Interest
User Interface
Interest
UV Editing
Interest
VFX & Video
Interest
Video Sequencer
Interest
Virtual Reality
Interest
Vulkan
Interest
Wayland
Interest
Workbench
Interest: X11
Legacy
Blender 2.8 Project
Legacy
Milestone 1: Basic, Local Asset Browser
Legacy
OpenGL Error
Meta
Good First Issue
Meta
Papercut
Meta
Retrospective
Meta
Security
Module
Animation & Rigging
Module
Core
Module
Development Management
Module
EEVEE & Viewport
Module
Grease Pencil
Module
Modeling
Module
Nodes & Physics
Module
Pipeline, Assets & IO
Module
Platforms, Builds & Tests
Module
Python API
Module
Render & Cycles
Module
Sculpt, Paint & Texture
Module
Triaging
Module
User Interface
Module
VFX & Video
Platform
FreeBSD
Platform
Linux
Platform
macOS
Platform
Windows
Priority
High
Priority
Low
Priority
Normal
Priority
Unbreak Now!
Status
Archived
Status
Confirmed
Status
Duplicate
Status
Needs Info from Developers
Status
Needs Information from User
Status
Needs Triage
Status
Resolved
Type
Bug
Type
Design
Type
Known Issue
Type
Patch
Type
Report
Type
To Do
No Milestone
No project
No Assignees
5 Participants
Notifications
Due Date
No due date set.
Dependencies
No dependencies set.
Reference: blender/blender#82160
Loading…
Reference in New Issue
No description provided.
Delete Branch "%!s(<nil>)"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Status:
Milestone 1 has been implemented in Blender 3.0.
It implies forward compatibility breakage (i.e. files saved in Blender 3.0 won't open in a fully correct state regarding inserted local override modifiers/constraints, in older versions). Some fix will be added to Blender 2.93 LTS to at least prevent crashes.
Milestone 2 is not yet scheduled.
Description
Big picture:
The insertion of new items in existing RNA collections of override data-blocks (e.g. object constraints or modifiers) was 'hacked' in in early stage of the project and unfortunately never properly reworked before it got into production.
It currently has a weird implementation and design flows that make it close to impossible to extend in practice (to support e.g. re-ordering, deletions, etc.)
Goals:
List Operations Design:
Ultimately this would include
Insertion
, possiblyre-ordering
of existing (from linked data) items. Supportingdeletion
operation remains an open topic, as it has fairly complicated implications, and is not strictly needed since items can usually be muted/de-activated anyway.Implementation comments are in italic below.
Current Design:
Order of operations defined in the override property is crucial, since if you insert a new item
A
, and then another itemB
afterA
, you will need to useA
as anchor forB
insertion. Insertion:Insertion operations are applied in a separate, second pass after all other override operations have been applied.* This is a mistake, insertion and ordering should be applied before any other override operation, such that order of items in destination data fully matches the one it had when operations were generated by diffing code). This is especially important for index-based operations or index-only lists.
If items in the collection can be referenced by unique names (e.g. modifiers, constraints), new items can be inserted anywhere, using any other existing item as anchor (insertion before/after that anchor item). If items in the collection can only be referenced by index (NLA tracks), new items can only be inserted after existing ones (i.e. after items coming from the linked reference datablock).
If name reference is defined, it takes precedence over index reference. If no valid reference is found, insertion point is assumed to be first (for insert after) or last (for insert before) position in the list.
*** Current implementation main flaw: it uses a single reference information,
subitem_local_name
/subitem_local_index
. This is used to find the anchor item after which the new one (matching current override operation) is inserted, in local override data. The one to be inserted is then always considered to be the next item in the list of the local (source, stored-in-blend) override data. This makesinsert after
operation the only possible one currently.*** Items inserted in local override data have a specific flag, different for each type (e.g.
eModifierFlag_OverrideLibrary_Local
for modifiers,CONSTRAINT_OVERRIDE_LIBRARY_LOCAL
for constraints, etc.).Re-ordering:* Not supported.
Target Design:
Insertion:* Fix current main issue by adding a third set of subitem data, called e.g.
subitem_storage_name
/subitem_storage_index
, to be used by insertion operations to find the source item from the local override data used as storage, when re-applying overrides over linked localized data.**** Versioning code could then simply detect that an insert operation has a define
subitem_local_xxx
info, but nosubitem_storage_xxx
one, and re-generate that one from the next item in the list of the local stored override data.*The rest of the current design for insertions should then be fine. Re-ordering:
***This should only be supported for lists that can reference items by their names (Index-based ones would be too prone to produce invalid data if linked reference data changes).Engineer plan:
Research the best way to
do_version
to use existing data model in the expected way for current, already implemented code/supported feature. This might imply a breaking of forward compatibility.Research more advanced algorithms able to detect deletion or re-ordering of existing items.
Work plan
Milestone 1 - Refactor Existing Code/Feature
33c5e7bcd5
,ec71054a9b
Try to find a way to ensure forward compatibility (not sure this is possible without unreasonable changes to the override data structures).Not practical to implement.95c82513ca
1d8d6c2f62
Milestone 2 - Advanced Diffing
Branch: ...
Relevant links:
Changed status from 'Needs Triage' to: 'Confirmed'
Added subscriber: @mont29
Added subscriber: @sebbas
Added subscriber: @RedMser
This issue was referenced by
33c5e7bcd5
Added subscriber: @timodriaan