Append: in localize All case, LibOverrides does not behave properly and will leave Main data in invalid state. #109004

Closed
opened 2023-06-15 11:41:39 +02:00 by Benoit Alexis · 7 comments

Problem

in Localize All case, append process does not handle LibOverride data properly currently: LibOverrides are kept as-is, but (some of) their dependencies may be made local directly. This leads to local liboverrides with local ID references, and/or linked data using local IDs...

Current append code relies on the premises that all newly linked data is made local. This is no longer applicable when linking liboverrides, since their linked references should never be made local (regardless of whether liboverrides themselves should be kept as liboverrides, or also be made fully local data).

Solving this issue is not trivial, and will require some careful thinking and design.

Solution Ideas

There are two possible ways to handle liboverrides when appending:

  1. Remove them to get fully local data without any linked dependency.
  2. Keep them as local liboverrides.

Currently, Blender tries to implement behavior 2. However, from a consistency and logical PoV, it would likely make more sense to rather go for behavior 1?

Regardless of the final outcome, Append process needs to be modified to take into account LibOverride data usages:

  • LibOverride reference pointers to linked data should not be ignored anymore.
  • These reference pointers should be handled in a specific way:
    • They should never become local.
    • Any data used by these linked reference must therefore be copied in case it needs to become local (and cannot be directly made local anymore).
    • These liboverride reference linked IDs should never get their ID pointers remapped to local copies either.
  • If behavior 2 is kept, there is the complex topic of what to do with 'regular linked data` from appended liboverrides (e.g. a liboverride Object usually still uses a simple linked obdata ID, not a liboverride of its reference obdata)?
    • This point alone is imho a strong reason to switch to behavior 1. Achieving proper consistency in behavior 2 feels simply impossible, even from a theoretical PoV.

Original Report

System Information
Operating system: Windows-10-10.0.19045-SP0 64 Bits
Graphics card: NVIDIA GeForce RTX 2080 Ti/PCIe/SSE2 NVIDIA Corporation 4.5.0 NVIDIA 516.94

Blender Version
Broken: version: 3.6.0 Beta, branch: blender-v3.6-release, commit date: 2023-06-13 23:42, hash: 962331c256b0
Worked: (newest version of Blender that worked as expected)

Short description of error
Based on my experiments for #108967 I found a new crash in 3.6. Drag and dropping a specific collection asset from the asset browser in Append mode will cause a crash if this same collection asset was previously imported.
(duplicating the collection works fine, and link mode 'twice' does nothing but change the position of the previously imported collection).

Context about attached files:
The collection asset in "VP_Simulator" was dropped in "WUX450ST.blend" in linked mode, with Instance collection unckecked. Then I parented the "projector" to the object "wux450st" and put the asset collection in another collection which I made an asset.

Exact steps for others to reproduce the error

  • Add attached files in asset library
  • Open new scene
  • Append WUX450ST collection to the scene (uncheck instance from redo panel)
  • Append same collection again (crash(

No crash if I import the asset collection projector_sim in append mode

# Problem in `Localize All` case, append process does not handle LibOverride data properly currently: LibOverrides are kept as-is, but (some of) their dependencies may be made local directly. This leads to local liboverrides with local ID references, and/or linked data using local IDs... Current append code relies on the premises that _all_ newly linked data is made local. This is no longer applicable when linking liboverrides, since their linked references should never be made local (regardless of whether liboverrides themselves should be kept as liboverrides, or also be made fully local data). Solving this issue is not trivial, and will require some careful thinking and design. # Solution Ideas There are two possible ways to handle liboverrides when appending: 1. Remove them to get fully local data without any linked dependency. 2. Keep them as local liboverrides. Currently, Blender tries to implement behavior `2`. However, from a consistency and logical PoV, it would likely make more sense to rather go for behavior `1`? Regardless of the final outcome, Append process needs to be modified to take into account LibOverride data usages: * LibOverride reference pointers to linked data should not be ignored anymore. * These reference pointers should be handled in a specific way: * They should never become local. * Any data used by these linked reference must therefore be copied in case it needs to become local (and cannot be directly made local anymore). * These liboverride reference linked IDs should never get their ID pointers remapped to local copies either. * If behavior `2` is kept, there is the complex topic of what to do with 'regular linked data` from appended liboverrides (e.g. a liboverride Object usually still uses a simple linked obdata ID, not a liboverride of its reference obdata)? * _This point alone is imho a *strong* reason to switch to behavior `1`. Achieving proper consistency in behavior `2` feels simply impossible, even from a theoretical PoV._ --------------------------------- _Original Report_ **System Information** Operating system: Windows-10-10.0.19045-SP0 64 Bits Graphics card: NVIDIA GeForce RTX 2080 Ti/PCIe/SSE2 NVIDIA Corporation 4.5.0 NVIDIA 516.94 **Blender Version** Broken: version: 3.6.0 Beta, branch: blender-v3.6-release, commit date: 2023-06-13 23:42, hash: `962331c256b0` Worked: (newest version of Blender that worked as expected) **Short description of error** Based on my experiments for #108967 I found a new crash in 3.6. Drag and dropping a specific collection asset from the asset browser in Append mode will cause a crash if this same collection asset was previously imported. (duplicating the collection works fine, and link mode 'twice' does nothing but change the position of the previously imported collection). Context about attached files: The collection asset in "VP_Simulator" was dropped in "WUX450ST.blend" in linked mode, with Instance collection unckecked. Then I parented the "projector" to the object "wux450st" and put the asset collection in another collection which I made an asset. **Exact steps for others to reproduce the error** - Add attached files in asset library - Open new scene - Append `WUX450ST` collection to the scene (uncheck `instance` from redo panel) - Append same collection again (crash( No crash if I import the asset collection `projector_sim` in append mode
Benoit Alexis added the
Type
Report
Status
Needs Triage
Severity
Normal
labels 2023-06-15 11:41:40 +02:00
Member

Thanks for the report. I can confirm the crash.

Thanks for the report. I can confirm the crash.
Member

I am able to replicate the crash with following steps:

  • new file
  • Append asset Projector_Sim into the scene twice

Edit:

Can't replicate the crash in debugger...

I am able to replicate the crash with following steps: - new file - `Append` asset `Projector_Sim` into the scene twice Edit: Can't replicate the crash in debugger...
Member

BlendfileLinkAppendContextItem id.lib is NULL in this case, but lib.filepath is accessed (same as BlendfileLinkAppendContextItem source_library)

1  BLI_strnlen                                 string.c                981  0x1560e5ff 
2  BLI_strncpy                                 string.c                67   0x15607607 
3  lib_weak_key_create                         main.c                  325  0xd4d284   
4  BKE_main_library_weak_reference_search_item main.c                  388  0xd4de19   
5  BKE_blendfile_append                        blendfile_link_append.c 1069 0xbc6ca30  
6  wm_file_link_append_datablock_ex            wm_files_link.c         539  0x3e46b6f  
7  WM_file_append_datablock                    wm_files_link.c         576  0x3e46ce0  
8  WM_drag_asset_id_import                     wm_dragdrop.cc          642  0x3dc6bc0  
9  view3d_collection_drop_copy_external_asset  space_view3d.cc         857  0xb57a25a  
10 wm_drop_prepare                             wm_dragdrop.cc          475  0x3dc528e  
11 wm_handlers_do_intern                       wm_event_system.cc      3324 0x3dfe43b  
12 wm_handlers_do                              wm_event_system.cc      3429 0x3dff326  
13 wm_event_do_region_handlers                 wm_event_system.cc      3855 0x3e06428  
14 wm_event_do_handlers_area_regions           wm_event_system.cc      3887 0x3e0681f  
15 wm_event_do_handlers                        wm_event_system.cc      4111 0x3e0a18d  
16 WM_main                                     wm.c                    646  0x3dc07db  
17 main                                        creator.c               583  0x804bbf   

@mont29 will know best here

`BlendfileLinkAppendContextItem` `id.lib` is NULL in this case, but `lib.filepath` is accessed (same as `BlendfileLinkAppendContextItem` `source_library`) ``` 1 BLI_strnlen string.c 981 0x1560e5ff 2 BLI_strncpy string.c 67 0x15607607 3 lib_weak_key_create main.c 325 0xd4d284 4 BKE_main_library_weak_reference_search_item main.c 388 0xd4de19 5 BKE_blendfile_append blendfile_link_append.c 1069 0xbc6ca30 6 wm_file_link_append_datablock_ex wm_files_link.c 539 0x3e46b6f 7 WM_file_append_datablock wm_files_link.c 576 0x3e46ce0 8 WM_drag_asset_id_import wm_dragdrop.cc 642 0x3dc6bc0 9 view3d_collection_drop_copy_external_asset space_view3d.cc 857 0xb57a25a 10 wm_drop_prepare wm_dragdrop.cc 475 0x3dc528e 11 wm_handlers_do_intern wm_event_system.cc 3324 0x3dfe43b 12 wm_handlers_do wm_event_system.cc 3429 0x3dff326 13 wm_event_do_region_handlers wm_event_system.cc 3855 0x3e06428 14 wm_event_do_handlers_area_regions wm_event_system.cc 3887 0x3e0681f 15 wm_event_do_handlers wm_event_system.cc 4111 0x3e0a18d 16 WM_main wm.c 646 0x3dc07db 17 main creator.c 583 0x804bbf ``` @mont29 will know best here

I cannot reproduce any crash currently, neither with 3.6 not 4.0?

I cannot reproduce any crash currently, neither with 3.6 not 4.0?
Bastien Montagne added
Status
Needs Information from User
and removed
Status
Confirmed
labels 2023-06-22 15:00:18 +02:00
Member

@mont29 : I can still repro in both 4.0 and 3.6 with the steps in the report description

@mont29 : I can still repro in both 4.0 and 3.6 with the steps in the report description
Philipp Oeser added
Status
Confirmed
and removed
Status
Needs Information from User
labels 2023-06-23 11:10:45 +02:00

Aaaaah ok it works with 'append and reuse' (my default), but not just 'append'.

Aaaaah ok it works with 'append and reuse' (my default), but not just 'append'.
Bastien Montagne added
Type
Bug
and removed
Type
Report
labels 2023-06-23 12:46:31 +02:00
Bastien Montagne added this to the 3.6 LTS milestone 2023-06-23 12:46:35 +02:00
Bastien Montagne changed title from Assets : Crash when drag and dropping a collection asset in Append mode twice to Append: Appending LibOverrides does not behave properly and will leave Main data in invalid state. 2023-06-23 16:21:54 +02:00

Updated description with actual problem and how it could be solved. This won't be trivial.

TL;DR: do not try to append LibOverride data until this issue is addressed.

Updated description with actual problem and how it could be solved. This won't be trivial. TL;DR: do not try to append LibOverride data until this issue is addressed.
Bastien Montagne changed title from Append: Appending LibOverrides does not behave properly and will leave Main data in invalid state. to Append: in `localize All` case, LibOverrides does not behave properly and will leave Main data in invalid state. 2023-06-23 16:58:18 +02:00
Blender Bot added
Status
Resolved
and removed
Status
Confirmed
labels 2023-06-27 18:32:09 +02:00
Sign in to join this conversation.
No Label
Interest
Alembic
Interest
Animation & Rigging
Interest
Asset System
Interest
Audio
Interest
Automated Testing
Interest
Blender Asset Bundle
Interest
BlendFile
Interest
Code Documentation
Interest
Collada
Interest
Compatibility
Interest
Compositing
Interest
Core
Interest
Cycles
Interest
Dependency Graph
Interest
Development Management
Interest
EEVEE
Interest
Freestyle
Interest
Geometry Nodes
Interest
Grease Pencil
Interest
ID Management
Interest
Images & Movies
Interest
Import Export
Interest
Line Art
Interest
Masking
Interest
Metal
Interest
Modeling
Interest
Modifiers
Interest
Motion Tracking
Interest
Nodes & Physics
Interest
OpenGL
Interest
Overlay
Interest
Overrides
Interest
Performance
Interest
Physics
Interest
Pipeline, Assets & IO
Interest
Platforms, Builds & Tests
Interest
Python API
Interest
Render & Cycles
Interest
Render Pipeline
Interest
Sculpt, Paint & Texture
Interest
Text Editor
Interest
Translations
Interest
Triaging
Interest
Undo
Interest
USD
Interest
User Interface
Interest
UV Editing
Interest
VFX & Video
Interest
Video Sequencer
Interest
Viewport & EEVEE
Interest
Virtual Reality
Interest
Vulkan
Interest
Wayland
Interest
Workbench
Interest: X11
Legacy
Asset Browser Project
Legacy
Blender 2.8 Project
Legacy
Milestone 1: Basic, Local Asset Browser
Legacy
OpenGL Error
Meta
Good First Issue
Meta
Papercut
Meta
Retrospective
Meta
Security
Module
Animation & Rigging
Module
Core
Module
Development Management
Module
Grease Pencil
Module
Modeling
Module
Nodes & Physics
Module
Pipeline, Assets & IO
Module
Platforms, Builds & Tests
Module
Python API
Module
Render & Cycles
Module
Sculpt, Paint & Texture
Module
Triaging
Module
User Interface
Module
VFX & Video
Module
Viewport & EEVEE
Platform
FreeBSD
Platform
Linux
Platform
macOS
Platform
Windows
Severity
High
Severity
Low
Severity
Normal
Severity
Unbreak Now!
Status
Archived
Status
Confirmed
Status
Duplicate
Status
Needs Info from Developers
Status
Needs Information from User
Status
Needs Triage
Status
Resolved
Type
Bug
Type
Design
Type
Known Issue
Type
Patch
Type
Report
Type
To Do
No Milestone
No project
No Assignees
5 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: blender/blender#109004
No description provided.