Add a mechanism to abort blend file reading on critical error. #105083

Closed
opened 2023-02-22 17:53:16 +01:00 by Bastien Montagne · 4 comments

There are many ways to forge invalid blend files, and random corruption can also happen.

Current blendfile reading code will usually misbehave severely in case some fundamentals of a blendfile structure are violated (typically leading to segfault and crash).

The goal of this design issue is to define a better way to survive such errors.

Proposal

The general idea is to introduce a C++-exception based mechanism to abort a blend file reading.

The generated Main is then marked as invalid, and calling code (like e.g. #BKE_blendfile_read etc.) will not proceed in such case.

This proposal does not cover the segfault case (as it is not a C++ exception). From a quick search on stackoverflow it does not seem to be recommended to try to convert a segfault into an exception either (or any other way to 'recover' from a segfault).

Proposal regarding usage of the new mechanism vs. existing ones:

  • There should be no more BLI_assert usage in reading code.
  • If an error is serious enough to be considered as a bug, but is recoverable, code should generate an ERROR report, and fix the issue.
  • If the error is not recoverable, code should use the new abort fileread mechanism.

Note:
this design was triggered by the #99836 issue.

There are many ways to forge invalid blend files, and random corruption can also happen. Current blendfile reading code will usually misbehave severely in case some fundamentals of a blendfile structure are violated (typically leading to segfault and crash). The goal of this design issue is to define a better way to survive such errors. ### Proposal The general idea is to introduce a C++-exception based mechanism to abort a blend file reading. The generated Main is then marked as invalid, and calling code (like e.g. #BKE_blendfile_read etc.) will not proceed in such case. This proposal does not cover the segfault case (as it is not a C++ exception). From a quick search on stackoverflow it does not seem to be recommended to try to convert a segfault into an exception either (or any other way to 'recover' from a segfault). Proposal regarding usage of the new mechanism vs. existing ones: - There should be no more `BLI_assert` usage in reading code. - If an error is serious enough to be considered as a bug, but is recoverable, code should generate an ERROR report, and fix the issue. - If the error is not recoverable, code should use the new abort fileread mechanism. > Note: > this design was triggered by the #99836 issue.
Bastien Montagne added the
Type
Design
Module
Core
Status
Confirmed
labels 2023-02-22 17:53:24 +01:00
Author
Owner

@brecht @sergey @JacquesLucke Guess you guys may be interested by the topic. ;)

@brecht @sergey @JacquesLucke Guess you guys may be interested by the topic. ;)
Member

I agree with the general goal of course, crashes due to bad .blend files should be minimized.

The general idea is to introduce a C++-exception based mechanism to abort a blend file reading.

Can work but probably takes lots of changes to get right, but maybe not as many to get something better than we have now (better have some memory leaks than a crash). It's always tricky to decide when exceptions should be used instead of error return values, but it seems ok to use exceptions for this case.

There should be no more BLI_assert usage in reading code.

That's a strong statement, and I don't agree with it. Asserts certainly should not directly be used on data that the user inputs (which includes .blend files). They should be used to state assumptions about the behavior of other code instead. So when an assert hits, there is always a bug in the code somewhere (even if just the assert is wrong). It should not be possible to trigger an assert with bad input data.

If an error is serious enough to be considered as a bug, but is recoverable, code should generate an ERROR report, and fix the issue.

Wonder how you would define recoverable. It's probably important to prevent the user from accidentally saving over an existing .blend when saving a file that has been partially loaded.

I agree with the general goal of course, crashes due to bad .blend files should be minimized. > The general idea is to introduce a C++-exception based mechanism to abort a blend file reading. Can work but probably takes lots of changes to get right, but maybe not as many to get something better than we have now (better have some memory leaks than a crash). It's always tricky to decide when exceptions should be used instead of error return values, but it seems ok to use exceptions for this case. > There should be no more `BLI_assert` usage in reading code. That's a strong statement, and I don't agree with it. Asserts certainly should not directly be used on data that the user inputs (which includes .blend files). They should be used to state assumptions about the behavior of other code instead. So when an assert hits, there is always a bug in the code somewhere (even if just the assert is wrong). It should not be possible to trigger an assert with bad input data. > If an error is serious enough to be considered as a bug, but is recoverable, code should generate an ERROR report, and fix the issue. Wonder how you would define recoverable. It's probably important to prevent the user from accidentally saving over an existing .blend when saving a file that has been partially loaded.
Author
Owner

It does work with reasonable amount of changes afaict, at least way less verbose than what would be required if implemented with return value or so imho (ref. #105085).

There should be no more BLI_assert usage in reading code.

That's a strong statement, and I don't agree with it. Asserts certainly should not directly be used on data that the user inputs (which includes .blend files). They should be used to state assumptions about the behavior of other code instead. So when an assert hits, there is always a bug in the code somewhere (even if just the assert is wrong). It should not be possible to trigger an assert with bad input data.

I agree with that statement. Maybe the issue here is the definition of 'reading code', for me it's code in BLO + dedicated callbacks in IDTypes. Other code called from readfile (e.g. BKE code adding missing data etc.) is not part of this context.

I don't think any code in BLO can be seen as not working with 'data from .blend file' ?

If an error is serious enough to be considered as a bug, but is recoverable, code should generate an ERROR report, and fix the issue.

Wonder how you would define recoverable. It's probably important to prevent the user from accidentally saving over an existing .blend when saving a file that has been partially loaded.

Recoverable means that the error is unambiguous, and that there is an unambiguous way to fix it, such that it is guaranteed that the result won't make the final Main data-base invalid in any way.

E.g. in #99836, there is a first stage fix that can be tried, which is assigning the first scene in given main to a screen with a null scene pointer. The fatal error would then happen only in case there is no scene at all in given main.

IMHO it's better to try and fix things as much as possible (with relevant warnings), even if some data is lost or modified, rather than making a whole .blend file fully unloadable.

It does work with reasonable amount of changes afaict, at least way less verbose than what would be required if implemented with return value or so imho (ref. #105085). >> There should be no more BLI_assert usage in reading code. > > That's a strong statement, and I don't agree with it. Asserts certainly should not directly be used on data that the user inputs (which includes .blend files). They should be used to state assumptions about the behavior of other code instead. So when an assert hits, there is always a bug in the code somewhere (even if just the assert is wrong). It should not be possible to trigger an assert with bad input data. I agree with that statement. Maybe the issue here is the definition of 'reading code', for me it's code in BLO + dedicated callbacks in IDTypes. Other code called from readfile (e.g. BKE code adding missing data etc.) is not part of this context. I don't think any code in BLO can be seen as not working with 'data from .blend file' ? >> If an error is serious enough to be considered as a bug, but is recoverable, code should generate an ERROR report, and fix the issue. > > Wonder how you would define recoverable. It's probably important to prevent the user from accidentally saving over an existing .blend when saving a file that has been partially loaded. Recoverable means that the error is unambiguous, and that there is an unambiguous way to fix it, such that it is guaranteed that the result won't make the final Main data-base invalid in any way. E.g. in #99836, there is a first stage fix that can be tried, which is assigning the first scene in given main to a screen with a null scene pointer. The fatal error would then happen only in case there is no scene at all in given main. IMHO it's better to try and fix things as much as possible (with relevant warnings), even if some data is lost or modified, rather than making a whole .blend file fully unloadable.
Bastien Montagne added the
Interest
BlendFile
Priority
Normal
labels 2023-02-23 16:34:06 +01:00
Author
Owner

Committed as b3f42d8e98.

Committed as b3f42d8e98.
Blender Bot added
Status
Archived
and removed
Status
Confirmed
labels 2023-03-01 12:30:29 +01:00
Sign in to join this conversation.
No Label
Interest
Alembic
Interest
Animation & Rigging
Interest
Asset Browser
Interest
Asset Browser Project Overview
Interest
Audio
Interest
Automated Testing
Interest
Blender Asset Bundle
Interest
BlendFile
Interest
Collada
Interest
Compatibility
Interest
Compositing
Interest
Core
Interest
Cycles
Interest
Dependency Graph
Interest
Development Management
Interest
EEVEE
Interest
EEVEE & Viewport
Interest
Freestyle
Interest
Geometry Nodes
Interest
Grease Pencil
Interest
ID Management
Interest
Images & Movies
Interest
Import Export
Interest
Line Art
Interest
Masking
Interest
Metal
Interest
Modeling
Interest
Modifiers
Interest
Motion Tracking
Interest
Nodes & Physics
Interest
OpenGL
Interest
Overlay
Interest
Overrides
Interest
Performance
Interest
Physics
Interest
Pipeline, Assets & IO
Interest
Platforms, Builds & Tests
Interest
Python API
Interest
Render & Cycles
Interest
Render Pipeline
Interest
Sculpt, Paint & Texture
Interest
Text Editor
Interest
Translations
Interest
Triaging
Interest
Undo
Interest
USD
Interest
User Interface
Interest
UV Editing
Interest
VFX & Video
Interest
Video Sequencer
Interest
Virtual Reality
Interest
Vulkan
Interest
Wayland
Interest
Workbench
Legacy
Blender 2.8 Project
Legacy
Milestone 1: Basic, Local Asset Browser
Legacy
OpenGL Error
Meta
Good First Issue
Meta
Papercut
Meta
Retrospective
Meta
Security
Module
Animation & Rigging
Module
Core
Module
Development Management
Module
EEVEE & Viewport
Module
Grease Pencil
Module
Modeling
Module
Nodes & Physics
Module
Pipeline, Assets & IO
Module
Platforms, Builds & Tests
Module
Python API
Module
Render & Cycles
Module
Sculpt, Paint & Texture
Module
Triaging
Module
User Interface
Module
VFX & Video
Platform
FreeBSD
Platform
Linux
Platform
macOS
Platform
Windows
Priority
High
Priority
Low
Priority
Normal
Priority
Unbreak Now!
Status
Archived
Status
Confirmed
Status
Duplicate
Status
Needs Info from Developers
Status
Needs Information from User
Status
Needs Triage
Status
Resolved
Type
Bug
Type
Design
Type
Known Issue
Type
Patch
Type
Report
Type
To Do
No Milestone
No project
No Assignees
2 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: blender/blender#105083
No description provided.