Add a mechanism to abort blend file reading on critical error. #105083

New Issue

Bastien Montagne · 2023-02-22T17:53:16+01:00

Bastien Montagne commented

2023-02-22 17:53:16 +01:00

There are many ways to forge invalid blend files, and random corruption can also happen.

Current blendfile reading code will usually misbehave severely in case some fundamentals of a blendfile structure are violated (typically leading to segfault and crash).

The goal of this design issue is to define a better way to survive such errors.

Proposal

The general idea is to introduce a C++-exception based mechanism to abort a blend file reading.

The generated Main is then marked as invalid, and calling code (like e.g. #BKE_blendfile_read etc.) will not proceed in such case.

This proposal does not cover the segfault case (as it is not a C++ exception). From a quick search on stackoverflow it does not seem to be recommended to try to convert a segfault into an exception either (or any other way to 'recover' from a segfault).

Proposal regarding usage of the new mechanism vs. existing ones:

There should be no more BLI_assert usage in reading code.
If an error is serious enough to be considered as a bug, but is recoverable, code should generate an ERROR report, and fix the issue.
If the error is not recoverable, code should use the new abort fileread mechanism.

Note:
this design was triggered by the #99836 issue.

There are many ways to forge invalid blend files, and random corruption can also happen. Current blendfile reading code will usually misbehave severely in case some fundamentals of a blendfile structure are violated (typically leading to segfault and crash). The goal of this design issue is to define a better way to survive such errors. ### Proposal The general idea is to introduce a C++-exception based mechanism to abort a blend file reading. The generated Main is then marked as invalid, and calling code (like e.g. #BKE_blendfile_read etc.) will not proceed in such case. This proposal does not cover the segfault case (as it is not a C++ exception). From a quick search on stackoverflow it does not seem to be recommended to try to convert a segfault into an exception either (or any other way to 'recover' from a segfault). Proposal regarding usage of the new mechanism vs. existing ones: - There should be no more `BLI_assert` usage in reading code. - If an error is serious enough to be considered as a bug, but is recoverable, code should generate an ERROR report, and fix the issue. - If the error is not recoverable, code should use the new abort fileread mechanism. > Note: > this design was triggered by the #99836 issue.

Bastien Montagne added the

labels 2023-02-22 17:53:24 +01:00

Bastien Montagne referenced this issue

2023-02-22 18:01:19 +01:00

WIP: Add a mechanism to abort a blend file reading on critical error. #105085

Bastien Montagne commented

2023-02-22 18:09:03 +01:00

@brecht @sergey @JacquesLucke Guess you guys may be interested by the topic. ;)

Jacques Lucke commented

2023-02-22 18:23:42 +01:00

I agree with the general goal of course, crashes due to bad .blend files should be minimized.

The general idea is to introduce a C++-exception based mechanism to abort a blend file reading.

Can work but probably takes lots of changes to get right, but maybe not as many to get something better than we have now (better have some memory leaks than a crash). It's always tricky to decide when exceptions should be used instead of error return values, but it seems ok to use exceptions for this case.

There should be no more BLI_assert usage in reading code.

That's a strong statement, and I don't agree with it. Asserts certainly should not directly be used on data that the user inputs (which includes .blend files). They should be used to state assumptions about the behavior of other code instead. So when an assert hits, there is always a bug in the code somewhere (even if just the assert is wrong). It should not be possible to trigger an assert with bad input data.

If an error is serious enough to be considered as a bug, but is recoverable, code should generate an ERROR report, and fix the issue.

Wonder how you would define recoverable. It's probably important to prevent the user from accidentally saving over an existing .blend when saving a file that has been partially loaded.

I agree with the general goal of course, crashes due to bad .blend files should be minimized. > The general idea is to introduce a C++-exception based mechanism to abort a blend file reading. Can work but probably takes lots of changes to get right, but maybe not as many to get something better than we have now (better have some memory leaks than a crash). It's always tricky to decide when exceptions should be used instead of error return values, but it seems ok to use exceptions for this case. > There should be no more `BLI_assert` usage in reading code. That's a strong statement, and I don't agree with it. Asserts certainly should not directly be used on data that the user inputs (which includes .blend files). They should be used to state assumptions about the behavior of other code instead. So when an assert hits, there is always a bug in the code somewhere (even if just the assert is wrong). It should not be possible to trigger an assert with bad input data. > If an error is serious enough to be considered as a bug, but is recoverable, code should generate an ERROR report, and fix the issue. Wonder how you would define recoverable. It's probably important to prevent the user from accidentally saving over an existing .blend when saving a file that has been partially loaded.

Bastien Montagne commented

2023-02-23 11:08:14 +01:00

It does work with reasonable amount of changes afaict, at least way less verbose than what would be required if implemented with return value or so imho (ref. #105085).

There should be no more BLI_assert usage in reading code.

That's a strong statement, and I don't agree with it. Asserts certainly should not directly be used on data that the user inputs (which includes .blend files). They should be used to state assumptions about the behavior of other code instead. So when an assert hits, there is always a bug in the code somewhere (even if just the assert is wrong). It should not be possible to trigger an assert with bad input data.

I agree with that statement. Maybe the issue here is the definition of 'reading code', for me it's code in BLO + dedicated callbacks in IDTypes. Other code called from readfile (e.g. BKE code adding missing data etc.) is not part of this context.

I don't think any code in BLO can be seen as not working with 'data from .blend file' ?

If an error is serious enough to be considered as a bug, but is recoverable, code should generate an ERROR report, and fix the issue.

Wonder how you would define recoverable. It's probably important to prevent the user from accidentally saving over an existing .blend when saving a file that has been partially loaded.

Recoverable means that the error is unambiguous, and that there is an unambiguous way to fix it, such that it is guaranteed that the result won't make the final Main data-base invalid in any way.

E.g. in #99836, there is a first stage fix that can be tried, which is assigning the first scene in given main to a screen with a null scene pointer. The fatal error would then happen only in case there is no scene at all in given main.

IMHO it's better to try and fix things as much as possible (with relevant warnings), even if some data is lost or modified, rather than making a whole .blend file fully unloadable.

It does work with reasonable amount of changes afaict, at least way less verbose than what would be required if implemented with return value or so imho (ref. #105085). >> There should be no more BLI_assert usage in reading code. > > That's a strong statement, and I don't agree with it. Asserts certainly should not directly be used on data that the user inputs (which includes .blend files). They should be used to state assumptions about the behavior of other code instead. So when an assert hits, there is always a bug in the code somewhere (even if just the assert is wrong). It should not be possible to trigger an assert with bad input data. I agree with that statement. Maybe the issue here is the definition of 'reading code', for me it's code in BLO + dedicated callbacks in IDTypes. Other code called from readfile (e.g. BKE code adding missing data etc.) is not part of this context. I don't think any code in BLO can be seen as not working with 'data from .blend file' ? >> If an error is serious enough to be considered as a bug, but is recoverable, code should generate an ERROR report, and fix the issue. > > Wonder how you would define recoverable. It's probably important to prevent the user from accidentally saving over an existing .blend when saving a file that has been partially loaded. Recoverable means that the error is unambiguous, and that there is an unambiguous way to fix it, such that it is guaranteed that the result won't make the final Main data-base invalid in any way. E.g. in #99836, there is a first stage fix that can be tried, which is assigning the first scene in given main to a screen with a null scene pointer. The fatal error would then happen only in case there is no scene at all in given main. IMHO it's better to try and fix things as much as possible (with relevant warnings), even if some data is lost or modified, rather than making a whole .blend file fully unloadable.

Bastien Montagne added the

Download

What's New

Blender Studio

Manual

Developers Blog

Documentation

Benchmark

Blender Conference

Development Fund

One-time Donations

Add a mechanism to abort blend file reading on critical error. #105083

Proposal