When a lot of libraries or linked IDs were missing/not found when
loading a .blend file, Blender used to show one warning report for each
missing item, potentially covering the user's screen with a giant
unuable popup.
Now it will instead generate a single warning with amount of missing lib
files and linked IDs. Each missing item is still reported individually,
but only as `INFO`, so it will still show up in the console or Info editor.
Move `eUndoStepDir` to `BKE_undo_system.h` and use its values
everywhere.
Note that this also introduce the `STEP_INVALID` value in that enum.
Finally, kept the matching struct members in some lower-level readfile
code as an `int` to avoid having to include `BKE_undo_system.h` in a lot
of unrelated files.
Instead of submitting tons of tiny IO syscalls, we can speed things up
significantly by `mmap`ing the .blend file into virtual memory and directly
accessing it.
In my local testing, this speeds up loading the Dweebs file with all its
linked files from 19sec to 10sec (on Linux).
As far as I can see, this should be supported on Linux, OSX and BSD.
For Windows, a second code path uses `CreateFileMapping` and
`MapViewOfFile` to achieve the same result.
Reviewed By: mont29, brecht
Differential Revision: https://developer.blender.org/D8246
Asset metadata is what turns a regular data-block into an asset. It is a small
data-structure, but a key part of the technical design of the asset system.
The design foresees that asset data-blocks store an `ID.asset_data` pointer of
type `AssetMetaData`. This data **must not** have dependencies on other
data-blocks or data-block data, it must be an independent unit. That way we can
read asset-metadata from .blends without reading anything else from the file.
The Asset Browser will use this metadata (together with the data-block name,
preview and file path) to represent assets in the file list.
Includes:
* New `ID.asset_data` for asset metadata.
* Asset tags, description and custom properties.
* BKE code to manage asset meta-data and asset tags.
* Code to read asset data from files, without reading IDs.
* RNA for asset metadata (including tags)
Part of the first Asset Browser milestone. Check the #asset_browser_milestone_1
project milestone on developer.blender.org.
Differential Revision: https://developer.blender.org/D9716
Reviewed by: Bastien Montagne, Brecht Van Lommel
BLO_version_defaults_userpref_blend -> blo_do_versions_userdef
The name was misleading as it was declared along with
BLO_update_defaults_startup_blend making it seem these functions were
related.
In fact preference defaults don't need to be updated as is done for
startup.blend since an in-memory blend file isn't used.
Rename the function to match other versioning functions
called from readfile.c. Also add/update comments on these differences.
Relying on pointer addresses across different data-blocks is extremely
not recommended (and should be strictly forbidden ideally), in
particular in direct_link step of blend file reading.
- It assumes a specific order in reading of data, which is not ensured
in future, and is in any case a very bad, non explicit, hidden
dependency on behaviors of other parts of the codebase.
- It is intrinsically unsafe (as in, it makes writing bad code and making
mistakes easy, see e.g. fix in rB84b3f6e049b35f9).
- It makes advanced handling of data-blocks harder (thinking about
partial undo code e.g., even though in this specific case it was not
an issue as we do not re-read neither windowmanagers nor worspaces
during undo).
New code uses windows' `winid` instead as 'anchor' to find again proper
workspace hook in windows at read time.
As a bonus, it will also cleanup the list of relations from any invalid
ones (afaict it was never done previously).
Differential Revision: https://developer.blender.org/D9073
When loading large files that are more than a couple weeks old
(such that DNA has changed in that time), a significant amount of
time is spent in `DNA_struct_reconstruct`. This function takes a struct
in the old layout and creates a struct in the new layout from it.
This was slow because it was computing the diff between the struct
layouts every time a struct is updated. Now the steps for the struct
reconstruction is computed only once per struct. This information is
then used to actually reconstruct all structs that changed.
I measured about 10-20% speedup when loading Spring files.
E.g. `10.6s -> 8.7s` for `06_055_A.anim.blend` in BKE_blendfile_read`.
This percentage varies a lot based on the number of blocks that have
to be reconstructed and how much DNA has changed since they have
been written. In none of my tests was the new code slower than the
old code.
Reviewers: campbellbarton
Differential Revision: https://developer.blender.org/D8959
This patch tries to sanitize the types of our size parameters across our read
and write code, which is currently fairly inconsistent (using `int`, `uint`,
`size_t`...), by using `size_t` everywhere. Since in Blender file themselves
we can only store chunk of size `MAX_INT`, added some asserts to ensure that
as well.
See {T79561} for details.
Differential Revision: https://developer.blender.org/D8672
This replaces header include guards with `#pragma once`.
A couple of include guards are not removed yet (e.g. `__RNA_TYPES_H__`),
because they are used in other places.
This patch has been generated by P1561 followed by `make format`.
Differential Revision: https://developer.blender.org/D8466
Main goal here is to have better specificity using cache keys, to avoid
same memroy address being re-used messing up with cache pointers
restoration after undo had to re-read a data-block.
Once all caches handling are ported to this new system, it should fix
random issues like the one reported in T76989.
Part of D8183, refactoring how we preserve caches across undo steps in
readfile code.
The feature is hidden behind an experimental option, you'll have to
enable it in the preferences to try it.
This feature is not yet considered fully stable, crashes may happen, as
well as .blend file corruptions (very unlikely, but still possible).
In a nutshell, the ideas behind this code are to:
* Detect unchanged IDs across an undo step.
* Reuse as much as possible existing IDs memory, even when its content
did change.
* Re-use existing depsgraphs instead of building new ones from scratch.
* Store accumulated recalc flags, to avoid needless re-compute of things
that did not change, when the ID itself is detected as modified.
See T60695 and D6580 for more technical details.
This commit also provide a compatibility code that will convert old
materials using Additive or Multiply Blend mode to their node equivalent.
This conversion is only done on outputs that are enabled for eevee.
Shader nodes are now shared with Eevee, so makes more sense to have it in
the core and not be Cycles specific.
Fix T62415: issues with append/link of old Cycles settings.
Node versioning code was added before there was a mechanism to do versioning
after lib linking. Now integrate with that system and make it less of a strange
exception. Node versioning is now skipped on undo, like other versioning code.
Regression in 358e07f447 for ms-windows since off_t is an int32_t
even on 64bit systems causing files over 2gb not to load.
Poison off_t so this doesn't happen again.
Delay loading all DATA sections of the blend file until they're needed.
Loading all data-blocks caused high peak memory usage especially with
libraries - since a lot of data may exist which isn't used directly.
In one test (spring project: 10_010_A.anim.blend),
peaked at ~12.5gig, dropping back to ~2.5gig once loaded.
With this change peaks memory usage reaches ~2.7gig while loading.
Besides this there are some minor gains from not having to read data
from the file-system and we can skip an alloc + memcpy reading data
written with the same version of Blender.
