CacheFile writing code would not write generic ID data (call for it has
been missing since the initial commit, rB61050f75b13e).
While potentially affecting other areas (mostly CustomProperties/IDProperties),
this was a critical failure for liboverrides. Also added some workaround
code to allow opening broken files (though the override of the CacheFile
data-block will be lost).
Attempt to work around some full-corruption cases created at the studio.
Not clear how those were created, so not really fixing anything here,
just detecting and 'solving' as best as possible some high corruption of
local overrides.
This is good to have in general anyway, might help prevent further
corruption to happen too.
Add new function `blo_bhead_is_id_valid_type()` to correctly check the
blend file block type.
File block type codes have four bytes, and two of those are only in use
when these blocks contain ID datablocks (like `"OB\0\0"`). However,
there are other types defined in `BLO_blend_defs.h` that have four
bytes, like `TEST`, `ENDB`, etc.
The function `BKE_idtype_idcode_is_valid(short idcode)` was used to
check for ID datablocks while reading a blend file. This only takes a
2-byte parameter, and thus its result is invalid for the 4-byte codes.
For `TEST` blocks, it would actually consider it a `TE` block, which is
a valid identifier for a Texture. This caused the heap buffer overflow,
as the datablock is not a valid ID, and thus the bytes that were
expected to form an ID name actually encode something completely
different.
Reviewed By: mont29
Differential Revision: https://developer.blender.org/D10703
This adds support for creating a `BlendFile` (internally called `Main`),
which is limited to a context.
Temporary data can now be created which can then use
`.libraries.load()` the same as with `bpy.data`.
To prevent errors caused by mixing the temporary ID's with data in
`bpy.data` they are tagged as temporary so they can't be assigned
to properties, however they can be passed as arguments to functions.
Reviewed By: mont29, sybren
Ref D10612
The `MAX_LIBARRAY` define was an annoying doublon to the `INDEX_ID_MAX` enum value
now defined in `DNA_ID.h`, and it is no more useful.
And comments were somewhat outdated. Also added an explanation about
chosen order for the `INDEX_ID_<IDTYPE>` order.
When a lot of libraries or linked IDs were missing/not found when
loading a .blend file, Blender used to show one warning report for each
missing item, potentially covering the user's screen with a giant
unuable popup.
Now it will instead generate a single warning with amount of missing lib
files and linked IDs. Each missing item is still reported individually,
but only as `INFO`, so it will still show up in the console or Info editor.
`object_in_any_collection` used during linking/appending to check
whether an object is already instanced by at least one collection, was
not taking into account embedded master collections from scenes.
Essentially, `lib_link_all` would perform some post-processing over data
in given `bmain` that **may** fail when not all data from all libraries
has been properly loaded yet.
This happens when `lib_link_all` is called from `read_libraries`, where
the bmains are split by libraries.
Now those post-processing is put into its own utils function, which
asserts that it is only called on a merged bmain.
Bonus point, this will avoid re-runing those not-so-cheap operations
more than once on the same data.
Reproducible in r1442 of Sprite repository when opening
`pro/animation_test/rex/performance/rex_crowdcamping/rex_crowdcamping.lighting.blend`
NOTE: Not so sure why we have to call `lib_link_all` several times (once
for each library, and then once on the whole merged bmain, including
local IDs then). So that it can get called for libs while we still have
that specific .blend file handle around?
In any case, the overhead here is minimal since we do ensure a data-block is
never lib-linked more than once, so this is not a serious concern right now.
Differential Revision: https://developer.blender.org/D10307
Move `eUndoStepDir` to `BKE_undo_system.h` and use its values
everywhere.
Note that this also introduce the `STEP_INVALID` value in that enum.
Finally, kept the matching struct members in some lower-level readfile
code as an `int` to avoid having to include `BKE_undo_system.h` in a lot
of unrelated files.
Instead of submitting tons of tiny IO syscalls, we can speed things up
significantly by `mmap`ing the .blend file into virtual memory and directly
accessing it.
In my local testing, this speeds up loading the Dweebs file with all its
linked files from 19sec to 10sec (on Linux).
As far as I can see, this should be supported on Linux, OSX and BSD.
For Windows, a second code path uses `CreateFileMapping` and
`MapViewOfFile` to achieve the same result.
Reviewed By: mont29, brecht
Differential Revision: https://developer.blender.org/D8246
The Asset Browser will be a sub-editor of the File Browser. This prepares the
File Browser code for that.
**File-Lists**
* Support loading assets with metadata read from external files into the
file-list.
* New main based file-list type, for the "Current File" asset library.
* Refresh file-list when switching between browse modes or asset libraries.
* Support empty file-lists (asset library with no assets).
* Store file previews as icons, so scripts can reference them via icon-id. See
previous commit.
**Space Data**
* Introduce "browse mode" to differeniate between file and asset browsing.
* Add `FileAssetSelectParams` to `SpaceFile`, with `FileSelectParams` as base.
Makes sure data is separated between asset and file browsing when switching
between them. The active params can be obtained through
`ED_fileselect_get_active_params()`.
* `FileAssetSelectParams` stores the currently visible asset library ID.
* Introduce file history abstraction so file and asset browsing can keep a
separate history (previous and next directories).
**General**
* Option to only show asset data-blocks while file browsing (not exposed here).
* Add "active_file" context member, so scripts can get and display info about
the active file.
* Add "active_id" context member, so `ED_OT_lib_id_load_custom_preview` can set
a custom ID preview. (Only for "Current File" asset library)
* Expose some of `FileDirEntry` in RNA as (non-editable). That way scripts can
obtain name, preview icon and asset-data.
Part of the first Asset Browser milestone. Check the #asset_browser_milestone_1
project milestone on developer.blender.org.
Differential Revision: https://developer.blender.org/D9724
Reviewed by: Bastien Montagne
One of the core design aspects of the Asset Browser is that users can "mount"
custom asset libraries via the Preferences. Currently an asset library is just
a directory with one or more .blend files in it. We could easily support a
single .blend file as asset library as well (rather than a directory). It's
just disabled currently.
Note that in earlier designs, asset libraries were called repositories.
Idea is simple: In Preferences > File Paths, you can create custom libraries,
by setting a name and selecting a path. The name is ensured to be unique. If
the name or path are empty, the Asset Browser will not show it in the list of
available asset libraries.
The library path is not checked for validity, the Asset Browser will allow
selecting invalid libraries, but show a message instead of the file list, to
help the user understand what's going on.
Of course the actual Asset Browser UI is not part of this commit, it's in one
of the following ones.
{F9497950}
Part of the first Asset Browser milestone. Check the #asset_browser_milestone_1
project milestone on developer.blender.org.
Differential Revision: https://developer.blender.org/D9722
Reviewed by: Brecht Van Lommel, Hans Goudey
Asset metadata is what turns a regular data-block into an asset. It is a small
data-structure, but a key part of the technical design of the asset system.
The design foresees that asset data-blocks store an `ID.asset_data` pointer of
type `AssetMetaData`. This data **must not** have dependencies on other
data-blocks or data-block data, it must be an independent unit. That way we can
read asset-metadata from .blends without reading anything else from the file.
The Asset Browser will use this metadata (together with the data-block name,
preview and file path) to represent assets in the file list.
Includes:
* New `ID.asset_data` for asset metadata.
* Asset tags, description and custom properties.
* BKE code to manage asset meta-data and asset tags.
* Code to read asset data from files, without reading IDs.
* RNA for asset metadata (including tags)
Part of the first Asset Browser milestone. Check the #asset_browser_milestone_1
project milestone on developer.blender.org.
Differential Revision: https://developer.blender.org/D9716
Reviewed by: Bastien Montagne, Brecht Van Lommel
Remove DNA headers, using forward declarations where possible.
Also removed duplicate header, header including it's self
and unnecessary inclusion of libc system headers from BKE header.
This way Outliner internal data stays internal, non-Outliner code will not be
able to access and mess with this. Further it allows us to use the real type
(rather than `void *`), change the type to a C++ container if needed and
slightly reduces the size for every Outliner stored in files.
Slightly changed how we set the `SO_TREESTORE_REBUILD` for this, but it should
effectively behave the same way as before.
