Initial error caused by 1efc94bb2f and fixed in 6a7917162c. However
the empty index files (empty as in, they contain the version number but
no assets) will have to be fixed somehow, since otherwise assets don't
show up at all for people who saved asset files in a broken version.
Delete empty index files if the modification timestamp indicates a time
when the bug was present (plus a day before and after, to address
possible time zone differences). This will basically make Blender skip
the optimization for .blend files without assets for one load, but even
then the index should still produce faster results than a completely
non-index read.
This can be removed after a while, it's just a (much needed) fix for
people who were using alpha/beta builds.
Differential Revision: https://developer.blender.org/D16678
Reviewed by: Jeroen Bakker
Steps to reproduce were:
- Open a .blend file that is located inside of an asset library and
contains assets.
- Save and close the file.
- Open a new file (Ctrl+N -> General).
- Open asset browser and load the asset library from above.
- If the assets from the file above still show up, press refresh button.
- -> Assets from the file above don't appear.
Likely fixes the underlying issue for T102610. A followup will be needed
to correct the empty asset index files written because of this bug.
We're in the process of moving responsibilities from the file/asset
browser backend to the asset system. 1efc94bb2f introduces a new
representation for asset, which would own the asset metadata now instead
of the file data.
Since the file-list code still does the loading of asset libraries,
ownership of the asset metadata has to be transferred to the asset
system. However, the asset indexing still requires it to be available,
so it can update the index with latest data. So transfer the ownership,
but still keep a non-owning pointer set.
Differential Revision: https://developer.blender.org/D16665
Reviewed by: Bastien Montagne
Adds a new `source/blender/asset_system` directory and moves asset
related files from BKE to it. More asset related code can follow
(e.g. asset indexing, ED_assetlist stuff) but needs further work to
untangle it. I also kept `BKE_asset.h` and `asset.cc` as is, since they
deal with asset DNA data mostly, thus make sense in BKE.
Motivation:
- Makes the asset system design more present (term wasn't even used in
code before).
- An `asset_system` directory is quite descriptive (trivial to identify
core asset system features) and makes it easy to find asset code.
- Asset system is mostly runtime data, with little relation to other
`Main`/BKE/DNA types.
- There's a lot of stuff in BKE already. It shouldn't be just a dump for
all stuff that seems core enough.
- Being its own directly helps us be more mindful about encapsulating
the module well, and avoiding dependencies on other modules.
- We can be more free with splitting files here than in BKE.
- In future there might be an asset system BPY module, which would then
map quite nicely to the `asset_system` directory.
Checked with some other core devs, consensus seems that this makes
sense.
These functions that retrieve strings from assets return stringrefs.
Storing them as std::strings is unnecessary and relies on binding to
the const references.
Use a shorter/simpler license convention, stops the header taking so
much space.
Follow the SPDX license specification: https://spdx.org/licenses
- C/C++/objc/objc++
- Python
- Shell Scripts
- CMake, GNUmakefile
While most of the source tree has been included
- `./extern/` was left out.
- `./intern/cycles` & `./intern/atomic` are also excluded because they
use different header conventions.
doc/license/SPDX-license-identifiers.txt has been added to list SPDX all
used identifiers.
See P2788 for the script that automated these edits.
Reviewed By: brecht, mont29, sergey
Ref D14069
Object/collection asset workflow would need the bounding box for snapping.
The bounding box is stored using ID properties in the scene. Currently ID properties
aren't stored in the asset index, what would break object snapping. For this reason
Asset Indexing is turned off in mater. This patch will introduce the indexing of ID
properties what will allow the indexing to be turned on again.
## Data Mapping ##
For data mapping we store the internal structure of IDProperty to the indexer (including meta-data) to be able to deserialize it back.
```
[
{
"name": ..,
"value": ..,
"type": ..,
/* `subtype` and `length` are only available for IDP_ARRAYs. */
"subtype": ..,
},
]
```
| **DNA** | **Serialize type** | **Note** |
| IDProperty.name | StringValue| |
| IDProperty.type | StringValue| "IDP_STRING", "IDP_INT", "IDP_FLOAT", "IDP_ARRAY", "IDP_GROUP", "IDP_DOUBLE"|
| IDProperty.subtype | StringValue| "IDP_INT", "IDP_FLOAT", "IDP_GROUP", "IDP_DOUBLE" |
| IDProperty.value | StringValue | When type is IDP_STRING |
| IDProperty.value | IntValue | When type is IDP_INT |
| IDProperty.value | DoubleValue | When type is IDP_FLOAT/IDP_DOUBLE |
| IDProperty.value | ArrayValue | When type is IDP_GROUP. Recursively uses the same structure as described in this section. |
| IDProperty.value | ArrayValue | When type is IDP_ARRAY. Each element holds a single element as described in this section. |
NOTE: IDP_ID and IDP_IDARRAY aren't supported. The entry will not be added.
Example
```
[
{
"name": "MyIntValue,
"type": "IDP_INT",
"value": 6,
},
{
"name": "myComplexArray",
"type": "IDP_ARRAY",
"subtype": "IDP_GROUP",
"value": [
[
{
"name": ..
....
}
]
]
}
]
```
## Considered alternatives ##
- Add conversion functions inside `asset_indexer`; makes generic code part of a specific solution.
- Add conversion functions inside `BLI_serialize`; would add data transformation responsibilities inside a unit that is currently only responsible for formatting.
- Use direct mapping between IDP properties and Values; leads to missing information and edge cases (empty primitive arrays) that could not be de-serialized.
Reviewed By: Severin, mont29, HooglyBoogly
Maniphest Tasks: T92306
Differential Revision: https://developer.blender.org/D12990
Using the `MEM_*` API from C++ code was a bit annoying:
* When converting C to C++ code, one often has to add a type cast on
returned `void *`. That leads to having the same type name three times
in the same line. This patch reduces the amount to two and removes the
`sizeof(...)` from the line.
* The existing alternative of using `OBJECT_GUARDED_NEW` looks a out
of place compared to other allocation methods. Sometimes
`MEM_CXX_CLASS_ALLOC_FUNCS` can be used when structs are defined
in C++ code. It doesn't look great but it's definitely better. The downside
is that it makes the name of the allocation less useful. That's because
the same name is used for all allocations of a type, independend of
where it is allocated.
This patch introduces three new functions: `MEM_new`, `MEM_cnew` and
`MEM_delete`. These cover the majority of use cases (array allocation is
not covered).
The `OBJECT_GUARDED_*` macros are removed because they are not
needed anymore.
Differential Revision: https://developer.blender.org/D13502
Use fixed-length string to convert `char[2]` to `std::string`. Otherwise
`strlen()` is called, which is problematic as the `char[2]` is not
zero-terminated.
Asset library indexing would store indexes of asset files to speed up
asset library browsing.
* Indexes are read when they are up to date
** Index should exist
** Index last modify data should be later than the file it indexes
** Index version should match
* The index of a file containing no assets can be load without opening
the index file. The size of the file should be below a 32 bytes.
* Indexes are stored on a persistent cache folder.
* Unused index files are automatically removed.
The structure of the index files contains all data needed for browsing assets:
```
{
"version": <file version number>,
"entries": [{
"name": "<asset name>",
"catalog_id": "<catalog_id>",
"catalog_name": "<catalog_name>",
"description": "<description>",
"author": "<author>",
"tags": ["<tag>"]
}]
}
```
Reviewed By: sybren, Severin
Maniphest Tasks: T91406
Differential Revision: https://developer.blender.org/D12693
