This should not be allowed in general, added some initial call to check
when user is allowed to delete a data to search for mandatory override
usages...
Proxy conversion is a fairly particular case of liboverride creation, in
which remapping all local usages of linked data probably makes more
sense, rather than only doing so whitin the overridden 'group' of IDs.
This commit also add an experimental userPreferences to prevent proxies
conversions on file load, and reporting for amount of coverted proxies
(and possible issues).
Note that potentially linked proxies from other libraries are not
hamdled here (this feature seems to be broken anyway in master
currently?).
This is used to find a valid collection in which to instantiate stray
objects and collections.
In some cases there will be no such active view layer, in which case we
can consider using the Scene's master collections children hierarchy
instead to find a valid instantiated parent collection for those stray
data.
LibOverride of scenes is not really supported currently, there are many
issues with it.
Will disable most user-accessible ways to create such overrides in a
following commit.
This makes the internal naming consistent with the public API. And also gives
us a visibility_flag rather than restrictflag that can be extended with more
flags.
Recent own rBabf3ce811f6e prevented any LayerCollection update during
the whole liboverride resync process, for both performances and feature
reasons.
However that means that the various runtime caches like the Base GHash
are not cleared anymore during ID remapping process, so we need to call
`BKE_main_collection_sync_remap` instead of `BKE_main_collection_sync`
when we finally are ready for this update.
Reported by @eyecandy (Andy Goralczyk) from Blender studio, thanks!
This is an easy & safe, yet not-so-nice way to address the
LayerCollections vs. Collections hierarchy resync problem.
Currently this resync is enforced everytime something changes in the
Collections hierarchy, which is extremely inneficient, and can even
produce 'loss' of LayerCollection data during complex Collection
processes.
Current example is during Library Overrides resync process. New code:
* Makes resync significantly faster (between 10 and 15%).
* Fixes 'disappearing' layer collections settings on sub-collections'
layers.
NOTE: This is not a proper fix for the underlying issue. However,
implementing and testing the 'lazy update' solution as proposed by
{T73411} requires a significant amount of time (especially in testing
and tracking all places where code would need to ensure LayerCollections
are up-to-date), which is not possible currently.
Differential Revision: https://developer.blender.org/D11889
Weird 'embedded for overrides' flag of embedded IDs (including ShapeKeys
in override context) was not properly cleaned up when making an override
fully local.
Reported by studio, thanks.
@jbakker should be backported to 2.93LTS if possible.
This change will ensure at least one 'local' collection can host the new
'local' override of all objects (indirectly) overridden by this
operation, such that no new override of object ends up in master
collection (which can become extremely messy in production files).
In practice, it means often at least one of the linked collection owning
those objects also has to be overridden.
NOTE: This only affect cases where root overridden linked object has
some dependencies outside of its own root linked collection. While this
situation should be avoided, it cannot always be, so we try to support
it as best as we can.
This reverts commit rB3a48147b8ab92, and fixes the issues with linking
etc.
Change compared to previous buggy commit (rBf8d219dfd4c31) is that
new `BlendFileReadReports` reports are now passed to the lowest level
function generating the `FileData` (`filedata_new()`), which ensures
(and asserts) that all code using it does have a valid non-NULL pointer
to a `BlendFileReadReport` data.
Sorry for the noise, it's always when you think a change is trivial and
do not test it well enough that you end up doing those kind of
mistakes...
Add direct user feedback (as a warning report) to user when recursive
resync of overrides was needed.
And some timing (as CLOG logs) about main readfile process steps.
This is essentially adding a new BlendFileReadReport structure that wraps
BKE_reports list, and adds some extra info (some timing, some info about
overrides and (recursive) resync, etc.).
After looking into task isolation issues with Sergey, we couldn't find the
reason behind the deadlocks that we are getting in T87938 and a Sprite Fright
file involving motion blur renders.
There is no apparent place where we adding or waiting on tasks in a task group
from different isolation regions, which is what is known to cause problems. Yet
it still hangs. Either we do not understand some limitation of TBB isolation,
or there is a bug in TBB, but we could not figure it out.
Instead the idea is to use isolation only where we know we need it: when
holding a mutex lock and then doing some multithreaded operation within that
locked region. Three places where we do this now:
* Generated images
* Cached BVH tree building
* OpenVDB lazy grid loading
Compared to the more automatic approach previously used, there is the downside
that it is easy to miss places where we need isolation. Yet doing it more
automatically is also causing unexpected issue and bugs that we found no
solution for, so this seems better.
Patch implemented by Sergey and me.
Differential Revision: https://developer.blender.org/D11603
Under some circumstances using task isolation can cause deadlocks.
Previously, our task pool implementation would run all tasks in an
isolated region. Now using task isolation is optional and can be
turned on/off for individual task pools.
Task pools that spawn new tasks recursively should never enable
task isolation. There is a new check that finds these cases at runtime.
Right now this check is disabled, so that this commit is a pure refactor.
It will be enabled in an upcoming commit.
This fixes T88598.
Differential Revision: https://developer.blender.org/D11415
In some cases e.g. only objects would actually need resync, so
collections on the override character would not be resynced, and if some
objects were sharing relationships with others those could be
lost/destroyed.
Ensure 'virtual' linked override IDs generated by the recursive resync
process are tagged as indirectly linked data.
This is needed to avoid the 'missing data' messages on those virtual
data-blocks after saving and reloading.
This commit fixes two different issues:
* In some cases, when an object was added to a sub-collection and used
into a different subcollection, and the root common collection would
not need to be resynced, it would end up creating multiple overrides
of the new object. This was affecting both normal and recursive
resync.
* In recurisve resync case, the barrier code to define what is part or
not of a override group/hierarchy was wrong.
Note that the current solution for the first issue is sub-optimal (it
goes back to the root of the override hierarchy and resync the whole
thing), a better solution is TODO for now.
We need to re-evaluate what needs to be resynced after each step of
processing overrides from a given 'indirect level' of libraries.
Otherwise, recusrive overrides (overrides of linked overrides) won't
work.
Note that this should not change too much in practice currently, since
there are other issues with recursive overrides yet.
Also, checks (CLOG errors) added show that some ID (node trees) seem to
be detected as needing resynced even after beig just resynced, this
needs further investigation still. Could be though that it is due to
limit currently set on nodetrees, those are always complicated
snowflakes to deal with...
This is not supposed to happen, but better be safe than sorry, and
assume it is beyond unlikely that someone would use chains of over 10k
linked libraries.
Very stupid mistake in libraries indirect-level building code, was not
skipping 'loop-back' ID pointers.
Note that we also need some level of checks for the case where there
would be an actual dependency loop between libraries, this is not
supposed to be possible, but better be safe than sorry. Will add in next
commit.
Recursive resync means also resyncing overrides that are linked from
other library files into current working file.
Note that this allows to get 'working' files even when their
dependencies are out of sync. However, since linked data is never
written/saved, this has to be re-done every time the working file is
loaded, until said dependencies are updated properly.
NOTE: This is still missing the 'report' side of things, which is part
of a larger task to enhance reports regarding both linking, and
liboverrides (see T88393).
----------
Technical notes:
Implementing this proved to be slightly more challenging than expected,
mainly because one of the key aspects of the feature was never done in
Blender before: manipulating, re-creating linked data.
This ended up moving the whole resync code to use temp IDs out of bmain,
which is better in the long run anyway (and more aligned with what we
generally want to do when manipulating temp ID data). It should also
give a marginal improvement in performances for regular resync.
This commit also had to carefully 'sort' libraries by level of indirect
usage, as we want to resync first the libraries that are the least directly
used, i.e. libraries that are most used by other libraries.
this is a followup to rB2bd85d9cc623, we cannot forcefully delete
obsolete overrides of object data (meshes etc.), as this implies also
deleting their user object, which might still be a perfectly valid
override, albeit in conflict regarding its obdata ID pointer...
