Overrides that are not created as part of an override hierarchy should
not be handled through (auto)resync at all. users are responsible to
hanlde those updates if they need it.
This is achieved by flagging overrides created outside of a hierarchical
process accordingly, and skipping them during resync process.
Goals of this refactor:
* More unified approach to updating everything that needs to be updated
after a change in a node tree.
* The updates should happen in the correct order and quadratic or worse
algorithms should be avoided.
* Improve detection of changes to the output to avoid tagging the depsgraph
when it's not necessary.
* Move towards a more declarative style of defining nodes by having a
more centralized update procedure.
The refactor consists of two main parts:
* Node tree tagging and update refactor.
* Generally, when changes are done to a node tree, it is tagged dirty
until a global update function is called that updates everything in
the correct order.
* The tagging is more fine-grained compared to before, to allow for more
precise depsgraph update tagging.
* Depsgraph changes.
* The shading specific depsgraph node for node trees as been removed.
* Instead, there is a new `NTREE_OUTPUT` depsgrap node, which is only
tagged when the output of the node tree changed (e.g. the Group Output
or Material Output node).
* The copy-on-write relation from node trees to the data block they are
embedded in is now non-flushing. This avoids e.g. triggering a material
update after the shader node tree changed in unrelated ways. Instead
the material has a flushing relation to the new `NTREE_OUTPUT` node now.
* The depsgraph no longer reports data block changes through to cycles
through `Depsgraph.updates` when only the node tree changed in ways
that do not affect the output.
Avoiding unnecessary updates seems to work well for geometry nodes and cycles.
The situation is a bit worse when there are drivers on the node tree, but that
could potentially be improved separately in the future.
Avoiding updates in eevee and the compositor is more tricky, but also less urgent.
* Eevee updates are triggered by calling `DRW_notify_view_update` in
`ED_render_view3d_update` indirectly from `DEG_editors_update`.
* Compositor updates are triggered by `ED_node_composite_job` in `node_area_refresh`.
This is triggered by calling `ED_area_tag_refresh` in `node_area_listener`.
Removing updates always has the risk of breaking some dependency that no
one was aware of. It's not unlikely that this will happen here as well. Adding
back missing updates should be quite a bit easier than getting rid of
unnecessary updates though.
Differential Revision: https://developer.blender.org/D13246
Basically, this fixes disappearing previews when editing asset metadata
or performing undo/redo actions.
The preview generation in a background job will eventually modify ID
data, but the undo push was done prior to that. So obviously, an undo
then would mean the preview is lost.
This patch makes it so undo/redo will regenerate the preview, if the preview
rendering was invoked but not finished in the undone/redone state.
The preview flag PRV_UNFINISHED wasn't entirely what we needed. So I had to
change it to a slightly different flag, with different semantics.
Original implementation was a quick prototype which should have never
landed as-is in master. It had very limiting constraints and did not
allow for any real further development.
This commit fixes the internal implementation to make more sensible,
maintainable and evolutive.
NOTE: This commit introduces another forward-incompatibility in the
Blender file format: Files saved after this commit won't open properly
in older versions of blender regarding local inserted constraints or
modifiers into overrides of linked data.
NOTE: Technical details: The 'anchor' item name/index is now stored in
`subitem_reference_` members, and the actual 'source' item name/index is
stored in `subitem_local_` members of the override property operation
data.
Previously, only the `subitem_local_` members were used, storing the
anchor item name/index, and assuming the 'source' item was always the
next in the list.
Milestone I of T82160.
Maniphest Tasks: T82160
Differential Revision: https://developer.blender.org/D13282
This commit adds to ID struct a new optional 'weak reference' to a
linked ID (in the form of a blend file library path and full ID name).
This can then be used on next append to try to find a matching local ID
instead of re-making the linked data local again.
Ref. T90545
NOTE: ID re-use will be disabled for regular append for the time being
(3.0 release), and only used for assets. Therefore, this commit should
not change anything user-wise.
Differential Revision: https://developer.blender.org/D12545
The storage of IDProperty UI data (min, max, default value, etc) is
quite complicated. For every property, retrieving a single one of these
values involves three string lookups. First for the "_RNA_UI" group
property, then another for a group with the property's name, then for
the data value name. Not only is this inefficient, it's hard to reason
about, unintuitive, and not at all self-explanatory.
This commit replaces that system with a UI data struct directly in the
IDProperty. If it's not used, the only cost is of a NULL pointer. Beyond
storing the description, name, and RNA subtype, derived structs are used
to store type specific UI data like min and max.
Note that this means that addons using (abusing) the `_RNA_UI` custom
property will have to be changed. A few places in the addons repository
will be changed after this commit with D9919.
**Before**
Before, first the _RNA_UI subgroup is retrieved the _RNA_UI group,
then the subgroup for the original property, then specific UI data
is accessed like any other IDProperty.
```
prop = rna_idprop_ui_prop_get(idproperties_owner, "prop_name", create=True)
prop["min"] = 1.0
```
**After**
After, the `id_properties_ui` function for RNA structs returns a python
object specifically for managing an IDProperty's UI data.
```
ui_data = idproperties_owner.id_properties_ui("prop_name")
ui_data.update(min=1.0)
```
In addition to `update`, there are now other functions:
- `as_dict`: Returns a dictionary of the property's UI data.
- `clear`: Removes the property's UI data.
- `update_from`: Copy UI data between properties,
even if they have different owners.
Differential Revision: https://developer.blender.org/D9697
This was really missing there (some COW tags behavior was also
documented in some code using them, like in `sound.c`, but not in their
definition).
Ref. T88555.
This is not supported currently,doing so through RNA API remains
possible, but from regular UI operations it should not be doable
anymore.
Ref. T90459.
During a mesh transformation in edit mode (Move, Rotate...), only part of
the batch cache needs to be updated.
This commit allows only update only the drawn batches seen in
`BKE_object_data_eval_batch_cache_deform_tag` if the new
`ID_RECALC_GEOMETRY_DEFORM` flag is used.
This new flag is used in the transforms operation for edit-mesh and
results in 1.6x overall speedup in heavy subdiv cube.
Differential Revision: https://developer.blender.org/D11599
Avoid computationally expensive copying operations
when only some settings have been modified.
This is done by adding support for updating parameters
without tagging for copy-on-write.
Currently only mesh data blocks are supported,
other data-blocks can be added individually.
This prepares for changing values such as edit-mesh auto-smooth angle
in edit-mode without duplicating all mesh-data.
The benefit will only be seen when the user interface no longer tags
all ID's for copy on write updates.
ID_RECALC_GEOMETRY_ALL_MODES has been added to support situations
where non edit-mode geometry is modified in edit-mode.
While this isn't something user are likely to do,
Python scripts may change the underlying mesh.
Reviewed By: sergey
Ref D11377
Recursive resync means also resyncing overrides that are linked from
other library files into current working file.
Note that this allows to get 'working' files even when their
dependencies are out of sync. However, since linked data is never
written/saved, this has to be re-done every time the working file is
loaded, until said dependencies are updated properly.
NOTE: This is still missing the 'report' side of things, which is part
of a larger task to enhance reports regarding both linking, and
liboverrides (see T88393).
----------
Technical notes:
Implementing this proved to be slightly more challenging than expected,
mainly because one of the key aspects of the feature was never done in
Blender before: manipulating, re-creating linked data.
This ended up moving the whole resync code to use temp IDs out of bmain,
which is better in the long run anyway (and more aligned with what we
generally want to do when manipulating temp ID data). It should also
give a marginal improvement in performances for regular resync.
This commit also had to carefully 'sort' libraries by level of indirect
usage, as we want to resync first the libraries that are the least directly
used, i.e. libraries that are most used by other libraries.
Resolve ownership ambiguity with shared physics pointers.
Previously, LIB_ID_CREATE_NO_MAIN allowed pointer sharing with
the source ID so physics caches can be shared between original and
evaluated data: (Object.soft.shared & Object.rigidbody_object.shared).
This only worked properly for LIB_TAG_COPIED_ON_WRITE ID's,
as LIB_TAG_NO_MAIN can be used in situations where the original ID's
lifetime limited by it's original data.
This commit adds `LIB_ID_COPY_SET_COPIED_ON_WRITE` so ID's only share
memory with original data for ID's evaluated in the depsgraph.
For all other uses, a full copy of physics data is made.
Ref D11228#287094
This fixes T87666 and T83252.
The boolean modifier and geometry nodes can depend on the geometry
of an entire collection. Before, the modifiers had to manually create relations
to all the objects in the collection. This worked for the most part, but was
cumbersome and did not solve all issues. For example, the modifiers were not
properly updated when objects were added/removed from the referenced collection.
This commit introduces the concept of "collection geometry" in the depsgraph.
The geometry of a collection depends on the transforms and geometry of all
the objects in it. The boolean modifier and geometry nodes can now just depend
on the collection geometry instead of creating all the dependencies themselves.
Differential Revision: https://developer.blender.org/D11053
This flag is set for liboverride IDs that are detected as no longer
needed by resync process, while having been user-edited, so
auto-handling code cannot silently delete them.
Exposing those to users will be part of the new incoming Override
Outliner view.
Part of D10855.
Move Palettes type with other weird, not-really-data ID types (like
PaintCurves and Brushes), higher in the process.
Those preset-like types may use a lot of other ID types, but should only
be used by UI-related (and Scene, for tool settings) types.
Move Brushes type with other weird, not-really-data ID types (like
Palette and Brushes), higher in the process.
Those preset-like types may use a lot of other ID types, but should only
be used by UI-related (and Scene, for tool settings) types.
Move PaintCurve type with other weird, not-really-data ID types (like
Palette and Brushes), higher in the process.
Those preset-like types may use a lot of other ID types, but should only
be used by UI-related (and Scene, for tool settings) types.
Move `OB_GR` processing just after scenes, and before objects.
This is much more sensible in general, and fixes glitches in auto-resync
process of library overrides in particular.
We can fairly easily detect some resync-needed cases when applying the
overrides operations on a Pointer RNA property.
This should cover all cases where an existing override's ID pointer is
changed in its linked data.
We still have to add code to detect when a not-yet-overridden linked ID
needs to become overridden (because its relations to other data-blocks
changed in a way that requires it).
Part of T83811 & D10649.
This adds support for creating a `BlendFile` (internally called `Main`),
which is limited to a context.
Temporary data can now be created which can then use
`.libraries.load()` the same as with `bpy.data`.
To prevent errors caused by mixing the temporary ID's with data in
`bpy.data` they are tagged as temporary so they can't be assigned
to properties, however they can be passed as arguments to functions.
Reviewed By: mont29, sybren
Ref D10612
The `MAX_LIBARRAY` define was an annoying doublon to the `INDEX_ID_MAX` enum value
now defined in `DNA_ID.h`, and it is no more useful.
And comments were somewhat outdated. Also added an explanation about
chosen order for the `INDEX_ID_<IDTYPE>` order.
This potentially could fix some missed cases in dependency tagging (when
dealing with overrides hierarchies), since relying on tag in ID itself
is not a good idea to check whether an ID has been propcessed or not
(exterior code may have forced that tag on some IDs e.g., which would
prevent them from ever being processed properly).
This avoids adding DNA_ID.h into other headers, recently changed in
cfa48c84d0
Note that other enums could be moved too, this is a smaller change
to avoid indirectly including DNA_ID.h in many places.
Building IDs which are not covered by copy-on-write process was not
implemented, which was causing parameters block not present, and, hence
causing crashes in areas which expected parameters to present.
First part of this change is related on making it so Copy-on-Write is
optional for ID nodes in the dependency graph.
Second part is related on using a generic builder for all ID types
which were not covered by Copy-on-Write before.
The final part is related on making it so build_id() is properly
handling ParticleSettings and Grease Pencil Data. Before they were not
covered there at all, and they need special handling because they do
have own build functions.
Not sure it worth trying to split those parts, as they are related to
each other and are not really possible to be tested standalone. Open
for a second opinion though.
Possible nut-tightening is to re-organize build_id() function so
that every branch does return and have an assert at the end, so that
missing ID type in the switch statement is easier to spot even when
using compilers which do not report missing switch cases.
As for question "why not use default" the answer is: to make it more
explicit and clear what is a decision when adding new ID types. We do
not want to quietly fall-back to a non-copy-on-write case for a newly
added ID types.
Differential Revision: https://developer.blender.org/D10075
Revert "Fix T83411: Crash when using a workspace/layout data path in a driver"
The fix for the crash exposed design violation in the viewport shading updates,
which is for some reason relying on dependency graph tag of interface data.
The viewport module did not respond to the issue in 2 weeks, and the architect
considered missing update for multiple users a more serious issue than a crash
in a very specific case.
This reverts commit 0f95f51361.
First step towards a better handling of relationships between IDs in
override context, especially when a resync is needed.
First, introduce a new flag to override operations,
`IDOVERRIDE_LIBRARY_FLAG_IDPOINTER_MATCH_REFERENCE`, for ID pointers.
It keeps track of whether an RNA ID pointer has been kept to its
'natural overriden ID' (in override hierarchy context), or has actually
been re-assigned to some other data-block.
Second, refactor how we deal with relationships between IDs in override
hierarchy code, especially in resync case. This will fixe several cases
listed in T83811, especially the case where an ID pointer to an existing
override needs to be updated to a new one due to a matching change in
linked data.
