The key used by find_operation consists of the operation code,
node name, and an index for array property drivers. All three
elements are required to unambiguously look up a node.
The idea of those flags is to avoid evaluation of operations
which are not needed to bring visible objects to an up to date
state.
Previously, dependency graph attempted to do combine those
into an ID level flag. In practice it proved to be rather
tricky, since there could be dependency cycles on ID level
which will not exist on component level.
This allows to force duplicated object to be fully evaluated for
display/draw.
Previously only transform component of duplicated object was
forced to become evaluated.
- When returning the number of items in a collection use BLI_*_len()
- Keep _size() for size in bytes.
- Keep _count() for data structures that don't store length
(hint this isn't a simple getter).
See P611 to apply instead of manually resolving conflicts.
The idea is to de-duplicate logic in DEG_id_tag_update() and flushing where we
need to translate depsgraph tag or component type to ID level recalc flag.
Currently unused, but is required for Blender 2.8.
This is crucial bit since batch cache is stored in the evaluated object,
meaning we can't tag it's hatch cache dirty from the notifier system.
Not easily at least. Better to leave this job to depsgraph, it knows
all the copies of data.
The is following: split copy on write update for node trees, and if we are only
tagging for uniform buffer update we skip whole datablock copy and only invoke
copy default_values form original nodetree to a copied one.
Thing which i'm not sure is: whether we need to use different branches in graph
itself to control such a conditional behavior, or whether we need to store tag
somewhere in the dependency graph. There are obviously cons and pros in both
approaches, and need to think about this. Maybe with more examples it becomes
more obvious which way is better.
This only fixes manual tweaks for now, animation support is coming.
< Dependency graph Copy-on-Write >
--------------------------------
\ ^__^
\ (oo)\_______
(__)\ )\/\
||----w |
|| ||
This is an initial commit of Copy-on-write support added to dependency graph.
Main priority for now: get playback (Alt-A) and all operators (selection,
transform etc) to work with the new concept of clear separation between
evaluated data coming from dependency graph and original data coming from
.blend file (and stored in bmain).
= How does this work? =
The idea is to support Copy-on-Write on the ID level. This means, we duplicate
the whole ID before we cann it's evaluaiton function. This is currently done
in the following way:
- At the depsgraph construction time we create "shallow" copy of the ID
datablock, just so we know it's pointer in memory and can use for function
bindings.
- At the evaluaiton time, the copy of ID get's "expanded" (needs a better
name internally, so it does not conflict with expanding datablocks during
library linking), which means the content of the datablock is being
copied over and all IDs are getting remapped to the copied ones.
Currently we do the whole copy, in the future we will support some tricks
here to prevent duplicating geometry arrays (verts, edges, loops, faces
and polys) when we don't need that.
- Evaluation functions are operating on copied datablocks and never touching
original datablock.
- There are some cases when we need to know non-ID pointers for function
bindings. This mainly applies to scene collections and armatures. The
idea of dealing with this is to "expand" copy-on-write datablock at
the dependency graph build time. This might introduce some slowdown to the
dependency graph construction time, but allows us to have minimal changes
in the code and avoid any hash look-up from evaluation function (one of
the ideas to avoid using pointers as function bindings is to pass name
of layer or a bone to the evaluation function and look up actual data based
on that name).
Currently there is a special function in depsgraph which does such a
synchronization, in the future we might want to make it more generic.
At some point we need to synchronize copy-on-write version of datablock with
the original version. This happens, i.e., when we change active object or
change selection. We don't want any actual evaluation of update flush happening
for such thins, so now we have a special update tag:
DEG_id_tag_update((id, DEG_TAG_COPY_ON_WRITE)
- For the render engines we now have special call for the dependency graph to
give evaluated datablock for the given original one. This isn't fully ideal
but allows to have Cycles viewport render.
This is definitely a subject for further investigation / improvement.
This call will tag copy-on-write component tagged for update without causing
updates to be flushed to any other objects, causing chain reaction of updates.
This tag is handy when selection in the scene changes.
This basically summarizes ideas underneath this commit. The code should be
reasonably documented.
Here is a demo of dependency graph with all copy-on-write stuff in it:
https://developer.blender.org/F635468
= What to expect to (not) work? =
- Only meshes are properly-ish aware of copy-on-write currently, Non-mesh
geometry will probably crash or will not work at all.
- Armatures will need similar depsgraph built-time expansion of the copied
datablock.
- There are some extra tags / relations added, to keep things demo-able but
which are slowing things down for evaluation.
- Edit mode works for until click selection is used (due to the selection
code using EditDerivedMesh created ad-hoc).
- Lots of tools will lack tagging synchronization of copied datablock for
sync with original ID.
= How to move forward? =
There is some tedious work related on going over all the tools, checking
whether they need to work with original or final evaluated object and make
the required changes.
Additionally, there need synchronization tag done in fair amount of tools
and operators as well. For example, currently it's not possible to change
render engine without re-opening the file or forcing dependency graph for
re-build via python console.
There is also now some thoughts required about copying evaluated properties
between objects or from collection to a new object. Perhaps easiest way
would be to move base flag flush to Object ID node and tag new objects for
update instead of doing manual copy.
here is some WIP patch which moves such evaluaiton / flush:
https://developer.blender.org/F635479
Lots of TODOs in the code, with possible optimization.
= How to test? =
This is a feature under heavy development, so obviously it is disabled by
default. The only reason it goes to 2.8 branch is to avoid possible merge
hell.
In order to enable this feature use WITH_DEPSGRAPH_COPY_ON_WRITE CMake
configuration option.
Avoids using copy-constructor invoked every time we pass function
to the builder functions.
Should lower number of CPU ticks spent during DEG construction.
These bits became obsolete with the new layer system, so we can
simplify some code around them or avoid existing workarounds which
were trying to keep things working for them.
There are still work needed to be done for on_visible_change to
avoid unnecessary updates, but that can also happen later.
This moves selectability/visibility flag flush from some hardcoded
places in the code to depsgraph. This way it is possible to simply
tag depsgraph to update those flags and rest it'll do on its own.
Using depsgraph for such flush is an overkill: those flags are fully
static and can not be animated, so it doesn't really make sense to
hook only those to depsgraph.
However, in the future we will have overrides on collections, which
ideally would need to be animatable and drivable and easiest way
to support this is to do this on depsgraph level, so it ensures
proper order of evaluation for animation and drivers. And it seems
logical to do both overrides and flags flush from depsgraph from
this point of view.
This commit also includes the evaluation of IDProperty for collections,
which basically are just another form of override. So once we implement
the other kind of overrides the flushing and collection evaluation won't
change.
Patch by Sergey Sharybin and Dalai Felinto
This is more proper way to go:
- Avoids re-compilation of all dependent files when implementation changes
without changed API,
- Linker should have much simpler time now de-duplicating and getting rid
of redundant implementations.
The idea here is to address issue that name on it's own is not
always unique: for example, when adding driver operations the
name used for nodes is the RNA path (and multiple drivers can
write to different array indices of the path). Basically, now
it's possible to pass extra integer value to distinguish
operations in such cases.
So now we've already switched from sprintf() to construct unique
operation name to pass RNA path and array index.
There should be no functional changes yet, but this work is
required for further work about replacing string with const
char*.
All in all, this patch adds an Alembic importer, an Alembic exporter,
and a new CacheFile data block which, for now, wraps around an Alembic
archive. This data block is made available through a new modifier ("Mesh
Sequence Cache") as well as a new constraint ("Transform Cache") to
somewhat properly support respectively geometric and transformation data
streaming from alembic caches.
A more in-depth documentation is to be found on the wiki, as well as a
guide to compile alembic: https://wiki.blender.org/index.php/
User:Kevindietrich/AlembicBasicIo.
Many thanks to everyone involved in this little project, and huge shout
out to "cgstrive" for the thorough testings with Maya, 3ds Max, Houdini
and Realflow as well as @fjuhec, @jensverwiebe and @jasperge for the
custom builds and compile fixes.
Reviewers: sergey, campbellbarton, mont29
Reviewed By: sergey, campbellbarton, mont29
Differential Revision: https://developer.blender.org/D2060
