Previously it was done during depsgraph iteration, which is not good at all,
since after evaluation nobody should really modify how object was evaluated.
This is crucial bit since batch cache is stored in the evaluated object,
meaning we can't tag it's hatch cache dirty from the notifier system.
Not easily at least. Better to leave this job to depsgraph, it knows
all the copies of data.
This makes code closer to id_override/assent-engine ones, which
introduce a new type of linked data, and hence reserve
ID_IS_LINKED_DATABLOCK to real linked datablocks.
Before it was a compile time option which was not very easy to use or test. Now
the project is getting more mature, so very soon we will be able to call for a
public tests of limited features.
The copy-on-write (which includes animation, modifiers) is enabled using
--enable-copy-on-write command line argument.
- NOCHECK -> ALL
- ALL -> MAYBE_ALL
Where 'MAYBE_ALL' checks to see if the mesh has changed.
This is clearer that `BKE_MESH_BATCH_DIRTY_ALL` is dirty and
going to be updated without any guess-work.
2.8x branch added bContext arg in many places,
pass eval-context instead since its not simple to reason about what
what nested functions do when they can access and change almost anything.
Also use const to prevent unexpected modifications.
This fixes crash loading files with shadows,
since off-screen buffers use a NULL context for rendering.
Note that some little parts of code have been dissabled because eval_ctx
was not available there. This should be resolved once DerivedMesh is
replaced.
UVs need specific data in the VBO, which is not computed unless the
shaders assigned to the mesh actually use UVs. When adding UVs to the
shader, the VBOs were not being recomputed to include the required data.
This adds a DEG relation between the shader and the mesh, and recomputes
the required data if the shader changed.
Thanks Sergey, for all the DEG stuff...
< Dependency graph Copy-on-Write >
--------------------------------
\ ^__^
\ (oo)\_______
(__)\ )\/\
||----w |
|| ||
This is an initial commit of Copy-on-write support added to dependency graph.
Main priority for now: get playback (Alt-A) and all operators (selection,
transform etc) to work with the new concept of clear separation between
evaluated data coming from dependency graph and original data coming from
.blend file (and stored in bmain).
= How does this work? =
The idea is to support Copy-on-Write on the ID level. This means, we duplicate
the whole ID before we cann it's evaluaiton function. This is currently done
in the following way:
- At the depsgraph construction time we create "shallow" copy of the ID
datablock, just so we know it's pointer in memory and can use for function
bindings.
- At the evaluaiton time, the copy of ID get's "expanded" (needs a better
name internally, so it does not conflict with expanding datablocks during
library linking), which means the content of the datablock is being
copied over and all IDs are getting remapped to the copied ones.
Currently we do the whole copy, in the future we will support some tricks
here to prevent duplicating geometry arrays (verts, edges, loops, faces
and polys) when we don't need that.
- Evaluation functions are operating on copied datablocks and never touching
original datablock.
- There are some cases when we need to know non-ID pointers for function
bindings. This mainly applies to scene collections and armatures. The
idea of dealing with this is to "expand" copy-on-write datablock at
the dependency graph build time. This might introduce some slowdown to the
dependency graph construction time, but allows us to have minimal changes
in the code and avoid any hash look-up from evaluation function (one of
the ideas to avoid using pointers as function bindings is to pass name
of layer or a bone to the evaluation function and look up actual data based
on that name).
Currently there is a special function in depsgraph which does such a
synchronization, in the future we might want to make it more generic.
At some point we need to synchronize copy-on-write version of datablock with
the original version. This happens, i.e., when we change active object or
change selection. We don't want any actual evaluation of update flush happening
for such thins, so now we have a special update tag:
DEG_id_tag_update((id, DEG_TAG_COPY_ON_WRITE)
- For the render engines we now have special call for the dependency graph to
give evaluated datablock for the given original one. This isn't fully ideal
but allows to have Cycles viewport render.
This is definitely a subject for further investigation / improvement.
This call will tag copy-on-write component tagged for update without causing
updates to be flushed to any other objects, causing chain reaction of updates.
This tag is handy when selection in the scene changes.
This basically summarizes ideas underneath this commit. The code should be
reasonably documented.
Here is a demo of dependency graph with all copy-on-write stuff in it:
https://developer.blender.org/F635468
= What to expect to (not) work? =
- Only meshes are properly-ish aware of copy-on-write currently, Non-mesh
geometry will probably crash or will not work at all.
- Armatures will need similar depsgraph built-time expansion of the copied
datablock.
- There are some extra tags / relations added, to keep things demo-able but
which are slowing things down for evaluation.
- Edit mode works for until click selection is used (due to the selection
code using EditDerivedMesh created ad-hoc).
- Lots of tools will lack tagging synchronization of copied datablock for
sync with original ID.
= How to move forward? =
There is some tedious work related on going over all the tools, checking
whether they need to work with original or final evaluated object and make
the required changes.
Additionally, there need synchronization tag done in fair amount of tools
and operators as well. For example, currently it's not possible to change
render engine without re-opening the file or forcing dependency graph for
re-build via python console.
There is also now some thoughts required about copying evaluated properties
between objects or from collection to a new object. Perhaps easiest way
would be to move base flag flush to Object ID node and tag new objects for
update instead of doing manual copy.
here is some WIP patch which moves such evaluaiton / flush:
https://developer.blender.org/F635479
Lots of TODOs in the code, with possible optimization.
= How to test? =
This is a feature under heavy development, so obviously it is disabled by
default. The only reason it goes to 2.8 branch is to avoid possible merge
hell.
In order to enable this feature use WITH_DEPSGRAPH_COPY_ON_WRITE CMake
configuration option.
Was internally a no-op operation, which only caused extra work
to be done during depsgrpah traversal and evaluation, without
making any measurable improvement.
The thing i'm really starting to hate is the requirement to specify both
operation code and node type. Seems to be duplicated enums without real
need for that.
Note: when in edit mode this depsgraph update is not being called. We are using DerivedMesh in those cases, so it is fine. I would like to investigate this though
This reverts commit 5aa19be912 and b4a721af69.
Due to postponement of particle system rewrite it was decided to put particle code
back into the 2.8 branch for the time being.
This reverts commit 47d0d9cca4.
Reverting the commit. Not only it did not solve all the cases of proxy popping,
but also broke real cases with single proxy involved.
Idea is to replace hard-to-track (id->lib != NULL) 'is linked datablock' check everywhere in Blender
by a macro doing the same thing. This will allow to easily spot those checks in future, and more importantly,
to easily change it (see work done in asset-engine branch).
Note: did not touch to readfile.c, since there most of the time 'id->lib' check actually concerns the pointer,
and not a check whether ID is linked or not. Will have a closer look at it later.
Reviewers: campbellbarton, brecht, sergey
Differential Revision: https://developer.blender.org/D2082
Replaces `G.is_rendering` with `use_render_params` argument.
This is needed for Cycles, which attempts to restore render-preview settings from particles,
after it gets its own particle data, but fails to restore because
`G.is_rendering` was being checked in psys_cache_paths (and other places).
Makes behavior of proxy_from backlink working similar to the old dependency graph.
it's nasty, but needed here in the studio to get proxies fixes ASAP.
- place return args last position
- move crazyspace function out of DerivedMesh header
- use bool for args
- flow control on own lines to ease debugging
It was possible that two threads will start calculating proxy group's inverted
matrix and store it in the object itself. This isn't good idea because it means
some threads might be using partially written matrix.
Previously it was very easy to run into situation when two objects are sharing
the same materials with drivers which will cause threading access issues.
This actually only needed for the old depsgraph, but since it's still the one
we're using by default we'd better solve this issue.
This commit integrates the work done so far on the new dependency graph system,
where goal was to replace legacy depsgraph with the new one, supporting loads of
neat features like:
- More granular dependency relation nature, which solves issues with fake cycles
in the dependencies.
- Move towards all-animatable, by better integration of drivers into the system.
- Lay down some basis for upcoming copy-on-write, overrides and so on.
The new system is living side-by-side with the previous one and disabled by
default, so nothing will become suddenly broken. The way to enable new depsgraph
is to pass `--new-depsgraph` command line argument.
It's a bit early to consider the system production-ready, there are some TODOs
and issues were discovered during the merge period, they'll be addressed ASAP.
But it's important to merge, because it's the only way to attract artists to
really start testing this system.
There are number of assorted documents related on the design of the new system:
* http://wiki.blender.org/index.php/User:Aligorith/GSoC2013_Depsgraph#Design_Documents
* http://wiki.blender.org/index.php/User:Nazg-gul/DependencyGraph
There are also some user-related information online:
* http://code.blender.org/2015/02/blender-dependency-graph-branch-for-users/
* http://code.blender.org/2015/03/more-dependency-graph-tricks/
Kudos to everyone who was involved into the project:
- Joshua "Aligorith" Leung -- design specification, initial code
- Lukas "lukas_t" Toenne -- integrating code into blender, with further fixes
- Sergey "Sergey" "Sharybin" -- some mocking around, trying to wrap up the
project and so
- Bassam "slikdigit" Kurdali -- stressing the new system, reporting all the
issues and recording/writing documentation.
- Everyone else who i forgot to mention here :)
This commit only adds callbacks which then later be used with major dependency
graph commit, keeping the upcoming commit more clean to follow.
Should be no functional changes so far still.
Currently it is just moving existing functions into a new file,
but in the future those new files will be grown much more due
to upcoming more granular scene updates.
Should be no functional changes.
