This repository has been archived on 2023-10-09. You can view files and clone it, but cannot push or open issues or pull requests.
Files
blender-archive/source/blender/depsgraph/intern/eval/deg_eval_flush.cc

250 lines
7.4 KiB
C++
Raw Normal View History

Depsgraph: Cleanup and code simplification This is mainly a maintenance commit which was aimed to make work with this module more pleasant and solve such issues as: - Annoyance with looong files, which had craftload in them - Usage of STL for the data structures we've got in BLI - Possible symbol conflicts - Not real clear layout of what is located where So in this commit the following changes are done: - STL is prohibited, it's not really predictable on various compilers, with our BLI algorithms we can predict things much better. There are still few usages of std::vector, but that we'll be solving later once we've got similar thing in BLI. - Simplify foreach loops, avoid using const_iterator all over the place. - New directory layout, which is hopefully easier to follow. - Some files were split, some of them will be split soon. The idea of this is to split huge functions into own files with good documentation and everything. - Removed stuff which was planned for use in the future but was never finished, tested or anything. Let's wipe it out for now, and bring back once we really start using it, so it'll be more clear if it solves our needs. - All the internal routines were moved to DEG namespace to separate them better from rest of blender. Some places now annoyingly using DEG::foo, but that we can olve by moving some utility functions inside of the namespace. While working on this we've found some hotspot in updates flush, so now playback of blenrig is few percent faster (something like 96fps with previous master and around 99-100fps after this change). Not saying it's something final, there is still room for cleanup and API simplification, but those might happen as a regular development now without doing any global changes.
2016-05-27 18:01:18 +02:00
/*
* ***** BEGIN GPL LICENSE BLOCK *****
*
* This program is free software; you can redistribute it and/or
* modify it under the terms of the GNU General Public License
* as published by the Free Software Foundation; either version 2
* of the License, or (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program; if not, write to the Free Software Foundation,
* Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
*
* The Original Code is Copyright (C) 2013 Blender Foundation.
* All rights reserved.
*
* Original Author: Joshua Leung
* Contributor(s): None Yet
*
* ***** END GPL LICENSE BLOCK *****
*/
/** \file blender/depsgraph/intern/eval/deg_eval_flush.cc
Depsgraph: Cleanup and code simplification This is mainly a maintenance commit which was aimed to make work with this module more pleasant and solve such issues as: - Annoyance with looong files, which had craftload in them - Usage of STL for the data structures we've got in BLI - Possible symbol conflicts - Not real clear layout of what is located where So in this commit the following changes are done: - STL is prohibited, it's not really predictable on various compilers, with our BLI algorithms we can predict things much better. There are still few usages of std::vector, but that we'll be solving later once we've got similar thing in BLI. - Simplify foreach loops, avoid using const_iterator all over the place. - New directory layout, which is hopefully easier to follow. - Some files were split, some of them will be split soon. The idea of this is to split huge functions into own files with good documentation and everything. - Removed stuff which was planned for use in the future but was never finished, tested or anything. Let's wipe it out for now, and bring back once we really start using it, so it'll be more clear if it solves our needs. - All the internal routines were moved to DEG namespace to separate them better from rest of blender. Some places now annoyingly using DEG::foo, but that we can olve by moving some utility functions inside of the namespace. While working on this we've found some hotspot in updates flush, so now playback of blenrig is few percent faster (something like 96fps with previous master and around 99-100fps after this change). Not saying it's something final, there is still room for cleanup and API simplification, but those might happen as a regular development now without doing any global changes.
2016-05-27 18:01:18 +02:00
* \ingroup depsgraph
*
* Core routines for how the Depsgraph works.
*/
#include "intern/eval/deg_eval_flush.h"
// TODO(sergey): Use some sort of wrapper.
#include <deque>
Depsgraph: Cleanup and code simplification This is mainly a maintenance commit which was aimed to make work with this module more pleasant and solve such issues as: - Annoyance with looong files, which had craftload in them - Usage of STL for the data structures we've got in BLI - Possible symbol conflicts - Not real clear layout of what is located where So in this commit the following changes are done: - STL is prohibited, it's not really predictable on various compilers, with our BLI algorithms we can predict things much better. There are still few usages of std::vector, but that we'll be solving later once we've got similar thing in BLI. - Simplify foreach loops, avoid using const_iterator all over the place. - New directory layout, which is hopefully easier to follow. - Some files were split, some of them will be split soon. The idea of this is to split huge functions into own files with good documentation and everything. - Removed stuff which was planned for use in the future but was never finished, tested or anything. Let's wipe it out for now, and bring back once we really start using it, so it'll be more clear if it solves our needs. - All the internal routines were moved to DEG namespace to separate them better from rest of blender. Some places now annoyingly using DEG::foo, but that we can olve by moving some utility functions inside of the namespace. While working on this we've found some hotspot in updates flush, so now playback of blenrig is few percent faster (something like 96fps with previous master and around 99-100fps after this change). Not saying it's something final, there is still room for cleanup and API simplification, but those might happen as a regular development now without doing any global changes.
2016-05-27 18:01:18 +02:00
#include "BLI_utildefines.h"
#include "BLI_task.h"
#include "BLI_ghash.h"
extern "C" {
#include "DNA_object_types.h"
Depsgraph: Cleanup and code simplification This is mainly a maintenance commit which was aimed to make work with this module more pleasant and solve such issues as: - Annoyance with looong files, which had craftload in them - Usage of STL for the data structures we've got in BLI - Possible symbol conflicts - Not real clear layout of what is located where So in this commit the following changes are done: - STL is prohibited, it's not really predictable on various compilers, with our BLI algorithms we can predict things much better. There are still few usages of std::vector, but that we'll be solving later once we've got similar thing in BLI. - Simplify foreach loops, avoid using const_iterator all over the place. - New directory layout, which is hopefully easier to follow. - Some files were split, some of them will be split soon. The idea of this is to split huge functions into own files with good documentation and everything. - Removed stuff which was planned for use in the future but was never finished, tested or anything. Let's wipe it out for now, and bring back once we really start using it, so it'll be more clear if it solves our needs. - All the internal routines were moved to DEG namespace to separate them better from rest of blender. Some places now annoyingly using DEG::foo, but that we can olve by moving some utility functions inside of the namespace. While working on this we've found some hotspot in updates flush, so now playback of blenrig is few percent faster (something like 96fps with previous master and around 99-100fps after this change). Not saying it's something final, there is still room for cleanup and API simplification, but those might happen as a regular development now without doing any global changes.
2016-05-27 18:01:18 +02:00
} /* extern "C" */
#include "DEG_depsgraph.h"
Depsgraph: Cleanup and code simplification This is mainly a maintenance commit which was aimed to make work with this module more pleasant and solve such issues as: - Annoyance with looong files, which had craftload in them - Usage of STL for the data structures we've got in BLI - Possible symbol conflicts - Not real clear layout of what is located where So in this commit the following changes are done: - STL is prohibited, it's not really predictable on various compilers, with our BLI algorithms we can predict things much better. There are still few usages of std::vector, but that we'll be solving later once we've got similar thing in BLI. - Simplify foreach loops, avoid using const_iterator all over the place. - New directory layout, which is hopefully easier to follow. - Some files were split, some of them will be split soon. The idea of this is to split huge functions into own files with good documentation and everything. - Removed stuff which was planned for use in the future but was never finished, tested or anything. Let's wipe it out for now, and bring back once we really start using it, so it'll be more clear if it solves our needs. - All the internal routines were moved to DEG namespace to separate them better from rest of blender. Some places now annoyingly using DEG::foo, but that we can olve by moving some utility functions inside of the namespace. While working on this we've found some hotspot in updates flush, so now playback of blenrig is few percent faster (something like 96fps with previous master and around 99-100fps after this change). Not saying it's something final, there is still room for cleanup and API simplification, but those might happen as a regular development now without doing any global changes.
2016-05-27 18:01:18 +02:00
#include "intern/nodes/deg_node.h"
#include "intern/nodes/deg_node_component.h"
#include "intern/nodes/deg_node_operation.h"
#include "intern/depsgraph_intern.h"
#include "intern/eval/deg_eval_copy_on_write.h"
Depsgraph: Cleanup and code simplification This is mainly a maintenance commit which was aimed to make work with this module more pleasant and solve such issues as: - Annoyance with looong files, which had craftload in them - Usage of STL for the data structures we've got in BLI - Possible symbol conflicts - Not real clear layout of what is located where So in this commit the following changes are done: - STL is prohibited, it's not really predictable on various compilers, with our BLI algorithms we can predict things much better. There are still few usages of std::vector, but that we'll be solving later once we've got similar thing in BLI. - Simplify foreach loops, avoid using const_iterator all over the place. - New directory layout, which is hopefully easier to follow. - Some files were split, some of them will be split soon. The idea of this is to split huge functions into own files with good documentation and everything. - Removed stuff which was planned for use in the future but was never finished, tested or anything. Let's wipe it out for now, and bring back once we really start using it, so it'll be more clear if it solves our needs. - All the internal routines were moved to DEG namespace to separate them better from rest of blender. Some places now annoyingly using DEG::foo, but that we can olve by moving some utility functions inside of the namespace. While working on this we've found some hotspot in updates flush, so now playback of blenrig is few percent faster (something like 96fps with previous master and around 99-100fps after this change). Not saying it's something final, there is still room for cleanup and API simplification, but those might happen as a regular development now without doing any global changes.
2016-05-27 18:01:18 +02:00
#include "util/deg_util_foreach.h"
namespace DEG {
enum {
COMPONENT_STATE_NONE = 0,
COMPONENT_STATE_SCHEDULED = 1,
COMPONENT_STATE_DONE = 2,
};
typedef std::deque<OperationDepsNode *> FlushQueue;
Depsgraph: Cleanup and code simplification This is mainly a maintenance commit which was aimed to make work with this module more pleasant and solve such issues as: - Annoyance with looong files, which had craftload in them - Usage of STL for the data structures we've got in BLI - Possible symbol conflicts - Not real clear layout of what is located where So in this commit the following changes are done: - STL is prohibited, it's not really predictable on various compilers, with our BLI algorithms we can predict things much better. There are still few usages of std::vector, but that we'll be solving later once we've got similar thing in BLI. - Simplify foreach loops, avoid using const_iterator all over the place. - New directory layout, which is hopefully easier to follow. - Some files were split, some of them will be split soon. The idea of this is to split huge functions into own files with good documentation and everything. - Removed stuff which was planned for use in the future but was never finished, tested or anything. Let's wipe it out for now, and bring back once we really start using it, so it'll be more clear if it solves our needs. - All the internal routines were moved to DEG namespace to separate them better from rest of blender. Some places now annoyingly using DEG::foo, but that we can olve by moving some utility functions inside of the namespace. While working on this we've found some hotspot in updates flush, so now playback of blenrig is few percent faster (something like 96fps with previous master and around 99-100fps after this change). Not saying it's something final, there is still room for cleanup and API simplification, but those might happen as a regular development now without doing any global changes.
2016-05-27 18:01:18 +02:00
static void flush_init_func(void *data_v, int i)
{
/* ID node's done flag is used to avoid multiple editors update
* for the same ID.
*/
Depsgraph *graph = (Depsgraph *)data_v;
OperationDepsNode *node = graph->operations[i];
ComponentDepsNode *comp_node = node->owner;
IDDepsNode *id_node = comp_node->owner;
Depsgraph: Cleanup and code simplification This is mainly a maintenance commit which was aimed to make work with this module more pleasant and solve such issues as: - Annoyance with looong files, which had craftload in them - Usage of STL for the data structures we've got in BLI - Possible symbol conflicts - Not real clear layout of what is located where So in this commit the following changes are done: - STL is prohibited, it's not really predictable on various compilers, with our BLI algorithms we can predict things much better. There are still few usages of std::vector, but that we'll be solving later once we've got similar thing in BLI. - Simplify foreach loops, avoid using const_iterator all over the place. - New directory layout, which is hopefully easier to follow. - Some files were split, some of them will be split soon. The idea of this is to split huge functions into own files with good documentation and everything. - Removed stuff which was planned for use in the future but was never finished, tested or anything. Let's wipe it out for now, and bring back once we really start using it, so it'll be more clear if it solves our needs. - All the internal routines were moved to DEG namespace to separate them better from rest of blender. Some places now annoyingly using DEG::foo, but that we can olve by moving some utility functions inside of the namespace. While working on this we've found some hotspot in updates flush, so now playback of blenrig is few percent faster (something like 96fps with previous master and around 99-100fps after this change). Not saying it's something final, there is still room for cleanup and API simplification, but those might happen as a regular development now without doing any global changes.
2016-05-27 18:01:18 +02:00
id_node->done = 0;
comp_node->done = COMPONENT_STATE_NONE;
Depsgraph: Cleanup and code simplification This is mainly a maintenance commit which was aimed to make work with this module more pleasant and solve such issues as: - Annoyance with looong files, which had craftload in them - Usage of STL for the data structures we've got in BLI - Possible symbol conflicts - Not real clear layout of what is located where So in this commit the following changes are done: - STL is prohibited, it's not really predictable on various compilers, with our BLI algorithms we can predict things much better. There are still few usages of std::vector, but that we'll be solving later once we've got similar thing in BLI. - Simplify foreach loops, avoid using const_iterator all over the place. - New directory layout, which is hopefully easier to follow. - Some files were split, some of them will be split soon. The idea of this is to split huge functions into own files with good documentation and everything. - Removed stuff which was planned for use in the future but was never finished, tested or anything. Let's wipe it out for now, and bring back once we really start using it, so it'll be more clear if it solves our needs. - All the internal routines were moved to DEG namespace to separate them better from rest of blender. Some places now annoyingly using DEG::foo, but that we can olve by moving some utility functions inside of the namespace. While working on this we've found some hotspot in updates flush, so now playback of blenrig is few percent faster (something like 96fps with previous master and around 99-100fps after this change). Not saying it's something final, there is still room for cleanup and API simplification, but those might happen as a regular development now without doing any global changes.
2016-05-27 18:01:18 +02:00
node->scheduled = false;
}
/* Flush updates from tagged nodes outwards until all affected nodes
* are tagged.
*/
void deg_graph_flush_updates(Main *bmain, Depsgraph *graph)
{
const bool use_copy_on_write = DEG_depsgraph_use_copy_on_write();
Depsgraph: Cleanup and code simplification This is mainly a maintenance commit which was aimed to make work with this module more pleasant and solve such issues as: - Annoyance with looong files, which had craftload in them - Usage of STL for the data structures we've got in BLI - Possible symbol conflicts - Not real clear layout of what is located where So in this commit the following changes are done: - STL is prohibited, it's not really predictable on various compilers, with our BLI algorithms we can predict things much better. There are still few usages of std::vector, but that we'll be solving later once we've got similar thing in BLI. - Simplify foreach loops, avoid using const_iterator all over the place. - New directory layout, which is hopefully easier to follow. - Some files were split, some of them will be split soon. The idea of this is to split huge functions into own files with good documentation and everything. - Removed stuff which was planned for use in the future but was never finished, tested or anything. Let's wipe it out for now, and bring back once we really start using it, so it'll be more clear if it solves our needs. - All the internal routines were moved to DEG namespace to separate them better from rest of blender. Some places now annoyingly using DEG::foo, but that we can olve by moving some utility functions inside of the namespace. While working on this we've found some hotspot in updates flush, so now playback of blenrig is few percent faster (something like 96fps with previous master and around 99-100fps after this change). Not saying it's something final, there is still room for cleanup and API simplification, but those might happen as a regular development now without doing any global changes.
2016-05-27 18:01:18 +02:00
/* Sanity check. */
if (graph == NULL) {
return;
}
/* Nothing to update, early out. */
if (BLI_gset_size(graph->entry_tags) == 0) {
return;
}
/* TODO(sergey): With a bit of flag magic we can get rid of this
* extra loop.
*/
const int num_operations = graph->operations.size();
const bool do_threads = num_operations > 256;
BLI_task_parallel_range(0,
num_operations,
graph,
flush_init_func,
do_threads);
FlushQueue queue;
/* Starting from the tagged "entry" nodes, flush outwards... */
/* NOTE: Also need to ensure that for each of these, there is a path back to
* root, or else they won't be done.
* NOTE: Count how many nodes we need to handle - entry nodes may be
* component nodes which don't count for this purpose!
*/
Depsgraph: Initial groundwork for copy-on-write support < Dependency graph Copy-on-Write > -------------------------------- \ ^__^ \ (oo)\_______ (__)\ )\/\ ||----w | || || This is an initial commit of Copy-on-write support added to dependency graph. Main priority for now: get playback (Alt-A) and all operators (selection, transform etc) to work with the new concept of clear separation between evaluated data coming from dependency graph and original data coming from .blend file (and stored in bmain). = How does this work? = The idea is to support Copy-on-Write on the ID level. This means, we duplicate the whole ID before we cann it's evaluaiton function. This is currently done in the following way: - At the depsgraph construction time we create "shallow" copy of the ID datablock, just so we know it's pointer in memory and can use for function bindings. - At the evaluaiton time, the copy of ID get's "expanded" (needs a better name internally, so it does not conflict with expanding datablocks during library linking), which means the content of the datablock is being copied over and all IDs are getting remapped to the copied ones. Currently we do the whole copy, in the future we will support some tricks here to prevent duplicating geometry arrays (verts, edges, loops, faces and polys) when we don't need that. - Evaluation functions are operating on copied datablocks and never touching original datablock. - There are some cases when we need to know non-ID pointers for function bindings. This mainly applies to scene collections and armatures. The idea of dealing with this is to "expand" copy-on-write datablock at the dependency graph build time. This might introduce some slowdown to the dependency graph construction time, but allows us to have minimal changes in the code and avoid any hash look-up from evaluation function (one of the ideas to avoid using pointers as function bindings is to pass name of layer or a bone to the evaluation function and look up actual data based on that name). Currently there is a special function in depsgraph which does such a synchronization, in the future we might want to make it more generic. At some point we need to synchronize copy-on-write version of datablock with the original version. This happens, i.e., when we change active object or change selection. We don't want any actual evaluation of update flush happening for such thins, so now we have a special update tag: DEG_id_tag_update((id, DEG_TAG_COPY_ON_WRITE) - For the render engines we now have special call for the dependency graph to give evaluated datablock for the given original one. This isn't fully ideal but allows to have Cycles viewport render. This is definitely a subject for further investigation / improvement. This call will tag copy-on-write component tagged for update without causing updates to be flushed to any other objects, causing chain reaction of updates. This tag is handy when selection in the scene changes. This basically summarizes ideas underneath this commit. The code should be reasonably documented. Here is a demo of dependency graph with all copy-on-write stuff in it: https://developer.blender.org/F635468 = What to expect to (not) work? = - Only meshes are properly-ish aware of copy-on-write currently, Non-mesh geometry will probably crash or will not work at all. - Armatures will need similar depsgraph built-time expansion of the copied datablock. - There are some extra tags / relations added, to keep things demo-able but which are slowing things down for evaluation. - Edit mode works for until click selection is used (due to the selection code using EditDerivedMesh created ad-hoc). - Lots of tools will lack tagging synchronization of copied datablock for sync with original ID. = How to move forward? = There is some tedious work related on going over all the tools, checking whether they need to work with original or final evaluated object and make the required changes. Additionally, there need synchronization tag done in fair amount of tools and operators as well. For example, currently it's not possible to change render engine without re-opening the file or forcing dependency graph for re-build via python console. There is also now some thoughts required about copying evaluated properties between objects or from collection to a new object. Perhaps easiest way would be to move base flag flush to Object ID node and tag new objects for update instead of doing manual copy. here is some WIP patch which moves such evaluaiton / flush: https://developer.blender.org/F635479 Lots of TODOs in the code, with possible optimization. = How to test? = This is a feature under heavy development, so obviously it is disabled by default. The only reason it goes to 2.8 branch is to avoid possible merge hell. In order to enable this feature use WITH_DEPSGRAPH_COPY_ON_WRITE CMake configuration option.
2017-06-14 10:26:24 +02:00
GSET_FOREACH_BEGIN(OperationDepsNode *, op_node, graph->entry_tags)
Depsgraph: Cleanup and code simplification This is mainly a maintenance commit which was aimed to make work with this module more pleasant and solve such issues as: - Annoyance with looong files, which had craftload in them - Usage of STL for the data structures we've got in BLI - Possible symbol conflicts - Not real clear layout of what is located where So in this commit the following changes are done: - STL is prohibited, it's not really predictable on various compilers, with our BLI algorithms we can predict things much better. There are still few usages of std::vector, but that we'll be solving later once we've got similar thing in BLI. - Simplify foreach loops, avoid using const_iterator all over the place. - New directory layout, which is hopefully easier to follow. - Some files were split, some of them will be split soon. The idea of this is to split huge functions into own files with good documentation and everything. - Removed stuff which was planned for use in the future but was never finished, tested or anything. Let's wipe it out for now, and bring back once we really start using it, so it'll be more clear if it solves our needs. - All the internal routines were moved to DEG namespace to separate them better from rest of blender. Some places now annoyingly using DEG::foo, but that we can olve by moving some utility functions inside of the namespace. While working on this we've found some hotspot in updates flush, so now playback of blenrig is few percent faster (something like 96fps with previous master and around 99-100fps after this change). Not saying it's something final, there is still room for cleanup and API simplification, but those might happen as a regular development now without doing any global changes.
2016-05-27 18:01:18 +02:00
{
queue.push_back(op_node);
op_node->scheduled = true;
Depsgraph: Cleanup and code simplification This is mainly a maintenance commit which was aimed to make work with this module more pleasant and solve such issues as: - Annoyance with looong files, which had craftload in them - Usage of STL for the data structures we've got in BLI - Possible symbol conflicts - Not real clear layout of what is located where So in this commit the following changes are done: - STL is prohibited, it's not really predictable on various compilers, with our BLI algorithms we can predict things much better. There are still few usages of std::vector, but that we'll be solving later once we've got similar thing in BLI. - Simplify foreach loops, avoid using const_iterator all over the place. - New directory layout, which is hopefully easier to follow. - Some files were split, some of them will be split soon. The idea of this is to split huge functions into own files with good documentation and everything. - Removed stuff which was planned for use in the future but was never finished, tested or anything. Let's wipe it out for now, and bring back once we really start using it, so it'll be more clear if it solves our needs. - All the internal routines were moved to DEG namespace to separate them better from rest of blender. Some places now annoyingly using DEG::foo, but that we can olve by moving some utility functions inside of the namespace. While working on this we've found some hotspot in updates flush, so now playback of blenrig is few percent faster (something like 96fps with previous master and around 99-100fps after this change). Not saying it's something final, there is still room for cleanup and API simplification, but those might happen as a regular development now without doing any global changes.
2016-05-27 18:01:18 +02:00
}
GSET_FOREACH_END();
DEGEditorUpdateContext update_ctx = {NULL};
update_ctx.bmain = bmain;
update_ctx.scene = graph->scene;
update_ctx.view_layer = graph->view_layer;
int num_flushed_objects = 0;
Depsgraph: Cleanup and code simplification This is mainly a maintenance commit which was aimed to make work with this module more pleasant and solve such issues as: - Annoyance with looong files, which had craftload in them - Usage of STL for the data structures we've got in BLI - Possible symbol conflicts - Not real clear layout of what is located where So in this commit the following changes are done: - STL is prohibited, it's not really predictable on various compilers, with our BLI algorithms we can predict things much better. There are still few usages of std::vector, but that we'll be solving later once we've got similar thing in BLI. - Simplify foreach loops, avoid using const_iterator all over the place. - New directory layout, which is hopefully easier to follow. - Some files were split, some of them will be split soon. The idea of this is to split huge functions into own files with good documentation and everything. - Removed stuff which was planned for use in the future but was never finished, tested or anything. Let's wipe it out for now, and bring back once we really start using it, so it'll be more clear if it solves our needs. - All the internal routines were moved to DEG namespace to separate them better from rest of blender. Some places now annoyingly using DEG::foo, but that we can olve by moving some utility functions inside of the namespace. While working on this we've found some hotspot in updates flush, so now playback of blenrig is few percent faster (something like 96fps with previous master and around 99-100fps after this change). Not saying it's something final, there is still room for cleanup and API simplification, but those might happen as a regular development now without doing any global changes.
2016-05-27 18:01:18 +02:00
while (!queue.empty()) {
OperationDepsNode *node = queue.front();
queue.pop_front();
Depsgraph: Cleanup and code simplification This is mainly a maintenance commit which was aimed to make work with this module more pleasant and solve such issues as: - Annoyance with looong files, which had craftload in them - Usage of STL for the data structures we've got in BLI - Possible symbol conflicts - Not real clear layout of what is located where So in this commit the following changes are done: - STL is prohibited, it's not really predictable on various compilers, with our BLI algorithms we can predict things much better. There are still few usages of std::vector, but that we'll be solving later once we've got similar thing in BLI. - Simplify foreach loops, avoid using const_iterator all over the place. - New directory layout, which is hopefully easier to follow. - Some files were split, some of them will be split soon. The idea of this is to split huge functions into own files with good documentation and everything. - Removed stuff which was planned for use in the future but was never finished, tested or anything. Let's wipe it out for now, and bring back once we really start using it, so it'll be more clear if it solves our needs. - All the internal routines were moved to DEG namespace to separate them better from rest of blender. Some places now annoyingly using DEG::foo, but that we can olve by moving some utility functions inside of the namespace. While working on this we've found some hotspot in updates flush, so now playback of blenrig is few percent faster (something like 96fps with previous master and around 99-100fps after this change). Not saying it's something final, there is still room for cleanup and API simplification, but those might happen as a regular development now without doing any global changes.
2016-05-27 18:01:18 +02:00
for (;;) {
node->flag |= DEPSOP_FLAG_NEEDS_UPDATE;
ComponentDepsNode *comp_node = node->owner;
IDDepsNode *id_node = comp_node->owner;
Depsgraph: Initial groundwork for copy-on-write support < Dependency graph Copy-on-Write > -------------------------------- \ ^__^ \ (oo)\_______ (__)\ )\/\ ||----w | || || This is an initial commit of Copy-on-write support added to dependency graph. Main priority for now: get playback (Alt-A) and all operators (selection, transform etc) to work with the new concept of clear separation between evaluated data coming from dependency graph and original data coming from .blend file (and stored in bmain). = How does this work? = The idea is to support Copy-on-Write on the ID level. This means, we duplicate the whole ID before we cann it's evaluaiton function. This is currently done in the following way: - At the depsgraph construction time we create "shallow" copy of the ID datablock, just so we know it's pointer in memory and can use for function bindings. - At the evaluaiton time, the copy of ID get's "expanded" (needs a better name internally, so it does not conflict with expanding datablocks during library linking), which means the content of the datablock is being copied over and all IDs are getting remapped to the copied ones. Currently we do the whole copy, in the future we will support some tricks here to prevent duplicating geometry arrays (verts, edges, loops, faces and polys) when we don't need that. - Evaluation functions are operating on copied datablocks and never touching original datablock. - There are some cases when we need to know non-ID pointers for function bindings. This mainly applies to scene collections and armatures. The idea of dealing with this is to "expand" copy-on-write datablock at the dependency graph build time. This might introduce some slowdown to the dependency graph construction time, but allows us to have minimal changes in the code and avoid any hash look-up from evaluation function (one of the ideas to avoid using pointers as function bindings is to pass name of layer or a bone to the evaluation function and look up actual data based on that name). Currently there is a special function in depsgraph which does such a synchronization, in the future we might want to make it more generic. At some point we need to synchronize copy-on-write version of datablock with the original version. This happens, i.e., when we change active object or change selection. We don't want any actual evaluation of update flush happening for such thins, so now we have a special update tag: DEG_id_tag_update((id, DEG_TAG_COPY_ON_WRITE) - For the render engines we now have special call for the dependency graph to give evaluated datablock for the given original one. This isn't fully ideal but allows to have Cycles viewport render. This is definitely a subject for further investigation / improvement. This call will tag copy-on-write component tagged for update without causing updates to be flushed to any other objects, causing chain reaction of updates. This tag is handy when selection in the scene changes. This basically summarizes ideas underneath this commit. The code should be reasonably documented. Here is a demo of dependency graph with all copy-on-write stuff in it: https://developer.blender.org/F635468 = What to expect to (not) work? = - Only meshes are properly-ish aware of copy-on-write currently, Non-mesh geometry will probably crash or will not work at all. - Armatures will need similar depsgraph built-time expansion of the copied datablock. - There are some extra tags / relations added, to keep things demo-able but which are slowing things down for evaluation. - Edit mode works for until click selection is used (due to the selection code using EditDerivedMesh created ad-hoc). - Lots of tools will lack tagging synchronization of copied datablock for sync with original ID. = How to move forward? = There is some tedious work related on going over all the tools, checking whether they need to work with original or final evaluated object and make the required changes. Additionally, there need synchronization tag done in fair amount of tools and operators as well. For example, currently it's not possible to change render engine without re-opening the file or forcing dependency graph for re-build via python console. There is also now some thoughts required about copying evaluated properties between objects or from collection to a new object. Perhaps easiest way would be to move base flag flush to Object ID node and tag new objects for update instead of doing manual copy. here is some WIP patch which moves such evaluaiton / flush: https://developer.blender.org/F635479 Lots of TODOs in the code, with possible optimization. = How to test? = This is a feature under heavy development, so obviously it is disabled by default. The only reason it goes to 2.8 branch is to avoid possible merge hell. In order to enable this feature use WITH_DEPSGRAPH_COPY_ON_WRITE CMake configuration option.
2017-06-14 10:26:24 +02:00
/* TODO(sergey): Do we need to pass original or evaluated ID here? */
ID *id_orig = id_node->id_orig;
ID *id_cow = id_node->id_cow;
2017-06-02 15:38:04 +10:00
if (id_node->done == 0) {
/* Copy tag from original data to CoW storage.
* This is because DEG_id_tag_update() sets tags on original
* data.
*/
id_cow->tag |= (id_orig->tag & LIB_TAG_ID_RECALC_ALL);
if (deg_copy_on_write_is_expanded(id_cow)) {
deg_editors_id_update(&update_ctx, id_cow);
}
lib_id_recalc_tag(bmain, id_orig);
/* TODO(sergey): For until we've got proper data nodes in the graph. */
lib_id_recalc_data_tag(bmain, id_orig);
}
Depsgraph: Initial groundwork for copy-on-write support < Dependency graph Copy-on-Write > -------------------------------- \ ^__^ \ (oo)\_______ (__)\ )\/\ ||----w | || || This is an initial commit of Copy-on-write support added to dependency graph. Main priority for now: get playback (Alt-A) and all operators (selection, transform etc) to work with the new concept of clear separation between evaluated data coming from dependency graph and original data coming from .blend file (and stored in bmain). = How does this work? = The idea is to support Copy-on-Write on the ID level. This means, we duplicate the whole ID before we cann it's evaluaiton function. This is currently done in the following way: - At the depsgraph construction time we create "shallow" copy of the ID datablock, just so we know it's pointer in memory and can use for function bindings. - At the evaluaiton time, the copy of ID get's "expanded" (needs a better name internally, so it does not conflict with expanding datablocks during library linking), which means the content of the datablock is being copied over and all IDs are getting remapped to the copied ones. Currently we do the whole copy, in the future we will support some tricks here to prevent duplicating geometry arrays (verts, edges, loops, faces and polys) when we don't need that. - Evaluation functions are operating on copied datablocks and never touching original datablock. - There are some cases when we need to know non-ID pointers for function bindings. This mainly applies to scene collections and armatures. The idea of dealing with this is to "expand" copy-on-write datablock at the dependency graph build time. This might introduce some slowdown to the dependency graph construction time, but allows us to have minimal changes in the code and avoid any hash look-up from evaluation function (one of the ideas to avoid using pointers as function bindings is to pass name of layer or a bone to the evaluation function and look up actual data based on that name). Currently there is a special function in depsgraph which does such a synchronization, in the future we might want to make it more generic. At some point we need to synchronize copy-on-write version of datablock with the original version. This happens, i.e., when we change active object or change selection. We don't want any actual evaluation of update flush happening for such thins, so now we have a special update tag: DEG_id_tag_update((id, DEG_TAG_COPY_ON_WRITE) - For the render engines we now have special call for the dependency graph to give evaluated datablock for the given original one. This isn't fully ideal but allows to have Cycles viewport render. This is definitely a subject for further investigation / improvement. This call will tag copy-on-write component tagged for update without causing updates to be flushed to any other objects, causing chain reaction of updates. This tag is handy when selection in the scene changes. This basically summarizes ideas underneath this commit. The code should be reasonably documented. Here is a demo of dependency graph with all copy-on-write stuff in it: https://developer.blender.org/F635468 = What to expect to (not) work? = - Only meshes are properly-ish aware of copy-on-write currently, Non-mesh geometry will probably crash or will not work at all. - Armatures will need similar depsgraph built-time expansion of the copied datablock. - There are some extra tags / relations added, to keep things demo-able but which are slowing things down for evaluation. - Edit mode works for until click selection is used (due to the selection code using EditDerivedMesh created ad-hoc). - Lots of tools will lack tagging synchronization of copied datablock for sync with original ID. = How to move forward? = There is some tedious work related on going over all the tools, checking whether they need to work with original or final evaluated object and make the required changes. Additionally, there need synchronization tag done in fair amount of tools and operators as well. For example, currently it's not possible to change render engine without re-opening the file or forcing dependency graph for re-build via python console. There is also now some thoughts required about copying evaluated properties between objects or from collection to a new object. Perhaps easiest way would be to move base flag flush to Object ID node and tag new objects for update instead of doing manual copy. here is some WIP patch which moves such evaluaiton / flush: https://developer.blender.org/F635479 Lots of TODOs in the code, with possible optimization. = How to test? = This is a feature under heavy development, so obviously it is disabled by default. The only reason it goes to 2.8 branch is to avoid possible merge hell. In order to enable this feature use WITH_DEPSGRAPH_COPY_ON_WRITE CMake configuration option.
2017-06-14 10:26:24 +02:00
if (comp_node->done != COMPONENT_STATE_DONE) {
2017-11-23 11:42:06 +01:00
/* Currently this is needed to get object->mesh to be replaced with
Depsgraph: Initial groundwork for copy-on-write support < Dependency graph Copy-on-Write > -------------------------------- \ ^__^ \ (oo)\_______ (__)\ )\/\ ||----w | || || This is an initial commit of Copy-on-write support added to dependency graph. Main priority for now: get playback (Alt-A) and all operators (selection, transform etc) to work with the new concept of clear separation between evaluated data coming from dependency graph and original data coming from .blend file (and stored in bmain). = How does this work? = The idea is to support Copy-on-Write on the ID level. This means, we duplicate the whole ID before we cann it's evaluaiton function. This is currently done in the following way: - At the depsgraph construction time we create "shallow" copy of the ID datablock, just so we know it's pointer in memory and can use for function bindings. - At the evaluaiton time, the copy of ID get's "expanded" (needs a better name internally, so it does not conflict with expanding datablocks during library linking), which means the content of the datablock is being copied over and all IDs are getting remapped to the copied ones. Currently we do the whole copy, in the future we will support some tricks here to prevent duplicating geometry arrays (verts, edges, loops, faces and polys) when we don't need that. - Evaluation functions are operating on copied datablocks and never touching original datablock. - There are some cases when we need to know non-ID pointers for function bindings. This mainly applies to scene collections and armatures. The idea of dealing with this is to "expand" copy-on-write datablock at the dependency graph build time. This might introduce some slowdown to the dependency graph construction time, but allows us to have minimal changes in the code and avoid any hash look-up from evaluation function (one of the ideas to avoid using pointers as function bindings is to pass name of layer or a bone to the evaluation function and look up actual data based on that name). Currently there is a special function in depsgraph which does such a synchronization, in the future we might want to make it more generic. At some point we need to synchronize copy-on-write version of datablock with the original version. This happens, i.e., when we change active object or change selection. We don't want any actual evaluation of update flush happening for such thins, so now we have a special update tag: DEG_id_tag_update((id, DEG_TAG_COPY_ON_WRITE) - For the render engines we now have special call for the dependency graph to give evaluated datablock for the given original one. This isn't fully ideal but allows to have Cycles viewport render. This is definitely a subject for further investigation / improvement. This call will tag copy-on-write component tagged for update without causing updates to be flushed to any other objects, causing chain reaction of updates. This tag is handy when selection in the scene changes. This basically summarizes ideas underneath this commit. The code should be reasonably documented. Here is a demo of dependency graph with all copy-on-write stuff in it: https://developer.blender.org/F635468 = What to expect to (not) work? = - Only meshes are properly-ish aware of copy-on-write currently, Non-mesh geometry will probably crash or will not work at all. - Armatures will need similar depsgraph built-time expansion of the copied datablock. - There are some extra tags / relations added, to keep things demo-able but which are slowing things down for evaluation. - Edit mode works for until click selection is used (due to the selection code using EditDerivedMesh created ad-hoc). - Lots of tools will lack tagging synchronization of copied datablock for sync with original ID. = How to move forward? = There is some tedious work related on going over all the tools, checking whether they need to work with original or final evaluated object and make the required changes. Additionally, there need synchronization tag done in fair amount of tools and operators as well. For example, currently it's not possible to change render engine without re-opening the file or forcing dependency graph for re-build via python console. There is also now some thoughts required about copying evaluated properties between objects or from collection to a new object. Perhaps easiest way would be to move base flag flush to Object ID node and tag new objects for update instead of doing manual copy. here is some WIP patch which moves such evaluaiton / flush: https://developer.blender.org/F635479 Lots of TODOs in the code, with possible optimization. = How to test? = This is a feature under heavy development, so obviously it is disabled by default. The only reason it goes to 2.8 branch is to avoid possible merge hell. In order to enable this feature use WITH_DEPSGRAPH_COPY_ON_WRITE CMake configuration option.
2017-06-14 10:26:24 +02:00
* original mesh (rather than being evaluated_mesh).
*
* TODO(sergey): This is something we need to avoid.
*/
if (use_copy_on_write && comp_node->depends_on_cow()) {
ComponentDepsNode *cow_comp =
id_node->find_component(DEG_NODE_TYPE_COPY_ON_WRITE);
cow_comp->tag_update(graph);
}
if (GS(id_orig->name) == ID_OB) {
2017-06-02 15:38:04 +10:00
if (id_node->done == 0) {
++num_flushed_objects;
}
}
foreach (OperationDepsNode *op, comp_node->operations) {
/* We don't want to flush tags in "upstream" direction for
* certain types of operations.
*
* TODO(sergey): Need a more generic solution for this.
*/
if (op->opcode == DEG_OPCODE_PARTICLE_SETTINGS_EVAL) {
continue;
}
op->flag |= DEPSOP_FLAG_NEEDS_UPDATE;
}
/* When some target changes bone, we might need to re-run the
* whole IK solver, otherwise result might be unpredictable.
*/
if (comp_node->type == DEG_NODE_TYPE_BONE) {
ComponentDepsNode *pose_comp =
id_node->find_component(DEG_NODE_TYPE_EVAL_POSE);
BLI_assert(pose_comp != NULL);
if (pose_comp->done == COMPONENT_STATE_NONE) {
queue.push_front(pose_comp->get_entry_operation());
pose_comp->done = COMPONENT_STATE_SCHEDULED;
}
}
}
id_node->done = 1;
comp_node->done = COMPONENT_STATE_DONE;
Depsgraph: Cleanup and code simplification This is mainly a maintenance commit which was aimed to make work with this module more pleasant and solve such issues as: - Annoyance with looong files, which had craftload in them - Usage of STL for the data structures we've got in BLI - Possible symbol conflicts - Not real clear layout of what is located where So in this commit the following changes are done: - STL is prohibited, it's not really predictable on various compilers, with our BLI algorithms we can predict things much better. There are still few usages of std::vector, but that we'll be solving later once we've got similar thing in BLI. - Simplify foreach loops, avoid using const_iterator all over the place. - New directory layout, which is hopefully easier to follow. - Some files were split, some of them will be split soon. The idea of this is to split huge functions into own files with good documentation and everything. - Removed stuff which was planned for use in the future but was never finished, tested or anything. Let's wipe it out for now, and bring back once we really start using it, so it'll be more clear if it solves our needs. - All the internal routines were moved to DEG namespace to separate them better from rest of blender. Some places now annoyingly using DEG::foo, but that we can olve by moving some utility functions inside of the namespace. While working on this we've found some hotspot in updates flush, so now playback of blenrig is few percent faster (something like 96fps with previous master and around 99-100fps after this change). Not saying it's something final, there is still room for cleanup and API simplification, but those might happen as a regular development now without doing any global changes.
2016-05-27 18:01:18 +02:00
/* Flush to nodes along links... */
/* TODO(sergey): This is mainly giving speedup due ot less queue pushes, which
* reduces number of memory allocations.
*
* We should try solve the allocation issue instead of doing crazy things here.
*/
if (node->outlinks.size() == 1) {
OperationDepsNode *to_node = (OperationDepsNode *)node->outlinks[0]->to;
if (to_node->scheduled == false) {
to_node->scheduled = true;
node = to_node;
}
else {
break;
}
}
else {
foreach (DepsRelation *rel, node->outlinks) {
OperationDepsNode *to_node = (OperationDepsNode *)rel->to;
if (to_node->scheduled == false) {
queue.push_front(to_node);
to_node->scheduled = true;
}
}
break;
Depsgraph: Cleanup and code simplification This is mainly a maintenance commit which was aimed to make work with this module more pleasant and solve such issues as: - Annoyance with looong files, which had craftload in them - Usage of STL for the data structures we've got in BLI - Possible symbol conflicts - Not real clear layout of what is located where So in this commit the following changes are done: - STL is prohibited, it's not really predictable on various compilers, with our BLI algorithms we can predict things much better. There are still few usages of std::vector, but that we'll be solving later once we've got similar thing in BLI. - Simplify foreach loops, avoid using const_iterator all over the place. - New directory layout, which is hopefully easier to follow. - Some files were split, some of them will be split soon. The idea of this is to split huge functions into own files with good documentation and everything. - Removed stuff which was planned for use in the future but was never finished, tested or anything. Let's wipe it out for now, and bring back once we really start using it, so it'll be more clear if it solves our needs. - All the internal routines were moved to DEG namespace to separate them better from rest of blender. Some places now annoyingly using DEG::foo, but that we can olve by moving some utility functions inside of the namespace. While working on this we've found some hotspot in updates flush, so now playback of blenrig is few percent faster (something like 96fps with previous master and around 99-100fps after this change). Not saying it's something final, there is still room for cleanup and API simplification, but those might happen as a regular development now without doing any global changes.
2016-05-27 18:01:18 +02:00
}
}
}
DEG_DEBUG_PRINTF("Update flushed to %d objects\n", num_flushed_objects);
Depsgraph: Cleanup and code simplification This is mainly a maintenance commit which was aimed to make work with this module more pleasant and solve such issues as: - Annoyance with looong files, which had craftload in them - Usage of STL for the data structures we've got in BLI - Possible symbol conflicts - Not real clear layout of what is located where So in this commit the following changes are done: - STL is prohibited, it's not really predictable on various compilers, with our BLI algorithms we can predict things much better. There are still few usages of std::vector, but that we'll be solving later once we've got similar thing in BLI. - Simplify foreach loops, avoid using const_iterator all over the place. - New directory layout, which is hopefully easier to follow. - Some files were split, some of them will be split soon. The idea of this is to split huge functions into own files with good documentation and everything. - Removed stuff which was planned for use in the future but was never finished, tested or anything. Let's wipe it out for now, and bring back once we really start using it, so it'll be more clear if it solves our needs. - All the internal routines were moved to DEG namespace to separate them better from rest of blender. Some places now annoyingly using DEG::foo, but that we can olve by moving some utility functions inside of the namespace. While working on this we've found some hotspot in updates flush, so now playback of blenrig is few percent faster (something like 96fps with previous master and around 99-100fps after this change). Not saying it's something final, there is still room for cleanup and API simplification, but those might happen as a regular development now without doing any global changes.
2016-05-27 18:01:18 +02:00
}
static void graph_clear_func(void *data_v, int i)
{
Depsgraph *graph = (Depsgraph *)data_v;
OperationDepsNode *node = graph->operations[i];
/* Clear node's "pending update" settings. */
node->flag &= ~(DEPSOP_FLAG_DIRECTLY_MODIFIED | DEPSOP_FLAG_NEEDS_UPDATE);
}
/* Clear tags from all operation nodes. */
void deg_graph_clear_tags(Depsgraph *graph)
{
/* Go over all operation nodes, clearing tags. */
const int num_operations = graph->operations.size();
const bool do_threads = num_operations > 256;
BLI_task_parallel_range(0, num_operations, graph, graph_clear_func, do_threads);
/* Clear any entry tags which haven't been flushed. */
BLI_gset_clear(graph->entry_tags, NULL);
}
} // namespace DEG