Similar issue/solution as in rB5188c14718c5 from this Monday actually,
there may be more of those still lurking around... Quite surprising they
all get reported now, this behavior has been in Blender since years.
Fact that those options are only used in a specific case, and that the
same behavior is ensured in a different part of the code in other cases,
is fairly confusing and unfortunate... At least document it.
Similar to other changes to ID remapping, gives huge speedups in some
cases, like certain types of liboverride creation.
Case from {T96092} goes from 1725 seconds (almost 30 minutes) to 45
seconds to generate the liboverride, on my machine.
Reviewed By: jbakker
Maniphest Tasks: T96092
Differential Revision: https://developer.blender.org/D14240
This patch increases the performance when remapping data.
{D13615} introduced a mechanism to remap multiple items in a single go.
This patch uses the same mechanism when remapping data inside ID datablocks.
Benchmark results when loading the village scene of sprite fright on AMD Ryzen 7 3800X 8-Core Processor
Before this patch 115 seconds
When patch applied less than 43 seconds
There is still some room for improvement by porting relink code.
Reviewed By: mont29
Maniphest Tasks: T95279
Differential Revision: https://developer.blender.org/D14043
Adds helper functions to debug IDRemapper data structure.
`BKE_id_remapper_result_string` converst a given IDRemapperApplyResult
to a readable form for logging purposes.
`BKE_id_remapper_print` prints out the rules inside a IDRemapper struct.
Use a shorter/simpler license convention, stops the header taking so
much space.
Follow the SPDX license specification: https://spdx.org/licenses
- C/C++/objc/objc++
- Python
- Shell Scripts
- CMake, GNUmakefile
While most of the source tree has been included
- `./extern/` was left out.
- `./intern/cycles` & `./intern/atomic` are also excluded because they
use different header conventions.
doc/license/SPDX-license-identifiers.txt has been added to list SPDX all
used identifiers.
See P2788 for the script that automated these edits.
Reviewed By: brecht, mont29, sergey
Ref D14069
Part of T91671.
Not much else to say, this is mainly a massive deletion of code.
Note that a few cleanups possible after this proxy removal were kept out
of this commit to try to reduce a bit its size.
Reviewed By: sergey, brecht
Maniphest Tasks: T91671
Differential Revision: https://developer.blender.org/D13995
During sprite fright loading of complex scenes would spend a long time in remapping ID's
The remapping process is done on a per ID instance that resulted in a very time consuming
process that goes over every possible ID reference to find out if it needs to be updated.
If there are N of references to ID blocks and there are M ID blocks that needed to be remapped
it would take N*M checks. These checks are scattered around the place and memory.
Each reference would only be updated at most once, but most of the time no update is needed at all.
Idea: By grouping the changes together will reduce the number of checks resulting in improved performance.
This would only require N checks. Additional benefits is improved data locality as data is only loaded once
in the L2 cache.
It has be implemented for the resyncing process and UI editors.
On an Intel(R) Core(TM) i7-6700 CPU @ 3.40GHz 16Gig the resyncing process went
from 170 seconds to 145 seconds (during hotspot recording).
After this patch has been applied we could add similar approach
to references (references between data blocks) and functionality (tagged deletion).
In my understanding this could reduce the resyncing process to less than a second.
Opening the village production file between 10 and 20 seconds.
Flame graphs showing that UI remapping isn't visible anymore (`WM_main_remap_editor_id_reference`)
* Master {F12769210 size=full}
* This patch {F12769211 size=full}
Reviewed By: mont29
Maniphest Tasks: T94185
Differential Revision: https://developer.blender.org/D13615
This reverts commit 948211679f.
This commit introduced some regressions in the test suite.
As this change is a core part of blender Bastien and I decided to revert
it as the solution isn't clear and needs more investigation.
The following tests FAILED:
62 - blendfile_liblink (SEGFAULT)
63 - blendfile_library_overrides (SEGFAULT)
It fails in (id_us_ensure_real)
During sprite fright loading of complex scenes would spend a long time in remapping ID's
The remapping process is done on a per ID instance that resulted in a very time consuming
process that goes over every possible ID reference to find out if it needs to be updated.
If there are N of references to ID blocks and there are M ID blocks that needed to be remapped
it would take N*M checks. These checks are scattered around the place and memory.
Each reference would only be updated at most once, but most of the time no update is needed at all.
Idea: By grouping the changes together will reduce the number of checks resulting in improved performance.
This would only require N checks. Additional benefits is improved data locality as data is only loaded once
in the L2 cache.
It has be implemented for the resyncing process and UI editors.
On an Intel(R) Core(TM) i7-6700 CPU @ 3.40GHz 16Gig the resyncing process went
from 170 seconds to 145 seconds (during hotspot recording).
After this patch has been applied we could add similar approach
to references (references between data blocks) and functionality (tagged deletion).
In my understanding this could reduce the resyncing process to less than a second.
Opening the village production file between 10 and 20 seconds.
Flame graphs showing that UI remapping isn't visible anymore (`WM_main_remap_editor_id_reference`)
* Master {F12769210 size=full}
* This patch {F12769211 size=full}
Reviewed By: mont29
Maniphest Tasks: T94185
Differential Revision: https://developer.blender.org/D13615
- Added space below non doc-string comments to make it clear
these aren't comments for the symbols directly below them.
- Use doxy sections for some headers.
- Minor improvements to doc-strings.
Ref T92709
rB43bc494892c3 switched `BKE_libblock_relink_to_newid` to use new ID
remapping and libquery code.
However, that new code does protect by default against remapping an
objects's data pointer when that object is in Edit mode, since this is
not a behavior that generic BKE code can handle (due to required editing
data for most obdata types when in edit mode).
So specific code that does create new IDs and need remapping in Edit
mode has to pass specific exception flags to remaping code.
This commit adds those remapping flags to `BKE_libblock_relink_to_newid`
and add said exception flag to the remapping call from
`ED_object_add_duplicate` when the object is in edit mode.
In theory we should never allow remapping of Objects' obdata ID pointer
when the object is in Edit mode. But there are some cases were this is
needed, so adding yet another exception option to remapping flags.
Preliminary change to fix T92629.
Current `BKE_libblock_relink_to_newid` is using its own simplistic,
limited and not really correct version of ID remapping.
While doing a full replacement would have been ideal, this is
risky/time-constrained for Blender 3.0 release, so for now we'll have
both versions co-existing.
While indeally we should only skip refcounting when relevant tag is set,
doing this in remapping code is too risky for now.
Related to previous commit and T88555.
In some cases (advanced, low-level), we also want to remap pointers like
`ID.newid` or `ID.orig_id`.
Only known case currently is `id_delete`, to avoid leaving potential access to freed memory. See next commit and T86501.
Keep the pointer location from the initial window-manager
between file load operations.
This is needed as the Python API may hold references to keymaps for e.g.
which are transferred to the newly loaded window manager,
without their `PointerRNA.owner_id` fields being updated.
Since there is only ever one window manager, keep the memory at the same location so the Python ID pointers stay valid.
Reviewed By: mont29
Ref D10690
This replaces header include guards with `#pragma once`.
A couple of include guards are not removed yet (e.g. `__RNA_TYPES_H__`),
because they are used in other places.
This patch has been generated by P1561 followed by `make format`.
Differential Revision: https://developer.blender.org/D8466
Surrounding includes with an 'extern "C"' block is not necessary anymore.
Also that made it harder to add any C++ code to some headers, or include headers
that have "optional" C++ code like `MEM_guardedalloc.h`.
I tested compilation on linux and windows (and got help from @LazyDodo).
If this still breaks compilation due to some linker error, the header containing
the symbol in question is probably missing an 'extern "C"' block.
Differential Revision: https://developer.blender.org/D7653
