This only applies to ID being copied outside of bmain. Handy for cases when it
is important to check if the copy corresponds to a data block coming from
library.
Example of that is proxy evaluation with copy on write.
Thanks Bastien for review!
This will allow much finer controll over how we copy data-blocks, from
full copy in Main database, to "lighter" ones (out of Main, inside an
already allocated datablock, etc.).
This commit also transfers a llot of what was previously handled by
per-ID-type custom code to generic ID handling code in BKE_library.
Hopefully will avoid in future inconsistencies and missing bits we had
all over the codebase in the past.
It also adds missing copying handling for a few types, most notably
Scene (which where using a fully customized handling previously).
Note that the type of allocation used during copying (regular in Main,
allocated but outside of Main, or not allocated by ID handling code at
all) is stored in ID's, which allows to handle them correctly when
freeing. This needs to be taken care of with caution when doing 'weird'
unusual things with ID copying and/or allocation!
As a final note, while rather noisy, this commit will hopefully not
break too much existing branches, old 'API' has been kept for the main
part, as a wrapper around new code. Cleaning it up will happen later.
Design task : T51804
Phab Diff: D2714
Noisy change, but safe, and better do it sooner than later if we are to
rework copying code. Also, previous commit shows this *is* useful to
catch some mistakes.
The absence of datablock properties "will certainly be resolved soon as the need for them is becoming obvious" said the [[http://wiki.blender.org/index.php/Dev:Ref/Release_Notes/2.67/Python_Nodes|Python Nodes release notes]]. So this patch allows Python scripts to create ID Properties which reference datablocks.
This functionality is implemented for `PointerProperty` and now such properties can be created with Python.
In addition to the standard update callback, `PointerProperty` can have a `poll` callback (standard RNA) which is useful for search menus. For details see the test included in this patch.
Original author: @artfunkel
Alexander (Blend4Web Team)
Reviewers: brecht, artfunkel, mont29, campbellbarton
Reviewed By: mont29, campbellbarton
Subscribers: jta, sergey, campbellbarton, wisaac, poseidon4o, mont29, homyachetser, Evgeny_Rodygin, AlexKowel, yurikovelenov, fjuhec, sharlybg, cardboard, duarteframos, blueprintrandom, a.romanov, BYOB, disnel, aditiapratama, bliblubli, dfelinto, lukastoenne
Maniphest Tasks: T37754
Differential Revision: https://developer.blender.org/D113
Use Main->relations in BKE_library_foreach_ID_link(), when possible
(i.e. IDWALK_READONLY is set), and if the data is available of course.
This is quite minor optimization, no sensible improvements are expected,
but does not hurt either to avoid potentially tens of looping over e.g.
objects constraints and modifiers, or heap of drivers...
The new MainIDRelations stores two mappings, one from ID users to ID
used, the other vice-versa.
That data is assumed to be short-living runtime, code creating it is
responsible to clear it asap. It will be much useful in places where we
handle relations between IDs for a lot of them at once.
Note: This commit is not fully functional, that is, the infamous, ugly,
PoS non-ID nodetrees will not be handled correctly when building relations.
Fix needed here is a bit noisy, so will be done in next own commit.
Was a waaaaayyyyy to much generic name for such a specific func, renamed
to much more descriptive BKE_libblock_relink_to_newid().
In near future (few weeks, to limit as much as possible silent mismatch
in branches), will rename BKE_libblock_relink_ex to BKE_libblock_relink,
this is the real generic data-block relinking func!
This aims at always ensuring that ID.newid (and relevant LIB_TAG_NEW)
stay in clean (i.e. cleared) state by default.
To achieve this, instead of clearing after all id copy call (would be
horribly noisy, and bad for performances), we try to completely remove
the setting of id->newid by default when copying a new ID.
This implies that areas actually needing that info (mainly, object editing
area (make single user...) and make local area) have to ensure they set
it themselves as needed.
This is far from simple change, many complex code paths to consider, so
will need some serious testing. :/
This has nothing to do here (freeing is not unlinking/remapping!), and
was actually redoing something already taken care of by
`BKE_libblock_relink_ex()` call in `BKE_libblock_free_ex()`.
Also, gives some noticeable speedup when removing datablocks with
do_unlink=True, about 5 to 10% quicker e.g. when deleting all objects
from a py console, in a big production file...
Added BKE_libblock_free_data_ex() which takes special do_id_user
argument which basically indicates whether main database was already
taken care about not having "dead" pointers.
Gives about 40% speedup of main database free with quadbot scene
(3.4sec vs. 5.4 sec on quite powerful desktop).
Issue here was that py API code was keeping references (pointers) to the
liniked data-blocks, which can actually be duplicated and then deleted
during the 'make local' process...
Would have like to find a better way than passing optional GHash to get
the oldid->newid mapping, but could not think of a better idea.
this patch resolves the following warnings;
```
Warning C4028 formal parameter 1 different from declaration blenkernel\intern\ocean.c 764
Warning C4098 'attach_stabilization_baseline_data': 'void' function returning a value blenkernel\intern\tracking_stabilize.c 139
Warning C4028 formal parameter 3 different from declaration blenkernel\intern\cachefile.c 148
Warning C4028 formal parameter 3 different from declaration blenkernel\intern\paint.c 413
Warning C4028 formal parameter 1 different from declaration blenkernel\intern\editderivedmesh.c 591
Warning C4028 formal parameter 3 different from declaration blenkernel\intern\library_remap.c 709
Warning C4028 formal parameter 1 different from declaration blenkernel\intern\ocean.c 754
Warning C4028 formal parameter 1 different from declaration blenkernel\intern\ocean.c 758
Warning C4028 formal parameter 1 different from declaration blenkernel\intern\ocean.c 759
Warning C4028 formal parameter 1 different from declaration blenkernel\intern\ocean.c 763
Warning C4028 formal parameter 1 different from declaration blenkernel\intern\ocean.c 764
Warning C4028 formal parameter 1 different from declaration blenkernel\intern\ocean.c 765
Warning C4028 formal parameter 1 different from declaration blenkernel\intern\ocean.c 769
Warning C4028 formal parameter 1 different from declaration blenkernel\intern\ocean.c 770
Warning C4028 formal parameter 1 different from declaration blenkernel\intern\DerivedMesh.c 3458
```
It's mostly things where the signature in the .h and the actual implementation in the .c do not match. And a bunch functions who do not match the TaskRunFunction declaration cause they leave out the __restrict keyword.
Reviewers: brecht, juicyfruit, sergey
Reviewed By: sergey
Subscribers: Blendify
Differential Revision: https://developer.blender.org/D2268
All in all, this patch adds an Alembic importer, an Alembic exporter,
and a new CacheFile data block which, for now, wraps around an Alembic
archive. This data block is made available through a new modifier ("Mesh
Sequence Cache") as well as a new constraint ("Transform Cache") to
somewhat properly support respectively geometric and transformation data
streaming from alembic caches.
A more in-depth documentation is to be found on the wiki, as well as a
guide to compile alembic: https://wiki.blender.org/index.php/
User:Kevindietrich/AlembicBasicIo.
Many thanks to everyone involved in this little project, and huge shout
out to "cgstrive" for the thorough testings with Maya, 3ds Max, Houdini
and Realflow as well as @fjuhec, @jensverwiebe and @jasperge for the
custom builds and compile fixes.
Reviewers: sergey, campbellbarton, mont29
Reviewed By: sergey, campbellbarton, mont29
Differential Revision: https://developer.blender.org/D2060
Turns out most BKE_foo_make_local datablock-specific functions are actually doing
exactly the same thing, only two currently need special additional operations
(object and brush ones). So added a BKE_id_make_local_generic instead
of copying same code over and over.
Also, changed a bit how make_local works in case we are localizing a whole library.
We need to do the 'remap' step (from old linked ID to new local one) in the second loop,
otherwise we miss some dependencies. This fixes main part of T48907.
Idea looked good, but we have too much custom situations here (some half-fake-sub-ID
being copied with their 'owner', animdata, etc.), let's let datablock copy functions
handle that themselves.
Also allows to safely call BKE_id_expand_local from all copy functions now (only when
copying linked data).
will be used by both make_local() and copy() datablock functions.
Note that this new func make 'extern' all IDs used by localized datablock,
not only refcounted ones as it used to be in each type's functions (with a few exceptions).
Totally stupid to not pass it, and then let (some) BKE_foo_make_local() use G.main!
Note: unused for now, much more refactoring still to come in make_local area!
This commit changes a lot of how IDs are handled internally, especially the unlinking/freeing
processes. So far, this was very fuzy, to summarize cleanly deleting or replacing a datablock
was pretty much impossible, except for a few special cases.
Also, unlinking was handled by each datatype, in a rather messy and prone-to-errors way (quite
a few ID usages were missed or wrongly handled that way).
One of the main goal of id-remap branch was to cleanup this, and fatorize ID links handling
by using library_query utils to allow generic handling of those, which is now the case
(now, generic ID links handling is only "knwon" from readfile.c and library_query.c).
This commit also adds backends to allow live replacement and deletion of datablocks in Blender
(so-called 'remapping' process, where we replace all usages of a given ID pointer by a new one,
or NULL one in case of unlinking).
This will allow nice new features, like ability to easily reload or relocate libraries, real immediate
deletion of datablocks in blender, replacement of one datablock by another, etc.
Some of those are for next commits.
A word of warning: this commit is highly risky, because it affects potentially a lot in Blender core.
Though it was tested rather deeply, being totally impossible to check all possible ID usage cases,
it's likely there are some remaining issues and bugs in new code... Please report them! ;)
Review task: D2027 (https://developer.blender.org/D2027).
Reviewed by campbellbarton, thanks a bunch.
Previous to 2.77, this used to be default behavior, was changed in rB591f4549c958b.
However, in most append cases, you do want a full localization of your data, so this new behavior
is kept by default, but there is now an option in append operator to only localize the 'first level'
of data (i.e. datablocks from linked library itself, and not those from other 'sub-libraries').
This has several benefits:
* User count remains coherent, regardless of the order in which you use 'user_one' & real refcounting users
(i.e. if you add to group, and then link in scene, or the reverse, you now always get same final user count).
* Avoids the need to check for potential 'user_one' extra user in count in several places in code (e.g. when
making IDs single users...).
* Users won't wonder why they cannot make 'single user' and ID even though its user count shows '2'!
* readfile.c now always uses code from BKE's library.c when modifying id->us. Which means we can consider
(asside from assignment during initialization) that id->us is read-only outside of library.c context.
Note that this commit reverts previous one (rB6b1d77a8052b) - please **do not** backport this one in 2.77.
Also re-reported through IRC by Thomas Beck (@plasmasolutions), thanks.
Though it's not ideal in theory, we have quite poor handling of object datablock currently
from user PoV - before this commit, it was not easily possible to get fully rid of an object
anymore if you did not removed it from all its groups before deleting it.
So for now, restore 2.76 behavior (namely, unlink an object from avaerything in Blender
once it is no more used by any scene).
Better handling of all this is TODO for later (also related to much more heavy changes
done in id-remap branch regarding sanitizing our ID deletion process).
BKE_main_id_tag_/BKE_main_id_flag_ were horrible naming now that we split those
into flags (for presistent one) and tags (for runtime ones).
Got rid of previous 'tag_' functions behavior (those who were dedicated shortcuts
to set/clear LIB_TAG_DOIT), so now '_tag_' functions affect tags, and '_flag_'
functions affect flags.
* Don't copy name before entering new_id function. new_id does that for
us already.
* Take a main argument to make the function possible to use with
different databases
* Append BKE_ to rename_id
This is purely internal sanitizing/cleanup, no change in behavior is expected at all.
This change was also needed because we were getting short on ID flags, and
future enhancement of 'user_one' ID behavior requires two new ones.
id->flag remains for persistent data (fakeuser only, so far!), this also allows us
100% backward & forward compatibility.
New id->tag is used for most flags. Though written in .blend files, its content
is cleared at read time.
Note that .blend file version was bumped, so that we can clear runtimeflags from
old .blends, important in case we add new persistent flags in future.
Also, behavior of tags (either status ones, or whether they need to be cleared before/after use)
has been added as comments to their declaration.
Reviewers: sergey, campbellbarton
Differential Revision: https://developer.blender.org/D1683
Idea is, instead of ignoring completely missing linked datablocks, to
create void placeholders for them.
That way, you can work on your file, save it, and find again your missing data once
lib becomes available again. Or you can edit missing lib's path (in Outliner),
save and reload the file, and you are done.
Also, Outliner now shows broken libraries (and placeholders) with a 'broken lib' icon.
Future plans are also to be able to relocate missing libs and reload them at runtime.
Code notes:
- Placeholder ID is just a regular datablock of same type as expected linked one,
with 'default' data, and a LIB_MISSING bitflag set.
- To allow creation of such datablocks, creation of datablocks in BKE was split in two step:
+ Allocation of memory itself.
+ Setting of all internal data to default values.
See also the design task (T43351).
Reviewed by @campbellbarton, thanks a bunch!
Differential Revision: https://developer.blender.org/D1394
Duplicate wasn't updating links,
so duplicatinvg a objects would still point to the originals for curve-taper, texmesh, drivers.
Use generic id-looper to handle replacing data.
Primary goal of this commit is to fix an annoying issue - when processing and saving .blend
files in background mode you lose their thumbnails, since it can only be generated with
an OpenGL context.
Solution to that is to read .blend thumbnail while reading .blend file (only done in background
mode currently), and store it in Main struct.
Also, this lead to removing .blend file reading code from thumb_blend (no need to have doublons).
We now have a small interface in regular reading code area, which keeps it reasonbaly light
by only reading/parsing header info, and first few BHead blocks.
This makes code reading .blend thumbnail about 3 to 4 times slower than previous highly specialized
one in blend_thumb.c, but overall thumbnail generation of a big .blend files folder only grows
of about 1%, think we can bare with it.
Finally, since thumbnail is now optionally stored in Main struct, it makes it easy to allow user
to define their own custom one (instead of auto-generated one). RNA API for this was not added though,
accessing that kind of .blend meta-data has to be rethought a bit on a bigger level first.
Reviewers: sergey, campbellbarton
Subscribers: Severin, psy-fi
Differential Revision: https://developer.blender.org/D1469
Assumed the `reference` pointer is an ID, currently true, but may not always be.
Add a callback specifically for this purpose since cleaning up notifiers and space-types are different operations.
