Optimization attempt with BKE_library_idtype_can_use_idtype() was not taking into account
the fact that drivers may link virtually against any datablock...
Has to be rethinked, but for after 2.78 release, this commit is safe to backport.
All in all, this patch adds an Alembic importer, an Alembic exporter,
and a new CacheFile data block which, for now, wraps around an Alembic
archive. This data block is made available through a new modifier ("Mesh
Sequence Cache") as well as a new constraint ("Transform Cache") to
somewhat properly support respectively geometric and transformation data
streaming from alembic caches.
A more in-depth documentation is to be found on the wiki, as well as a
guide to compile alembic: https://wiki.blender.org/index.php/
User:Kevindietrich/AlembicBasicIo.
Many thanks to everyone involved in this little project, and huge shout
out to "cgstrive" for the thorough testings with Maya, 3ds Max, Houdini
and Realflow as well as @fjuhec, @jensverwiebe and @jasperge for the
custom builds and compile fixes.
Reviewers: sergey, campbellbarton, mont29
Reviewed By: sergey, campbellbarton, mont29
Differential Revision: https://developer.blender.org/D2060
Hating all those not-so-real ID types... Here there were two causes for the issue:
1) Linked shapekey ID was made local twice (once from mesh's make local, once by itself).
Solved by not explicitely making shapekeys (nor any other non-linkable datatype) local.
2) Key->from 'back pointer' to its owner was messing 'still in used' detection of linked data
after localization. Fixed with a hack for now, thinking correct solution might actually
be to not consider this pointer at all in libquery ID looper, since it's nothing like
and actual usage of mesh/lattice/curve.
Again, shapekeys as ID is a joke, those should be mere struct, they have absolutely nothing to do in Main, period. :(
Point 2) still demonstrates the need for better handling of IDs dependencies though,
so far we only hit corner cases, but think there could also be valid cases generating those
'dependency cycles' between IDs (ID a using ID b which uses ID a), this will have to be addressed some day...
Check to ensure we do not delete last instancing of an indirectly used object (forbidden
because it creates 'ghost' objects user have no real way to re-instance) was defeated by
the backward pointer 'proxy_from', which generated a 'false' indirect usage of (local)
proxy object itself (the one we are trying to delete).
Fixed by actually considering that ID usage as local if proxy object itself is local
(because that pointer actually does not 'pertain' to the object holding it, but to its proxy).
Yeah, it's... complicated, twisted and ugly - it's proxy.
libquery now passes an extra flag info to the callback, in case that specific
ID usage is considered as indirect.
In most cases, it's just set from ID_IS_LINKED_DATABLOCK() result on datablock owner,
but in proxy object case we also consider ob->data, materials and pose usages as indirect.
Does not fixes all issues yet, but should already make behavior with proxy object saner.
Not really happy with this, could lead to a **lot** of callback executions...
But libquery foreach looper must go over **all** ID pointers,
otherwise remapping & co is broken!
A possible fix for future would be to have 'image slots' defined at root of poly/facetex layers,
and just a (short) index in each face/poly item - would also save a reasonable amount of memory...
This should allow us to avoid a lot of useless processing when iterating over the
whole main database (unlink/remap/usages checks/etc.).
Note that some ID types report they can use any type for now, due to
fuzzyness/indefined nature of some usages (like constraints/modifiers/game logic,
or ID pointer of nodes...). Maybe we could address this (like e.g. adding defines
in relevant headers to restrict ID types used by constraints, by modifiers, etc.).
But don’t think this is top priority for now.
Idea is to replace hard-to-track (id->lib != NULL) 'is linked datablock' check everywhere in Blender
by a macro doing the same thing. This will allow to easily spot those checks in future, and more importantly,
to easily change it (see work done in asset-engine branch).
Note: did not touch to readfile.c, since there most of the time 'id->lib' check actually concerns the pointer,
and not a check whether ID is linked or not. Will have a closer look at it later.
Reviewers: campbellbarton, brecht, sergey
Differential Revision: https://developer.blender.org/D2082
This is in fact very hairy situation here... Objects are only refcounted by scenes,
any other usage is 'free', which means once all object instanciations are gone Blender
considers it can delete it.
There is a trap here though: indirect usages. Typically, we should never modify linked data
(because it is essencially useless, changes would be ignored and ost on next reload or
even undo/redo). This means indirect usages are not affected by default 'safe' remapping/unlinking.
For unlinking preceeding deletion however, this is not acceptable - we are likely to end with
a zero-user ID (aka deletable one) which is still actually used by other linked data.
Solution choosen here is double:
I) From 'user-space' (i.e. outliner, operators...), we check for cases where deleting datablocks
should not be allowed (indirect data or indirectly used data), and abort (with report) if needed.
II) From 'lower' level (BKE_library_remap and RNA), we also unlink from linked data,
which makes actual deletion possible and safe.
Note that with previous behavior (2.77 one), linked object would be deleted, including from linked data -
but then, once file is saved and reloaded, indirect usage would link back the deleted object,
without any instanciation in scene, which made it somehow virtual and unreachable...
With new behavior, this is no more possible, but on the other hand it means that in situations of dependency cycles
(two linked objects using each other), linked objects become impossible to delete (from user space).
Not sure what's best here, behavior with those corner cases of library linking is very poorly defined... :(
This commit changes a lot of how IDs are handled internally, especially the unlinking/freeing
processes. So far, this was very fuzy, to summarize cleanly deleting or replacing a datablock
was pretty much impossible, except for a few special cases.
Also, unlinking was handled by each datatype, in a rather messy and prone-to-errors way (quite
a few ID usages were missed or wrongly handled that way).
One of the main goal of id-remap branch was to cleanup this, and fatorize ID links handling
by using library_query utils to allow generic handling of those, which is now the case
(now, generic ID links handling is only "knwon" from readfile.c and library_query.c).
This commit also adds backends to allow live replacement and deletion of datablocks in Blender
(so-called 'remapping' process, where we replace all usages of a given ID pointer by a new one,
or NULL one in case of unlinking).
This will allow nice new features, like ability to easily reload or relocate libraries, real immediate
deletion of datablocks in blender, replacement of one datablock by another, etc.
Some of those are for next commits.
A word of warning: this commit is highly risky, because it affects potentially a lot in Blender core.
Though it was tested rather deeply, being totally impossible to check all possible ID usage cases,
it's likely there are some remaining issues and bugs in new code... Please report them! ;)
Review task: D2027 (https://developer.blender.org/D2027).
Reviewed by campbellbarton, thanks a bunch.
Some area were still not in sync with readfile.c, now should be better.
Note that readfile.c has been used as référence here re us refcounting,
not sure how accurate it is, time will say :|
This commit:
* Fixes bad handling of 'stop iteration' (by adding a status flag, so that we can actually
stop in helper functions too, and jumping to a finalize label instead of raw return, to
allow propper clean up).
* Adds optional recursion into 'ID tree' - callback can also decide to exclude current id_pp
from recursion. Note that this implies 'readonly', modifying IDs while recursing is not
something we want to support!
* Changes callback signature/expected behavior: return behavior is now handled through flags,
and 'parent' ID of id_pp is also passed (since it may not always be root id anymore).
Reviewers: sergey, campbellbarton
Differential Revision: https://developer.blender.org/D1869
This mainly adds bpy.data.user_map() method, which goes over the whole Main database
to build a mapping (dict) {ID: {users_of_that_ID}}.
Very handy to check and debug ID usages, but could also be really valuable for py addons
creating temporary scenes, or some exporters, etc.
Note: current code in master's libquery misses some IDs (and reports some it should not,
like nodetrees), this is fixed in id-remap but still needs serious review before going to master.
This basically means that current bpy.data.user_map() **will not** report a complete and exhaustive
state of dependencies between IDs. Should work OK in most cases though.
Original work/idea comes from id-remap branch, was heavily reworked by @campbellbarton
and myself for master.
Reviewers: campbellbarton, sergey
Differential Revision: https://developer.blender.org/D1678
Looping twice on same ID pointer may not be an issue in master currently, but with work done in id-remap
branch this should be avoided as much as possible, so for now assuming we do not need this here.
Note: if we really need this and have to add it back, then please at least use IDWALK_USER, and not
IDWALK_NOP flag!
As we can see in (original) read code, ob pointer in groupobject is actually a 'USER_ONE' case.
This was not done in 'add object to group' code, probably because we can assume objects always
have at least one user in that case? Made it explicit now. Also fixed foreach_ID_link looper.
In general we have waaayyyyyy too much 'own handling' of ID->us count in code currently,
will clean up that...
Duplicate wasn't updating links,
so duplicatinvg a objects would still point to the originals for curve-taper, texmesh, drivers.
Use generic id-looper to handle replacing data.
This func is long enough, there's no real need to make it even longer with
verbose local varnames and multi-line for loops...
Also, avoid mono-leters names as well for data pointers.
