OVERVIEW
* In 2.7 terminology, all layers and groups are now collection datablocks.
* These collections are nestable, linkable, instanceable, overrideable, ..
which opens up new ways to set up scenes and link + override data.
* Viewport/render visibility and selectability are now a part of the collection
and shared across all view layers and linkable.
* View layers define which subset of the scene collection hierarchy is excluded
for each. For many workflows one view layer can be used, these are more of an
advanced feature now.
OUTLINER
* The outliner now has a "View Layer" display mode instead of "Collections",
which can display the collections and/or objects in the view layer.
* In this display mode, collections can be excluded with the right click menu.
These will then be greyed out and their objects will be excluded.
* To view collections not linked to any scene, the "Blender File" display mode
can be used, with the new filtering option to just see Colleciton datablocks.
* The outliner right click menus for collections and objects were reorganized.
* Drag and drop still needs to be improved. Like before, dragging the icon or
text gives different results, we'll unify this later.
LINKING AND OVERRIDES
* Collections can now be linked into the scene without creating an instance,
with the link/append operator or from the collections view in the outliner.
* Collections can get static overrides with the right click menu in the outliner,
but this is rather unreliable and not clearly communicated at the moment.
* We still need to improve the make override operator to turn collection instances
into collections with overrides directly in the scene.
PERFORMANCE
* We tried to make performance not worse than before and improve it in some
cases. The main thing that's still a bit slower is multiple scenes, we have to
change the layer syncing to only updated affected scenes.
* Collections keep a list of their parent collections for faster incremental
updates in syncing and caching.
* View layer bases are now in a object -> base hash to avoid quadratic time
lookups internally and in API functions like visible_get().
VERSIONING
* Compatibility with 2.7 files should be improved due to the new visibility
controls. Of course users may not want to set up their scenes differently
now to avoid having separate layers and groups.
* Compatibility with 2.8 is mostly there, and was tested on Eevee demo and Hero
files. There's a few things which are know to be not quite compatible, like
nested layer collections inside groups.
* The versioning code for 2.8 files is quite complicated, and isolated behind
#ifdef so it can be removed at the end of the release cycle.
KNOWN ISSUES
* The G-key group operators in the 3D viewport were left mostly as is, they
need to be modified still to fit better.
* Same for the groups panel in the object properties. This needs to be updated
still, or perhaps replaced by something better.
* Collections must all have a unique name. Less restrictive namespacing is to
be done later, we'll have to see how important this is as all objects within
the collections must also have a unique name anyway.
* Full scene copy and delete scene are exactly doing the right thing yet.
Differential Revision: https://developer.blender.org/D3383https://code.blender.org/2018/05/collections-and-groups/
The RenderResult struct still has a listbase of RenderLayer, but that's ok
since this is strictly for rendering.
* Subversion bump (to 2.80.2)
* DNA low level doversion (renames) - only for .blend created since 2.80 started
Note: We can't use DNA_struct_elem_find or get file version in init_structDNA,
so we are manually iterating over the array of the SDNA elements instead.
Note 2: This doversion change with renames can be reverted in a few months. But
so far it's required for 2.8 files created between October 2016 and now.
Reviewers: campbellbarton, sergey
Differential Revision: https://developer.blender.org/D2927
We do have an history of those pieces of evil in our code, would be nice
to get fully rid of it, but at the very least let's not add more of them
in new code. :)
Previous code (same as what `BKE_libblock_free_us()` is doing when
usercount reach 0) was probably OK in that specific case, but still not
good idea, and potentially risky.
Design Documents
----------------
* https://wiki.blender.org/index.php/Dev:2.8/Source/Layers
* https://wiki.blender.org/index.php/Dev:2.8/Source/DataDesignRevised
User Commit Log
---------------
* New Layer and Collection system to replace render layers and viewport layers.
* A layer is a set of collections of objects (and their drawing options) required for specific tasks.
* A collection is a set of objects, equivalent of the old layers in Blender. A collection can be shared across multiple layers.
* All Scenes have a master collection that all other collections are children of.
* New collection "context" tab (in Properties Editor)
* New temporary viewport "collections" panel to control per-collection
visibility
Missing User Features
---------------------
* Collection "Filter"
Option to add objects based on their names
* Collection Manager operators
The existing buttons are placeholders
* Collection Manager drawing
The editor main region is empty
* Collection Override
* Per-Collection engine settings
This will come as a separate commit, as part of the clay-engine branch
Dev Commit Log
--------------
* New DNA file (DNA_layer_types.h) with the new structs
We are replacing Base by a new extended Base while keeping it backward
compatible with some legacy settings (i.e., lay, flag_legacy).
Renamed all Base to BaseLegacy to make it clear the areas of code that
still need to be converted
Note: manual changes were required on - deg_builder_nodes.h, rna_object.c, KX_Light.cpp
* Unittesting for main syncronization requirements
- read, write, add/copy/remove objects, copy scene, collection
link/unlinking, context)
* New Editor: Collection Manager
Based on patch by Julian Eisel
This is extracted from the layer-manager branch. With the following changes:
- Renamed references of layer manager to collections manager
- I doesn't include the editors/space_collections/ draw and util files
- The drawing code itself will be implemented separately by Julian
* Base / Object:
A little note about them. Original Blender code would try to keep them
in sync through the code, juggling flags back and forth. This will now
be handled by Depsgraph, keeping Object and Bases more separated
throughout the non-rendering code.
Scene.base is being cleared in doversion, and the old viewport drawing
code was poorly converted to use the new bases while the new viewport
code doesn't get merged and replace the old one.
Python API Changes
------------------
```
- scene.layers
+ # no longer exists
- scene.objects
+ scene.scene_layers.active.objects
- scene.objects.active
+ scene.render_layers.active.objects.active
- bpy.context.scene.objects.link()
+ bpy.context.scene_collection.objects.link()
- bpy_extras.object_utils.object_data_add(context, obdata, operator=None, use_active_layer=True, name=None)
+ bpy_extras.object_utils.object_data_add(context, obdata, operator=None, name=None)
- bpy.context.object.select
+ bpy.context.object.select = True
+ bpy.context.object.select = False
+ bpy.context.object.select_get()
+ bpy.context.object.select_set(action='SELECT')
+ bpy.context.object.select_set(action='DESELECT')
-AddObjectHelper.layers
+ # no longer exists
```
This is in fact very hairy situation here... Objects are only refcounted by scenes,
any other usage is 'free', which means once all object instanciations are gone Blender
considers it can delete it.
There is a trap here though: indirect usages. Typically, we should never modify linked data
(because it is essencially useless, changes would be ignored and ost on next reload or
even undo/redo). This means indirect usages are not affected by default 'safe' remapping/unlinking.
For unlinking preceeding deletion however, this is not acceptable - we are likely to end with
a zero-user ID (aka deletable one) which is still actually used by other linked data.
Solution choosen here is double:
I) From 'user-space' (i.e. outliner, operators...), we check for cases where deleting datablocks
should not be allowed (indirect data or indirectly used data), and abort (with report) if needed.
II) From 'lower' level (BKE_library_remap and RNA), we also unlink from linked data,
which makes actual deletion possible and safe.
Note that with previous behavior (2.77 one), linked object would be deleted, including from linked data -
but then, once file is saved and reloaded, indirect usage would link back the deleted object,
without any instanciation in scene, which made it somehow virtual and unreachable...
With new behavior, this is no more possible, but on the other hand it means that in situations of dependency cycles
(two linked objects using each other), linked objects become impossible to delete (from user space).
Not sure what's best here, behavior with those corner cases of library linking is very poorly defined... :(
This commit changes a lot of how IDs are handled internally, especially the unlinking/freeing
processes. So far, this was very fuzy, to summarize cleanly deleting or replacing a datablock
was pretty much impossible, except for a few special cases.
Also, unlinking was handled by each datatype, in a rather messy and prone-to-errors way (quite
a few ID usages were missed or wrongly handled that way).
One of the main goal of id-remap branch was to cleanup this, and fatorize ID links handling
by using library_query utils to allow generic handling of those, which is now the case
(now, generic ID links handling is only "knwon" from readfile.c and library_query.c).
This commit also adds backends to allow live replacement and deletion of datablocks in Blender
(so-called 'remapping' process, where we replace all usages of a given ID pointer by a new one,
or NULL one in case of unlinking).
This will allow nice new features, like ability to easily reload or relocate libraries, real immediate
deletion of datablocks in blender, replacement of one datablock by another, etc.
Some of those are for next commits.
A word of warning: this commit is highly risky, because it affects potentially a lot in Blender core.
Though it was tested rather deeply, being totally impossible to check all possible ID usage cases,
it's likely there are some remaining issues and bugs in new code... Please report them! ;)
Review task: D2027 (https://developer.blender.org/D2027).
Reviewed by campbellbarton, thanks a bunch.
Added a small menu with a few helper oerators next to each group panel:
* Remove group from all objects
* Select objects in group
More could be added possibly in the future.
Thanks to Campbell for the advice here.
before entering the recursion check.
Now use group tags instead of object tags, which could be a little more
efficient and was used before this patch too.
The new recursion check for groups duplicating themselves has to un-set the LIB_DOIT flag after each object, otherwise it will prevent duplicating a (non-recursive) group multiple times.
rB568f0c7 added a recursion check that is supposed to prevent cyclic
cases where a group includes itself via dupli instancing.
The check function was descending into all groups nested inside the
target group - which works for single level recursion like in the test
case, but does not handle generic recursion. Basically it asked:
"is object X in the group already or in any instanced dupligroup?"
The new check instead asks:
"is group G dupli'd by X or any instanced subgroup thereof?"
which is what we really need to know.
Add Selected to Active Group presented a menu with groups to choose from, while
this seemingly inverse operation did not and used all groups of the active
object.
Now both operators present a menu, with as first option "All Groups".
when the group contained two objects duplicating the same group. Also added the
dependency check to the "add to active group" operator now for consistency.
Couple of issues here:
- User shouldn't be able to run into dupligroup recursion.
It was checking already when setting a group for dupli.
Added check to operator which adds object to group.
- It's still possible files with recursion are hanging around,
so made simplify function robust to such kind of crap.
besides performance in some cases.
* DAG_scene_sort is now removed and replaced by DAG_relations_tag_update in
most cases. This will clear the dependency graph, and only rebuild it right
before it's needed again when the scene is re-evaluated.
This is done because DAG_scene_sort is slow when called many times from
python operators. Further the scene argument is not needed because most
operations can potentially affect more than the current scene.
* DAG_scene_relations_update will now rebuild the dependency graph if it's not
there yet, and DAG_scene_relations_rebuild will force a rebuild for the rare
cases that need it.
* Remove various places where ob->recalc was set manually. This should go
through DAG_id_tag_update() in nearly all cases instead since this is now
a fast operation. Also removed DAG_ids_flush_update that goes along with
such manual tagging of ob->recalc.
Issue was caused by couple of circumstances:
- Normal Map node requires tesselated faces to compute tangent space
- All temporary meshes needed for Cycles export were adding to G.main
- Undo pushes would temporary set meshes tessfaces to NULL
- Moving node will cause undo push and tree re-evaluate fr preview
All this leads to threading conflict between preview render and undo
system.
Solved it in way that all temporary meshes are adding to that exact
Main which was passed to Cycles via BlendData. This required couple
of mechanic changes like adding extra parameter to *_add() functions
and adding some *_ex() functions to make it possible RNA adds objects
to Main passed to new() RNA function.
This was tricky to pass Main to RNA function and IMO that's not so
nice to pass main to function, so ended up with such decision:
- Object.to_mesh() will add temp mesh to G.main
- Added Main.meshes.new_from_object() which does the same as to_mesh,
but adds temporary mesh to specified Main.
So now all temporary meshes needed for preview render would be added
to preview_main which does not conflict with undo pushes.
Viewport render shall not be an issue because object sync happens from
main thread in this case.
It could be some issues with final render, but that's not so much
likely to happen, so shall be fine.
Thanks to Brecht for review!
