The changes come from the `functions` branch, where I'm using
these structures a lot.
This also includes a new `BLI::Optional<T>` type, which is similar
to `std::Optional<T>` which can be used when Blender starts using
C++17.
This is a more correct fix to the issue Brecht was fixing in D6600.
While the fix in that patch worked fine for linking it broke ASAN
runtime under some circumstances.
For example, `make full debug developer` would compile, but trying
to start blender will cause assert failure in ASAN (related on check
that ASAN is not running already).
Top-level idea: leave it to CMake to keep track of dependency graph.
The root of the issue comes to the fact that target like "blender" is
configured to use a lot of static libraries coming from Blender sources
and to use external static libraries. There is nothing which ensures
order between blender's and external libraries. Only order of blender
libraries is guaranteed.
It was possible that due to a cycle or other circumstances some of
blender libraries would have been passed to linker after libraries
it uses, causing linker errors.
For example, this order will likely fail:
libbf_blenfont.a libfreetype6.a libbf_blenfont.a
This change makes it so blender libraries are explicitly provided
their dependencies to an external libraries, which allows CMake to
ensure they are always linked against them.
General rule here: if bf_foo depends on an external library it is
to be provided to LIBS for bf_foo.
For example, if bf_blenkernel depends on opensubdiv then LIBS in
blenkernel's CMakeLists.txt is to include OPENSUBDIB_LIBRARIES.
The change is made based on searching for used include folders
such as OPENSUBDIV_INCLUDE_DIRS and adding corresponding libraries
to LIBS ion that CMakeLists.txt. Transitive dependencies are not
simplified by this approach, but I am not aware of any downside of
this: CMake should be smart enough to simplify them on its side.
And even if not, this shouldn't affect linking time.
Benefit of not relying on transitive dependencies is that build
system is more robust towards future changes. For example, if
bf_intern_opensubiv is no longer depends on OPENSUBDIV_LIBRARIES
and all such code is moved to bf_blenkernel this will not break
linking.
The not-so-trivial part is change to blender_add_lib (and its
version in Cycles). The complexity is caused by libraries being
provided as a single list argument which doesn't allow to use
different release and debug libraries on Windows. The idea is:
- Have every library prefixed as "optimized" or "debug" if
separation is needed (non-prefixed libraries will be considered
"generic").
- Loop through libraries passed to function and do simple parsing
which will look for "optimized" and "debug" words and specify
following library to corresponding category.
This isn't something particularly great. Alternative would be to
use target_link_libraries() directly, which sounds like more code
but which is more explicit and allows to have more flexibility
and control comparing to wrapper approach.
Tested the following configurations on Linux, macOS and Windows:
- make full debug developer
- make full release developer
- make lite debug developer
- make lite release developer
NOTE: Linux libraries needs to be compiled with D6641 applied,
otherwise, depending on configuration, it's possible to run into
duplicated zlib symbols error.
Differential Revision: https://developer.blender.org/D6642
This commit adds some new hashing based data structures to blenlib.
All of them use open addressing with probing currently.
Furthermore, they support small object optimization, but it is not
customizable yet. I'll add support for this when necessary.
The following main data structures are included:
**Set**
A collection of values, where every value must exist at most once.
This is similar to a Python `set`.
**SetVector**
A combination of a Set and a Vector. It supports fast search for
elements and maintains insertion order when there are no deletes.
All elements are stored in a continuous array. So they can be
iterated over using a normal `ArrayRef`.
**Map**
A set of key-value-pairs, where every key must exist at most once.
This is similar to a Python `dict`.
**StringMap**
A special map for the case when the keys are strings. This case is
fairly common and allows for some optimizations. Most importantly,
many unnecessary allocations can be avoided by storing strings in
a single buffer. Furthermore, the interface of this class uses
`StringRef` to avoid unnecessary conversions.
This commit is a continuation of rB369d5e8ad2bb7.
These two data structures reference strings somewhere in memory.
They do not own the referenced string. The string is considered
const.
A string referenced by StringRefNull can be expected to be
null-terminated. That is not the case for StringRef.
This commit is a continuation of rB369d5e8ad2bb7c2.
Many generic C++ data structures have been developed in the
functions branch. This commit merges a first chunk of them into
master. The following new data structures are included:
Array: Owns a memory buffer with a fixed size. It is different
from std::array in that the size is not part of the type.
ArrayRef: References an array owned by someone else. All elements
in the referenced array are considered to be const. This should
be the preferred parameter type for functions that take arrays
as input.
MutableArrayRef: References an array owned by someone else. The
elements in the referenced array can be changed.
IndexRange: Specifies a continuous range of integers with a start
and end index.
IntrusiveListBaseWrapper: A utility class that allows iterating
over ListBase instances where the prev and next pointer are
stored in the objects directly.
Stack: A stack implemented on top of a vector.
Vector: An array that can grow dynamically.
Allocators: Three allocator types are included that can be used
by the container types to support different use cases.
The Stack and Vector support small object optimization. So when
the amount of elements in them is below a certain threshold, no
memory allocation is performed.
Additionally, most methods have unit tests.
I'm merging this without normal code review, after I checked the
code roughly with Sergey, and after we talked about it with Brecht.
Preparing for the bigger changes which will be related on passing
dependency graph to various callbacks which need it.
Differential Revision: https://developer.blender.org/D5725
See Design task T68277, and patch D5423.
This commit includes edits by @ideasman42 to patch in
branch temp-D5423-update, plus responses to his comments.
This is really close to BLI_mempool but uses an array to keep track of the
chunks of memory. There is no tagging necessary to clear the whole
structure so reuse is fast.
Naturally supports iteration but does not support freeing.
No functional change, this adds LIB definition and args to cmake files.
Without this it's difficult to migrate away from 'BLENDER_SORTED_LIBS'
since there are many platforms/configurations that could break when
changing linking order.
Manually add and enable WITHOUT_SORTED_LIBS to try building
without sorted libs (currently fails since all variables are empty).
This check will eventually be removed.
See T46725.
Some users of the 3D versions were storing 2D data in it.
Using a 3D tree for 2D data adds a spatially redundant branch
every 3rd level, as well as some extra memory use, best avoid this.
This moves logic into kdtree_impl.h which is included in a source
file that defines the number of dimensions - so we can easily support
different numbers of dimensions as needed
(currently 3D and 4D are supported).
Macro use isn't so nice but avoids a lot of duplicate code.
There is no point having operations that iterate over the whole
bit array as macros, so convert BLI_BITMAP_SET_ALL to a function.
Also, add more utilities for copying and manipulating masks.
Reviewers: brecht, campbellbarton
Differential Revision: https://developer.blender.org/D4101
The idea is to make main thread and job threads to be scheduled
on CPU dies which has direct access to memory (those are NUMA
nodes 0 and 2).
We also do this for new EPYC CPUs since their NUMA nodes 1 and 3
do have access but only to a higher range DDR slots. By preferring
nodes 0 and 2 on EPYC we make it so users with partially filled
DDR slots has fast memory access.
One thing which is not really solved yet is localization of
memory allocation: we do not guarantee that memory is allocated
on the closest to the NUMA node DDR slot and hope that memory
manager of OS is acting in favor of us.
There is a new `bpy.app.timers` api.
For more details, look in the Python API documentation.
Reviewers: campbellbarton
Differential Revision: https://developer.blender.org/D3994
Simple isn't a good prefix for library names since
lots of unrelated modules could be called 'simple'.
Include 'py' in module name since this is a subset of Python,
one of the main motivations for this is to be Python like/compatible.
Recently @sergey found that hard-coding evaluation of certain very
common driver expressions without calling the Python interpreter
produces a 30-40% performance improvement. Since hard-coding is
obviously not suitable for production, I implemented a proper
parser and interpreter for simple arithmetic expressions in C.
The evaluator supports +, -, *, /, (), ==, !=, <, <=, >, >=,
and, or, not, ternary if; driver variables, frame, pi, True, False,
and a subset of standard math functions that seem most useful.
Booleans are represented as numbers, since within the supported
operation set it seems to be impossible to distinguish True/False
from 1.0/0.0. Boolean operations properly implement lazy evaluation
with jumps, and comparisons support chaining like 'a < b < c...'.
Expressions are parsed into a very simple stack machine program
that can then be safely evaluated in multiple threads.
Reviewers: sergey, campbellbarton
Differential Revision: https://developer.blender.org/D3698
This patch adds a new matte node that implements the Cryptomatte specification.
It also incluces a custom eye dropper that works outside of a color picker.
Cryptomatte export for the Cycles render engine will be in a separate patch.
Reviewers: brecht
Reviewed By: brecht
Subscribers: brecht
Tags: #compositing
Differential Revision: https://developer.blender.org/D3531
While the feature is interesting, it's not much from what we can tell.
Retargeting is an important feature but needs
to fit in better with typical animation work-flows.
See: T52809
They are used to start and end colored output in console.
Use with care, it is up to you to check that console actually
supports Truecolor ANSII.
In thew future we can extend this to other consoles and platforms.
- Each allocation can be a different size
(but should be smaller than the chunk size).
- Result can be looped over in order of allocation.
- Allocations are aligned to pointer size to avoid unaligned reads.
Design Documents
----------------
* https://wiki.blender.org/index.php/Dev:2.8/Source/Layers
* https://wiki.blender.org/index.php/Dev:2.8/Source/DataDesignRevised
User Commit Log
---------------
* New Layer and Collection system to replace render layers and viewport layers.
* A layer is a set of collections of objects (and their drawing options) required for specific tasks.
* A collection is a set of objects, equivalent of the old layers in Blender. A collection can be shared across multiple layers.
* All Scenes have a master collection that all other collections are children of.
* New collection "context" tab (in Properties Editor)
* New temporary viewport "collections" panel to control per-collection
visibility
Missing User Features
---------------------
* Collection "Filter"
Option to add objects based on their names
* Collection Manager operators
The existing buttons are placeholders
* Collection Manager drawing
The editor main region is empty
* Collection Override
* Per-Collection engine settings
This will come as a separate commit, as part of the clay-engine branch
Dev Commit Log
--------------
* New DNA file (DNA_layer_types.h) with the new structs
We are replacing Base by a new extended Base while keeping it backward
compatible with some legacy settings (i.e., lay, flag_legacy).
Renamed all Base to BaseLegacy to make it clear the areas of code that
still need to be converted
Note: manual changes were required on - deg_builder_nodes.h, rna_object.c, KX_Light.cpp
* Unittesting for main syncronization requirements
- read, write, add/copy/remove objects, copy scene, collection
link/unlinking, context)
* New Editor: Collection Manager
Based on patch by Julian Eisel
This is extracted from the layer-manager branch. With the following changes:
- Renamed references of layer manager to collections manager
- I doesn't include the editors/space_collections/ draw and util files
- The drawing code itself will be implemented separately by Julian
* Base / Object:
A little note about them. Original Blender code would try to keep them
in sync through the code, juggling flags back and forth. This will now
be handled by Depsgraph, keeping Object and Bases more separated
throughout the non-rendering code.
Scene.base is being cleared in doversion, and the old viewport drawing
code was poorly converted to use the new bases while the new viewport
code doesn't get merged and replace the old one.
Python API Changes
------------------
```
- scene.layers
+ # no longer exists
- scene.objects
+ scene.scene_layers.active.objects
- scene.objects.active
+ scene.render_layers.active.objects.active
- bpy.context.scene.objects.link()
+ bpy.context.scene_collection.objects.link()
- bpy_extras.object_utils.object_data_add(context, obdata, operator=None, use_active_layer=True, name=None)
+ bpy_extras.object_utils.object_data_add(context, obdata, operator=None, name=None)
- bpy.context.object.select
+ bpy.context.object.select = True
+ bpy.context.object.select = False
+ bpy.context.object.select_get()
+ bpy.context.object.select_set(action='SELECT')
+ bpy.context.object.select_set(action='DESELECT')
-AddObjectHelper.layers
+ # no longer exists
```
Things like `BLI_uniquename` had nothing, but really nothing to do in
BLI_path_util files!
Also, got rid of length limitation in `BLI_uniquename_cb`, we can use
alloca here to avoid overhead of malloc while keeping free size (within
reasonable limits of course).
