Previously, the lifetimes of anonymous attributes were determined by
reference counts which were non-deterministic when multiple threads
are used. Now the lifetimes of anonymous attributes are handled
more explicitly and deterministically. This is a prerequisite for any kind
of caching, because caching the output of nodes that do things
non-deterministically and have "invisible inputs" (reference counts)
doesn't really work.
For more details for how deterministic lifetimes are achieved, see D16858.
No functional changes are expected. Small performance changes are expected
as well (within few percent, anything larger regressions should be reported as
bugs).
Differential Revision: https://developer.blender.org/D16858
This reduces logging overhead. The performance difference is only
significant when there are many fast nodes. In my test file with many
math nodes, the performance improved from 720ms to 630ms.
This refactors the geometry nodes evaluation system. No changes for the
user are expected. At a high level the goals are:
* Support using geometry nodes outside of the geometry nodes modifier.
* Support using the evaluator infrastructure for other purposes like field evaluation.
* Support more nodes, especially when many of them are disabled behind switch nodes.
* Support doing preprocessing on node groups.
For more details see T98492.
There are fairly detailed comments in the code, but here is a high level overview
for how it works now:
* There is a new "lazy-function" system. It is similar in spirit to the multi-function
system but with different goals. Instead of optimizing throughput for highly
parallelizable work, this system is designed to compute only the data that is actually
necessary. What data is necessary can be determined dynamically during evaluation.
Many lazy-functions can be composed in a graph to form a new lazy-function, which can
again be used in a graph etc.
* Each geometry node group is converted into a lazy-function graph prior to evaluation.
To evaluate geometry nodes, one then just has to evaluate that graph. Node groups are
no longer inlined into their parents.
Next steps for the evaluation system is to reduce the use of threads in some situations
to avoid overhead. Many small node groups don't benefit from multi-threading at all.
This is much easier to do now because not everything has to be inlined in one huge
node tree anymore.
Differential Revision: https://developer.blender.org/D15914
The purpose of `NodeTreeRef` was to speed up various queries on a read-only
`bNodeTree`. Not that we have runtime data in nodes and sockets, we can also
store the result of some queries there. This has some benefits:
* No need for a read-only separate node tree data structure which increased
complexity.
* Makes it easier to reuse cached queries in more parts of Blender that can
benefit from it.
A downside is that we loose some type safety that we got by having different
types for input and output sockets, as well as internal and non-internal links.
This patch also refactors `DerivedNodeTree` so that it does not use
`NodeTreeRef` anymore, but uses `bNodeTree` directly instead.
To provide a convenient API (that is also close to what `NodeTreeRef` has), a
new approach is implemented: `bNodeTree`, `bNode`, `bNodeSocket` and `bNodeLink`
now have C++ methods declared in `DNA_node_types.h` which are implemented in
`BKE_node_runtime.hh`. To make this work, `makesdna` now skips c++ sections when
parsing dna header files.
No user visible changes are expected.
Differential Revision: https://developer.blender.org/D15491
Using the same `GeometryComponentFieldContext` for all situations,
even when only one geometry type is supported is misleading, and mixes
too many different abstraction levels into code that could be simpler.
With the attribute API moved out of geometry components recently,
the "component" system is just getting in the way here.
This commit adds specific field contexts for geometry types: meshes,
curves, point clouds, and instances. There are also separate field input
helper classes, to help reduce boilerplate for fields that only support
specific geometry types.
Another benefit of this change is that it separates geometry components
from fields, which makes it easier to see the purpose of the two concepts,
and how they relate.
Because we want to be able to evaluate a field on just `CurvesGeometry`
rather than the full `Curves` data-block, the generic "geometry context"
had to be changed to avoid using `GeometryComponent`, since there is
no corresponding geometry component type. The resulting void pointer
is ugly, but only turns up in three places in practice. When Apple clang
supports `std::variant`, that could be used instead.
Differential Revision: https://developer.blender.org/D15519
Previously, curves sculpt tools only worked on original data. This was
very limiting, because one could effectively only sculpt the curves when
all procedural effects were turned off. This patch adds support for curves
sculpting while looking the result of procedural effects (like deformation
based on the surface mesh). This functionality is also known as "crazy space"
support in Blender.
For more details see D15407.
Differential Revision: https://developer.blender.org/D15407
Currently, there are two attribute API. The first, defined in `BKE_attribute.h` is
accessible from RNA and C code. The second is implemented with `GeometryComponent`
and is only accessible in C++ code. The second is widely used, but only being
accessible through the `GeometrySet` API makes it awkward to use, and even impossible
for types that don't correspond directly to a geometry component like `CurvesGeometry`.
This patch adds a new attribute API, designed to replace the `GeometryComponent`
attribute API now, and to eventually replace or be the basis of the other one.
The basic idea is that there is an `AttributeAccessor` class that allows code to
interact with a set of attributes owned by some geometry. The accessor itself has
no ownership. `AttributeAccessor` is a simple type that can be passed around by
value. That makes it easy to return it from functions and to store it in containers.
For const-correctness, there is also a `MutableAttributeAccessor` that allows
changing individual and can add or remove attributes.
Currently, `AttributeAccessor` is composed of two pointers. The first is a pointer
to the owner of the attribute data. The second is a pointer to a struct with
function pointers, that is similar to a virtual function table. The functions
know how to access attributes on the owner.
The actual attribute access for geometries is still implemented with the `AttributeProvider`
pattern, which makes it easy to support different sources of attributes on a
geometry and simplifies dealing with built-in attributes.
There are different ways to get an attribute accessor for a geometry:
* `GeometryComponent.attributes()`
* `CurvesGeometry.attributes()`
* `bke::mesh_attributes(const Mesh &)`
* `bke::pointcloud_attributes(const PointCloud &)`
All of these also have a `_for_write` variant that returns a `MutabelAttributeAccessor`.
Differential Revision: https://developer.blender.org/D15280
This adds a new node editor overlay that helps users to see where
named attributes are used. This is important, because named
attributes can have name collisions between independent node
groups which can lead to hard to find issues.
Differential Revision: https://developer.blender.org/D14618
This commit moves declarations that depend on `FN_field.hh` out of
`BKE_geometry_set.hh` into `BKE_geometry_fields.hh`. This helps to
reduce the number of areas that need to depend on the functions module,
which recently came in in review of D11591.
In the future we may have a library of standard field inputs in order to
make composing algorithms easier, so it makes sense to have a header
that could contain them and some basic related utilities relating the
concepts of geometry and fields.
Reducing use of unnecessary headers may also reduce compilation time.
Differential Revision: https://developer.blender.org/D14517
Use a shorter/simpler license convention, stops the header taking so
much space.
Follow the SPDX license specification: https://spdx.org/licenses
- C/C++/objc/objc++
- Python
- Shell Scripts
- CMake, GNUmakefile
While most of the source tree has been included
- `./extern/` was left out.
- `./intern/cycles` & `./intern/atomic` are also excluded because they
use different header conventions.
doc/license/SPDX-license-identifiers.txt has been added to list SPDX all
used identifiers.
See P2788 for the script that automated these edits.
Reviewed By: brecht, mont29, sergey
Ref D14069
This adds `blender::is_same_any_v` which is the almost the same as
`std::is_same_v`. The difference is that it allows for checking multiple
types at the same time.
Differential Revision: https://developer.blender.org/D13673
This patch refactors the instance-realization code and adds new functionality.
* Named and anonymous attributes are propagated from instances to the
realized geometry. If the same attribute exists on the geometry and on an
instance, the attribute on the geometry has precedence.
* The id attribute has special handling to avoid creating the same id on many
output points. This is necessary to make e.g. the Random Value node work
as expected afterwards.
Realizing instance attributes has an effect on existing files, especially due to the
id attribute. To avoid breaking existing files, the Realize Instances node now has
a legacy option that is enabled for all already existing Realize Instances nodes.
Removing this legacy behavior does affect some existing files (although not many).
We can decide whether it's worth to remove the old behavior as a separate step.
This refactor also improves performance when realizing instances. That is mainly
due to multi-threading. See D13446 to get the file used for benchmarking. The
curve code is not as optimized as it could be yet. That's mainly because the storage
for these attributes might change soonish and it wasn't worth optimizing for the
current storage format right now.
```
1,000,000 x mesh vertex: 530 ms -> 130 ms
1,000,000 x simple cube: 1290 ms -> 190 ms
1,000,000 x point: 1000 ms -> 150 ms
1,000,000 x curve spiral: 1740 ms -> 330 ms
1,000,000 x curve line: 1110 ms -> 210 ms
10,000 x subdivided cylinder: 170 ms -> 40 ms
10 x subdivided spiral: 180 ms -> 180 ms
```
Differential Revision: https://developer.blender.org/D13446
Most of our field inputs are currently specific to geometry. This patch introduces
a new `GeometryFieldInput` that reduces the overhead of adding new geometry
field input.
Differential Revision: https://developer.blender.org/D13489
Currently the geometry nodes evaluator always stores a field for every
type that supports it, even if it is just a single value. This results in a lot
of overhead when there are many sockets that just contain a single
value, which is often the case.
This introduces a new `ValueOrField<T>` type that is used by the geometry
nodes evaluator. Now a field will only be created when it is actually
necessary. See D13307 for more details. In extrem cases this can speed
up the evaluation 2-3x (those cases are probably never hit in practice
though, but it's good to get rid of unnecessary overhead nevertheless).
Differential Revision: https://developer.blender.org/D13307
Goals of this refactor:
* Simplify creating virtual arrays.
* Simplify passing virtual arrays around.
* Simplify converting between typed and generic virtual arrays.
* Reduce memory allocations.
As a quick reminder, a virtual arrays is a data structure that behaves like an
array (i.e. it can be accessed using an index). However, it may not actually
be stored as array internally. The two most important implementations
of virtual arrays are those that correspond to an actual plain array and those
that have the same value for every index. However, many more
implementations exist for various reasons (interfacing with legacy attributes,
unified iterator over all points in multiple splines, ...).
With this refactor the core types (`VArray`, `GVArray`, `VMutableArray` and
`GVMutableArray`) can be used like "normal values". They typically live
on the stack. Before, they were usually inside a `std::unique_ptr`. This makes
passing them around much easier. Creation of new virtual arrays is also
much simpler now due to some constructors. Memory allocations are
reduced by making use of small object optimization inside the core types.
Previously, `VArray` was a class with virtual methods that had to be overridden
to change the behavior of a the virtual array. Now,`VArray` has a fixed size
and has no virtual methods. Instead it contains a `VArrayImpl` that is
similar to the old `VArray`. `VArrayImpl` should rarely ever be used directly,
unless a new virtual array implementation is added.
To support the small object optimization for many `VArrayImpl` classes,
a new `blender::Any` type is added. It is similar to `std::any` with two
additional features. It has an adjustable inline buffer size and alignment.
The inline buffer size of `std::any` can't be relied on and is usually too
small for our use case here. Furthermore, `blender::Any` can store
additional user-defined type information without increasing the
stack size.
Differential Revision: https://developer.blender.org/D12986
Previously, every node had to create warnings for unsupported input
geometry manually. Now this is automated. Nodes just have to specify
the geometry types they support in the node declaration.
Differential Revision: https://developer.blender.org/D12899
This adds a new Instance on Points node that is a replacement
for the old Point Instance node. Contrary to the old node,
it does not have a mode to instance objects or collections
directly. Instead, the node has to be used with an Object/
Collection Info to achieve the same effect.
Rotation and scale of the instances can be adjusted in the node
directly or can be controlled with a field to get some variation
between instances.
The node supports placing different instances on different points.
The user has control over which instance is placed on which point
using an Instance Index input. If that functionality is used, the
Instance Geometry has to contain multiple instances that can are
instanced separately.
Differential Revision: https://developer.blender.org/D12478
This implements the initial core framework for fields and anonymous
attributes (also see T91274).
The new functionality is hidden behind the "Geometry Nodes Fields"
feature flag. When enabled in the user preferences, the following
new nodes become available: `Position`, `Index`, `Normal`,
`Set Position` and `Attribute Capture`.
Socket inspection has not been updated to work with fields yet.
Besides these changes at the user level, this patch contains the
ground work for:
* building and evaluating fields at run-time (`FN_fields.hh`) and
* creating and accessing anonymous attributes on geometry
(`BKE_anonymous_attribute.h`).
For evaluating fields we use a new so called multi-function procedure
(`FN_multi_function_procedure.hh`). It allows composing multi-functions
in arbitrary ways and supports efficient evaluation as is required by
fields. See `FN_multi_function_procedure.hh` for more details on how
this evaluation mechanism can be used.
A new `AttributeIDRef` has been added which allows handling named
and anonymous attributes in the same way in many places.
Hans and I worked on this patch together.
Differential Revision: https://developer.blender.org/D12414
Many ui features for geometry nodes need access to information generated
during evaluation:
* Node warnings.
* Attribute search.
* Viewer node.
* Socket inspection (not in master yet).
The way we logged the required information before had some disadvantages:
* Viewer node used a completely separate system from node warnings and
attribute search.
* Most of the context of logged information is lost when e.g. the same node
group is used multiple times.
* A global lock was needed every time something is logged.
This new implementation solves these problems:
* All four mentioned ui features use the same underlying logging system.
* All context information for logged values is kept intact.
* Every thread has its own local logger. The logged informatiton is combined
in the end.
Differential Revision: https://developer.blender.org/D11785
The old geometry nodes evaluator was quite basic and missed many features.
It was useful to get the geometry nodes project started. However, nowadays
we run into its limitations from time to time.
The new evaluator is more complex, but comes with new capabilities.
The two most important capabilities are that it can now execute nodes in
parallel and it supports lazy evaluation.
The performance improvement by multi-threading depends a lot on the specific
node tree. In our demo files, the speedup is measurable but not huge. This
is mainly because they are bottlenecked by one or two nodes that have to be
executed one after the other (often the Boolean or Attribute Proximity nodes)
or because the bottleneck is multi-threaded already (often openvdb nodes).
Lazy evaluation of inputs is only supported by the Switch node for now.
Previously, geometry nodes would always compute both inputs and then just
discard the one that is not used. Now, only the input that is required
is computed.
For some more details read D11191, T87620 and the in-code documentation.
Differential Revision: https://developer.blender.org/D11191
Those were mostly just left over from previous work on particle nodes.
They solved the problem of keeping a reference to an object over
multiple frames and in a cache. Currently, we do not have this problem
in geometry nodes, so we can also remove this layer of complexity
for now.
This is a first step towards T87620.
It should not have any functional changes.
Goals of this refactor:
* Move the evaluator out of `MOD_nodes.cc`. That makes it easier to
improve it in isolation.
* Extract core input/out parameter management out of `GeoNodeExecParams`.
Managing this is the responsibility of the evaluator. This separation of
concerns will be useful once we have lazy evaluation of certain inputs/outputs.
Differential Revision: https://developer.blender.org/D11085
A virtual array is a data structure that is similar to a normal array
in that its elements can be accessed by an index. However, a virtual
array does not have to be a contiguous array internally. Instead, its
elements can be layed out arbitrarily while element access happens
through a virtual function call. However, the virtual array data
structures are designed so that the virtual function call can be avoided
in cases where it could become a bottleneck.
Most commonly, a virtual array is backed by an actual array/span or
is a single value internally, that is the same for every index.
Besides those, there are many more specialized virtual arrays like the
ones that provides vertex positions based on the `MVert` struct or
vertex group weights.
Not all attributes used by geometry nodes are stored in simple contiguous
arrays. To provide uniform access to all kinds of attributes, the attribute
API has to provide virtual array functionality that hides the implementation
details of attributes.
Before this refactor, the attribute API provided its own virtual array
implementation as part of the `ReadAttribute` and `WriteAttribute` types.
That resulted in unnecessary code duplication with the virtual array system.
Even worse, it bound many algorithms used by geometry nodes to the specifics
of the attribute API, even though they could also use different data sources
(such as data from sockets, default values, later results of expressions, ...).
This refactor removes the `ReadAttribute` and `WriteAttribute` types and
replaces them with `GVArray` and `GVMutableArray` respectively. The `GV`
stands for "generic virtual". The "generic" means that the data type contained
in those virtual arrays is only known at run-time. There are the corresponding
statically typed types `VArray<T>` and `VMutableArray<T>` as well.
No regressions are expected from this refactor. It does come with one
improvement for users. The attribute API can convert the data type
on write now. This is especially useful when writing to builtin attributes
like `material_index` with e.g. the Attribute Math node (which usually
just writes to float attributes, while `material_index` is an integer attribute).
Differential Revision: https://developer.blender.org/D10994
This is a complete rewrite of the derived node tree data structure.
It is a much thinner abstraction about `NodeTreeRef` than before.
This gives the user of the derived node tree more control and allows
for greater introspection capabilities (e.g. before muted nodes were
completely abstracted away; this was convenient, but came with
limitations).
Another nice benefit of the new structure is that it is much cheaper
to build, because it does not inline all nodes and sockets in nested
node groups.
Differential Revision: https://developer.blender.org/D10620
This patch adds icons to the right side of nodes when they encounter a
a problem. When hovered, a tooltip displays describing the encountered
while evaluating the node.
Some examples are: attribute doesn't exist, mesh has no faces,
incorrect attribute type, etc. Exposing more messages to the system
will be an ongoing process. Multiple warnings per node are supported.
The system is implemented somewhat generically so that the basic
structure can also be used to store more information from evaluation
for the interface, like a list of available attributes.
Currently the messages are just button tooltips. They could be styled
differently in the future. Another limitation is that every instance of
a node group in a parent node tree will have the same error messages,
the "evaluation context" used to decide when to display the tooltips
must be extended to support node tree paths.
Differential Revision: https://developer.blender.org/D10290
Since the derived node tree is already build for the evaluation system,
it's simpler to pass a derived node to the params struct. This will also
allow context lookups in nested node groups for node error messages,
since the derived node has that information readily accessible.
Currently every attribute node assumes that the attribute exists on the
"points" domain, so it generally isn't possible to work with attributes
on other domains like edges, polygons, and corners.
This commit adds a heuristic to each attribute node to determine the
correct domain for the result attribute. In general, it works like this:
- If the output attribute already exists, use that domain.
- Otherwise, use the highest priority domain of the input attributes.
- If none of the inputs are attributes, use the default domain (points).
For the implementation I abstracted the check a bit, but in each
node has a slightly different situation, so we end up with slightly
different `get_result_domain` functions in each node. I think this makes
sense, it keeps the code flexible and more easily understandable.
Note that we might eventually want to expose a domain drop-down to some
of the nodes. But that will be a separate discussion; this commit focuses
on making a more useful choice automatically.
Differential Revision: https://developer.blender.org/D10389
Normally sockets only have one input link. This commit adds the back-end
changes needed to use multiple input links per socket.
Multi-input sockets can be defined with a new flag in `bNodeSocketType`.
The changes necessary to make the sockets work in the geometry nodes
evaluator are generalizing input socket values as a vector of values,
and supporting this in the derived node tree structure.
This patch should contain no functional changes. Two upcoming patches
will use this system for the "Join Geometry" node and expose link picking
and updated display in the UI: D10069 and D10181.
Reviewed By: Jacques Lucke, Hans Goudey
Differential Revision: https://developer.blender.org/D10067
This node allows sampling a texture for every vertex based on some
mapping attribute. Typical attribute names are the name of a uv map
(e.g. "UVMap") and "position". However, every attribute that can be
converted to a vector implicitly is supported.
It should be noted that as of right now, uv map attributes can only be
accessed after a Point Distribute node.
Ref T82584.
Differential Revision: https://developer.blender.org/D10121
