archive/blender-archive
Archived
1
1
Fork 0
Code Pull Requests Packages Activity
This repository has been archived on 2023-10-09. You can view files and clone it, but cannot push or open issues or pull requests.
Files
b3146200a8c4265de7fba2e2d40ef84f99d979e2
blender-archive/source/blender/functions/FN_multi_function_builder.hh

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

693 lines
28 KiB
C++
Raw Normal View History

File headers: SPDX License migration Use a shorter/simpler license convention, stops the header taking so much space. Follow the SPDX license specification: https://spdx.org/licenses - C/C++/objc/objc++ - Python - Shell Scripts - CMake, GNUmakefile While most of the source tree has been included - `./extern/` was left out. - `./intern/cycles` & `./intern/atomic` are also excluded because they use different header conventions. doc/license/SPDX-license-identifiers.txt has been added to list SPDX all used identifiers. See P2788 for the script that automated these edits. Reviewed By: brecht, mont29, sergey Ref D14069
2022-02-11 09:07:11 +11:00
/* SPDX-License-Identifier: GPL-2.0-or-later */
Functions: add utilities that allow creating some multi-functions with less typing
2020-06-22 15:50:08 +02:00
#pragma once
/** \file
* \ingroup fn
*
* This file contains several utilities to create multi-functions with less redundant code.
*/
#include "FN_multi_function.hh"
Functions: refactor multi-function builder API * New `build_mf` namespace for the multi-function builders. * The type name of the created multi-functions is now "private", i.e. the caller has to use `auto`. This has the benefit that the implementation can change more freely without affecting the caller. * `CustomMF` does not use `std::function` internally anymore. This reduces some overhead during code generation and at run-time. * `CustomMF` now supports single-mutable parameters.
2023-01-07 16:19:59 +01:00
namespace blender::fn::build_mf {
Functions: add utilities that allow creating some multi-functions with less typing
2020-06-22 15:50:08 +02:00
/**
Geometry Nodes: refactor array devirtualization Goals: * Better high level control over where devirtualization occurs. There is always a trade-off between performance and compile-time/binary-size. * Simplify using array devirtualization. * Better performance for cases where devirtualization wasn't used before. Many geometry nodes accept fields as inputs. Internally, that means that the execution functions have to accept so called "virtual arrays" as inputs. Those can be e.g. actual arrays, just single values, or lazily computed arrays. Due to these different possible virtual arrays implementations, access to individual elements is slower than it would be if everything was just a normal array (access does through a virtual function call). For more complex execution functions, this overhead does not matter, but for small functions (like a simple addition) it very much does. The virtual function call also prevents the compiler from doing some optimizations (e.g. loop unrolling and inserting simd instructions). The solution is to "devirtualize" the virtual arrays for small functions where the overhead is measurable. Essentially, the function is generated many times with different array types as input. Then there is a run-time dispatch that calls the best implementation. We have been doing devirtualization in e.g. math nodes for a long time already. This patch just generalizes the concept and makes it easier to control. It also makes it easier to investigate the different trade-offs when it comes to devirtualization. Nodes that we've optimized using devirtualization before didn't get a speedup. However, a couple of nodes are using devirtualization now, that didn't before. Those got a 2-4x speedup in common cases. * Map Range * Random Value * Switch * Combine XYZ Differential Revision: https://developer.blender.org/D14628
2022-04-26 17:12:34 +02:00
* These presets determine what code is generated for a #CustomMF. Different presets make different
* trade-offs between run-time performance and compile-time/binary size.
Functions: add utilities that allow creating some multi-functions with less typing
2020-06-22 15:50:08 +02:00
*/
Functions: refactor multi-function builder API * New `build_mf` namespace for the multi-function builders. * The type name of the created multi-functions is now "private", i.e. the caller has to use `auto`. This has the benefit that the implementation can change more freely without affecting the caller. * `CustomMF` does not use `std::function` internally anymore. This reduces some overhead during code generation and at run-time. * `CustomMF` now supports single-mutable parameters.
2023-01-07 16:19:59 +01:00
namespace exec_presets {
Geometry Nodes: refactor array devirtualization Goals: * Better high level control over where devirtualization occurs. There is always a trade-off between performance and compile-time/binary-size. * Simplify using array devirtualization. * Better performance for cases where devirtualization wasn't used before. Many geometry nodes accept fields as inputs. Internally, that means that the execution functions have to accept so called "virtual arrays" as inputs. Those can be e.g. actual arrays, just single values, or lazily computed arrays. Due to these different possible virtual arrays implementations, access to individual elements is slower than it would be if everything was just a normal array (access does through a virtual function call). For more complex execution functions, this overhead does not matter, but for small functions (like a simple addition) it very much does. The virtual function call also prevents the compiler from doing some optimizations (e.g. loop unrolling and inserting simd instructions). The solution is to "devirtualize" the virtual arrays for small functions where the overhead is measurable. Essentially, the function is generated many times with different array types as input. Then there is a run-time dispatch that calls the best implementation. We have been doing devirtualization in e.g. math nodes for a long time already. This patch just generalizes the concept and makes it easier to control. It also makes it easier to investigate the different trade-offs when it comes to devirtualization. Nodes that we've optimized using devirtualization before didn't get a speedup. However, a couple of nodes are using devirtualization now, that didn't before. Those got a 2-4x speedup in common cases. * Map Range * Random Value * Switch * Combine XYZ Differential Revision: https://developer.blender.org/D14628
2022-04-26 17:12:34 +02:00
/** Method to execute a function in case devirtualization was not possible. */
enum class FallbackMode {
/** Access all elements in virtual arrays through virtual function calls. */
Simple,
/** Process elements in chunks to reduce virtual function call overhead. */
Materialized,
};
Functions: add utilities that allow creating some multi-functions with less typing
2020-06-22 15:50:08 +02:00
Geometry Nodes: refactor array devirtualization Goals: * Better high level control over where devirtualization occurs. There is always a trade-off between performance and compile-time/binary-size. * Simplify using array devirtualization. * Better performance for cases where devirtualization wasn't used before. Many geometry nodes accept fields as inputs. Internally, that means that the execution functions have to accept so called "virtual arrays" as inputs. Those can be e.g. actual arrays, just single values, or lazily computed arrays. Due to these different possible virtual arrays implementations, access to individual elements is slower than it would be if everything was just a normal array (access does through a virtual function call). For more complex execution functions, this overhead does not matter, but for small functions (like a simple addition) it very much does. The virtual function call also prevents the compiler from doing some optimizations (e.g. loop unrolling and inserting simd instructions). The solution is to "devirtualize" the virtual arrays for small functions where the overhead is measurable. Essentially, the function is generated many times with different array types as input. Then there is a run-time dispatch that calls the best implementation. We have been doing devirtualization in e.g. math nodes for a long time already. This patch just generalizes the concept and makes it easier to control. It also makes it easier to investigate the different trade-offs when it comes to devirtualization. Nodes that we've optimized using devirtualization before didn't get a speedup. However, a couple of nodes are using devirtualization now, that didn't before. Those got a 2-4x speedup in common cases. * Map Range * Random Value * Switch * Combine XYZ Differential Revision: https://developer.blender.org/D14628
2022-04-26 17:12:34 +02:00
/**
* The "naive" method for executing a #CustomMF. Every element is processed separately and input
* values are retrieved from the virtual arrays one by one. This generates the least amount of
* code, but is also the slowest method.
*/
struct Simple {
static constexpr bool use_devirtualization = false;
static constexpr FallbackMode fallback_mode = FallbackMode::Simple;
};
/**
* This is an improvement over the #Simple method. It still generates a relatively small amount of
* code, because the function is only instantiated once. It's generally faster than #Simple,
* because inputs are retrieved from the virtual arrays in chunks, reducing virtual method call
* overhead.
*/
struct Materialized {
static constexpr bool use_devirtualization = false;
static constexpr FallbackMode fallback_mode = FallbackMode::Materialized;
};
/**
* The most efficient preset, but also potentially generates a lot of code (exponential in the
* number of inputs of the function). It generates separate optimized loops for all combinations of
* inputs. This should be used for small functions of which all inputs are likely to be single
* values or spans, and the number of inputs is relatively small.
*/
struct AllSpanOrSingle {
static constexpr bool use_devirtualization = true;
static constexpr FallbackMode fallback_mode = FallbackMode::Materialized;
Functions: improve devirtualization in multi-function builder This refactors how devirtualization is done in general and how multi-functions use it. * The old `Devirtualizer` class has been removed in favor of a simpler solution. It is also more general in the sense that it is not coupled with `IndexMask` and `VArray`. Instead there is a function that has inputs which control how different types are devirtualized. The new implementation is currently less general with regard to the number of parameters it supports. This can be changed in the future, but does not seem necessary now and would make the code less obvious. * Devirtualizers for different types are now defined in their respective headers. * The multi-function builder works with the `GVArray` stored in `MFParams` directly now, instead of first converting it to a `VArray<T>`. This reduces some constant overhead, which makes the multi-function slightly faster. This is only noticable when very few elements are processed though. No functional changes or performance regressions are expected.
2023-01-07 12:55:48 +01:00
template<typename... ParamTags, typename... LoadedParams, size_t... I>
auto create_devirtualizers(TypeSequence<ParamTags...> /*param_tags*/,
std::index_sequence<I...> /*indices*/,
const IndexMask &mask,
Functions: refactor multi-function builder API * New `build_mf` namespace for the multi-function builders. * The type name of the created multi-functions is now "private", i.e. the caller has to use `auto`. This has the benefit that the implementation can change more freely without affecting the caller. * `CustomMF` does not use `std::function` internally anymore. This reduces some overhead during code generation and at run-time. * `CustomMF` now supports single-mutable parameters.
2023-01-07 16:19:59 +01:00
const std::tuple<LoadedParams...> &loaded_params) const
Functions: add utilities that allow creating some multi-functions with less typing
2020-06-22 15:50:08 +02:00
{
Functions: improve devirtualization in multi-function builder This refactors how devirtualization is done in general and how multi-functions use it. * The old `Devirtualizer` class has been removed in favor of a simpler solution. It is also more general in the sense that it is not coupled with `IndexMask` and `VArray`. Instead there is a function that has inputs which control how different types are devirtualized. The new implementation is currently less general with regard to the number of parameters it supports. This can be changed in the future, but does not seem necessary now and would make the code less obvious. * Devirtualizers for different types are now defined in their respective headers. * The multi-function builder works with the `GVArray` stored in `MFParams` directly now, instead of first converting it to a `VArray<T>`. This reduces some constant overhead, which makes the multi-function slightly faster. This is only noticable when very few elements are processed though. No functional changes or performance regressions are expected.
2023-01-07 12:55:48 +01:00
return std::make_tuple(IndexMaskDevirtualizer<true, true>{mask}, [&]() {
typedef ParamTags ParamTag;
typedef typename ParamTag::base_type T;
if constexpr (ParamTag::category == MFParamCategory::SingleInput) {
const GVArrayImpl &varray_impl = *std::get<I>(loaded_params);
return GVArrayDevirtualizer<T, true, true>{varray_impl};
}
Functions: refactor multi-function builder API * New `build_mf` namespace for the multi-function builders. * The type name of the created multi-functions is now "private", i.e. the caller has to use `auto`. This has the benefit that the implementation can change more freely without affecting the caller. * `CustomMF` does not use `std::function` internally anymore. This reduces some overhead during code generation and at run-time. * `CustomMF` now supports single-mutable parameters.
2023-01-07 16:19:59 +01:00
else if constexpr (ELEM(ParamTag::category,
MFParamCategory::SingleOutput,
MFParamCategory::SingleMutable)) {
Functions: improve devirtualization in multi-function builder This refactors how devirtualization is done in general and how multi-functions use it. * The old `Devirtualizer` class has been removed in favor of a simpler solution. It is also more general in the sense that it is not coupled with `IndexMask` and `VArray`. Instead there is a function that has inputs which control how different types are devirtualized. The new implementation is currently less general with regard to the number of parameters it supports. This can be changed in the future, but does not seem necessary now and would make the code less obvious. * Devirtualizers for different types are now defined in their respective headers. * The multi-function builder works with the `GVArray` stored in `MFParams` directly now, instead of first converting it to a `VArray<T>`. This reduces some constant overhead, which makes the multi-function slightly faster. This is only noticable when very few elements are processed though. No functional changes or performance regressions are expected.
2023-01-07 12:55:48 +01:00
T *ptr = std::get<I>(loaded_params);
return BasicDevirtualizer<T *>{ptr};
}
}()...);
Functions: add utilities that allow creating some multi-functions with less typing
2020-06-22 15:50:08 +02:00
}
Geometry Nodes: refactor array devirtualization Goals: * Better high level control over where devirtualization occurs. There is always a trade-off between performance and compile-time/binary-size. * Simplify using array devirtualization. * Better performance for cases where devirtualization wasn't used before. Many geometry nodes accept fields as inputs. Internally, that means that the execution functions have to accept so called "virtual arrays" as inputs. Those can be e.g. actual arrays, just single values, or lazily computed arrays. Due to these different possible virtual arrays implementations, access to individual elements is slower than it would be if everything was just a normal array (access does through a virtual function call). For more complex execution functions, this overhead does not matter, but for small functions (like a simple addition) it very much does. The virtual function call also prevents the compiler from doing some optimizations (e.g. loop unrolling and inserting simd instructions). The solution is to "devirtualize" the virtual arrays for small functions where the overhead is measurable. Essentially, the function is generated many times with different array types as input. Then there is a run-time dispatch that calls the best implementation. We have been doing devirtualization in e.g. math nodes for a long time already. This patch just generalizes the concept and makes it easier to control. It also makes it easier to investigate the different trade-offs when it comes to devirtualization. Nodes that we've optimized using devirtualization before didn't get a speedup. However, a couple of nodes are using devirtualization now, that didn't before. Those got a 2-4x speedup in common cases. * Map Range * Random Value * Switch * Combine XYZ Differential Revision: https://developer.blender.org/D14628
2022-04-26 17:12:34 +02:00
};
Functions: add utilities that allow creating some multi-functions with less typing
2020-06-22 15:50:08 +02:00
Geometry Nodes: refactor array devirtualization Goals: * Better high level control over where devirtualization occurs. There is always a trade-off between performance and compile-time/binary-size. * Simplify using array devirtualization. * Better performance for cases where devirtualization wasn't used before. Many geometry nodes accept fields as inputs. Internally, that means that the execution functions have to accept so called "virtual arrays" as inputs. Those can be e.g. actual arrays, just single values, or lazily computed arrays. Due to these different possible virtual arrays implementations, access to individual elements is slower than it would be if everything was just a normal array (access does through a virtual function call). For more complex execution functions, this overhead does not matter, but for small functions (like a simple addition) it very much does. The virtual function call also prevents the compiler from doing some optimizations (e.g. loop unrolling and inserting simd instructions). The solution is to "devirtualize" the virtual arrays for small functions where the overhead is measurable. Essentially, the function is generated many times with different array types as input. Then there is a run-time dispatch that calls the best implementation. We have been doing devirtualization in e.g. math nodes for a long time already. This patch just generalizes the concept and makes it easier to control. It also makes it easier to investigate the different trade-offs when it comes to devirtualization. Nodes that we've optimized using devirtualization before didn't get a speedup. However, a couple of nodes are using devirtualization now, that didn't before. Those got a 2-4x speedup in common cases. * Map Range * Random Value * Switch * Combine XYZ Differential Revision: https://developer.blender.org/D14628
2022-04-26 17:12:34 +02:00
/**
Cleanup: spelling in comments Also use back-slashes for doxy commands.
2022-04-28 14:03:49 +10:00
* A slightly weaker variant of #AllSpanOrSingle. It generates less code, because it assumes that
Geometry Nodes: refactor array devirtualization Goals: * Better high level control over where devirtualization occurs. There is always a trade-off between performance and compile-time/binary-size. * Simplify using array devirtualization. * Better performance for cases where devirtualization wasn't used before. Many geometry nodes accept fields as inputs. Internally, that means that the execution functions have to accept so called "virtual arrays" as inputs. Those can be e.g. actual arrays, just single values, or lazily computed arrays. Due to these different possible virtual arrays implementations, access to individual elements is slower than it would be if everything was just a normal array (access does through a virtual function call). For more complex execution functions, this overhead does not matter, but for small functions (like a simple addition) it very much does. The virtual function call also prevents the compiler from doing some optimizations (e.g. loop unrolling and inserting simd instructions). The solution is to "devirtualize" the virtual arrays for small functions where the overhead is measurable. Essentially, the function is generated many times with different array types as input. Then there is a run-time dispatch that calls the best implementation. We have been doing devirtualization in e.g. math nodes for a long time already. This patch just generalizes the concept and makes it easier to control. It also makes it easier to investigate the different trade-offs when it comes to devirtualization. Nodes that we've optimized using devirtualization before didn't get a speedup. However, a couple of nodes are using devirtualization now, that didn't before. Those got a 2-4x speedup in common cases. * Map Range * Random Value * Switch * Combine XYZ Differential Revision: https://developer.blender.org/D14628
2022-04-26 17:12:34 +02:00
* some of the inputs are most likely single values. It should be used for small functions which
* have too many inputs to make #AllSingleOrSpan a reasonable choice.
*/
template<size_t... Indices> struct SomeSpanOrSingle {
static constexpr bool use_devirtualization = true;
static constexpr FallbackMode fallback_mode = FallbackMode::Materialized;
Functions: improve devirtualization in multi-function builder This refactors how devirtualization is done in general and how multi-functions use it. * The old `Devirtualizer` class has been removed in favor of a simpler solution. It is also more general in the sense that it is not coupled with `IndexMask` and `VArray`. Instead there is a function that has inputs which control how different types are devirtualized. The new implementation is currently less general with regard to the number of parameters it supports. This can be changed in the future, but does not seem necessary now and would make the code less obvious. * Devirtualizers for different types are now defined in their respective headers. * The multi-function builder works with the `GVArray` stored in `MFParams` directly now, instead of first converting it to a `VArray<T>`. This reduces some constant overhead, which makes the multi-function slightly faster. This is only noticable when very few elements are processed though. No functional changes or performance regressions are expected.
2023-01-07 12:55:48 +01:00
template<typename... ParamTags, typename... LoadedParams, size_t... I>
auto create_devirtualizers(TypeSequence<ParamTags...> /*param_tags*/,
std::index_sequence<I...> /*indices*/,
const IndexMask &mask,
Functions: refactor multi-function builder API * New `build_mf` namespace for the multi-function builders. * The type name of the created multi-functions is now "private", i.e. the caller has to use `auto`. This has the benefit that the implementation can change more freely without affecting the caller. * `CustomMF` does not use `std::function` internally anymore. This reduces some overhead during code generation and at run-time. * `CustomMF` now supports single-mutable parameters.
2023-01-07 16:19:59 +01:00
const std::tuple<LoadedParams...> &loaded_params) const
Functions: add utilities that allow creating some multi-functions with less typing
2020-06-22 15:50:08 +02:00
{
Functions: improve devirtualization in multi-function builder This refactors how devirtualization is done in general and how multi-functions use it. * The old `Devirtualizer` class has been removed in favor of a simpler solution. It is also more general in the sense that it is not coupled with `IndexMask` and `VArray`. Instead there is a function that has inputs which control how different types are devirtualized. The new implementation is currently less general with regard to the number of parameters it supports. This can be changed in the future, but does not seem necessary now and would make the code less obvious. * Devirtualizers for different types are now defined in their respective headers. * The multi-function builder works with the `GVArray` stored in `MFParams` directly now, instead of first converting it to a `VArray<T>`. This reduces some constant overhead, which makes the multi-function slightly faster. This is only noticable when very few elements are processed though. No functional changes or performance regressions are expected.
2023-01-07 12:55:48 +01:00
return std::make_tuple(IndexMaskDevirtualizer<true, true>{mask}, [&]() {
typedef ParamTags ParamTag;
typedef typename ParamTag::base_type T;
if constexpr (ParamTag::category == MFParamCategory::SingleInput) {
constexpr bool UseSpan = ValueSequence<size_t, Indices...>::template contains<I>();
const GVArrayImpl &varray_impl = *std::get<I>(loaded_params);
return GVArrayDevirtualizer<T, true, UseSpan>{varray_impl};
}
Functions: refactor multi-function builder API * New `build_mf` namespace for the multi-function builders. * The type name of the created multi-functions is now "private", i.e. the caller has to use `auto`. This has the benefit that the implementation can change more freely without affecting the caller. * `CustomMF` does not use `std::function` internally anymore. This reduces some overhead during code generation and at run-time. * `CustomMF` now supports single-mutable parameters.
2023-01-07 16:19:59 +01:00
else if constexpr (ELEM(ParamTag::category,
MFParamCategory::SingleOutput,
MFParamCategory::SingleMutable)) {
Functions: improve devirtualization in multi-function builder This refactors how devirtualization is done in general and how multi-functions use it. * The old `Devirtualizer` class has been removed in favor of a simpler solution. It is also more general in the sense that it is not coupled with `IndexMask` and `VArray`. Instead there is a function that has inputs which control how different types are devirtualized. The new implementation is currently less general with regard to the number of parameters it supports. This can be changed in the future, but does not seem necessary now and would make the code less obvious. * Devirtualizers for different types are now defined in their respective headers. * The multi-function builder works with the `GVArray` stored in `MFParams` directly now, instead of first converting it to a `VArray<T>`. This reduces some constant overhead, which makes the multi-function slightly faster. This is only noticable when very few elements are processed though. No functional changes or performance regressions are expected.
2023-01-07 12:55:48 +01:00
T *ptr = std::get<I>(loaded_params);
return BasicDevirtualizer<T *>{ptr};
}
}()...);
Functions: add utilities that allow creating some multi-functions with less typing
2020-06-22 15:50:08 +02:00
}
Geometry Nodes: refactor array devirtualization Goals: * Better high level control over where devirtualization occurs. There is always a trade-off between performance and compile-time/binary-size. * Simplify using array devirtualization. * Better performance for cases where devirtualization wasn't used before. Many geometry nodes accept fields as inputs. Internally, that means that the execution functions have to accept so called "virtual arrays" as inputs. Those can be e.g. actual arrays, just single values, or lazily computed arrays. Due to these different possible virtual arrays implementations, access to individual elements is slower than it would be if everything was just a normal array (access does through a virtual function call). For more complex execution functions, this overhead does not matter, but for small functions (like a simple addition) it very much does. The virtual function call also prevents the compiler from doing some optimizations (e.g. loop unrolling and inserting simd instructions). The solution is to "devirtualize" the virtual arrays for small functions where the overhead is measurable. Essentially, the function is generated many times with different array types as input. Then there is a run-time dispatch that calls the best implementation. We have been doing devirtualization in e.g. math nodes for a long time already. This patch just generalizes the concept and makes it easier to control. It also makes it easier to investigate the different trade-offs when it comes to devirtualization. Nodes that we've optimized using devirtualization before didn't get a speedup. However, a couple of nodes are using devirtualization now, that didn't before. Those got a 2-4x speedup in common cases. * Map Range * Random Value * Switch * Combine XYZ Differential Revision: https://developer.blender.org/D14628
2022-04-26 17:12:34 +02:00
};
Functions: add utilities that allow creating some multi-functions with less typing
2020-06-22 15:50:08 +02:00
Functions: refactor multi-function builder API * New `build_mf` namespace for the multi-function builders. * The type name of the created multi-functions is now "private", i.e. the caller has to use `auto`. This has the benefit that the implementation can change more freely without affecting the caller. * `CustomMF` does not use `std::function` internally anymore. This reduces some overhead during code generation and at run-time. * `CustomMF` now supports single-mutable parameters.
2023-01-07 16:19:59 +01:00
} // namespace exec_presets
Geometry Nodes: refactor array devirtualization Goals: * Better high level control over where devirtualization occurs. There is always a trade-off between performance and compile-time/binary-size. * Simplify using array devirtualization. * Better performance for cases where devirtualization wasn't used before. Many geometry nodes accept fields as inputs. Internally, that means that the execution functions have to accept so called "virtual arrays" as inputs. Those can be e.g. actual arrays, just single values, or lazily computed arrays. Due to these different possible virtual arrays implementations, access to individual elements is slower than it would be if everything was just a normal array (access does through a virtual function call). For more complex execution functions, this overhead does not matter, but for small functions (like a simple addition) it very much does. The virtual function call also prevents the compiler from doing some optimizations (e.g. loop unrolling and inserting simd instructions). The solution is to "devirtualize" the virtual arrays for small functions where the overhead is measurable. Essentially, the function is generated many times with different array types as input. Then there is a run-time dispatch that calls the best implementation. We have been doing devirtualization in e.g. math nodes for a long time already. This patch just generalizes the concept and makes it easier to control. It also makes it easier to investigate the different trade-offs when it comes to devirtualization. Nodes that we've optimized using devirtualization before didn't get a speedup. However, a couple of nodes are using devirtualization now, that didn't before. Those got a 2-4x speedup in common cases. * Map Range * Random Value * Switch * Combine XYZ Differential Revision: https://developer.blender.org/D14628
2022-04-26 17:12:34 +02:00
namespace detail {
Functions: optimize simple generated multi-functions This implements two optimizations: * Reduce virtual function call overhead when a non-standard virtual array is used as input. * Use a lambda in `type_conversion.cc`. In my test setup, which creates a float attribute filled with the index, the running time drops from `4.0 ms` to `2.0 ms`. Differential Revision: https://developer.blender.org/D14585
2022-04-07 18:48:14 +02:00
Geometry Nodes: refactor array devirtualization Goals: * Better high level control over where devirtualization occurs. There is always a trade-off between performance and compile-time/binary-size. * Simplify using array devirtualization. * Better performance for cases where devirtualization wasn't used before. Many geometry nodes accept fields as inputs. Internally, that means that the execution functions have to accept so called "virtual arrays" as inputs. Those can be e.g. actual arrays, just single values, or lazily computed arrays. Due to these different possible virtual arrays implementations, access to individual elements is slower than it would be if everything was just a normal array (access does through a virtual function call). For more complex execution functions, this overhead does not matter, but for small functions (like a simple addition) it very much does. The virtual function call also prevents the compiler from doing some optimizations (e.g. loop unrolling and inserting simd instructions). The solution is to "devirtualize" the virtual arrays for small functions where the overhead is measurable. Essentially, the function is generated many times with different array types as input. Then there is a run-time dispatch that calls the best implementation. We have been doing devirtualization in e.g. math nodes for a long time already. This patch just generalizes the concept and makes it easier to control. It also makes it easier to investigate the different trade-offs when it comes to devirtualization. Nodes that we've optimized using devirtualization before didn't get a speedup. However, a couple of nodes are using devirtualization now, that didn't before. Those got a 2-4x speedup in common cases. * Map Range * Random Value * Switch * Combine XYZ Differential Revision: https://developer.blender.org/D14628
2022-04-26 17:12:34 +02:00
/**
* Executes #element_fn for all indices in the mask. The passed in #args contain the input as well
* as output parameters. Usually types in #args are devirtualized (e.g. a `Span<int>` is passed in
* instead of a `VArray<int>`).
*/
template<typename MaskT, typename... Args, typename... ParamTags, size_t... I, typename ElementFn>
Functions: improve devirtualization in multi-function builder This refactors how devirtualization is done in general and how multi-functions use it. * The old `Devirtualizer` class has been removed in favor of a simpler solution. It is also more general in the sense that it is not coupled with `IndexMask` and `VArray`. Instead there is a function that has inputs which control how different types are devirtualized. The new implementation is currently less general with regard to the number of parameters it supports. This can be changed in the future, but does not seem necessary now and would make the code less obvious. * Devirtualizers for different types are now defined in their respective headers. * The multi-function builder works with the `GVArray` stored in `MFParams` directly now, instead of first converting it to a `VArray<T>`. This reduces some constant overhead, which makes the multi-function slightly faster. This is only noticable when very few elements are processed though. No functional changes or performance regressions are expected.
2023-01-07 12:55:48 +01:00
void execute_array(TypeSequence<ParamTags...> /*param_tags*/,
std::index_sequence<I...> /*indices*/,
Geometry Nodes: refactor array devirtualization Goals: * Better high level control over where devirtualization occurs. There is always a trade-off between performance and compile-time/binary-size. * Simplify using array devirtualization. * Better performance for cases where devirtualization wasn't used before. Many geometry nodes accept fields as inputs. Internally, that means that the execution functions have to accept so called "virtual arrays" as inputs. Those can be e.g. actual arrays, just single values, or lazily computed arrays. Due to these different possible virtual arrays implementations, access to individual elements is slower than it would be if everything was just a normal array (access does through a virtual function call). For more complex execution functions, this overhead does not matter, but for small functions (like a simple addition) it very much does. The virtual function call also prevents the compiler from doing some optimizations (e.g. loop unrolling and inserting simd instructions). The solution is to "devirtualize" the virtual arrays for small functions where the overhead is measurable. Essentially, the function is generated many times with different array types as input. Then there is a run-time dispatch that calls the best implementation. We have been doing devirtualization in e.g. math nodes for a long time already. This patch just generalizes the concept and makes it easier to control. It also makes it easier to investigate the different trade-offs when it comes to devirtualization. Nodes that we've optimized using devirtualization before didn't get a speedup. However, a couple of nodes are using devirtualization now, that didn't before. Those got a 2-4x speedup in common cases. * Map Range * Random Value * Switch * Combine XYZ Differential Revision: https://developer.blender.org/D14628
2022-04-26 17:12:34 +02:00
ElementFn element_fn,
MaskT mask,
/* Use restrict to tell the compiler that pointer inputs do not alias each
* other. This is important for some compiler optimizations. */
Args &&__restrict... args)
{
for (const int64_t i : mask) {
element_fn([&]() -> decltype(auto) {
using ParamTag = typename TypeSequence<ParamTags...>::template at_index<I>;
if constexpr (ParamTag::category == MFParamCategory::SingleInput) {
/* For inputs, pass the value (or a reference to it) to the function. */
return args[i];
Functions: optimize simple generated multi-functions This implements two optimizations: * Reduce virtual function call overhead when a non-standard virtual array is used as input. * Use a lambda in `type_conversion.cc`. In my test setup, which creates a float attribute filled with the index, the running time drops from `4.0 ms` to `2.0 ms`. Differential Revision: https://developer.blender.org/D14585
2022-04-07 18:48:14 +02:00
}
Geometry Nodes: refactor array devirtualization Goals: * Better high level control over where devirtualization occurs. There is always a trade-off between performance and compile-time/binary-size. * Simplify using array devirtualization. * Better performance for cases where devirtualization wasn't used before. Many geometry nodes accept fields as inputs. Internally, that means that the execution functions have to accept so called "virtual arrays" as inputs. Those can be e.g. actual arrays, just single values, or lazily computed arrays. Due to these different possible virtual arrays implementations, access to individual elements is slower than it would be if everything was just a normal array (access does through a virtual function call). For more complex execution functions, this overhead does not matter, but for small functions (like a simple addition) it very much does. The virtual function call also prevents the compiler from doing some optimizations (e.g. loop unrolling and inserting simd instructions). The solution is to "devirtualize" the virtual arrays for small functions where the overhead is measurable. Essentially, the function is generated many times with different array types as input. Then there is a run-time dispatch that calls the best implementation. We have been doing devirtualization in e.g. math nodes for a long time already. This patch just generalizes the concept and makes it easier to control. It also makes it easier to investigate the different trade-offs when it comes to devirtualization. Nodes that we've optimized using devirtualization before didn't get a speedup. However, a couple of nodes are using devirtualization now, that didn't before. Those got a 2-4x speedup in common cases. * Map Range * Random Value * Switch * Combine XYZ Differential Revision: https://developer.blender.org/D14628
2022-04-26 17:12:34 +02:00
else if constexpr (ParamTag::category == MFParamCategory::SingleOutput) {
/* For outputs, pass a pointer to the function. This is done instead of passing a
* reference, because the pointer points to uninitialized memory. */
Functions: improve devirtualization in multi-function builder This refactors how devirtualization is done in general and how multi-functions use it. * The old `Devirtualizer` class has been removed in favor of a simpler solution. It is also more general in the sense that it is not coupled with `IndexMask` and `VArray`. Instead there is a function that has inputs which control how different types are devirtualized. The new implementation is currently less general with regard to the number of parameters it supports. This can be changed in the future, but does not seem necessary now and would make the code less obvious. * Devirtualizers for different types are now defined in their respective headers. * The multi-function builder works with the `GVArray` stored in `MFParams` directly now, instead of first converting it to a `VArray<T>`. This reduces some constant overhead, which makes the multi-function slightly faster. This is only noticable when very few elements are processed though. No functional changes or performance regressions are expected.
2023-01-07 12:55:48 +01:00
return args + i;
Geometry Nodes: refactor array devirtualization Goals: * Better high level control over where devirtualization occurs. There is always a trade-off between performance and compile-time/binary-size. * Simplify using array devirtualization. * Better performance for cases where devirtualization wasn't used before. Many geometry nodes accept fields as inputs. Internally, that means that the execution functions have to accept so called "virtual arrays" as inputs. Those can be e.g. actual arrays, just single values, or lazily computed arrays. Due to these different possible virtual arrays implementations, access to individual elements is slower than it would be if everything was just a normal array (access does through a virtual function call). For more complex execution functions, this overhead does not matter, but for small functions (like a simple addition) it very much does. The virtual function call also prevents the compiler from doing some optimizations (e.g. loop unrolling and inserting simd instructions). The solution is to "devirtualize" the virtual arrays for small functions where the overhead is measurable. Essentially, the function is generated many times with different array types as input. Then there is a run-time dispatch that calls the best implementation. We have been doing devirtualization in e.g. math nodes for a long time already. This patch just generalizes the concept and makes it easier to control. It also makes it easier to investigate the different trade-offs when it comes to devirtualization. Nodes that we've optimized using devirtualization before didn't get a speedup. However, a couple of nodes are using devirtualization now, that didn't before. Those got a 2-4x speedup in common cases. * Map Range * Random Value * Switch * Combine XYZ Differential Revision: https://developer.blender.org/D14628
2022-04-26 17:12:34 +02:00
}
Functions: refactor multi-function builder API * New `build_mf` namespace for the multi-function builders. * The type name of the created multi-functions is now "private", i.e. the caller has to use `auto`. This has the benefit that the implementation can change more freely without affecting the caller. * `CustomMF` does not use `std::function` internally anymore. This reduces some overhead during code generation and at run-time. * `CustomMF` now supports single-mutable parameters.
2023-01-07 16:19:59 +01:00
else if constexpr (ParamTag::category == MFParamCategory::SingleMutable) {
/* For mutables, pass a mutable reference to the function. */
return args[i];
}
Geometry Nodes: refactor array devirtualization Goals: * Better high level control over where devirtualization occurs. There is always a trade-off between performance and compile-time/binary-size. * Simplify using array devirtualization. * Better performance for cases where devirtualization wasn't used before. Many geometry nodes accept fields as inputs. Internally, that means that the execution functions have to accept so called "virtual arrays" as inputs. Those can be e.g. actual arrays, just single values, or lazily computed arrays. Due to these different possible virtual arrays implementations, access to individual elements is slower than it would be if everything was just a normal array (access does through a virtual function call). For more complex execution functions, this overhead does not matter, but for small functions (like a simple addition) it very much does. The virtual function call also prevents the compiler from doing some optimizations (e.g. loop unrolling and inserting simd instructions). The solution is to "devirtualize" the virtual arrays for small functions where the overhead is measurable. Essentially, the function is generated many times with different array types as input. Then there is a run-time dispatch that calls the best implementation. We have been doing devirtualization in e.g. math nodes for a long time already. This patch just generalizes the concept and makes it easier to control. It also makes it easier to investigate the different trade-offs when it comes to devirtualization. Nodes that we've optimized using devirtualization before didn't get a speedup. However, a couple of nodes are using devirtualization now, that didn't before. Those got a 2-4x speedup in common cases. * Map Range * Random Value * Switch * Combine XYZ Differential Revision: https://developer.blender.org/D14628
2022-04-26 17:12:34 +02:00
}()...);
}
}
Functions: optimize simple generated multi-functions This implements two optimizations: * Reduce virtual function call overhead when a non-standard virtual array is used as input. * Use a lambda in `type_conversion.cc`. In my test setup, which creates a float attribute filled with the index, the running time drops from `4.0 ms` to `2.0 ms`. Differential Revision: https://developer.blender.org/D14585
2022-04-07 18:48:14 +02:00
Functions: refactor multi-function builder API * New `build_mf` namespace for the multi-function builders. * The type name of the created multi-functions is now "private", i.e. the caller has to use `auto`. This has the benefit that the implementation can change more freely without affecting the caller. * `CustomMF` does not use `std::function` internally anymore. This reduces some overhead during code generation and at run-time. * `CustomMF` now supports single-mutable parameters.
2023-01-07 16:19:59 +01:00
enum class MaterializeArgMode {
Geometry Nodes: refactor array devirtualization Goals: * Better high level control over where devirtualization occurs. There is always a trade-off between performance and compile-time/binary-size. * Simplify using array devirtualization. * Better performance for cases where devirtualization wasn't used before. Many geometry nodes accept fields as inputs. Internally, that means that the execution functions have to accept so called "virtual arrays" as inputs. Those can be e.g. actual arrays, just single values, or lazily computed arrays. Due to these different possible virtual arrays implementations, access to individual elements is slower than it would be if everything was just a normal array (access does through a virtual function call). For more complex execution functions, this overhead does not matter, but for small functions (like a simple addition) it very much does. The virtual function call also prevents the compiler from doing some optimizations (e.g. loop unrolling and inserting simd instructions). The solution is to "devirtualize" the virtual arrays for small functions where the overhead is measurable. Essentially, the function is generated many times with different array types as input. Then there is a run-time dispatch that calls the best implementation. We have been doing devirtualization in e.g. math nodes for a long time already. This patch just generalizes the concept and makes it easier to control. It also makes it easier to investigate the different trade-offs when it comes to devirtualization. Nodes that we've optimized using devirtualization before didn't get a speedup. However, a couple of nodes are using devirtualization now, that didn't before. Those got a 2-4x speedup in common cases. * Map Range * Random Value * Switch * Combine XYZ Differential Revision: https://developer.blender.org/D14628
2022-04-26 17:12:34 +02:00
Unknown,
Single,
Span,
Materialized,
};
Functions: optimize simple generated multi-functions This implements two optimizations: * Reduce virtual function call overhead when a non-standard virtual array is used as input. * Use a lambda in `type_conversion.cc`. In my test setup, which creates a float attribute filled with the index, the running time drops from `4.0 ms` to `2.0 ms`. Differential Revision: https://developer.blender.org/D14585
2022-04-07 18:48:14 +02:00
Functions: refactor multi-function builder API * New `build_mf` namespace for the multi-function builders. * The type name of the created multi-functions is now "private", i.e. the caller has to use `auto`. This has the benefit that the implementation can change more freely without affecting the caller. * `CustomMF` does not use `std::function` internally anymore. This reduces some overhead during code generation and at run-time. * `CustomMF` now supports single-mutable parameters.
2023-01-07 16:19:59 +01:00
template<typename ParamTag> struct MaterializeArgInfo {
MaterializeArgMode mode = MaterializeArgMode::Unknown;
Geometry Nodes: refactor array devirtualization Goals: * Better high level control over where devirtualization occurs. There is always a trade-off between performance and compile-time/binary-size. * Simplify using array devirtualization. * Better performance for cases where devirtualization wasn't used before. Many geometry nodes accept fields as inputs. Internally, that means that the execution functions have to accept so called "virtual arrays" as inputs. Those can be e.g. actual arrays, just single values, or lazily computed arrays. Due to these different possible virtual arrays implementations, access to individual elements is slower than it would be if everything was just a normal array (access does through a virtual function call). For more complex execution functions, this overhead does not matter, but for small functions (like a simple addition) it very much does. The virtual function call also prevents the compiler from doing some optimizations (e.g. loop unrolling and inserting simd instructions). The solution is to "devirtualize" the virtual arrays for small functions where the overhead is measurable. Essentially, the function is generated many times with different array types as input. Then there is a run-time dispatch that calls the best implementation. We have been doing devirtualization in e.g. math nodes for a long time already. This patch just generalizes the concept and makes it easier to control. It also makes it easier to investigate the different trade-offs when it comes to devirtualization. Nodes that we've optimized using devirtualization before didn't get a speedup. However, a couple of nodes are using devirtualization now, that didn't before. Those got a 2-4x speedup in common cases. * Map Range * Random Value * Switch * Combine XYZ Differential Revision: https://developer.blender.org/D14628
2022-04-26 17:12:34 +02:00
Span<typename ParamTag::base_type> internal_span;
};
Functions: optimize simple generated multi-functions This implements two optimizations: * Reduce virtual function call overhead when a non-standard virtual array is used as input. * Use a lambda in `type_conversion.cc`. In my test setup, which creates a float attribute filled with the index, the running time drops from `4.0 ms` to `2.0 ms`. Differential Revision: https://developer.blender.org/D14585
2022-04-07 18:48:14 +02:00
Geometry Nodes: refactor array devirtualization Goals: * Better high level control over where devirtualization occurs. There is always a trade-off between performance and compile-time/binary-size. * Simplify using array devirtualization. * Better performance for cases where devirtualization wasn't used before. Many geometry nodes accept fields as inputs. Internally, that means that the execution functions have to accept so called "virtual arrays" as inputs. Those can be e.g. actual arrays, just single values, or lazily computed arrays. Due to these different possible virtual arrays implementations, access to individual elements is slower than it would be if everything was just a normal array (access does through a virtual function call). For more complex execution functions, this overhead does not matter, but for small functions (like a simple addition) it very much does. The virtual function call also prevents the compiler from doing some optimizations (e.g. loop unrolling and inserting simd instructions). The solution is to "devirtualize" the virtual arrays for small functions where the overhead is measurable. Essentially, the function is generated many times with different array types as input. Then there is a run-time dispatch that calls the best implementation. We have been doing devirtualization in e.g. math nodes for a long time already. This patch just generalizes the concept and makes it easier to control. It also makes it easier to investigate the different trade-offs when it comes to devirtualization. Nodes that we've optimized using devirtualization before didn't get a speedup. However, a couple of nodes are using devirtualization now, that didn't before. Those got a 2-4x speedup in common cases. * Map Range * Random Value * Switch * Combine XYZ Differential Revision: https://developer.blender.org/D14628
2022-04-26 17:12:34 +02:00
/**
* Similar to #execute_array but accepts two mask inputs, one for inputs and one for outputs.
*/
template<typename... ParamTags, typename ElementFn, typename... Chunks>
Functions: improve devirtualization in multi-function builder This refactors how devirtualization is done in general and how multi-functions use it. * The old `Devirtualizer` class has been removed in favor of a simpler solution. It is also more general in the sense that it is not coupled with `IndexMask` and `VArray`. Instead there is a function that has inputs which control how different types are devirtualized. The new implementation is currently less general with regard to the number of parameters it supports. This can be changed in the future, but does not seem necessary now and would make the code less obvious. * Devirtualizers for different types are now defined in their respective headers. * The multi-function builder works with the `GVArray` stored in `MFParams` directly now, instead of first converting it to a `VArray<T>`. This reduces some constant overhead, which makes the multi-function slightly faster. This is only noticable when very few elements are processed though. No functional changes or performance regressions are expected.
2023-01-07 12:55:48 +01:00
void execute_materialized_impl(TypeSequence<ParamTags...> /*param_tags*/,
Geometry Nodes: refactor array devirtualization Goals: * Better high level control over where devirtualization occurs. There is always a trade-off between performance and compile-time/binary-size. * Simplify using array devirtualization. * Better performance for cases where devirtualization wasn't used before. Many geometry nodes accept fields as inputs. Internally, that means that the execution functions have to accept so called "virtual arrays" as inputs. Those can be e.g. actual arrays, just single values, or lazily computed arrays. Due to these different possible virtual arrays implementations, access to individual elements is slower than it would be if everything was just a normal array (access does through a virtual function call). For more complex execution functions, this overhead does not matter, but for small functions (like a simple addition) it very much does. The virtual function call also prevents the compiler from doing some optimizations (e.g. loop unrolling and inserting simd instructions). The solution is to "devirtualize" the virtual arrays for small functions where the overhead is measurable. Essentially, the function is generated many times with different array types as input. Then there is a run-time dispatch that calls the best implementation. We have been doing devirtualization in e.g. math nodes for a long time already. This patch just generalizes the concept and makes it easier to control. It also makes it easier to investigate the different trade-offs when it comes to devirtualization. Nodes that we've optimized using devirtualization before didn't get a speedup. However, a couple of nodes are using devirtualization now, that didn't before. Those got a 2-4x speedup in common cases. * Map Range * Random Value * Switch * Combine XYZ Differential Revision: https://developer.blender.org/D14628
2022-04-26 17:12:34 +02:00
const ElementFn element_fn,
const IndexRange in_mask,
const IndexMask out_mask,
Chunks &&__restrict... chunks)
{
BLI_assert(in_mask.size() == out_mask.size());
for (const int64_t i : IndexRange(in_mask.size())) {
const int64_t in_i = in_mask[i];
const int64_t out_i = out_mask[i];
element_fn([&]() -> decltype(auto) {
using ParamTag = ParamTags;
if constexpr (ParamTag::category == MFParamCategory::SingleInput) {
return chunks[in_i];
Functions: optimize simple generated multi-functions This implements two optimizations: * Reduce virtual function call overhead when a non-standard virtual array is used as input. * Use a lambda in `type_conversion.cc`. In my test setup, which creates a float attribute filled with the index, the running time drops from `4.0 ms` to `2.0 ms`. Differential Revision: https://developer.blender.org/D14585
2022-04-07 18:48:14 +02:00
}
Geometry Nodes: refactor array devirtualization Goals: * Better high level control over where devirtualization occurs. There is always a trade-off between performance and compile-time/binary-size. * Simplify using array devirtualization. * Better performance for cases where devirtualization wasn't used before. Many geometry nodes accept fields as inputs. Internally, that means that the execution functions have to accept so called "virtual arrays" as inputs. Those can be e.g. actual arrays, just single values, or lazily computed arrays. Due to these different possible virtual arrays implementations, access to individual elements is slower than it would be if everything was just a normal array (access does through a virtual function call). For more complex execution functions, this overhead does not matter, but for small functions (like a simple addition) it very much does. The virtual function call also prevents the compiler from doing some optimizations (e.g. loop unrolling and inserting simd instructions). The solution is to "devirtualize" the virtual arrays for small functions where the overhead is measurable. Essentially, the function is generated many times with different array types as input. Then there is a run-time dispatch that calls the best implementation. We have been doing devirtualization in e.g. math nodes for a long time already. This patch just generalizes the concept and makes it easier to control. It also makes it easier to investigate the different trade-offs when it comes to devirtualization. Nodes that we've optimized using devirtualization before didn't get a speedup. However, a couple of nodes are using devirtualization now, that didn't before. Those got a 2-4x speedup in common cases. * Map Range * Random Value * Switch * Combine XYZ Differential Revision: https://developer.blender.org/D14628
2022-04-26 17:12:34 +02:00
else if constexpr (ParamTag::category == MFParamCategory::SingleOutput) {
Functions: improve devirtualization in multi-function builder This refactors how devirtualization is done in general and how multi-functions use it. * The old `Devirtualizer` class has been removed in favor of a simpler solution. It is also more general in the sense that it is not coupled with `IndexMask` and `VArray`. Instead there is a function that has inputs which control how different types are devirtualized. The new implementation is currently less general with regard to the number of parameters it supports. This can be changed in the future, but does not seem necessary now and would make the code less obvious. * Devirtualizers for different types are now defined in their respective headers. * The multi-function builder works with the `GVArray` stored in `MFParams` directly now, instead of first converting it to a `VArray<T>`. This reduces some constant overhead, which makes the multi-function slightly faster. This is only noticable when very few elements are processed though. No functional changes or performance regressions are expected.
2023-01-07 12:55:48 +01:00
return chunks + out_i;
Geometry Nodes: refactor array devirtualization Goals: * Better high level control over where devirtualization occurs. There is always a trade-off between performance and compile-time/binary-size. * Simplify using array devirtualization. * Better performance for cases where devirtualization wasn't used before. Many geometry nodes accept fields as inputs. Internally, that means that the execution functions have to accept so called "virtual arrays" as inputs. Those can be e.g. actual arrays, just single values, or lazily computed arrays. Due to these different possible virtual arrays implementations, access to individual elements is slower than it would be if everything was just a normal array (access does through a virtual function call). For more complex execution functions, this overhead does not matter, but for small functions (like a simple addition) it very much does. The virtual function call also prevents the compiler from doing some optimizations (e.g. loop unrolling and inserting simd instructions). The solution is to "devirtualize" the virtual arrays for small functions where the overhead is measurable. Essentially, the function is generated many times with different array types as input. Then there is a run-time dispatch that calls the best implementation. We have been doing devirtualization in e.g. math nodes for a long time already. This patch just generalizes the concept and makes it easier to control. It also makes it easier to investigate the different trade-offs when it comes to devirtualization. Nodes that we've optimized using devirtualization before didn't get a speedup. However, a couple of nodes are using devirtualization now, that didn't before. Those got a 2-4x speedup in common cases. * Map Range * Random Value * Switch * Combine XYZ Differential Revision: https://developer.blender.org/D14628
2022-04-26 17:12:34 +02:00
}
Functions: refactor multi-function builder API * New `build_mf` namespace for the multi-function builders. * The type name of the created multi-functions is now "private", i.e. the caller has to use `auto`. This has the benefit that the implementation can change more freely without affecting the caller. * `CustomMF` does not use `std::function` internally anymore. This reduces some overhead during code generation and at run-time. * `CustomMF` now supports single-mutable parameters.
2023-01-07 16:19:59 +01:00
else if constexpr (ParamTag::category == MFParamCategory::SingleMutable) {
return chunks[out_i];
}
Geometry Nodes: refactor array devirtualization Goals: * Better high level control over where devirtualization occurs. There is always a trade-off between performance and compile-time/binary-size. * Simplify using array devirtualization. * Better performance for cases where devirtualization wasn't used before. Many geometry nodes accept fields as inputs. Internally, that means that the execution functions have to accept so called "virtual arrays" as inputs. Those can be e.g. actual arrays, just single values, or lazily computed arrays. Due to these different possible virtual arrays implementations, access to individual elements is slower than it would be if everything was just a normal array (access does through a virtual function call). For more complex execution functions, this overhead does not matter, but for small functions (like a simple addition) it very much does. The virtual function call also prevents the compiler from doing some optimizations (e.g. loop unrolling and inserting simd instructions). The solution is to "devirtualize" the virtual arrays for small functions where the overhead is measurable. Essentially, the function is generated many times with different array types as input. Then there is a run-time dispatch that calls the best implementation. We have been doing devirtualization in e.g. math nodes for a long time already. This patch just generalizes the concept and makes it easier to control. It also makes it easier to investigate the different trade-offs when it comes to devirtualization. Nodes that we've optimized using devirtualization before didn't get a speedup. However, a couple of nodes are using devirtualization now, that didn't before. Those got a 2-4x speedup in common cases. * Map Range * Random Value * Switch * Combine XYZ Differential Revision: https://developer.blender.org/D14628
2022-04-26 17:12:34 +02:00
}()...);
Functions: allow for better compiler optimization This extracts the inner loops into a separate function. There are two main reasons for this: * Allows using `__restrict` to indicate that no other parameter aliases with the output array. This allows for better optimization. * Makes it easier to search for the generated assembly code, especially with the `BLI_NOINLINE`.
2022-04-04 11:57:39 +02:00
}
Geometry Nodes: refactor array devirtualization Goals: * Better high level control over where devirtualization occurs. There is always a trade-off between performance and compile-time/binary-size. * Simplify using array devirtualization. * Better performance for cases where devirtualization wasn't used before. Many geometry nodes accept fields as inputs. Internally, that means that the execution functions have to accept so called "virtual arrays" as inputs. Those can be e.g. actual arrays, just single values, or lazily computed arrays. Due to these different possible virtual arrays implementations, access to individual elements is slower than it would be if everything was just a normal array (access does through a virtual function call). For more complex execution functions, this overhead does not matter, but for small functions (like a simple addition) it very much does. The virtual function call also prevents the compiler from doing some optimizations (e.g. loop unrolling and inserting simd instructions). The solution is to "devirtualize" the virtual arrays for small functions where the overhead is measurable. Essentially, the function is generated many times with different array types as input. Then there is a run-time dispatch that calls the best implementation. We have been doing devirtualization in e.g. math nodes for a long time already. This patch just generalizes the concept and makes it easier to control. It also makes it easier to investigate the different trade-offs when it comes to devirtualization. Nodes that we've optimized using devirtualization before didn't get a speedup. However, a couple of nodes are using devirtualization now, that didn't before. Those got a 2-4x speedup in common cases. * Map Range * Random Value * Switch * Combine XYZ Differential Revision: https://developer.blender.org/D14628
2022-04-26 17:12:34 +02:00
}
Functions: allow for better compiler optimization This extracts the inner loops into a separate function. There are two main reasons for this: * Allows using `__restrict` to indicate that no other parameter aliases with the output array. This allows for better optimization. * Makes it easier to search for the generated assembly code, especially with the `BLI_NOINLINE`.
2022-04-04 11:57:39 +02:00
Geometry Nodes: refactor array devirtualization Goals: * Better high level control over where devirtualization occurs. There is always a trade-off between performance and compile-time/binary-size. * Simplify using array devirtualization. * Better performance for cases where devirtualization wasn't used before. Many geometry nodes accept fields as inputs. Internally, that means that the execution functions have to accept so called "virtual arrays" as inputs. Those can be e.g. actual arrays, just single values, or lazily computed arrays. Due to these different possible virtual arrays implementations, access to individual elements is slower than it would be if everything was just a normal array (access does through a virtual function call). For more complex execution functions, this overhead does not matter, but for small functions (like a simple addition) it very much does. The virtual function call also prevents the compiler from doing some optimizations (e.g. loop unrolling and inserting simd instructions). The solution is to "devirtualize" the virtual arrays for small functions where the overhead is measurable. Essentially, the function is generated many times with different array types as input. Then there is a run-time dispatch that calls the best implementation. We have been doing devirtualization in e.g. math nodes for a long time already. This patch just generalizes the concept and makes it easier to control. It also makes it easier to investigate the different trade-offs when it comes to devirtualization. Nodes that we've optimized using devirtualization before didn't get a speedup. However, a couple of nodes are using devirtualization now, that didn't before. Those got a 2-4x speedup in common cases. * Map Range * Random Value * Switch * Combine XYZ Differential Revision: https://developer.blender.org/D14628
2022-04-26 17:12:34 +02:00
/**
* Executes #element_fn for all indices in #mask. However, instead of processing every element
* separately, processing happens in chunks. This allows retrieving from input virtual arrays in
* chunks, which reduces virtual function call overhead.
*/
Functions: improve devirtualization in multi-function builder This refactors how devirtualization is done in general and how multi-functions use it. * The old `Devirtualizer` class has been removed in favor of a simpler solution. It is also more general in the sense that it is not coupled with `IndexMask` and `VArray`. Instead there is a function that has inputs which control how different types are devirtualized. The new implementation is currently less general with regard to the number of parameters it supports. This can be changed in the future, but does not seem necessary now and would make the code less obvious. * Devirtualizers for different types are now defined in their respective headers. * The multi-function builder works with the `GVArray` stored in `MFParams` directly now, instead of first converting it to a `VArray<T>`. This reduces some constant overhead, which makes the multi-function slightly faster. This is only noticable when very few elements are processed though. No functional changes or performance regressions are expected.
2023-01-07 12:55:48 +01:00
template<typename... ParamTags, size_t... I, typename ElementFn, typename... LoadedParams>
Geometry Nodes: refactor array devirtualization Goals: * Better high level control over where devirtualization occurs. There is always a trade-off between performance and compile-time/binary-size. * Simplify using array devirtualization. * Better performance for cases where devirtualization wasn't used before. Many geometry nodes accept fields as inputs. Internally, that means that the execution functions have to accept so called "virtual arrays" as inputs. Those can be e.g. actual arrays, just single values, or lazily computed arrays. Due to these different possible virtual arrays implementations, access to individual elements is slower than it would be if everything was just a normal array (access does through a virtual function call). For more complex execution functions, this overhead does not matter, but for small functions (like a simple addition) it very much does. The virtual function call also prevents the compiler from doing some optimizations (e.g. loop unrolling and inserting simd instructions). The solution is to "devirtualize" the virtual arrays for small functions where the overhead is measurable. Essentially, the function is generated many times with different array types as input. Then there is a run-time dispatch that calls the best implementation. We have been doing devirtualization in e.g. math nodes for a long time already. This patch just generalizes the concept and makes it easier to control. It also makes it easier to investigate the different trade-offs when it comes to devirtualization. Nodes that we've optimized using devirtualization before didn't get a speedup. However, a couple of nodes are using devirtualization now, that didn't before. Those got a 2-4x speedup in common cases. * Map Range * Random Value * Switch * Combine XYZ Differential Revision: https://developer.blender.org/D14628
2022-04-26 17:12:34 +02:00
void execute_materialized(TypeSequence<ParamTags...> /* param_tags */,
std::index_sequence<I...> /* indices */,
const ElementFn element_fn,
const IndexMask mask,
Functions: improve devirtualization in multi-function builder This refactors how devirtualization is done in general and how multi-functions use it. * The old `Devirtualizer` class has been removed in favor of a simpler solution. It is also more general in the sense that it is not coupled with `IndexMask` and `VArray`. Instead there is a function that has inputs which control how different types are devirtualized. The new implementation is currently less general with regard to the number of parameters it supports. This can be changed in the future, but does not seem necessary now and would make the code less obvious. * Devirtualizers for different types are now defined in their respective headers. * The multi-function builder works with the `GVArray` stored in `MFParams` directly now, instead of first converting it to a `VArray<T>`. This reduces some constant overhead, which makes the multi-function slightly faster. This is only noticable when very few elements are processed though. No functional changes or performance regressions are expected.
2023-01-07 12:55:48 +01:00
const std::tuple<LoadedParams...> &loaded_params)
Geometry Nodes: refactor array devirtualization Goals: * Better high level control over where devirtualization occurs. There is always a trade-off between performance and compile-time/binary-size. * Simplify using array devirtualization. * Better performance for cases where devirtualization wasn't used before. Many geometry nodes accept fields as inputs. Internally, that means that the execution functions have to accept so called "virtual arrays" as inputs. Those can be e.g. actual arrays, just single values, or lazily computed arrays. Due to these different possible virtual arrays implementations, access to individual elements is slower than it would be if everything was just a normal array (access does through a virtual function call). For more complex execution functions, this overhead does not matter, but for small functions (like a simple addition) it very much does. The virtual function call also prevents the compiler from doing some optimizations (e.g. loop unrolling and inserting simd instructions). The solution is to "devirtualize" the virtual arrays for small functions where the overhead is measurable. Essentially, the function is generated many times with different array types as input. Then there is a run-time dispatch that calls the best implementation. We have been doing devirtualization in e.g. math nodes for a long time already. This patch just generalizes the concept and makes it easier to control. It also makes it easier to investigate the different trade-offs when it comes to devirtualization. Nodes that we've optimized using devirtualization before didn't get a speedup. However, a couple of nodes are using devirtualization now, that didn't before. Those got a 2-4x speedup in common cases. * Map Range * Random Value * Switch * Combine XYZ Differential Revision: https://developer.blender.org/D14628
2022-04-26 17:12:34 +02:00
{
/* In theory, all elements could be processed in one chunk. However, that has the disadvantage
* that large temporary arrays are needed. Using small chunks allows using small arrays, which
* are reused multiple times, which improves cache efficiency. The chunk size also shouldn't be
* too small, because then overhead of the outer loop over chunks becomes significant again. */
static constexpr int64_t MaxChunkSize = 32;
const int64_t mask_size = mask.size();
const int64_t buffer_size = std::min(mask_size, MaxChunkSize);
/* Local buffers that are used to temporarily store values retrieved from virtual arrays. */
std::tuple<TypedBuffer<typename ParamTags::base_type, MaxChunkSize>...> buffers_owner;
/* A span for each parameter which is either empty or points to memory in #buffers_owner. */
std::tuple<MutableSpan<typename ParamTags::base_type>...> buffers;
/* Information about every parameter. */
Functions: refactor multi-function builder API * New `build_mf` namespace for the multi-function builders. * The type name of the created multi-functions is now "private", i.e. the caller has to use `auto`. This has the benefit that the implementation can change more freely without affecting the caller. * `CustomMF` does not use `std::function` internally anymore. This reduces some overhead during code generation and at run-time. * `CustomMF` now supports single-mutable parameters.
2023-01-07 16:19:59 +01:00
std::tuple<MaterializeArgInfo<ParamTags>...> args_info;
Geometry Nodes: refactor array devirtualization Goals: * Better high level control over where devirtualization occurs. There is always a trade-off between performance and compile-time/binary-size. * Simplify using array devirtualization. * Better performance for cases where devirtualization wasn't used before. Many geometry nodes accept fields as inputs. Internally, that means that the execution functions have to accept so called "virtual arrays" as inputs. Those can be e.g. actual arrays, just single values, or lazily computed arrays. Due to these different possible virtual arrays implementations, access to individual elements is slower than it would be if everything was just a normal array (access does through a virtual function call). For more complex execution functions, this overhead does not matter, but for small functions (like a simple addition) it very much does. The virtual function call also prevents the compiler from doing some optimizations (e.g. loop unrolling and inserting simd instructions). The solution is to "devirtualize" the virtual arrays for small functions where the overhead is measurable. Essentially, the function is generated many times with different array types as input. Then there is a run-time dispatch that calls the best implementation. We have been doing devirtualization in e.g. math nodes for a long time already. This patch just generalizes the concept and makes it easier to control. It also makes it easier to investigate the different trade-offs when it comes to devirtualization. Nodes that we've optimized using devirtualization before didn't get a speedup. However, a couple of nodes are using devirtualization now, that didn't before. Those got a 2-4x speedup in common cases. * Map Range * Random Value * Switch * Combine XYZ Differential Revision: https://developer.blender.org/D14628
2022-04-26 17:12:34 +02:00
(
/* Setup information for all parameters. */
[&] {
Cleanup: add hint that typedef is used as compiler bug workaround https://developercommunity.visualstudio.com/t/Alias-template-inside-fold-expression-fa/10040507
2022-09-20 10:42:25 +02:00
/* Use `typedef` instead of `using` to work around a compiler bug. */
Workaround for msvc compiler bug https://developercommunity.visualstudio.com/t/Alias-template-inside-fold-expression-fa/10040507
2022-05-12 13:38:22 +02:00
typedef ParamTags ParamTag;
typedef typename ParamTag::base_type T;
Functions: refactor multi-function builder API * New `build_mf` namespace for the multi-function builders. * The type name of the created multi-functions is now "private", i.e. the caller has to use `auto`. This has the benefit that the implementation can change more freely without affecting the caller. * `CustomMF` does not use `std::function` internally anymore. This reduces some overhead during code generation and at run-time. * `CustomMF` now supports single-mutable parameters.
2023-01-07 16:19:59 +01:00
[[maybe_unused]] MaterializeArgInfo<ParamTags> &arg_info = std::get<I>(args_info);
Geometry Nodes: refactor array devirtualization Goals: * Better high level control over where devirtualization occurs. There is always a trade-off between performance and compile-time/binary-size. * Simplify using array devirtualization. * Better performance for cases where devirtualization wasn't used before. Many geometry nodes accept fields as inputs. Internally, that means that the execution functions have to accept so called "virtual arrays" as inputs. Those can be e.g. actual arrays, just single values, or lazily computed arrays. Due to these different possible virtual arrays implementations, access to individual elements is slower than it would be if everything was just a normal array (access does through a virtual function call). For more complex execution functions, this overhead does not matter, but for small functions (like a simple addition) it very much does. The virtual function call also prevents the compiler from doing some optimizations (e.g. loop unrolling and inserting simd instructions). The solution is to "devirtualize" the virtual arrays for small functions where the overhead is measurable. Essentially, the function is generated many times with different array types as input. Then there is a run-time dispatch that calls the best implementation. We have been doing devirtualization in e.g. math nodes for a long time already. This patch just generalizes the concept and makes it easier to control. It also makes it easier to investigate the different trade-offs when it comes to devirtualization. Nodes that we've optimized using devirtualization before didn't get a speedup. However, a couple of nodes are using devirtualization now, that didn't before. Those got a 2-4x speedup in common cases. * Map Range * Random Value * Switch * Combine XYZ Differential Revision: https://developer.blender.org/D14628
2022-04-26 17:12:34 +02:00
if constexpr (ParamTag::category == MFParamCategory::SingleInput) {
Functions: improve devirtualization in multi-function builder This refactors how devirtualization is done in general and how multi-functions use it. * The old `Devirtualizer` class has been removed in favor of a simpler solution. It is also more general in the sense that it is not coupled with `IndexMask` and `VArray`. Instead there is a function that has inputs which control how different types are devirtualized. The new implementation is currently less general with regard to the number of parameters it supports. This can be changed in the future, but does not seem necessary now and would make the code less obvious. * Devirtualizers for different types are now defined in their respective headers. * The multi-function builder works with the `GVArray` stored in `MFParams` directly now, instead of first converting it to a `VArray<T>`. This reduces some constant overhead, which makes the multi-function slightly faster. This is only noticable when very few elements are processed though. No functional changes or performance regressions are expected.
2023-01-07 12:55:48 +01:00
const GVArrayImpl &varray_impl = *std::get<I>(loaded_params);
const CommonVArrayInfo common_info = varray_impl.common_info();
if (common_info.type == CommonVArrayInfo::Type::Single) {
Geometry Nodes: refactor array devirtualization Goals: * Better high level control over where devirtualization occurs. There is always a trade-off between performance and compile-time/binary-size. * Simplify using array devirtualization. * Better performance for cases where devirtualization wasn't used before. Many geometry nodes accept fields as inputs. Internally, that means that the execution functions have to accept so called "virtual arrays" as inputs. Those can be e.g. actual arrays, just single values, or lazily computed arrays. Due to these different possible virtual arrays implementations, access to individual elements is slower than it would be if everything was just a normal array (access does through a virtual function call). For more complex execution functions, this overhead does not matter, but for small functions (like a simple addition) it very much does. The virtual function call also prevents the compiler from doing some optimizations (e.g. loop unrolling and inserting simd instructions). The solution is to "devirtualize" the virtual arrays for small functions where the overhead is measurable. Essentially, the function is generated many times with different array types as input. Then there is a run-time dispatch that calls the best implementation. We have been doing devirtualization in e.g. math nodes for a long time already. This patch just generalizes the concept and makes it easier to control. It also makes it easier to investigate the different trade-offs when it comes to devirtualization. Nodes that we've optimized using devirtualization before didn't get a speedup. However, a couple of nodes are using devirtualization now, that didn't before. Those got a 2-4x speedup in common cases. * Map Range * Random Value * Switch * Combine XYZ Differential Revision: https://developer.blender.org/D14628
2022-04-26 17:12:34 +02:00
/* If an input #VArray is a single value, we have to fill the buffer with that value
* only once. The same unchanged buffer can then be reused in every chunk. */
MutableSpan<T> in_chunk{std::get<I>(buffers_owner).ptr(), buffer_size};
Functions: improve devirtualization in multi-function builder This refactors how devirtualization is done in general and how multi-functions use it. * The old `Devirtualizer` class has been removed in favor of a simpler solution. It is also more general in the sense that it is not coupled with `IndexMask` and `VArray`. Instead there is a function that has inputs which control how different types are devirtualized. The new implementation is currently less general with regard to the number of parameters it supports. This can be changed in the future, but does not seem necessary now and would make the code less obvious. * Devirtualizers for different types are now defined in their respective headers. * The multi-function builder works with the `GVArray` stored in `MFParams` directly now, instead of first converting it to a `VArray<T>`. This reduces some constant overhead, which makes the multi-function slightly faster. This is only noticable when very few elements are processed though. No functional changes or performance regressions are expected.
2023-01-07 12:55:48 +01:00
const T &in_single = *static_cast<const T *>(common_info.data);
Geometry Nodes: refactor array devirtualization Goals: * Better high level control over where devirtualization occurs. There is always a trade-off between performance and compile-time/binary-size. * Simplify using array devirtualization. * Better performance for cases where devirtualization wasn't used before. Many geometry nodes accept fields as inputs. Internally, that means that the execution functions have to accept so called "virtual arrays" as inputs. Those can be e.g. actual arrays, just single values, or lazily computed arrays. Due to these different possible virtual arrays implementations, access to individual elements is slower than it would be if everything was just a normal array (access does through a virtual function call). For more complex execution functions, this overhead does not matter, but for small functions (like a simple addition) it very much does. The virtual function call also prevents the compiler from doing some optimizations (e.g. loop unrolling and inserting simd instructions). The solution is to "devirtualize" the virtual arrays for small functions where the overhead is measurable. Essentially, the function is generated many times with different array types as input. Then there is a run-time dispatch that calls the best implementation. We have been doing devirtualization in e.g. math nodes for a long time already. This patch just generalizes the concept and makes it easier to control. It also makes it easier to investigate the different trade-offs when it comes to devirtualization. Nodes that we've optimized using devirtualization before didn't get a speedup. However, a couple of nodes are using devirtualization now, that didn't before. Those got a 2-4x speedup in common cases. * Map Range * Random Value * Switch * Combine XYZ Differential Revision: https://developer.blender.org/D14628
2022-04-26 17:12:34 +02:00
uninitialized_fill_n(in_chunk.data(), in_chunk.size(), in_single);
std::get<I>(buffers) = in_chunk;
Functions: refactor multi-function builder API * New `build_mf` namespace for the multi-function builders. * The type name of the created multi-functions is now "private", i.e. the caller has to use `auto`. This has the benefit that the implementation can change more freely without affecting the caller. * `CustomMF` does not use `std::function` internally anymore. This reduces some overhead during code generation and at run-time. * `CustomMF` now supports single-mutable parameters.
2023-01-07 16:19:59 +01:00
arg_info.mode = MaterializeArgMode::Single;
Geometry Nodes: refactor array devirtualization Goals: * Better high level control over where devirtualization occurs. There is always a trade-off between performance and compile-time/binary-size. * Simplify using array devirtualization. * Better performance for cases where devirtualization wasn't used before. Many geometry nodes accept fields as inputs. Internally, that means that the execution functions have to accept so called "virtual arrays" as inputs. Those can be e.g. actual arrays, just single values, or lazily computed arrays. Due to these different possible virtual arrays implementations, access to individual elements is slower than it would be if everything was just a normal array (access does through a virtual function call). For more complex execution functions, this overhead does not matter, but for small functions (like a simple addition) it very much does. The virtual function call also prevents the compiler from doing some optimizations (e.g. loop unrolling and inserting simd instructions). The solution is to "devirtualize" the virtual arrays for small functions where the overhead is measurable. Essentially, the function is generated many times with different array types as input. Then there is a run-time dispatch that calls the best implementation. We have been doing devirtualization in e.g. math nodes for a long time already. This patch just generalizes the concept and makes it easier to control. It also makes it easier to investigate the different trade-offs when it comes to devirtualization. Nodes that we've optimized using devirtualization before didn't get a speedup. However, a couple of nodes are using devirtualization now, that didn't before. Those got a 2-4x speedup in common cases. * Map Range * Random Value * Switch * Combine XYZ Differential Revision: https://developer.blender.org/D14628
2022-04-26 17:12:34 +02:00
}
Functions: improve devirtualization in multi-function builder This refactors how devirtualization is done in general and how multi-functions use it. * The old `Devirtualizer` class has been removed in favor of a simpler solution. It is also more general in the sense that it is not coupled with `IndexMask` and `VArray`. Instead there is a function that has inputs which control how different types are devirtualized. The new implementation is currently less general with regard to the number of parameters it supports. This can be changed in the future, but does not seem necessary now and would make the code less obvious. * Devirtualizers for different types are now defined in their respective headers. * The multi-function builder works with the `GVArray` stored in `MFParams` directly now, instead of first converting it to a `VArray<T>`. This reduces some constant overhead, which makes the multi-function slightly faster. This is only noticable when very few elements are processed though. No functional changes or performance regressions are expected.
2023-01-07 12:55:48 +01:00
else if (common_info.type == CommonVArrayInfo::Type::Span) {
Geometry Nodes: refactor array devirtualization Goals: * Better high level control over where devirtualization occurs. There is always a trade-off between performance and compile-time/binary-size. * Simplify using array devirtualization. * Better performance for cases where devirtualization wasn't used before. Many geometry nodes accept fields as inputs. Internally, that means that the execution functions have to accept so called "virtual arrays" as inputs. Those can be e.g. actual arrays, just single values, or lazily computed arrays. Due to these different possible virtual arrays implementations, access to individual elements is slower than it would be if everything was just a normal array (access does through a virtual function call). For more complex execution functions, this overhead does not matter, but for small functions (like a simple addition) it very much does. The virtual function call also prevents the compiler from doing some optimizations (e.g. loop unrolling and inserting simd instructions). The solution is to "devirtualize" the virtual arrays for small functions where the overhead is measurable. Essentially, the function is generated many times with different array types as input. Then there is a run-time dispatch that calls the best implementation. We have been doing devirtualization in e.g. math nodes for a long time already. This patch just generalizes the concept and makes it easier to control. It also makes it easier to investigate the different trade-offs when it comes to devirtualization. Nodes that we've optimized using devirtualization before didn't get a speedup. However, a couple of nodes are using devirtualization now, that didn't before. Those got a 2-4x speedup in common cases. * Map Range * Random Value * Switch * Combine XYZ Differential Revision: https://developer.blender.org/D14628
2022-04-26 17:12:34 +02:00
/* Remember the span so that it doesn't have to be retrieved in every iteration. */
Functions: improve devirtualization in multi-function builder This refactors how devirtualization is done in general and how multi-functions use it. * The old `Devirtualizer` class has been removed in favor of a simpler solution. It is also more general in the sense that it is not coupled with `IndexMask` and `VArray`. Instead there is a function that has inputs which control how different types are devirtualized. The new implementation is currently less general with regard to the number of parameters it supports. This can be changed in the future, but does not seem necessary now and would make the code less obvious. * Devirtualizers for different types are now defined in their respective headers. * The multi-function builder works with the `GVArray` stored in `MFParams` directly now, instead of first converting it to a `VArray<T>`. This reduces some constant overhead, which makes the multi-function slightly faster. This is only noticable when very few elements are processed though. No functional changes or performance regressions are expected.
2023-01-07 12:55:48 +01:00
const T *ptr = static_cast<const T *>(common_info.data);
arg_info.internal_span = Span<T>(ptr, varray_impl.size());
Geometry Nodes: refactor array devirtualization Goals: * Better high level control over where devirtualization occurs. There is always a trade-off between performance and compile-time/binary-size. * Simplify using array devirtualization. * Better performance for cases where devirtualization wasn't used before. Many geometry nodes accept fields as inputs. Internally, that means that the execution functions have to accept so called "virtual arrays" as inputs. Those can be e.g. actual arrays, just single values, or lazily computed arrays. Due to these different possible virtual arrays implementations, access to individual elements is slower than it would be if everything was just a normal array (access does through a virtual function call). For more complex execution functions, this overhead does not matter, but for small functions (like a simple addition) it very much does. The virtual function call also prevents the compiler from doing some optimizations (e.g. loop unrolling and inserting simd instructions). The solution is to "devirtualize" the virtual arrays for small functions where the overhead is measurable. Essentially, the function is generated many times with different array types as input. Then there is a run-time dispatch that calls the best implementation. We have been doing devirtualization in e.g. math nodes for a long time already. This patch just generalizes the concept and makes it easier to control. It also makes it easier to investigate the different trade-offs when it comes to devirtualization. Nodes that we've optimized using devirtualization before didn't get a speedup. However, a couple of nodes are using devirtualization now, that didn't before. Those got a 2-4x speedup in common cases. * Map Range * Random Value * Switch * Combine XYZ Differential Revision: https://developer.blender.org/D14628
2022-04-26 17:12:34 +02:00
}
}
}(),
...);
/* Outer loop over all chunks. */
for (int64_t chunk_start = 0; chunk_start < mask_size; chunk_start += MaxChunkSize) {
BLI: Follow up and fix recent span slicing change a5e7657ceeb6cc didn't account for slices of zero sizes, and the asserts were slightly incorrect otherwise. Also, the change didn't apply to `Span`, only `MutableSpan`, which was a mistake. This also adds "safe" methods to `IndexMask`, and switches function calls where necessary.
2022-11-23 11:35:59 -06:00
const IndexMask sliced_mask = mask.slice_safe(chunk_start, MaxChunkSize);
Geometry Nodes: refactor array devirtualization Goals: * Better high level control over where devirtualization occurs. There is always a trade-off between performance and compile-time/binary-size. * Simplify using array devirtualization. * Better performance for cases where devirtualization wasn't used before. Many geometry nodes accept fields as inputs. Internally, that means that the execution functions have to accept so called "virtual arrays" as inputs. Those can be e.g. actual arrays, just single values, or lazily computed arrays. Due to these different possible virtual arrays implementations, access to individual elements is slower than it would be if everything was just a normal array (access does through a virtual function call). For more complex execution functions, this overhead does not matter, but for small functions (like a simple addition) it very much does. The virtual function call also prevents the compiler from doing some optimizations (e.g. loop unrolling and inserting simd instructions). The solution is to "devirtualize" the virtual arrays for small functions where the overhead is measurable. Essentially, the function is generated many times with different array types as input. Then there is a run-time dispatch that calls the best implementation. We have been doing devirtualization in e.g. math nodes for a long time already. This patch just generalizes the concept and makes it easier to control. It also makes it easier to investigate the different trade-offs when it comes to devirtualization. Nodes that we've optimized using devirtualization before didn't get a speedup. However, a couple of nodes are using devirtualization now, that didn't before. Those got a 2-4x speedup in common cases. * Map Range * Random Value * Switch * Combine XYZ Differential Revision: https://developer.blender.org/D14628
2022-04-26 17:12:34 +02:00
const int64_t chunk_size = sliced_mask.size();
const bool sliced_mask_is_range = sliced_mask.is_range();
execute_materialized_impl(
TypeSequence<ParamTags...>(),
element_fn,
/* Inputs are "compressed" into contiguous arrays without gaps. */
IndexRange(chunk_size),
/* Outputs are written directly into the correct place in the output arrays. */
sliced_mask,
/* Prepare every parameter for this chunk. */
[&] {
using ParamTag = ParamTags;
using T = typename ParamTag::base_type;
Functions: refactor multi-function builder API * New `build_mf` namespace for the multi-function builders. * The type name of the created multi-functions is now "private", i.e. the caller has to use `auto`. This has the benefit that the implementation can change more freely without affecting the caller. * `CustomMF` does not use `std::function` internally anymore. This reduces some overhead during code generation and at run-time. * `CustomMF` now supports single-mutable parameters.
2023-01-07 16:19:59 +01:00
[[maybe_unused]] MaterializeArgInfo<ParamTags> &arg_info = std::get<I>(args_info);
Geometry Nodes: refactor array devirtualization Goals: * Better high level control over where devirtualization occurs. There is always a trade-off between performance and compile-time/binary-size. * Simplify using array devirtualization. * Better performance for cases where devirtualization wasn't used before. Many geometry nodes accept fields as inputs. Internally, that means that the execution functions have to accept so called "virtual arrays" as inputs. Those can be e.g. actual arrays, just single values, or lazily computed arrays. Due to these different possible virtual arrays implementations, access to individual elements is slower than it would be if everything was just a normal array (access does through a virtual function call). For more complex execution functions, this overhead does not matter, but for small functions (like a simple addition) it very much does. The virtual function call also prevents the compiler from doing some optimizations (e.g. loop unrolling and inserting simd instructions). The solution is to "devirtualize" the virtual arrays for small functions where the overhead is measurable. Essentially, the function is generated many times with different array types as input. Then there is a run-time dispatch that calls the best implementation. We have been doing devirtualization in e.g. math nodes for a long time already. This patch just generalizes the concept and makes it easier to control. It also makes it easier to investigate the different trade-offs when it comes to devirtualization. Nodes that we've optimized using devirtualization before didn't get a speedup. However, a couple of nodes are using devirtualization now, that didn't before. Those got a 2-4x speedup in common cases. * Map Range * Random Value * Switch * Combine XYZ Differential Revision: https://developer.blender.org/D14628
2022-04-26 17:12:34 +02:00
if constexpr (ParamTag::category == MFParamCategory::SingleInput) {
Functions: refactor multi-function builder API * New `build_mf` namespace for the multi-function builders. * The type name of the created multi-functions is now "private", i.e. the caller has to use `auto`. This has the benefit that the implementation can change more freely without affecting the caller. * `CustomMF` does not use `std::function` internally anymore. This reduces some overhead during code generation and at run-time. * `CustomMF` now supports single-mutable parameters.
2023-01-07 16:19:59 +01:00
if (arg_info.mode == MaterializeArgMode::Single) {
Geometry Nodes: refactor array devirtualization Goals: * Better high level control over where devirtualization occurs. There is always a trade-off between performance and compile-time/binary-size. * Simplify using array devirtualization. * Better performance for cases where devirtualization wasn't used before. Many geometry nodes accept fields as inputs. Internally, that means that the execution functions have to accept so called "virtual arrays" as inputs. Those can be e.g. actual arrays, just single values, or lazily computed arrays. Due to these different possible virtual arrays implementations, access to individual elements is slower than it would be if everything was just a normal array (access does through a virtual function call). For more complex execution functions, this overhead does not matter, but for small functions (like a simple addition) it very much does. The virtual function call also prevents the compiler from doing some optimizations (e.g. loop unrolling and inserting simd instructions). The solution is to "devirtualize" the virtual arrays for small functions where the overhead is measurable. Essentially, the function is generated many times with different array types as input. Then there is a run-time dispatch that calls the best implementation. We have been doing devirtualization in e.g. math nodes for a long time already. This patch just generalizes the concept and makes it easier to control. It also makes it easier to investigate the different trade-offs when it comes to devirtualization. Nodes that we've optimized using devirtualization before didn't get a speedup. However, a couple of nodes are using devirtualization now, that didn't before. Those got a 2-4x speedup in common cases. * Map Range * Random Value * Switch * Combine XYZ Differential Revision: https://developer.blender.org/D14628
2022-04-26 17:12:34 +02:00
/* The single value has been filled into a buffer already reused for every chunk. */
return Span<T>(std::get<I>(buffers));
}
else {
if (sliced_mask_is_range) {
if (!arg_info.internal_span.is_empty()) {
/* In this case we can just use an existing span instead of "compressing" it into
* a new temporary buffer. */
const IndexRange sliced_mask_range = sliced_mask.as_range();
Functions: refactor multi-function builder API * New `build_mf` namespace for the multi-function builders. * The type name of the created multi-functions is now "private", i.e. the caller has to use `auto`. This has the benefit that the implementation can change more freely without affecting the caller. * `CustomMF` does not use `std::function` internally anymore. This reduces some overhead during code generation and at run-time. * `CustomMF` now supports single-mutable parameters.
2023-01-07 16:19:59 +01:00
arg_info.mode = MaterializeArgMode::Span;
Geometry Nodes: refactor array devirtualization Goals: * Better high level control over where devirtualization occurs. There is always a trade-off between performance and compile-time/binary-size. * Simplify using array devirtualization. * Better performance for cases where devirtualization wasn't used before. Many geometry nodes accept fields as inputs. Internally, that means that the execution functions have to accept so called "virtual arrays" as inputs. Those can be e.g. actual arrays, just single values, or lazily computed arrays. Due to these different possible virtual arrays implementations, access to individual elements is slower than it would be if everything was just a normal array (access does through a virtual function call). For more complex execution functions, this overhead does not matter, but for small functions (like a simple addition) it very much does. The virtual function call also prevents the compiler from doing some optimizations (e.g. loop unrolling and inserting simd instructions). The solution is to "devirtualize" the virtual arrays for small functions where the overhead is measurable. Essentially, the function is generated many times with different array types as input. Then there is a run-time dispatch that calls the best implementation. We have been doing devirtualization in e.g. math nodes for a long time already. This patch just generalizes the concept and makes it easier to control. It also makes it easier to investigate the different trade-offs when it comes to devirtualization. Nodes that we've optimized using devirtualization before didn't get a speedup. However, a couple of nodes are using devirtualization now, that didn't before. Those got a 2-4x speedup in common cases. * Map Range * Random Value * Switch * Combine XYZ Differential Revision: https://developer.blender.org/D14628
2022-04-26 17:12:34 +02:00
return arg_info.internal_span.slice(sliced_mask_range);
}
}
Functions: improve devirtualization in multi-function builder This refactors how devirtualization is done in general and how multi-functions use it. * The old `Devirtualizer` class has been removed in favor of a simpler solution. It is also more general in the sense that it is not coupled with `IndexMask` and `VArray`. Instead there is a function that has inputs which control how different types are devirtualized. The new implementation is currently less general with regard to the number of parameters it supports. This can be changed in the future, but does not seem necessary now and would make the code less obvious. * Devirtualizers for different types are now defined in their respective headers. * The multi-function builder works with the `GVArray` stored in `MFParams` directly now, instead of first converting it to a `VArray<T>`. This reduces some constant overhead, which makes the multi-function slightly faster. This is only noticable when very few elements are processed though. No functional changes or performance regressions are expected.
2023-01-07 12:55:48 +01:00
const GVArrayImpl &varray_impl = *std::get<I>(loaded_params);
Geometry Nodes: refactor array devirtualization Goals: * Better high level control over where devirtualization occurs. There is always a trade-off between performance and compile-time/binary-size. * Simplify using array devirtualization. * Better performance for cases where devirtualization wasn't used before. Many geometry nodes accept fields as inputs. Internally, that means that the execution functions have to accept so called "virtual arrays" as inputs. Those can be e.g. actual arrays, just single values, or lazily computed arrays. Due to these different possible virtual arrays implementations, access to individual elements is slower than it would be if everything was just a normal array (access does through a virtual function call). For more complex execution functions, this overhead does not matter, but for small functions (like a simple addition) it very much does. The virtual function call also prevents the compiler from doing some optimizations (e.g. loop unrolling and inserting simd instructions). The solution is to "devirtualize" the virtual arrays for small functions where the overhead is measurable. Essentially, the function is generated many times with different array types as input. Then there is a run-time dispatch that calls the best implementation. We have been doing devirtualization in e.g. math nodes for a long time already. This patch just generalizes the concept and makes it easier to control. It also makes it easier to investigate the different trade-offs when it comes to devirtualization. Nodes that we've optimized using devirtualization before didn't get a speedup. However, a couple of nodes are using devirtualization now, that didn't before. Those got a 2-4x speedup in common cases. * Map Range * Random Value * Switch * Combine XYZ Differential Revision: https://developer.blender.org/D14628
2022-04-26 17:12:34 +02:00
/* As a fallback, do a virtual function call to retrieve all elements in the current
* chunk. The elements are stored in a temporary buffer reused for every chunk. */
MutableSpan<T> in_chunk{std::get<I>(buffers_owner).ptr(), chunk_size};
Functions: improve devirtualization in multi-function builder This refactors how devirtualization is done in general and how multi-functions use it. * The old `Devirtualizer` class has been removed in favor of a simpler solution. It is also more general in the sense that it is not coupled with `IndexMask` and `VArray`. Instead there is a function that has inputs which control how different types are devirtualized. The new implementation is currently less general with regard to the number of parameters it supports. This can be changed in the future, but does not seem necessary now and would make the code less obvious. * Devirtualizers for different types are now defined in their respective headers. * The multi-function builder works with the `GVArray` stored in `MFParams` directly now, instead of first converting it to a `VArray<T>`. This reduces some constant overhead, which makes the multi-function slightly faster. This is only noticable when very few elements are processed though. No functional changes or performance regressions are expected.
2023-01-07 12:55:48 +01:00
varray_impl.materialize_compressed_to_uninitialized(sliced_mask, in_chunk.data());
Geometry Nodes: refactor array devirtualization Goals: * Better high level control over where devirtualization occurs. There is always a trade-off between performance and compile-time/binary-size. * Simplify using array devirtualization. * Better performance for cases where devirtualization wasn't used before. Many geometry nodes accept fields as inputs. Internally, that means that the execution functions have to accept so called "virtual arrays" as inputs. Those can be e.g. actual arrays, just single values, or lazily computed arrays. Due to these different possible virtual arrays implementations, access to individual elements is slower than it would be if everything was just a normal array (access does through a virtual function call). For more complex execution functions, this overhead does not matter, but for small functions (like a simple addition) it very much does. The virtual function call also prevents the compiler from doing some optimizations (e.g. loop unrolling and inserting simd instructions). The solution is to "devirtualize" the virtual arrays for small functions where the overhead is measurable. Essentially, the function is generated many times with different array types as input. Then there is a run-time dispatch that calls the best implementation. We have been doing devirtualization in e.g. math nodes for a long time already. This patch just generalizes the concept and makes it easier to control. It also makes it easier to investigate the different trade-offs when it comes to devirtualization. Nodes that we've optimized using devirtualization before didn't get a speedup. However, a couple of nodes are using devirtualization now, that didn't before. Those got a 2-4x speedup in common cases. * Map Range * Random Value * Switch * Combine XYZ Differential Revision: https://developer.blender.org/D14628
2022-04-26 17:12:34 +02:00
/* Remember that this parameter has been materialized, so that the values are
* destructed properly when the chunk is done. */
Functions: refactor multi-function builder API * New `build_mf` namespace for the multi-function builders. * The type name of the created multi-functions is now "private", i.e. the caller has to use `auto`. This has the benefit that the implementation can change more freely without affecting the caller. * `CustomMF` does not use `std::function` internally anymore. This reduces some overhead during code generation and at run-time. * `CustomMF` now supports single-mutable parameters.
2023-01-07 16:19:59 +01:00
arg_info.mode = MaterializeArgMode::Materialized;
Geometry Nodes: refactor array devirtualization Goals: * Better high level control over where devirtualization occurs. There is always a trade-off between performance and compile-time/binary-size. * Simplify using array devirtualization. * Better performance for cases where devirtualization wasn't used before. Many geometry nodes accept fields as inputs. Internally, that means that the execution functions have to accept so called "virtual arrays" as inputs. Those can be e.g. actual arrays, just single values, or lazily computed arrays. Due to these different possible virtual arrays implementations, access to individual elements is slower than it would be if everything was just a normal array (access does through a virtual function call). For more complex execution functions, this overhead does not matter, but for small functions (like a simple addition) it very much does. The virtual function call also prevents the compiler from doing some optimizations (e.g. loop unrolling and inserting simd instructions). The solution is to "devirtualize" the virtual arrays for small functions where the overhead is measurable. Essentially, the function is generated many times with different array types as input. Then there is a run-time dispatch that calls the best implementation. We have been doing devirtualization in e.g. math nodes for a long time already. This patch just generalizes the concept and makes it easier to control. It also makes it easier to investigate the different trade-offs when it comes to devirtualization. Nodes that we've optimized using devirtualization before didn't get a speedup. However, a couple of nodes are using devirtualization now, that didn't before. Those got a 2-4x speedup in common cases. * Map Range * Random Value * Switch * Combine XYZ Differential Revision: https://developer.blender.org/D14628
2022-04-26 17:12:34 +02:00
return Span<T>(in_chunk);
}
}
Functions: refactor multi-function builder API * New `build_mf` namespace for the multi-function builders. * The type name of the created multi-functions is now "private", i.e. the caller has to use `auto`. This has the benefit that the implementation can change more freely without affecting the caller. * `CustomMF` does not use `std::function` internally anymore. This reduces some overhead during code generation and at run-time. * `CustomMF` now supports single-mutable parameters.
2023-01-07 16:19:59 +01:00
else if constexpr (ELEM(ParamTag::category,
MFParamCategory::SingleOutput,
MFParamCategory::SingleMutable)) {
Geometry Nodes: refactor array devirtualization Goals: * Better high level control over where devirtualization occurs. There is always a trade-off between performance and compile-time/binary-size. * Simplify using array devirtualization. * Better performance for cases where devirtualization wasn't used before. Many geometry nodes accept fields as inputs. Internally, that means that the execution functions have to accept so called "virtual arrays" as inputs. Those can be e.g. actual arrays, just single values, or lazily computed arrays. Due to these different possible virtual arrays implementations, access to individual elements is slower than it would be if everything was just a normal array (access does through a virtual function call). For more complex execution functions, this overhead does not matter, but for small functions (like a simple addition) it very much does. The virtual function call also prevents the compiler from doing some optimizations (e.g. loop unrolling and inserting simd instructions). The solution is to "devirtualize" the virtual arrays for small functions where the overhead is measurable. Essentially, the function is generated many times with different array types as input. Then there is a run-time dispatch that calls the best implementation. We have been doing devirtualization in e.g. math nodes for a long time already. This patch just generalizes the concept and makes it easier to control. It also makes it easier to investigate the different trade-offs when it comes to devirtualization. Nodes that we've optimized using devirtualization before didn't get a speedup. However, a couple of nodes are using devirtualization now, that didn't before. Those got a 2-4x speedup in common cases. * Map Range * Random Value * Switch * Combine XYZ Differential Revision: https://developer.blender.org/D14628
2022-04-26 17:12:34 +02:00
/* For outputs, just pass a pointer. This is important so that `__restrict` works. */
Functions: improve devirtualization in multi-function builder This refactors how devirtualization is done in general and how multi-functions use it. * The old `Devirtualizer` class has been removed in favor of a simpler solution. It is also more general in the sense that it is not coupled with `IndexMask` and `VArray`. Instead there is a function that has inputs which control how different types are devirtualized. The new implementation is currently less general with regard to the number of parameters it supports. This can be changed in the future, but does not seem necessary now and would make the code less obvious. * Devirtualizers for different types are now defined in their respective headers. * The multi-function builder works with the `GVArray` stored in `MFParams` directly now, instead of first converting it to a `VArray<T>`. This reduces some constant overhead, which makes the multi-function slightly faster. This is only noticable when very few elements are processed though. No functional changes or performance regressions are expected.
2023-01-07 12:55:48 +01:00
return std::get<I>(loaded_params);
Geometry Nodes: refactor array devirtualization Goals: * Better high level control over where devirtualization occurs. There is always a trade-off between performance and compile-time/binary-size. * Simplify using array devirtualization. * Better performance for cases where devirtualization wasn't used before. Many geometry nodes accept fields as inputs. Internally, that means that the execution functions have to accept so called "virtual arrays" as inputs. Those can be e.g. actual arrays, just single values, or lazily computed arrays. Due to these different possible virtual arrays implementations, access to individual elements is slower than it would be if everything was just a normal array (access does through a virtual function call). For more complex execution functions, this overhead does not matter, but for small functions (like a simple addition) it very much does. The virtual function call also prevents the compiler from doing some optimizations (e.g. loop unrolling and inserting simd instructions). The solution is to "devirtualize" the virtual arrays for small functions where the overhead is measurable. Essentially, the function is generated many times with different array types as input. Then there is a run-time dispatch that calls the best implementation. We have been doing devirtualization in e.g. math nodes for a long time already. This patch just generalizes the concept and makes it easier to control. It also makes it easier to investigate the different trade-offs when it comes to devirtualization. Nodes that we've optimized using devirtualization before didn't get a speedup. However, a couple of nodes are using devirtualization now, that didn't before. Those got a 2-4x speedup in common cases. * Map Range * Random Value * Switch * Combine XYZ Differential Revision: https://developer.blender.org/D14628
2022-04-26 17:12:34 +02:00
}
}()...);
(
/* Destruct values that have been materialized before. */
[&] {
Cleanup: add hint that typedef is used as compiler bug workaround https://developercommunity.visualstudio.com/t/Alias-template-inside-fold-expression-fa/10040507
2022-09-20 10:42:25 +02:00
/* Use `typedef` instead of `using` to work around a compiler bug. */
Workaround for msvc compiler bug https://developercommunity.visualstudio.com/t/Alias-template-inside-fold-expression-fa/10040507
2022-05-12 13:38:22 +02:00
typedef ParamTags ParamTag;
typedef typename ParamTag::base_type T;
Functions: refactor multi-function builder API * New `build_mf` namespace for the multi-function builders. * The type name of the created multi-functions is now "private", i.e. the caller has to use `auto`. This has the benefit that the implementation can change more freely without affecting the caller. * `CustomMF` does not use `std::function` internally anymore. This reduces some overhead during code generation and at run-time. * `CustomMF` now supports single-mutable parameters.
2023-01-07 16:19:59 +01:00
[[maybe_unused]] MaterializeArgInfo<ParamTags> &arg_info = std::get<I>(args_info);
Geometry Nodes: refactor array devirtualization Goals: * Better high level control over where devirtualization occurs. There is always a trade-off between performance and compile-time/binary-size. * Simplify using array devirtualization. * Better performance for cases where devirtualization wasn't used before. Many geometry nodes accept fields as inputs. Internally, that means that the execution functions have to accept so called "virtual arrays" as inputs. Those can be e.g. actual arrays, just single values, or lazily computed arrays. Due to these different possible virtual arrays implementations, access to individual elements is slower than it would be if everything was just a normal array (access does through a virtual function call). For more complex execution functions, this overhead does not matter, but for small functions (like a simple addition) it very much does. The virtual function call also prevents the compiler from doing some optimizations (e.g. loop unrolling and inserting simd instructions). The solution is to "devirtualize" the virtual arrays for small functions where the overhead is measurable. Essentially, the function is generated many times with different array types as input. Then there is a run-time dispatch that calls the best implementation. We have been doing devirtualization in e.g. math nodes for a long time already. This patch just generalizes the concept and makes it easier to control. It also makes it easier to investigate the different trade-offs when it comes to devirtualization. Nodes that we've optimized using devirtualization before didn't get a speedup. However, a couple of nodes are using devirtualization now, that didn't before. Those got a 2-4x speedup in common cases. * Map Range * Random Value * Switch * Combine XYZ Differential Revision: https://developer.blender.org/D14628
2022-04-26 17:12:34 +02:00
if constexpr (ParamTag::category == MFParamCategory::SingleInput) {
Functions: refactor multi-function builder API * New `build_mf` namespace for the multi-function builders. * The type name of the created multi-functions is now "private", i.e. the caller has to use `auto`. This has the benefit that the implementation can change more freely without affecting the caller. * `CustomMF` does not use `std::function` internally anymore. This reduces some overhead during code generation and at run-time. * `CustomMF` now supports single-mutable parameters.
2023-01-07 16:19:59 +01:00
if (arg_info.mode == MaterializeArgMode::Materialized) {
Geometry Nodes: refactor array devirtualization Goals: * Better high level control over where devirtualization occurs. There is always a trade-off between performance and compile-time/binary-size. * Simplify using array devirtualization. * Better performance for cases where devirtualization wasn't used before. Many geometry nodes accept fields as inputs. Internally, that means that the execution functions have to accept so called "virtual arrays" as inputs. Those can be e.g. actual arrays, just single values, or lazily computed arrays. Due to these different possible virtual arrays implementations, access to individual elements is slower than it would be if everything was just a normal array (access does through a virtual function call). For more complex execution functions, this overhead does not matter, but for small functions (like a simple addition) it very much does. The virtual function call also prevents the compiler from doing some optimizations (e.g. loop unrolling and inserting simd instructions). The solution is to "devirtualize" the virtual arrays for small functions where the overhead is measurable. Essentially, the function is generated many times with different array types as input. Then there is a run-time dispatch that calls the best implementation. We have been doing devirtualization in e.g. math nodes for a long time already. This patch just generalizes the concept and makes it easier to control. It also makes it easier to investigate the different trade-offs when it comes to devirtualization. Nodes that we've optimized using devirtualization before didn't get a speedup. However, a couple of nodes are using devirtualization now, that didn't before. Those got a 2-4x speedup in common cases. * Map Range * Random Value * Switch * Combine XYZ Differential Revision: https://developer.blender.org/D14628
2022-04-26 17:12:34 +02:00
T *in_chunk = std::get<I>(buffers_owner).ptr();
destruct_n(in_chunk, chunk_size);
}
}
}(),
...);
Functions: optimize simple generated multi-functions This implements two optimizations: * Reduce virtual function call overhead when a non-standard virtual array is used as input. * Use a lambda in `type_conversion.cc`. In my test setup, which creates a float attribute filled with the index, the running time drops from `4.0 ms` to `2.0 ms`. Differential Revision: https://developer.blender.org/D14585
2022-04-07 18:48:14 +02:00
}
Geometry Nodes: refactor array devirtualization Goals: * Better high level control over where devirtualization occurs. There is always a trade-off between performance and compile-time/binary-size. * Simplify using array devirtualization. * Better performance for cases where devirtualization wasn't used before. Many geometry nodes accept fields as inputs. Internally, that means that the execution functions have to accept so called "virtual arrays" as inputs. Those can be e.g. actual arrays, just single values, or lazily computed arrays. Due to these different possible virtual arrays implementations, access to individual elements is slower than it would be if everything was just a normal array (access does through a virtual function call). For more complex execution functions, this overhead does not matter, but for small functions (like a simple addition) it very much does. The virtual function call also prevents the compiler from doing some optimizations (e.g. loop unrolling and inserting simd instructions). The solution is to "devirtualize" the virtual arrays for small functions where the overhead is measurable. Essentially, the function is generated many times with different array types as input. Then there is a run-time dispatch that calls the best implementation. We have been doing devirtualization in e.g. math nodes for a long time already. This patch just generalizes the concept and makes it easier to control. It also makes it easier to investigate the different trade-offs when it comes to devirtualization. Nodes that we've optimized using devirtualization before didn't get a speedup. However, a couple of nodes are using devirtualization now, that didn't before. Those got a 2-4x speedup in common cases. * Map Range * Random Value * Switch * Combine XYZ Differential Revision: https://developer.blender.org/D14628
2022-04-26 17:12:34 +02:00
(
/* Destruct buffers for single value inputs. */
[&] {
Cleanup: add hint that typedef is used as compiler bug workaround https://developercommunity.visualstudio.com/t/Alias-template-inside-fold-expression-fa/10040507
2022-09-20 10:42:25 +02:00
/* Use `typedef` instead of `using` to work around a compiler bug. */
Workaround for msvc compiler bug https://developercommunity.visualstudio.com/t/Alias-template-inside-fold-expression-fa/10040507
2022-05-12 13:38:22 +02:00
typedef ParamTags ParamTag;
typedef typename ParamTag::base_type T;
Functions: refactor multi-function builder API * New `build_mf` namespace for the multi-function builders. * The type name of the created multi-functions is now "private", i.e. the caller has to use `auto`. This has the benefit that the implementation can change more freely without affecting the caller. * `CustomMF` does not use `std::function` internally anymore. This reduces some overhead during code generation and at run-time. * `CustomMF` now supports single-mutable parameters.
2023-01-07 16:19:59 +01:00
[[maybe_unused]] MaterializeArgInfo<ParamTags> &arg_info = std::get<I>(args_info);
Geometry Nodes: refactor array devirtualization Goals: * Better high level control over where devirtualization occurs. There is always a trade-off between performance and compile-time/binary-size. * Simplify using array devirtualization. * Better performance for cases where devirtualization wasn't used before. Many geometry nodes accept fields as inputs. Internally, that means that the execution functions have to accept so called "virtual arrays" as inputs. Those can be e.g. actual arrays, just single values, or lazily computed arrays. Due to these different possible virtual arrays implementations, access to individual elements is slower than it would be if everything was just a normal array (access does through a virtual function call). For more complex execution functions, this overhead does not matter, but for small functions (like a simple addition) it very much does. The virtual function call also prevents the compiler from doing some optimizations (e.g. loop unrolling and inserting simd instructions). The solution is to "devirtualize" the virtual arrays for small functions where the overhead is measurable. Essentially, the function is generated many times with different array types as input. Then there is a run-time dispatch that calls the best implementation. We have been doing devirtualization in e.g. math nodes for a long time already. This patch just generalizes the concept and makes it easier to control. It also makes it easier to investigate the different trade-offs when it comes to devirtualization. Nodes that we've optimized using devirtualization before didn't get a speedup. However, a couple of nodes are using devirtualization now, that didn't before. Those got a 2-4x speedup in common cases. * Map Range * Random Value * Switch * Combine XYZ Differential Revision: https://developer.blender.org/D14628
2022-04-26 17:12:34 +02:00
if constexpr (ParamTag::category == MFParamCategory::SingleInput) {
Functions: refactor multi-function builder API * New `build_mf` namespace for the multi-function builders. * The type name of the created multi-functions is now "private", i.e. the caller has to use `auto`. This has the benefit that the implementation can change more freely without affecting the caller. * `CustomMF` does not use `std::function` internally anymore. This reduces some overhead during code generation and at run-time. * `CustomMF` now supports single-mutable parameters.
2023-01-07 16:19:59 +01:00
if (arg_info.mode == MaterializeArgMode::Single) {
Geometry Nodes: refactor array devirtualization Goals: * Better high level control over where devirtualization occurs. There is always a trade-off between performance and compile-time/binary-size. * Simplify using array devirtualization. * Better performance for cases where devirtualization wasn't used before. Many geometry nodes accept fields as inputs. Internally, that means that the execution functions have to accept so called "virtual arrays" as inputs. Those can be e.g. actual arrays, just single values, or lazily computed arrays. Due to these different possible virtual arrays implementations, access to individual elements is slower than it would be if everything was just a normal array (access does through a virtual function call). For more complex execution functions, this overhead does not matter, but for small functions (like a simple addition) it very much does. The virtual function call also prevents the compiler from doing some optimizations (e.g. loop unrolling and inserting simd instructions). The solution is to "devirtualize" the virtual arrays for small functions where the overhead is measurable. Essentially, the function is generated many times with different array types as input. Then there is a run-time dispatch that calls the best implementation. We have been doing devirtualization in e.g. math nodes for a long time already. This patch just generalizes the concept and makes it easier to control. It also makes it easier to investigate the different trade-offs when it comes to devirtualization. Nodes that we've optimized using devirtualization before didn't get a speedup. However, a couple of nodes are using devirtualization now, that didn't before. Those got a 2-4x speedup in common cases. * Map Range * Random Value * Switch * Combine XYZ Differential Revision: https://developer.blender.org/D14628
2022-04-26 17:12:34 +02:00
MutableSpan<T> in_chunk = std::get<I>(buffers);
destruct_n(in_chunk.data(), in_chunk.size());
}
}
}(),
...);
}
Functions: add utilities that allow creating some multi-functions with less typing
2020-06-22 15:50:08 +02:00
Functions: refactor multi-function builder API * New `build_mf` namespace for the multi-function builders. * The type name of the created multi-functions is now "private", i.e. the caller has to use `auto`. This has the benefit that the implementation can change more freely without affecting the caller. * `CustomMF` does not use `std::function` internally anymore. This reduces some overhead during code generation and at run-time. * `CustomMF` now supports single-mutable parameters.
2023-01-07 16:19:59 +01:00
template<typename ElementFn, typename ExecPreset, typename... ParamTags, size_t... I>
inline void execute_element_fn_as_multi_function(const ElementFn element_fn,
const ExecPreset exec_preset,
const IndexMask mask,
MFParams params,
TypeSequence<ParamTags...> /*param_tags*/,
std::index_sequence<I...> /*indices*/)
{
Geometry Nodes: refactor array devirtualization Goals: * Better high level control over where devirtualization occurs. There is always a trade-off between performance and compile-time/binary-size. * Simplify using array devirtualization. * Better performance for cases where devirtualization wasn't used before. Many geometry nodes accept fields as inputs. Internally, that means that the execution functions have to accept so called "virtual arrays" as inputs. Those can be e.g. actual arrays, just single values, or lazily computed arrays. Due to these different possible virtual arrays implementations, access to individual elements is slower than it would be if everything was just a normal array (access does through a virtual function call). For more complex execution functions, this overhead does not matter, but for small functions (like a simple addition) it very much does. The virtual function call also prevents the compiler from doing some optimizations (e.g. loop unrolling and inserting simd instructions). The solution is to "devirtualize" the virtual arrays for small functions where the overhead is measurable. Essentially, the function is generated many times with different array types as input. Then there is a run-time dispatch that calls the best implementation. We have been doing devirtualization in e.g. math nodes for a long time already. This patch just generalizes the concept and makes it easier to control. It also makes it easier to investigate the different trade-offs when it comes to devirtualization. Nodes that we've optimized using devirtualization before didn't get a speedup. However, a couple of nodes are using devirtualization now, that didn't before. Those got a 2-4x speedup in common cases. * Map Range * Random Value * Switch * Combine XYZ Differential Revision: https://developer.blender.org/D14628
2022-04-26 17:12:34 +02:00
Functions: refactor multi-function builder API * New `build_mf` namespace for the multi-function builders. * The type name of the created multi-functions is now "private", i.e. the caller has to use `auto`. This has the benefit that the implementation can change more freely without affecting the caller. * `CustomMF` does not use `std::function` internally anymore. This reduces some overhead during code generation and at run-time. * `CustomMF` now supports single-mutable parameters.
2023-01-07 16:19:59 +01:00
/* Load parameters from #MFParams. */
/* Contains `const GVArrayImpl *` for inputs and `T *` for outputs. */
const auto loaded_params = std::make_tuple([&]() {
/* Use `typedef` instead of `using` to work around a compiler bug. */
typedef ParamTags ParamTag;
typedef typename ParamTag::base_type T;
Geometry Nodes: refactor array devirtualization Goals: * Better high level control over where devirtualization occurs. There is always a trade-off between performance and compile-time/binary-size. * Simplify using array devirtualization. * Better performance for cases where devirtualization wasn't used before. Many geometry nodes accept fields as inputs. Internally, that means that the execution functions have to accept so called "virtual arrays" as inputs. Those can be e.g. actual arrays, just single values, or lazily computed arrays. Due to these different possible virtual arrays implementations, access to individual elements is slower than it would be if everything was just a normal array (access does through a virtual function call). For more complex execution functions, this overhead does not matter, but for small functions (like a simple addition) it very much does. The virtual function call also prevents the compiler from doing some optimizations (e.g. loop unrolling and inserting simd instructions). The solution is to "devirtualize" the virtual arrays for small functions where the overhead is measurable. Essentially, the function is generated many times with different array types as input. Then there is a run-time dispatch that calls the best implementation. We have been doing devirtualization in e.g. math nodes for a long time already. This patch just generalizes the concept and makes it easier to control. It also makes it easier to investigate the different trade-offs when it comes to devirtualization. Nodes that we've optimized using devirtualization before didn't get a speedup. However, a couple of nodes are using devirtualization now, that didn't before. Those got a 2-4x speedup in common cases. * Map Range * Random Value * Switch * Combine XYZ Differential Revision: https://developer.blender.org/D14628
2022-04-26 17:12:34 +02:00
Functions: refactor multi-function builder API * New `build_mf` namespace for the multi-function builders. * The type name of the created multi-functions is now "private", i.e. the caller has to use `auto`. This has the benefit that the implementation can change more freely without affecting the caller. * `CustomMF` does not use `std::function` internally anymore. This reduces some overhead during code generation and at run-time. * `CustomMF` now supports single-mutable parameters.
2023-01-07 16:19:59 +01:00
if constexpr (ParamTag::category == MFParamCategory::SingleInput) {
return params.readonly_single_input(I).get_implementation();
Geometry Nodes: refactor array devirtualization Goals: * Better high level control over where devirtualization occurs. There is always a trade-off between performance and compile-time/binary-size. * Simplify using array devirtualization. * Better performance for cases where devirtualization wasn't used before. Many geometry nodes accept fields as inputs. Internally, that means that the execution functions have to accept so called "virtual arrays" as inputs. Those can be e.g. actual arrays, just single values, or lazily computed arrays. Due to these different possible virtual arrays implementations, access to individual elements is slower than it would be if everything was just a normal array (access does through a virtual function call). For more complex execution functions, this overhead does not matter, but for small functions (like a simple addition) it very much does. The virtual function call also prevents the compiler from doing some optimizations (e.g. loop unrolling and inserting simd instructions). The solution is to "devirtualize" the virtual arrays for small functions where the overhead is measurable. Essentially, the function is generated many times with different array types as input. Then there is a run-time dispatch that calls the best implementation. We have been doing devirtualization in e.g. math nodes for a long time already. This patch just generalizes the concept and makes it easier to control. It also makes it easier to investigate the different trade-offs when it comes to devirtualization. Nodes that we've optimized using devirtualization before didn't get a speedup. However, a couple of nodes are using devirtualization now, that didn't before. Those got a 2-4x speedup in common cases. * Map Range * Random Value * Switch * Combine XYZ Differential Revision: https://developer.blender.org/D14628
2022-04-26 17:12:34 +02:00
}
Functions: refactor multi-function builder API * New `build_mf` namespace for the multi-function builders. * The type name of the created multi-functions is now "private", i.e. the caller has to use `auto`. This has the benefit that the implementation can change more freely without affecting the caller. * `CustomMF` does not use `std::function` internally anymore. This reduces some overhead during code generation and at run-time. * `CustomMF` now supports single-mutable parameters.
2023-01-07 16:19:59 +01:00
else if constexpr (ParamTag::category == MFParamCategory::SingleOutput) {
return static_cast<T *>(params.uninitialized_single_output(I).data());
Geometry Nodes: refactor array devirtualization Goals: * Better high level control over where devirtualization occurs. There is always a trade-off between performance and compile-time/binary-size. * Simplify using array devirtualization. * Better performance for cases where devirtualization wasn't used before. Many geometry nodes accept fields as inputs. Internally, that means that the execution functions have to accept so called "virtual arrays" as inputs. Those can be e.g. actual arrays, just single values, or lazily computed arrays. Due to these different possible virtual arrays implementations, access to individual elements is slower than it would be if everything was just a normal array (access does through a virtual function call). For more complex execution functions, this overhead does not matter, but for small functions (like a simple addition) it very much does. The virtual function call also prevents the compiler from doing some optimizations (e.g. loop unrolling and inserting simd instructions). The solution is to "devirtualize" the virtual arrays for small functions where the overhead is measurable. Essentially, the function is generated many times with different array types as input. Then there is a run-time dispatch that calls the best implementation. We have been doing devirtualization in e.g. math nodes for a long time already. This patch just generalizes the concept and makes it easier to control. It also makes it easier to investigate the different trade-offs when it comes to devirtualization. Nodes that we've optimized using devirtualization before didn't get a speedup. However, a couple of nodes are using devirtualization now, that didn't before. Those got a 2-4x speedup in common cases. * Map Range * Random Value * Switch * Combine XYZ Differential Revision: https://developer.blender.org/D14628
2022-04-26 17:12:34 +02:00
}
Functions: refactor multi-function builder API * New `build_mf` namespace for the multi-function builders. * The type name of the created multi-functions is now "private", i.e. the caller has to use `auto`. This has the benefit that the implementation can change more freely without affecting the caller. * `CustomMF` does not use `std::function` internally anymore. This reduces some overhead during code generation and at run-time. * `CustomMF` now supports single-mutable parameters.
2023-01-07 16:19:59 +01:00
else if constexpr (ParamTag::category == MFParamCategory::SingleMutable) {
return static_cast<T *>(params.single_mutable(I).data());
}
}()...);
/* Try execute devirtualized if enabled and the input types allow it. */
bool executed_devirtualized = false;
if constexpr (ExecPreset::use_devirtualization) {
const auto devirtualizers = exec_preset.create_devirtualizers(
TypeSequence<ParamTags...>(), std::index_sequence<I...>(), mask, loaded_params);
executed_devirtualized = call_with_devirtualized_parameters(
devirtualizers, [&](auto &&...args) {
execute_array(TypeSequence<ParamTags...>(),
std::index_sequence<I...>(),
element_fn,
std::forward<decltype(args)>(args)...);
});
Functions: add utilities that allow creating some multi-functions with less typing
2020-06-22 15:50:08 +02:00
}
Functions: refactor multi-function builder API * New `build_mf` namespace for the multi-function builders. * The type name of the created multi-functions is now "private", i.e. the caller has to use `auto`. This has the benefit that the implementation can change more freely without affecting the caller. * `CustomMF` does not use `std::function` internally anymore. This reduces some overhead during code generation and at run-time. * `CustomMF` now supports single-mutable parameters.
2023-01-07 16:19:59 +01:00
/* If devirtualized execution was disabled or not possible, use a fallback method which is
* slower but always works. */
if (!executed_devirtualized) {
/* The materialized method is most common because it avoids most virtual function overhead but
* still instantiates the function only once. */
if constexpr (ExecPreset::fallback_mode == exec_presets::FallbackMode::Materialized) {
execute_materialized(TypeSequence<ParamTags...>(),
std::index_sequence<I...>(),
element_fn,
mask,
loaded_params);
}
else {
/* This fallback is slower because it uses virtual method calls for every element. */
execute_array(
TypeSequence<ParamTags...>(), std::index_sequence<I...>(), element_fn, mask, [&]() {
/* Use `typedef` instead of `using` to work around a compiler bug. */
typedef ParamTags ParamTag;
typedef typename ParamTag::base_type T;
if constexpr (ParamTag::category == MFParamCategory::SingleInput) {
const GVArrayImpl &varray_impl = *std::get<I>(loaded_params);
return GVArray(&varray_impl).typed<T>();
}
else if constexpr (ELEM(ParamTag::category,
MFParamCategory::SingleOutput,
MFParamCategory::SingleMutable)) {
T *ptr = std::get<I>(loaded_params);
return ptr;
}
}()...);
}
Functions: add two more customizable multi-functions
2020-06-30 17:59:33 +02:00
}
Functions: refactor multi-function builder API * New `build_mf` namespace for the multi-function builders. * The type name of the created multi-functions is now "private", i.e. the caller has to use `auto`. This has the benefit that the implementation can change more freely without affecting the caller. * `CustomMF` does not use `std::function` internally anymore. This reduces some overhead during code generation and at run-time. * `CustomMF` now supports single-mutable parameters.
2023-01-07 16:19:59 +01:00
}
Functions: add two more customizable multi-functions
2020-06-30 17:59:33 +02:00
Geometry Nodes: support Vector Rotate node Differential Revision: https://developer.blender.org/D10410
2021-03-08 11:37:37 +01:00
/**
Functions: refactor multi-function builder API * New `build_mf` namespace for the multi-function builders. * The type name of the created multi-functions is now "private", i.e. the caller has to use `auto`. This has the benefit that the implementation can change more freely without affecting the caller. * `CustomMF` does not use `std::function` internally anymore. This reduces some overhead during code generation and at run-time. * `CustomMF` now supports single-mutable parameters.
2023-01-07 16:19:59 +01:00
* `element_fn` is expected to return nothing and to have the following parameters:
* - For single-inputs: const value or reference.
* - For single-mutables: non-const reference.
* - For single-outputs: non-const pointer.
Geometry Nodes: support Vector Rotate node Differential Revision: https://developer.blender.org/D10410
2021-03-08 11:37:37 +01:00
*/
Functions: refactor multi-function builder API * New `build_mf` namespace for the multi-function builders. * The type name of the created multi-functions is now "private", i.e. the caller has to use `auto`. This has the benefit that the implementation can change more freely without affecting the caller. * `CustomMF` does not use `std::function` internally anymore. This reduces some overhead during code generation and at run-time. * `CustomMF` now supports single-mutable parameters.
2023-01-07 16:19:59 +01:00
template<typename ElementFn, typename ExecPreset, typename... ParamTags>
inline auto build_multi_function_call_from_element_fn(const ElementFn element_fn,
const ExecPreset exec_preset,
TypeSequence<ParamTags...> /*param_tags*/)
{
return [element_fn, exec_preset](const IndexMask mask, MFParams params) {
execute_element_fn_as_multi_function(element_fn,
exec_preset,
mask,
params,
TypeSequence<ParamTags...>(),
std::make_index_sequence<sizeof...(ParamTags)>());
};
}
Geometry Nodes: support Vector Rotate node Differential Revision: https://developer.blender.org/D10410
2021-03-08 11:37:37 +01:00
Functions: add utilities that allow creating some multi-functions with less typing
2020-06-22 15:50:08 +02:00
/**
Functions: refactor multi-function builder API * New `build_mf` namespace for the multi-function builders. * The type name of the created multi-functions is now "private", i.e. the caller has to use `auto`. This has the benefit that the implementation can change more freely without affecting the caller. * `CustomMF` does not use `std::function` internally anymore. This reduces some overhead during code generation and at run-time. * `CustomMF` now supports single-mutable parameters.
2023-01-07 16:19:59 +01:00
* A multi function that just invokes the provided function in its #call method.
Functions: add utilities that allow creating some multi-functions with less typing
2020-06-22 15:50:08 +02:00
*/
Functions: refactor multi-function builder API * New `build_mf` namespace for the multi-function builders. * The type name of the created multi-functions is now "private", i.e. the caller has to use `auto`. This has the benefit that the implementation can change more freely without affecting the caller. * `CustomMF` does not use `std::function` internally anymore. This reduces some overhead during code generation and at run-time. * `CustomMF` now supports single-mutable parameters.
2023-01-07 16:19:59 +01:00
template<typename CallFn, typename... ParamTags> class CustomMF : public MultiFunction {
Functions: add utilities that allow creating some multi-functions with less typing
2020-06-22 15:50:08 +02:00
private:
Functions: make multi functions smaller and cheaper to construct in many cases Previously, the signature of a `MultiFunction` was always embedded into the function. There are two issues with that. First, `MFSignature` is relatively large, because it contains multiple strings and vectors. Secondly, constructing it can add overhead that should not be necessary, because often the same signature can be reused. The solution is to only keep a pointer to a signature in `MultiFunction` that is set during construction. Child classes are responsible for making sure that the signature lives long enough. In most cases, the signature is either embedded into the child class or it is allocated statically (and is only created once).
2021-03-22 11:57:24 +01:00
MFSignature signature_;
Functions: refactor multi-function builder API * New `build_mf` namespace for the multi-function builders. * The type name of the created multi-functions is now "private", i.e. the caller has to use `auto`. This has the benefit that the implementation can change more freely without affecting the caller. * `CustomMF` does not use `std::function` internally anymore. This reduces some overhead during code generation and at run-time. * `CustomMF` now supports single-mutable parameters.
2023-01-07 16:19:59 +01:00
CallFn call_fn_;
Functions: add utilities that allow creating some multi-functions with less typing
2020-06-22 15:50:08 +02:00
public:
Functions: refactor multi-function builder API * New `build_mf` namespace for the multi-function builders. * The type name of the created multi-functions is now "private", i.e. the caller has to use `auto`. This has the benefit that the implementation can change more freely without affecting the caller. * `CustomMF` does not use `std::function` internally anymore. This reduces some overhead during code generation and at run-time. * `CustomMF` now supports single-mutable parameters.
2023-01-07 16:19:59 +01:00
CustomMF(const char *name, CallFn call_fn, TypeSequence<ParamTags...> /*param_tags*/)
: call_fn_(std::move(call_fn))
Functions: add utilities that allow creating some multi-functions with less typing
2020-06-22 15:50:08 +02:00
{
Functions: make multi functions smaller and cheaper to construct in many cases Previously, the signature of a `MultiFunction` was always embedded into the function. There are two issues with that. First, `MFSignature` is relatively large, because it contains multiple strings and vectors. Secondly, constructing it can add overhead that should not be necessary, because often the same signature can be reused. The solution is to only keep a pointer to a signature in `MultiFunction` that is set during construction. Child classes are responsible for making sure that the signature lives long enough. In most cases, the signature is either embedded into the child class or it is allocated statically (and is only created once).
2021-03-22 11:57:24 +01:00
MFSignatureBuilder signature{name};
Functions: refactor multi-function builder API * New `build_mf` namespace for the multi-function builders. * The type name of the created multi-functions is now "private", i.e. the caller has to use `auto`. This has the benefit that the implementation can change more freely without affecting the caller. * `CustomMF` does not use `std::function` internally anymore. This reduces some overhead during code generation and at run-time. * `CustomMF` now supports single-mutable parameters.
2023-01-07 16:19:59 +01:00
/* Loop over all parameter types and add an entry for each in the signature. */
([&] { signature.add(ParamTags(), ""); }(), ...);
Functions: make multi functions smaller and cheaper to construct in many cases Previously, the signature of a `MultiFunction` was always embedded into the function. There are two issues with that. First, `MFSignature` is relatively large, because it contains multiple strings and vectors. Secondly, constructing it can add overhead that should not be necessary, because often the same signature can be reused. The solution is to only keep a pointer to a signature in `MultiFunction` that is set during construction. Child classes are responsible for making sure that the signature lives long enough. In most cases, the signature is either embedded into the child class or it is allocated statically (and is only created once).
2021-03-22 11:57:24 +01:00
signature_ = signature.build();
this->set_signature(&signature_);
Functions: add utilities that allow creating some multi-functions with less typing
2020-06-22 15:50:08 +02:00
}
Cleanup: replace UNUSED macro with commented args in C++ code This is the conventional way of dealing with unused arguments in C++, since it works on all compilers. Regex find and replace: `UNUSED\((\w+)\)` -> `/*$1*/`
2022-10-03 17:37:25 -05:00
void call(IndexMask mask, MFParams params, MFContext /*context*/) const override
Functions: add utilities that allow creating some multi-functions with less typing
2020-06-22 15:50:08 +02:00
{
Functions: refactor multi-function builder API * New `build_mf` namespace for the multi-function builders. * The type name of the created multi-functions is now "private", i.e. the caller has to use `auto`. This has the benefit that the implementation can change more freely without affecting the caller. * `CustomMF` does not use `std::function` internally anymore. This reduces some overhead during code generation and at run-time. * `CustomMF` now supports single-mutable parameters.
2023-01-07 16:19:59 +01:00
call_fn_(mask, params);
Functions: add utilities that allow creating some multi-functions with less typing
2020-06-22 15:50:08 +02:00
}
};
Functions: refactor multi-function builder API * New `build_mf` namespace for the multi-function builders. * The type name of the created multi-functions is now "private", i.e. the caller has to use `auto`. This has the benefit that the implementation can change more freely without affecting the caller. * `CustomMF` does not use `std::function` internally anymore. This reduces some overhead during code generation and at run-time. * `CustomMF` now supports single-mutable parameters.
2023-01-07 16:19:59 +01:00
template<typename Out, typename... In, typename ElementFn, typename ExecPreset>
inline auto build_multi_function_with_n_inputs_one_output(const char *name,
const ElementFn element_fn,
const ExecPreset exec_preset,
TypeSequence<In...> /*in_types*/)
{
constexpr auto param_tags = TypeSequence<MFParamTag<MFParamCategory::SingleInput, In>...,
MFParamTag<MFParamCategory::SingleOutput, Out>>();
auto call_fn = build_multi_function_call_from_element_fn(
[element_fn](const In &...in, Out *out) { new (out) Out(element_fn(in...)); },
exec_preset,
param_tags);
return CustomMF(name, call_fn, param_tags);
}
} // namespace detail
/** Build multi-function with 1 single-input and 1 single-output parameter. */
template<typename In1,
typename Out1,
typename ElementFn,
typename ExecPreset = exec_presets::Materialized>
inline auto SI1_SO(const char *name,
const ElementFn element_fn,
const ExecPreset exec_preset = exec_presets::Materialized())
{
return detail::build_multi_function_with_n_inputs_one_output<Out1>(
name, element_fn, exec_preset, TypeSequence<In1>());
}
/** Build multi-function with 2 single-input and 1 single-output parameter. */
template<typename In1,
typename In2,
typename Out1,
typename ElementFn,
typename ExecPreset = exec_presets::Materialized>
inline auto SI2_SO(const char *name,
const ElementFn element_fn,
const ExecPreset exec_preset = exec_presets::Materialized())
{
return detail::build_multi_function_with_n_inputs_one_output<Out1>(
name, element_fn, exec_preset, TypeSequence<In1, In2>());
}
/** Build multi-function with 3 single-input and 1 single-output parameter. */
template<typename In1,
typename In2,
typename In3,
typename Out1,
typename ElementFn,
typename ExecPreset = exec_presets::Materialized>
inline auto SI3_SO(const char *name,
const ElementFn element_fn,
const ExecPreset exec_preset = exec_presets::Materialized())
{
return detail::build_multi_function_with_n_inputs_one_output<Out1>(
name, element_fn, exec_preset, TypeSequence<In1, In2, In3>());
}
/** Build multi-function with 4 single-input and 1 single-output parameter. */
template<typename In1,
typename In2,
typename In3,
typename In4,
typename Out1,
typename ElementFn,
typename ExecPreset = exec_presets::Materialized>
inline auto SI4_SO(const char *name,
const ElementFn element_fn,
const ExecPreset exec_preset = exec_presets::Materialized())
{
return detail::build_multi_function_with_n_inputs_one_output<Out1>(
name, element_fn, exec_preset, TypeSequence<In1, In2, In3, In4>());
}
/** Build multi-function with 5 single-input and 1 single-output parameter. */
template<typename In1,
typename In2,
typename In3,
typename In4,
typename In5,
typename Out1,
typename ElementFn,
typename ExecPreset = exec_presets::Materialized>
inline auto SI5_SO(const char *name,
const ElementFn element_fn,
const ExecPreset exec_preset = exec_presets::Materialized())
{
return detail::build_multi_function_with_n_inputs_one_output<Out1>(
name, element_fn, exec_preset, TypeSequence<In1, In2, In3, In4, In5>());
}
/** Build multi-function with 6 single-input and 1 single-output parameter. */
template<typename In1,
typename In2,
typename In3,
typename In4,
typename In5,
typename In6,
typename Out1,
typename ElementFn,
typename ExecPreset = exec_presets::Materialized>
inline auto SI6_SO(const char *name,
const ElementFn element_fn,
const ExecPreset exec_preset = exec_presets::Materialized())
{
return detail::build_multi_function_with_n_inputs_one_output<Out1>(
name, element_fn, exec_preset, TypeSequence<In1, In2, In3, In4, In5, In6>());
}
/** Build multi-function with 1 single-mutable parameter. */
template<typename Mut1, typename ElementFn, typename ExecPreset = exec_presets::AllSpanOrSingle>
inline auto SM(const char *name,
const ElementFn element_fn,
const ExecPreset exec_preset = exec_presets::AllSpanOrSingle())
{
constexpr auto param_tags = TypeSequence<MFParamTag<MFParamCategory::SingleMutable, Mut1>>();
auto call_fn = detail::build_multi_function_call_from_element_fn(
element_fn, exec_preset, param_tags);
return detail::CustomMF(name, call_fn, param_tags);
}
} // namespace blender::fn::build_mf
namespace blender::fn {
Functions: add generic functions that output constants
2020-07-07 19:34:35 +02:00
/**
* A multi-function that outputs the same value every time. The value is not owned by an instance
Geometry Nodes: add field support for socket inspection Since fields were committed to master, socket inspection did not work correctly for all socket types anymore. Now the same functionality as before is back. Furthermore, fields that depend on some input will now show the inputs in the socket inspection. I added support for evaluating constant fields more immediately. This has the benefit that the same constant field is not evaluated more than once. It also helps with making the field independent of the multi-functions that it uses. We might still want to change the ownership handling for the multi-functions of nodes a bit, but that can be done separately. Differential Revision: https://developer.blender.org/D12444
2021-09-11 13:05:20 +02:00
* of this function. If #make_value_copy is false, the caller is responsible for destructing and
* freeing the value.
Functions: add generic functions that output constants
2020-07-07 19:34:35 +02:00
*/
class CustomMF_GenericConstant : public MultiFunction {
private:
const CPPType &type_;
const void *value_;
Functions: make multi functions smaller and cheaper to construct in many cases Previously, the signature of a `MultiFunction` was always embedded into the function. There are two issues with that. First, `MFSignature` is relatively large, because it contains multiple strings and vectors. Secondly, constructing it can add overhead that should not be necessary, because often the same signature can be reused. The solution is to only keep a pointer to a signature in `MultiFunction` that is set during construction. Child classes are responsible for making sure that the signature lives long enough. In most cases, the signature is either embedded into the child class or it is allocated statically (and is only created once).
2021-03-22 11:57:24 +01:00
MFSignature signature_;
Geometry Nodes: add field support for socket inspection Since fields were committed to master, socket inspection did not work correctly for all socket types anymore. Now the same functionality as before is back. Furthermore, fields that depend on some input will now show the inputs in the socket inspection. I added support for evaluating constant fields more immediately. This has the benefit that the same constant field is not evaluated more than once. It also helps with making the field independent of the multi-functions that it uses. We might still want to change the ownership handling for the multi-functions of nodes a bit, but that can be done separately. Differential Revision: https://developer.blender.org/D12444
2021-09-11 13:05:20 +02:00
bool owns_value_;
Functions: add generic functions that output constants
2020-07-07 19:34:35 +02:00
Functions: initial hash/equals implementation for constant multi-functions
2020-07-08 15:04:28 +02:00
template<typename T> friend class CustomMF_Constant;
Functions: add generic functions that output constants
2020-07-07 19:34:35 +02:00
public:
Geometry Nodes: add field support for socket inspection Since fields were committed to master, socket inspection did not work correctly for all socket types anymore. Now the same functionality as before is back. Furthermore, fields that depend on some input will now show the inputs in the socket inspection. I added support for evaluating constant fields more immediately. This has the benefit that the same constant field is not evaluated more than once. It also helps with making the field independent of the multi-functions that it uses. We might still want to change the ownership handling for the multi-functions of nodes a bit, but that can be done separately. Differential Revision: https://developer.blender.org/D12444
2021-09-11 13:05:20 +02:00
CustomMF_GenericConstant(const CPPType &type, const void *value, bool make_value_copy);
~CustomMF_GenericConstant();
Functions: add generic functions that output constants
2020-07-07 19:34:35 +02:00
void call(IndexMask mask, MFParams params, MFContext context) const override;
Refactor: Update integer type usage This updates the usage of integer types in code I wrote according to our new style guides. Major changes: * Use signed instead of unsigned integers in many places. * C++ containers in blenlib use `int64_t` for size and indices now (instead of `uint`). * Hash values for C++ containers are 64 bit wide now (instead of 32 bit). I do hope that I broke no builds, but it is quite likely that some compiler reports slightly different errors. Please let me know when there are any errors. If the fix is small, feel free to commit it yourself. I compiled successfully on linux with gcc and on windows.
2020-07-20 12:16:20 +02:00
uint64_t hash() const override;
Functions: initial hash/equals implementation for constant multi-functions
2020-07-08 15:04:28 +02:00
bool equals(const MultiFunction &other) const override;
Functions: add generic functions that output constants
2020-07-07 19:34:35 +02:00
};
/**
* A multi-function that outputs the same array every time. The array is not owned by in instance
* of this function. The caller is responsible for destructing and freeing the values.
*/
class CustomMF_GenericConstantArray : public MultiFunction {
private:
GSpan array_;
Functions: make multi functions smaller and cheaper to construct in many cases Previously, the signature of a `MultiFunction` was always embedded into the function. There are two issues with that. First, `MFSignature` is relatively large, because it contains multiple strings and vectors. Secondly, constructing it can add overhead that should not be necessary, because often the same signature can be reused. The solution is to only keep a pointer to a signature in `MultiFunction` that is set during construction. Child classes are responsible for making sure that the signature lives long enough. In most cases, the signature is either embedded into the child class or it is allocated statically (and is only created once).
2021-03-22 11:57:24 +01:00
MFSignature signature_;
Functions: add generic functions that output constants
2020-07-07 19:34:35 +02:00
public:
CustomMF_GenericConstantArray(GSpan array);
void call(IndexMask mask, MFParams params, MFContext context) const override;
};
Functions: initial hash/equals implementation for constant multi-functions
2020-07-08 15:04:28 +02:00
/**
* Generates a multi-function that outputs a constant value.
*/
template<typename T> class CustomMF_Constant : public MultiFunction {
private:
T value_;
Functions: make multi functions smaller and cheaper to construct in many cases Previously, the signature of a `MultiFunction` was always embedded into the function. There are two issues with that. First, `MFSignature` is relatively large, because it contains multiple strings and vectors. Secondly, constructing it can add overhead that should not be necessary, because often the same signature can be reused. The solution is to only keep a pointer to a signature in `MultiFunction` that is set during construction. Child classes are responsible for making sure that the signature lives long enough. In most cases, the signature is either embedded into the child class or it is allocated statically (and is only created once).
2021-03-22 11:57:24 +01:00
MFSignature signature_;
Functions: initial hash/equals implementation for constant multi-functions
2020-07-08 15:04:28 +02:00
public:
template<typename U> CustomMF_Constant(U &&value) : value_(std::forward<U>(value))
{
Functions: make multi functions smaller and cheaper to construct in many cases Previously, the signature of a `MultiFunction` was always embedded into the function. There are two issues with that. First, `MFSignature` is relatively large, because it contains multiple strings and vectors. Secondly, constructing it can add overhead that should not be necessary, because often the same signature can be reused. The solution is to only keep a pointer to a signature in `MultiFunction` that is set during construction. Child classes are responsible for making sure that the signature lives long enough. In most cases, the signature is either embedded into the child class or it is allocated statically (and is only created once).
2021-03-22 11:57:24 +01:00
MFSignatureBuilder signature{"Constant"};
Functions: use static names for multi-functions Previously, the function names were stored in `std::string` and were often created dynamically (especially when the function just output a constant). This resulted in a lot of overhead. Now the function name is just a `const char *` that should be statically allocated. This is good enough for the majority of cases. If a multi-function needs a more dynamic name, it can override the `MultiFunction::debug_name` method. In my test file with >400,000 simple math nodes, the execution time improves from 3s to 1s.
2021-11-21 12:37:04 +01:00
signature.single_output<T>("Value");
Functions: make multi functions smaller and cheaper to construct in many cases Previously, the signature of a `MultiFunction` was always embedded into the function. There are two issues with that. First, `MFSignature` is relatively large, because it contains multiple strings and vectors. Secondly, constructing it can add overhead that should not be necessary, because often the same signature can be reused. The solution is to only keep a pointer to a signature in `MultiFunction` that is set during construction. Child classes are responsible for making sure that the signature lives long enough. In most cases, the signature is either embedded into the child class or it is allocated statically (and is only created once).
2021-03-22 11:57:24 +01:00
signature_ = signature.build();
this->set_signature(&signature_);
Functions: initial hash/equals implementation for constant multi-functions
2020-07-08 15:04:28 +02:00
}
Cleanup: replace UNUSED macro with commented args in C++ code This is the conventional way of dealing with unused arguments in C++, since it works on all compilers. Regex find and replace: `UNUSED\((\w+)\)` -> `/*$1*/`
2022-10-03 17:37:25 -05:00
void call(IndexMask mask, MFParams params, MFContext /*context*/) const override
Functions: initial hash/equals implementation for constant multi-functions
2020-07-08 15:04:28 +02:00
{
MutableSpan<T> output = params.uninitialized_single_output<T>(0);
Functions: move loops into function builders This simplifies debugging, and can help improve performance by making it easier for the compiler. More optimization might still be possible by using `__restrict` in a few places.
2022-03-29 10:11:49 +02:00
mask.to_best_mask_type([&](const auto &mask) {
for (const int64_t i : mask) {
new (&output[i]) T(value_);
}
});
Functions: initial hash/equals implementation for constant multi-functions
2020-07-08 15:04:28 +02:00
}
Refactor: Update integer type usage This updates the usage of integer types in code I wrote according to our new style guides. Major changes: * Use signed instead of unsigned integers in many places. * C++ containers in blenlib use `int64_t` for size and indices now (instead of `uint`). * Hash values for C++ containers are 64 bit wide now (instead of 32 bit). I do hope that I broke no builds, but it is quite likely that some compiler reports slightly different errors. Please let me know when there are any errors. If the fix is small, feel free to commit it yourself. I compiled successfully on linux with gcc and on windows.
2020-07-20 12:16:20 +02:00
uint64_t hash() const override
Functions: initial hash/equals implementation for constant multi-functions
2020-07-08 15:04:28 +02:00
{
BLI: simplify using DefaultHash
2021-03-25 16:01:28 +01:00
return get_default_hash(value_);
Functions: initial hash/equals implementation for constant multi-functions
2020-07-08 15:04:28 +02:00
}
bool equals(const MultiFunction &other) const override
{
const CustomMF_Constant *other1 = dynamic_cast<const CustomMF_Constant *>(&other);
if (other1 != nullptr) {
return value_ == other1->value_;
}
const CustomMF_GenericConstant *other2 = dynamic_cast<const CustomMF_GenericConstant *>(
&other);
if (other2 != nullptr) {
Functions: use new is-equal and hash function of CPPType
2020-07-10 12:56:57 +02:00
const CPPType &type = CPPType::get<T>();
if (type == other2->type_) {
Functions: improve CPPType * Reduce code duplication. * Give methods more standardized names (e.g. `move_to_initialized` -> `move_assign`). * Support wrapping arbitrary C++ types, even those that e.g. are not copyable.
2021-06-28 13:13:52 +02:00
return type.is_equal_or_false(static_cast<const void *>(&value_), other2->value_);
Functions: initial hash/equals implementation for constant multi-functions
2020-07-08 15:04:28 +02:00
}
}
return false;
}
};
Nodes: support default function for partially implemented nodes
2020-07-16 13:38:23 +02:00
class CustomMF_DefaultOutput : public MultiFunction {
private:
Refactor: Update integer type usage This updates the usage of integer types in code I wrote according to our new style guides. Major changes: * Use signed instead of unsigned integers in many places. * C++ containers in blenlib use `int64_t` for size and indices now (instead of `uint`). * Hash values for C++ containers are 64 bit wide now (instead of 32 bit). I do hope that I broke no builds, but it is quite likely that some compiler reports slightly different errors. Please let me know when there are any errors. If the fix is small, feel free to commit it yourself. I compiled successfully on linux with gcc and on windows.
2020-07-20 12:16:20 +02:00
int output_amount_;
Functions: make multi functions smaller and cheaper to construct in many cases Previously, the signature of a `MultiFunction` was always embedded into the function. There are two issues with that. First, `MFSignature` is relatively large, because it contains multiple strings and vectors. Secondly, constructing it can add overhead that should not be necessary, because often the same signature can be reused. The solution is to only keep a pointer to a signature in `MultiFunction` that is set during construction. Child classes are responsible for making sure that the signature lives long enough. In most cases, the signature is either embedded into the child class or it is allocated statically (and is only created once).
2021-03-22 11:57:24 +01:00
MFSignature signature_;
Nodes: support default function for partially implemented nodes
2020-07-16 13:38:23 +02:00
public:
Functions: use static names for multi-functions Previously, the function names were stored in `std::string` and were often created dynamically (especially when the function just output a constant). This resulted in a lot of overhead. Now the function name is just a `const char *` that should be statically allocated. This is good enough for the majority of cases. If a multi-function needs a more dynamic name, it can override the `MultiFunction::debug_name` method. In my test file with >400,000 simple math nodes, the execution time improves from 3s to 1s.
2021-11-21 12:37:04 +01:00
CustomMF_DefaultOutput(Span<MFDataType> input_types, Span<MFDataType> output_types);
Nodes: support default function for partially implemented nodes
2020-07-16 13:38:23 +02:00
void call(IndexMask mask, MFParams params, MFContext context) const override;
};
Geometry Nodes: fields and anonymous attributes This implements the initial core framework for fields and anonymous attributes (also see T91274). The new functionality is hidden behind the "Geometry Nodes Fields" feature flag. When enabled in the user preferences, the following new nodes become available: `Position`, `Index`, `Normal`, `Set Position` and `Attribute Capture`. Socket inspection has not been updated to work with fields yet. Besides these changes at the user level, this patch contains the ground work for: * building and evaluating fields at run-time (`FN_fields.hh`) and * creating and accessing anonymous attributes on geometry (`BKE_anonymous_attribute.h`). For evaluating fields we use a new so called multi-function procedure (`FN_multi_function_procedure.hh`). It allows composing multi-functions in arbitrary ways and supports efficient evaluation as is required by fields. See `FN_multi_function_procedure.hh` for more details on how this evaluation mechanism can be used. A new `AttributeIDRef` has been added which allows handling named and anonymous attributes in the same way in many places. Hans and I worked on this patch together. Differential Revision: https://developer.blender.org/D12414
2021-09-09 12:54:20 +02:00
class CustomMF_GenericCopy : public MultiFunction {
private:
MFSignature signature_;
public:
Functions: use static names for multi-functions Previously, the function names were stored in `std::string` and were often created dynamically (especially when the function just output a constant). This resulted in a lot of overhead. Now the function name is just a `const char *` that should be statically allocated. This is good enough for the majority of cases. If a multi-function needs a more dynamic name, it can override the `MultiFunction::debug_name` method. In my test file with >400,000 simple math nodes, the execution time improves from 3s to 1s.
2021-11-21 12:37:04 +01:00
CustomMF_GenericCopy(MFDataType data_type);
Geometry Nodes: fields and anonymous attributes This implements the initial core framework for fields and anonymous attributes (also see T91274). The new functionality is hidden behind the "Geometry Nodes Fields" feature flag. When enabled in the user preferences, the following new nodes become available: `Position`, `Index`, `Normal`, `Set Position` and `Attribute Capture`. Socket inspection has not been updated to work with fields yet. Besides these changes at the user level, this patch contains the ground work for: * building and evaluating fields at run-time (`FN_fields.hh`) and * creating and accessing anonymous attributes on geometry (`BKE_anonymous_attribute.h`). For evaluating fields we use a new so called multi-function procedure (`FN_multi_function_procedure.hh`). It allows composing multi-functions in arbitrary ways and supports efficient evaluation as is required by fields. See `FN_multi_function_procedure.hh` for more details on how this evaluation mechanism can be used. A new `AttributeIDRef` has been added which allows handling named and anonymous attributes in the same way in many places. Hans and I worked on this patch together. Differential Revision: https://developer.blender.org/D12414
2021-09-09 12:54:20 +02:00
void call(IndexMask mask, MFParams params, MFContext context) const override;
};
Cleanup: use nested namespaces
2020-07-03 14:25:20 +02:00
} // namespace blender::fn
Copy Permalink