Matrix operations and sockets for geometry nodes #105408

Lukas Tönne · 2023-03-03T17:02:11+01:00

Lukas Tönne commented

2023-03-03 17:02:11 +01:00

Support for matrix operations in geometry nodes (and function nodes generally)

customdata layer types for 2x2, 3x3, 4x4 matrices
New socket type 'Matrix' to represent 4x4 matrix data
Basic set of function nodes for matrix operations:
- Input nodes to specify an explicit matrix by components (default matrix is 4x4 identity)
- 'Combine' node to construct matrices by columns, rows, or individual elements
- 'Separate' node to extract columns, rows, or elements
- Translate/Rotate/Scale nodes to generate affine matrices
- 'Decompose' node to return translation, rotation, and scale.
- Matrix Math for common operations, such as matrix/scalar multiplication, inverse, determinant, etc.
- 'Transform Vector' node to apply a matrix as a transformation to a vector

The socket represents a 4x4 matrix as the only type. This is to simplify matrix operations for users. Supporting 2x2 and 3x3 matrix types could have better performance and require less memory, but would make conversions necessary and force users to pick the correct variant of node for the respective matrix type. Eventually this might be supported with a generic socket type.

Support for matrix operations in geometry nodes (and function nodes generally) * customdata layer types for 2x2, 3x3, 4x4 matrices * New socket type 'Matrix' to represent 4x4 matrix data * Basic set of function nodes for matrix operations: - Input nodes to specify an explicit matrix by components (default matrix is 4x4 identity) ![image](/attachments/6e714b98-2809-40c5-af9e-0f21ec13cf2e) - 'Combine' node to construct matrices by columns, rows, or individual elements ![image](/attachments/3bb7c0fe-4e09-44f0-b6a9-5bd44bcbe584) - 'Separate' node to extract columns, rows, or elements ![image](/attachments/f3355225-110e-4731-b3c7-cdbcbaeb7ce7) - Translate/Rotate/Scale nodes to generate affine matrices ![image](/attachments/a11de734-e28f-4768-961c-b7ccf7de575e) - 'Decompose' node to return translation, rotation, and scale. ![image](/attachments/95c0de44-7703-4c0b-878d-48526953791b) - Matrix Math for common operations, such as matrix/scalar multiplication, inverse, determinant, etc. ![image](/attachments/a47724e1-5199-47fa-a04e-7759a1cff47d) - 'Transform Vector' node to apply a matrix as a transformation to a vector ![image](/attachments/f145b367-e4ec-4e1a-b2f6-00c8b178fd90) The socket represents a 4x4 matrix as the only type. This is to simplify matrix operations for users. Supporting 2x2 and 3x3 matrix types could have better performance and require less memory, but would make conversions necessary and force users to pick the correct variant of node for the respective matrix type. Eventually this might be supported with a generic socket type.

image.png

14 KiB

image.png

7.7 KiB

image.png

4.7 KiB

image.png

17 KiB

image.png

29 KiB

image.png

7.2 KiB

image.png

47 KiB

👍 1 👀 2 ❤️ 12 🎉 1

Lukas Tönne added 35 commits 2023-03-03 17:02:29 +01:00

a3b7f972d0 Some initial files for matrix nodes.

87004bd122 Blenkernel socket identifiers.

9a55a9bdd5 Plumbing for the SOCK_MATRIX types.

8bfd84a3c9 Finished majority of boilerplate code for matrix socket types.

726ad272aa Implementation of the input, combine and separate nodes for matrix_4x4.

bfa19daccc Stub implementation of the matrix4x4 math node.

8394e73a2f Register CppType for float3x3.

6156c1f8e1 Hash function for the float3x3 type.

f773604aae Declare field CppTypes for float3x3 and float4x4.

9cd8342506 Fixed callback signature for matrix multifunctions.

eac7726df3 Added missing implementation of socket declaration builders.

88cbc107f1 Include runtime C++ functions for bNode.

6b9e700ed1 Custom data layer types for float matrix types.

b5f6919a80 Node menu entries for matrix nodes.

147a3e635e Fix vec3 input of the matrix input node.

eea512fbcc Matrix input mode for the instance-on-points node.

9c01a58ceb Added nodes for both 3x3 and 4x4 matrix variants.

83e4fef23c Revert "Matrix input mode for the instance-on-points node."

This reverts commit eea512fbcc.
Should be decided separately if a matrix input is better suited for the
instance-on-points node or other existing nodes.

edbd778923 Merge branch 'master' into nodes-matrix-types

95f1b7c904 Fix incorrect output socket indices in math nodes.

1b90ba3d04 Use correct storage callbacks for matrix input nodes.

ccaeb31e46 Use the large size preset for matrix input nodes to accomodate all the columns.

b2825c4176 Fixed socket lookup functions in CombineMatrix4x4 node.

ddb172eef3 Fixed incorrect socket lookup in a few places.

8ee84bc82d Added a few more nodes for affine (de)composition.

6d107b9f5a Merge branch 'main' into nodes-matrix-types

ad36523191 Merge branch 'main' into nodes-matrix-types

49b22cc187 Fix crash when socket UI creates an empty layout.

cdd86e85f6 Moved the matrix nodes menu into the Utilities category.

bfde6c8a9c Removed the 2x2 and 3x3 matrix types.

Having different matrix types is likely to be confusing for users
and complicates node trees by requiring conversions.
For now there will be only a the 4x4 matrix type, which can encode
3x3 and 2x2 matrices as well at the cost of some performance.

Eventually a generic socket type could make this easier and allow
the introduction of additional matrix types without too much
inconvenience for users.

79ff2f88e2 Renamed matrix nodes without the 4x4 suffix.

Matrix nodes should be generic enough to work with all kinds
of matrices. Right now that will only be 4x4 matrices but the
nodes don't need that in their names.

4580828dd9 Fix indexing errors in the matrix decompose node.

ddad2b8337 Implemented remaining matrix math operations.

95875b8cc7 Added a transform node for applying matrices to vectors.

4f74ca2b79 Renamed matrix socket type as well by removing the 4x4 suffix.

Iliya Katushenock commented

2023-03-03 17:25:54 +01:00

I'm glad that finally there are steps to this!
A couple of things at a glance:

Do we want a matrix rotation node? Would it be more extensible to have a rotation matrix generator node capable of doing this for euler, normal axis, ... ?
Do we need to enter all 16 components as separate entries? 2 vectors, 3 vectors, 4 vectors and 4 values... Not sure about the purity, but there are even more sockets than the statistics node!

I'm glad that finally there are steps to this! A couple of things at a glance: 1. Do we want a matrix rotation node? Would it be more extensible to have a rotation matrix generator node capable of doing this for euler, normal axis, ... ? 2. Do we need to enter all 16 components as separate entries? 2 vectors, 3 vectors, 4 vectors and 4 values... Not sure about the purity, but there are even more sockets than the statistics node!

❤️ 1

Kenzie commented

2023-03-03 17:29:58 +01:00

I would personally recommend against going this route and instead present the user with the abstract concept of a transform. Matrices are lossy and can be easily deformed in a way that causes bugs further down the line.

Storing the location, rotation (quaternion, similarly abstracted), and scale as well as order is plenty to define and carry a transform. Imo to an end user matrices should be only used as a baked representation of these transforms (or black boxed formulas such as perspective). My approach to using them is that if avoidable they should not be manually modified, inversed (if possible to avoid loss of precision in world space), or decomposed back into LRS transforms. Possibly never exposed at all to the end user.

I would personally recommend against going this route and instead present the user with the abstract concept of a transform. Matrices are lossy and can be easily deformed in a way that causes bugs further down the line. Storing the location, rotation (quaternion, similarly abstracted), and scale as well as order is plenty to define and carry a transform. Imo to an end user matrices should be only used as a baked representation of these transforms (or black boxed formulas such as perspective). My approach to using them is that if avoidable they should not be manually modified, inversed (if possible to avoid loss of precision in world space), or decomposed back into LRS transforms. Possibly never exposed at all to the end user.

Iliya Katushenock commented

2023-03-03 17:32:50 +01:00

@KenzieMac130 Matrices are a very important mathematical abstraction that will give you a lot of possibilities. The quality of data accuracy is more of a general issue. And it is a consequence of poor user experience. Even transformations won't save her.

Kenzie commented

2023-03-03 17:46:25 +01:00

For the average user of 3D software they will only ever use location, rotation, and scale. Decomposing is lossy with rotation and scale, therefore not ideal for storing those values to retrieve later. Shears and other non-afine transforms to be applied on vertices are likely to be done by the average user using vector math instead nowadays. Giving the user the ability to arbitrarily define matrices is a guaranteed way to get unsanitary data. From what I have seen plenty of other 3D packages and engines avoid exposing them directly to the user.

Lukas Tönne added this to the Nodes & Physics project 2023-03-03 17:57:49 +01:00

Lukas Tönne requested review from Hans Goudey 2023-03-03 17:59:19 +01:00

Steff Kempink commented

2023-03-03 18:00:26 +01:00

First-time contributor

I'd love to join into the design discussion, but isn't a (not yet existing) design issue the right place for that? Or was this initial design already approved by the Nodes & Physics team?

Iliya Katushenock commented

2023-03-03 18:04:17 +01:00

@McSwiff No design, Lukas took up prototyping matrix nodes after chatting in geometry nodes channel in chat

👍 1

Iliya Katushenock commented

2023-03-03 18:07:33 +01:00

@KenzieMac130 Matrix is a mathematical abstraction for describing any linear 3D transformation.

Their correctness is a consequence of the conscious operation of the user.
If we consider the user to be unfamiliar with mathematics, then we simply have to make a button: Make Ok.
In the case of other parametric programs, it seems that matrices are included in the list of primitives, which should be in the first place.

@KenzieMac130 Matrix is a mathematical abstraction for describing any linear 3D transformation. - Their correctness is a consequence of the conscious operation of the user. - If we consider the user to be unfamiliar with mathematics, then we simply have to make a button: Make Ok. - In the case of other parametric programs, it seems that matrices are included in the list of primitives, which should be in the first place.

Steff Kempink commented

2023-03-03 18:16:10 +01:00

First-time contributor

Okay then, so minor design discussions are allowed in PRs now?
Given that matrix math is already a very sharp tool with which you can easily cut yourself. I would advocate for a "Compose/Recompose Matrix" node as a mirror to the "Decompose Matrix".
I've used it in the past, knowing fully well the loss of precision that comes with it. But I'm at the end of my chain of calculations. I just needed a simple to express and easy to read way to take my transform matrix and set the rotation of it based on something I calculated.
Is that a strong enough reason?

Okay then, so minor design discussions are allowed in PRs now? Given that matrix math is already a very sharp tool with which you can easily cut yourself. I would advocate for a "Compose/Recompose Matrix" node as a mirror to the "Decompose Matrix". I've used it in the past, knowing fully well the loss of precision that comes with it. But I'm at the end of my chain of calculations. I just needed a simple to express and easy to read way to take my transform matrix and set the rotation of it based on something I calculated. Is that a strong enough reason?

Lukas Tönne added 2 commits 2023-03-06 11:59:44 +01:00

997dbf8cc5 Fixed outdated matrix input menu entry.

dd55a1f93d Merge remote-tracking branch 'origin/main' into nodes-matrix-types

Iliya Katushenock added the

Download

What's New

Blender Studio

Manual

Developers Blog

Documentation

Benchmark

Blender Conference

Development Fund

One-time Donations

Matrix operations and sockets for geometry nodes #105408

Pull request closed