Expandable component sockets #105491

New Issue

Node editing can simplified by implementing expandable sockets, i.e. a feature whereby certain struct-like socket types (vectors, colors, transforms) can be expanded into component sockets within the same node. The functionality is the same as when inserting a Combine node before inputs or a Separate node after outputs.

Current combine/separate nodes:

Proposed expandable sockets:

This is mostly a UI feature to reduce the need for inserting explicit socket Separate/Combine nodes. It does not involve the implementation of generic, user-defined structs, which are a larger feature.

List of required changes:

bNodeSocketType API to determine expandable socket types and what components they expand into
BKE methods for bNodeSocket to get component/parent sockets in the same node
bNodeSocket flag for expanded state
Node verification changes to add/remove (or show/hide?) component sockets, as expansion state changes
Socket draw code additions for expansion button and clearly identifiable component sockets
Node execution changes (Function nodes, geometry nodes, shaders?) to support expanded sockets

Node editing can simplified by implementing *expandable sockets*, i.e. a feature whereby certain struct-like socket types (vectors, colors, transforms) can be expanded into *component sockets* within the same node. The functionality is the same as when inserting a `Combine` node before inputs or a `Separate` node after outputs. Current combine/separate nodes: ![image](/attachments/1244fe79-4ca0-4335-bde5-cdfac4148fe3) Proposed expandable sockets: ![image](/attachments/18f4912c-e10d-4ea5-beb1-382d4228a3f2) This is mostly a UI feature to reduce the need for inserting explicit socket Separate/Combine nodes. It does **not** involve the implementation of generic, user-defined structs, which are a larger feature. List of required changes: - [ ] `bNodeSocketType` API to determine expandable socket types and what components they expand into - [ ] BKE methods for `bNodeSocket` to get component/parent sockets in the same node - [ ] `bNodeSocket` flag for expanded state - [ ] Node verification changes to add/remove (or show/hide?) component sockets, as expansion state changes - [ ] Socket draw code additions for expansion button and clearly identifiable component sockets - [ ] Node execution changes (Function nodes, geometry nodes, shaders?) to support expanded sockets

image.png

122 KiB

image.png

88 KiB

image.png

122 KiB

image.png

88 KiB

What opportunities will be open if some kind of infrastructure for this appears:

Dynamic declarations will be able to extend multi-input sockets to multiple sockets for links already inserted.
Many nodes (mainly for geometry primitives) will be able to transfer the function of the input variability of the geometric parameters of the primitive (define cuboid by x,y,z axis values, or by min/max points? xyz axis also split into individual inputs, or consisted as vector?...) to the input sockets.
Including enums, not only arrow button.

What opportunities will be open if some kind of infrastructure for this appears: - Dynamic declarations will be able to extend multi-input sockets to multiple sockets for links already inserted. - Many nodes (mainly for geometry primitives) will be able to transfer the function of the input variability of the geometric parameters of the primitive (define cuboid by x,y,z axis values, or by min/max points? xyz axis also split into individual inputs, or consisted as vector?...) to the input sockets. Including enums, not only arrow button.

I have two big wishes that would help tremendously with the Animation & Rigging efforts towards a node-based rigging system. FYI, this system is prototyped at PowerShip.

Compound-to-compound expansion

IMO the 'expandability' of sockets shouldn't be limited to expanding to individual float values. Conceptually matrices should be 'expandable' into their location/rotation/scale components (being vector3, quaternion, vector3).

Connection-sensitive operation

It would be great if this could be lifted beyond the "avoid combine/separate nodes" use case.

For example, the above Set Control node is implemented such that it leaves the transform components alone when their socket is not connected. So the above one would keep rotation and scale of the control as-is, and only updates its location.

In contrast, using a "Combine Matrix" node that only has its 'location' socket connected will produce a matrix that would reset the control rotation and scale. This would hamper usability of the node system, because you then cannot use separate nodes to control loc/rot/scale individually.

This idea might also impact the lower-level float components that make up the vectors/quaternions. Not sure what the design impact of that would be.

Underlying design

For me the idea of 'matrices for all transforms' is tempting, but also has its limitations. For example, a matrix contains a shear component as well, and if it's nonzero, it can get in the way of decomposing the matrix into its loc/rot/scale components. Such a decomposition would certainly loose the shear part.

It might also be computationally heavier than necessary to convert individual loc/rot/scale components into a matrix, only to pass it through a noodle, and then be decomposed again.

Maybe this could lead to a new 'transform' socket (and maybe noodle) type? Something that could convey loc/rot/scale as two vectors and a quaternion, which are then only combined into a matrix when they have to be. Connecting the collapsed 'transform' socket would thus 'under water' connect all three loc/rot/scale sub-sockets with one noodle, and not bother with matrix conversions.

I have two big wishes that would help tremendously with the Animation & Rigging efforts towards a node-based rigging system. FYI, this system is prototyped at [PowerShip](https://projects.blender.org/dr.sybren/powership). #### Compound-to-compound expansion IMO the 'expandability' of sockets shouldn't be limited to expanding to individual `float` values. Conceptually matrices should be 'expandable' into their location/rotation/scale components (being vector3, quaternion, vector3). #### Connection-sensitive operation It would be great if this could be lifted beyond the "avoid combine/separate nodes" use case. ![image](/attachments/55c05424-91b3-4caf-b30b-9257a6d11f28) For example, the above _Set Control_ node is implemented such that it leaves the transform components alone when their socket is not connected. So the above one would keep rotation and scale of the control as-is, and only updates its location. In contrast, using a "Combine Matrix" node that only has its 'location' socket connected will produce a matrix that would _reset_ the control rotation and scale. This would hamper usability of the node system, because you then cannot use separate nodes to control loc/rot/scale individually. This idea might also impact the lower-level float components that make up the vectors/quaternions. Not sure what the design impact of that would be. #### Underlying design For me the idea of 'matrices for all transforms' is tempting, but also has its limitations. For example, a matrix contains a shear component as well, and if it's nonzero, it can get in the way of decomposing the matrix into its loc/rot/scale components. Such a decomposition would certainly loose the shear part. It might also be computationally heavier than necessary to convert individual loc/rot/scale components into a matrix, only to pass it through a noodle, and then be decomposed again. Maybe this could lead to a new 'transform' socket (and maybe noodle) type? Something that could convey loc/rot/scale as two vectors and a quaternion, which are then only combined into a matrix when they have to be. Connecting the collapsed 'transform' socket would thus 'under water' connect all three loc/rot/scale sub-sockets with one noodle, and not bother with matrix conversions.

image.png

31 KiB

@dr.sybren A more complex version of socket components is on track, as far as I can see.
But a dynamic type with computational complexity balancing... it need much more thinkings...

@dr.sybren A more complex version of socket components is on track, as far as I can see. But a dynamic type with computational complexity balancing... it need much more thinkings...

Download

What's New

Blender Studio

Manual

Developers Blog

Documentation

Benchmark

Blender Conference

Development Fund

One-time Donations

Expandable component sockets #105491

Compound-to-compound expansion

Connection-sensitive operation

Underlying design