Geometry Nodes: Freeze Node #106497

New Issue

Jacques Lucke · 2023-04-03T15:52:21+02:00

Jacques Lucke commented

2023-04-03 15:52:21 +02:00

The Freeze Geometry node can store the procedurally generated input geometry, allowing it to output the geometry without having to recompute it every time. It's also capable of storing different geometries for different frames. This is also a first step towards the strictly more general Edit Geometry node that has been discussed before.

Even without the ability to manually edit the frozen geometry (which is what the Edit Geometry node would allow) this node has important use cases:

Bake simulation output to allow random access of individual frames.
Speedup node tree evaluation by manually freezing parts of it that are not expected to change (previous work).
Access baked animated geometry at arbitrary frames.

Baking using Freeze Geometry vs. special purpose simulation baking

The main reason I'm writing this document now is that I was working on baking for simulation nodes. Doing that, I noticed that what makes this hard is the same thing that would make the Freeze Geometry node hard. There are some benefits to using the more general solution than something that is specific to simulation:

Only the output of the simulation is frozen instead of the entire simulation state which might contain data that is only necessary during the simulation.
The user has more control over what is baked and what is not.
Better separation of concerns (simulation vs. storing animated geometry).
Implementation actually seems to become easier, because the Freeze Geometry node can be worked on independently of simulation nodes.
People would misuse the simulation caching mechanism as general geometry freezing for performance reason, better provide the right solution directly instead of giving people the chance to start using simulations for that purpose (which may be problematic in some cases because something that should be static geometry suddenly contains a simulation and people might not actually know if it's abstracted away by a node group).

There are also some disadvantages which don't outweigh the benefits currently:

Baking simulations is generally necessary for rendering. This implies that users most likely have to use an additional node in order to work with simulations. This gives more control to the user but might make things slightly more complicated in simple setups.
Since the Freeze Geometry node does not store the entire simulation state (but only the output geometry), it's not (easily) possible to continue baking a partially baked simulation after restarting Blender. This functionality could be added separately in the future of course.

Storing the Geometry

It should be possible to store the frozen geometry in within the .blend file and to store it externally.

The storage within the .blend file is not entirely trivial because a GeometrySet is not just an ID data block that is in Main. Also we likely want to support compression across multiple non-main geometry data blocks, because animated geometry often contains duplicate data at different frames. Handling instances is also a bit tricky since they are not an ID data block either (other than e.g. Mesh).

An alternative would be to actually store the frozen geometry data blocks in Main (except for instances for now, probably). The downside is that this might add a relatively large number of data blocks, especially when freezing an animation. It's unclear whether all the individually frozen geometries should just be normal data blocks or whether they should be hidden from the user in some way.

For storing frozen geometry externally, the same considerations done in #105251 apply. A custom format likely works best for this use case because it allows us to store all the data we want without overcomplicating the .blend file format (this is kind of similar to .geo and .bgeo files).

Manual vs. Automatic Caching

Ideally, we would implement automatic caching in geometry nodes before manually freezing geometries, because that would solve some use cases of the Freeze Geometry node in a better way. The problem with that is that people learn to use the Freeze Geometry node in cases where it is not necessary in the future. Longer term, it's likely less relevant which form of caching is implemented first, because it shouldn't be too hard to unlearn adding a Freeze Geometry node once it doesn't help performance anymore.

Store geometry per node vs. per node per context

There are two main places where the frozen geometry can be stored. Both approaches have valid use cases, so it may be necessary to support both (now or longer term).

In the node itself: If the frozen geometry is stored in the node directly, that means that if the node is in a node groups that's used in multiple places, it will output the same geometry everywhere, even if the input geometry differs in different contexts. This has the benefit that the node can easily be moved around/copied etc. without loosing the frozen data. The UI of the node will be the same no matter in which context it is looked at.
In the modifier per node context: This would mean that the modifier has a mapping from node-contexts to frozen geometry. If the node is in a node group that is used multiple times, it may be frozen in some contexts and not in others. This likely also means that the node UI can look different depending on the current context in the node tree.

Anonymous Attributes

Anonymous attributes present a challenge because which attributes a node propagates depends on what happens to the right of the node (more precisely on which anonymous attributes are used there). The consequence of that is that anonymous attributes that are not needed when the geometry is frozen are not available later on when the anonymous attribute is used because something later in the tree changed.

There are three possible solutions here:

Never freeze any anonymous attributes. This makes it harder to add a Freeze Geometry in the middle of an existing node setup and users would be forced to use named attributes.
Explicitly pass anonymous attributes through the Freeze Geometry node using additional sockets (this was originally mentioned here). This makes it possible for users to use the freeze node without having to resort to named attributes, but it's still not trivial to add the node in the middle of an existing setup. This also has the problem that it doesn't work well when an anonymous attribute only exists on parts of the geometry set (e.g. only on a mesh, but there is also a point cloud that doesn't have it).
Warn the user when the set of propagated anonymous attributes changes and a re-freeze is necessary. This would make it trivial to add the node in existing setups and does not suffer from the issues above. It just may be less obvious why some anonymous attribute is not available after the node, hence the warning is necessary.

Currently, I think option 3 would work best. This is easier now than it used to be during the last discussion because of 2ffd08e952.

Global Simulation Baking

In cases where there are multiple simulations on potentially multiple objects, we want to have the ability to freeze them all with the click of one button. It's not entirely clear to me yet how this should be done in a good way. Somehow the user would have to specify which Freeze Geometry nodes should be frozen. If there are multiple simulations in a chain, it should only be necessary to cache the output of the last simulation (I expect this case to become fairly common as simple simulations are abstracted away using node group assets). The even more tricky thing is to unfreeze, because we likely don't actually want to unfreeze everything in case the user explicitly wanted to keep something frozen.

In either way, with the Freeze Geometry node replacing other special purpose simulation baking, the user would have to add and configure the node whenever an object has a simulation. Otherwise it can't be baked using a global button. In a production environment that is unlikely to be an issue and might even be benefitial because of the added control. Simple Python scripts could be used to freeze/unfreeze exactly as desired. It's mostly the simple "out of the box" experience that suffers a bit..

Main Unknowns

How to store the frozen geometry in the .blend file exactly. Is it ok to add a potentially large number of ID data blocks? What about sharing data between those data blocks when e.g. every mesh at every frame is stored at separate data block?
How to support a global "Bake" button that freezes everything that needs to be frozen and unfrozen? Is that even feasible or what it be much better to support simulation-specific baking for that (which would mean that most of this Freeze Geometry node proposal would be postponed again).

The `Freeze Geometry` node can store the procedurally generated input geometry, allowing it to output the geometry without having to recompute it every time. It's also capable of storing different geometries for different frames. This is also a first step towards the strictly more general `Edit Geometry` node that has been discussed [before](https://code.blender.org/2022/11/geometry-nodes-workshop-2022/#geometry-object). Even without the ability to manually edit the frozen geometry (which is what the `Edit Geometry` node would allow) this node has important use cases: * Bake simulation output to allow random access of individual frames. * Speedup node tree evaluation by manually freezing parts of it that are not expected to change ([previous work](https://devtalk.blender.org/t/geometry-nodes-caching-prototype/25518)). * Access baked animated geometry at arbitrary frames. ![image](/attachments/1964da8e-596b-48e9-8cf2-51437ce1f4a1) ### Baking using Freeze Geometry vs. special purpose simulation baking The main reason I'm writing this document now is that I was working on baking for simulation nodes. Doing that, I noticed that what makes this hard is the same thing that would make the `Freeze Geometry` node hard. There are some benefits to using the more general solution than something that is specific to simulation: * Only the output of the simulation is frozen instead of the entire simulation state which might contain data that is only necessary during the simulation. * The user has more control over what is baked and what is not. * Better separation of concerns (simulation vs. storing animated geometry). * Implementation actually seems to become easier, because the `Freeze Geometry` node can be worked on independently of simulation nodes. * People would misuse the simulation caching mechanism as general geometry freezing for performance reason, better provide the right solution directly instead of giving people the chance to start using simulations for that purpose (which may be problematic in some cases because something that should be static geometry suddenly contains a simulation and people might not actually know if it's abstracted away by a node group). There are also some disadvantages which don't outweigh the benefits currently: * Baking simulations is generally necessary for rendering. This implies that users most likely have to use an additional node in order to work with simulations. This gives more control to the user but might make things slightly more complicated in simple setups. * Since the `Freeze Geometry` node does not store the entire simulation state (but only the output geometry), it's not (easily) possible to continue baking a partially baked simulation after restarting Blender. This functionality could be added separately in the future of course. ### Storing the Geometry It should be possible to store the frozen geometry in within the .blend file and to store it externally. The storage within the .blend file is not entirely trivial because a `GeometrySet` is not just an ID data block that is in `Main`. Also we likely want to support compression across multiple non-main geometry data blocks, because animated geometry often contains duplicate data at different frames. Handling instances is also a bit tricky since they are not an `ID` data block either (other than e.g. `Mesh`). An alternative would be to actually store the frozen geometry data blocks in `Main` (except for instances for now, probably). The downside is that this might add a relatively large number of data blocks, especially when freezing an animation. It's unclear whether all the individually frozen geometries should just be normal data blocks or whether they should be hidden from the user in some way. For storing frozen geometry externally, the same considerations done in #105251 apply. A custom format likely works best for this use case because it allows us to store all the data we want without overcomplicating the `.blend` file format (this is kind of similar to `.geo` and `.bgeo` files). ### Manual vs. Automatic Caching Ideally, we would implement automatic caching in geometry nodes before manually freezing geometries, because that would solve some use cases of the `Freeze Geometry` node in a better way. The problem with that is that people learn to use the `Freeze Geometry` node in cases where it is not necessary in the future. Longer term, it's likely less relevant which form of caching is implemented first, because it shouldn't be too hard to unlearn adding a `Freeze Geometry` node once it doesn't help performance anymore. ### Store geometry per node vs. per node per context There are two main places where the frozen geometry can be stored. Both approaches have valid use cases, so it may be necessary to support both (now or longer term). 1. In the node itself: If the frozen geometry is stored in the node directly, that means that if the node is in a node groups that's used in multiple places, it will output the same geometry everywhere, even if the input geometry differs in different contexts. This has the benefit that the node can easily be moved around/copied etc. without loosing the frozen data. The UI of the node will be the same no matter in which context it is looked at. 2. In the modifier per node context: This would mean that the modifier has a mapping from node-contexts to frozen geometry. If the node is in a node group that is used multiple times, it may be frozen in some contexts and not in others. This likely also means that the node UI can look different depending on the current context in the node tree. ### Anonymous Attributes Anonymous attributes present a challenge because which attributes a node propagates depends on what happens to the right of the node (more precisely on which anonymous attributes are used there). The consequence of that is that anonymous attributes that are not needed when the geometry is frozen are not available later on when the anonymous attribute is used because something later in the tree changed. There are three possible solutions here: 1. Never freeze any anonymous attributes. This makes it harder to add a `Freeze Geometry` in the middle of an existing node setup and users would be forced to use named attributes. 2. Explicitly pass anonymous attributes through the `Freeze Geometry` node using additional sockets (this was originally mentioned [here](https://code.blender.org/2022/11/geometry-nodes-workshop-2022/#geometry-object)). This makes it possible for users to use the freeze node without having to resort to named attributes, but it's still not trivial to add the node in the middle of an existing setup. This also has the problem that it doesn't work well when an anonymous attribute only exists on parts of the geometry set (e.g. only on a mesh, but there is also a point cloud that doesn't have it). 3. Warn the user when the set of propagated anonymous attributes changes and a re-freeze is necessary. This would make it trivial to add the node in existing setups and does not suffer from the issues above. It just may be less obvious why some anonymous attribute is not available after the node, hence the warning is necessary. Currently, I think option 3 would work best. This is easier now than it used to be during the last discussion because of 2ffd08e95249. ### Global Simulation Baking In cases where there are multiple simulations on potentially multiple objects, we want to have the ability to freeze them all with the click of one button. It's not entirely clear to me yet how this should be done in a good way. Somehow the user would have to specify which `Freeze Geometry` nodes should be frozen. If there are multiple simulations in a chain, it should only be necessary to cache the output of the last simulation (I expect this case to become fairly common as simple simulations are abstracted away using node group assets). The even more tricky thing is to unfreeze, because we likely don't actually want to unfreeze everything in case the user explicitly wanted to keep something frozen. In either way, with the `Freeze Geometry` node replacing other special purpose simulation baking, the user would have to add and configure the node whenever an object has a simulation. Otherwise it can't be baked using a global button. In a production environment that is unlikely to be an issue and might even be benefitial because of the added control. Simple Python scripts could be used to freeze/unfreeze exactly as desired. It's mostly the simple "out of the box" experience that suffers a bit.. ## Main Unknowns * How to store the frozen geometry in the .blend file exactly. Is it ok to add a potentially large number of ID data blocks? What about sharing data between those data blocks when e.g. every mesh at every frame is stored at separate data block? * How to support a global "Bake" button that freezes everything that needs to be frozen and unfrozen? Is that even feasible or what it be much better to support simulation-specific baking for that (which would mean that most of this Freeze Geometry node proposal would be postponed again).

image.png

17 KiB

freeze_geometry_most_simple_usage.mp4

677 KiB

👍 4 🎉 3

Jacques Lucke added the

Type

To Do

label 2023-04-03 15:52:21 +02:00

Jacques Lucke added this to the Nodes & Physics project 2023-04-03 15:52:22 +02:00

Lukas Tönne commented

2023-04-04 10:27:52 +02:00

This is also a first step towards the strictly more general Edit Geometry node

What would a combination of Freeze and Edit nodes look like? It seems to me that Edit Geometry is an extension of Freeze, with the edit mode built on top of the existing caching. Will the Edit node be usable on its own or do you still need to Freeze first and then Edit, and does that mean caching data twice?

Baking simulations is generally necessary for rendering

Would it be fair to say that baking is necessary for efficient (parallel) rendering? In principle you can currently render single-threaded and in sequence, doing simulations on-the-fly. That's not recommended for final rendering, but should currently be possible by design (at least if our physics code would support it properly).

If there was a general "Prepare for Render" stage at the end to cache all the things, would it be able to detect existing cache nodes and re-use them to avoid unnecessary duplication?

> This is also a first step towards the strictly more general Edit Geometry node What would a combination of `Freeze` and `Edit` nodes look like? It seems to me that `Edit Geometry` is an extension of `Freeze`, with the edit mode built on top of the existing caching. Will the `Edit` node be usable on its own or do you still need to `Freeze` first and then `Edit`, and does that mean caching data twice? > Baking simulations is generally necessary for rendering Would it be fair to say that baking is necessary for _efficient_ (parallel) rendering? In principle you can currently render single-threaded and in sequence, doing simulations on-the-fly. That's not recommended for final rendering, but should currently be possible by design (at least if our physics code [would support it properly](https://projects.blender.org/blender/blender/issues/94235#issuecomment-913698)). If there was a general "Prepare for Render" stage at the end to cache all the things, would it be able to detect existing cache nodes and re-use them to avoid unnecessary duplication?

Jacques Lucke commented

2023-04-04 13:46:28 +02:00

What would a combination of Freeze and Edit nodes look like? It seems to me that Edit Geometry is an extension of Freeze, with the edit mode built on top of the existing caching.

That's exactly what I meant.

Will the Edit node be usable on its own or do you still need to Freeze first and then Edit, and does that mean caching data twice?

We don't have a full design of the edit functionality yet, and it could probably be implemented either way.

Would it be fair to say that baking is necessary for efficient (parallel) rendering?

Yeah, I think so. The main difficulty here might be that the rendering code might have to know substeps the simulation needs exactly

If there was a general "Prepare for Render" stage at the end to cache all the things, would it be able to detect existing cache nodes and re-use them to avoid unnecessary duplication?

Long term, we should try that even if there are multiple cache nodes, there is no significant memory duplication even when they store the same geometries. At run-time that will be relatively easy to do with implicit-sharing, but the .blend file format doesn't really support that yet.

> What would a combination of Freeze and Edit nodes look like? It seems to me that Edit Geometry is an extension of Freeze, with the edit mode built on top of the existing caching. That's exactly what I meant. > Will the Edit node be usable on its own or do you still need to Freeze first and then Edit, and does that mean caching data twice? We don't have a full design of the edit functionality yet, and it could probably be implemented either way. > Would it be fair to say that baking is necessary for efficient (parallel) rendering? Yeah, I think so. The main difficulty here might be that the rendering code might have to know substeps the simulation needs exactly > If there was a general "Prepare for Render" stage at the end to cache all the things, would it be able to detect existing cache nodes and re-use them to avoid unnecessary duplication? Long term, we should try that even if there are multiple cache nodes, there is no significant memory duplication even when they store the same geometries. At run-time that will be relatively easy to do with implicit-sharing, but the .blend file format doesn't really support that yet.

Dalai Felinto commented

2023-04-04 18:11:34 +02:00

For the records, this patch was discussed here: https://devtalk.blender.org/t/2023-04-04-nodes-physics-module-meeting/28457

Jacques Lucke added

and removed

labels 2023-04-04 19:30:45 +02:00

Dalai Felinto commented

2023-04-07 18:34:31 +02:00

I've been thinking about an alternative. Basically to embrace the concept of checkpoints a bit more.

What is a check-point? Any point in the node-tree that you can bake and use the baked result instead.

Checkpoints:

Simulation zone
Freeze Node
Edit Geometry (maybe an upgraded version of freeze node, maybe its own node)

The checkpoints are scattered inside the node-trees and they store specific baking setting. There is an overview on the object level (or scene level?) that allow you to see all the checkpoints inside an object.

On that level you can check for each of the checkpoints:

If the checkpoint is baked.
If it should be baked as part of object-level baking.
Any checkpoint baking specific setting (baking folder, ...).

That means that a simulation zone is a check-point on its own, but it doesn't need to be included when baking. Also chained simulations can be disabled for object-level baking, leaving only the last one (or a freezing node).

Final thoughts:

Mixing edit geometry and the other baking solutions (freeze, simulation) adds overhead for when users don't want to edit and yet need to setup datablock for the baked data.
It should be possible to convert between freeze and edit geometry.
The Outliner may be a good place to see the checkpoints and their settings.
Having the baking operator (button or menu entry) outside the nodetree lead to a better separation between data (nodes) and operations (the operators: free, cache, ...).
Users can still have a way to quickly focus on the operators related to bake a specific checkpoint (like we do for texture settings, by changing a nearby editor to the appropriate type and subtype/tab).

I've been thinking about an alternative. Basically to embrace the concept of checkpoints a bit more. What is a check-point? Any point in the node-tree that you can bake and use the baked result instead. Checkpoints: * Simulation zone * Freeze Node * Edit Geometry (maybe an upgraded version of freeze node, maybe its own node) The checkpoints are scattered inside the node-trees and they store specific baking setting. There is an overview on the object level (or scene level?) that allow you to see all the checkpoints inside an object. On that level you can check for each of the checkpoints: * If the checkpoint is baked. * If it should be baked as part of object-level baking. * Any checkpoint baking specific setting (baking folder, ...). That means that a simulation zone is a check-point on its own, but it doesn't need to be included when baking. Also chained simulations can be disabled for object-level baking, leaving only the last one (or a freezing node). ---- Final thoughts: * Mixing edit geometry and the other baking solutions (freeze, simulation) adds overhead for when users don't want to edit and yet need to setup datablock for the baked data. * It should be possible to convert between freeze and edit geometry. * The Outliner may be a good place to see the checkpoints and their settings. * Having the baking operator (button or menu entry) outside the nodetree lead to a better separation between data (nodes) and operations (the operators: free, cache, ...). * Users can still have a way to quickly focus on the operators related to bake a specific checkpoint (like we do for texture settings, by changing a nearby editor to the appropriate type and subtype/tab).

👍 1

Brecht Van Lommel commented

2023-04-07 19:17:03 +02:00

I agree freezing and caching/baking are very similar, and the node implementation and UI can be almost entirely shared. But I think there should still be some distinction on the UI level.

Freeze/unfreeze to me is an explicit action that a user takes for a specific point in the node graph or modifier stack. Where you say, I'm done editing this part of the node graph, and now I will continue editing beyond this either with more procedural nodes or manual editing.

Caching/baking on the other hand should be mostly automatic and globally controlled. Here I think a users wants to set things up in such a way that they can safely clear the cache/bake and re-bake, or have a render farm do it. Whereas unfreezing all frozen nodes would not be something you can do so casually.

I don't really care if cache/bake is built into simulation nodes somehow, or rather a separate node that is part of default node groups. But I think if it's a separate node it should not be the same as the freeze node even practically if all its settings end up being the same.

I agree freezing and caching/baking are very similar, and the node implementation and UI can be almost entirely shared. But I think there should still be some distinction on the UI level. Freeze/unfreeze to me is an explicit action that a user takes for a specific point in the node graph or modifier stack. Where you say, I'm done editing this part of the node graph, and now I will continue editing beyond this either with more procedural nodes or manual editing. Caching/baking on the other hand should be mostly automatic and globally controlled. Here I think a users wants to set things up in such a way that they can safely clear the cache/bake and re-bake, or have a render farm do it. Whereas unfreezing all frozen nodes would not be something you can do so casually. I don't really care if cache/bake is built into simulation nodes somehow, or rather a separate node that is part of default node groups. But I think if it's a separate node it should not be the same as the freeze node even practically if all its settings end up being the same.

👍 1

Jacques Lucke commented

2023-05-07 12:03:39 +02:00

There is a new proposal that replaces this one.

https://devtalk.blender.org/t/geometry-nodes-checkpoints-proposal/29266

There is a new proposal that replaces this one. https://devtalk.blender.org/t/geometry-nodes-checkpoints-proposal/29266

Jacques Lucke closed this issue

2023-05-07 12:03:40 +02:00

Blender Bot added the

Status

Archived

label 2023-05-07 12:03:42 +02:00