Geometry Nodes Baking #110222

New Issue

Jacques Lucke · 2023-07-17T21:18:50+02:00

Jacques Lucke commented

2023-07-17 21:18:50 +02:00

As mentioned on the blog the goal is to support the following features:

Editing procedurally generated geometry destructively in the viewport.
Prepare and manage simulations for the render farm which shouldn't need to run the simulation itself.
Explicitly cache parts of the node tree to avoid recomputing them after every change.
Re-time animated geometries, i.e. access geometry generated on one frame in another frame.

At a high level these goals are achieved by allowing the user to bake still or animated geometry at arbitrary positions in a geometry nodes tree using a new Bake node. The baked data can then be used without having to recompute the inputs of the Bake node. It can be edited destructively using Blender's existing edit modes. Furthermore, the baked data can be imported at specific frames.

Implementing all of these features in one go would be too big of an undertaking, so below are smaller releasable milestones that can be worked on one after another.

Milestone 1: Bake Node

After this milestone users will be able to bake still or animated geometries at arbitrary positions in the node tree. The main use cases are:

Prepare simulations (with optional post-processing) for a renderfarm.
Speedup working with very heavy node trees (e.g. static landscape generation) by explicitly caching the generated data.

There are a few requirements for this milestone:

New Bake node with a dynamic number of sockets for geometries and their attributes.
Baking works by pressing on a bake button in the node itself.
- For baking the user has to choose between still or animated geometry and optionally a frame range.
- A bake directory on disk has to be choosen.
Deleting the baked data should be possible from the node as well.
Potential settings and the baked data are stored per Bake node instance. A Bake node that is in a reused node group is counted as multiple Bake node instances. The actual data per Bake is stored in the geometry nodes modifier on the object.

It may be enough to keep UI changes local to the new Bake node.

Milestone 2: Manage Bakes

In production files, there are typically many simulations and other heavy effects that require baking. The goal of this milestone is to give the user tools to manage many bakes.

There are multiple parts to this which could potentially be split up into somewhat independent tasks, but I keep them together until further discussion happened:

Support naming Bake node instances.
- While working with just Bake nodes in the node tree, this is less important, because the location of the Bake node in the tree "identifies" it to the use. When dealing with many Bake node instances at once, one can't rely on their location but needs to use names.
- Just using node names does not work well for this, because a node only has one name but might be reused many times.
Support tagging Bake node instances.
- As we've seen in the intro, not all baking happens for the same reasons. Therefore it is important that tools that work with multiple bakes can filter what bakes to work on.
- Supporting custom tags can help the user organize baked data.
UI for dealing with all bakes in a scene or file.
- There should be some kind of UI list that shows all the Bake node instances.
- It should show which data is currently baked and which not.
- Filtering can happen using tags and object selection.
- Various new operators can work on multiple bakes at the same time: bake, delete bake, set bake directory.
Python API for dealing with baked data.
- While the UI we provide for managing baked data should be good enough for most use cases, it seems unrealistic that we can provide all the tools that a larger production needs out of the box. Therefore, it is necessary to make it easy to manage bakes from addons as well.

Milestone 3: Import Bake

The Bake node does two things implicitly, it indicates where in the node group the baking happens and it also imports the baked data. Having both of these in the same node is desirable for most use cases, but sometimes it's benefitial to split both tasks. For example:

Baking an animated geometry and the loading it at different frames for re-timing.
Load multiple baked frames of the same Bake within the same frame.
Baking a geometry in one .blend file and then loading it as a cache in another.

All of these use-cases turn our baking format into a more general purpose interchange format like Alembic that is Blender specific. Baking into other formats will be possible eventually as well, but it not part of this project. Using a custom format allows us to optimize it for our needs and avoids the problem that Blenders data does not always map 1 to 1 to how data is stored in common formats. Also see this task.

The import can be done with a new Import Bake node which takes a (file) path and a frame as input and outputs the geometry. There are some problems that need to be overcome:

The Import Bake node ideally has the same sockets as the corresponding Bake node, but this can't be guaranteed when they are in separate .blend files.
While not supported yet, it will be possible to pack baked data into the .blend file. Therefore, the path input is sometimes a path on disk and sometimes some internal identifier.
It should be possible to use the combination of a Bake node with one or more Import Bake nodes within a node group without having to hard code the file path for all of these nodes. Therefore, it seems reasonable that the Bake node also outputs where the baked data is stored. Also see this older proposal.

Milestone 4: Editing Baked Data

The idea is that the user can just go into edit, sculpt or other modes on the baked data and edit it in place. For animated geometries, every frame can be destructively edited separately. The goal here is not to record all manual edits the user did. After baking again, the edits will generally be gone.

While this might seem like a small feature, if done right, it can have huge impacts on the way people work with geometry in Blender in the long-term. That is because it could theoretically allow us to get rid of the idea of "original geometry" in Blender (e.g. the mesh referenced by object->data). To give a glimpse into what could become possible: Adding a cube object could just add an "empty" object with Cube node in geometry nodes. This allows the primitive mesh to remain procedural. Going into edit mode could implicitly (or explicitly) add a Bake node and bake the cube so that it can be edited destructively.

As for this milestone, the target use cases are more minor and mostly revolve around fixing up "bad" data in animated geometry. For example, in a particle simulation, a single particle got too close to the camera and blocks the entire view. It should be possible to just delete it manually.

Parallel Track 1: UI for Unused Nodes

There are various situations in which nodes in a node group are unused:

Nodes not connected to the output.
Nodes that would compute the unused input of a Switch node.
Nodes that were used to compute a simulation that is now cached.
Nodes that compute the inputs to the Bake node which is now baked.

Currently, there is no obvious way for the user to determine which nodes have been evaluated by the last evaluation. However, this information would be very useful because it allows seeing at a glance when data is cached/baked.

Logging the set of nodes that has been evaluated is relatively straight forward. However, we still need the a good UI for these nodes. The obvious solution is to just gray out these nodes. While that can work, it's not ideal because the nodes can and should still be editable and graying out usually indicates that the user is not expected to edit them.

Parallel Track 2: Store Baked Data in .blend File

We already support packing external files into .blend files. That system just needs to be enhanced to also support baking entire folders full with baked data. Ideally, it is possible to load the baked data lazily when loading a .blend file, but that's probably quite a bit more challenging. It remains to be seen how much of a performance hit loading all frames at once has.

As mentioned on the [blog](https://code.blender.org/2023/06/node-tools-interface-and-baking/) the goal is to support the following features: * Editing procedurally generated geometry destructively in the viewport. * Prepare and manage simulations for the render farm which shouldn't need to run the simulation itself. * Explicitly cache parts of the node tree to avoid recomputing them after every change. * Re-time animated geometries, i.e. access geometry generated on one frame in another frame. At a high level these goals are achieved by allowing the user to bake still or animated geometry at arbitrary positions in a geometry nodes tree using a new Bake node. The baked data can then be used without having to recompute the inputs of the Bake node. It can be edited destructively using Blender's existing edit modes. Furthermore, the baked data can be imported at specific frames. Implementing all of these features in one go would be too big of an undertaking, so below are smaller releasable milestones that can be worked on one after another. ## Milestone 1: Bake Node After this milestone users will be able to bake still or animated geometries at arbitrary positions in the node tree. The main use cases are: * Prepare simulations (with optional post-processing) for a renderfarm. * Speedup working with very heavy node trees (e.g. static landscape generation) by explicitly caching the generated data. There are a few requirements for this milestone: * New Bake node with a dynamic number of sockets for geometries and their attributes. * Baking works by pressing on a bake button in the node itself. * For baking the user has to choose between still or animated geometry and optionally a frame range. * A bake directory on disk has to be choosen. * Deleting the baked data should be possible from the node as well. * Potential settings and the baked data are stored per Bake node instance. A Bake node that is in a reused node group is counted as multiple Bake node instances. The actual data per Bake is stored in the geometry nodes modifier on the object. It may be enough to keep UI changes local to the new Bake node. ## Milestone 2: Manage Bakes In production files, there are typically many simulations and other heavy effects that require baking. The goal of this milestone is to give the user tools to manage many bakes. There are multiple parts to this which could potentially be split up into somewhat independent tasks, but I keep them together until further discussion happened: * Support naming Bake node instances. * While working with just Bake nodes in the node tree, this is less important, because the location of the Bake node in the tree "identifies" it to the use. When dealing with many Bake node instances at once, one can't rely on their location but needs to use names. * Just using node names does not work well for this, because a node only has one name but might be reused many times. * Support tagging Bake node instances. * As we've seen in the intro, not all baking happens for the same reasons. Therefore it is important that tools that work with multiple bakes can filter what bakes to work on. * Supporting custom tags can help the user organize baked data. * UI for dealing with all bakes in a scene or file. * There should be some kind of UI list that shows all the Bake node instances. * It should show which data is currently baked and which not. * Filtering can happen using tags and object selection. * Various new operators can work on multiple bakes at the same time: bake, delete bake, set bake directory. * Python API for dealing with baked data. * While the UI we provide for managing baked data should be good enough for most use cases, it seems unrealistic that we can provide all the tools that a larger production needs out of the box. Therefore, it is necessary to make it easy to manage bakes from addons as well. ## Milestone 3: Import Bake The Bake node does two things implicitly, it indicates where in the node group the baking happens and it also imports the baked data. Having both of these in the same node is desirable for most use cases, but sometimes it's benefitial to split both tasks. For example: * Baking an animated geometry and the loading it at different frames for re-timing. * Load multiple baked frames of the same Bake within the same frame. * Baking a geometry in one .blend file and then loading it as a cache in another. All of these use-cases turn our baking format into a more general purpose interchange format like Alembic that is Blender specific. Baking into other formats will be possible eventually as well, but it not part of this project. Using a custom format allows us to optimize it for our needs and avoids the problem that Blenders data does not always map 1 to 1 to how data is stored in common formats. Also see [this](https://projects.blender.org/blender/blender/issues/105251) task. The import can be done with a new Import Bake node which takes a (file) path and a frame as input and outputs the geometry. There are some problems that need to be overcome: * The Import Bake node ideally has the same sockets as the corresponding Bake node, but this can't be guaranteed when they are in separate .blend files. * While not supported yet, it will be possible to pack baked data into the .blend file. Therefore, the path input is sometimes a path on disk and sometimes some internal identifier. * It should be possible to use the combination of a Bake node with one or more Import Bake nodes within a node group without having to hard code the file path for all of these nodes. Therefore, it seems reasonable that the Bake node also outputs where the baked data is stored. Also see [this](https://devtalk.blender.org/t/geometry-nodes-checkpoints-proposal/29266) older proposal. ## Milestone 4: Editing Baked Data The idea is that the user can just go into edit, sculpt or other modes on the baked data and edit it in place. For animated geometries, every frame can be destructively edited separately. The goal here is not to record all manual edits the user did. After baking again, the edits will generally be gone. While this might seem like a small feature, if done right, it can have huge impacts on the way people work with geometry in Blender in the long-term. That is because it could theoretically allow us to get rid of the idea of "original geometry" in Blender (e.g. the mesh referenced by `object->data`). To give a glimpse into what could become possible: Adding a cube object could just add an "empty" object with Cube node in geometry nodes. This allows the primitive mesh to remain procedural. Going into edit mode could implicitly (or explicitly) add a Bake node and bake the cube so that it can be edited destructively. As for this milestone, the target use cases are more minor and mostly revolve around fixing up "bad" data in animated geometry. For example, in a particle simulation, a single particle got too close to the camera and blocks the entire view. It should be possible to just delete it manually. ## Parallel Track 1: UI for Unused Nodes There are various situations in which nodes in a node group are unused: * Nodes not connected to the output. * Nodes that would compute the unused input of a Switch node. * Nodes that were used to compute a simulation that is now cached. * Nodes that compute the inputs to the Bake node which is now baked. Currently, there is no obvious way for the user to determine which nodes have been evaluated by the last evaluation. However, this information would be very useful because it allows seeing at a glance when data is cached/baked. Logging the set of nodes that has been evaluated is relatively [straight forward](https://projects.blender.org/blender/blender/pulls/107780). However, we still need the a good UI for these nodes. The obvious solution is to just gray out these nodes. While that can work, it's not ideal because the nodes can and should still be editable and graying out usually indicates that the user is not expected to edit them. ## Parallel Track 2: Store Baked Data in .blend File We already support packing external files into .blend files. That system just needs to be enhanced to also support baking entire folders full with baked data. Ideally, it is possible to load the baked data lazily when loading a .blend file, but that's probably quite a bit more challenging. It remains to be seen how much of a performance hit loading all frames at once has.

🚀 2

Jacques Lucke added the

Type

Design

label 2023-07-17 21:18:50 +02:00

Jacques Lucke added this to the Nodes & Physics project 2023-07-17 21:19:17 +02:00

Lukas Tönne commented

2023-07-18 10:58:41 +02:00

I like it, it's a reasonable set of milestones. I guess the editing part has the most unknowns still.

The Import Bake node ideally has the same sockets as the corresponding Bake node, but this can't be guaranteed when they are in separate .blend files.

I would expect a bake to contain metadata about the contained attributes and that would be used to create appropriate sockets. Downside is that baking with a different attribute set can cause changes to import nodes on file load, similar to how changing a node group adds and removes(!) sockets on validation. An alternative might be to detect mismatch of bake metadata and sockets and show a warning, but leave it to users to fix outdated import nodes (keep dead sockets until links are removed or reconnected).

I like it, it's a reasonable set of milestones. I guess the editing part has the most unknowns still. > The Import Bake node ideally has the same sockets as the corresponding Bake node, but this can't be guaranteed when they are in separate .blend files. I would expect a bake to contain metadata about the contained attributes and that would be used to create appropriate sockets. Downside is that baking with a different attribute set can cause changes to import nodes on file load, similar to how changing a node group adds and removes(!) sockets on validation. An alternative might be to detect mismatch of bake metadata and sockets and show a warning, but leave it to users to fix outdated import nodes (keep dead sockets until links are removed or reconnected).

Jacques Lucke commented

2023-07-18 11:01:49 +02:00

I would expect a bake to contain metadata about the contained attributes and that would be used to create appropriate sockets. Downside is that baking with a different attribute set can cause changes to import nodes on file load, similar to how changing a node group adds and removes(!) sockets on validation. An alternative might be to detect mismatch of bake metadata and sockets and show a warning, but leave it to users to fix outdated import nodes (keep dead sockets until links are removed or reconnected).

Yeah, I tend towards showing a warning with a fix button as well. Updating sockets automatically would be way more tricky when the same Import Bake node is used with different bakes (e.g. where the file path is passed as input into the node group).

> I would expect a bake to contain metadata about the contained attributes and that would be used to create appropriate sockets. Downside is that baking with a different attribute set can cause changes to import nodes on file load, similar to how changing a node group adds and removes(!) sockets on validation. An alternative might be to detect mismatch of bake metadata and sockets and show a warning, but leave it to users to fix outdated import nodes (keep dead sockets until links are removed or reconnected). Yeah, I tend towards showing a warning with a fix button as well. Updating sockets automatically would be way more tricky when the same Import Bake node is used with different bakes (e.g. where the file path is passed as input into the node group).

Lukas Tönne commented

2023-07-18 11:13:51 +02:00

Updating sockets automatically would be way more tricky when the same Import Bake node is used with different bakes (e.g. where the file path is passed as input into the node group).

Tangentially related: This problem is also one of the weak points in the "function sockets" experiment. When the actual evaluated graph is only known at evaluation time there is no way to automatically generate sockets beforehand. My idea (not yet implemented) was to use a "template node group" to define the sockets and then find the best match with the actual graph at eval time (similar to an abstract virtual function declaration). That won't work here but i thought i'd mention it.

> Updating sockets automatically would be way more tricky when the same Import Bake node is used with different bakes (e.g. where the file path is passed as input into the node group). Tangentially related: This problem is also one of the weak points in the ["function sockets" experiment](https://projects.blender.org/blender/blender/pulls/107842). When the actual evaluated graph is only known at evaluation time there is no way to automatically generate sockets beforehand. My idea (not yet implemented) was to use a "template node group" to define the sockets and then find the best match with the actual graph at eval time (similar to an abstract virtual function declaration). That won't work here but i thought i'd mention it.

Michal-Podziomek commented

2024-02-06 22:59:34 +01:00

Relevant to #86839 - Mesh Cache modifier and Mesh Sequence Cache modifier.

Assuming the baked cache contains full vertex data (color attrib, weights, other), this results in a feature superset of Alembic and Lightwave point cache.

There's also potential for a better overall integration with different pipelines:

Current cache modifiers miss features, effectively limiting them to pre-baked animation playback and editing.

Being able to use caches with geonodes could remove these limitations. Two that are pretty great and that I've tested preliminarily:

better shapekeys with custom, complex, non-linear interpolations, and ability to 'key' other data - like weights and vertex colors.
sculpt layers with good performance for up to ~25M verts per object - not achievable with Multires or shape keys.

Relevant to #86839 - Mesh Cache modifier and Mesh Sequence Cache modifier. Assuming the baked cache contains full vertex data (color attrib, weights, other), this results in a feature superset of Alembic and Lightwave point cache. There's also potential for a better overall integration with different pipelines: Current cache modifiers miss features, effectively limiting them to pre-baked animation playback and editing. Being able to use caches with geonodes could remove these limitations. Two that are pretty great and that I've tested preliminarily: - better shapekeys with custom, complex, non-linear interpolations, and ability to 'key' other data - like weights and vertex colors. - sculpt layers with good performance for up to ~25M verts per object - not achievable with Multires or shape keys.

Sign in to join this conversation.

No Label

Download

What's New

Blender Studio

Manual

Developers Blog

Documentation

Benchmark

Blender Conference

Development Fund

One-time Donations

Geometry Nodes Baking #110222