Simulation nodes: Automatic cache reset behavior #105724

Closed
opened 2023-03-13 15:15:19 +01:00 by Hans Goudey · 1 comment
Member

Terminology

To make discussion of inputs and outputs easier, the following naming convention is be used:

  • Input sockets of the “Simulation Input” node are simulation inputs
  • Output sockets of the “Simulation Input” node are iteration inputs
  • Input sockets of the “Simulation Output” node are iteration outputs
  • Output sockets of the “Simulation Output” node are simulation outputs
    image

Stepping

Some node evaluations don’t involve a time step (e.g. loading a file). In those cases the simulation inputs are used unmodified as simulation outputs.
image

If an iteration is computed during node evaluation then the simulation outputs are defined as follows:

  1. When cached data is available for the exact current frame the outputs simply expose the cached data.
    Simulation inputs are ignored.
    image

  2. When cached data for the current frame is not available, but exists for an earlier frame, an iteration is performed with the necessary time step.
    Simulation inputs are ignored.
    Iteration inputs are cached data.
    image

  3. When previous data is unavailable or invalid, then the current simulation inputs are used and piped straight to iteration inputs.
    This is only correct for the start frame (to keep results deterministic), so on other frames a warning should be shown if no cached input is available.
    image

Invalidating Caches

A cache becomes invalid if the scene data is changed such that the output for the current frame will be different. This could be because

  1. The simulation input on the start frame is different.
  2. Scene data used during the iteration (e.g. an Object Info node) changed.

In practice it will be difficult to determine precisely when changes do or do not affect the simulation. False positives have to be expected, that is: invalidating the cache when it wouldn’t have to be. This is because we have to rely on coarse depsgraph dependencies. It is preferable to invalidate the cache unnecessarily than to keep cached results and miss potential changes. Over time the dependencies affecting nodes may be refined for more stable caching.

Reset

The term "reset" describes the point when a simulation reverts to the initial scene data, typically when animation jumps back to the start frame (Point 3 in Stepping).

Conditions that can cause a reset (only when cache is invalid!)

  • Jump to start frame
  • Jump to other frame, with a warning
  • Start frame is changed (might invalidate cache)

A reset should not happen just because the cache got invalidated. For example: A particle simulation might use an object as emitter. The user moves this object manually during simulation, which constantly invalidates the cache. However, the particle simulation should not reset: all the current particles remain in the simulation.

The last valid simulation state is always retained even if the rest of the cache is invalidated. That way simulations can be tested dynamically.

Baking

Baked caches are never invalidated automatically.

If a baked cache exists then the simulation input should never be used and no further iterations should happen. The simulation output is always read from the cache.

A baked cache must be freed explicitly, at which point it reverts back to a dynamic cache.

Notes

  • “Memory cache” vs “disk cache” distinction should not be relevant to most of the examples below.
  • Undo is effectively a file load operation too.
# Terminology To make discussion of inputs and outputs easier, the following naming convention is be used: * Input sockets of the “Simulation Input” node are *simulation inputs* * Output sockets of the “Simulation Input” node are *iteration inputs* * Input sockets of the “Simulation Output” node are *iteration outputs* * Output sockets of the “Simulation Output” node are *simulation outputs* ![image](/attachments/5d53bb74-1004-4b35-a48c-643583e488dc) # Stepping Some node evaluations don’t involve a time step (e.g. loading a file). In those cases the simulation inputs are used unmodified as simulation outputs. ![image](/attachments/b573bc70-777b-465d-a724-d80687c98897) If an iteration is computed during node evaluation then the simulation outputs are defined as follows: 1. When cached data is available for the exact current frame the outputs simply expose the cached data. Simulation inputs are ignored. ![image](/attachments/cfcde9a3-5793-41b8-8337-82508f203bc5) 2. When cached data for the current frame is not available, but exists for an earlier frame, an iteration is performed with the necessary time step. Simulation inputs are ignored. Iteration inputs are cached data. ![image](/attachments/ac84e6a7-148a-46be-9f21-967c3984bc35) 3. When previous data is unavailable or invalid, then the current simulation inputs are used and piped straight to iteration inputs. This is only correct for the start frame (to keep results deterministic), so on other frames a warning should be shown if no cached input is available. ![image](/attachments/0f1c503d-d68f-4ebc-81a4-d7b1e0b972a2) # Invalidating Caches A cache becomes invalid if the scene data is changed such that the output for the current frame will be different. This could be because 1. The simulation input on the start frame is different. 2. Scene data used during the iteration (e.g. an Object Info node) changed. In practice it will be difficult to determine precisely when changes do or do not affect the simulation. False positives have to be expected, that is: invalidating the cache when it wouldn’t have to be. This is because we have to rely on coarse depsgraph dependencies. It is preferable to invalidate the cache unnecessarily than to keep cached results and miss potential changes. Over time the dependencies affecting nodes may be refined for more stable caching. ## Reset The term "reset" describes the point when a simulation reverts to the initial scene data, typically when animation jumps back to the start frame (Point 3 in [Stepping](#stepping)). Conditions that can cause a reset (only when cache is invalid!) * Jump to start frame * Jump to other frame, with a warning * Start frame is changed (might invalidate cache) A reset should not happen just because the cache got invalidated. For example: A particle simulation might use an object as emitter. The user moves this object manually during simulation, which constantly invalidates the cache. However, the particle simulation should not *reset*: all the current particles remain in the simulation. The last valid simulation state is always retained even if the rest of the cache is invalidated. That way simulations can be tested dynamically. # Baking Baked caches are never invalidated automatically. If a baked cache exists then the simulation input should never be used and no further iterations should happen. The simulation output is always read from the cache. A baked cache must be freed explicitly, at which point it reverts back to a dynamic cache. # Notes * “Memory cache” vs “disk cache” distinction should not be relevant to most of the examples below. * Undo is effectively a file load operation too.
Hans Goudey added the
Type
Design
Interest
Geometry Nodes
labels 2023-03-13 15:15:20 +01:00
Hans Goudey added this to the Nodes & Physics project 2023-03-13 15:15:20 +01:00
Lukas Tönne self-assigned this 2023-03-13 15:52:26 +01:00
Member

Updated Hans' initial comment.

Updated Hans' initial comment.
Blender Bot added the
Status
Archived
label 2023-10-23 12:42:24 +02:00
Sign in to join this conversation.
No Label
Interest
Alembic
Interest
Animation & Rigging
Interest
Asset Browser
Interest
Asset Browser Project Overview
Interest
Audio
Interest
Automated Testing
Interest
Blender Asset Bundle
Interest
BlendFile
Interest
Collada
Interest
Compatibility
Interest
Compositing
Interest
Core
Interest
Cycles
Interest
Dependency Graph
Interest
Development Management
Interest
EEVEE
Interest
EEVEE & Viewport
Interest
Freestyle
Interest
Geometry Nodes
Interest
Grease Pencil
Interest
ID Management
Interest
Images & Movies
Interest
Import Export
Interest
Line Art
Interest
Masking
Interest
Metal
Interest
Modeling
Interest
Modifiers
Interest
Motion Tracking
Interest
Nodes & Physics
Interest
OpenGL
Interest
Overlay
Interest
Overrides
Interest
Performance
Interest
Physics
Interest
Pipeline, Assets & IO
Interest
Platforms, Builds & Tests
Interest
Python API
Interest
Render & Cycles
Interest
Render Pipeline
Interest
Sculpt, Paint & Texture
Interest
Text Editor
Interest
Translations
Interest
Triaging
Interest
Undo
Interest
USD
Interest
User Interface
Interest
UV Editing
Interest
VFX & Video
Interest
Video Sequencer
Interest
Virtual Reality
Interest
Vulkan
Interest
Wayland
Interest
Workbench
Interest: X11
Legacy
Blender 2.8 Project
Legacy
Milestone 1: Basic, Local Asset Browser
Legacy
OpenGL Error
Meta
Good First Issue
Meta
Papercut
Meta
Retrospective
Meta
Security
Module
Animation & Rigging
Module
Core
Module
Development Management
Module
EEVEE & Viewport
Module
Grease Pencil
Module
Modeling
Module
Nodes & Physics
Module
Pipeline, Assets & IO
Module
Platforms, Builds & Tests
Module
Python API
Module
Render & Cycles
Module
Sculpt, Paint & Texture
Module
Triaging
Module
User Interface
Module
VFX & Video
Platform
FreeBSD
Platform
Linux
Platform
macOS
Platform
Windows
Priority
High
Priority
Low
Priority
Normal
Priority
Unbreak Now!
Status
Archived
Status
Confirmed
Status
Duplicate
Status
Needs Info from Developers
Status
Needs Information from User
Status
Needs Triage
Status
Resolved
Type
Bug
Type
Design
Type
Known Issue
Type
Patch
Type
Report
Type
To Do
No Milestone
No Assignees
2 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: blender/blender#105724
No description provided.