Design: Animation Caching #73481

New Issue

Jeroen Bakker · 2020-01-29T09:32:59+01:00

Jeroen Bakker commented

2020-01-29 09:32:59 +01:00

Status: Initial, not validated

Glossary

Character	Any geometry object that can be animated. Does not have to reflect a character per say

Overview

User Stories

As an animator I want that the cache is updated directly after a change has been made so playing back the animation would in an as high as possible frame-rate.
As an animator I want that the cache only updated the characters I am animating so the result can be seen as fast as possible
As an animator I want the updating of the cache to be done in the background so playing back the animation would be in an as high as possible frame-rate as soon as possible.
As an animator I want that the cache only marks frames invalid that are influenced by the change so the result can be seen as fast as possible
As an animator I want to be able to watch the progress of the background job so I know when I can expect the animation to be cached.
As an animator I want to be able to see the validity of the cache in the timeline so I know what part of the animation is cached.
As an animator I want to be able to playback the animation in the 3d view-port at the frames per seconds of the scene so the animation is smooth and not distracting.
As an animator I want to enable/disable the cache globally so I can perform non-animating (like modelling) task more efficiently
As an animator I want the view-port rendering to use the animation cache so the export is fast.
As an animator I want to be able to clear the animation cache so unfortunate errors can be removed.
As an animator I want to be able to clear the animation cache so computer resources can be used for other tasks.
As an animator I want the cache to be refreshed when I open a shot/sequence file so cache reflects all changes made to linked resources.

Alternatives

The next are workflow related alternatives. Technical alternatives will be described in the Engineering Plan.

Alternative 1: Cache Final Geometry Only

This option will cache the result of the DependencyGraph of any Character.

Benefits

Very Easy to setup.
Can support storing cache to disk per shot.

Cons

Not able to fine-tune memory usage and can become un-usable on huge scenes.

Alternative 2: Use Mesh Sequence Cache Modifier

Blender has a Mesh Sequence Cache Modifier that currently only loads from an alembic cache. We could use this modifier (or a new one). The modifier is then used for reading and writing (animation) caches.
When a cache is valid only the Mesh Sequence Cache modifier and the modifiers below in the modifier stack are being updated. When updating the cache all the modifiers above the Mesh Sequence Cache are executed and the result will then be stored. The idea behind this is that the animation speed could be fine-tuned for memory usage or playback speed by moving the Mesh Sequence Cache modifier to another location in the modifier stack.

Benefits

Animation cache can be optimized for speed or memory usage.
Needs to be setup a single time.

Cons

Is difficult when used for linked in characters the cache is fixed in the original character blend-file.
** Limitation could be reduced by Library Overrides.
Hard to make persistent caches per shot as the CacheFile is set in the original character blend-file.

Alternative 3: Use a Virtual Modifier

To reduce the limitation of Alternative 2: Use Mesh Sequence Cache Modifier we could have a configuration per shot animation file. Inside the configuration an object can be added (armature, mesh, etc) with a setting where the a virtual cache modifier will be added to the modifier list. This cache modifier will not be visible to the user.
Possible options where the virtual modifier could be added could be, this could be set per character:

No Caching: The cache modifier isn't added.
Final Mesh: The cache modifier is added as last modifier in the modifier list.
Before Last Subsurface Modifier: The cache modifier is added just before the last subsurface modifier. If no subsurface modifier is available the cache modifier will be added as last modifier.
After Last Armature Modifier: The cache modifier is added just after the last armature modifier. If no armature modifier is available the cache modifier will be added as last modifier.
...

Note that this list isn't complete and can be completed later on. We could also use 'After Last Deformer Modifier'

Mockup Scene Animation Cache Panel

Mockup Object Animation Cache Panel

This could be setup during animation preparation phase of a shot, but even during animation the animator can change it depending on the character and shot complexity. There is also possibilities to do put this setting in more generic (Simplify, Scene setting), or to be semi-automatically selected by the caching system based on the active selection and other settings.

Benefits

Animator can fine-tune memory usage and speed of the shot
Animator is able to turn off/on the animation cache for a character based on the workflow.
** During blocking the already blocked characters could be cached, but the currently being blocked character could not be cached as (frame range of could give undesired results)
Can support storing cache to disk per shot.

Cons

More detailed control leads to more time to be setup. This time can be minimized by having a specialized operators (Set Animation Cache Profile on all (all animate-able or selected) objects) or optimization profiles (Optimize For Performance / Memory Saving).

Comparison

Detailed description

 This section will be completed after an alternative has been chosen.

Open Questions

Question	Answer
What are the needs for a global enable/disable of the cache? not saying we should not do it, but just so we would implement a solution that matches the needs
What does the user see when the animation cache is enabled and the frame is still invalid? Block animation / Show last cached version of the frame / ...
I can imagine scenarios where you don't want that the animation cache is refreshed or enabled when opening? Should we add a `disable animation cache` in the load file operator

Open Tasks

- Create first version
- Advertise design
- Discuss the alternatives and make a selection of the alternative we will be implementing
- Describe the selected alternative in more detail
- Create an engineering plan

**Status**: `Initial, not validated` ## Glossary | **Character** | Any geometry object that can be animated. Does not have to reflect a character per say | | -- | -- | ## Overview **User Stories** * As an animator I want that the cache is updated directly after a change has been made so playing back the animation would in an as high as possible frame-rate. * As an animator I want that the cache only updated the characters I am animating so the result can be seen as fast as possible * As an animator I want the updating of the cache to be done in the background so playing back the animation would be in an as high as possible frame-rate as soon as possible. * As an animator I want that the cache only marks frames invalid that are influenced by the change so the result can be seen as fast as possible * As an animator I want to be able to watch the progress of the background job so I know when I can expect the animation to be cached. * As an animator I want to be able to see the validity of the cache in the timeline so I know what part of the animation is cached. * As an animator I want to be able to playback the animation in the 3d view-port at the frames per seconds of the scene so the animation is smooth and not distracting. * As an animator I want to enable/disable the cache globally so I can perform non-animating (like modelling) task more efficiently * As an animator I want the view-port rendering to use the animation cache so the export is fast. * As an animator I want to be able to clear the animation cache so unfortunate errors can be removed. * As an animator I want to be able to clear the animation cache so computer resources can be used for other tasks. * As an animator I want the cache to be refreshed when I open a shot/sequence file so cache reflects all changes made to linked resources. ## Alternatives The next are workflow related alternatives. Technical alternatives will be described in the Engineering Plan. **Alternative 1: Cache Final Geometry Only** This option will cache the result of the `DependencyGraph` of any Character. **Benefits** * Very Easy to setup. * Can support storing cache to disk per shot. **Cons** * Not able to fine-tune memory usage and can become un-usable on huge scenes. **Alternative 2: Use Mesh Sequence Cache Modifier** Blender has a Mesh Sequence Cache Modifier that currently only loads from an alembic cache. We could use this modifier (or a new one). The modifier is then used for reading and writing (animation) caches. When a cache is valid only the Mesh Sequence Cache modifier and the modifiers below in the modifier stack are being updated. When updating the cache all the modifiers above the Mesh Sequence Cache are executed and the result will then be stored. The idea behind this is that the animation speed could be fine-tuned for memory usage or playback speed by moving the Mesh Sequence Cache modifier to another location in the modifier stack. ![T73481_modifier_stack.png](https://archive.blender.org/developer/F8310104/T73481_modifier_stack.png) **Benefits** * Animation cache can be optimized for speed or memory usage. * Needs to be setup a single time. **Cons** * Is difficult when used for linked in characters the cache is fixed in the original character blend-file. ** Limitation could be reduced by Library Overrides. * Hard to make persistent caches per shot as the CacheFile is set in the original character blend-file. **Alternative 3: Use a Virtual Modifier** To reduce the limitation of __Alternative 2: Use Mesh Sequence Cache Modifier__ we could have a configuration per shot animation file. Inside the configuration an object can be added (armature, mesh, etc) with a setting where the a virtual cache modifier will be added to the modifier list. This cache modifier will not be visible to the user. Possible options where the virtual modifier could be added could be, this could be set per character: * **No Caching**: The cache modifier isn't added. * **Final Mesh**: The cache modifier is added as last modifier in the modifier list. * **Before Last Subsurface Modifier**: The cache modifier is added just before the last subsurface modifier. If no subsurface modifier is available the cache modifier will be added as last modifier. * **After Last Armature Modifier**: The cache modifier is added just after the last armature modifier. If no armature modifier is available the cache modifier will be added as last modifier. * ... ![T73481_virtual_modifier_stack.png](https://archive.blender.org/developer/F8310106/T73481_virtual_modifier_stack.png) ```Note that this list isn't complete and can be completed later on. We could also use 'After Last Deformer Modifier'``` **Mockup Scene Animation Cache Panel** ![image.png](https://archive.blender.org/developer/F8310419/image.png) **Mockup Object Animation Cache Panel** ![image.png](https://archive.blender.org/developer/F8310686/image.png) This could be setup during animation preparation phase of a shot, but even during animation the animator can change it depending on the character and shot complexity. There is also possibilities to do put this setting in more generic (Simplify, Scene setting), or to be semi-automatically selected by the caching system based on the active selection and other settings. **Benefits** * Animator can fine-tune memory usage and speed of the shot * Animator is able to turn off/on the animation cache for a character based on the workflow. ** During blocking the already blocked characters could be cached, but the currently being blocked character could not be cached as (frame range of could give undesired results) * Can support storing cache to disk per shot. **Cons** * More detailed control leads to more time to be setup. This time can be minimized by having a specialized operators (Set Animation Cache Profile on all (all animate-able or selected) objects) or optimization profiles (Optimize For Performance / Memory Saving). **Comparison** ## Detailed description ``` This section will be completed after an alternative has been chosen. ``` ## Open Questions | **Question** | **Answer** | | -- | -- | | What are the needs for a global enable/disable of the cache? not saying we should not do it, but just so we would implement a solution that matches the needs | | | What does the user see when the animation cache is enabled and the frame is still invalid? Block animation / Show last cached version of the frame / ... | | | I can imagine scenarios where you don't want that the animation cache is refreshed or enabled when opening? Should we add a `disable animation cache` in the load file operator | | ## Open Tasks * - [ ] Create first version * - [ ] Advertise design * - [ ] Discuss the alternatives and make a selection of the alternative we will be implementing * - [ ] Describe the selected alternative in more detail * - [ ] Create an engineering plan

Jeroen Bakker commented

2020-01-29 09:32:59 +01:00

Changed status from 'Needs Triage' to: 'Confirmed'

Jeroen Bakker commented

2020-01-29 09:32:59 +01:00

Added subscribers: @Jeroen-Bakker, @AditiaA.Pratama, @Scaredyfish, @JorgeBernalMartinez, @LucianoMunoz, @RedMser, @StanislavOvcharov, @Sergey, @Hjalti, @Mets, @brecht, @MarcelLegindi, @sozap, @item412, @dgsantana, @WilliamReynish, @Phigon, @dfelinto

Jeroen Bakker changed title from ~~[WIP] Design: Animation Caching~~ to Design: Animation Caching

2020-01-29 14:43:40 +01:00

William Reynish commented

2020-01-30 00:26:18 +01:00

This seems good, although as you point out, the less users have to worry about setting it up, the better. If this requires users to do lots of extra work, they are much less likely to use it, and won’t benefit from the faster playback.

Preferably this would happen similar to how modern NLEs work when they render clips in the background, without users having to really do anything at all. The result is smooth playback.

I would hope we can make this system equally smart so it just knows to cache things when needed, and perform the caching in the background without interrupting the user.

This seems good, although as you point out, the less users have to worry about setting it up, the better. If this requires users to do lots of extra work, they are much less likely to use it, and won’t benefit from the faster playback. Preferably this would happen similar to how modern NLEs work when they render clips in the background, without users having to really do anything at all. The result is smooth playback. I would hope we can make this system equally smart so it just knows to cache things when needed, and perform the caching in the background without interrupting the user.

Stanislav Ovcharov commented

2020-02-01 20:31:22 +01:00

I totally agree with @LucianoMunoz in https://developer.blender.org/T68908
and especially:

Caching feel seamless and run in the background like caching video on a video editor (not vse XP) while editing.
Also whatever thing is visible should be cached, if there is hair, cloth, it should all be able to be part of this cache.

Everything visible should be cached into RAM, not geometry only, whole scene. Without modifiers and without writing to disk

I totally agree with @LucianoMunoz in https://developer.blender.org/T68908 and especially: > Caching feel seamless and run in the background like caching video on a video editor (not vse XP) while editing. > Also whatever thing is visible should be cached, if there is hair, cloth, it should all be able to be part of this cache. > Everything visible should be cached into RAM, not geometry only, whole scene. Without modifiers and without writing to disk

Sam Green commented

2020-02-25 11:02:10 +01:00

Added subscriber: @SamGreen

Brecht Van Lommel commented

2020-03-27 15:56:57 +01:00

I think "Alternative 1" is the right solution. Modifiers only handle the mesh case and while it's the most important one, we can have something more generic and automatic.

We can cache entire datablocks at the dependency graph level, and then for memory usage look at how we can keep that as low as possible. The basic thing is that unchanged datablocks that have not been reevaluated by the dependency graph should be shared of course.

In typical scene setups for animators, I think the main memory usage would be meshes and hair. Strategies to reduce memory usage are:

Custom data layer mem compare or modified tagging to share data layers.
Implementation OpenSubdiv GPU acceleration, so we can use low poly meshes.
For hair, implement GPU accelerated hair deform.

These are all about improving animation playback itself too, anything that we can avoid re-evaluating on the CPU we don't need to cache for every frame either. In a more general mechanism, you can imagine a per-datablock diffing function that can compare the datablock at a reference frame and the cached frame, and implement that for datablocks where we think we can gain significant memory.

Disk caching I would not consider important for an initial implementation.

The dependency graph already manages ownership of evaluated datablocks, and adding caching + restoring to that seems doable to me. We may need some special handling for GPU buffers, ideally a fast update path if we know only coordinates have changed. But again this is something we'd like anyway to improve playback performance itself.

I think "Alternative 1" is the right solution. Modifiers only handle the mesh case and while it's the most important one, we can have something more generic and automatic. We can cache entire datablocks at the dependency graph level, and then for memory usage look at how we can keep that as low as possible. The basic thing is that unchanged datablocks that have not been reevaluated by the dependency graph should be shared of course. In typical scene setups for animators, I think the main memory usage would be meshes and hair. Strategies to reduce memory usage are: * Custom data layer mem compare or modified tagging to share data layers. * Implementation OpenSubdiv GPU acceleration, so we can use low poly meshes. * For hair, implement GPU accelerated hair deform. These are all about improving animation playback itself too, anything that we can avoid re-evaluating on the CPU we don't need to cache for every frame either. In a more general mechanism, you can imagine a per-datablock diffing function that can compare the datablock at a reference frame and the cached frame, and implement that for datablocks where we think we can gain significant memory. Disk caching I would not consider important for an initial implementation. The dependency graph already manages ownership of evaluated datablocks, and adding caching + restoring to that seems doable to me. We may need some special handling for GPU buffers, ideally a fast update path if we know only coordinates have changed. But again this is something we'd like anyway to improve playback performance itself.

hujin commented

2020-10-16 18:07:16 +02:00

Added subscriber: @2046411367