Hair Dynamics: cannot generate a valid particle cache #62563

New Issue

Garry R. Osgood · 2019-03-14T02:28:33+01:00

Garry R. Osgood commented

2019-03-14 02:28:33 +01:00

System Information
Operating system: 4.20.12-gentoo Linux
Graphics card: Radeon(TM) Pro WX9100 (VEGA10, DRM 3.27.0, 4.20.12-gentoo, LLVM 6.0.1)

Blender Version
Broken: 2.80 (sub 48) commit hash: bf9904ec80 Build: 2019-03-13 01:14:16 Linux Release branch: blender2.7

Worked: (optional) 2.79b commit hash and date unknown; compiled from Gentoo repository

Short description of error
Fails to make a valid particle cache when rendering a dynamic hair system via Cntl + F12. Hair particle motion appears compromised after the first frame. see video clip dyn_on.mkv.

Require
A .blend file with a mesh object that has a hair particle system defined on it. Hair dynamics is turned on in the particle system properties panel. The file should have been previously saved without a cache. Blend file dyn_on.blend provided for this requirement.

Exact steps for others to reproduce the error

Open dyn_on.blend
Cntl + F12 to render animation (implicitly build in-memory cache)
Observe:

 a. Cache indicator on timeline shows only 1 or 2 valid initial frames; remaining 248 - 249 frames are invalid.
 b. Hair particles exhibit only limited rotational dynamics. They do not follow the surface of the sphere (i.e., they appear "unanchored"), nor do they seem affected by gravity. See **dyn_on.mkv** video clip.
 c. After the last frame has been rendered, click on the rewind control in the timeline. The cache section in the Particles properties reports the state of the cache: "1 frames in memory (216 KB), not exact since frame 0"

Workaround

Bake particle dynamics first or
Run the animation through in the 3D viewport to create an in-memory or on disk cache before rendering.

Rendering with an in-memory, on-disk, or with baked dynamics appears to generate an unaffected animation. See dyn_on_cache.mkv video clip.

Notes:

Examples rendered in Cycles, but EEVEE and Workbench rendering does not change behavior.
Hair particles rendered with a collection of simple mesh geometry in the examples, but rendering with single mesh objects or rendering hair paths does not change behavior.
Running in single thread mode (-t 1 on command line) does not change behavior.
Rendering via CPU or GPU does not change behavior.
Problem seems specific to hair particle systems. Emitter particle systems without pre-computed caches can be rendered with Cntl + F12; the cache seems fully valid in the time line, dynamics of falling emitter particles do not seem unusual and the cache section in the Particles properties reports that there are 250 valid frames in memory at the conclusion of the animation.

Related:
#62439 describes a failure in particle updates which seems very similar in appearance exhibited by particles in video clip dyn_on.mkv.

dyn_on_cache.mkv - dyn_on_cache.mkv
dyn_on.mkv - dyn_on.mkv
dyn_on.blend - dyn_on.blend

**System Information** Operating system: 4.20.12-gentoo Linux Graphics card: Radeon(TM) Pro WX9100 (VEGA10, DRM 3.27.0, 4.20.12-gentoo, LLVM 6.0.1) **Blender Version** Broken: 2.80 (sub 48) commit hash: bf9904ec8018 Build: 2019-03-13 01:14:16 Linux Release branch: blender2.7 Worked: (optional) 2.79b commit hash and date unknown; compiled from Gentoo repository **Short description of error** Fails to make a valid particle cache when rendering a dynamic hair system via Cntl + F12. Hair particle motion appears compromised after the first frame. see video clip **dyn_on.mkv**. **Require** A .blend file with a mesh object that has a hair particle system defined on it. Hair dynamics is turned on in the particle system properties panel. The file should have been previously saved without a cache. Blend file **dyn_on.blend** provided for this requirement. **Exact steps for others to reproduce the error** 1. Open **dyn_on.blend** 2. Cntl + F12 to render animation (implicitly build in-memory cache) 3. Observe: ``` a. Cache indicator on timeline shows only 1 or 2 valid initial frames; remaining 248 - 249 frames are invalid. b. Hair particles exhibit only limited rotational dynamics. They do not follow the surface of the sphere (i.e., they appear "unanchored"), nor do they seem affected by gravity. See **dyn_on.mkv** video clip. c. After the last frame has been rendered, click on the rewind control in the timeline. The cache section in the Particles properties reports the state of the cache: "1 frames in memory (216 KB), not exact since frame 0" ``` **Workaround** 1. Bake particle dynamics first or 2. Run the animation through in the 3D viewport to create an in-memory or on disk cache before rendering. Rendering with an in-memory, on-disk, or with baked dynamics appears to generate an unaffected animation. See **dyn_on_cache.mkv** video clip. **Notes:** 1. Examples rendered in Cycles, but EEVEE and Workbench rendering does not change behavior. 2. Hair particles rendered with a collection of simple mesh geometry in the examples, but rendering with single mesh objects or rendering hair paths does not change behavior. 3. Running in single thread mode (-t 1 on command line) does not change behavior. 4. Rendering via CPU or GPU does not change behavior. 5. Problem seems specific to hair particle systems. Emitter particle systems without pre-computed caches can be rendered with Cntl + F12; the cache seems fully valid in the time line, dynamics of falling emitter particles do not seem unusual and the cache section in the Particles properties reports that there are 250 valid frames in memory at the conclusion of the animation. **Related:** #62439 describes a failure in particle updates which seems very similar in appearance exhibited by particles in video clip **dyn_on.mkv.** [dyn_on_cache.mkv](https://archive.blender.org/developer/F6818202/dyn_on_cache.mkv) - dyn_on_cache.mkv [dyn_on.mkv](https://archive.blender.org/developer/F6818196/dyn_on.mkv) - dyn_on.mkv [dyn_on.blend](https://archive.blender.org/developer/F6818180/dyn_on.blend) - dyn_on.blend

Garry R. Osgood commented

2019-03-14 02:28:33 +01:00

Added subscriber: @grosgood

blender-admin commented

2019-03-14 02:28:33 +01:00

#71996 was marked as duplicate of this issue

Sebastian Parborg commented

2019-03-14 12:53:37 +01:00

Added subscribers: @dfelinto, @ZedDB

Dalai Felinto was assigned by Sebastian Parborg

2019-03-14 12:53:37 +01:00

Sebastian Parborg commented

2019-03-14 12:53:37 +01:00

I could have sworn there was already a ticket for this (particles not simulating when rendering without a cache), but I can't find it if that is the case.

@dfelinto reassign if this is not for you.

I could have sworn there was already a ticket for this (particles not simulating when rendering without a cache), but I can't find it if that is the case. @dfelinto reassign if this is not for you.

Garry R. Osgood commented

2019-03-18 12:54:58 +01:00

In blenkernel/intern/particle.c:psys_cache_paths, a N X (S + 1) array of "hair segment vertices" is built for the N hairs in the particle system, each with S segments. So for four segment hairs, there is a root, tip, and three interior hair vertices, for a total of 5. These "hair vertices" are distinct from model geometry vertices.

This pathcache "bucket" is created fresh at the start of rendering each frame and populates the particle system's pathcache field. psys_cache_paths() appears to develop a "snapshot" of positions, velocities and rotations of each hair vertex at the present frame.

It is not clear to me why this is so, but it seems in the case of Cntl + F12, without prior baking or creating an in-memory/disk cache, that the position, rotation or velocity of the underlying mesh appears not to be incorporated in the calculation of hair vertex positions, rotations or velocities. Watching the content of the pathcache in GDB during a rendering reveals unchanging positions frame after frame. This is not the case in baked/prior pointcache circumstances;
the pathcache appears to reflect the dynamics of the underlying object mesh.

One is tempted to think of an omitted transform somewhere, but I suspect that the problem is more diffuse than a dropped matrix. Fortunately, the severity of this bug is somewhere between a paper cut and a mild annoyance, since the workaround is simply to generate a point cache or bake the dynamics, and do such workarounds until the Glorious Rewrite of Particle Systems takes place. On the other hand, one would like to think that a shiny, production-ready 2.80 is going to be feature complete in every respect. One hates to let go of ideals.

In blenkernel/intern/particle.c:psys_cache_paths, a N X (S + 1) array of "hair segment vertices" is built for the N hairs in the particle system, each with S segments. So for four segment hairs, there is a root, tip, and three interior hair vertices, for a total of 5. These "hair vertices" are distinct from model geometry vertices. This pathcache "bucket" is created fresh at the start of rendering each frame and populates the particle system's pathcache field. psys_cache_paths() appears to develop a "snapshot" of positions, velocities and rotations of each hair vertex at the present frame. It is not clear to me why this is so, but it seems in the case of Cntl + F12, without prior baking or creating an in-memory/disk cache, that the position, rotation or velocity of the underlying mesh appears not to be incorporated in the calculation of hair vertex positions, rotations or velocities. Watching the content of the pathcache in GDB during a rendering reveals unchanging positions frame after frame. This is not the case in baked/prior pointcache circumstances; the pathcache appears to reflect the dynamics of the underlying object mesh. One is tempted to think of an omitted transform somewhere, but I suspect that the problem is more diffuse than a dropped matrix. Fortunately, the severity of this bug is somewhere between a paper cut and a mild annoyance, since the workaround is simply to generate a point cache or bake the dynamics, and do such workarounds until the Glorious Rewrite of Particle Systems takes place. On the other hand, one would like to think that a shiny, production-ready 2.80 is going to be feature complete in *every* respect. One hates to let go of ideals.

Dalai Felinto commented

2019-03-22 22:06:00 +01:00

Added subscribers: @Sergey, @dr.sybren

Dalai Felinto commented

2019-03-22 22:06:00 +01:00

@ZedDB let's find someone else that actually is familiar with this area ;) @Sergey worked on hair, @dr.sybren worked on cache, ...

Garry R. Osgood commented

2019-03-24 17:03:48 +01:00

Notes based on blender 2.80 commit: 7454fa927b Sat Mar 23 14:25:29 2019 +1100 #62563 still persists in this blender version.

Close to the heart of the matter is a - perhaps incomplete - meshing of designs of how dependency graph (depsgraph) updates proceed in the
rendering context and view layer context.

Frame-to-frame dependency graphs updates occur in two contexts: rendering, the consequence of CNTL F12, and view layer updates, the
consequence of running the timeline pointer forward.

These two contexts do not manage dependency graph updates in exactly the same way. In the rendering context, a per-frame call to
BKE_object_free(Object *ob), object.c:553, takes place, which frees the particle systems in ListBase ob->particlesystem. It is an array of
PointCaches deeper in these structures which encapsulate the state of frame T for hair simulators, which compute from them frame T + 1.
These PointCaches are freed as well and are not reconstructed in the course of the dependency graph update for frame T + 1. This
circumstance gives rise to the main feature of this bug: a hair simulation that essentially restarts from an initial T = 1 state
on every frame.

In contrast, the view layer context does not employ the draconian data management policy of BKE_object_free() at all. Only by toggling on the
Hair Dynamics checkbox property in the Particle panel does one see BKE_object_free(Object *ob) called on mesh objects with particle
systems. It is not called per-frame when one scrubs the timeline forward. Consequently, PointCache state persists from frame-to-frame and
hair simulation proceeds in an expected fashion.

Details in partsystem_notes.txt, attached.partsystem_notes.txt

Notes based on blender 2.80 commit: 7454fa927b80 Sat Mar 23 14:25:29 2019 +1100 #62563 still persists in this blender version. Close to the heart of the matter is a - perhaps incomplete - meshing of designs of how dependency graph (depsgraph) updates proceed in the rendering context and view layer context. Frame-to-frame dependency graphs updates occur in two contexts: rendering, the consequence of CNTL F12, and view layer updates, the consequence of running the timeline pointer forward. These two contexts do not manage dependency graph updates in exactly the same way. In the rendering context, a per-frame call to BKE_object_free(Object *ob), object.c:553, takes place, which frees the particle systems in ListBase ob->particlesystem. It is an array of PointCaches deeper in these structures which encapsulate the state of frame T for hair simulators, which compute from them frame T + 1. These PointCaches are freed as well and are not reconstructed in the course of the dependency graph update for frame T + 1. This circumstance gives rise to the main feature of this bug: a hair simulation that essentially restarts from an initial T = 1 state on every frame. In contrast, the view layer context does not employ the draconian data management policy of BKE_object_free() at all. Only by toggling on the Hair Dynamics checkbox property in the Particle panel does one see BKE_object_free(Object *ob) called on mesh objects with particle systems. It is not called per-frame when one scrubs the timeline forward. Consequently, PointCache state persists from frame-to-frame and hair simulation proceeds in an expected fashion. Details in partsystem_notes.txt, attached.[partsystem_notes.txt](https://archive.blender.org/developer/F6876637/partsystem_notes.txt)

Garry R. Osgood commented

2019-04-03 13:27:35 +02:00

[Commit ]]: 5494683651 (November 9, 2018) appears to have set the stage for this bug, which is a side effect. Very close to the heart of the matter is the hair dynamics simulation depending on the persistence of the same ParticleSystem object across the initial three frames of animation. With the commit, the prerequisite continuity of ParticleSystems can no longer be guaranteed, as they are reinitialized with the frame-by-frame deletion and recreation of the rendering dependency graph. Currently I'm wandering through the rabbit hutch that is [ https:*developer.blender.org/diffusion/B/browse/master/source/blender/blenkernel/intern/particle_system.c | particle_system_update() and friends, looking for either Big Buck Bunny, a contemporary of the code, it seems, or a way for ParticleSystems to be stateless - or at least less stateful. Simpler flavors of simulations, such as hair without dynamics or particle emitters, do not have equivalent dependencies on persisting ParticleSystem objects, and were unaffected by the November commit.

Details in partsystem_notes_2.txt, attached.
partsystem_notes_2.txt

[Commit ]]: 549468365157 (November 9, 2018) appears to have set the stage for this bug, which is a side effect. Very close to the heart of the matter is the hair dynamics simulation depending on the persistence of the same ParticleSystem object across the initial three frames of animation. With the commit, the prerequisite continuity of ParticleSystems can no longer be guaranteed, as they are reinitialized with the frame-by-frame deletion and recreation of the rendering dependency graph. Currently I'm wandering through the rabbit hutch that is [[ https:*developer.blender.org/diffusion/B/browse/master/source/blender/blenkernel/intern/particle_system.c | particle_system_update() ](https:*developer.blender.org/rB549468365157a075949f2d4b8d9496ff719cefbf) and friends, looking for either Big Buck Bunny, a contemporary of the code, it seems, or a way for ParticleSystems to be stateless - or at least less stateful. Simpler flavors of simulations, such as hair without dynamics or particle emitters, do not have equivalent dependencies on persisting ParticleSystem objects, and were unaffected by the November commit. Details in partsystem_notes_2.txt, attached. [partsystem_notes_2.txt](https://archive.blender.org/developer/F6908892/partsystem_notes_2.txt)

Philipp Oeser changed title from ~~Cannot generate a valid particle cache for dynamic hair particles~~ to Hair Dynamics: cannot generate a valid particle cache

2019-09-04 20:22:30 +02:00

Dalai Felinto was unassigned by Philipp Oeser

2019-09-04 20:22:30 +02:00

Philipp Oeser commented

2019-11-28 12:55:14 +01:00

Added subscribers: @PaulHartsuyker, @lichtwerk

Philipp Oeser commented

2019-11-28 12:56:21 +01:00

#71996 (Hair doesn't work, simple turnaround animation.) has a file that I cannot bake at all [there is never more that one valid frame in the cache.... no time to investigate more, just saying...]

Garry R. Osgood commented

2019-12-25 15:17:58 +01:00

Per [Tracker Curfew ]], [ https:*lists.blender.org/pipermail/bf-committers/2019-December/050383.html|Tracker curfew and developer.blender.org changes and #71418
Known issue: user documentation could use a Note box to the effect that caches cannot be generated for dynamic hair by means of a final render ( Cntl + F12 ), but generating a cache for dynamic hair by previewing the animation in the view port or baking hair dynamics works. In these two cases the particle system interacts with a persistent depsgraph in Viewport mode. Render depsgraphs do not persist but are recreated frame-by-frame. The hair particle system has not been designed to interact with transient depsgraphs.

Per [Tracker Curfew ]], [[ https:*lists.blender.org/pipermail/bf-committers/2019-December/050383.html|Tracker curfew and developer.blender.org changes ](https:*code.blender.org/2019/12/tracker-curfew/) and #71418 Known issue: user documentation could use a Note box to the effect that caches cannot be generated for dynamic hair by means of a final render ( Cntl + F12 ), but generating a cache for dynamic hair by previewing the animation in the view port or baking hair dynamics works. In these two cases the particle system interacts with a persistent depsgraph in Viewport mode. Render depsgraphs do not persist but are recreated frame-by-frame. The hair particle system has not been designed to interact with transient depsgraphs.

Garry R. Osgood commented

2019-12-25 15:25:47 +01:00

Removed Tracker Curfew tag per discussion between @dr.sybren and @dfelinto: Re: Tracker curfew and developer.blender.org changes Identified as a known issue.

Removed Tracker Curfew tag per discussion between @dr.sybren and @dfelinto: [Re: Tracker curfew and developer.blender.org changes ](https://lists.blender.org/pipermail/bf-committers/2019-December/050386.html) Identified as a known issue.

Philipp Oeser removed the

Download

What's New

Blender Studio

Manual

Developers Blog

Documentation

Benchmark

Blender Conference

Development Fund

One-time Donations

Hair Dynamics: cannot generate a valid particle cache #62563