Crashing with animation rendering and sound #62653

New Issue

Tomasz Kaye · 2019-03-16T13:26:38+01:00

Tomasz Kaye commented

2019-03-16 13:26:38 +01:00

System Information
Operating system: MacOS 10.14
Graphics card: Radeon Pro 580 8192 MB

Blender Version
Broken: 2.80, 3600bb79d05d-x86_64, 2019-03-16

Short description of error
Blender exits while rendering this project as an animation (fails to render). Usually seems to render first few tens of frames, but exits before getting to frame 100.

Exact steps for others to reproduce the error
Open file. Edit animation output location if necessary. Render animation.

cantrender_noaudio.blend

**System Information** Operating system: MacOS 10.14 Graphics card: Radeon Pro 580 8192 MB **Blender Version** Broken: 2.80, 3600bb79d05d-x86_64, 2019-03-16 **Short description of error** Blender exits while rendering this project as an animation (fails to render). Usually seems to render first few tens of frames, but exits before getting to frame 100. **Exact steps for others to reproduce the error** Open file. Edit animation output location if necessary. Render animation. [cantrender_noaudio.blend](https://archive.blender.org/developer/F6834338/cantrender_noaudio.blend)

Tomasz Kaye commented

2019-03-16 13:26:38 +01:00

Added subscriber: @info-27

Tomasz Kaye commented

2019-03-16 16:01:06 +01:00

Changed status from 'Open' to: 'Archived'

Tomasz Kaye closed this issue

2019-03-16 16:01:06 +01:00

Antonio Vazquez commented

2019-03-16 16:01:35 +01:00

Added subscriber: @antoniov

Antonio Vazquez commented

2019-03-16 16:01:35 +01:00

The crash is related to sound. Here the call stack:

>	blender.exe!std::_Tree<std::_Tset_traits<void *,std::less<void *>,std::allocator<void *>,0> >::_Orphan_ptr(std::_Tree_node<void *,void *> * _Ptr) Line 2097	C++

 	blender.exe!std::_Tree<std::_Tset_traits<void *,std::less<void *>,std::allocator<void *>,0> >::clear() Line 1410	C++
 	blender.exe!std::_Tree<std::_Tset_traits<void *,std::less<void *>,std::allocator<void *>,0> >::erase(std::_Tree_const_iterator<std::_Tree_val<std::_Tree_simple_types<void *> > > _First, std::_Tree_const_iterator<std::_Tree_val<std::_Tree_simple_types<void *> > > _Last) Line 1389	C++
 	blender.exe!std::_Tree<std::_Tset_traits<void *,std::less<void *>,std::allocator<void *>,0> >::_Tidy() Line 2113	C++
 	blender.exe!std::_Tree<std::_Tset_traits<void *,std::less<void *>,std::allocator<void *>,0> >::~_Tree<std::_Tset_traits<void *,std::less<void *>,std::allocator<void *>,0> >() Line 1168	C++
 	blender.exe!std::set<void *,std::less<void *>,std::allocator<void *> >::~set<void *,std::less<void *>,std::allocator<void *> >()	C++
 	blender.exe!std::set<void *,std::less<void *>,std::allocator<void *> >::`scalar deleting destructor'(unsigned int)	C++
 	blender.exe!AUD_destroySet(void * set) Line 36	C++
 	blender.exe!BKE_sound_update_scene(Main * bmain, Scene * scene) Line 912	C
 	blender.exe!BKE_scene_graph_update_for_newframe(Depsgraph * depsgraph, Main * bmain) Line 1562	C
 	blender.exe!engine_depsgraph_init(RenderEngine * engine, ViewLayer * view_layer) Line 497	C
 	blender.exe!RE_engine_render(Render * re, int do_all) Line 727	C
 	blender.exe!do_render_3d(Render * re) Line 1081	C
 	blender.exe!do_render(Render * re) Line 1160	C
 	blender.exe!do_render_composite(Render * re) Line 1366	C
 	blender.exe!do_render_all_options(Render * re) Line 1635	C
 	blender.exe!RE_BlenderAnim(Render * re, Main * bmain, Scene * scene, ViewLayer * single_layer, Object * camera_override, int sfra, int efra, int tfra) Line 2487	C
 	blender.exe!render_startjob(void * rjv, short * stop, short * do_update, float * progress) Line 612	C
 	blender.exe!do_job_thread(void * job_v) Line 331	C
 	blender.exe!tslot_thread_start(void * tslot_p) Line 252	C

I don't know who is the person to fix this.

The crash is related to sound. Here the call stack: ``` > blender.exe!std::_Tree<std::_Tset_traits<void *,std::less<void *>,std::allocator<void *>,0> >::_Orphan_ptr(std::_Tree_node<void *,void *> * _Ptr) Line 2097 C++ blender.exe!std::_Tree<std::_Tset_traits<void *,std::less<void *>,std::allocator<void *>,0> >::clear() Line 1410 C++ blender.exe!std::_Tree<std::_Tset_traits<void *,std::less<void *>,std::allocator<void *>,0> >::erase(std::_Tree_const_iterator<std::_Tree_val<std::_Tree_simple_types<void *> > > _First, std::_Tree_const_iterator<std::_Tree_val<std::_Tree_simple_types<void *> > > _Last) Line 1389 C++ blender.exe!std::_Tree<std::_Tset_traits<void *,std::less<void *>,std::allocator<void *>,0> >::_Tidy() Line 2113 C++ blender.exe!std::_Tree<std::_Tset_traits<void *,std::less<void *>,std::allocator<void *>,0> >::~_Tree<std::_Tset_traits<void *,std::less<void *>,std::allocator<void *>,0> >() Line 1168 C++ blender.exe!std::set<void *,std::less<void *>,std::allocator<void *> >::~set<void *,std::less<void *>,std::allocator<void *> >() C++ blender.exe!std::set<void *,std::less<void *>,std::allocator<void *> >::`scalar deleting destructor'(unsigned int) C++ blender.exe!AUD_destroySet(void * set) Line 36 C++ blender.exe!BKE_sound_update_scene(Main * bmain, Scene * scene) Line 912 C blender.exe!BKE_scene_graph_update_for_newframe(Depsgraph * depsgraph, Main * bmain) Line 1562 C blender.exe!engine_depsgraph_init(RenderEngine * engine, ViewLayer * view_layer) Line 497 C blender.exe!RE_engine_render(Render * re, int do_all) Line 727 C blender.exe!do_render_3d(Render * re) Line 1081 C blender.exe!do_render(Render * re) Line 1160 C blender.exe!do_render_composite(Render * re) Line 1366 C blender.exe!do_render_all_options(Render * re) Line 1635 C blender.exe!RE_BlenderAnim(Render * re, Main * bmain, Scene * scene, ViewLayer * single_layer, Object * camera_override, int sfra, int efra, int tfra) Line 2487 C blender.exe!render_startjob(void * rjv, short * stop, short * do_update, float * progress) Line 612 C blender.exe!do_job_thread(void * job_v) Line 331 C blender.exe!tslot_thread_start(void * tslot_p) Line 252 C ``` ![image.png](https://archive.blender.org/developer/F6834833/image.png) I don't know who is the person to fix this.

Tomasz Kaye commented

2019-03-16 16:01:53 +01:00

Checking console log showed that it was a problem with a 3rd party add-on (snapscrubber). Removing the add on fixed the problem.

Tomasz Kaye commented

2019-03-16 16:02:18 +01:00

Changed status from 'Archived' to: 'Open'

Tomasz Kaye reopened this issue

2019-03-16 16:02:18 +01:00

Antonio Vazquez commented

2019-03-16 16:02:43 +01:00

@info-27 I got an error with sound rendering your file. Are you sure we can close it?

Antonio Vazquez commented

2019-03-16 16:06:00 +01:00

Added subscriber: @brecht

Antonio Vazquez commented

2019-03-16 16:06:00 +01:00

@brecht What can we do here?

Tomasz Kaye commented

2019-03-16 16:06:16 +01:00

@antoniov will comment to confirm after this render is done, But on frame 1000 without problems so far.

Tomasz Kaye commented

2019-03-16 17:14:46 +01:00

Ah seems there's still a problem. Crashed again with this complaint:

Append frame 3764
 Time: 00:00.68 (Saving: 00:00.06)
blender(57290,0x11544c5c0) malloc: *** error for object 0x7fe864f1b870: pointer being freed was not allocated
blender(57290,0x11544c5c0) malloc: *** set a breakpoint in malloc_error_break to debug
Abort trap: 6
logout
Saving session...
...copying shared history...
...saving history...truncating history files...
...completed.

Ah seems there's still a problem. Crashed again with this complaint: ``` Append frame 3764 Time: 00:00.68 (Saving: 00:00.06) blender(57290,0x11544c5c0) malloc: *** error for object 0x7fe864f1b870: pointer being freed was not allocated blender(57290,0x11544c5c0) malloc: *** set a breakpoint in malloc_error_break to debug Abort trap: 6 logout Saving session... ...copying shared history... ...saving history...truncating history files... ...completed. ```

Brecht Van Lommel commented

2019-03-18 14:23:57 +01:00

I have not been able to reproduce this crash on Linux. There may be random memory corruption involved, or something with thread timing.

It's likely possible to further pin down the issue.

Render with an empty scene, or with Cycles. This may be unrelated to grease pencil.
Render to jpg file instead of ffmpeg video, as this may be audio related.
Disable audaspace in the build and render (for developers).

I have not been able to reproduce this crash on Linux. There may be random memory corruption involved, or something with thread timing. It's likely possible to further pin down the issue. * Render with an empty scene, or with Cycles. This may be unrelated to grease pencil. * Render to jpg file instead of ffmpeg video, as this may be audio related. * Disable audaspace in the build and render (for developers).

Ray molenkamp commented

2019-04-08 20:56:36 +02:00

Added subscribers: @neXyon, @LazyDodo

Joerg Mueller was assigned by Ray molenkamp

2019-04-08 20:56:36 +02:00

Ray molenkamp commented

2019-04-08 20:56:36 +02:00

Can still confirm on windows, different trace than @antoniov but still coming out of audaspace though, @neXyon mind taking a peek here?

>	blender.exe!std::_Tree<std::_Tset_traits<void *,std::less<void *>,std::allocator<void *>,0> >::begin() Line 1187	C++

 	blender.exe!AUD_getSet(void * set=0x000000018b292fe0) Line 58	C++
 	blender.exe!BKE_sound_update_scene(Main * bmain=0x000000008223b958, Scene * scene=0x00000000822716f8) Line 901	C
 	blender.exe!BKE_scene_graph_update_tagged(Depsgraph * depsgraph=0x0000000038e2cef8, Main * bmain=0x000000008223b958) Line 1512	C
 	blender.exe!wm_event_do_depsgraph(bContext * C=0x0000000013b8af78) Line 343	C
 	blender.exe!wm_event_do_refresh_wm_and_depsgraph(bContext * C=0x0000000013b8af78) Line 369	C
 	blender.exe!wm_event_do_notifiers(bContext * C=0x0000000013b8af78) Line 528	C
 	blender.exe!WM_main(bContext * C=0x0000000013b8af78) Line 424	C
 	blender.exe!main(int argc=1, const unsigned char * * UNUSED_argv_c=0x0000000008e5afa0) Line 507	C
 	[External Code]

Can still confirm on windows, different trace than @antoniov but still coming out of audaspace though, @neXyon mind taking a peek here? ``` > blender.exe!std::_Tree<std::_Tset_traits<void *,std::less<void *>,std::allocator<void *>,0> >::begin() Line 1187 C++ blender.exe!AUD_getSet(void * set=0x000000018b292fe0) Line 58 C++ blender.exe!BKE_sound_update_scene(Main * bmain=0x000000008223b958, Scene * scene=0x00000000822716f8) Line 901 C blender.exe!BKE_scene_graph_update_tagged(Depsgraph * depsgraph=0x0000000038e2cef8, Main * bmain=0x000000008223b958) Line 1512 C blender.exe!wm_event_do_depsgraph(bContext * C=0x0000000013b8af78) Line 343 C blender.exe!wm_event_do_refresh_wm_and_depsgraph(bContext * C=0x0000000013b8af78) Line 369 C blender.exe!wm_event_do_notifiers(bContext * C=0x0000000013b8af78) Line 528 C blender.exe!WM_main(bContext * C=0x0000000013b8af78) Line 424 C blender.exe!main(int argc=1, const unsigned char * * UNUSED_argv_c=0x0000000008e5afa0) Line 507 C [External Code] ```

Joerg Mueller commented

2019-04-09 10:13:17 +02:00

I tried to reproduce the crash, but it also didn't happen on my linux. Given the stack traces though, it looks like the issue is BKE_sound_update_scene being called at the same time in different threads. As such a mutex in this function should solve the problem?

I tried to reproduce the crash, but it also didn't happen on my linux. Given the stack traces though, it looks like the issue is `BKE_sound_update_scene` being called at the same time in different threads. As such a mutex in this function should solve the problem?

matc commented

2019-04-13 02:34:03 +02:00

Added subscriber: @matc

matc commented

2019-04-14 04:03:06 +02:00

The problem seams to have been introduced in 48f9e24f0c Enable dependency graph update while rendering.

During rendering scene->speaker_handles gets manipulated from both the render thread and the window event thread. This leads to a race condition, even when not using audio, as both threads attempt to delete and replace the scene->speaker_handles. I'm not sure whether it even makes sense to add/remove sound strips to/from the same scene->sound_scene from two places, even in a thread safe manner. Viewport and renderer would probably need each its own instance. I guess this could be solved by (TODO, move to depsgraph).

This would be a solution to revert the problem in the meantime:

diff --git a/source/blender/blenkernel/intern/scene.c b/source/blender/blenkernel/intern/scene.c
index 644eb704655..e48d1ba2d82 100644
--- a/source/blender/blenkernel/intern/scene.c
+++ b/source/blender/blenkernel/intern/scene.c
@@ -65,6 +65,7 @@
 - include "BKE_editmesh.h"
 - include "BKE_fcurve.h"
 #include "BKE_freestyle.h"
+#include "BKE_global.h"
 - include "BKE_gpencil.h"
 - include "BKE_icons.h"
 #include "BKE_idprop.h"
@@ -1506,7 +1507,9 @@ void BKE_scene_graph_update_tagged(Depsgraph *depsgraph,
 	 */
 	DEG_evaluate_on_refresh(depsgraph);
 	/* Update sound system animation (TODO, move to depsgraph). */
-	BKE_sound_update_scene(bmain, scene);
+	if (!G.is_rendering) {
+		BKE_sound_update_scene(bmain, scene);
+	}
 
 	/* Notify python about depsgraph update */
 	if (run_callbacks) {

The problem seams to have been introduced in 48f9e24f0c1e *Enable dependency graph update while rendering*. During rendering `scene->speaker_handles` gets manipulated from both the render thread and the window event thread. This leads to a race condition, even when not using audio, as both threads attempt to delete and replace the `scene->speaker_handles`. I'm not sure whether it even makes sense to add/remove sound strips to/from the same `scene->sound_scene` from two places, even in a thread safe manner. Viewport and renderer would probably need each its own instance. I guess this could be solved by `(TODO, move to depsgraph)`. This would be a solution to revert the problem in the meantime: ``` diff --git a/source/blender/blenkernel/intern/scene.c b/source/blender/blenkernel/intern/scene.c index 644eb704655..e48d1ba2d82 100644 --- a/source/blender/blenkernel/intern/scene.c +++ b/source/blender/blenkernel/intern/scene.c @@ -65,6 +65,7 @@ - include "BKE_editmesh.h" - include "BKE_fcurve.h" #include "BKE_freestyle.h" +#include "BKE_global.h" - include "BKE_gpencil.h" - include "BKE_icons.h" #include "BKE_idprop.h" @@ -1506,7 +1507,9 @@ void BKE_scene_graph_update_tagged(Depsgraph *depsgraph, */ DEG_evaluate_on_refresh(depsgraph); /* Update sound system animation (TODO, move to depsgraph). */ - BKE_sound_update_scene(bmain, scene); + if (!G.is_rendering) { + BKE_sound_update_scene(bmain, scene); + } /* Notify python about depsgraph update */ if (run_callbacks) { ```

Joerg Mueller commented

2019-04-14 10:09:09 +02:00

Nice find on where the problem was introduced! I'd be fine with the temporary fix. Are you sure that this doesn't prevent both calls of the two threads during rendering? One should still be done. I'm not sure what moving to the dependency graph will change here, does the dependency graph get updated from two threads as well? Or is it two different instances and thus as you said viewport and renderer each need their own sound_scene instance?

matc commented

2019-04-14 21:51:04 +02:00

The two calls to BKE_sound_update_scene both come from a depencygraph. One holds the copies for the viewport and the other one for the renderer. Both are running in their own thread. BKE_scene_graph_update_tagged happens to not be called for the render thread. But BKE_scene_graph_update_for_newframe is called for both, which is a problem.

Apparently each dependency graph already creates a copy of sound_scene and speaker_handles. So this should be working (only tested without audio):

diff --git a/source/blender/blenkernel/intern/scene.c b/source/blender/blenkernel/intern/scene.c
index 644eb704655..77ee3731766 100644
--- a/source/blender/blenkernel/intern/scene.c
+++ b/source/blender/blenkernel/intern/scene.c
@@ -1506,7 +1506,7 @@ void BKE_scene_graph_update_tagged(Depsgraph *depsgraph,
 	 */
 	DEG_evaluate_on_refresh(depsgraph);
 	/* Update sound system animation (TODO, move to depsgraph). */
-	BKE_sound_update_scene(bmain, scene);
+	BKE_sound_update_scene(bmain, DEG_get_evaluated_scene(depsgraph));
 
 	/* Notify python about depsgraph update */
 	if (run_callbacks) {
@@ -1551,7 +1551,7 @@ void BKE_scene_graph_update_for_newframe(Depsgraph *depsgraph,
 	 */
 	DEG_evaluate_on_framechange(bmain, depsgraph, ctime);
 	/* Update sound system animation (TODO, move to depsgraph). */
-	BKE_sound_update_scene(bmain, scene);
+	BKE_sound_update_scene(bmain, DEG_get_evaluated_scene(depsgraph));
 	/* Notify editors and python about recalc. */
 	BLI_callback_exec(bmain, &scene->id, BLI_CB_EVT_FRAME_CHANGE_POST);
 	/* Inform editors about possible changes. */

The two calls to BKE_sound_update_scene both come from a depencygraph. One holds the copies for the viewport and the other one for the renderer. Both are running in their own thread. BKE_scene_graph_update_tagged happens to not be called for the render thread. But BKE_scene_graph_update_for_newframe is called for both, which is a problem. Apparently each dependency graph already creates a copy of sound_scene and speaker_handles. So this should be working (only tested without audio): ``` diff --git a/source/blender/blenkernel/intern/scene.c b/source/blender/blenkernel/intern/scene.c index 644eb704655..77ee3731766 100644 --- a/source/blender/blenkernel/intern/scene.c +++ b/source/blender/blenkernel/intern/scene.c @@ -1506,7 +1506,7 @@ void BKE_scene_graph_update_tagged(Depsgraph *depsgraph, */ DEG_evaluate_on_refresh(depsgraph); /* Update sound system animation (TODO, move to depsgraph). */ - BKE_sound_update_scene(bmain, scene); + BKE_sound_update_scene(bmain, DEG_get_evaluated_scene(depsgraph)); /* Notify python about depsgraph update */ if (run_callbacks) { @@ -1551,7 +1551,7 @@ void BKE_scene_graph_update_for_newframe(Depsgraph *depsgraph, */ DEG_evaluate_on_framechange(bmain, depsgraph, ctime); /* Update sound system animation (TODO, move to depsgraph). */ - BKE_sound_update_scene(bmain, scene); + BKE_sound_update_scene(bmain, DEG_get_evaluated_scene(depsgraph)); /* Notify editors and python about recalc. */ BLI_callback_exec(bmain, &scene->id, BLI_CB_EVT_FRAME_CHANGE_POST); /* Inform editors about possible changes. */ ```

matc commented

2019-04-15 01:11:30 +02:00

I added an audio track in the sequencer to do some testing. Everything seams to work fine. Only when adding another track while rendering, the added track is part of the remaining render. But this seems to be a general problem, as moving the cube in the default scene while rendering does show up in the render too. I assume this is limited to animation rendering.

I also tested one scene with audio as a track in another scene with audio. Which worked out too.

I added an audio track in the sequencer to do some testing. Everything seams to work fine. Only when adding another track while rendering, the added track is part of the remaining render. But this seems to be a general problem, as moving the cube in the default scene while rendering does show up in the render too. I assume this is limited to animation rendering. I also tested one scene with audio as a track in another scene with audio. Which worked out too.

Brecht Van Lommel changed title from ~~Crashing while rendering animation. Grease pencil.~~ to Crashing with animation rendering and sound

2019-04-15 17:45:00 +02:00

Joerg Mueller was unassigned by Brecht Van Lommel

2019-04-15 17:45:00 +02:00

Sergey Sharybin was assigned by Brecht Van Lommel

2019-04-15 17:45:00 +02:00

Brecht Van Lommel commented

2019-04-15 17:46:20 +02:00

So this fix goes in the right direction, but it's not complete. All the sound handles and state should only exist and get updated on the evaluated scene, not the original scene. That means saving audio and playback should also get data from the evaluated scene.

Assigning to @Sergey to look into.

So this fix goes in the right direction, but it's not complete. All the sound handles and state should only exist and get updated on the evaluated scene, not the original scene. That means saving audio and playback should also get data from the evaluated scene. Assigning to @Sergey to look into.

Sergey Sharybin commented

2019-05-03 15:59:15 +02:00

Changed status from 'Open' to: 'Resolved'

Sergey Sharybin closed this issue

2019-05-03 15:59:15 +02:00

Sergey Sharybin commented

2019-05-03 15:59:15 +02:00

I did a series of changes, which should have made the audio system compatible with copy-on-write concept. The latest change is in a460e97212.

Give it a test, and if there are more issues discovered make a new report.

I did a series of changes, which should have made the audio system compatible with copy-on-write concept. The latest change is in a460e97212. Give it a test, and if there are more issues discovered make a new report.

Sergey Sharybin commented

2019-05-07 11:38:13 +02:00

Changed status from 'Resolved' to: 'Open'

Sergey Sharybin reopened this issue

2019-05-07 11:38:13 +02:00

Sergey Sharybin commented

2019-05-07 11:38:13 +02:00

Re-opening. Need to re-iterate over the fix.

matc commented

2019-06-02 00:19:16 +02:00

I can still trigger this bug fairly frequently just by rendering the default .blend file at a low resolution.

Primarily crashing in AUD_destroySet and occasionally in AUD_getSet.

I can still trigger this bug fairly frequently just by rendering the default .blend file at a low resolution. Primarily crashing in AUD_destroySet and occasionally in AUD_getSet.

Sergey Sharybin commented

2019-06-05 14:25:48 +02:00

Changed status from 'Open' to: 'Resolved'

Sergey Sharybin closed this issue

2019-06-05 14:25:48 +02:00

Sergey Sharybin commented

2019-06-05 14:25:48 +02:00

Committed reworked patch at bbaa1bffe9. The issue is now expected to be solved.

Thanks for the report, closing.

Committed reworked patch at bbaa1bffe9. The issue is now expected to be solved. Thanks for the report, closing.

Sign in to join this conversation.

No Label

Download

What's New

Blender Studio

Manual

Developers Blog

Documentation

Benchmark

Blender Conference

Development Fund

One-time Donations

Crashing with animation rendering and sound #62653