Blender crashes when attempting to save the blend file from a script prior to rendering #92704

New Issue

Jeducious · 2021-11-01T10:58:50+01:00

Jeducious commented

2021-11-01 10:58:50 +01:00

System Information
System info file for Blender 3.0 Beta tests bl_3_0_beta_system-info.txt
System info file for Blender 3.1 alpha tests
bl_3.1_alpha_system-info.txt
Crash Dump
blender.crash.txt
python_module_crash.txt
Minimal python script for reproducing the crash
render_engine_script.py

Blender Version
Broken: See above sys info files
Worked: see bl_2_93_4_system-info.txt

Short description of error
When attempting to render using the Crowdrender addon, pressing F12 or using the render still/animation menu options causes Blender to crash. Since I am a developer for Crowdrender, I investigated until I could create a minimum test case for causing the crash. That script is attached above.

The cause appears to be calling the bpy.ops.wm.savemainfile or bpy.ops.wm.save_as_mainfile operators whilst in a render_init handler. Further investigation showed that this seems to only occur if I modify a property in the scene or in the bpy.context.window_manager. This combination of modifying such a property, then calling a render, and whilst handling the render_init event, attempting to call bpy.ops.wm.savemainfile or bpy.ops.wm.save_as_mainfile appears to be what causes the crash.

I have done more tests and if no such property is modified, then there is no crash, so I am fairly confident this combination is causing the crash.

All of this works fine and has worked fine since Blender 2.83 up to 2.94.

Desired outcome
@brecht has mentioned that what we are doing is the best way to enable our addon to work well with Blender. So my hope is that this problem can be fixed in Blender.

Exact steps for others to reproduce the error

open Blender 3.0 beta or 3.1 alpha
With the default scene loaded, open the scripting workspace
In the scripting workspace, in the text editor, open the script attached to this bug report and click the run script button. It will register a render engine and a handler for render_init events
Press the F12 button, or use the render still menu option
Blender will exit immediately

**System Information** System info file for Blender 3.0 Beta tests [bl_3_0_beta_system-info.txt](https://archive.blender.org/developer/F11656556/bl_3_0_beta_system-info.txt) System info file for Blender 3.1 alpha tests [bl_3.1_alpha_system-info.txt](https://archive.blender.org/developer/F11656553/bl_3.1_alpha_system-info.txt) **Crash Dump** [blender.crash.txt](https://archive.blender.org/developer/F11656557/blender.crash.txt) [python_module_crash.txt](https://archive.blender.org/developer/F11656555/python_module_crash.txt) **Minimal python script for reproducing the crash** [render_engine_script.py](https://archive.blender.org/developer/F11656554/render_engine_script.py) **Blender Version** Broken: See above sys info files Worked: see [bl_2_93_4_system-info.txt](https://archive.blender.org/developer/F11656625/bl_2_93_4_system-info.txt) **Short description of error** When attempting to render using the Crowdrender addon, pressing F12 or using the render still/animation menu options causes Blender to crash. Since I am a developer for Crowdrender, I investigated until I could create a minimum test case for causing the crash. That script is attached above. The cause appears to be calling the bpy.ops.wm.savemainfile or bpy.ops.wm.save_as_mainfile operators whilst in a render_init handler. Further investigation showed that this seems to only occur if I modify a property in the scene or in the bpy.context.window_manager. This combination of modifying such a property, then calling a render, and whilst handling the render_init event, attempting to call bpy.ops.wm.savemainfile or bpy.ops.wm.save_as_mainfile appears to be what causes the crash. I have done more tests and if no such property is modified, then there is no crash, so I am fairly confident this combination is causing the crash. All of this works fine and has worked fine since Blender 2.83 up to 2.94. **Desired outcome** @brecht has [mentioned that what we are doing is the best way ](https://devtalk.blender.org/t/changes-in-blender-3-0-3-1-leading-to-a-segfault-when-rendering-using-crowdrender/21097) to enable our addon to work well with Blender. So my hope is that this problem can be fixed in Blender. **Exact steps for others to reproduce the error** - open Blender 3.0 beta or 3.1 alpha - With the default scene loaded, open the scripting workspace - In the scripting workspace, in the text editor, open the script attached to this bug report and click the run script button. It will register a render engine and a handler for render_init events - Press the F12 button, or use the render still menu option - Blender will exit immediately

Jeducious commented

2021-11-01 10:58:50 +01:00

Added subscribers: @brecht, @JamesCrowther

Jeducious commented

2021-11-02 00:51:25 +01:00

Update, have done more testing and its also broken on MacOS as well. Haven't tested linux yet though.

So, on macOS I get the following stack dump blender.crash.txt, the sys_info for the machine that created this is system-info.txt.

The problem on macOS is a little different. I have modified my code now so that I do not modify any properties in the scene or window manager. This allows me to render consecutively. The code still saves a copy of the blend file from the bpy.app.handlers.render_init handler I have registered.

On Windows the crash still happens instantly the moment a render is attempted using the crowdrender engine. So modifying properties doesn't appear to be a factor on that platform. However, on MacOS, I can render consecutively, as I said, but if I then modify the scene, I can get Blender to crash when pressing render. This doesn't appear to be consistent, however. There are times when the modification to the scene, and then attempting a render will not cause a crash. The problem is intermittent. I have not done an empirical study yet, so I can't really comment on the percentage of attempts that cause a crash.

Update, have done more testing and its also broken on MacOS as well. Haven't tested linux yet though. So, on macOS I get the following stack dump [blender.crash.txt](https://archive.blender.org/developer/F11668146/blender.crash.txt), the sys_info for the machine that created this is [system-info.txt](https://archive.blender.org/developer/F11668180/system-info.txt). The problem on macOS is a little different. I have modified my code now so that I do not modify any properties in the scene or window manager. This allows me to render consecutively. The code still saves a copy of the blend file from the bpy.app.handlers.render_init handler I have registered. On Windows the crash still happens instantly the moment a render is attempted using the crowdrender engine. So modifying properties doesn't appear to be a factor on that platform. However, on MacOS, I can render consecutively, as I said, but if I then modify the scene, I can get Blender to crash when pressing render. This doesn't appear to be consistent, however. There are times when the modification to the scene, and then attempting a render will not cause a crash. The problem is intermittent. I have not done an empirical study yet, so I can't really comment on the percentage of attempts that cause a crash.

Jeducious commented

2021-11-02 01:15:34 +01:00

Further testing done, I checked the latest python API spec for the bpy.ops.wm.save_as_mainfile operator. Check existing is set to true to warn if the file already exists. To eliminate this as a potential issue I tested setting this to False in my code. Sadly this still causes an instant crash on Windows so this seems to not help.

My thinking was that since warning the user might display a pop-up or write a message to the user, and this is where the code seems to be crashing, that I should perhaps test preventing this information from being displayed. However, my tests showed that setting the check_existing argument of the save_as_mainfile operator did not prevent the crash.

Further testing done, I checked the latest python API spec for the bpy.ops.wm.save_as_mainfile operator. Check existing is set to true to warn if the file already exists. To eliminate this as a potential issue I tested setting this to False in my code. Sadly this still causes an instant crash on Windows so this seems to not help. My thinking was that since warning the user might display a pop-up or write a message to the user, and this is where the code seems to be crashing, that I should perhaps test preventing this information from being displayed. However, my tests showed that setting the check_existing argument of the save_as_mainfile operator did not prevent the crash.

Jeducious commented

2021-11-03 00:48:50 +01:00

@brecht Finally tracked down the issue and commit that introduced the bug! This issue was caused by a change back in September 17th. https://developer.blender.org/D12507 was created to deal with the following...

When saving blend files close any menus that might be open, show
"waiting" mouse cursor right away, before creating preview.

However this seems to cause a race condition when the bpy.ops.wm.save_as_mainfile or bpy.ops.wm.save_mainfile operators are run from within a handler registered for the bpy.app.handlers.render_init, render_pre or in the render method of the RenderEngine class.

When calling the wm_file_write method in this context, the crash occurs.

I've recompiled Blender and commented out the following from wm_file_write


  if (!G.background) {
    /* Redraw to remove menus that might be open. */
    WM_redraw_windows(C);
  }

Once recompiled, the crash is now gone, so this looking like the cause of the crash alright.

We've had a chat amongst us devs here at Crowdrender, and have a suggestion, this redraw call is unavoidable when wanting to save the blend file from a script, which will not have any windows open when it does this. So in that case, we'd not want to redraw the UI, no need to. The user is now rendering and probably won't have any UI elements open.

Its also an unexpected side effect of saving a file. I get it that the UI would be altered if the user was saving the file using the UI, but this is a different case.

I think this could be handled better if the redraw call was done from the invoke part of the save operators, rather than the wm_file_write method. Having the redraw call in the code that is writing to the file seems like going against the philosophy of separation of concerns, and in this case is directly responsible for causing a crash.

I am going to experiment by seeing if I can move the redraw call, or achieve the same effect by moving the same call, or something similar to the invoke section of the bpy.ops.wm.save_X operators, since it makes more sense to me that those would be where the user is engaging the save functions of Blender and more appropriate for redrawing the UI.

Be happy to hear your and any other module owners thoughts though..

@brecht Finally tracked down the issue and commit that introduced the bug! This issue was caused by a change back in September 17th. https://developer.blender.org/D12507 was created to deal with the following... > > When saving blend files close any menus that might be open, show > "waiting" mouse cursor right away, before creating preview. However this seems to cause a race condition when the bpy.ops.wm.save_as_mainfile or bpy.ops.wm.save_mainfile operators are run from within a handler registered for the bpy.app.handlers.render_init, render_pre or in the render method of the RenderEngine class. When calling the wm_file_write method in this context, the crash occurs. I've recompiled Blender and commented out the following from [wm_file_write ](https://github.com/blender/blender/blob/ffd3dd63769c8e97eceda54c8fa4067d872655be/source/blender/windowmanager/intern/wm_files.c#L1796) ``` if (!G.background) { /* Redraw to remove menus that might be open. */ WM_redraw_windows(C); } ``` Once recompiled, the crash is now gone, so this looking like the cause of the crash alright. We've had a chat amongst us devs here at Crowdrender, and have a suggestion, this redraw call is unavoidable when wanting to save the blend file from a script, which will not have any windows open when it does this. So in that case, we'd not want to redraw the UI, no need to. The user is now rendering and probably won't have any UI elements open. Its also an unexpected side effect of saving a file. I get it that the UI would be altered if the user was saving the file using the UI, but this is a different case. I think this could be handled better if the redraw call was done from the invoke part of the save operators, rather than the wm_file_write method. Having the redraw call in the code that is writing to the file seems like going against the philosophy of separation of concerns, and in this case is directly responsible for causing a crash. I am going to experiment by seeing if I can move the redraw call, or achieve the same effect by moving the same call, or something similar to the invoke section of the bpy.ops.wm.save_X operators, since it makes more sense to me that those would be where the user is engaging the save functions of Blender and more appropriate for redrawing the UI. Be happy to hear your and any other module owners thoughts though..

Jeducious commented

2021-11-03 01:08:23 +01:00

ok final tests confirmed that I can move the call to redraw to the [wm_save_as_mainfile_invoke ]] in [ https:*github.com/blender/blender/blob/ffd3dd63769c8e97eceda54c8fa4067d872655be/source/blender/windowmanager/intern/wm_files.c | wm_files.c .

I recompiled Blender again with the following change to wm_save_as_mainfile_invoke in addition to removing the call to redraw from wm_file_write.

static int wm_save_as_mainfile_invoke(bContext *C, wmOperator *op, const wmEvent *UNUSED(event))
{

  save_set_compress(op);
  save_set_filepath(C, op);

  if (!G.background) {
    /* Redraw to remove menus that might be open. */
    WM_redraw_windows(C);
  }

  WM_event_add_fileselect(C, op);

  return OPERATOR_RUNNING_MODAL;
}

All tested fine in master on windows 10. I was able to save from the UI and run our addon with no crashes and the file still saves in both cases as expected, and there are no glitches in the UI that I found.

Would like to offer these changes as a patch :)

ok final tests confirmed that I can move the call to redraw to the [wm_save_as_mainfile_invoke ]] in [[ https:*github.com/blender/blender/blob/ffd3dd63769c8e97eceda54c8fa4067d872655be/source/blender/windowmanager/intern/wm_files.c | wm_files.c ](https:*github.com/blender/blender/blob/ffd3dd63769c8e97eceda54c8fa4067d872655be/source/blender/windowmanager/intern/wm_files.c#L3073). I recompiled Blender again with the following change to wm_save_as_mainfile_invoke in addition to removing the call to redraw from wm_file_write. ``` static int wm_save_as_mainfile_invoke(bContext *C, wmOperator *op, const wmEvent *UNUSED(event)) { save_set_compress(op); save_set_filepath(C, op); if (!G.background) { /* Redraw to remove menus that might be open. */ WM_redraw_windows(C); } WM_event_add_fileselect(C, op); return OPERATOR_RUNNING_MODAL; } ``` All tested fine in master on windows 10. I was able to save from the UI and run our addon with no crashes and the file still saves in both cases as expected, and there are no glitches in the UI that I found. Would like to offer these changes as a patch :)

Campbell Barton commented

2021-11-03 03:19:20 +01:00

Added subscriber: @ideasman42

Campbell Barton commented

2021-11-03 03:19:20 +01:00

The test script isn't working with factory settings.

Traceback (most recent call last):
  File "/render_engine_script.py", line 68, in before_render
  File "/render_engine_script.py", line 41, in create_temp_directory
AttributeError: 'Scene' object has no attribute 'crowd_render'

The test script isn't working with factory settings. ``` Traceback (most recent call last): File "/render_engine_script.py", line 68, in before_render File "/render_engine_script.py", line 41, in create_temp_directory AttributeError: 'Scene' object has no attribute 'crowd_render' ```

Jeducious commented

2021-11-03 03:23:31 +01:00

@ideasman42 Yeah, thats def on me :( apologies, I missed the fact I had references to our addon in that script. I will fix and re-post the script to remove references to our addon.

Brecht Van Lommel commented

2021-11-03 04:54:18 +01:00

The problem D12507 intended to solve is for the case where you click Save in the menu without opening a file browser. The suggested solution however moves the code to the file browser case, which undoes that fix and also may make the file menu appear in the screenshot.

Instead I suggest to move the call to WM_redraw_windows inside if (BLI_thread_is_main()) { just below, so it will not be called from the render thread.

The problem [D12507](https://archive.blender.org/developer/D12507) intended to solve is for the case where you click Save in the menu without opening a file browser. The suggested solution however moves the code to the file browser case, which undoes that fix and also may make the file menu appear in the screenshot. Instead I suggest to move the call to `WM_redraw_windows` inside `if (BLI_thread_is_main()) {` just below, so it will not be called from the render thread.

Jeducious commented

2021-11-03 05:15:33 +01:00

@brecht I'll make those changes in my branch and report back. Happy to submit a patch like that if it fixes the crash and [D12507 ](https://developer.blender.org/D12507) still has its intended effect.

@brecht I'll make those changes in my branch and report back. Happy to submit a patch like that if it fixes the crash and [[D12507](https://archive.blender.org/developer/D12507) ](https://developer.blender.org/D12507) still has its intended effect.

Jeducious commented

2021-11-03 12:00:41 +01:00

@brecht, @ideasman42 ok, I've reverted the change in my branch and implemented the suggestion to use if (BLI_thread_is_main()). The testing was a success with no more crashes and the thumbnails/screenshots taken when pressing save do not contain open menus in them, so I think I'll be submitting a branch with these changes as a patch for to close this issue, so long as it passes the review process :P

@brecht, @ideasman42 ok, I've reverted the change in my branch and implemented the suggestion to use `if (BLI_thread_is_main())`. The testing was a success with no more crashes and the thumbnails/screenshots taken when pressing save do not contain open menus in them, so I think I'll be submitting a branch with these changes as a patch for to close this issue, so long as it passes the review process :P

Brecht Van Lommel commented

2021-11-03 15:10:31 +01:00

Yes, a patch would be welcome.

Raimund Klink commented

2021-11-03 15:20:20 +01:00

Added subscriber: @Raimund58

Jeducious commented

2021-11-05 00:46:27 +01:00

@brecht Patch is incoming, I am getting my dev env setup, I was using the GitHub mirror, but after reading the guidelines on the wiki for blender development, seems you guys prefer arcanist? I just installed php and am working on installing arcanist tonight, then hopefully I'll be able to get setup for dev again using the official blender repository rather than the GitHub mirror.

Bay Raitt commented

2021-11-06 11:15:57 +01:00

Added subscriber: @spiraloid-3

Bay Raitt commented

2021-11-06 11:16:34 +01:00

Hitting this too. so glad to find this thread.

Jeducious commented

2021-11-07 03:03:40 +01:00

@spiraloid-3 Welcome :) Are you at all able to contribute by submitting a minimum test case for your particular situation? happy to test that on my patch to make sure it actually fixes your case as well :)

Jeducious self-assigned this 2021-11-08 01:52:35 +01:00

Jeducious commented

2021-11-08 01:52:35 +01:00

Guess I may as well claim this? I mean, I have created a differential for it now so I think that makes me the person responsible for it xD.

@brecht I've added you as reviewer, should I add anyone else?

Guess I may as well claim this? I mean, I have created a differential for it now so I think that makes me the person responsible for it xD. @brecht I've added you as reviewer, should I add anyone else?

Campbell Barton commented

2021-11-08 02:09:21 +01:00

@JamesCrowther could you update the test script? note that I tried modifying it and couldn't redo the crash in this report.

Jeducious commented

2021-11-09 00:46:36 +01:00

@ideasman42 I tried, and I failed, urhghghg 😵 .

Ok, the test script I wrote appears to have crashed because it was using the same properties as the addon. On its own the test script seems to not crash when Crowdrender is disabled.

So I went with the simpler option of fixing up the addon itself so you can pretty please try testing with that, its attached below, maybe give it a try? I used factory startup and enabled this copy of the addon and I got it to crash. Happy to update the task description too, just thought it might be better to verify this works (um crashes) for you first, otherwise I have more work to do.

To reproduce,
0. On windows 10 64 bit, Start Blender 3.1 alpha or 3.0 beta in factory startup mode (step zero cause I initially missed this step out on the instructions 😝 ). Default scene is fine

install the addon
enable the addon
Go to render properties panel in Blender and select "crowdrender" as the render engine from the drop down menu where cycles/eevee normally are
A new panel called "Crowdrender" should appear with a single button labelled "START", press start
Press F12 to render the current scene, Blender should crash with an access violation. The crash dump should point to the wm_file_write method where it calls the redraw of the UI.

crowdrender.zip

@ideasman42 I tried, and I failed, urhghghg 😵 . Ok, the test script I wrote appears to have crashed because it was using the same properties as the addon. On its own the test script seems to not crash when Crowdrender is disabled. So I went with the simpler option of fixing up the addon itself so you can pretty please try testing with that, its attached below, maybe give it a try? I used factory startup and enabled this copy of the addon and I got it to crash. Happy to update the task description too, just thought it might be better to verify this works (um crashes) for you first, otherwise I have more work to do. To reproduce, 0. On windows 10 64 bit, Start Blender 3.1 alpha or 3.0 beta in factory startup mode (step zero cause I initially missed this step out on the instructions 😝 ). Default scene is fine 1. install the addon 2. enable the addon 3. Go to render properties panel in Blender and select "crowdrender" as the render engine from the drop down menu where cycles/eevee normally are 4. A new panel called "Crowdrender" should appear with a single button labelled "START", press start 5. Press F12 to render the current scene, Blender should crash with an access violation. The crash dump should point to the wm_file_write method where it calls the redraw of the UI. [crowdrender.zip](https://archive.blender.org/developer/F11770850/crowdrender.zip)

blender-admin commented

2021-11-09 05:13:40 +01:00

This issue was referenced by d5d97e4169

This issue was referenced by d5d97e41699982110747d252daff427ae96a0f48

blender-admin commented

2021-11-09 05:13:40 +01:00

This issue was referenced by afc60f9957

This issue was referenced by afc60f995701ec6d41a05823b248db624ea1fcc6

Campbell Barton commented

2021-11-09 05:22:05 +01:00

Changed status from 'Needs Triage' to: 'Resolved'

Campbell Barton closed this issue

2021-11-09 05:22:05 +01:00

Jeducious commented

2021-11-10 00:17:50 +01:00

Thanks @ideasman42 & @brecht 😄 appreciate the help on getting this patch in.

Bay Raitt commented

2021-12-08 07:46:04 +01:00

@JamesCrowther man I had to chase this for ages to get a repro. I wrote an addon to do it. this addon loads a template file with a camera in it and tries to save it to a new file. it has something to do with just the presence of a camera in the loaded template scene.

it's not the camera itself (I've rebuilt the camera from scratch) it's just any camera in the loaded scene will crash. scratching my head. I included a blank new scene (no camera) that works just fine. to toggle between them just comment/uncomment the crashme.blend and crashmenot.blend lines (or add a camera to a blank scene as save over the crashme.blend)

such an odd bug. curious if anyone can figure out why this would crash blender 3.0

why would just the presence of a brand new camera in the template scene cause the crash....

(btw I also reset to factory defaults, deactivated all addons, updated startup file. same crash.)

addon installs normally and puts two blend files next to it. comments on the important bits. (you also need to define the filepath variable location to put the new folder in)

crashme.zip

@JamesCrowther man I had to chase this for ages to get a repro. I wrote an addon to do it. this addon loads a template file with a camera in it and tries to save it to a new file. it has something to do with just the presence of a camera in the loaded template scene. it's not the camera itself (I've rebuilt the camera from scratch) it's just any camera in the loaded scene will crash. scratching my head. I included a blank new scene (no camera) that works just fine. to toggle between them just comment/uncomment the crashme.blend and crashmenot.blend lines (or add a camera to a blank scene as save over the crashme.blend) such an odd bug. curious if anyone can figure out why this would crash blender 3.0 why would just the presence of a brand new camera in the template scene cause the crash.... (btw I also reset to factory defaults, deactivated all addons, updated startup file. same crash.) ![image.png](https://archive.blender.org/developer/F12708728/image.png) addon installs normally and puts two blend files next to it. comments on the important bits. (you also need to define the filepath variable location to put the new folder in) [crashme.zip](https://archive.blender.org/developer/F12708723/crashme.zip)

Brecht Van Lommel commented

2021-12-08 14:30:35 +01:00

@spiraloid-3, please create a new bug report.