Standalone bpy module crashes Python on exit #125376

New Issue

Andrew Wason · 2024-07-24T18:37:35+02:00

Andrew Wason commented

2024-07-24 18:37:35 +02:00

System Information
Operating system: macOS-14.5-arm64-arm-64bit 64 Bits
Graphics card: Metal API Apple M2 Pro 1.2

Blender Version
Broken: fd4fb94751d100af967cf0b2743a1fbf33a6fc8f
Worked: 4.2.0

Short description of error
Using the standalone bpy python module can crash Python on exit because the PyModuleDef.m_free handler is not invoked.

Exact steps for others to reproduce the error
Blender keeps a static list of SpaceType instances in get_space_types()
Normally these are cleaned up in BKE_spacetypes_free() which is called from the bpy PyModuleDef.m_free handler.

In some cases, this m_free handler is not called at all and so the get_space_types() instances are automatically destructed at program exit, as part of their destruction they call into the python interpreter (PyGILState_Ensure()) and this crashes because the interpreter has already been torn down earlier.

A non-crashing example is:

import bpy
print(bpy.context.blend_data.cameras[0])

We can see the static SpaceTypes are destroyed normally from the PyModuleDef.m_free callback (bpy_module_free):

* thread #1, queue = 'com.apple.main-thread', stop reason = breakpoint 2.12
  * frame #0: 0x000000014ccaf684 __init__.so`SpaceType::~SpaceType(this=0x00000001048264b0) at screen.cc:208:1
    frame #1: 0x000000014ccafc1c __init__.so`std::__1::default_delete<SpaceType>::operator()[abi:ue170006](this=0x00000001048553f8, __ptr=0x00000001048264b0) const at unique_ptr.h:68:5
    frame #2: 0x000000014ccafb90 __init__.so`std::__1::unique_ptr<SpaceType, std::__1::default_delete<SpaceType>>::reset[abi:ue170006](this=0x00000001048553f8, __p=0x0000000000000000) at unique_ptr.h:300:7
    frame #3: 0x000000014ccafb18 __init__.so`std::__1::unique_ptr<SpaceType, std::__1::default_delete<SpaceType>>::~unique_ptr[abi:ue170006](this=0x00000001048553f8) at unique_ptr.h:266:75
    frame #4: 0x000000014ccafae8 __init__.so`std::__1::unique_ptr<SpaceType, std::__1::default_delete<SpaceType>>::~unique_ptr[abi:ue170006](this=0x00000001048553f8) at unique_ptr.h:266:73
    frame #5: 0x000000014ccafac0 __init__.so`void std::__1::__destroy_at[abi:ue170006]<std::__1::unique_ptr<SpaceType, std::__1::default_delete<SpaceType>>, 0>(__loc=0x00000001048553f8) at construct_at.h:69:13
    frame #6: 0x000000014ccafa78 __init__.so`std::__1::unique_ptr<SpaceType, std::__1::default_delete<SpaceType>>* std::__1::destroy_n[abi:ue170006]<std::__1::unique_ptr<SpaceType, std::__1::default_delete<SpaceType>>*, long long>(__first=0x00000001048553f8, __n=19) at construct_at.h:125:9
    frame #7: 0x000000014ccaf974 __init__.so`void blender::destruct_n<std::__1::unique_ptr<SpaceType, std::__1::default_delete<SpaceType>>>(ptr=0x00000001048553f8, n=19) at BLI_memory_utils.hh:42:3
    frame #8: 0x000000014ccaf774 __init__.so`blender::Vector<std::__1::unique_ptr<SpaceType, std::__1::default_delete<SpaceType>>, 4ll, blender::GuardedAllocator>::clear_and_shrink(this=0x0000000158cc92c8) at BLI_vector.hh:412:5
    frame #9: 0x000000014ccaf6a8 __init__.so`BKE_spacetypes_free() at screen.cc:238:21
    frame #10: 0x000000014c50ba70 __init__.so`BKE_blender_free() at blender.cc:67:3
    frame #11: 0x000000014d828ce8 __init__.so`WM_exit_ex(C=0x0000600002f9d0e8, do_python_exit=true, do_user_exit_actions=false) at wm_init_exit.cc:598:3
    frame #12: 0x000000014c3d8130 __init__.so`main_python_exit() at creator.cc:610:3
    frame #13: 0x000000014e4d3d80 __init__.so`bpy_module_free((null)=0x0000000100131fd0) at bpy_interface.cc:952:3
    frame #14: 0x00000001009ccf7c libpython3.11.dylib`module_dealloc + 124
    frame #15: 0x00000001009a673c libpython3.11.dylib`list_ass_slice + 412
    frame #16: 0x0000000100aa782c libpython3.11.dylib`_PyInterpreterState_ClearModules + 172
    frame #17: 0x0000000100aa3abc libpython3.11.dylib`finalize_modules + 2740
    frame #18: 0x0000000100aa2c8c libpython3.11.dylib`Py_FinalizeEx + 116
    frame #19: 0x0000000100ac88dc libpython3.11.dylib`Py_RunMain + 228
    frame #20: 0x0000000100ac92e0 libpython3.11.dylib`pymain_main + 324
    frame #21: 0x0000000100ac9380 libpython3.11.dylib`Py_BytesMain + 40
    frame #22: 0x00000001953560e0 dyld`start + 2360

A crashing example is:

import typing as ta
import bpy

T = ta.TypeVar("T")

class Wrapper(ta.Generic[T]):
    def item(self) -> T | None:
        return None

w: Wrapper[bpy.types.Camera]

We can see the static SpaceTypes are destroyed as the process exits - and this crashes since the Python interpreter has already been torn down earlier:

* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x28)
  * frame #0: 0x0000000100aa7298 libpython3.11.dylib`new_threadstate + 24
    frame #1: 0x0000000100aa8614 libpython3.11.dylib`PyGILState_Ensure + 84
    frame #2: 0x00000001464f6958 __init__.so`bpy_class_free(pyob_ptr=0x0000000104e97c90) at bpy_rna.cc:8964:14
    frame #3: 0x0000000144caf57c __init__.so`SpaceType::~SpaceType(this=0x000000010d7dc7a0) at screen.cc:217:9
    frame #4: 0x0000000144caf688 __init__.so`SpaceType::~SpaceType(this=0x000000010d7dc7a0) at screen.cc:208:1
    frame #5: 0x0000000144cafc1c __init__.so`std::__1::default_delete<SpaceType>::operator()[abi:ue170006](this=0x000000010490f788, __ptr=0x000000010d7dc7a0) const at unique_ptr.h:68:5
    frame #6: 0x0000000144cafb90 __init__.so`std::__1::unique_ptr<SpaceType, std::__1::default_delete<SpaceType>>::reset[abi:ue170006](this=0x000000010490f788, __p=0x0000000000000000) at unique_ptr.h:300:7
    frame #7: 0x0000000144cafb18 __init__.so`std::__1::unique_ptr<SpaceType, std::__1::default_delete<SpaceType>>::~unique_ptr[abi:ue170006](this=0x000000010490f788) at unique_ptr.h:266:75
    frame #8: 0x0000000144cafae8 __init__.so`std::__1::unique_ptr<SpaceType, std::__1::default_delete<SpaceType>>::~unique_ptr[abi:ue170006](this=0x000000010490f788) at unique_ptr.h:266:73
    frame #9: 0x0000000144cafac0 __init__.so`void std::__1::__destroy_at[abi:ue170006]<std::__1::unique_ptr<SpaceType, std::__1::default_delete<SpaceType>>, 0>(__loc=0x000000010490f788) at construct_at.h:69:13
    frame #10: 0x0000000144cafa78 __init__.so`std::__1::unique_ptr<SpaceType, std::__1::default_delete<SpaceType>>* std::__1::destroy_n[abi:ue170006]<std::__1::unique_ptr<SpaceType, std::__1::default_delete<SpaceType>>*, long long>(__first=0x000000010490f788, __n=19) at construct_at.h:125:9
    frame #11: 0x0000000144caf974 __init__.so`void blender::destruct_n<std::__1::unique_ptr<SpaceType, std::__1::default_delete<SpaceType>>>(ptr=0x000000010490f788, n=19) at BLI_memory_utils.hh:42:3
    frame #12: 0x0000000144caf90c __init__.so`blender::Vector<std::__1::unique_ptr<SpaceType, std::__1::default_delete<SpaceType>>, 4ll, blender::GuardedAllocator>::~Vector(this=0x0000000150cc92c8) at BLI_vector.hh:265:5
    frame #13: 0x0000000144caf828 __init__.so`blender::Vector<std::__1::unique_ptr<SpaceType, std::__1::default_delete<SpaceType>>, 4ll, blender::GuardedAllocator>::~Vector(this=0x0000000150cc92c8) at BLI_vector.hh:264:3
    frame #14: 0x000000019559c2e8 libsystem_c.dylib`__cxa_finalize_ranges + 476
    frame #15: 0x000000019559c070 libsystem_c.dylib`exit + 44
    frame #16: 0x00000001956f78d0 libdyld.dylib`dyld4::LibSystemHelpers::exit(int) const + 20
    frame #17: 0x000000019535612c dyld`start + 2436

Also running each of the above examples with python -v (which logs module initialization and destruction), we can see the non-crashing example logs:

# destroy _datetime
# destroy bpy
# destroy keyingsets_utils
# destroy numpy.core._exceptions

and the crashing example logs:

# destroy _datetime
# destroy numpy.core._exceptions

i.e. it is completely missing destroying the bpy module. The "# destroy ..." are logged from module_dealloc() and that is also where m_free is invoked. So Python is never deallocing the bpy module nor invoking it's m_free.

I git bisected and this started happening with this commit
It's not clear how that would cause this though.

**System Information** Operating system: macOS-14.5-arm64-arm-64bit 64 Bits Graphics card: Metal API Apple M2 Pro 1.2 **Blender Version** Broken: `fd4fb94751d100af967cf0b2743a1fbf33a6fc8f` Worked: `4.2.0` **Short description of error** Using the standalone `bpy` python module can crash Python on exit because the [PyModuleDef.m_free](https://docs.python.org/3.11/c-api/module.html#c.PyModuleDef.m_free) handler is not invoked. **Exact steps for others to reproduce the error** Blender keeps a static list of `SpaceType` instances in [get_space_types()](https://projects.blender.org/blender/blender/src/commit/97fc0d2716eaba57f4d69e73fd5c4eff945dbba7/source/blender/blenkernel/intern/screen.cc#L202) Normally these are cleaned up in `BKE_spacetypes_free()` which is called from the `bpy` [PyModuleDef.m_free](https://projects.blender.org/blender/blender/src/commit/97fc0d2716eaba57f4d69e73fd5c4eff945dbba7/source/blender/python/intern/bpy_interface.cc#L831) handler. In some cases, this `m_free` handler is not called at all and so the `get_space_types()` instances are automatically destructed at program exit, as part of their destruction they call into the python interpreter (`PyGILState_Ensure()`) and this crashes because the interpreter has already been torn down earlier. A non-crashing example is: ```python import bpy print(bpy.context.blend_data.cameras[0]) ``` We can see the static `SpaceType`s are destroyed normally from the `PyModuleDef.m_free` callback (`bpy_module_free`): ``` * thread #1, queue = 'com.apple.main-thread', stop reason = breakpoint 2.12 * frame #0: 0x000000014ccaf684 __init__.so`SpaceType::~SpaceType(this=0x00000001048264b0) at screen.cc:208:1 frame #1: 0x000000014ccafc1c __init__.so`std::__1::default_delete<SpaceType>::operator()[abi:ue170006](this=0x00000001048553f8, __ptr=0x00000001048264b0) const at unique_ptr.h:68:5 frame #2: 0x000000014ccafb90 __init__.so`std::__1::unique_ptr<SpaceType, std::__1::default_delete<SpaceType>>::reset[abi:ue170006](this=0x00000001048553f8, __p=0x0000000000000000) at unique_ptr.h:300:7 frame #3: 0x000000014ccafb18 __init__.so`std::__1::unique_ptr<SpaceType, std::__1::default_delete<SpaceType>>::~unique_ptr[abi:ue170006](this=0x00000001048553f8) at unique_ptr.h:266:75 frame #4: 0x000000014ccafae8 __init__.so`std::__1::unique_ptr<SpaceType, std::__1::default_delete<SpaceType>>::~unique_ptr[abi:ue170006](this=0x00000001048553f8) at unique_ptr.h:266:73 frame #5: 0x000000014ccafac0 __init__.so`void std::__1::__destroy_at[abi:ue170006]<std::__1::unique_ptr<SpaceType, std::__1::default_delete<SpaceType>>, 0>(__loc=0x00000001048553f8) at construct_at.h:69:13 frame #6: 0x000000014ccafa78 __init__.so`std::__1::unique_ptr<SpaceType, std::__1::default_delete<SpaceType>>* std::__1::destroy_n[abi:ue170006]<std::__1::unique_ptr<SpaceType, std::__1::default_delete<SpaceType>>*, long long>(__first=0x00000001048553f8, __n=19) at construct_at.h:125:9 frame #7: 0x000000014ccaf974 __init__.so`void blender::destruct_n<std::__1::unique_ptr<SpaceType, std::__1::default_delete<SpaceType>>>(ptr=0x00000001048553f8, n=19) at BLI_memory_utils.hh:42:3 frame #8: 0x000000014ccaf774 __init__.so`blender::Vector<std::__1::unique_ptr<SpaceType, std::__1::default_delete<SpaceType>>, 4ll, blender::GuardedAllocator>::clear_and_shrink(this=0x0000000158cc92c8) at BLI_vector.hh:412:5 frame #9: 0x000000014ccaf6a8 __init__.so`BKE_spacetypes_free() at screen.cc:238:21 frame #10: 0x000000014c50ba70 __init__.so`BKE_blender_free() at blender.cc:67:3 frame #11: 0x000000014d828ce8 __init__.so`WM_exit_ex(C=0x0000600002f9d0e8, do_python_exit=true, do_user_exit_actions=false) at wm_init_exit.cc:598:3 frame #12: 0x000000014c3d8130 __init__.so`main_python_exit() at creator.cc:610:3 frame #13: 0x000000014e4d3d80 __init__.so`bpy_module_free((null)=0x0000000100131fd0) at bpy_interface.cc:952:3 frame #14: 0x00000001009ccf7c libpython3.11.dylib`module_dealloc + 124 frame #15: 0x00000001009a673c libpython3.11.dylib`list_ass_slice + 412 frame #16: 0x0000000100aa782c libpython3.11.dylib`_PyInterpreterState_ClearModules + 172 frame #17: 0x0000000100aa3abc libpython3.11.dylib`finalize_modules + 2740 frame #18: 0x0000000100aa2c8c libpython3.11.dylib`Py_FinalizeEx + 116 frame #19: 0x0000000100ac88dc libpython3.11.dylib`Py_RunMain + 228 frame #20: 0x0000000100ac92e0 libpython3.11.dylib`pymain_main + 324 frame #21: 0x0000000100ac9380 libpython3.11.dylib`Py_BytesMain + 40 frame #22: 0x00000001953560e0 dyld`start + 2360 ``` A crashing example is: ```python import typing as ta import bpy T = ta.TypeVar("T") class Wrapper(ta.Generic[T]): def item(self) -> T | None: return None w: Wrapper[bpy.types.Camera] ``` We can see the static `SpaceType`s are destroyed as the process exits - and this crashes since the Python interpreter has already been torn down earlier: ``` * thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x28) * frame #0: 0x0000000100aa7298 libpython3.11.dylib`new_threadstate + 24 frame #1: 0x0000000100aa8614 libpython3.11.dylib`PyGILState_Ensure + 84 frame #2: 0x00000001464f6958 __init__.so`bpy_class_free(pyob_ptr=0x0000000104e97c90) at bpy_rna.cc:8964:14 frame #3: 0x0000000144caf57c __init__.so`SpaceType::~SpaceType(this=0x000000010d7dc7a0) at screen.cc:217:9 frame #4: 0x0000000144caf688 __init__.so`SpaceType::~SpaceType(this=0x000000010d7dc7a0) at screen.cc:208:1 frame #5: 0x0000000144cafc1c __init__.so`std::__1::default_delete<SpaceType>::operator()[abi:ue170006](this=0x000000010490f788, __ptr=0x000000010d7dc7a0) const at unique_ptr.h:68:5 frame #6: 0x0000000144cafb90 __init__.so`std::__1::unique_ptr<SpaceType, std::__1::default_delete<SpaceType>>::reset[abi:ue170006](this=0x000000010490f788, __p=0x0000000000000000) at unique_ptr.h:300:7 frame #7: 0x0000000144cafb18 __init__.so`std::__1::unique_ptr<SpaceType, std::__1::default_delete<SpaceType>>::~unique_ptr[abi:ue170006](this=0x000000010490f788) at unique_ptr.h:266:75 frame #8: 0x0000000144cafae8 __init__.so`std::__1::unique_ptr<SpaceType, std::__1::default_delete<SpaceType>>::~unique_ptr[abi:ue170006](this=0x000000010490f788) at unique_ptr.h:266:73 frame #9: 0x0000000144cafac0 __init__.so`void std::__1::__destroy_at[abi:ue170006]<std::__1::unique_ptr<SpaceType, std::__1::default_delete<SpaceType>>, 0>(__loc=0x000000010490f788) at construct_at.h:69:13 frame #10: 0x0000000144cafa78 __init__.so`std::__1::unique_ptr<SpaceType, std::__1::default_delete<SpaceType>>* std::__1::destroy_n[abi:ue170006]<std::__1::unique_ptr<SpaceType, std::__1::default_delete<SpaceType>>*, long long>(__first=0x000000010490f788, __n=19) at construct_at.h:125:9 frame #11: 0x0000000144caf974 __init__.so`void blender::destruct_n<std::__1::unique_ptr<SpaceType, std::__1::default_delete<SpaceType>>>(ptr=0x000000010490f788, n=19) at BLI_memory_utils.hh:42:3 frame #12: 0x0000000144caf90c __init__.so`blender::Vector<std::__1::unique_ptr<SpaceType, std::__1::default_delete<SpaceType>>, 4ll, blender::GuardedAllocator>::~Vector(this=0x0000000150cc92c8) at BLI_vector.hh:265:5 frame #13: 0x0000000144caf828 __init__.so`blender::Vector<std::__1::unique_ptr<SpaceType, std::__1::default_delete<SpaceType>>, 4ll, blender::GuardedAllocator>::~Vector(this=0x0000000150cc92c8) at BLI_vector.hh:264:3 frame #14: 0x000000019559c2e8 libsystem_c.dylib`__cxa_finalize_ranges + 476 frame #15: 0x000000019559c070 libsystem_c.dylib`exit + 44 frame #16: 0x00000001956f78d0 libdyld.dylib`dyld4::LibSystemHelpers::exit(int) const + 20 frame #17: 0x000000019535612c dyld`start + 2436 ``` Also running each of the above examples with `python -v` (which logs module initialization and destruction), we can see the non-crashing example logs: ``` # destroy _datetime # destroy bpy # destroy keyingsets_utils # destroy numpy.core._exceptions ``` and the crashing example logs: ``` # destroy _datetime # destroy numpy.core._exceptions ``` i.e. it is completely missing destroying the `bpy` module. The "# destroy ..." are logged from [module_dealloc()](https://github.com/python/cpython/blob/fda6bd842a2b93a501526f1b830eb900d935ac73/Objects/moduleobject.c#L775) and that is also where `m_free` is invoked. So Python is never deallocing the `bpy` module nor invoking it's `m_free`. I `git bisect`ed and this started happening with [this commit](https://projects.blender.org/blender/blender/commit/fd4fb94751d100af967cf0b2743a1fbf33a6fc8f) It's not clear how that would cause this though.

Andrew Wason added the

labels 2024-07-24 18:37:36 +02:00

Andrew Wason changed title from ~~Declaring generic Python type crashes bpy on exit~~ to Standalone `bpy` module crashes Python on exit

2024-08-01 23:27:11 +02:00

Richard Antalik commented

2024-08-05 09:46:11 +02:00

@ideasman42 Can you check?

Bart van der Braak added

and removed

labels 2024-08-14 12:55:00 +02:00

Philipp Oeser commented

2024-09-10 12:36:21 +02:00

Cannot repro on linux in 884d882155 (dont have my MAC with me today, but could this possibly be a MAC-only issue?)

Cannot repro on linux in 884d8821558b (dont have my MAC with me today, but could this possibly be a MAC-only issue?)

Andrew Wason commented

2024-09-10 20:31:03 +02:00

Cannot repro on linux in 884d882155 (dont have my MAC with me today, but could this possibly be a MAC-only issue?)

I suspect it could be (or at least my sample reproduces on macos, there may be other cases that repro on linux). The commit that triggered this enabled audaspace with standalone bpy, and the audio systems stuff is quite different on linux than mac.
I just tried with 488b5cda54 and am still able to repro on macos:

Assertion failed: (gilstate->autoInterpreterState), function PyGILState_Ensure, file pystate.c, line 1691.
Process 28937 stopped
* thread #1, queue = 'com.apple.main-thread', stop reason = hit program assert
    frame #4: 0x0000000100f87500 libpython3.11d.dylib`PyGILState_Ensure at pystate.c:1691:5
   1688	    /* Ensure that _PyEval_InitThreads() and _PyGILState_Init() have been
   1689	       called by Py_Initialize() */
   1690	    assert(_PyEval_ThreadsInitialized(runtime));
-> 1691	    assert(gilstate->autoInterpreterState);
   1692
   1693	    PyThreadState *tcur = (PyThreadState *)PyThread_tss_get(&gilstate->autoTSSkey);
   1694	    int current;
Target 0: (python) stopped.
(lldb) bt
* thread #1, queue = 'com.apple.main-thread', stop reason = hit program assert
    frame #0: 0x00000001840a55f0 libsystem_kernel.dylib`__pthread_kill + 8
    frame #1: 0x00000001840ddc20 libsystem_pthread.dylib`pthread_kill + 288
    frame #2: 0x0000000183feaa30 libsystem_c.dylib`abort + 180
    frame #3: 0x0000000183fe9d20 libsystem_c.dylib`__assert_rtn + 284
  * frame #4: 0x0000000100f87500 libpython3.11d.dylib`PyGILState_Ensure at pystate.c:1691:5
    frame #5: 0x00000003021a427c __init__.so`bpy_class_free(pyob_ptr=0x0000000139a3e780) at bpy_rna.cc:9039:14
    frame #6: 0x00000003008f31e8 __init__.so`SpaceType::~SpaceType(this=0x0000000159ebd510) at screen.cc:219:9
    frame #7: 0x00000003008f32f4 __init__.so`SpaceType::~SpaceType(this=0x0000000159ebd510) at screen.cc:210:1
    frame #8: 0x00000003008f3888 __init__.so`std::__1::default_delete<SpaceType>::operator()[abi:ue170006](this=0x0000000159eec458, __ptr=0x0000000159ebd510) const at unique_ptr.h:68:5
    frame #9: 0x00000003008f37fc __init__.so`std::__1::unique_ptr<SpaceType, std::__1::default_delete<SpaceType>>::reset[abi:ue170006](this=0x0000000159eec458, __p=0x0000000000000000) at unique_ptr.h:300:7
    frame #10: 0x00000003008f3784 __init__.so`std::__1::unique_ptr<SpaceType, std::__1::default_delete<SpaceType>>::~unique_ptr[abi:ue170006](this=0x0000000159eec458) at unique_ptr.h:266:75
    frame #11: 0x00000003008f3754 __init__.so`std::__1::unique_ptr<SpaceType, std::__1::default_delete<SpaceType>>::~unique_ptr[abi:ue170006](this=0x0000000159eec458) at unique_ptr.h:266:73
    frame #12: 0x00000003008f372c __init__.so`void std::__1::__destroy_at[abi:ue170006]<std::__1::unique_ptr<SpaceType, std::__1::default_delete<SpaceType>>, 0>(__loc=0x0000000159eec458) at construct_at.h:69:13
    frame #13: 0x00000003008f36e4 __init__.so`std::__1::unique_ptr<SpaceType, std::__1::default_delete<SpaceType>>* std::__1::destroy_n[abi:ue170006]<std::__1::unique_ptr<SpaceType, std::__1::default_delete<SpaceType>>*, long long>(__first=0x0000000159eec458, __n=19) at construct_at.h:125:9
    frame #14: 0x00000003008f35e0 __init__.so`void blender::destruct_n<std::__1::unique_ptr<SpaceType, std::__1::default_delete<SpaceType>>>(ptr=0x0000000159eec458, n=19) at BLI_memory_utils.hh:42:3
    frame #15: 0x00000003008f3578 __init__.so`blender::Vector<std::__1::unique_ptr<SpaceType, std::__1::default_delete<SpaceType>>, 4ll, blender::GuardedAllocator>::~Vector(this=0x000000030cb1fde8) at BLI_vector.hh:265:5
    frame #16: 0x00000003008f3494 __init__.so`blender::Vector<std::__1::unique_ptr<SpaceType, std::__1::default_delete<SpaceType>>, 4ll, blender::GuardedAllocator>::~Vector(this=0x000000030cb1fde8) at BLI_vector.hh:264:3
    frame #17: 0x0000000183f9b2e8 libsystem_c.dylib`__cxa_finalize_ranges + 476
    frame #18: 0x0000000183f9b070 libsystem_c.dylib`exit + 44
    frame #19: 0x00000001840f6850 libdyld.dylib`dyld4::LibSystemHelpers::exit(int) const + 20
    frame #20: 0x0000000183d531a0 dyld`start + 2552

> Cannot repro on linux in 884d8821558b (dont have my MAC with me today, but could this possibly be a MAC-only issue?) I suspect it could be (or at least my sample reproduces on macos, there may be other cases that repro on linux). The commit that triggered this enabled audaspace with standalone bpy, and the audio systems stuff is quite different on linux than mac. I just tried with https://projects.blender.org/blender/blender/commit/488b5cda5442c3067cc31d812fa430e0e9850ac9 and am still able to repro on macos: ``` Assertion failed: (gilstate->autoInterpreterState), function PyGILState_Ensure, file pystate.c, line 1691. Process 28937 stopped * thread #1, queue = 'com.apple.main-thread', stop reason = hit program assert frame #4: 0x0000000100f87500 libpython3.11d.dylib`PyGILState_Ensure at pystate.c:1691:5 1688 /* Ensure that _PyEval_InitThreads() and _PyGILState_Init() have been 1689 called by Py_Initialize() */ 1690 assert(_PyEval_ThreadsInitialized(runtime)); -> 1691 assert(gilstate->autoInterpreterState); 1692 1693 PyThreadState *tcur = (PyThreadState *)PyThread_tss_get(&gilstate->autoTSSkey); 1694 int current; Target 0: (python) stopped. (lldb) bt * thread #1, queue = 'com.apple.main-thread', stop reason = hit program assert frame #0: 0x00000001840a55f0 libsystem_kernel.dylib`__pthread_kill + 8 frame #1: 0x00000001840ddc20 libsystem_pthread.dylib`pthread_kill + 288 frame #2: 0x0000000183feaa30 libsystem_c.dylib`abort + 180 frame #3: 0x0000000183fe9d20 libsystem_c.dylib`__assert_rtn + 284 * frame #4: 0x0000000100f87500 libpython3.11d.dylib`PyGILState_Ensure at pystate.c:1691:5 frame #5: 0x00000003021a427c __init__.so`bpy_class_free(pyob_ptr=0x0000000139a3e780) at bpy_rna.cc:9039:14 frame #6: 0x00000003008f31e8 __init__.so`SpaceType::~SpaceType(this=0x0000000159ebd510) at screen.cc:219:9 frame #7: 0x00000003008f32f4 __init__.so`SpaceType::~SpaceType(this=0x0000000159ebd510) at screen.cc:210:1 frame #8: 0x00000003008f3888 __init__.so`std::__1::default_delete<SpaceType>::operator()[abi:ue170006](this=0x0000000159eec458, __ptr=0x0000000159ebd510) const at unique_ptr.h:68:5 frame #9: 0x00000003008f37fc __init__.so`std::__1::unique_ptr<SpaceType, std::__1::default_delete<SpaceType>>::reset[abi:ue170006](this=0x0000000159eec458, __p=0x0000000000000000) at unique_ptr.h:300:7 frame #10: 0x00000003008f3784 __init__.so`std::__1::unique_ptr<SpaceType, std::__1::default_delete<SpaceType>>::~unique_ptr[abi:ue170006](this=0x0000000159eec458) at unique_ptr.h:266:75 frame #11: 0x00000003008f3754 __init__.so`std::__1::unique_ptr<SpaceType, std::__1::default_delete<SpaceType>>::~unique_ptr[abi:ue170006](this=0x0000000159eec458) at unique_ptr.h:266:73 frame #12: 0x00000003008f372c __init__.so`void std::__1::__destroy_at[abi:ue170006]<std::__1::unique_ptr<SpaceType, std::__1::default_delete<SpaceType>>, 0>(__loc=0x0000000159eec458) at construct_at.h:69:13 frame #13: 0x00000003008f36e4 __init__.so`std::__1::unique_ptr<SpaceType, std::__1::default_delete<SpaceType>>* std::__1::destroy_n[abi:ue170006]<std::__1::unique_ptr<SpaceType, std::__1::default_delete<SpaceType>>*, long long>(__first=0x0000000159eec458, __n=19) at construct_at.h:125:9 frame #14: 0x00000003008f35e0 __init__.so`void blender::destruct_n<std::__1::unique_ptr<SpaceType, std::__1::default_delete<SpaceType>>>(ptr=0x0000000159eec458, n=19) at BLI_memory_utils.hh:42:3 frame #15: 0x00000003008f3578 __init__.so`blender::Vector<std::__1::unique_ptr<SpaceType, std::__1::default_delete<SpaceType>>, 4ll, blender::GuardedAllocator>::~Vector(this=0x000000030cb1fde8) at BLI_vector.hh:265:5 frame #16: 0x00000003008f3494 __init__.so`blender::Vector<std::__1::unique_ptr<SpaceType, std::__1::default_delete<SpaceType>>, 4ll, blender::GuardedAllocator>::~Vector(this=0x000000030cb1fde8) at BLI_vector.hh:264:3 frame #17: 0x0000000183f9b2e8 libsystem_c.dylib`__cxa_finalize_ranges + 476 frame #18: 0x0000000183f9b070 libsystem_c.dylib`exit + 44 frame #19: 0x00000001840f6850 libdyld.dylib`dyld4::LibSystemHelpers::exit(int) const + 20 frame #20: 0x0000000183d531a0 dyld`start + 2552 ```

Jesse Yurkovich commented

2024-09-10 20:40:57 +02:00

This crash is similar to that seen in #126807

Something weird is happening, somehow, with the typing module indeed.

This crash is similar to that seen in [#126807](https://projects.blender.org/blender/blender/issues/126807#issuecomment-1290367) Something weird is happening, somehow, with the `typing` module indeed.

Andrew Wason commented

2024-09-10 22:41:59 +02:00

The problem is somehow bpy module has one additional refcount (3 instead of 2) at finalization time, and so normally it ends up being decref'd and freed (bpy_module_free ends up calling BKE_spacetypes_free to free that static global list). But in this case it still has a ref and so we don't bpy_module_free, and so the static SpaceType list is freed at exit which is after the python interpreter has already shut down, and part of freeing that list attempts to call back into python which asserts/crashes.

I tried comparing the behavior before and after commit fd4fb94751 - but I couldn't figure out where that additional reference on bpy was coming from.

If you run python -v so it logs module import/cleanup and grep bpy you can see the non-crashing version logs an additional destroy bpy when the python interpreter destroys the module, in the crashing case it is never destroyed:

  # cleanup[3] wiping _bpy
  # destroy bpy.utils.units
  # destroy _bpy
  # cleanup[3] wiping _bpy_hydra
  # cleanup[3] wiping _bpy_path
+ # destroy bpy
  # destroy _bpy.ops
  # destroy bpy._utils_previews

The problem is somehow `bpy` module has one additional refcount (3 instead of 2) at finalization time, and so normally it ends up being decref'd and freed (`bpy_module_free` ends up calling `BKE_spacetypes_free` to free that static global list). But in this case it still has a ref and so we don't `bpy_module_free`, and so the static `SpaceType` list is freed at exit which is after the python interpreter has already shut down, and part of freeing that list attempts to call back into python which asserts/crashes. I tried comparing the behavior before and after commit https://projects.blender.org/blender/blender/commit/fd4fb94751d100af967cf0b2743a1fbf33a6fc8f - but I couldn't figure out where that additional reference on `bpy` was coming from. If you run `python -v` so it logs module import/cleanup and grep `bpy` you can see the non-crashing version logs an additional `destroy bpy` when the python interpreter destroys the module, in the crashing case it is never destroyed: ```diff # cleanup[3] wiping _bpy # destroy bpy.utils.units # destroy _bpy # cleanup[3] wiping _bpy_hydra # cleanup[3] wiping _bpy_path + # destroy bpy # destroy _bpy.ops # destroy bpy._utils_previews ```

Sign in to join this conversation.

No Label

Download

What's New

Blender Studio

Manual

Developers Blog

Documentation

Benchmark

Blender Conference

Development Fund

One-time Donations

Standalone bpy module crashes Python on exit #125376

Standalone `bpy` module crashes Python on exit #125376