windows/mantaflow/clang: Fatal Python error: Python memory allocator called without holding the GIL #72612

New Issue

Ray molenkamp · 2019-12-20T18:38:50+01:00

Ray molenkamp commented

2019-12-20 18:38:50 +01:00

System Information
Operating system: Windows-10 x64 - clang build
Graphics card: gtx1660

Blender Version
Broken: master 5804f18b1f
Worked: (optional)

Short description of error
When running a clang build with mantaflow enabled, blender crashes. when doing a debug build it asserts the following error

Fatal Python error: Python memory allocator called without holding the GIL

stack trace:

KernelBase.dll!00007ff9fd160192()
python37_d.dll!fatal_error(const char * prefix, const char * msg, int status) Line 2187
python37_d.dll!Py_FatalError(const char * msg) Line 2198
python37_d.dll!_PyMem_DebugCheckGIL() Line 2224
python37_d.dll!_PyMem_DebugMalloc(void * ctx, unsigned __int64 nbytes) Line 2230
python37_d.dll!PyObject_Malloc(unsigned __int64 size) Line 634
python37_d.dll!_PyBytes_FromSize(__int64 size, int use_calloc) Line 85
python37_d.dll!PyBytes_FromStringAndSize(const char * str, __int64 size) Line 119
python37_d.dll!_PyUnicode_AsUTF8String(_object * unicode, const char * errors) Line 5197
python37_d.dll!PyUnicode_AsUTF8String(_object * unicode) Line 5237
blender.exe!pyObjectToString(_object * inputObject) Line 2031
blender.exe!MANTA::setPointers<int>(std::vector<std::tuple<int **,std::string,std::string,bool>,std::allocator<std::tuple<int **,std::string,std::string,bool>>> objects) Line 2488
blender.exe!MANTA::updatePointers() Line 2659
blender.exe!MANTA::MANTA(int * res, FluidModifierData * mmd) Line 250
blender.exe!manta_init(int * res, FluidModifierData * mmd) Line 32
blender.exe!BKE_fluid_reallocate_fluid(FluidDomainSettings * mds, int * res, int free_old) Line 118
blender.exe!BKE_fluid_modifier_init(FluidModifierData * mmd, Depsgraph * depsgraph, Object * ob, Scene * scene, Mesh * me) Line 542
blender.exe!BKE_fluid_modifier_processDomain(FluidModifierData * mmd, Depsgraph * depsgraph, Scene * scene, Object * ob, Mesh * me, const int scene_framenr) Line 3563
blender.exe!BKE_fluid_modifier_process(FluidModifierData * mmd, Depsgraph * depsgraph, Scene * scene, Object * ob, Mesh * me) Line 3841
blender.exe!BKE_fluid_modifier_do(FluidModifierData * mmd, Depsgraph * depsgraph, Scene * scene, Object * ob, Mesh * me) Line 3856
blender.exe!applyModifier(ModifierData * md, const ModifierEvalContext * ctx, Mesh * me) Line 120
blender.exe!modwrap_applyModifier(ModifierData * md, const ModifierEvalContext * ctx, Mesh * me) Line 961
blender.exe!mesh_calc_modifiers(Depsgraph * depsgraph, Scene * scene, Object * ob, int useDeform, const bool need_mapping, const CustomData_MeshMasks * dataMask, const int index, const bool use_cache, const bool allow_shared_mesh, Mesh * * r_deform, Mesh * * r_final) Line 1195
blender.exe!mesh_build_data(Depsgraph * depsgraph, Scene * scene, Object * ob, const CustomData_MeshMasks * dataMask, const bool need_mapping) Line 1836
blender.exe!makeDerivedMesh(Depsgraph * depsgraph, Scene * scene, Object * ob, BMEditMesh * em, const CustomData_MeshMasks * dataMask) Line 1950
blender.exe!BKE_object_handle_data_update(Depsgraph * depsgraph, Scene * scene, Object * ob) Line 189
blender.exe!BKE_object_eval_uber_data(Depsgraph * depsgraph, Scene * scene, Object * ob) Line 359
blender.exe!std::_Invoker_functor::_Call<void (*&)(struct Depsgraph *,struct Scene *,struct Object *),struct Depsgraph *,struct Scene * &,struct Object * &>(void (*&)(struct Depsgraph *,struct Scene *,struct Object *),struct Depsgraph * &&,struct Scene * &,struct Object * &)
blender.exe!std::invoke<void (*&)(struct Depsgraph *,struct Scene *,struct Object *),struct Depsgraph *,struct Scene * &,struct Object * &>(void (*&)(struct Depsgraph *,struct Scene *,struct Object *),struct Depsgraph * &&,struct Scene * &,struct Object * &)
blender.exe!std::_Invoker_ret<struct std::_Unforced,0>::_Call<void (*&)(struct Depsgraph *,struct Scene *,struct Object *),struct Depsgraph *,struct Scene * &,struct Object * &>(void (*&)(struct Depsgraph *,struct Scene *,struct Object *),struct Depsgraph * &&,struct Scene * &,struct Object * &)
blender.exe!std::_Call_binder<std::_Unforced,0,1,2,void (*)(Depsgraph *, Scene *, Object *),std::tuple<std::_Ph<1>,Scene *,Object *>,std::tuple<Depsgraph *&&>>(std::_Invoker_ret<std::_Unforced,0>, std::integer_sequence<unsigned long long,0,1,2>, void(*)(Depsgraph *, Scene *, Object *) & _Obj, std::tuple<std::_Ph<1>,Scene *,Object *> & _Tpl, std::tuple<Depsgraph *&&> && _Ut) Line 1858
blender.exe!std::_Binder<std::_Unforced,void (&)(Depsgraph *, Scene *, Object *),const std::_Ph<1> &,Scene *&,Object *&>::operator()<Depsgraph *>(Depsgraph * && _Unbargs) Line 1914
blender.exe!std::_Invoker_functor::_Call<class std::_Binder<struct std::_Unforced,void (&)(struct Depsgraph *,struct Scene *,struct Object *),struct std::_Ph<1> const &,struct Scene * &,struct Object * &> &,struct Depsgraph *>(class std::_Binder<struct std::_Unforced,void (&)(struct Depsgraph *,struct Scene *,struct Object *),struct std::_Ph<1> const &,struct Scene * &,struct Object * &> &,struct Depsgraph * &&)
blender.exe!std::invoke<class std::_Binder<struct std::_Unforced,void (&)(struct Depsgraph *,struct Scene *,struct Object *),struct std::_Ph<1> const &,struct Scene * &,struct Object * &> &,struct Depsgraph *>(class std::_Binder<struct std::_Unforced,void (&)(struct Depsgraph *,struct Scene *,struct Object *),struct std::_Ph<1> const &,struct Scene * &,struct Object * &> &,struct Depsgraph * &&)
blender.exe!std::_Invoker_ret<void,1>::_Call<class std::_Binder<struct std::_Unforced,void (&)(struct Depsgraph *,struct Scene *,struct Object *),struct std::_Ph<1> const &,struct Scene * &,struct Object * &> &,struct Depsgraph *>(class std::_Binder<struct std::_Unforced,void (&)(struct Depsgraph *,struct Scene *,struct Object *),struct std::_Ph<1> const &,struct Scene * &,struct Object * &> &,struct Depsgraph * &&)
blender.exe!std::_Func_impl_no_alloc<class std::_Binder<struct std::_Unforced,void (&)(struct Depsgraph *,struct Scene *,struct Object *),struct std::_Ph<1> const &,struct Scene * &,struct Object * &>,void,struct Depsgraph *>::_Do_call(struct Depsgraph * &&)
blender.exe!std::_Func_class<void,struct Depsgraph *>::operator()(struct Depsgraph *)
blender.exe!DEG::deg_task_run_func(TaskPool * pool, void * taskdata, int thread_id) Line 89
[Inline Frame] blender.exe!handle_local_queue(TaskThreadLocalStorage * tls, const int thread_id) Line 420
blender.exe!task_scheduler_thread_run(void * thread_p) Line 457
blender.exe!__ptw32_threadStart()
ucrtbased.dll!00007ff9c80f542c()
kernel32.dll!00007ff9fe127bd4()
ntdll.dll!00007ff9ff7aced1()

this seems to fix it, but i don't know the Python integration well enough to know if it is a bad fix or not, @ideasman42 may have some insights there.

diff --git a/intern/mantaflow/intern/MANTA_main.cpp b/intern/mantaflow/intern/MANTA_main.cpp
index 0828fae5351..27eef331356 100644
--- a/intern/mantaflow/intern/MANTA_main.cpp
+++ b/intern/mantaflow/intern/MANTA_main.cpp
@@ -2476,7 +2476,8 @@ template<typename T>
 void MANTA::setPointers(std::vector<std::tuple<T **, std::string, std::string, bool>> objects)
 {
   PyObject *mantaObject = NULL;
-
+  PyGILState_STATE gstate;
+  gstate = PyGILState_Ensure();
   for (typename std::vector<std::tuple<T **, std::string, std::string, bool>>::iterator it =
            objects.begin();
        it != objects.end();
@@ -2491,6 +2492,7 @@ void MANTA::setPointers(std::vector<std::tuple<T **, std::string, std::string, b
       (*std::get<0>(*it)) = NULL;
     }
   }
+  PyGILState_Release(gstate);
 }```


**Exact steps for others to reproduce the error**
- Build with clang on windows (regular msvc build seems fine)
- Start blender with factory defaults
- Add quick smoke
- Bake
- Poof

**System Information** Operating system: Windows-10 x64 - clang build Graphics card: gtx1660 **Blender Version** Broken: master 5804f18b1f Worked: (optional) **Short description of error** When running a clang build with mantaflow enabled, blender crashes. when doing a debug build it asserts the following error `Fatal Python error: Python memory allocator called without holding the GIL` stack trace: ``` KernelBase.dll!00007ff9fd160192() python37_d.dll!fatal_error(const char * prefix, const char * msg, int status) Line 2187 python37_d.dll!Py_FatalError(const char * msg) Line 2198 python37_d.dll!_PyMem_DebugCheckGIL() Line 2224 python37_d.dll!_PyMem_DebugMalloc(void * ctx, unsigned __int64 nbytes) Line 2230 python37_d.dll!PyObject_Malloc(unsigned __int64 size) Line 634 python37_d.dll!_PyBytes_FromSize(__int64 size, int use_calloc) Line 85 python37_d.dll!PyBytes_FromStringAndSize(const char * str, __int64 size) Line 119 python37_d.dll!_PyUnicode_AsUTF8String(_object * unicode, const char * errors) Line 5197 python37_d.dll!PyUnicode_AsUTF8String(_object * unicode) Line 5237 blender.exe!pyObjectToString(_object * inputObject) Line 2031 blender.exe!MANTA::setPointers<int>(std::vector<std::tuple<int **,std::string,std::string,bool>,std::allocator<std::tuple<int **,std::string,std::string,bool>>> objects) Line 2488 blender.exe!MANTA::updatePointers() Line 2659 blender.exe!MANTA::MANTA(int * res, FluidModifierData * mmd) Line 250 blender.exe!manta_init(int * res, FluidModifierData * mmd) Line 32 blender.exe!BKE_fluid_reallocate_fluid(FluidDomainSettings * mds, int * res, int free_old) Line 118 blender.exe!BKE_fluid_modifier_init(FluidModifierData * mmd, Depsgraph * depsgraph, Object * ob, Scene * scene, Mesh * me) Line 542 blender.exe!BKE_fluid_modifier_processDomain(FluidModifierData * mmd, Depsgraph * depsgraph, Scene * scene, Object * ob, Mesh * me, const int scene_framenr) Line 3563 blender.exe!BKE_fluid_modifier_process(FluidModifierData * mmd, Depsgraph * depsgraph, Scene * scene, Object * ob, Mesh * me) Line 3841 blender.exe!BKE_fluid_modifier_do(FluidModifierData * mmd, Depsgraph * depsgraph, Scene * scene, Object * ob, Mesh * me) Line 3856 blender.exe!applyModifier(ModifierData * md, const ModifierEvalContext * ctx, Mesh * me) Line 120 blender.exe!modwrap_applyModifier(ModifierData * md, const ModifierEvalContext * ctx, Mesh * me) Line 961 blender.exe!mesh_calc_modifiers(Depsgraph * depsgraph, Scene * scene, Object * ob, int useDeform, const bool need_mapping, const CustomData_MeshMasks * dataMask, const int index, const bool use_cache, const bool allow_shared_mesh, Mesh * * r_deform, Mesh * * r_final) Line 1195 blender.exe!mesh_build_data(Depsgraph * depsgraph, Scene * scene, Object * ob, const CustomData_MeshMasks * dataMask, const bool need_mapping) Line 1836 blender.exe!makeDerivedMesh(Depsgraph * depsgraph, Scene * scene, Object * ob, BMEditMesh * em, const CustomData_MeshMasks * dataMask) Line 1950 blender.exe!BKE_object_handle_data_update(Depsgraph * depsgraph, Scene * scene, Object * ob) Line 189 blender.exe!BKE_object_eval_uber_data(Depsgraph * depsgraph, Scene * scene, Object * ob) Line 359 blender.exe!std::_Invoker_functor::_Call<void (*&)(struct Depsgraph *,struct Scene *,struct Object *),struct Depsgraph *,struct Scene * &,struct Object * &>(void (*&)(struct Depsgraph *,struct Scene *,struct Object *),struct Depsgraph * &&,struct Scene * &,struct Object * &) blender.exe!std::invoke<void (*&)(struct Depsgraph *,struct Scene *,struct Object *),struct Depsgraph *,struct Scene * &,struct Object * &>(void (*&)(struct Depsgraph *,struct Scene *,struct Object *),struct Depsgraph * &&,struct Scene * &,struct Object * &) blender.exe!std::_Invoker_ret<struct std::_Unforced,0>::_Call<void (*&)(struct Depsgraph *,struct Scene *,struct Object *),struct Depsgraph *,struct Scene * &,struct Object * &>(void (*&)(struct Depsgraph *,struct Scene *,struct Object *),struct Depsgraph * &&,struct Scene * &,struct Object * &) blender.exe!std::_Call_binder<std::_Unforced,0,1,2,void (*)(Depsgraph *, Scene *, Object *),std::tuple<std::_Ph<1>,Scene *,Object *>,std::tuple<Depsgraph *&&>>(std::_Invoker_ret<std::_Unforced,0>, std::integer_sequence<unsigned long long,0,1,2>, void(*)(Depsgraph *, Scene *, Object *) & _Obj, std::tuple<std::_Ph<1>,Scene *,Object *> & _Tpl, std::tuple<Depsgraph *&&> && _Ut) Line 1858 blender.exe!std::_Binder<std::_Unforced,void (&)(Depsgraph *, Scene *, Object *),const std::_Ph<1> &,Scene *&,Object *&>::operator()<Depsgraph *>(Depsgraph * && _Unbargs) Line 1914 blender.exe!std::_Invoker_functor::_Call<class std::_Binder<struct std::_Unforced,void (&)(struct Depsgraph *,struct Scene *,struct Object *),struct std::_Ph<1> const &,struct Scene * &,struct Object * &> &,struct Depsgraph *>(class std::_Binder<struct std::_Unforced,void (&)(struct Depsgraph *,struct Scene *,struct Object *),struct std::_Ph<1> const &,struct Scene * &,struct Object * &> &,struct Depsgraph * &&) blender.exe!std::invoke<class std::_Binder<struct std::_Unforced,void (&)(struct Depsgraph *,struct Scene *,struct Object *),struct std::_Ph<1> const &,struct Scene * &,struct Object * &> &,struct Depsgraph *>(class std::_Binder<struct std::_Unforced,void (&)(struct Depsgraph *,struct Scene *,struct Object *),struct std::_Ph<1> const &,struct Scene * &,struct Object * &> &,struct Depsgraph * &&) blender.exe!std::_Invoker_ret<void,1>::_Call<class std::_Binder<struct std::_Unforced,void (&)(struct Depsgraph *,struct Scene *,struct Object *),struct std::_Ph<1> const &,struct Scene * &,struct Object * &> &,struct Depsgraph *>(class std::_Binder<struct std::_Unforced,void (&)(struct Depsgraph *,struct Scene *,struct Object *),struct std::_Ph<1> const &,struct Scene * &,struct Object * &> &,struct Depsgraph * &&) blender.exe!std::_Func_impl_no_alloc<class std::_Binder<struct std::_Unforced,void (&)(struct Depsgraph *,struct Scene *,struct Object *),struct std::_Ph<1> const &,struct Scene * &,struct Object * &>,void,struct Depsgraph *>::_Do_call(struct Depsgraph * &&) blender.exe!std::_Func_class<void,struct Depsgraph *>::operator()(struct Depsgraph *) blender.exe!DEG::deg_task_run_func(TaskPool * pool, void * taskdata, int thread_id) Line 89 [Inline Frame] blender.exe!handle_local_queue(TaskThreadLocalStorage * tls, const int thread_id) Line 420 blender.exe!task_scheduler_thread_run(void * thread_p) Line 457 blender.exe!__ptw32_threadStart() ucrtbased.dll!00007ff9c80f542c() kernel32.dll!00007ff9fe127bd4() ntdll.dll!00007ff9ff7aced1() ``` this seems to fix it, but i don't know the Python integration well enough to know if it is a bad fix or not, @ideasman42 may have some insights there. ``` diff --git a/intern/mantaflow/intern/MANTA_main.cpp b/intern/mantaflow/intern/MANTA_main.cpp index 0828fae5351..27eef331356 100644 --- a/intern/mantaflow/intern/MANTA_main.cpp +++ b/intern/mantaflow/intern/MANTA_main.cpp @@ -2476,7 +2476,8 @@ template<typename T> void MANTA::setPointers(std::vector<std::tuple<T **, std::string, std::string, bool>> objects) { PyObject *mantaObject = NULL; - + PyGILState_STATE gstate; + gstate = PyGILState_Ensure(); for (typename std::vector<std::tuple<T **, std::string, std::string, bool>>::iterator it = objects.begin(); it != objects.end(); @@ -2491,6 +2492,7 @@ void MANTA::setPointers(std::vector<std::tuple<T **, std::string, std::string, b (*std::get<0>(*it)) = NULL; } } + PyGILState_Release(gstate); }``` **Exact steps for others to reproduce the error** - Build with clang on windows (regular msvc build seems fine) - Start blender with factory defaults - Add quick smoke - Bake - Poof

Ray molenkamp commented

2019-12-20 18:38:50 +01:00

Added subscribers: @ideasman42, @LazyDodo

Sebastián Barschkis was assigned by Ray molenkamp

2019-12-20 18:40:38 +01:00

Sebastián Barschkis commented

2019-12-21 00:39:39 +01:00

@LazyDodo Ah, I see, interesting that this pops up on windows. Just committed 4d0a91d03d which ensures that the GIL get acquired (similar to what you did).
Can you check if it fixes the issue?

@LazyDodo Ah, I see, interesting that this pops up on windows. Just committed 4d0a91d03d1c which ensures that the GIL get acquired (similar to what you did). Can you check if it fixes the issue?

Ray molenkamp commented

2019-12-21 02:49:12 +01:00

Changed status from 'Open' to: 'Resolved'

Ray molenkamp closed this issue

2019-12-21 02:49:12 +01:00

Ray molenkamp commented

2019-12-21 02:49:12 +01:00

Can confirm it fixes this one, there's a bunch of other pointer errors, but i'll bug you separately about those. (if you can't wait, for proper reports, try doing an asan build, it should catch a bunch of these )

Sign in to join this conversation.

No Label

Download

What's New

Blender Studio

Manual

Developers Blog

Documentation

Benchmark

Blender Conference

Development Fund

One-time Donations

windows/mantaflow/clang: Fatal Python error: Python memory allocator called without holding the GIL #72612