Crash using 'ctypes' (fixed in Python >= 3.8), causes crashes in A.N.T. Landscape and various other addons #84752

New Issue

Simon I. · 2021-01-15T16:40:12+01:00

Simon I. commented

2021-01-15 16:40:12 +01:00

System Information
Operating system: Ubuntu 20.10, Linux-5.8.0-38-generic-x86_64-with-glibc2.17 64 Bits
Graphics card: Mesa DRI Intel(R) HD Graphics 620 (KBL GT2) Intel Open Source Technology Center 4.6 (Core Profile) Mesa 20.3.1 (git-d90622910b)
CPU: Intel Core i5-7200U

Not reproducible on Ubuntu 20.04.

Addon Information
Name: A.N.T.Landscape (0, 1, 8)
Author: Jimmy Hazevoet

Blender Version
Broken:

2.91.0, branch: master, commit date: 2020-11-25 08:34, hash: 0f45cab862, type: Release Snap and portable
2.92.0 Beta, branch: master, commit date: 2021-01-18 17:08, hash: b5c3f26cba, type: Release portable
2.92.0 Beta, branch: master, commit date: 2021-01-14 10:38, hash: 442b6e5e09, type: Release snap
Worked:
2.91.0, branch: master, commit date: 2020-11-25 08:34, hash: 0f45cab862, type: Release Flatpak

Short description of error
Whenever I want to enable the A.N.T. Landscape Addon by clicking the little box it crashes blender to desktop. This happens in the package installed via official Snap. It does work in the flatpak version however...

Exact steps for others to reproduce the error

Open Blender
Click on Edit -> Preferences
Go to Addons
Click the little checkbox next to Add Mesh: A.N.T. Landscape --> Crash

Analysis
The problem appears to be caused by the call to CFUNCTYPE(c_int)(lambda: None) by _reset_cache() in [version]/python/lib/python3.7/ctypes/__init__.py. This line appears to have been added as workaround for a failing test, but has since been removed from by the following commit:

It was originally added by commit:

674e9389e9

Removing the line in question prevents the crash.

Older information:

What I've tried so far but did'nt help

Delete user config directory
uninstall Blender and reinstall
enable other addons - no problem
delete scripts/addons/ant_landscape/__pycache__/*
compare system-info.txt of both versions - except for flatpak / snap specific file paths there identical (Build flags, Python version, other enabled addons etc.)

debugging information
I ran Blender with blender --debug --debug-gpu > ~/blender_debug_output.txt 2>&1 as recommended by @rjg file only contains:

Aborted (core dumped)

The core dump is nowhere to be found, neither the blender crash log. I looked through /tmp (only contains quit.blend file) and /var/crash, nothing there.
I ran blender again with blender --debug --debug-all > ~/blender_debug_output.txt 2>&1, here's the output:
blender_debug_all_output.txt
And again with blender --log "*" --log-level -1 --log-file ~/blender-log.txt, here's that:
blender-log.txt

Here's the Blender systems-info.txt:
system-info.txt

**System Information** Operating system: Ubuntu 20.10, Linux-5.8.0-38-generic-x86_64-with-glibc2.17 64 Bits Graphics card: Mesa DRI Intel(R) HD Graphics 620 (KBL GT2) Intel Open Source Technology Center 4.6 (Core Profile) Mesa 20.3.1 (git-d90622910b) CPU: Intel Core i5-7200U *Not reproducible on Ubuntu 20.04.* **Addon Information** Name: A.N.T.Landscape (0, 1, 8) Author: Jimmy Hazevoet **Blender Version** Broken: - 2.91.0, branch: master, commit date: 2020-11-25 08:34, hash: 0f45cab862b8, type: Release **Snap** and **portable** - 2.92.0 Beta, branch: master, commit date: 2021-01-18 17:08, hash: b5c3f26cba81, type: Release **portable** - 2.92.0 Beta, branch: master, commit date: 2021-01-14 10:38, hash: 442b6e5e0951, type: Release **snap** Worked: - 2.91.0, branch: master, commit date: 2020-11-25 08:34, hash: 0f45cab862b8, type: Release **Flatpak** **Short description of error** Whenever I want to enable the A.N.T. Landscape Addon by clicking the little box it crashes blender to desktop. This happens in the package installed via official Snap. It does work in the flatpak version however... **Exact steps for others to reproduce the error** 1. Open Blender 2. Click on Edit -> Preferences 3. Go to Addons 4. Click the little checkbox next to Add Mesh: A.N.T. Landscape --> Crash **Analysis** The problem appears to be caused by the call to `CFUNCTYPE(c_int)(lambda: None)` by `_reset_cache()` in `[version]/python/lib/python3.7/ctypes/__init__.py`. This line appears to have been added as workaround for a failing test, but has since been removed from by the following commit: - https://github.com/python/cpython/commit/e6b247c8e524dbe5fc03b3492f628d0d5348bc49#diff-edff4421a88bdc56335a84a1e5a0ce0eafe5cd0c37dcb1aff03ff075bfb9b57c - https://bugs.python.org/issue35523 It was originally added by commit: - https://github.com/python/cpython/commit/674e9389e9fdadd622829f4833367ac3b38475b5 Removing the line in question prevents the crash. ----- Older information: **What I've tried so far but did'nt help** - Delete user config directory - uninstall Blender and reinstall - enable other addons - no problem - delete `scripts/addons/ant_landscape/__pycache__/*` - compare `system-info.txt` of both versions - except for flatpak / snap specific file paths there identical (Build flags, Python version, other enabled addons etc.) **debugging information** I ran Blender with `blender --debug --debug-gpu > ~/blender_debug_output.txt 2>&1` as recommended by @rjg file only contains: ``` Aborted (core dumped) ``` The core dump is nowhere to be found, neither the blender crash log. I looked through `/tmp` (only contains quit.blend file) and `/var/crash`, nothing there. I ran blender again with `blender --debug --debug-all > ~/blender_debug_output.txt 2>&1`, here's the output: [blender_debug_all_output.txt](https://archive.blender.org/developer/F9581858/blender_debug_all_output.txt) And again with `blender --log "*" --log-level -1 --log-file ~/blender-log.txt`, here's that: [blender-log.txt](https://archive.blender.org/developer/F9581862/blender-log.txt) Here's the Blender systems-info.txt: [system-info.txt](https://archive.blender.org/developer/F9586981/system-info.txt)

Simon I. commented

2021-01-15 16:40:12 +01:00

Added subscriber: @Simon_I

blender-admin commented

2021-01-15 16:40:12 +01:00

#87242 was marked as duplicate of this issue

blender-admin commented

2021-01-15 16:40:12 +01:00

#83685 was marked as duplicate of this issue

blender-admin commented

2021-01-15 16:40:12 +01:00

blender/blender-addons#85022 was marked as duplicate of this issue

blender-admin commented

2021-01-15 16:40:12 +01:00

#84366 was marked as duplicate of this issue

blender-admin commented

2021-01-15 16:40:12 +01:00

blender/blender-addons#84537 was marked as duplicate of this issue

blender-admin commented

2021-01-15 16:40:12 +01:00

#84880 was marked as duplicate of this issue

Robert Guetzkow commented

2021-01-16 11:59:41 +01:00

Added subscriber: @rjg

Robert Guetzkow commented

2021-01-16 11:59:41 +01:00

Changed status from 'Needs Triage' to: 'Needs User Info'

Robert Guetzkow commented

2021-01-16 11:59:41 +01:00

I can't reproduce the problem. Can you please try to start Blender with the following command:

blender --debug --debug-gpu > ~/blender_debug_output.txt 2>&1

Please also check if Blender has written a crash log in the /tmp directory. Upload both the crash log and blender_debug_output.txt.

I can't reproduce the problem. Can you please try to start Blender with the following command: ``` blender --debug --debug-gpu > ~/blender_debug_output.txt 2>&1 ``` Please also check if Blender has written a crash log in the `/tmp` directory. Upload both the crash log and `blender_debug_output.txt`.

Robert Guetzkow commented

2021-01-16 14:19:34 +01:00

Changed status from 'Needs User Info' to: 'Needs Triage'

Scurest commented

2021-01-19 06:44:26 +01:00

Added subscriber: @scurest

Scurest commented

2021-01-19 06:44:26 +01:00

This is likely the same as blender/blender-addons#84537 since A.N.T. Landscape imports numpy.

Simon I. commented

2021-01-19 08:46:51 +01:00

@scurest I can confirm that the crash seems to be related. I opened the blender python consoles and did a import numpy, immediately crashes. I didn't see the other report blender/blender-addons#84537 when I opened this one, but it looks very much the same, thanks for pointing that out!

@scurest I can confirm that the crash seems to be related. I opened the blender python consoles and did a `import numpy`, immediately crashes. I didn't see the other report blender/blender-addons#84537 when I opened this one, but it looks very much the same, thanks for pointing that out!

Robert Guetzkow commented

2021-01-19 09:53:47 +01:00

@Simon_I Am I reading your report correctly that the issue also occurs in the official daily builds from https://builder.blender.org/download/, labeled "portable" in the ticket?

Simon I. commented

2021-01-19 09:53:56 +01:00

Here are other addons which make blender crash, too:

3D View: Precision Drawing Tools
Add Mesh: Extra Objects, A.N.T.Landscape, Archipack
Mesh: Tissue
Object: Carver, Grease Pencil Tools
Render: POV:
System: Manage UI Translations
All these addons import numpy at some point, except for Manage UI Translations, which crashes because it imports ctypes. In fact, numpy itself relies on importing ctypes lib, so this might be the actual problem

I tried running ctypes' __init__.py script from 2.92/python/lib/python3.7/ctypes in Blender's Python console, it crashes. I also tried all the imports from the script in Blender, it crashes when I do from ctypes._endian import BigEndianStructure, LittleEndianStructure (in __init__.py you'll find it in line 533), but ctypes._endian again does a from ctypes import *, so I don't really know where to look further...

Here are other addons which make blender crash, too: - 3D View: Precision Drawing Tools - Add Mesh: Extra Objects, A.N.T.Landscape, Archipack - Mesh: Tissue - Object: Carver, Grease Pencil Tools - Render: POV: - System: Manage UI Translations All these addons import numpy at some point, except for Manage UI Translations, which crashes because it imports ctypes. In fact, numpy itself relies on importing **ctypes** lib, so this might be the actual problem I tried running ctypes' `__init__.py` script from `2.92/python/lib/python3.7/ctypes` in Blender's Python console, it crashes. I also tried all the imports from the script in Blender, it crashes when I do `from ctypes._endian import BigEndianStructure, LittleEndianStructure` (in `__init__.py` you'll find it in line 533), but `ctypes._endian` again does a `from ctypes import *`, so I don't really know where to look further...

Simon I. commented

2021-01-19 09:55:48 +01:00

In #84752#1096308, @rjg wrote:
@Simon_I Am I reading your report correctly that the issue also occurs in the official daily builds from https://builder.blender.org/download/, labeled "portable" in the ticket?

Yep just tried again this morning with the 2.92.0 beta downloaded from blender.org --> Download Blender --> Download Blender Experimental

> In #84752#1096308, @rjg wrote: > @Simon_I Am I reading your report correctly that the issue also occurs in the official daily builds from https://builder.blender.org/download/, labeled "portable" in the ticket? Yep just tried again this morning with the 2.92.0 beta downloaded from blender.org --> Download Blender --> Download Blender Experimental

Simon I. changed title from ~~Blender Snap package crash when enabling A.N.T. Landscape addon~~ to Blender crash when enabling A.N.T. Landscape and various other addons that use ctypes

2021-01-19 09:57:57 +01:00

Robert Guetzkow commented

2021-01-19 10:37:20 +01:00

Thank your for looking into this. Could you please specify what processor you are using and attach your system information from Help > Save System Info, just in case this is hardware dependent?

Thank your for looking into this. Could you please specify what processor you are using and attach your system information from *Help > Save System Info*, just in case this is hardware dependent?

Robert Guetzkow commented

2021-01-19 10:37:26 +01:00

Changed status from 'Needs Triage' to: 'Needs User Info'

Simon I. commented

2021-01-19 10:47:08 +01:00

@rjg I added the Info in the report. Could also provide full lscpu output if that's of interest. But how could the flatpak version still work if that was a hardware related issue?

@rjg I added the Info in the report. Could also provide full `lscpu` output if that's of interest. But how could the flatpak version still work if that was a hardware related issue?

Simon I. commented

2021-01-19 10:54:12 +01:00

Here's another strange thing I've noticed: as stated above, Blender crashes when running import ctypes in it's Python console. If however I open Blender's Python interpreter from 2.92/python/bin/python3.7m in my operations system's terminal, I can import ctypes and numpy without having it crash or throwing an exception.

Here's another strange thing I've noticed: as stated above, Blender crashes when running `import ctypes` in it's Python console. If however I open Blender's Python interpreter from `2.92/python/bin/python3.7m` in my operations system's terminal, I can import ctypes and numpy without having it crash or throwing an exception.

Scurest commented

2021-01-19 10:58:51 +01:00

Can you sprinkle some print()s around ctype's __init__.py to find out exactly where it dies?

Can you sprinkle some `print()`s around ctype's `__init__.py` to find out exactly where it dies?

Robert Guetzkow commented

2021-01-19 11:16:31 +01:00

@Simon_I thanks for the info. That is indeed a curious case.

Could you please open Blender and get the content of sys.path and also check if any of the following environment variables have been set:

BLENDER_SYSTEM_PYTHON
BLENDER_SYSTEM_SCRIPTS
BLENDER_SYSTEM_DATAFILES
BLENDER_USER_SCRIPTS

import os

for key, value in os.environ.items():
    print(f"{key}: {value}")

Please check if there are any notable differences to the flatpak version.

@Simon_I thanks for the info. That is indeed a curious case. Could you please open Blender and get the content of `sys.path` and also check if any of the following environment variables have been set: - `BLENDER_SYSTEM_PYTHON` - `BLENDER_SYSTEM_SCRIPTS` - `BLENDER_SYSTEM_DATAFILES` - `BLENDER_USER_SCRIPTS` ``` import os for key, value in os.environ.items(): print(f"{key}: {value}") ``` Please check if there are any notable differences to the flatpak version.

Simon I. commented

2021-01-19 12:08:37 +01:00

@scurest I tried, but same result as before: script crashes at from ctypes._endian import BigEndianStructure, LittleEndianStructure I can however not debug that file any further by using prints / file writes, as it again imports ctypes itself, so _endian.py crashes at from ctypes import * which kinda brings back to where I started...

@rjg notable differences in os.environ.items():

Variable	Flatpak	Snap
SHELL:	/bin/sh	/bin/bash
XAUTHORITY:	/run/user/1000/Xauthority	/run/user/1000/.mutter-Xwaylandauth.1VGAX0
AT_SPI_BUS_ADDRESS:	unix:path=/run/user/1000/at-spi-bus	-
XDG_CACHE_HOME:	~/.var/app/org.blender.Blender/cache	-
	Couple of entries for GJS_DEBUG...	-
GI_TYPELIB_PATH:	/app/lib/girepository-1.0	-
SHLVL	0	1
GST_PLUGIN_SYSTEM_PATH:	/app/lib/gstreamer-1.0:/usr/lib/extensions/gstreamer-1.0:/usr/lib/x86_64-linux-gnu/gstreamer-1.0	-
JOURNAL_STREAM:	8:487937	-
PATH:	/app/bin:/usr/bin	~/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin
LESSOPEN:	-	[pipe character] /usr/bin/lesspipe %s
LESSCLOSE:	-	/usr/bin/lesspipe %s %s
COLORTERM:	-	truecolor
TERM:	-	xterm-256color
DISPLAY:	:99.0	:0
VTE_VERSION:	-	6200

(note that I use Gnome on wayland, but crash also happened on X11 session)

Here's Flatpaks sys.path():
['/app/blender/2.91/scripts/startup', '/app/blender/2.91/scripts/modules', '/app/blender/2.91/python/lib/python37.zip', '/app/blender/2.91/python/lib/python3.7', '/app/blender/2.91/python/lib/python3.7/lib-dynload', '/app/blender/2.91/python/lib/python3.7/site-packages', '/app/blender/2.91/scripts/freestyle/modules', '/app/blender/2.91/scripts/addons/modules', '~/.var/app/org.blender.Blender/config/blender/2.91/scripts/addons/modules', '/app/blender/2.91/scripts/addons']

Here's Snaps sys.path():
['/snap/blender/63/2.92/scripts/startup', '/snap/blender/63/2.92/scripts/modules', '/snap/blender/63/2.92/python/lib/python37.zip', '/snap/blender/63/2.92/python/lib/python3.7', '/snap/blender/63/2.92/python/lib/python3.7/lib-dynload', '/snap/blender/63/2.92/python/lib/python3.7/site-packages', '/snap/blender/63/2.92/scripts/freestyle/modules', '/snap/blender/63/2.92/scripts/addons/modules', '~/.config/blender/2.92/scripts/addons/modules', '/snap/blender/63/2.92/scripts/addons', '/snap/blender/63/2.92/scripts/addons_contrib']

only difference seems to be the addons_contrib directory.

For the other variables you mentioned please provide information on how to get them.

@scurest I tried, but same result as before: script crashes at `from ctypes._endian import BigEndianStructure, LittleEndianStructure` I can however not debug that file any further by using prints / file writes, as it again imports ctypes itself, so `_endian.py` crashes at `from ctypes import *` which kinda brings back to where I started... @rjg notable differences in `os.environ.items()`: | Variable | Flatpak | Snap | |---------|-------|--| | SHELL: | /bin/sh | /bin/bash | | XAUTHORITY: | /run/user/1000/Xauthority | /run/user/1000/.mutter-Xwaylandauth.1VGAX0 | | AT_SPI_BUS_ADDRESS: | unix:path=/run/user/1000/at-spi-bus | - | | XDG_CACHE_HOME: | ~/.var/app/org.blender.Blender/cache | - | | | Couple of entries for GJS_DEBUG... | - | | GI_TYPELIB_PATH: | /app/lib/girepository-1.0 | - | | SHLVL | 0 | 1 | | GST_PLUGIN_SYSTEM_PATH: | /app/lib/gstreamer-1.0:/usr/lib/extensions/gstreamer-1.0:/usr/lib/x86_64-linux-gnu/gstreamer-1.0 | - | | JOURNAL_STREAM: | 8:487937 | - | | PATH: | /app/bin:/usr/bin | ~/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin | | LESSOPEN: | - | [pipe character] /usr/bin/lesspipe %s | | LESSCLOSE: | - | /usr/bin/lesspipe %s %s | | COLORTERM: | - | truecolor | | TERM: | - | xterm-256color | | DISPLAY: | :99.0 | :0 | | VTE_VERSION: | - | 6200 | (note that I use Gnome on wayland, but crash also happened on X11 session) Here's Flatpaks `sys.path()`: ['/app/blender/2.91/scripts/startup', '/app/blender/2.91/scripts/modules', '/app/blender/2.91/python/lib/python37.zip', '/app/blender/2.91/python/lib/python3.7', '/app/blender/2.91/python/lib/python3.7/lib-dynload', '/app/blender/2.91/python/lib/python3.7/site-packages', '/app/blender/2.91/scripts/freestyle/modules', '/app/blender/2.91/scripts/addons/modules', '~/.var/app/org.blender.Blender/config/blender/2.91/scripts/addons/modules', '/app/blender/2.91/scripts/addons'] Here's Snaps `sys.path()`: ['/snap/blender/63/2.92/scripts/startup', '/snap/blender/63/2.92/scripts/modules', '/snap/blender/63/2.92/python/lib/python37.zip', '/snap/blender/63/2.92/python/lib/python3.7', '/snap/blender/63/2.92/python/lib/python3.7/lib-dynload', '/snap/blender/63/2.92/python/lib/python3.7/site-packages', '/snap/blender/63/2.92/scripts/freestyle/modules', '/snap/blender/63/2.92/scripts/addons/modules', '~/.config/blender/2.92/scripts/addons/modules', '/snap/blender/63/2.92/scripts/addons', '/snap/blender/63/2.92/scripts/addons_contrib'] only difference seems to be the addons_contrib directory. For the other variables you mentioned please provide information on how to get them.

Scurest commented

2021-01-19 12:34:38 +01:00

so _endian.py crashes at from ctypes import * which kinda brings back to where I started...

When you import a module that's already been imported, you just get a reference to the module that's been executed so far, ie. everything defined up to line 533. It shouldn't actually run any more code, so this would be a very weird place to crash. Can you replace from ctypes._endian import BigEndianStructure, LittleEndianStructure with the contents of _endian.py (minus the from ctypes import * line) to get further?

> so `_endian.py` crashes at `from ctypes import *` which kinda brings back to where I started... When you import a module that's already been imported, you just get a reference to the module that's been executed so far, ie. everything defined up to line 533. It shouldn't actually run any more code, so this would be a very weird place to crash. Can you replace `from ctypes._endian import BigEndianStructure, LittleEndianStructure` with the contents of `_endian.py` (minus the `from ctypes import *` line) to get further?

Simon I. commented

2021-01-19 12:46:41 +01:00

@scurest Thanks a lot. ctypes' __init__.py script crashes at line 551 _reset_cache()
@rjg could this be related to the fact that Snap package's os.environ.items() does not contain any specification of cache directories?

@scurest Thanks a lot. ctypes' `__init__.py` script crashes at line 551 `_reset_cache()` @rjg could this be related to the fact that Snap package's `os.environ.items()` does not contain any specification of cache directories?

Scurest commented

2021-01-19 12:52:53 +01:00

Can you check which line in the _reset_cache function?

Can you check which line in the `_reset_cache` function?

Simon I. commented

2021-01-19 12:56:19 +01:00

@scurest function _reset_cache() crashes at line 273 CFUNCTYPE(c_int)(lambda: None)
Also note the comment that was left directly above that line:

_pointer_type_cache[None] = c_void_p
- XXX for whatever reasons, creating the first instance of a callback
- function is needed for the unittests on Win64 to succeed.  This MAY
- be a compiler bug, since the problem occurs only when _ctypes is
- compiled with the MS SDK compiler.  Or an uninitialized variable?
CFUNCTYPE(c_int)(lambda: None)

Don't know if that refers to the line above or below the comment...

@scurest function `_reset_cache()` crashes at line 273 `CFUNCTYPE(c_int)(lambda: None)` Also note the comment that was left directly above that line: ``` _pointer_type_cache[None] = c_void_p - XXX for whatever reasons, creating the first instance of a callback - function is needed for the unittests on Win64 to succeed. This MAY - be a compiler bug, since the problem occurs only when _ctypes is - compiled with the MS SDK compiler. Or an uninitialized variable? CFUNCTYPE(c_int)(lambda: None) ``` Don't know if that refers to the line above or below the comment...

Scurest commented

2021-01-19 12:57:08 +01:00

What happens if you comment it out?

Simon I. commented

2021-01-19 13:13:50 +01:00

When commenting that line out it runs the script without errors or crashes.

That class CFUNCTYPE stops at line 104: return CFunctionType
See this part of class CFUNCTYPE:

    try:
        return _c_functype_cache[(restype, argtypes, flags)]
    except KeyError:
        class CFunctionType(_CFuncPtr):
            _argtypes_ = argtypes
            _restype_ = restype
            _flags_ = flags
        _c_functype_cache[(restype, argtypes, flags)] = CFunctionType
        return CFunctionType

When running the script it jumps to except branch, then runs through that class once, does _c_functiontyp_cache ... and crashes when trying to return CFunctionType, the print()s I put into the class are not printed a second time.

When commenting that line out it runs the script without errors or crashes. That class `CFUNCTYPE` stops at line 104: `return CFunctionType` See this part of class `CFUNCTYPE`: ``` try: return _c_functype_cache[(restype, argtypes, flags)] except KeyError: class CFunctionType(_CFuncPtr): _argtypes_ = argtypes _restype_ = restype _flags_ = flags _c_functype_cache[(restype, argtypes, flags)] = CFunctionType return CFunctionType ``` When running the script it jumps to `except` branch, then runs through that class once, does `_c_functiontyp_cache ...` and crashes when trying to `return CFunctionType`, the `print()`s I put into the class are not printed a second time.

Simon I. commented

2021-01-19 13:23:39 +01:00

When I run the portable Blender version with line 273 in 2.92/python/lib/python3.7/ctypes/__init__.py commented out, I can enable all the addons mentioned above that where previously crashing Blender. I only tested real quick with A.N.TLandscape, it generates a landscape and most of it's operators seem to behave normal. Nice.
Can't do it in snap however, as it's files are read only even for su.
Anyways someone with programming knowledge should try to find out why this line was needed before, I suspect just deleting that line could produce other problems...

When I run the portable Blender version with line 273 in `2.92/python/lib/python3.7/ctypes/__init__.py` commented out, I can enable all the addons mentioned above that where previously crashing Blender. I only tested real quick with A.N.TLandscape, it generates a landscape and most of it's operators seem to behave normal. Nice. Can't do it in snap however, as it's files are read only even for su. Anyways someone with programming knowledge should try to find out why this line was needed before, I suspect just deleting that line could produce other problems...

Scurest commented

2021-01-19 13:25:43 +01:00

This comment was removed by @scurest

*This comment was removed by @scurest*

Robert Guetzkow commented

2021-01-19 13:32:11 +01:00

@Simon_I That XDG_CACHE_HOME isn't set, shouldn't be a problem. It is supposed to default to $HOME/.cache in that case. It is correct that you aren't finding the environment variables I've listed. It would've been potentially problematic if they would be defined and Blender would use them.

@Simon_I That `XDG_CACHE_HOME` isn't set, shouldn't be a problem. It is supposed to default to `$HOME/.cache` in that case. It is correct that you aren't finding the environment variables I've listed. It would've been potentially problematic if they would be defined and Blender would use them.

Robert Guetzkow commented

2021-01-19 13:37:22 +01:00

@Simon_I thank you for investigating this further, we will look into this.

Robert Guetzkow commented

2021-01-19 14:15:15 +01:00

Changed status from 'Needs User Info' to: 'Needs Developer To Reproduce'

Robert Guetzkow commented

2021-01-19 14:39:34 +01:00

The code that causes issues for you exists in Python 3.7.7 but was removed by commit

The line was apparently introduced as a workaround for a failing test quite a long time ago.

674e9389e9

I still don't know why this fails on your system, but judging by numerous reports about crashes in this functions this doesn't seem to be an issues isolated to Blender. From the current information I do not know whether the crash indicates a problem elsewhere or if this was just a bug in ctypes. This might need another look just to double check.

If this is indeed the cause, then we would either need to patch the Python version 3.7.7 or update to a newer version that contains the fix from upstream.

The code that causes issues for you exists in Python 3.7.7 but was removed by commit - https://github.com/python/cpython/commit/e6b247c8e524dbe5fc03b3492f628d0d5348bc49#diff-edff4421a88bdc56335a84a1e5a0ce0eafe5cd0c37dcb1aff03ff075bfb9b57c - https://bugs.python.org/issue35523 The line was apparently introduced as a workaround for a failing test quite a long time ago. - https://github.com/python/cpython/commit/674e9389e9fdadd622829f4833367ac3b38475b5 I still don't know why this fails on your system, but judging by numerous reports about crashes in this functions this doesn't seem to be an issues isolated to Blender. From the current information I do not know whether the crash indicates a problem elsewhere or if this was just a bug in ctypes. This might need another look just to double check. If this is indeed the cause, then we would either need to patch the Python version 3.7.7 or update to a newer version that contains the fix from upstream.

Simon I. commented

2021-01-19 14:53:50 +01:00

@rjg thanks for the info. Let me know if you need more information from me, otherwise I'll just hope it's as easy as removing these few lines... Have a blessed week!

Robert Guetzkow commented

2021-01-19 15:09:16 +01:00

@Simon_I Thank you for identifying the function that caused the problem! Do you happen to use SELinux or other security extensions?

https://bugzilla.redhat.com/show_bug.cgi?id=814391

Within ctypes, objects of _ctypes.CThunkObject wrap machine code "thunks", containing run-time generated machine code hooks for bridging between Python callables and C functions. Instantiating one of these objects requires an allocation of a page of RAM that can be both written to and executed. In RHEL's python we patch ctypes to use libffi's ffi_closure_alloc for this, which uses a trick involving mapping a tempfile in /tmp multiple times to avoid SELinux's complaining about pages of memory being both writable and executable.

Edit: Obviously @scurest thanks to you as well for helping debug this.

@Simon_I Thank you for identifying the function that caused the problem! Do you happen to use SELinux or other security extensions? https://bugzilla.redhat.com/show_bug.cgi?id=814391 > Within ctypes, objects of _ctypes.CThunkObject wrap machine code "thunks", containing run-time generated machine code hooks for bridging between Python callables and C functions. Instantiating one of these objects requires an allocation of a page of RAM that can be both written to and executed. In RHEL's python we patch ctypes to use libffi's ffi_closure_alloc for this, which uses a trick involving mapping a tempfile in /tmp multiple times to avoid SELinux's complaining about pages of memory being both writable and executable. Edit: Obviously @scurest thanks to you as well for helping debug this.

Simon I. commented

2021-01-19 15:24:45 +01:00

@rjg SELinux not as far as I know. This is a common Ubuntu installation. Ubuntu uses AppArmor though. The AppArmor profile for blender is in complain mode however, since Blender snaps are installed with --classic, so I would guess it's not enforcing any restrictions... Also running aa-notify does not register any AppArmor denials when Blender crashes.

@rjg SELinux not as far as I know. This is a common Ubuntu installation. Ubuntu uses AppArmor though. The AppArmor profile for blender is in complain mode however, since Blender snaps are installed with `--classic`, so I would guess it's not enforcing any restrictions... Also running aa-notify does not register any AppArmor denials when Blender crashes.

Scurest commented

2021-01-20 01:45:58 +01:00

@Simon_I

I only tested real quick with A.N.TLandscape, it generates a landscape and most of it's operators seem to behave normal.

Sorry, just want to make sure: I believe only the erosion operator actually uses numpy (and therefore presumably ctypes), did you try that one? You could also try exporting a cube or something to glTF since I know that uses numpy too.

@Simon_I > I only tested real quick with A.N.TLandscape, it generates a landscape and most of it's operators seem to behave normal. Sorry, just want to make sure: I believe only the erosion operator actually *uses* numpy (and therefore presumably ctypes), did you try that one? You could also try exporting a cube or something to glTF since I know that uses numpy too.

Simon I. commented

2021-01-20 10:24:43 +01:00

@scurest Using the Landscape Erosion and exporting to glTF works now.

Scurest commented

2021-01-20 10:36:04 +01:00

@Simon_I Great, thanks for checking all that stuff.

Robert Guetzkow commented

2021-01-20 12:10:10 +01:00

Added subscriber: @LuisYayo

Robert Guetzkow commented

2021-01-20 12:10:35 +01:00

Added subscribers: @skittlituted_scorpion, @Mkats13

Robert Guetzkow commented

2021-01-20 12:11:26 +01:00

Changed status from 'Needs Developer To Reproduce' to: 'Confirmed'

Robert Guetzkow commented

2021-01-20 12:17:58 +01:00

Changed status from 'Confirmed' to: 'Needs Developer To Reproduce'

Robert Guetzkow commented

2021-01-20 13:14:39 +01:00

Changed status from 'Needs Developer To Reproduce' to: 'Confirmed'

Robert Guetzkow commented

2021-01-20 13:14:39 +01:00

I can reproduce this with a release build of Blender 2.91.0 on Ubuntu 20.10, but not 20.04.

Philipp Oeser commented

2021-01-20 14:17:30 +01:00

Added subscribers: @ideasman42, @lichtwerk

Philipp Oeser commented

2021-01-20 14:17:30 +01:00

Will CC @ideasman42 here to share his wisdom.

Campbell Barton commented

2021-01-20 14:34:23 +01:00

We can patch the Python that's distributed with Blender, although I'm not keen on this as a precedent.

Note that we could update this at run-time via monkey-patching (not especially nice either).

Unfortunately this is being kept for the 3.7 series (as it's still in 3.7.9).

Judging by #83246, it's possible we'll upgrade Python, although we didn't yet make a decision on this.

We can patch the Python that's distributed with Blender, although I'm not keen on this as a precedent. Note that we could update this at run-time via monkey-patching (not especially nice either). Unfortunately this is being kept for the 3.7 series (as it's still in 3.7.9). ---- Judging by #83246, it's possible we'll upgrade Python, although we didn't yet make a decision on this.

Inkabod Chrane commented

2021-01-20 16:51:33 +01:00

So does that mean that for now, we know there's a bug, it'll be fixed when python is upgraded ?

Sybren A. Stüvel commented

2021-01-21 15:13:57 +01:00

Added subscriber: @dr.sybren

Sybren A. Stüvel commented

2021-01-21 15:13:57 +01:00

Which Python version fixes this? I mean, upgrading to 3.7.9 is quite different from upgrading to 3.10-alpha.

Robert Guetzkow commented

2021-01-21 15:21:12 +01:00

@dr.sybren Based on the tags of the commit, this appears to be in 3.8.0 and later.

Please note that I haven't investigated (yet) why it doesn't crash when using Flatpak.

@dr.sybren Based on the tags of the commit, this appears to be in 3.8.0 and later. Please note that I haven't investigated (yet) why it doesn't crash when using Flatpak.

Robert Guetzkow commented

2021-01-25 11:22:21 +01:00

Added subscriber: @tomb

Robert Guetzkow commented

2021-01-26 10:10:31 +01:00

Added subscriber: @luifernandi

Campbell Barton changed title from ~~Blender crash when enabling A.N.T. Landscape and various other addons that use ctypes~~ to Crash using 'ctypes' (fixed in Python >= 3.8), causes crashes in A.N.T. Landscape and various other addons

2021-01-28 20:54:31 +01:00

Tomasz Biernacki commented

2021-01-31 18:27:50 +01:00

With the new Blender 2.93.0 Alpha export of GLTF works fine. Many thanks!

Robert Guetzkow commented

2021-01-31 23:59:40 +01:00

@tomb That is a bit strange, we didn't patch ctypes or switch the Python version yet. Are you using one of the daily builds from https://builder.blender.org/download/ ?

Tomasz Biernacki commented

2021-02-01 08:51:43 +01:00

Yes now I'm using the daily builds.
I reinstalled python from the console. Then I launched the newest Blender 2.93.0 Alpha daily build, downloaded by me yesterday. I don't know, but it seems installed python have nothing to do with it, because older versions of Blender keep crashing when trying to export gltf. Blender 2.93.0 Alpha works fine for me.

Yes now I'm using the daily builds. I reinstalled python from the console. Then I launched the newest Blender 2.93.0 Alpha daily build, downloaded by me yesterday. I don't know, but it seems installed python have nothing to do with it, because older versions of Blender keep crashing when trying to export gltf. Blender 2.93.0 Alpha works fine for me.

Robert Guetzkow commented

2021-02-01 10:54:56 +01:00

Added subscribers: @carlf, @JulienDuroure, @ry167

Simon I. commented

2021-02-01 11:24:25 +01:00

@tomb Can't confirm, exporting the default cube to gltf still crashes for me when using the latest 2.93 alpha

Tomasz Biernacki commented

2021-02-02 21:38:15 +01:00

@Simon_I @rjg Yes, right, sorry for the confusion, still crashes. I don't know how I managed to export 2 days ago, but really without crash. Anyway I see export window, before I didn't see even the window.

Campbell Barton commented

2021-02-12 10:05:31 +01:00

Changed status from 'Confirmed' to: 'Resolved'

Campbell Barton closed this issue

2021-02-12 10:05:31 +01:00

Campbell Barton self-assigned this 2021-02-12 10:05:31 +01:00

Campbell Barton commented

2021-02-12 10:05:31 +01:00

Closing, as Blender has moved to Python 3.9.

Robert Guetzkow commented

2021-04-06 18:49:23 +02:00

Added subscriber: @vm

Sign in to join this conversation.

No Label

Download

What's New

Blender Studio

Manual

Developers Blog

Documentation

Benchmark

Blender Conference

Development Fund

One-time Donations

Crash using 'ctypes' (fixed in Python >= 3.8), causes crashes in A.N.T. Landscape and various other addons #84752