Python: Importing zstandard fails #107137

New Issue

Simon Thommes · 2023-04-19T18:40:27+02:00

Simon Thommes commented

2023-04-19 18:40:27 +02:00

System Information
Operating system: Linux-5.17.15-76051715-generic-x86_64-with-glibc2.35 64 Bits, X11 UI
Graphics card: Quadro RTX 6000/PCIe/SSE2 NVIDIA Corporation 4.5.0 NVIDIA 515.48.07

Blender Version
Broken: version: 3.6.0 Alpha, branch: main, commit date: 2023-04-19 14:50, hash: 45c0762f1b9b
Worked: (newest version of Blender that worked as expected)

Short description of error
Importing zstandard with import zstandard gives the following error:

Traceback (most recent call last):
  File "<blender_console>", line 1, in <module>
  File "/home/guest/.local/lib/python3.10/site-packages/zstandard/__init__.py", line 39, in <module>
    from .backend_c import *  # type: ignore
ImportError: zstd C API versions mismatch; Python bindings were not compiled/linked against expected zstd version (10500 returned by the lib, 10502 hardcoded in zstd headers, 10502 hardcoded in the cext)

Exact steps for others to reproduce the error
import zstandard

**System Information** Operating system: Linux-5.17.15-76051715-generic-x86_64-with-glibc2.35 64 Bits, X11 UI Graphics card: Quadro RTX 6000/PCIe/SSE2 NVIDIA Corporation 4.5.0 NVIDIA 515.48.07 **Blender Version** Broken: version: 3.6.0 Alpha, branch: main, commit date: 2023-04-19 14:50, hash: `45c0762f1b9b` Worked: (newest version of Blender that worked as expected) **Short description of error** Importing zstandard with `import zstandard` gives the following error: ``` Traceback (most recent call last): File "<blender_console>", line 1, in <module> File "/home/guest/.local/lib/python3.10/site-packages/zstandard/__init__.py", line 39, in <module> from .backend_c import * # type: ignore ImportError: zstd C API versions mismatch; Python bindings were not compiled/linked against expected zstd version (10500 returned by the lib, 10502 hardcoded in zstd headers, 10502 hardcoded in the cext) ``` **Exact steps for others to reproduce the error** `import zstandard`

Simon Thommes added the

labels 2023-04-19 18:40:28 +02:00

Sergey Sharybin commented

2023-04-19 18:47:31 +02:00

We were looking into the issue with Simon. We did not notice that the zstandard is coming from the local pip install.

But this is actually something that feels strange. One would imagine that Blender will use the zstandard shipped with it, and not the one from pip. Otherwise it feels to be too easy to run into library version mismatch like that.

We were looking into the issue with Simon. We did not notice that the `zstandard` is coming from the local pip install. But this is actually something that feels strange. One would imagine that Blender will use the `zstandard ` shipped with it, and not the one from pip. Otherwise it feels to be too easy to run into library version mismatch like that.

Ray molenkamp commented

2023-04-19 18:51:42 +02:00

It's already a bit of a hassle to keep this in check, we ship very specific versions of both zstd and the python addon for this reason. The odd thing here is it using site packages outside the blender folder which as far as i know shouldn't happen unless you start blender with the --python-use-system-env flag?

It's already a bit of a hassle to keep this in check, we ship very specific versions of both zstd and the python addon for this reason. The odd thing here is it using site packages outside the blender folder which as far as i know shouldn't happen unless you start blender with the `--python-use-system-env` flag?

Sergey Sharybin commented

2023-04-19 20:14:33 +02:00

We just downloaded Blender from the buildbot and run it form the terminal without extra command line arguments.
What I am not sure about is the environment variables. Is there such variable with the same meaning as the --python-use-system-env flag?

We just downloaded Blender from the buildbot and run it form the terminal without extra command line arguments. What I am not sure about is the environment variables. Is there such variable with the same meaning as the `--python-use-system-env` flag?

Ray molenkamp commented

2023-04-19 20:26:00 +02:00

not that i'm aware of, we're venturing into campbell territory real quick, best diagnostics i have is

[root@SRV blender-3.6.0-alpha+main.45c0762f1b9b-linux.x86_64-release]# ./blender --log "bpy.interface" --log-level -1 -b --python-expr "import zstandard"
Blender 3.6.0 Alpha (hash 45c0762f1b9b built 2023-04-19 15:11:26)
shared memfd open() failed: Function not implemented
INFO (bpy.interface): source/blender/python/intern/bpy_interface.c:320 BPY_python_start: Initializing *without* support for the systems Python environment such as 'PYTHONPATH' and the user-site directory.

Blender quit

that'll tell you if it's allowing env vars to be respected or not

not that i'm aware of, we're venturing into campbell territory real quick, best diagnostics i have is ``` [root@SRV blender-3.6.0-alpha+main.45c0762f1b9b-linux.x86_64-release]# ./blender --log "bpy.interface" --log-level -1 -b --python-expr "import zstandard" Blender 3.6.0 Alpha (hash 45c0762f1b9b built 2023-04-19 15:11:26) shared memfd open() failed: Function not implemented INFO (bpy.interface): source/blender/python/intern/bpy_interface.c:320 BPY_python_start: Initializing *without* support for the systems Python environment such as 'PYTHONPATH' and the user-site directory. Blender quit ``` that'll tell you if it's allowing env vars to be respected or not

Sergey Sharybin commented

2023-04-20 11:57:31 +02:00

I kind of managed to reproduce the issue on my workstation by placing zstandard from python3.10 pip to my ~/.local/lib/python3.10/site-packages/zstandard (I had to do it manually since there is no easy way to me to install actual python3.10).

After that using the current buildbot build:

sergey@ws-sergey:~/Downloads/blender-3.6.0-alpha+main.f87e474af004-linux.x86_64-release$ ./blender --factory-startup --log "bpy.interface" --log-level -1 -b --python-expr "import zstandard"
Blender 3.6.0 Alpha (hash f87e474af004 built 2023-04-20 02:16:03)
Failed to open dir (No such file or directory): /run/user/1000/gvfs/
INFO (bpy.interface): source/blender/python/intern/bpy_interface.c:320 BPY_python_start: Initializing *without* support for the systems Python environment such as 'PYTHONPATH' and the user-site directory.
Error: Python: Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/sergey/.local/lib/python3.10/site-packages/zstandard/__init__.py", line 39, in <module>
    from .backend_c import *  # type: ignore
ImportError: zstd C API versions mismatch; Python bindings were not compiled/linked against expected zstd version (10500 returned by the lib, 10502 hardcoded in zstd headers, 10502 hardcoded in the cext)

Blender quit

And to my knowledge I do not have any special Blender or python environment variables.
The log also says *without* support for the systems Python environment such as 'PYTHONPATH' and the user-site directory, which does not seem to be the observed behavior.

I kind of managed to reproduce the issue on my workstation by placing zstandard from python3.10 pip to my `~/.local/lib/python3.10/site-packages/zstandard` (I had to do it manually since there is no easy way to me to install actual python3.10). After that using the current buildbot build: ``` sergey@ws-sergey:~/Downloads/blender-3.6.0-alpha+main.f87e474af004-linux.x86_64-release$ ./blender --factory-startup --log "bpy.interface" --log-level -1 -b --python-expr "import zstandard" Blender 3.6.0 Alpha (hash f87e474af004 built 2023-04-20 02:16:03) Failed to open dir (No such file or directory): /run/user/1000/gvfs/ INFO (bpy.interface): source/blender/python/intern/bpy_interface.c:320 BPY_python_start: Initializing *without* support for the systems Python environment such as 'PYTHONPATH' and the user-site directory. Error: Python: Traceback (most recent call last): File "<string>", line 1, in <module> File "/home/sergey/.local/lib/python3.10/site-packages/zstandard/__init__.py", line 39, in <module> from .backend_c import * # type: ignore ImportError: zstd C API versions mismatch; Python bindings were not compiled/linked against expected zstd version (10500 returned by the lib, 10502 hardcoded in zstd headers, 10502 hardcoded in the cext) Blender quit ``` And to my knowledge I do not have any special Blender or python environment variables. The log also says `*without* support for the systems Python environment such as 'PYTHONPATH' and the user-site directory`, which does not seem to be the observed behavior.

Philipp Oeser commented

2023-04-21 11:07:44 +02:00

Looks like this is confirmed.
While not API related, will put on the Python API modules desk and summon @ideasman42 for further investigation.

Looks like this is confirmed. While not API related, will put on the `Python API` modules desk and summon @ideasman42 for further investigation.

Philipp Oeser added

Download

What's New

Blender Studio

Manual

Developers Blog

Documentation

Benchmark

Blender Conference

Development Fund

One-time Donations

Python: Importing zstandard fails #107137