user site-packages path no longer in sys.path, resulting in modules installed there can no longer be imported any more by Blender's python #76993

New Issue

MACHIN3 · 2020-05-23T13:52:55+02:00

MACHIN3 commented

2020-05-23 13:52:55 +02:00

System Information
Operating system: Linux-4.15.0-96-generic-x86_64-with-debian-buster-sid 64 Bits
Graphics card: GeForce GTX 1050/PCIe/SSE2 NVIDIA Corporation 4.5.0 NVIDIA 435.21

Blender Version
Broken: version: 2.83 (sub 15), branch: master, commit date: 2020-05-03 18:42, hash: 331bf04fad
Worked: '1239cab11f'

Short description of error
Blender no longer considers python modules installed to the user site-packages path (if it exists), as of 331bf04fad(first build I've noticed it on, probably earlier though).
As a consequence any modules installed in this location can no longer be imported from Blender's python.

The user site-packages path on Windows is
C:\Users\name\AppData\Roaming\Python\Python37\site-packages
On Linux
/home/name/.local/lib/python3.7/site-packages
On MacOS (I think)
/Users/name/Library/Python/3.7/lib/python/site-packages

It can be fetched by doing

import site
site.getusersitepackages()

Here are two system_info files to demonstrate it:
system-info_31bf04fad93.txt user site-packages path missing
system-info_1239cab11ff9.txt user site-packages path present: /home/x/.local/lib/python3.7/site-packages

Exact steps for others to reproduce the error

save out the system info in 1239cab11f or earlier, as well as 331bf04fad or newer
alternatively run the following snippet in the python console
look for the user site-packages path

    import sys
    for p in sys.path:
        print(p)

**System Information** Operating system: Linux-4.15.0-96-generic-x86_64-with-debian-buster-sid 64 Bits Graphics card: GeForce GTX 1050/PCIe/SSE2 NVIDIA Corporation 4.5.0 NVIDIA 435.21 **Blender Version** Broken: version: 2.83 (sub 15), branch: master, commit date: 2020-05-03 18:42, hash: `331bf04fad` Worked: '1239cab11f' **Short description of error** Blender no longer considers python modules installed to the user site-packages path (if it exists), as of `331bf04fad`(first build I've noticed it on, probably earlier though). As a consequence any modules installed in this location can no longer be imported from Blender's python. The user site-packages path on Windows is `C:\Users\name\AppData\Roaming\Python\Python37\site-packages` On Linux `/home/name/.local/lib/python3.7/site-packages` On MacOS (I think) `/Users/name/Library/Python/3.7/lib/python/site-packages` It can be fetched by doing ``` import site site.getusersitepackages() ``` Here are two system_info files to demonstrate it: [system-info_31bf04fad93.txt](https://archive.blender.org/developer/F8549264/system-info_31bf04fad93.txt) user site-packages path missing [system-info_1239cab11ff9.txt](https://archive.blender.org/developer/F8549265/system-info_1239cab11ff9.txt) user site-packages path present: `/home/x/.local/lib/python3.7/site-packages` **Exact steps for others to reproduce the error** * save out the system info in `1239cab11f` or earlier, as well as `331bf04fad` or newer * alternatively run the following snippet in the python console * look for the user site-packages path ``` import sys for p in sys.path: print(p) ```

MACHIN3 commented

2020-05-23 13:52:55 +02:00

Added subscriber: @MACHIN3

MACHIN3 changed title from ~~user site-packages path no longer in sys.path, resulting in modules installed there can't be imported any more by Blender's python~~ to user site-packages path no longer in sys.path, resulting in modules installed there can no longer be imported any more by Blender's python

2020-05-23 13:53:25 +02:00

MACHIN3 commented

2020-05-23 14:12:47 +02:00

Apparently, its intentional? https://developer.blender.org/rBS79a58eef059ffc3f12d11bd68938cfb1b4cd2462

MACHIN3 commented

2020-05-23 14:18:43 +02:00

Added subscriber: @brecht

MACHIN3 commented

2020-05-23 14:18:43 +02:00

I'm ready to close this but can somebody offer a suggestion what addons relying on non-supplied modules should do?
I can just append the path to sys.path when registering of course. @brecht?

I'm ready to close this but can somebody offer a suggestion what addons relying on non-supplied modules should do? I can just append the path to sys.path when registering of course. @brecht?

Robert Guetzkow commented

2020-05-23 14:23:12 +02:00

Added subscriber: @rjg

Robert Guetzkow commented

2020-05-23 14:23:12 +02:00

In #71420 it was debated whether Blender would get it's own site-packages directory for the ones installed with the --user flag by setting PYTHONUSERBASE . Not sure if there's been a decision for these changes yet.

Installing the packages directly appears to be the way to go at the moment.

In #71420 it was debated whether Blender would get it's own site-packages directory for the ones installed with the `--user` flag by setting `PYTHONUSERBASE` . Not sure if there's been a decision for these changes yet. [Installing the packages directly appears to be the way to go at the moment](https://blender.stackexchange.com/questions/168448/bundling-python-library-with-addon/168482#168482).

CansecoGPC commented

2020-05-23 17:32:07 +02:00

Added subscriber: @CansecoGPC

CansecoGPC commented

2020-05-23 17:32:07 +02:00

I was wondering why an add-on i was using didn't pick up it's dependencies, good to know.

Sadly the solution provided doesn't work, because as soon as i try to install with pip without --user flag on Linux this is what happens.

"Defaulting to user installation because normal site-packages is not writeable"

I hope the solution isn't launching Blender with root privileges.

I was wondering why an add-on i was using didn't pick up it's dependencies, good to know. Sadly the solution provided doesn't work, because as soon as i try to install with pip without --user flag on Linux this is what happens. "Defaulting to user installation because normal site-packages is not writeable" I hope the solution isn't launching Blender with root privileges.

MACHIN3 commented

2020-05-23 17:38:40 +02:00

"Defaulting to user installation because normal site-packages is not writeable"

As it should be to be honest.

user installation is definitely prefered IMO.

I just run this now, when registering my addon:

def verify_user_sitepackages():
    usersitepackagespath = site.getusersitepackages()

    if os.path.exists(usersitepackagespath) and usersitepackagespath not in sys.path:
        sys.path.append(usersitepackagespath)

> "Defaulting to user installation because normal site-packages is not writeable" As it should be to be honest. - user installation is definitely prefered IMO. I just run this now, when registering my addon: ``` def verify_user_sitepackages(): usersitepackagespath = site.getusersitepackages() if os.path.exists(usersitepackagespath) and usersitepackagespath not in sys.path: sys.path.append(usersitepackagespath) ```

Robert Guetzkow commented

2020-05-23 17:57:20 +02:00

@MACHIN3 @brecht Both certainly works, not sure what is currently recommended in the context of Blender. Having a proper and unified way of installing, managing and isolating dependencies would be nice. Perhaps that's a project I could get involved in once my thesis is done.

MACHIN3 commented

2020-05-23 18:00:53 +02:00

It's a worthy cause for sure :) Good luck with your thesis and thanks for your input!

Robert Guetzkow commented

2020-05-23 18:07:04 +02:00

@MACHIN3 Thank you! No problem, you're welcome.

CansecoGPC commented

2020-05-23 18:13:55 +02:00

In #76993#937328, @MACHIN3 wrote:

"Defaulting to user installation because normal site-packages is not writeable"

As it should be to be honest.

--user installation is definitely prefered IMO.

I just run this now, when registering my addon:
def verify_user_sitepackages():
    usersitepackagespath = site.getusersitepackages()

    if os.path.exists(usersitepackagespath) and usersitepackagespath not in sys.path:
        sys.path.append(usersitepackagespath)

Just needed to import os, sys, and site and it worked. Thanks for the solution.

> In #76993#937328, @MACHIN3 wrote: >> "Defaulting to user installation because normal site-packages is not writeable" > As it should be to be honest. > > --user installation is definitely prefered IMO. > > I just run this now, when registering my addon: > > ``` > def verify_user_sitepackages(): > usersitepackagespath = site.getusersitepackages() > > if os.path.exists(usersitepackagespath) and usersitepackagespath not in sys.path: > sys.path.append(usersitepackagespath) > > ``` Just needed to import os, sys, and site and it worked. Thanks for the solution.

Campbell Barton commented

2020-05-25 09:49:54 +02:00

Added subscriber: @ideasman42

Campbell Barton commented

2020-05-25 09:49:54 +02:00

Changed status from 'Needs Triage' to: 'Archived'

Campbell Barton closed this issue

2020-05-25 09:49:54 +02:00

Campbell Barton self-assigned this 2020-05-25 09:49:54 +02:00

Campbell Barton commented

2020-05-25 09:49:54 +02:00

This is working as intended, pass --python-use-system-env for previous behavior.

This is working as intended, pass `--python-use-system-env` for previous behavior.

Robert Guetzkow commented

2020-05-25 10:10:30 +02:00

@ideasman42 Since add-on developers can't expect that Blender is started with this command line flag, I assume that installing packages without the --user flag is currently the intended approach?

@ideasman42 Since add-on developers can't expect that Blender is started with this command line flag, I assume that installing packages *without* the `--user` flag is currently the intended approach?

MACHIN3 commented

2020-05-25 10:30:23 +02:00

This is working as intended, pass --python-use-system-env for previous behavior.

I can understand that, but honestly, this is a terrible thing to ask of addon users.

> This is working as intended, pass --python-use-system-env for previous behavior. I can understand that, but honestly, this is a terrible thing to ask of addon users.

stephen leger commented

2020-05-25 14:05:40 +02:00

Added subscriber: @StephenLeger

stephen leger commented

2020-05-25 14:05:40 +02:00

This is working as intended, pass --python-use-system-env for previous behavior.

Would rather say this is not working for average user anymore.

Create a site_package in user folder as working alternative out of the box and set the path according instead make a bit more sense than simply removing and let addon devs deal without.

%user%/scripts/site_package/

> This is working as intended, pass --python-use-system-env for previous behavior. Would rather say this is not working for average user anymore. Create a site_package in user folder as working alternative out of the box and set the path according instead make a bit more sense than simply removing and let addon devs deal without. %user%/scripts/site_package/

James commented

2022-04-14 21:05:40 +02:00

Added subscriber: @martiaus1028

James commented

2022-04-14 21:05:40 +02:00

@MACHIN3 Thank you for the tip! did you include that snippet in the register function just once? and then you still pip install and just have it go to the user installation?

MACHIN3 commented

2022-04-14 21:38:21 +02:00

Yes, I only run it once in __init__.py in register()

To install pip, I do

cmd = [pythonbinpath, ensurepippath, "--upgrade", "--user"]
pip = subprocess.run(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE)

to install a module - PIL/Pillow in my case, I do:

cmd = [pythonbinpath, "-m", "pip", "install", "--upgrade", "--user", "Pillow==%s" % (version)]
pil = subprocess.run(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE)

In both cases --user ensures the user site packages path is used.

Yes, I only run it once in `__init__.py` in `register()` To install pip, I do ``` cmd = [pythonbinpath, ensurepippath, "--upgrade", "--user"] pip = subprocess.run(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE) ``` to install a module - PIL/Pillow in my case, I do: ``` cmd = [pythonbinpath, "-m", "pip", "install", "--upgrade", "--user", "Pillow==%s" % (version)] pil = subprocess.run(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE) ``` In both cases `--user` ensures the user site packages path is used.

Sign in to join this conversation.

No Label

Download

What's New

Blender Studio

Manual

Developers Blog

Documentation

Benchmark

Blender Conference

Development Fund

One-time Donations

user site-packages path no longer in sys.path, resulting in modules installed there can no longer be imported any more by Blender's python #76993