Fails to compile on Python 3.11b3 due to opaque PyFrameObject #99277

New Issue

Ben Beasley · 2022-06-29T18:56:59+02:00

Ben Beasley commented

2022-06-29 18:56:59 +02:00

System Information
Operating system: Fedora Rawhide (development)
Graphics card: N/A

Blender Version
Broken: 3.2.0
Worked: N/A

Short description of error

Blender fails to compile with Python 3.11b3 because:

The fields of PyFrameObject are now opaque (defined in an internal header); the function PyFrame_GetCode should be used to get the code object from the frame object.
Similarly, the function PyThreadState_GetFrame should be used to get the frame object from the thread state.

[ 52%] Generating buildinfo.h_fake, buildinfo.h
/builddir/build/BUILD/blender-3.2.0/source/blender/python/generic/py_capi_utils.c: In function 'PyC_FileAndNum':
/builddir/build/BUILD/blender-3.2.0/source/blender/python/generic/py_capi_utils.c:652:38: error: 'PyThreadState' {aka 'struct _ts'} has no member named 'frame'; did you mean 'cframe'?
  652 |   if (!(frame = PyThreadState_GET()->frame)) {
      |                                      ^~~~~
      |                                      cframe
/builddir/build/BUILD/blender-3.2.0/source/blender/python/generic/py_capi_utils.c:658:41: error: invalid use of incomplete typedef 'PyFrameObject' {aka 'struct _frame'}
  658 |     *r_filename = PyUnicode_AsUTF8(frame->f_code->co_filename);
      |                                         ^~
cd /builddir/build/BUILD/blender-3.2.0/redhat-linux-build/source/creator && /usr/bin/cmake -DSOURCE_DIR=/builddir/build/BUILD/blender-3.2.0 -DBUILD_DATE= -DBUILD_TIME= -P /builddir/build/BUILD/blender-3.2.0/build_files/cmake/buildinfo.cmake

Exact steps for others to reproduce the error

Build Blender normally, but with Python 3.11b3.

Additional details

I proposed the following fix for a similar issue in USD: https://github.com/PixarAnimationStudios/USD/pull/1928

**System Information** Operating system: Fedora Rawhide (development) Graphics card: N/A **Blender Version** Broken: 3.2.0 Worked: N/A **Short description of error** Blender fails to compile with Python 3.11b3 because: - The fields of PyFrameObject are now opaque (defined in an internal header); the function [PyFrame_GetCode](https://docs.python.org/3/c-api/reflection.html?highlight=pyframe_getcode#c.PyFrame_GetCode) should be used to get the code object from the frame object. - Similarly, the function [PyThreadState_GetFrame](https://docs.python.org/3/c-api/init.html?highlight=pythreadstate#c.PyThreadState_GetFrame) should be used to get the frame object from the thread state. ``` [ 52%] Generating buildinfo.h_fake, buildinfo.h /builddir/build/BUILD/blender-3.2.0/source/blender/python/generic/py_capi_utils.c: In function 'PyC_FileAndNum': /builddir/build/BUILD/blender-3.2.0/source/blender/python/generic/py_capi_utils.c:652:38: error: 'PyThreadState' {aka 'struct _ts'} has no member named 'frame'; did you mean 'cframe'? 652 | if (!(frame = PyThreadState_GET()->frame)) { | ^~~~~ | cframe /builddir/build/BUILD/blender-3.2.0/source/blender/python/generic/py_capi_utils.c:658:41: error: invalid use of incomplete typedef 'PyFrameObject' {aka 'struct _frame'} 658 | *r_filename = PyUnicode_AsUTF8(frame->f_code->co_filename); | ^~ cd /builddir/build/BUILD/blender-3.2.0/redhat-linux-build/source/creator && /usr/bin/cmake -DSOURCE_DIR=/builddir/build/BUILD/blender-3.2.0 -DBUILD_DATE= -DBUILD_TIME= -P /builddir/build/BUILD/blender-3.2.0/build_files/cmake/buildinfo.cmake ``` **Exact steps for others to reproduce the error** Build Blender normally, but with Python 3.11b3. **Additional details** I proposed the following fix for a similar issue in USD: https://github.com/PixarAnimationStudios/USD/pull/1928

Ben Beasley commented

2022-06-29 18:56:59 +02:00

Added subscriber: @music

Ben Beasley commented

2022-06-29 20:10:45 +02:00

While working on a patch for these errors, I found an additional error:

/builddir/build/BUILD/blender-3.2.0/source/blender/python/intern/bpy_driver.c: In function 'bpy_driver_secure_bytecode_validate':
/builddir/build/BUILD/blender-3.2.0/source/blender/python/intern/bpy_driver.c:386:36: error: 'PyCodeObject' has no member named 'co_code'
  386 |     PyBytes_AsStringAndSize(py_code->co_code, (char **)&codestr, &code_len);
      |                                    ^~

In Python 3.11 (https://docs.python.org/3.11/whatsnew/3.11.html#id6), “PyCodeObject no longer has a co_code field. Instead, use PyObject_GetAttrString(code_object, "co_code") or PyCode_GetCode() to get the underlying bytes object.” This link also has further information on the PyThreadState and PyFrameObject changes.

While working on a patch for these errors, I found an additional error: ``` /builddir/build/BUILD/blender-3.2.0/source/blender/python/intern/bpy_driver.c: In function 'bpy_driver_secure_bytecode_validate': /builddir/build/BUILD/blender-3.2.0/source/blender/python/intern/bpy_driver.c:386:36: error: 'PyCodeObject' has no member named 'co_code' 386 | PyBytes_AsStringAndSize(py_code->co_code, (char **)&codestr, &code_len); | ^~ ``` In Python 3.11 (https://docs.python.org/3.11/whatsnew/3.11.html#id6), “PyCodeObject no longer has a co_code field. Instead, use PyObject_GetAttrString(code_object, "co_code") or PyCode_GetCode() to get the underlying bytes object.” This link also has further information on the PyThreadState and PyFrameObject changes.

Pratik Borhade commented

2022-06-30 08:12:41 +02:00

Added subscriber: @PratikPB2123

Pratik Borhade commented

2022-06-30 08:12:41 +02:00

Changed status from 'Needs Triage' to: 'Archived'

Pratik Borhade closed this issue

2022-06-30 08:12:41 +02:00

Pratik Borhade commented

2022-06-30 08:12:41 +02:00

Hi, thanks for opening a report. But tracker is not usually used for build related questions. See What not to report here
You can reach out to devs in [#blender-builds ]] channel for such questions. Or you can open a topic on [ https:*devtalk.blender.org/c/blender/building-blender/11 | devtalk

Hi, thanks for opening a report. But tracker is not usually used for build related questions. See [What not to report here ](https://developer.blender.org/maniphest/task/edit/form/1/) You can reach out to devs in [#blender-builds ]] channel for such questions. Or you can open a topic on [[ https:*devtalk.blender.org/c/blender/building-blender/11 | devtalk ](https:*blender.chat/channel/blender-builds)

Ben Beasley commented

2022-06-30 15:23:58 +02:00

Thanks for your response, but I’m a little confused. I’m not asking for build support, but reporting a set of CPython API changes that will require code changes in Blender in order to support the upcoming Python 3.11 release. I’m working on the necessary patch. It doesn’t seem like a chat channel is the right venue for that.

Pratik Borhade commented

2022-06-30 15:37:06 +02:00

Changed status from 'Archived' to: 'Needs Triage'

Pratik Borhade reopened this issue

2022-06-30 15:37:06 +02:00

Germano Cavalcante commented

2022-07-01 15:22:28 +02:00

Added subscribers: @ideasman42, @mano-wii

Germano Cavalcante commented

2022-07-01 15:22:28 +02:00

Changed status from 'Needs Triage' to: 'Archived'

Germano Cavalcante closed this issue

2022-07-01 15:22:28 +02:00

Germano Cavalcante commented

2022-07-01 15:22:28 +02:00

Build issues are off-topic on the bug tracker as explained by the submission form .

What not to report here
For feature requests, feedback, questions or issues building Blender, see communication channels .

Please ask about this on [blender.chat ]] or [ https:*devtalk.blender.org/c/blender | DevTalk .
You can also propose a patch in the session Submit Code .

I suggest contacting @ideasman42 to propose early support for Python 3.11.

(This task can be reopened as To Do if approved by the developers)

Build issues are off-topic on the bug tracker as explained by the [submission form ](https://developer.blender.org/maniphest/task/edit/form/1/). > **What not to report here** > For feature requests, feedback, questions or issues building Blender, see [communication channels ](https://wiki.blender.org/wiki/Communication/Contact#User_Feedback_and_Requests). Please ask about this on [blender.chat ]] or [[ https:*devtalk.blender.org/c/blender | DevTalk ](https:*blender.chat/channel/blender-coders). You can also propose a patch in the session [Submit Code ](https://developer.blender.org/differential/diff/create/). I suggest contacting @ideasman42 to propose early support for Python 3.11. (This task can be reopened as `To Do` if approved by the developers)

Campbell Barton commented

2022-07-04 13:16:45 +02:00

Changed status from 'Archived' to: 'Confirmed'

Campbell Barton reopened this issue

2022-07-04 13:16:45 +02:00

Campbell Barton self-assigned this 2022-07-04 13:16:45 +02:00

Campbell Barton commented

2022-07-04 13:16:45 +02:00

Recently I looked into supporting Python 3.11, but it needs some careful consideration based on changes to allowed op-codes considered safe for evaluation of PyDrivers. It should be finished in the next day or so.