Benchmark CLI segfaults #92343

Open
opened 2021-10-19 12:25:52 +02:00 by Paul Melis · 2 comments

System Information
Operating system: RHEL 8.4
Graphics card: NVIDIA A100-SXM4-40GB ;-)

Blender Version
Broken: 2.93.1 (official binaries)
Worked: own debug-built version of 2.93.1

Short description of error

Running benchmark-launcher cli 2.0.1 and using CPU for rendering results in a segfault:

snellius paulm@gcn19 12:20 ~$ cat /tmp/blender.crash.txt
# Blender 2.93.1, Commit date: 2021-06-22 05:57, Hash 1b8d33b18c2f

# backtrace
/home/paulm/.cache/blender-benchmark-launcher/blender-versions/6159316089de79ce0c3edec8cfdc5f08a6ca79e6a1fa8642538bc850c43970fe/blender-2.93.1-linux-x64/blender(BLI_system_backtrace+0x20) [0xa5915a0]
/home/paulm/.cache/blender-benchmark-launcher/blender-versions/6159316089de79ce0c3edec8cfdc5f08a6ca79e6a1fa8642538bc850c43970fe/blender-2.93.1-linux-x64/blender() [0xf5f99d]
/lib64/libc.so.6(+0x37400) [0x7fc716283400]
/home/paulm/.cache/blender-benchmark-launcher/blender-versions/6159316089de79ce0c3edec8cfdc5f08a6ca79e6a1fa8642538bc850c43970fe/blender-2.93.1-linux-x64/blender(BLI_findstring+0x35) [0xa51a665]
/home/paulm/.cache/blender-benchmark-launcher/blender-versions/6159316089de79ce0c3edec8cfdc5f08a6ca79e6a1fa8642538bc850c43970fe/blender-2.93.1-linux-x64/blender(IDP_AddToGroup+0x15) [0xf90da5]
/home/paulm/.cache/blender-benchmark-launcher/blender-versions/6159316089de79ce0c3edec8cfdc5f08a6ca79e6a1fa8642538bc850c43970fe/blender-2.93.1-linux-x64/blender(RNA_property_boolean_set+0xce) [0x173995e]
/home/paulm/.cache/blender-benchmark-launcher/blender-versions/6159316089de79ce0c3edec8cfdc5f08a6ca79e6a1fa8642538bc850c43970fe/blender-2.93.1-linux-x64/blender() [0x1878fae]
/home/paulm/.cache/blender-benchmark-launcher/blender-versions/6159316089de79ce0c3edec8cfdc5f08a6ca79e6a1fa8642538bc850c43970fe/blender-2.93.1-linux-x64/blender() [0x187983a]
/home/paulm/.cache/blender-benchmark-launcher/blender-versions/6159316089de79ce0c3edec8cfdc5f08a6ca79e6a1fa8642538bc850c43970fe/blender-2.93.1-linux-x64/blender(PyObject_SetAttr+0x4f) [0x875580f]
/home/paulm/.cache/blender-benchmark-launcher/blender-versions/6159316089de79ce0c3edec8cfdc5f08a6ca79e6a1fa8642538bc850c43970fe/blender-2.93.1-linux-x64/blender(_PyEval_EvalFrameDefault+0x1491) [0xf54461]
/home/paulm/.cache/blender-benchmark-launcher/blender-versions/6159316089de79ce0c3edec8cfdc5f08a6ca79e6a1fa8642538bc850c43970fe/blender-2.93.1-linux-x64/blender() [0xf5203b]
/home/paulm/.cache/blender-benchmark-launcher/blender-versions/6159316089de79ce0c3edec8cfdc5f08a6ca79e6a1fa8642538bc850c43970fe/blender-2.93.1-linux-x64/blender(_PyEval_EvalFrameDefault+0x59cc) [0xf5899c]
/home/paulm/.cache/blender-benchmark-launcher/blender-versions/6159316089de79ce0c3edec8cfdc5f08a6ca79e6a1fa8642538bc850c43970fe/blender-2.93.1-linux-x64/blender() [0xf5203b]
/home/paulm/.cache/blender-benchmark-launcher/blender-versions/6159316089de79ce0c3edec8cfdc5f08a6ca79e6a1fa8642538bc850c43970fe/blender-2.93.1-linux-x64/blender(_PyEval_EvalFrameDefault+0x59cc) [0xf5899c]
/home/paulm/.cache/blender-benchmark-launcher/blender-versions/6159316089de79ce0c3edec8cfdc5f08a6ca79e6a1fa8642538bc850c43970fe/blender-2.93.1-linux-x64/blender() [0x87bfeb4]
/home/paulm/.cache/blender-benchmark-launcher/blender-versions/6159316089de79ce0c3edec8cfdc5f08a6ca79e6a1fa8642538bc850c43970fe/blender-2.93.1-linux-x64/blender(_PyFunction_Vectorcall+0x9a) [0x871246a]
/home/paulm/.cache/blender-benchmark-launcher/blender-versions/6159316089de79ce0c3edec8cfdc5f08a6ca79e6a1fa8642538bc850c43970fe/blender-2.93.1-linux-x64/blender(_PyEval_EvalFrameDefault+0x661d) [0xf595ed]
/home/paulm/.cache/blender-benchmark-launcher/blender-versions/6159316089de79ce0c3edec8cfdc5f08a6ca79e6a1fa8642538bc850c43970fe/blender-2.93.1-linux-x64/blender() [0xf5203b]
/home/paulm/.cache/blender-benchmark-launcher/blender-versions/6159316089de79ce0c3edec8cfdc5f08a6ca79e6a1fa8642538bc850c43970fe/blender-2.93.1-linux-x64/blender(_PyEval_EvalFrameDefault+0x59cc) [0xf5899c]
/home/paulm/.cache/blender-benchmark-launcher/blender-versions/6159316089de79ce0c3edec8cfdc5f08a6ca79e6a1fa8642538bc850c43970fe/blender-2.93.1-linux-x64/blender() [0xf5203b]
/home/paulm/.cache/blender-benchmark-launcher/blender-versions/6159316089de79ce0c3edec8cfdc5f08a6ca79e6a1fa8642538bc850c43970fe/blender-2.93.1-linux-x64/blender(_PyEval_EvalFrameDefault+0x59cc) [0xf5899c]
/home/paulm/.cache/blender-benchmark-launcher/blender-versions/6159316089de79ce0c3edec8cfdc5f08a6ca79e6a1fa8642538bc850c43970fe/blender-2.93.1-linux-x64/blender() [0x87bfeb4]
/home/paulm/.cache/blender-benchmark-launcher/blender-versions/6159316089de79ce0c3edec8cfdc5f08a6ca79e6a1fa8642538bc850c43970fe/blender-2.93.1-linux-x64/blender(_PyEval_EvalCodeWithName+0x4e) [0x87c01fe]
/home/paulm/.cache/blender-benchmark-launcher/blender-versions/6159316089de79ce0c3edec8cfdc5f08a6ca79e6a1fa8642538bc850c43970fe/blender-2.93.1-linux-x64/blender(PyEval_EvalCodeEx+0x3e) [0x87c024e]
/home/paulm/.cache/blender-benchmark-launcher/blender-versions/6159316089de79ce0c3edec8cfdc5f08a6ca79e6a1fa8642538bc850c43970fe/blender-2.93.1-linux-x64/blender(PyEval_EvalCode+0x1b) [0x87c027b]
/home/paulm/.cache/blender-benchmark-launcher/blender-versions/6159316089de79ce0c3edec8cfdc5f08a6ca79e6a1fa8642538bc850c43970fe/blender-2.93.1-linux-x64/blender() [0x87fabe5]
/home/paulm/.cache/blender-benchmark-launcher/blender-versions/6159316089de79ce0c3edec8cfdc5f08a6ca79e6a1fa8642538bc850c43970fe/blender-2.93.1-linux-x64/blender(PyRun_FileExFlags+0xad) [0x87fcced]
/home/paulm/.cache/blender-benchmark-launcher/blender-versions/6159316089de79ce0c3edec8cfdc5f08a6ca79e6a1fa8642538bc850c43970fe/blender-2.93.1-linux-x64/blender() [0x186c02f]
/home/paulm/.cache/blender-benchmark-launcher/blender-versions/6159316089de79ce0c3edec8cfdc5f08a6ca79e6a1fa8642538bc850c43970fe/blender-2.93.1-linux-x64/blender() [0xf5dc1a]
/home/paulm/.cache/blender-benchmark-launcher/blender-versions/6159316089de79ce0c3edec8cfdc5f08a6ca79e6a1fa8642538bc850c43970fe/blender-2.93.1-linux-x64/blender(BLI_args_parse+0xc7) [0xa4ec367]
/home/paulm/.cache/blender-benchmark-launcher/blender-versions/6159316089de79ce0c3edec8cfdc5f08a6ca79e6a1fa8642538bc850c43970fe/blender-2.93.1-linux-x64/blender(main+0x2d5) [0xe872d5]
/lib64/libc.so.6(__libc_start_main+0xf3) [0x7fc71626f493]
/home/paulm/.cache/blender-benchmark-launcher/blender-versions/6159316089de79ce0c3edec8cfdc5f08a6ca79e6a1fa8642538bc850c43970fe/blender-2.93.1-linux-x64/blender() [0xf5c31c]

# Python backtrace
  File "/home/paulm/.cache/blender-benchmark-launcher/benchmark-scripts/dee17c82d883838f6da21e7c86f368cda2ab8399eac10b660e199628aca09ec0/blender-benchmark-script-2.0.1/render.py", line 31 in _enable_cpu_device
  File "/home/paulm/.cache/blender-benchmark-launcher/benchmark-scripts/dee17c82d883838f6da21e7c86f368cda2ab8399eac10b660e199628aca09ec0/blender-benchmark-script-2.0.1/render.py", line 83 in _enable_device
  File "/home/paulm/.cache/blender-benchmark-launcher/benchmark-scripts/dee17c82d883838f6da21e7c86f368cda2ab8399eac10b660e199628aca09ec0/blender-benchmark-script-2.0.1/render.py", line 111 in render
  File "/home/paulm/.cache/blender-benchmark-launcher/benchmark-scripts/dee17c82d883838f6da21e7c86f368cda2ab8399eac10b660e199628aca09ec0/blender-benchmark-script-2.0.1/main.py", line 72 in _benchmark
  File "/home/paulm/.cache/blender-benchmark-launcher/benchmark-scripts/dee17c82d883838f6da21e7c86f368cda2ab8399eac10b660e199628aca09ec0/blender-benchmark-script-2.0.1/main.py", line 98 in main
  File "/home/paulm/.cache/blender-benchmark-launcher/benchmark-scripts/dee17c82d883838f6da21e7c86f368cda2ab8399eac10b660e199628aca09ec0/blender-benchmark-script-2.0.1/main.py", line 102 in <module>

I tried with a debug build of 2.93.1, but then the error doesn't show. See also https://devtalk.blender.org/t/benchmark-cli-segfault/20953/2

Exact steps for others to reproduce the error

  1. Download and extract benchmark-launcher cli 2.0.1
  2. $ ./benchmark-launcher-cli --verbosity 3
  3. Pick 2.93.1
  4. Choose bmw27 scene (download if needed)
  5. Choose CPU device (Intel Xeon Platinum 8360Y CPU @ 2.40GHz (CPU) in this case)
  6. Start benchmarking

By luck I found that I can make the segfault go away by adding an extra call to info._get_compute_devices() in render.py in the _enable_cpu_device() function, calling it just before the for compute_device in info._get_compute_devices(): loop.

**System Information** Operating system: RHEL 8.4 Graphics card: NVIDIA A100-SXM4-40GB ;-) **Blender Version** Broken: 2.93.1 (official binaries) Worked: own debug-built version of 2.93.1 **Short description of error** Running benchmark-launcher cli 2.0.1 and using CPU for rendering results in a segfault: ``` snellius paulm@gcn19 12:20 ~$ cat /tmp/blender.crash.txt # Blender 2.93.1, Commit date: 2021-06-22 05:57, Hash 1b8d33b18c2f # backtrace /home/paulm/.cache/blender-benchmark-launcher/blender-versions/6159316089de79ce0c3edec8cfdc5f08a6ca79e6a1fa8642538bc850c43970fe/blender-2.93.1-linux-x64/blender(BLI_system_backtrace+0x20) [0xa5915a0] /home/paulm/.cache/blender-benchmark-launcher/blender-versions/6159316089de79ce0c3edec8cfdc5f08a6ca79e6a1fa8642538bc850c43970fe/blender-2.93.1-linux-x64/blender() [0xf5f99d] /lib64/libc.so.6(+0x37400) [0x7fc716283400] /home/paulm/.cache/blender-benchmark-launcher/blender-versions/6159316089de79ce0c3edec8cfdc5f08a6ca79e6a1fa8642538bc850c43970fe/blender-2.93.1-linux-x64/blender(BLI_findstring+0x35) [0xa51a665] /home/paulm/.cache/blender-benchmark-launcher/blender-versions/6159316089de79ce0c3edec8cfdc5f08a6ca79e6a1fa8642538bc850c43970fe/blender-2.93.1-linux-x64/blender(IDP_AddToGroup+0x15) [0xf90da5] /home/paulm/.cache/blender-benchmark-launcher/blender-versions/6159316089de79ce0c3edec8cfdc5f08a6ca79e6a1fa8642538bc850c43970fe/blender-2.93.1-linux-x64/blender(RNA_property_boolean_set+0xce) [0x173995e] /home/paulm/.cache/blender-benchmark-launcher/blender-versions/6159316089de79ce0c3edec8cfdc5f08a6ca79e6a1fa8642538bc850c43970fe/blender-2.93.1-linux-x64/blender() [0x1878fae] /home/paulm/.cache/blender-benchmark-launcher/blender-versions/6159316089de79ce0c3edec8cfdc5f08a6ca79e6a1fa8642538bc850c43970fe/blender-2.93.1-linux-x64/blender() [0x187983a] /home/paulm/.cache/blender-benchmark-launcher/blender-versions/6159316089de79ce0c3edec8cfdc5f08a6ca79e6a1fa8642538bc850c43970fe/blender-2.93.1-linux-x64/blender(PyObject_SetAttr+0x4f) [0x875580f] /home/paulm/.cache/blender-benchmark-launcher/blender-versions/6159316089de79ce0c3edec8cfdc5f08a6ca79e6a1fa8642538bc850c43970fe/blender-2.93.1-linux-x64/blender(_PyEval_EvalFrameDefault+0x1491) [0xf54461] /home/paulm/.cache/blender-benchmark-launcher/blender-versions/6159316089de79ce0c3edec8cfdc5f08a6ca79e6a1fa8642538bc850c43970fe/blender-2.93.1-linux-x64/blender() [0xf5203b] /home/paulm/.cache/blender-benchmark-launcher/blender-versions/6159316089de79ce0c3edec8cfdc5f08a6ca79e6a1fa8642538bc850c43970fe/blender-2.93.1-linux-x64/blender(_PyEval_EvalFrameDefault+0x59cc) [0xf5899c] /home/paulm/.cache/blender-benchmark-launcher/blender-versions/6159316089de79ce0c3edec8cfdc5f08a6ca79e6a1fa8642538bc850c43970fe/blender-2.93.1-linux-x64/blender() [0xf5203b] /home/paulm/.cache/blender-benchmark-launcher/blender-versions/6159316089de79ce0c3edec8cfdc5f08a6ca79e6a1fa8642538bc850c43970fe/blender-2.93.1-linux-x64/blender(_PyEval_EvalFrameDefault+0x59cc) [0xf5899c] /home/paulm/.cache/blender-benchmark-launcher/blender-versions/6159316089de79ce0c3edec8cfdc5f08a6ca79e6a1fa8642538bc850c43970fe/blender-2.93.1-linux-x64/blender() [0x87bfeb4] /home/paulm/.cache/blender-benchmark-launcher/blender-versions/6159316089de79ce0c3edec8cfdc5f08a6ca79e6a1fa8642538bc850c43970fe/blender-2.93.1-linux-x64/blender(_PyFunction_Vectorcall+0x9a) [0x871246a] /home/paulm/.cache/blender-benchmark-launcher/blender-versions/6159316089de79ce0c3edec8cfdc5f08a6ca79e6a1fa8642538bc850c43970fe/blender-2.93.1-linux-x64/blender(_PyEval_EvalFrameDefault+0x661d) [0xf595ed] /home/paulm/.cache/blender-benchmark-launcher/blender-versions/6159316089de79ce0c3edec8cfdc5f08a6ca79e6a1fa8642538bc850c43970fe/blender-2.93.1-linux-x64/blender() [0xf5203b] /home/paulm/.cache/blender-benchmark-launcher/blender-versions/6159316089de79ce0c3edec8cfdc5f08a6ca79e6a1fa8642538bc850c43970fe/blender-2.93.1-linux-x64/blender(_PyEval_EvalFrameDefault+0x59cc) [0xf5899c] /home/paulm/.cache/blender-benchmark-launcher/blender-versions/6159316089de79ce0c3edec8cfdc5f08a6ca79e6a1fa8642538bc850c43970fe/blender-2.93.1-linux-x64/blender() [0xf5203b] /home/paulm/.cache/blender-benchmark-launcher/blender-versions/6159316089de79ce0c3edec8cfdc5f08a6ca79e6a1fa8642538bc850c43970fe/blender-2.93.1-linux-x64/blender(_PyEval_EvalFrameDefault+0x59cc) [0xf5899c] /home/paulm/.cache/blender-benchmark-launcher/blender-versions/6159316089de79ce0c3edec8cfdc5f08a6ca79e6a1fa8642538bc850c43970fe/blender-2.93.1-linux-x64/blender() [0x87bfeb4] /home/paulm/.cache/blender-benchmark-launcher/blender-versions/6159316089de79ce0c3edec8cfdc5f08a6ca79e6a1fa8642538bc850c43970fe/blender-2.93.1-linux-x64/blender(_PyEval_EvalCodeWithName+0x4e) [0x87c01fe] /home/paulm/.cache/blender-benchmark-launcher/blender-versions/6159316089de79ce0c3edec8cfdc5f08a6ca79e6a1fa8642538bc850c43970fe/blender-2.93.1-linux-x64/blender(PyEval_EvalCodeEx+0x3e) [0x87c024e] /home/paulm/.cache/blender-benchmark-launcher/blender-versions/6159316089de79ce0c3edec8cfdc5f08a6ca79e6a1fa8642538bc850c43970fe/blender-2.93.1-linux-x64/blender(PyEval_EvalCode+0x1b) [0x87c027b] /home/paulm/.cache/blender-benchmark-launcher/blender-versions/6159316089de79ce0c3edec8cfdc5f08a6ca79e6a1fa8642538bc850c43970fe/blender-2.93.1-linux-x64/blender() [0x87fabe5] /home/paulm/.cache/blender-benchmark-launcher/blender-versions/6159316089de79ce0c3edec8cfdc5f08a6ca79e6a1fa8642538bc850c43970fe/blender-2.93.1-linux-x64/blender(PyRun_FileExFlags+0xad) [0x87fcced] /home/paulm/.cache/blender-benchmark-launcher/blender-versions/6159316089de79ce0c3edec8cfdc5f08a6ca79e6a1fa8642538bc850c43970fe/blender-2.93.1-linux-x64/blender() [0x186c02f] /home/paulm/.cache/blender-benchmark-launcher/blender-versions/6159316089de79ce0c3edec8cfdc5f08a6ca79e6a1fa8642538bc850c43970fe/blender-2.93.1-linux-x64/blender() [0xf5dc1a] /home/paulm/.cache/blender-benchmark-launcher/blender-versions/6159316089de79ce0c3edec8cfdc5f08a6ca79e6a1fa8642538bc850c43970fe/blender-2.93.1-linux-x64/blender(BLI_args_parse+0xc7) [0xa4ec367] /home/paulm/.cache/blender-benchmark-launcher/blender-versions/6159316089de79ce0c3edec8cfdc5f08a6ca79e6a1fa8642538bc850c43970fe/blender-2.93.1-linux-x64/blender(main+0x2d5) [0xe872d5] /lib64/libc.so.6(__libc_start_main+0xf3) [0x7fc71626f493] /home/paulm/.cache/blender-benchmark-launcher/blender-versions/6159316089de79ce0c3edec8cfdc5f08a6ca79e6a1fa8642538bc850c43970fe/blender-2.93.1-linux-x64/blender() [0xf5c31c] # Python backtrace File "/home/paulm/.cache/blender-benchmark-launcher/benchmark-scripts/dee17c82d883838f6da21e7c86f368cda2ab8399eac10b660e199628aca09ec0/blender-benchmark-script-2.0.1/render.py", line 31 in _enable_cpu_device File "/home/paulm/.cache/blender-benchmark-launcher/benchmark-scripts/dee17c82d883838f6da21e7c86f368cda2ab8399eac10b660e199628aca09ec0/blender-benchmark-script-2.0.1/render.py", line 83 in _enable_device File "/home/paulm/.cache/blender-benchmark-launcher/benchmark-scripts/dee17c82d883838f6da21e7c86f368cda2ab8399eac10b660e199628aca09ec0/blender-benchmark-script-2.0.1/render.py", line 111 in render File "/home/paulm/.cache/blender-benchmark-launcher/benchmark-scripts/dee17c82d883838f6da21e7c86f368cda2ab8399eac10b660e199628aca09ec0/blender-benchmark-script-2.0.1/main.py", line 72 in _benchmark File "/home/paulm/.cache/blender-benchmark-launcher/benchmark-scripts/dee17c82d883838f6da21e7c86f368cda2ab8399eac10b660e199628aca09ec0/blender-benchmark-script-2.0.1/main.py", line 98 in main File "/home/paulm/.cache/blender-benchmark-launcher/benchmark-scripts/dee17c82d883838f6da21e7c86f368cda2ab8399eac10b660e199628aca09ec0/blender-benchmark-script-2.0.1/main.py", line 102 in <module> ``` I tried with a debug build of 2.93.1, but then the error doesn't show. See also https://devtalk.blender.org/t/benchmark-cli-segfault/20953/2 **Exact steps for others to reproduce the error** 1. Download and extract benchmark-launcher cli 2.0.1 2. $ ./benchmark-launcher-cli --verbosity 3 3. Pick 2.93.1 4. Choose bmw27 scene (download if needed) 5. Choose CPU device (Intel Xeon Platinum 8360Y CPU @ 2.40GHz (CPU) in this case) 6. Start benchmarking By luck I found that I can make the segfault go away by adding an extra call to `info._get_compute_devices()` in render.py in the `_enable_cpu_device()` function, calling it just before the `for compute_device in info._get_compute_devices():` loop.
Author

Added subscriber: @PaulMelis

Added subscriber: @PaulMelis
Author

The above hack only fixes the CPU-based benchmark, for the GPU-based one it needs the same fix, but in _enable_compute_device() calling info._get_compute_devices() before the devices = info._get_compute_devices() line.

The above hack only fixes the CPU-based benchmark, for the GPU-based one it needs the same fix, but in `_enable_compute_device()` calling `info._get_compute_devices()` before the `devices = info._get_compute_devices()` line.
This repo is archived. You cannot comment on issues.
No description provided.