Blender performance under Linux is flakey ( on my system? ) #54807

New Issue

Kor Boerema · 2018-04-25T11:07:55+02:00

Kor Boerema commented

2018-04-25 11:07:55 +02:00

Hi,

I was hesitant to report this, because I don't really know if it is a bug or if something is wrong with my system, but I've tried almost every trick I know to get it working right and something is still not as it should be.

When running Blender on Linux and doing some sculpting, the system begins to lag. It will take a couple of seconds for the cursor to keep up with the mouse and generally everything runs slowly. This is with a simple scene with one object and 600.000 faces, when trying to quit Blender it will hang for about two to three minutes pegging one CPU at 100%. Switch from object to sculpt mode also takes quite a while.

When running the exact same file on my Windows installation ( dual boot ) everything is snappy and behaves like it should. On my Linux installation I have a VM that has a GPU passed through to it and that also works as it should ( same file ), this has made me startup the VM within Linux and use that for sculpting because it is much faster, which is a bit weird.

System Information
AMD ThreadRipper 1950X
Asrock Taichi
32GB RAM
Geforce 1080GTX TI
Geforce 1080GTX
nvme boot drive ( Windows and Linux each have their own drive, the VM resides on the Linux drive )

Windows VM:
6 cores
8GB RAM
Virtio harddisk
Above mentioned Geforce 1080GTX passed through using KVM,.

Software:
Windows 10 FCU ( VM )
Windows 10 ( Dual boot )
KUbuntu 17.10 Tried drivers: Nouveau, Nvidia 384, 390, 396.
Ubuntu 17.10 ( fresh install different drive )

For Linux I've tried kernels: 4.13 up to 4.14.
Blender Version
On Windows: Blender 279a both for the VM and the dualboot
Linux: Blender 279a, 279b, the latest built of 279 of April 24 and Blender 280 also April 24.

Short description of error
Sculpting lags in Linux, Windows is fine on the same hardware.

Exact steps for others to reproduce the error
On Linux just open the file, do some sculpting and it begins to lag, try to close Blender and one CPU will be pegged for 100% for about 4 minutes. On Windows sculpt some, everything is snappy close Blender without saving ( it prompts ) but closes immediately.
I've included the file ( compressed with 7zip )Dolphin.7z

Hi, I was hesitant to report this, because I don't really know if it is a bug or if something is wrong with my system, but I've tried almost every trick I know to get it working right and something is still not as it should be. When running Blender on Linux and doing some sculpting, the system begins to lag. It will take a couple of seconds for the cursor to keep up with the mouse and generally everything runs slowly. This is with a simple scene with one object and 600.000 faces, when trying to quit Blender it will hang for about two to three minutes pegging one CPU at 100%. Switch from object to sculpt mode also takes quite a while. When running the exact same file on my Windows installation ( dual boot ) everything is snappy and behaves like it should. On my Linux installation I have a VM that has a GPU passed through to it and that also works as it should ( same file ), this has made me startup the VM within Linux and use that for sculpting because it is much faster, which is a bit weird. **System Information** AMD ThreadRipper 1950X Asrock Taichi 32GB RAM Geforce 1080GTX TI Geforce 1080GTX nvme boot drive ( Windows and Linux each have their own drive, the VM resides on the Linux drive ) Windows VM: 6 cores 8GB RAM Virtio harddisk Above mentioned Geforce 1080GTX passed through using KVM,. Software: Windows 10 FCU ( VM ) Windows 10 ( Dual boot ) KUbuntu 17.10 Tried drivers: Nouveau, Nvidia 384, 390, 396. Ubuntu 17.10 ( fresh install different drive ) For Linux I've tried kernels: 4.13 up to 4.14. **Blender Version** On Windows: Blender 279a both for the VM and the dualboot Linux: Blender 279a, 279b, the latest built of 279 of April 24 and Blender 280 also April 24. **Short description of error** Sculpting lags in Linux, Windows is fine on the same hardware. **Exact steps for others to reproduce the error** On Linux just open the file, do some sculpting and it begins to lag, try to close Blender and one CPU will be pegged for 100% for about 4 minutes. On Windows sculpt some, everything is snappy close Blender without saving ( it prompts ) but closes immediately. I've included the file ( compressed with 7zip )[Dolphin.7z](https://archive.blender.org/developer/F2998488/Dolphin.7z)

Kor Boerema commented

2018-04-25 11:07:55 +02:00

Added subscriber: @boemak

Bastien Montagne commented

2018-04-25 11:25:21 +02:00

Added subscriber: @mont29

Bastien Montagne commented

2018-04-25 11:25:21 +02:00

Do you have nvidia drivers installed on linux?

Kor Boerema commented

2018-04-25 11:42:27 +02:00

I've tried both Nouveau and Nvidia drivers. I've been working on this for a couple of days now and saw that there were performance regressions with the newer drivers ( 390 ), so I reverted back to the 384 drivers. I've tried 384 all the way up to the latest drivers ( 396 ). On the KUbuntu and the Ubuntu installation I have also tried the Nouveau drivers, no difference.

Bastien Montagne commented

2018-04-25 18:46:28 +02:00

So not the drivers... could be threading, iirc we had issues with that and ThreadRipper CPUs, but this is supposed to be fixed… Can you try those things:

Try to start Blender in factory settings (--factory-startup commandline option) (this will ensure whether this is a userpref or addon issue or not).
Try the latest build from our buildbot.
Try to run blender from a terminal with single thread (-t 1) option.

So not the drivers... could be threading, iirc we had issues with that and ThreadRipper CPUs, but this is supposed to be fixed… Can you try those things: * Try to start Blender in factory settings (`--factory-startup` commandline option) (this will ensure whether this is a userpref or addon issue or not). * Try the latest build from [our buildbot](https://builder.blender.org/download). * Try to run blender from a terminal with single thread (`-t 1`) option.

Kor Boerema commented

2018-04-25 20:59:55 +02:00

When trying to get some statistics for you I came upon something interesting, it seems that switching from Sculpt mode to Object mode and vice versa just hangs Blender on my machine when using Linux.

When loading a file that is initially saved as being in Sculpt mode, it will easily take 1 min and 40 seconds or more to just open the file. When the file is saved in Object mode it takes 3 seconds or something when opening it again.
When a file is loaded as initially being in Object Mode, switching to Sculpt mode will take from 40 seconds to over a minute. After I click a couple of times from Sculpt mode to Object Mode it works quickly, provided I don't change anything on the object. Seems like something is cached.

Both installations are running with the commandline parameters you provided. ( single threaded, factory settings. )

On Windows none of this is an issue, it just works and is snappy.

Output:
Linux

user@hepheastus:~/Downloads/blender-2.79-92f36586e30-linux-glibc219-x86_64$ ./blender --factory-startup  -t 1
found bundled python: /home/user/Downloads/blender-2.79-92f36586e30-linux-glibc219-x86_64/2.79/python
Read blend: /home/user/Dolphin.blend
Saved session recovery to '/tmp/quit.blend'
Error: Not freed memory blocks: 5, total unfreed memory 0.010681 MB

Blender quit

Windows:

C:\Program Files\Blender Foundation\Blender>blender --factory-startup -t 1
AL lib: (EE) UpdateDeviceParams: Failed to set 44100hz, got 48000hz instead
found bundled python: C:\Program Files\Blender Foundation\Blender\2.79\python
Read blend: Z:\Graphics\Blender\Dolphin.blend
Saved session recovery to 'C:\Users\user\AppData\Local\Temp\quit.blend'

Blender quit

C:\Program Files\Blender Foundation\Blender>

The first run I got an error saying "Warning: Object has non-uniform scale, sculpting may be unpredictable". So I applied CRTL + A and resaved the file, this does not seem to make a difference.

It looks like, and I don't presume to be a Blender developer or anything, don't get me wrong, that if the mesh is changed in any way during sculpting, switching to Object Mode, saving the file will just peg a CPU for a long period of time. Maybe there are a lot of vertex transformations being done at that time to convert the mesh from a dynotopo object to a regular object? The sculpting itself is still laggy after a while.

When trying to get some statistics for you I came upon something interesting, it seems that switching from Sculpt mode to Object mode and vice versa just hangs Blender on my machine when using Linux. When loading a file that is initially saved as being in Sculpt mode, it will easily take 1 min and 40 seconds or more to just open the file. When the file is saved in Object mode it takes 3 seconds or something when opening it again. When a file is loaded as initially being in Object Mode, switching to Sculpt mode will take from 40 seconds to over a minute. After I click a couple of times from Sculpt mode to Object Mode it works quickly, provided I don't change anything on the object. Seems like something is cached. Both installations are running with the commandline parameters you provided. ( single threaded, factory settings. ) On Windows none of this is an issue, it just works and is snappy. Output: Linux ``` user@hepheastus:~/Downloads/blender-2.79-92f36586e30-linux-glibc219-x86_64$ ./blender --factory-startup -t 1 found bundled python: /home/user/Downloads/blender-2.79-92f36586e30-linux-glibc219-x86_64/2.79/python Read blend: /home/user/Dolphin.blend Saved session recovery to '/tmp/quit.blend' Error: Not freed memory blocks: 5, total unfreed memory 0.010681 MB Blender quit ``` Windows: ``` C:\Program Files\Blender Foundation\Blender>blender --factory-startup -t 1 AL lib: (EE) UpdateDeviceParams: Failed to set 44100hz, got 48000hz instead found bundled python: C:\Program Files\Blender Foundation\Blender\2.79\python Read blend: Z:\Graphics\Blender\Dolphin.blend Saved session recovery to 'C:\Users\user\AppData\Local\Temp\quit.blend' Blender quit C:\Program Files\Blender Foundation\Blender> ``` The first run I got an error saying "Warning: Object has non-uniform scale, sculpting may be unpredictable". So I applied CRTL + A and resaved the file, this does not seem to make a difference. It looks like, and I don't presume to be a Blender developer or anything, don't get me wrong, that if the mesh is changed in any way during sculpting, switching to Object Mode, saving the file will just peg a CPU for a long period of time. Maybe there are a lot of vertex transformations being done at that time to convert the mesh from a dynotopo object to a regular object? The sculpting itself is still laggy after a while.

Kor Boerema commented

2018-04-25 23:41:01 +02:00

I did some more digging, because I want to use Blender on Linux instead of Windows:

When running strace on Blender I am seeing a lot of the following:

madvise(0x7f4461091000, 12288, MADV_DONTNEED) = 0
madvise(0x7f44a867b000, 24576, MADV_DONTNEED) = 0
madvise(0x7f44a90fb000, 8192, MADV_DONTNEED) = 0
madvise(0x7f44a8cf1000, 16384, MADV_DONTNEED) = 0
madvise(0x7f44aab7e000, 8192, MADV_DONTNEED) = 0
madvise(0x7f445bec3000, 12288, MADV_DONTNEED) = 0
madvise(0x7f44ab247000, 8192, MADV_DONTNEED) = 0
madvise(0x7f444dc63000, 12288, MADV_DONTNEED) = 0
madvise(0x7f444ddca000, 4096, MADV_DONTNEED) = 0
madvise(0x7f444ddf3000, 4096, MADV_DONTNEED) = 0
madvise(0x7f444de49000, 4096, MADV_DONTNEED) = 0
madvise(0x7f444defe000, 4096, MADV_DONTNEED) = 0
madvise(0x7f444df2c000, 1314816, MADV_DONTNEED) = 0

When the program hangs switching from Sculpt mode to Object Mode.

When running the perf tool I see the following:

perf:

   51.03%  blender       [kernel.kallsyms]        [k] page_fault                                                                                                                                                   
   6.80%  blender       blender                  [.] BM_face_as_array_vert_tri                                                                                                                                    
   5.28%  blender       blender                  [.] BLI_ghash_lookup                                                                                                                                             
   2.41%  blender       blender                  [.] BKE_pbvh_bmesh_update_topology                                                                                                                               
   1.82%  blender       blender                  [.] closest_on_tri_to_point_v3                                                                                                                                   
   1.42%  blender       blender                  [.] BM_vert_calc_normal                                                                                                                                          
   1.25%  blender       blender                  [.] BB_expand                                                                                                                                                    
   1.11%  blender       blender                  [.] GPU_pbvh_bmesh_buffers_update                                                                                                                                
   0.93%  blender       libm-2.26.so             [.] __acosf_finite                                                                                                                                              
   0.83%  blender       blender                  [.] BM_face_calc_normal                                                                                                                                          
   0.79%  blender       blender                  [.] BLI_ghash_ensure_p                                                                                                                                           
   0.67%  blender       blender                  [.] BLI_ghashIterator_step                                                                                                                                       
   0.62%  blender       blender                  [.] normal_tri_v3                                                                                                                                                
   0.62%  blender       blender                  [.] BM_edge_calc_length_squared                                                                                                                             
   0.53%  blender       blender                  [.] 0x0000000000fa23fd                                                                                                                                           
   0.52%  blender       blender                  [.] bmesh_radial_loop_append   

perf record -e page-faults -ag -p blender-pid

+   21.29%     0.00%  blender      blender                  [.] 0x0000000000441f0f
+   21.05%     0.00%  blender      blender                  [.] BLI_ghashutil_ptrcmp
+   11.54%    11.54%  blender      blender                  [.] BLI_mempool_iterstep
+   11.41%    11.41%  blender      blender                  [.] 0x000000000162c369
+    9.79%     9.79%  blender      blender                  [.] BLI_ghash_lookup
+    9.48%     0.00%  blender      [unknown]                [.] 0000000000000000
+    8.40%     0.00%  blender      [unknown]                [.] 0x00007f1dc81ade48
+    5.66%     0.68%  blender      blender                  [.] MEM_lockfree_allocN_len
+    5.58%     0.00%  blender      [unknown]                [.] 0x74db8548ffffffd4
+    5.31%     5.31%  blender      blender                  [.] BKE_pbvh_bmesh_node_save_orig
+    5.26%     5.26%  blender      blender                  [.] BM_face_as_array_vert_tri
+    5.23%     5.23%  blender      libc-2.26.so             [.] __memcpy_ssse3
+    4.96%     4.96%  blender      blender                  [.] BLI_mempool_destroy
+    4.41%     4.41%  blender      blender                  [.] 0x000000000162c3b9
+    4.04%     4.04%  blender      blender                  [.] bmiter__loop_of_face_step
+    3.31%     3.31%  blender      blender                  [.] MEM_lockfree_mallocN
+    2.59%     2.59%  blender      blender                  [.] BM_vert_calc_normal
+    2.24%     2.24%  blender      blender                  [.] BM_mesh_remap
+    2.07%     2.07%  blender      blender                  [.] BLI_ghash_reinsert
+    1.84%     1.84%  blender      blender                  [.] BM_mesh_bm_to_me
+    1.63%     1.63%  blender      blender                  [.] BLI_ghash_insert
+    1.48%     1.48%  blender      libc-2.26.so             [.] __memset_avx2_unaligned_erms
+    1.44%     1.44%  blender      blender                  [.] BKE_pbvh_bmesh_update_topology
     1.39%     1.39%  blender      blender                  [.] 0x0000000001625c38

perf record -p blender-pid -F99
  90.05%  blender      [kernel.kallsyms]        [k] page_fault
   0.99%  blender      blender                  [.] BLI_ghash_lookup
   0.75%  blender      blender                  [.] BM_face_as_array_vert_tri
   0.32%  blender      blender                  [.] BKE_pbvh_bmesh_update_topology
   0.25%  blender      blender                  [.] closest_on_tri_to_point_v3
   0.19%  blender      blender                  [.] BM_mesh_bm_to_me
   0.16%  blender      blender                  [.] BB_expand
   0.16%  blender      blender                  [.] BLI_ghash_ensure_p
   0.14%  blender      [kernel.kallsyms]        [k] __indirect_thunk_start
   0.13%  blender      blender                  [.] bmesh_radial_loop_append
   0.13%  blender      libm-2.26.so             [.] __acosf_finite
   0.12%  blender      blender                  [.] BM_face_calc_normal
   0.12%  blender      blender                  [.] bmesh_disk_edge_append
   0.12%  blender      blender                  [.] GPU_pbvh_bmesh_buffers_update
   0.12%  blender      blender                  [.] BM_vert_calc_normal

When switching modes. It seems there is something going on that causes a lot of page faults.
When searching why page-faults maybe high I came across this: http://sdf.org/~riley/blog/2014/10/27/why-are-my-page-faults-to-high/
My machine uses:

ldd (Ubuntu GLIBC 2.26-0ubuntu2.1) 2.26
Copyright (C) 2017 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
Written by Roland McGrath and Ulrich Drepper.

Blender is compiled against 2.19 right? Can it be that there are syscalls being rerouted on Linux that just destroys the performance?

When the sculpting lags a lot of these show up:

futex(0x7f44d26e98f0, FUTEX_WAKE_PRIVATE, 1) = 1
madvise(0x7f442380e000, 4096, MADV_DONTNEED) = 0
futex(0x7f44d26e9940, FUTEX_WAKE_PRIVATE, 2147483647) = 1
futex(0x7f44d26e9944, FUTEX_WAKE_PRIVATE, 2147483647) = 1
futex(0x7f44d26e9940, FUTEX_WAKE_PRIVATE, 2147483647) = 1
futex(0x7f44d26e9944, FUTEX_WAKE_PRIVATE, 2147483647) = 1
futex(0x7f44d26e9940, FUTEX_WAKE_PRIVATE, 2147483647) = 1
futex(0x7f44d26e98f0, FUTEX_WAKE_PRIVATE, 1) = 1

Hope this helps.

BTW the perf captures come from a freshly installed Fadora system, the LDD output is from my main KUbuntu system. KUbuntu is running from a NVME drive and Fedora from it's own 840Pro. Fedora runs nouveau., I wanted to not run a Ubuntu kernel to see if that makes a difference.
Another thing I've tried is running taskset to pin Blender on one CPU, my machine is running in NUMA configuration, no luck though.

I did some more digging, because I want to use Blender on Linux instead of Windows: When running strace on Blender I am seeing a lot of the following: ``` madvise(0x7f4461091000, 12288, MADV_DONTNEED) = 0 madvise(0x7f44a867b000, 24576, MADV_DONTNEED) = 0 madvise(0x7f44a90fb000, 8192, MADV_DONTNEED) = 0 madvise(0x7f44a8cf1000, 16384, MADV_DONTNEED) = 0 madvise(0x7f44aab7e000, 8192, MADV_DONTNEED) = 0 madvise(0x7f445bec3000, 12288, MADV_DONTNEED) = 0 madvise(0x7f44ab247000, 8192, MADV_DONTNEED) = 0 madvise(0x7f444dc63000, 12288, MADV_DONTNEED) = 0 madvise(0x7f444ddca000, 4096, MADV_DONTNEED) = 0 madvise(0x7f444ddf3000, 4096, MADV_DONTNEED) = 0 madvise(0x7f444de49000, 4096, MADV_DONTNEED) = 0 madvise(0x7f444defe000, 4096, MADV_DONTNEED) = 0 madvise(0x7f444df2c000, 1314816, MADV_DONTNEED) = 0 ``` When the program hangs switching from Sculpt mode to Object Mode. When running the perf tool I see the following: ``` perf: 51.03% blender [kernel.kallsyms] [k] page_fault 6.80% blender blender [.] BM_face_as_array_vert_tri 5.28% blender blender [.] BLI_ghash_lookup 2.41% blender blender [.] BKE_pbvh_bmesh_update_topology 1.82% blender blender [.] closest_on_tri_to_point_v3 1.42% blender blender [.] BM_vert_calc_normal 1.25% blender blender [.] BB_expand 1.11% blender blender [.] GPU_pbvh_bmesh_buffers_update 0.93% blender libm-2.26.so [.] __acosf_finite 0.83% blender blender [.] BM_face_calc_normal 0.79% blender blender [.] BLI_ghash_ensure_p 0.67% blender blender [.] BLI_ghashIterator_step 0.62% blender blender [.] normal_tri_v3 0.62% blender blender [.] BM_edge_calc_length_squared 0.53% blender blender [.] 0x0000000000fa23fd 0.52% blender blender [.] bmesh_radial_loop_append perf record -e page-faults -ag -p blender-pid + 21.29% 0.00% blender blender [.] 0x0000000000441f0f + 21.05% 0.00% blender blender [.] BLI_ghashutil_ptrcmp + 11.54% 11.54% blender blender [.] BLI_mempool_iterstep + 11.41% 11.41% blender blender [.] 0x000000000162c369 + 9.79% 9.79% blender blender [.] BLI_ghash_lookup + 9.48% 0.00% blender [unknown] [.] 0000000000000000 + 8.40% 0.00% blender [unknown] [.] 0x00007f1dc81ade48 + 5.66% 0.68% blender blender [.] MEM_lockfree_allocN_len + 5.58% 0.00% blender [unknown] [.] 0x74db8548ffffffd4 + 5.31% 5.31% blender blender [.] BKE_pbvh_bmesh_node_save_orig + 5.26% 5.26% blender blender [.] BM_face_as_array_vert_tri + 5.23% 5.23% blender libc-2.26.so [.] __memcpy_ssse3 + 4.96% 4.96% blender blender [.] BLI_mempool_destroy + 4.41% 4.41% blender blender [.] 0x000000000162c3b9 + 4.04% 4.04% blender blender [.] bmiter__loop_of_face_step + 3.31% 3.31% blender blender [.] MEM_lockfree_mallocN + 2.59% 2.59% blender blender [.] BM_vert_calc_normal + 2.24% 2.24% blender blender [.] BM_mesh_remap + 2.07% 2.07% blender blender [.] BLI_ghash_reinsert + 1.84% 1.84% blender blender [.] BM_mesh_bm_to_me + 1.63% 1.63% blender blender [.] BLI_ghash_insert + 1.48% 1.48% blender libc-2.26.so [.] __memset_avx2_unaligned_erms + 1.44% 1.44% blender blender [.] BKE_pbvh_bmesh_update_topology 1.39% 1.39% blender blender [.] 0x0000000001625c38 perf record -p blender-pid -F99 90.05% blender [kernel.kallsyms] [k] page_fault 0.99% blender blender [.] BLI_ghash_lookup 0.75% blender blender [.] BM_face_as_array_vert_tri 0.32% blender blender [.] BKE_pbvh_bmesh_update_topology 0.25% blender blender [.] closest_on_tri_to_point_v3 0.19% blender blender [.] BM_mesh_bm_to_me 0.16% blender blender [.] BB_expand 0.16% blender blender [.] BLI_ghash_ensure_p 0.14% blender [kernel.kallsyms] [k] __indirect_thunk_start 0.13% blender blender [.] bmesh_radial_loop_append 0.13% blender libm-2.26.so [.] __acosf_finite 0.12% blender blender [.] BM_face_calc_normal 0.12% blender blender [.] bmesh_disk_edge_append 0.12% blender blender [.] GPU_pbvh_bmesh_buffers_update 0.12% blender blender [.] BM_vert_calc_normal ``` When switching modes. It seems there is something going on that causes a lot of page faults. When searching why page-faults maybe high I came across this: http://sdf.org/~riley/blog/2014/10/27/why-are-my-page-faults-to-high/ My machine uses: ``` ldd (Ubuntu GLIBC 2.26-0ubuntu2.1) 2.26 Copyright (C) 2017 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. Written by Roland McGrath and Ulrich Drepper. ``` Blender is compiled against 2.19 right? Can it be that there are syscalls being rerouted on Linux that just destroys the performance? When the sculpting lags a lot of these show up: ``` futex(0x7f44d26e98f0, FUTEX_WAKE_PRIVATE, 1) = 1 madvise(0x7f442380e000, 4096, MADV_DONTNEED) = 0 futex(0x7f44d26e9940, FUTEX_WAKE_PRIVATE, 2147483647) = 1 futex(0x7f44d26e9944, FUTEX_WAKE_PRIVATE, 2147483647) = 1 futex(0x7f44d26e9940, FUTEX_WAKE_PRIVATE, 2147483647) = 1 futex(0x7f44d26e9944, FUTEX_WAKE_PRIVATE, 2147483647) = 1 futex(0x7f44d26e9940, FUTEX_WAKE_PRIVATE, 2147483647) = 1 futex(0x7f44d26e98f0, FUTEX_WAKE_PRIVATE, 1) = 1 ``` Hope this helps. BTW the perf captures come from a freshly installed Fadora system, the LDD output is from my main KUbuntu system. KUbuntu is running from a NVME drive and Fedora from it's own 840Pro. Fedora runs nouveau., I wanted to not run a Ubuntu kernel to see if that makes a difference. Another thing I've tried is running taskset to pin Blender on one CPU, my machine is running in NUMA configuration, no luck though.

Bastien Montagne commented

2018-04-26 16:07:46 +02:00

Added subscriber: @Sergey

Bastien Montagne commented

2018-04-26 16:07:46 +02:00

There is definitively something wrong with memory handling here, and yes, we use oldish glibc to be able to run on oldish distros. Afraid am not much knowledgeable here, @Sergey should be able to help more hopefully?

Ray molenkamp commented

2018-04-26 16:10:59 +02:00

Added subscriber: @LazyDodo

Ray molenkamp commented

2018-04-26 16:10:59 +02:00

#54287 might be related?

YAFU commented

2018-04-26 16:40:57 +02:00

Added subscriber: @YAFU

YAFU commented

2018-04-26 16:40:57 +02:00

Hi. Kubuntu Linux 18.04, i7-3770, GTX 960, 16GB RAM

I can not reproduce extreme lag or hang with default Dolphin.blend.

But if I zoom out view (or use big size brush) and I sculpt on dolphin, hang and lag start happening. If I reduce "Spacing" percentage in Dyntopo Stroke, things get worse (I have to kill Blender process).
I do not have Windows to make comparisons.

Hi. Kubuntu Linux 18.04, i7-3770, GTX 960, 16GB RAM I can not reproduce extreme lag or hang with default Dolphin.blend. But if I zoom out view (or use big size brush) and I sculpt on dolphin, hang and lag start happening. If I reduce "Spacing" percentage in Dyntopo Stroke, things get worse (I have to kill Blender process). I do not have Windows to make comparisons.

YAFU commented

2018-04-26 17:03:00 +02:00

LazyDodo , Sorry, I had not paid attention to your message.
Supposedly, that problem was solved (reverted) for 2.79b?. This problem also occurs in Blender 2.79b, here the system graph while Blender is hanging (big brush and spacing = 1%):

But I do not have Windows to compare, I'm not sure if this is a normal behavior due to a heavy amount of vertices.

LazyDodo , Sorry, I had not paid attention to your message. Supposedly, that problem was solved (reverted) for 2.79b?. This problem also occurs in Blender 2.79b, here the system graph while Blender is hanging (big brush and spacing = 1%): ![System.jpg](https://archive.blender.org/developer/F3032315/System.jpg) But I do not have Windows to compare, I'm not sure if this is a normal behavior due to a heavy amount of vertices.

Sergey Sharybin commented

2018-04-26 17:15:39 +02:00

We dynamically link against older libc. This means, we only limiting maximum feature-set we use from libc, but we don't "lock" implementation of those features to older libc. All the functions form libc are coming from library on your particular machine.

There are two known issues with those generations of CPUs to me. One of them we've fixed in 55696b56d9. The other one needs microcode update and (AFAIR) some kernel command line argument. That issue is visible in dmesg. So would be interesting to see if there's anything obvious wrong in there.

We dynamically link against older libc. This means, we only limiting maximum feature-set we use from libc, but we don't "lock" implementation of those features to older libc. All the functions form libc are coming from library on your particular machine. There are two known issues with those generations of CPUs to me. One of them we've fixed in 55696b56d9. The other one needs microcode update and (AFAIR) some kernel command line argument. That issue is visible in `dmesg`. So would be interesting to see if there's anything obvious wrong in there.

Kor Boerema commented

2018-04-26 17:42:29 +02:00

I was wondering if maybe the newer libc versions might have done something with certain memory management systemcalls.

I am interested in the microcode update you mentioned and the kernel parameters.

I was wondering if maybe the newer libc versions might have done something with certain memory management systemcalls. I am interested in the microcode update you mentioned and the kernel parameters.

Sergey Sharybin commented

2018-04-26 18:07:31 +02:00

That will all depend on your specific linux distro. For microcode on Debian based system there is amd64-microcode package.

The kernel parameter i've referenced to is ASLR (address space randomization). Is actually done via sysctl setting kernel.randomize_va_space = 0. Look up online how to set that on your specific distro.

P.S. ASLR works on a page level, and it might be using page faults for something? Hrrm...

That will all depend on your specific linux distro. For microcode on Debian based system there is amd64-microcode package. The kernel parameter i've referenced to is ASLR (address space randomization). Is actually done via sysctl setting `kernel.randomize_va_space = 0`. Look up online how to set that on your specific distro. P.S. ASLR works on a page level, and it might be using page faults for something? Hrrm...

Kor Boerema commented

2018-04-27 21:13:51 +02:00

I won't have the chance to change this for a couple of days, I will post back when I have the chance to change my system to non-ASLR and run some benchmarks.

Kor Boerema commented

2018-05-02 23:57:52 +02:00

I finally had a chance to test it without ASLR and I am sorry to say that it does not seem to make a difference.

I used this command:

user@host:~$ echo 0 | sudo tee /proc/sys/kernel/randomize_va_space 
[sudo] password for user: 
0

Following this advice on how to disable ASLR on Ubuntu systems: https://askubuntu.com/questions/318315/how-can-i-temporarily-disable-aslr-address-space-layout-randomization

Any other things I might try?

I finally had a chance to test it without ASLR and I am sorry to say that it does not seem to make a difference. I used this command: ``` user@host:~$ echo 0 | sudo tee /proc/sys/kernel/randomize_va_space [sudo] password for user: 0 ``` Following this advice on how to disable ASLR on Ubuntu systems: https://askubuntu.com/questions/318315/how-can-i-temporarily-disable-aslr-address-space-layout-randomization Any other things I might try?

Christian Hubert commented

2018-05-28 07:43:24 +02:00

Added subscriber: @christian

Lukas commented

2018-05-31 13:26:53 +02:00

Added subscriber: @lsstratmann

Kor Boerema commented

2018-06-06 13:32:27 +02:00

So, can I test anything else? I have some time to spend on this and I'd like to use Linux instead of Windows.

James Gregory commented

2019-03-21 20:44:13 +01:00

Added subscriber: @j4m3z0r

James Gregory commented

2019-03-21 20:44:13 +01:00

I've been doing some experimenting as part of working through this bug: https://developer.blender.org/T62718 and discovered something that might be of interest here: I just downloaded the 2.8 beta build of Blender for win64 and ran it under Wine (using winehq's wine-stable package) on Ubuntu 18.04. This seems to get me very close to the performance I see on Windows. I'm not sure I'd actually recommend this as a general solution, but it would lend evidence to the idea that this is something different with the Blender binaries produced for Linux vs Windows, as with WINE we're ultimately dependent on Linux's system libraries (I had initially thought it might be the allocator, but swapping in jemalloc via LD_PRELOAD didn't make any difference).

My test case was using dyntopo per the ticket I linked to above, so I have no idea if this will generalize to the other use-cases you were seeing issues on. However it seems like a relatively simple test and may help guide us on where we should be looking.

I've been doing some experimenting as part of working through this bug: https://developer.blender.org/T62718 and discovered something that might be of interest here: I just downloaded the 2.8 beta build of Blender for win64 and ran it under Wine (using winehq's wine-stable package) on Ubuntu 18.04. This seems to get me very close to the performance I see on Windows. I'm not sure I'd actually recommend this as a general solution, but it would lend evidence to the idea that this is something different with the Blender binaries produced for Linux vs Windows, as with WINE we're ultimately dependent on Linux's system libraries (I had initially thought it might be the allocator, but swapping in jemalloc via LD_PRELOAD didn't make any difference). My test case was using dyntopo per the ticket I linked to above, so I have no idea if this will generalize to the other use-cases you were seeing issues on. However it seems like a relatively simple test and may help guide us on where we should be looking.

Ray molenkamp commented

2019-03-21 21:22:10 +01:00

I think the default build of blender will try to statically link jmemalloc on linux, so it may not be using the system libs, could be worth trying a build with the cmake option DWITH_MEM_JEMALLOC set to Off just to see if its our copy of the lib or not that is causing issues.

I think the default build of blender will try to statically link jmemalloc on linux, so it may not be using the system libs, could be worth trying a build with the cmake option `DWITH_MEM_JEMALLOC` set to `Off` just to see if its our copy of the lib or not that is causing issues.

Kor Boerema commented

2019-03-21 21:36:01 +01:00

I thought this would never be updated anymore, glad it is still being investigated.

I also use Looking Glass on Linux to run a Windows VM and they ran into memcopy issues using GCC. On a lark I compiled blender using -fno-builtin-memcpy just to see what it would do, turns out not much.

The same performance degradation is still there. On Windows no problem.

Thread on SO: https://stackoverflow.com/questions/50422510/why-is-memcpy-slow-in-32-bit-mode-with-gcc-march-native-on-ryzen-for-large-buf

I thought this would never be updated anymore, glad it is still being investigated. I also use Looking Glass on Linux to run a Windows VM and they ran into memcopy issues using GCC. On a lark I compiled blender using -fno-builtin-memcpy just to see what it would do, turns out not much. The same performance degradation is still there. On Windows no problem. Thread on SO: https://stackoverflow.com/questions/50422510/why-is-memcpy-slow-in-32-bit-mode-with-gcc-march-native-on-ryzen-for-large-buf

Kor Boerema commented

2019-03-21 21:37:54 +01:00

Have you seen this: https://blender.stackexchange.com/questions/119999/linux-ubuntu-18-04-sculpting-performance-dyntopo-is-terrible-in-comparison-t

Seems I am not the only one.

Have you seen this: https://blender.stackexchange.com/questions/119999/linux-ubuntu-18-04-sculpting-performance-dyntopo-is-terrible-in-comparison-t Seems I am not the only one.

YAFU commented

2019-03-21 21:44:20 +01:00

LazyDodo, Hi.
I am building by myself in Linux and I do not notice difference between my builds, my builds with DWITH_MEM_JEMALLOC set to Off, or buildbot builds. The performance is still worse in Linux, for example with the file that I have shared here:
https://developer.blender.org/T62718#645732

LazyDodo, Hi. I am building by myself in Linux and I do not notice difference between my builds, my builds with DWITH_MEM_JEMALLOC set to Off, or buildbot builds. The performance is still worse in Linux, for example with the file that I have shared here: https://developer.blender.org/T62718#645732

James Gregory commented

2019-03-21 21:47:20 +01:00

Ok, so another random data-point: I ran Blender under Linux's 'perf' tool, and spent a couple of minutes doing this grap-with-dyntopo test I've been investigating. Here's an excerpt:

Samples: 187K of event 'cycles:ppp', Event count (approx.): 181242304472, Thread: blender, DSO: blender
Overhead  Command  Symbol                                                                                                       ◆
  34.01%  blender  [.] BLI_ghash_lookup                                                                                         ▒
  11.13%  blender  [.] update_node_vb                                                                                           ▒
   8.08%  blender  [.] BLI_ghashIterator_step                                                                                   ▒
   7.36%  blender  [.] sculpt_undo_bmesh_push                                                                                   ▒
   6.58%  blender  [.] BLI_ghash_ensure_p                                                                                       ▒
   3.80%  blender  [.] paint_mesh_restore_co_task_cb                                                                            ▒
   2.68%  blender  [.] BB_expand                                                                                                ▒

That BLI_ghash_lookup looks pretty innocuous, so I have to assume it is appearing so frequently as a result of it being called many times. It does use the UNLIKELY hint to the compiler. Is it possible that the hints are making things worse?

Based on the data-point that the Windows binaries under WINE are faster (suggesting possibly better code generation from the Windows compilers), I wondered if rebuilding with a different compiler might help, so went and installed LLVM 8, which gets me the following crash:

# backtrace
/home/james/src/3rdparty/blender-git/build-linux/bin/blender(BLI_system_backtrace+0x20) [0x13b9f00]
/home/james/src/3rdparty/blender-git/build-linux/bin/blender() [0x94c2fb]
/lib/x86_64-linux-gnu/libc.so.6(+0x3ef20) [0x7f8e1c19ff20]
/home/james/src/3rdparty/blender-git/build-linux/bin/blender(gpu_context_remove_batch+0x4c) [0xeb3a5c]
/home/james/src/3rdparty/blender-git/build-linux/bin/blender(GPU_batch_vao_cache_clear+0x103) [0xeacab3]
/home/james/src/3rdparty/blender-git/build-linux/bin/blender(GPU_batch_discard+0x9) [0xeaccb9]
/home/james/src/3rdparty/blender-git/build-linux/bin/blender() [0xecc81e]
/home/james/src/3rdparty/blender-git/build-linux/bin/blender(DRW_mesh_batch_cache_free+0x9) [0xecc489]
/home/james/src/3rdparty/blender-git/build-linux/bin/blender(BKE_mesh_runtime_clear_cache+0x6a) [0x10e221a]
/home/james/src/3rdparty/blender-git/build-linux/bin/blender(BKE_mesh_free+0x13) [0x10c7c03]
/home/james/src/3rdparty/blender-git/build-linux/bin/blender(BKE_object_free_derived_caches+0x24a) [0x10ff0ba]
/home/james/src/3rdparty/blender-git/build-linux/bin/blender(BKE_object_free+0x23) [0x10ffa33]
/home/james/src/3rdparty/blender-git/build-linux/bin/blender(_ZN3DEG32deg_free_copy_on_write_datablockEP2ID+0x1e1) [0x13db6a1]
/home/james/src/3rdparty/blender-git/build-linux/bin/blender(_ZN3DEG6IDNode7destroyEv+0x39) [0x13e1229]
/home/james/src/3rdparty/blender-git/build-linux/bin/blender(_ZN3DEG9Depsgraph14clear_id_nodesEv+0x133) [0x13c25e3]
/home/james/src/3rdparty/blender-git/build-linux/bin/blender(_ZN3DEG9DepsgraphD1Ev+0xc) [0x13c23fc]
/home/james/src/3rdparty/blender-git/build-linux/bin/blender(DEG_graph_free+0xe) [0x13c308e]
/home/james/src/3rdparty/blender-git/build-linux/bin/blender(BLI_ghash_free+0x58) [0x135a5b8]
/home/james/src/3rdparty/blender-git/build-linux/bin/blender(BKE_scene_free_ex+0x157) [0x1144ae7]
/home/james/src/3rdparty/blender-git/build-linux/bin/blender(BKE_id_free_ex+0x9a) [0x10b343a]
/home/james/src/3rdparty/blender-git/build-linux/bin/blender(BKE_main_free+0x292) [0x10b5c62]
/home/james/src/3rdparty/blender-git/build-linux/bin/blender(BKE_blender_globals_clear+0x10) [0x10130e0]
/home/james/src/3rdparty/blender-git/build-linux/bin/blender() [0x10146d7]
/home/james/src/3rdparty/blender-git/build-linux/bin/blender(BKE_blendfile_read+0xe4) [0x10143d4]
/home/james/src/3rdparty/blender-git/build-linux/bin/blender(WM_file_read+0x1ec) [0x95745c]
/home/james/src/3rdparty/blender-git/build-linux/bin/blender() [0x959b4c]
/home/james/src/3rdparty/blender-git/build-linux/bin/blender() [0x955642]
/home/james/src/3rdparty/blender-git/build-linux/bin/blender() [0x9509ab]
/home/james/src/3rdparty/blender-git/build-linux/bin/blender(WM_operator_name_call_ptr+0x19) [0x951569]
/home/james/src/3rdparty/blender-git/build-linux/bin/blender() [0xb0a70c]
/home/james/src/3rdparty/blender-git/build-linux/bin/blender() [0xb06247]
/home/james/src/3rdparty/blender-git/build-linux/bin/blender() [0x955ea0]
/home/james/src/3rdparty/blender-git/build-linux/bin/blender() [0x9529bc]
/home/james/src/3rdparty/blender-git/build-linux/bin/blender(wm_event_do_handlers+0x551) [0x952081]
/home/james/src/3rdparty/blender-git/build-linux/bin/blender(WM_main+0x20) [0x94ccc0]
/home/james/src/3rdparty/blender-git/build-linux/bin/blender(main+0x373) [0x948d73]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xe7) [0x7f8e1c182b97]
/home/james/src/3rdparty/blender-git/build-linux/bin/blender(_start+0x2a) [0x94893a]

However, this is all pretty superficial analysis. I suspect we'll need to look more closely at what's going on to resolve this.

Ok, so another random data-point: I ran Blender under Linux's 'perf' tool, and spent a couple of minutes doing this grap-with-dyntopo test I've been investigating. Here's an excerpt: ``` Samples: 187K of event 'cycles:ppp', Event count (approx.): 181242304472, Thread: blender, DSO: blender Overhead Command Symbol ◆ 34.01% blender [.] BLI_ghash_lookup ▒ 11.13% blender [.] update_node_vb ▒ 8.08% blender [.] BLI_ghashIterator_step ▒ 7.36% blender [.] sculpt_undo_bmesh_push ▒ 6.58% blender [.] BLI_ghash_ensure_p ▒ 3.80% blender [.] paint_mesh_restore_co_task_cb ▒ 2.68% blender [.] BB_expand ▒ ``` That BLI_ghash_lookup looks pretty innocuous, so I have to assume it is appearing so frequently as a result of it being called many times. It does use the UNLIKELY hint to the compiler. Is it possible that the hints are making things worse? Based on the data-point that the Windows binaries under WINE are faster (suggesting possibly better code generation from the Windows compilers), I wondered if rebuilding with a different compiler might help, so went and installed LLVM 8, which gets me the following crash: ``` # backtrace /home/james/src/3rdparty/blender-git/build-linux/bin/blender(BLI_system_backtrace+0x20) [0x13b9f00] /home/james/src/3rdparty/blender-git/build-linux/bin/blender() [0x94c2fb] /lib/x86_64-linux-gnu/libc.so.6(+0x3ef20) [0x7f8e1c19ff20] /home/james/src/3rdparty/blender-git/build-linux/bin/blender(gpu_context_remove_batch+0x4c) [0xeb3a5c] /home/james/src/3rdparty/blender-git/build-linux/bin/blender(GPU_batch_vao_cache_clear+0x103) [0xeacab3] /home/james/src/3rdparty/blender-git/build-linux/bin/blender(GPU_batch_discard+0x9) [0xeaccb9] /home/james/src/3rdparty/blender-git/build-linux/bin/blender() [0xecc81e] /home/james/src/3rdparty/blender-git/build-linux/bin/blender(DRW_mesh_batch_cache_free+0x9) [0xecc489] /home/james/src/3rdparty/blender-git/build-linux/bin/blender(BKE_mesh_runtime_clear_cache+0x6a) [0x10e221a] /home/james/src/3rdparty/blender-git/build-linux/bin/blender(BKE_mesh_free+0x13) [0x10c7c03] /home/james/src/3rdparty/blender-git/build-linux/bin/blender(BKE_object_free_derived_caches+0x24a) [0x10ff0ba] /home/james/src/3rdparty/blender-git/build-linux/bin/blender(BKE_object_free+0x23) [0x10ffa33] /home/james/src/3rdparty/blender-git/build-linux/bin/blender(_ZN3DEG32deg_free_copy_on_write_datablockEP2ID+0x1e1) [0x13db6a1] /home/james/src/3rdparty/blender-git/build-linux/bin/blender(_ZN3DEG6IDNode7destroyEv+0x39) [0x13e1229] /home/james/src/3rdparty/blender-git/build-linux/bin/blender(_ZN3DEG9Depsgraph14clear_id_nodesEv+0x133) [0x13c25e3] /home/james/src/3rdparty/blender-git/build-linux/bin/blender(_ZN3DEG9DepsgraphD1Ev+0xc) [0x13c23fc] /home/james/src/3rdparty/blender-git/build-linux/bin/blender(DEG_graph_free+0xe) [0x13c308e] /home/james/src/3rdparty/blender-git/build-linux/bin/blender(BLI_ghash_free+0x58) [0x135a5b8] /home/james/src/3rdparty/blender-git/build-linux/bin/blender(BKE_scene_free_ex+0x157) [0x1144ae7] /home/james/src/3rdparty/blender-git/build-linux/bin/blender(BKE_id_free_ex+0x9a) [0x10b343a] /home/james/src/3rdparty/blender-git/build-linux/bin/blender(BKE_main_free+0x292) [0x10b5c62] /home/james/src/3rdparty/blender-git/build-linux/bin/blender(BKE_blender_globals_clear+0x10) [0x10130e0] /home/james/src/3rdparty/blender-git/build-linux/bin/blender() [0x10146d7] /home/james/src/3rdparty/blender-git/build-linux/bin/blender(BKE_blendfile_read+0xe4) [0x10143d4] /home/james/src/3rdparty/blender-git/build-linux/bin/blender(WM_file_read+0x1ec) [0x95745c] /home/james/src/3rdparty/blender-git/build-linux/bin/blender() [0x959b4c] /home/james/src/3rdparty/blender-git/build-linux/bin/blender() [0x955642] /home/james/src/3rdparty/blender-git/build-linux/bin/blender() [0x9509ab] /home/james/src/3rdparty/blender-git/build-linux/bin/blender(WM_operator_name_call_ptr+0x19) [0x951569] /home/james/src/3rdparty/blender-git/build-linux/bin/blender() [0xb0a70c] /home/james/src/3rdparty/blender-git/build-linux/bin/blender() [0xb06247] /home/james/src/3rdparty/blender-git/build-linux/bin/blender() [0x955ea0] /home/james/src/3rdparty/blender-git/build-linux/bin/blender() [0x9529bc] /home/james/src/3rdparty/blender-git/build-linux/bin/blender(wm_event_do_handlers+0x551) [0x952081] /home/james/src/3rdparty/blender-git/build-linux/bin/blender(WM_main+0x20) [0x94ccc0] /home/james/src/3rdparty/blender-git/build-linux/bin/blender(main+0x373) [0x948d73] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xe7) [0x7f8e1c182b97] /home/james/src/3rdparty/blender-git/build-linux/bin/blender(_start+0x2a) [0x94893a] ``` However, this is all pretty superficial analysis. I suspect we'll need to look more closely at what's going on to resolve this.

Kor Boerema commented

2019-03-21 21:56:58 +01:00

Gut feelings and assumptions and all, but memory allocation issues? Sculpting involves some pretty large vertex arrays, or some other data structures I'll bet.

I would not know how to profile those on Linux though.

Gut feelings and assumptions and all, but memory allocation issues? Sculpting involves some pretty large vertex arrays, or some other data structures I'll bet. I would not know how to profile those on Linux though.

Simon G. commented

2019-03-21 22:04:44 +01:00

Added subscriber: @intrigus-3

James Gregory commented

2019-03-21 22:33:26 +01:00

Ok, so back-tracking on this issue a little bit, I just wanted to see if we have consensus about the actual problem. It seems like the issue is specifically: Linux is significantly slower than Windows when using dynamic topology for sculpting. I'm able to repro this on the dolphin file. There may be other performance gaps, but that seems to be the issue that is impacting this ticket in particular. Is that a fair characterization of the issue?

James Gregory commented

2019-03-21 22:41:58 +01:00

Ok, and running perf on blender on a session when I load up the dolphin and just try to do a simple grab with dyntopo enabled reveals this:

Samples: 1M of event 'cycles:ppp', Event count (approx.): 1081833253304
Overhead  Command      Shared Object                Symbol                                                                              ◆
  90.76%  blender      blender                      [.] BLI_mempool_free                                                                ▒
   1.05%  blender      blender                      [.] BM_face_as_array_vert_tri                                                       ▒
   0.60%  blender      [kernel]                     [k] 0xffffffffb4171258                                                              ▒
   0.55%  blender      blender                      [.] BLI_ghash_haskey                                                                ▒
   0.55%  blender      blender                      [.] BLI_ghash_lookup                                                                ▒

Ok, so we're looking for something that invokes that BLI_mempool_free method and doesn't happen (or happens less) on Windows. Fortunately there are only 13 files that contain references to BLI_mempool_free.

Ok, and running perf on blender on a session when I load up the dolphin and just try to do a simple grab with dyntopo enabled reveals this: ``` Samples: 1M of event 'cycles:ppp', Event count (approx.): 1081833253304 Overhead Command Shared Object Symbol ◆ 90.76% blender blender [.] BLI_mempool_free ▒ 1.05% blender blender [.] BM_face_as_array_vert_tri ▒ 0.60% blender [kernel] [k] 0xffffffffb4171258 ▒ 0.55% blender blender [.] BLI_ghash_haskey ▒ 0.55% blender blender [.] BLI_ghash_lookup ▒ ``` Ok, so we're looking for something that invokes that BLI_mempool_free method and doesn't happen (or happens less) on Windows. Fortunately there are only 13 files that contain references to BLI_mempool_free.

James Gregory commented

2019-03-21 22:56:34 +01:00

Ok, and so just to clarify, the above was on the system version of Blender I had installed. Re-running this on Blender 2.8 official binaries yields something different. I also ran perf with -g this time, to capture stack traces. Unfortunately the official builds don't seem to have complete debug symbols:

  Children      Self  Command          Shared Object                Symbol                                       ◆
-   30.86%    30.69%  blender          blender                      [.] BM_face_as_array_vert_tri                ▒
   - 9.91% 0x4ffffffff                                                                                           ▒
        BM_face_as_array_vert_tri                                                                                ▒
     5.89% BM_face_as_array_vert_tri                                                                             ▒
   - 0.52% 0x7f648c188c68                                                                                        ▒
        0x7f64469188d0                                                                                           ▒
-   23.97%     0.00%  blender          [kernel]                     [k] 0x00007f64469188d0                       ▒
     0x7f64469188d0                                                                                              ▒
-   19.69%    19.54%  blender          blender                      [.] BLI_ghash_lookup                         ▒
   - 10.04% 0x441f0f                                                                                             ▒
        BLI_ghashutil_intcmp                                                                                     ▒
        BLI_ghash_lookup                                                                                         ▒
   - 9.36% 0x1f0fc3f80904e8                                                                                      ▒
        BLI_ghashutil_ptrcmp                                                                                     ▒
        BLI_ghash_lookup                                                                                         ▒
-   17.24%     0.00%  blender          [kernel]                     [k] 0x00000004ffffffff                       ▒
   - 0x4ffffffff                                                                                                 ▒
        9.97% BM_face_as_array_vert_tri                                                                          ▒
        2.92% BKE_pbvh_bmesh_update_topology                                                                     ▒
        2.30% closest_on_tri_to_point_v3                                                                         ▒
        0.71% BM_edge_calc_length_squared                                                                        ▒
-   11.59%     0.05%  blender          blender                      [.] BLI_ghashutil_intcmp                     ▒
   - 11.54% BLI_ghashutil_intcmp                                                                                 ▒
        10.11% BLI_ghash_lookup                                                                                  ▒
        1.23% BLI_ghash_ensure_p                                                                                 ▒
-   11.52%     0.00%  blender          blender                      [.] 0x0000000000441f0f                       ▒
   - 0x441f0f                                                                                                    ▒
      - 11.52% BLI_ghashutil_intcmp                                                                              ▒
           10.11% BLI_ghash_lookup                                                                               ▒
           1.18% BLI_ghash_ensure_p                                                                              ▒
-    9.84%     0.06%  blender          blender                      [.] BLI_ghashutil_ptrcmp                     ▒
   - 9.78% BLI_ghashutil_ptrcmp                                                                                  ▒
        9.43% BLI_ghash_lookup                                                                                   ▒
-    9.84%     0.00%  blender          [kernel]                     [k] 0x001f0fc3f80904e8                       ▒
     0x1f0fc3f80904e8                                                                                            ▒
   - BLI_ghashutil_ptrcmp                                                                                        ▒
        9.43% BLI_ghash_lookup                                                                                   ▒
-    7.19%     7.15%  blender          blender                      [.] closest_on_tri_to_point_v3               ▒
   - 2.29% 0x4ffffffff                                                                                           ▒
        closest_on_tri_to_point_v3                                                                               ▒
     1.37% closest_on_tri_to_point_v3                                                                            ▒
-    6.05%     0.25%  blender          [kernel]                     [k] 0000000000000000                         ▒
   - 5.88% 0                                                                                                     ▒
        1.63% sculpt_combine_proxies_task_cb                                                                     ▒

That's all the time I have to spend on this today. I'm new to tinkering with Blender's code (and have a rocky past with cmake), so if anyone has a recipe I can paste into my terminal to get a build of Blender with debug symbols for Blender and all of its dependencies, that would help a lot.

Ok, and so just to clarify, the above was on the system version of Blender I had installed. Re-running this on Blender 2.8 official binaries yields something different. I also ran perf with -g this time, to capture stack traces. Unfortunately the official builds don't seem to have complete debug symbols: ``` Children Self Command Shared Object Symbol ◆ - 30.86% 30.69% blender blender [.] BM_face_as_array_vert_tri ▒ - 9.91% 0x4ffffffff ▒ BM_face_as_array_vert_tri ▒ 5.89% BM_face_as_array_vert_tri ▒ - 0.52% 0x7f648c188c68 ▒ 0x7f64469188d0 ▒ - 23.97% 0.00% blender [kernel] [k] 0x00007f64469188d0 ▒ 0x7f64469188d0 ▒ - 19.69% 19.54% blender blender [.] BLI_ghash_lookup ▒ - 10.04% 0x441f0f ▒ BLI_ghashutil_intcmp ▒ BLI_ghash_lookup ▒ - 9.36% 0x1f0fc3f80904e8 ▒ BLI_ghashutil_ptrcmp ▒ BLI_ghash_lookup ▒ - 17.24% 0.00% blender [kernel] [k] 0x00000004ffffffff ▒ - 0x4ffffffff ▒ 9.97% BM_face_as_array_vert_tri ▒ 2.92% BKE_pbvh_bmesh_update_topology ▒ 2.30% closest_on_tri_to_point_v3 ▒ 0.71% BM_edge_calc_length_squared ▒ - 11.59% 0.05% blender blender [.] BLI_ghashutil_intcmp ▒ - 11.54% BLI_ghashutil_intcmp ▒ 10.11% BLI_ghash_lookup ▒ 1.23% BLI_ghash_ensure_p ▒ - 11.52% 0.00% blender blender [.] 0x0000000000441f0f ▒ - 0x441f0f ▒ - 11.52% BLI_ghashutil_intcmp ▒ 10.11% BLI_ghash_lookup ▒ 1.18% BLI_ghash_ensure_p ▒ - 9.84% 0.06% blender blender [.] BLI_ghashutil_ptrcmp ▒ - 9.78% BLI_ghashutil_ptrcmp ▒ 9.43% BLI_ghash_lookup ▒ - 9.84% 0.00% blender [kernel] [k] 0x001f0fc3f80904e8 ▒ 0x1f0fc3f80904e8 ▒ - BLI_ghashutil_ptrcmp ▒ 9.43% BLI_ghash_lookup ▒ - 7.19% 7.15% blender blender [.] closest_on_tri_to_point_v3 ▒ - 2.29% 0x4ffffffff ▒ closest_on_tri_to_point_v3 ▒ 1.37% closest_on_tri_to_point_v3 ▒ - 6.05% 0.25% blender [kernel] [k] 0000000000000000 ▒ - 5.88% 0 ▒ 1.63% sculpt_combine_proxies_task_cb ▒ ``` That's all the time I have to spend on this today. I'm new to tinkering with Blender's code (and have a rocky past with cmake), so if anyone has a recipe I can paste into my terminal to get a build of Blender with debug symbols for Blender and all of its dependencies, that would help a lot.

YAFU commented

2019-03-22 12:45:31 +01:00

James.
Here are the instructions to build Blender:

https://wiki.blender.org/wiki/Building_Blender

To install dependencies in Linux, after downloading the sources you just run in the linux terminal the "install_deps.sh" script in "blender/build_files/build_environment/". "master" branch is 2.8.
To compile with debug symbols you must set CMAKE_BUILD_TYPE=Debug.
Here are the options for debugging:

https://wiki.blender.org/wiki/Tools/Debugging

Running with the blender binary from terminal "./blender --help" you will see options about how to launch blender for debug.

James. Here are the instructions to build Blender: https://wiki.blender.org/wiki/Building_Blender To install dependencies in Linux, after downloading the sources you just run in the linux terminal the "install_deps.sh" script in "blender/build_files/build_environment/". "master" branch is 2.8. To compile with debug symbols you must set CMAKE_BUILD_TYPE=Debug. Here are the options for debugging: https://wiki.blender.org/wiki/Tools/Debugging Running with the blender binary from terminal "./blender --help" you will see options about how to launch blender for debug.

Sergey Sharybin commented

2019-04-04 09:54:08 +02:00

Dynamic topology is known to behave slower on Linux comparing to Windows. This is something what was reported here in the studio as well.

It boils down to Windows sending much less mouse motion events than Linux. We do have a system in place which ignores motion events when they are too close to each other (is configured by the brush's step size), but probably on Windows events are more sparse than the step size.

Dynamic topology is known to behave slower on Linux comparing to Windows. This is something what was reported here in the studio as well. It boils down to Windows sending much less mouse motion events than Linux. We do have a system in place which ignores motion events when they are too close to each other (is configured by the brush's step size), but probably on Windows events are more sparse than the step size.

Kor Boerema commented

2019-04-04 13:32:22 +02:00

Hi Sergey,

That sounds reasonable, the only thing is that it doesn't explain the huge amount of time it takes from going from sculpt mode to object mode and vice versa.

Hi Sergey, That sounds reasonable, the only thing is that it doesn't explain the huge amount of time it takes from going from sculpt mode to object mode and vice versa.

Sergey Sharybin commented

2019-04-04 14:03:32 +02:00

@boemak, toggling sculpt mode creates all the internal structures like BVH. Also, when using dynamic topology this will also create a special BMesh representation of the object.

But this time is expected to be the same on Linux and Windows. Is it very different for you?

@boemak, toggling sculpt mode creates all the internal structures like BVH. Also, when using dynamic topology this will also create a special BMesh representation of the object. But this time is expected to be the same on Linux and Windows. Is it very different for you?

Kor Boerema commented

2019-04-05 17:30:57 +02:00

On Linux it will easily take a minute or more. On Windows it is done in seconds.

See my above posts about the time it takes to get from Sculpt to Object mode in Linux compared to Windows.

On Linux it will easily take a minute or more. On Windows it is done in seconds. See my above posts about the time it takes to get from Sculpt to Object mode in Linux compared to Windows.

James Gregory commented

2019-04-08 21:11:26 +02:00

I had wondered about the event system. Would it work to simply drop events that arrive while the sculpting logic is still running? I guess the main issue with that is that you want the same behavior on all platforms, as far as is possible. On the other hand, it seems like there is already different behavior here.

Alternatively, since I saw (much) better performance on sculpting by running Blender-win64 under Wine, perhaps looking at how Wine routes events might point to a viable path forward?

I had wondered about the event system. Would it work to simply drop events that arrive while the sculpting logic is still running? I guess the main issue with that is that you want the same behavior on all platforms, as far as is possible. On the other hand, it seems like there is already different behavior here. Alternatively, since I saw (much) better performance on sculpting by running Blender-win64 under Wine, perhaps looking at how Wine routes events might point to a viable path forward?

Ray molenkamp commented

2019-04-08 21:15:35 +02:00

I'd rather we repro the issue on linux, and run it through a profiler to see whats going on rather than randomly lifting code from other projects based on a hunch.

Pablo Dobarro commented

2020-01-10 16:15:12 +01:00

Added subscriber: @PabloDobarro

Pablo Dobarro commented

2020-01-10 16:15:12 +01:00

In 2.81 we introduced code that limits the number of input events that are processed in the brush code, so performance problems related to that should be fixed on Linux. We can still improve this by tweaking the limits of how many events per second are allowed to be processed in the different paint modes while keeping them responsive or by implementing a solution like D5676 to decouple the events from the paint functions in all systems, so that is something that we can improve.

In 2.81 we introduced code that limits the number of input events that are processed in the brush code, so performance problems related to that should be fixed on Linux. We can still improve this by tweaking the limits of how many events per second are allowed to be processed in the different paint modes while keeping them responsive or by implementing a solution like [D5676](https://archive.blender.org/developer/D5676) to decouple the events from the paint functions in all systems, so that is something that we can improve.

Philipp Oeser removed the

Download

What's New

Blender Studio

Manual

Developers Blog

Documentation

Benchmark

Blender Conference

Development Fund

One-time Donations

Blender performance under Linux is flakey ( on my system? ) #54807