Slow performance importing USD scene with many objects sharing the same basename #110483

New Issue

Bin-Zhao · 2023-07-26T04:45:47+02:00

Bin-Zhao commented

2023-07-26 04:45:47 +02:00

System Information
Operating system: Windows 10
Graphics card: GTX 1070

Blender Version
Broken: 3.6
Worked: None

Short description of error

It takes a long time to load some particular USD scenes.

Exact steps for others to reproduce the error

Download this file and try to import limits_32.usdc. It takes longer than 30 seconds for USD import job to finish. limits_48.usdc takes even longer , 4min 33sec. For comparison, both Omniverse 2023.1 and Tellusim can import limits_48.usdc in less than 20 and 5 seconds, respectively, as reported here

I have profiled the process and identified one big bottleneck is id_sort_by_name function. For each object created, id_sort_by_name needs to search a long linked list backward to find a place for the newly created object. The search is supposed to be fast as the new object typically has larger extension number. However, the search is based on alphabet comparison so a smaller extension number , e.g. 9999 could be bigger alphabetically than 13223. I believe this had incurred a O(N^2) penalty in id_sort_by_name.

**System Information** Operating system: Windows 10 Graphics card: GTX 1070 **Blender Version** Broken: 3.6 Worked: None **Short description of error** It takes a long time to load some particular USD scenes. **Exact steps for others to reproduce the error** Download [this file](https://tellusim.com/download/import/limits.zip) and try to import ```limits_32.usdc```. It takes longer than 30 seconds for USD import job to finish. ```limits_48.usdc``` takes even longer , 4min 33sec. For comparison, both **Omniverse 2023.1** and **Tellusim** can import ```limits_48.usdc``` in less than 20 and 5 seconds, respectively, as reported [here](https://tellusim.com/import/) I have profiled the process and identified one big bottleneck is ```id_sort_by_name``` function. For each object created, ```id_sort_by_name``` needs to search a long linked list backward to find a place for the newly created object. The search is supposed to be fast as the new object typically has larger extension number. However, the search is based on alphabet comparison so a smaller extension number , e.g. 9999 could be bigger alphabetically than 13223. I believe this had incurred a O(N^2) penalty in ```id_sort_by_name```.

Bin-Zhao added the

labels 2023-07-26 04:45:48 +02:00

Richard Antalik commented

2023-08-03 02:56:51 +02:00

@mont29 Do you think, this is something that can be resolved easily?

Bastien Montagne commented

2023-08-03 11:11:24 +02:00

This is a very well known issue yes. Historically Blender was not designed to handle tens of thousands of data-blocks...

The generic task covering these cases is #73359, not sure if @aras_p also had more specific tasks/patches too?

Thanks for the report, but don't think it's worth keeping it open, this is really a known issue involving design and development, not a bug.

This is a very well known issue yes. Historically Blender was not designed to handle tens of thousands of data-blocks... The generic task covering these cases is #73359, not sure if @aras_p also had more specific tasks/patches too? Thanks for the report, but don't think it's worth keeping it open, this is really a known issue involving design and development, not a bug.

Bastien Montagne closed this issue

2023-08-03 11:11:26 +02:00

Blender Bot added

and removed

labels 2023-08-03 11:11:29 +02:00

Sign in to join this conversation.