Blender File Browser not generating previews for large (filezise) images #96023

New Issue

Shady Hamza · 2022-02-25T06:32:04+01:00

Shady Hamza commented

2022-02-25 06:32:04 +01:00

System Information
Operating system: Windows-10-10.0.22000-SP0 64 Bits
Graphics card: NVIDIA GeForce RTX 2060 SUPER/PCIe/SSE2 NVIDIA Corporation 4.5.0 NVIDIA 511.65

Blender Version
Broken: version: 3.2.0 Alpha, branch: master, commit date: 2022-02-24 21:51, hash: 7d7dd66ba7
Worked: never

Short description of error
Blender File Browser in thumbnail view doesn't load thumbnails at all (or takes very long time and freezes blender when loading large png, hdri, alphas for sculpting..etc in thumbnail view ,files 4k and above)

Exact steps for others to reproduce the error
#96023.zip

folder contains two 8k images from HDRI haven (one is 73MB, the other 328MB)
open a File Browser
browse to that folder
show thumbnails
larger file will never show a thumbnail

{F12886546 size=full}

**System Information** Operating system: Windows-10-10.0.22000-SP0 64 Bits Graphics card: NVIDIA GeForce RTX 2060 SUPER/PCIe/SSE2 NVIDIA Corporation 4.5.0 NVIDIA 511.65 **Blender Version** Broken: version: 3.2.0 Alpha, branch: master, commit date: 2022-02-24 21:51, hash: `7d7dd66ba7` Worked: never **Short description of error** Blender File Browser in thumbnail view doesn't load thumbnails at all (or takes very long time and freezes blender when loading large png, hdri, alphas for sculpting..etc in thumbnail view ,files 4k and above) **Exact steps for others to reproduce the error** [#96023.zip](https://archive.blender.org/developer/F12886425/T96023.zip) - folder contains two 8k images from HDRI haven (one is 73MB, the other 328MB) - open a File Browser - browse to that folder - show thumbnails - larger file will never show a thumbnail {[F12886546](https://archive.blender.org/developer/F12886546/image.png) size=full}

Shady Hamza commented

2022-02-25 06:32:04 +01:00

Added subscriber: @Shady.Hamza

Omar Emara commented

2022-02-25 08:25:14 +01:00

Added subscriber: @OmarEmaraDev

Omar Emara commented

2022-02-25 08:25:14 +01:00

Changed status from 'Needs Triage' to: 'Needs User Info'

Omar Emara commented

2022-02-25 08:25:14 +01:00

Did this used to work better in any previous version? You mentioned 3.0+, so does it work better in 2.93?
By loading you mean the generation of thumbnails, not loading them into Blender, right?

Did this used to work better in any previous version? You mentioned 3.0+, so does it work better in 2.93? By loading you mean the generation of thumbnails, not loading them into Blender, right?

Shady Hamza commented

2022-02-25 08:54:37 +01:00

It's used to be the same but i don't use blender 2.93 anymore so I cannot confirm if it's still exists, maybe it's fixed in the LTS bug fixes.
And yup generating thumbnails in the file viewer, i have to open windows browser to pick exactly what image im looking for, loading is fine, sometimes also it does generate low res thumbs unless i refresh the file viewer, but larger images is pain

It's used to be the same but i don't use blender 2.93 anymore so I cannot confirm if it's still exists, maybe it's fixed in the LTS bug fixes. And yup generating thumbnails in the file viewer, i have to open windows browser to pick exactly what image im looking for, loading is fine, sometimes also it does generate low res thumbs unless i refresh the file viewer, but larger images is pain

Philipp Oeser commented

2022-02-25 09:34:53 +01:00

Added subscriber: @lichtwerk

Philipp Oeser commented

2022-02-25 09:34:53 +01:00

In #96023#1313721, @Shady.Hamza wrote:
It's used to be the same but i don't use blender 2.93 anymore so I cannot confirm if it's still exists, maybe it's fixed in the LTS bug fixes.

Please test this, you can get previous builds here https://download.blender.org/release/.
If this is just a request for improved performance, this will likely not be considered a bug.

Blender File Viewer takes very long time and freezes blender when loading large png, hdri, alphas for sculpting..etc in thumbnail view ,files 4k and above, sometimes doesn't load thumbnails at all, this is an old issue

Not loading thumbnails at all would be considered a bug though, but we'd need to have a reproducable scenario here, can you give reproduction steps to when this happens?

> In #96023#1313721, @Shady.Hamza wrote: > It's used to be the same but i don't use blender 2.93 anymore so I cannot confirm if it's still exists, maybe it's fixed in the LTS bug fixes. Please test this, you can get previous builds here https://download.blender.org/release/. If this is just a request for improved performance, this will likely not be considered a bug. > Blender File Viewer takes very long time and freezes blender when loading large png, hdri, alphas for sculpting..etc in thumbnail view ,files 4k and above, sometimes doesn't load thumbnails at all, this is an old issue Not loading thumbnails at all would be considered a bug though, but we'd need to have a reproducable scenario here, can you give reproduction steps to when this happens?

Omar Emara commented

2022-02-25 09:38:30 +01:00

This comment was removed by @OmarEmaraDev

*This comment was removed by @OmarEmaraDev*

Shady Hamza commented

2022-02-25 09:57:42 +01:00

3.2: not loading 1k Thumbs unless I refresh multiple times each times loads part of it, But doesn't load 4k and above at all
2.93: slow Load but loads smaller images, on the other hand larger images 6k and above freezes blender and whole PC :D

3.2: not loading 1k Thumbs unless I refresh multiple times each times loads part of it, But doesn't load 4k and above at all 2.93: slow Load but loads smaller images, on the other hand larger images 6k and above freezes blender and whole PC :D

Shady Hamza commented

2022-02-25 10:02:52 +01:00

I go to texture Properties and open an Alpha Image for Sculpt, from folders containing 100+ 2k, 4k and 16k alphas

Shady Hamza commented

2022-02-25 10:07:27 +01:00

ok one more thing this happens less with jpgs they load much much faster than pngs and exrs, PNGs and EXRs tend to not load or load slowly even if both have same res

Philipp Oeser changed title from ~~Blender File Viewer 3.0+~~ to Blender Browser not generating previews for large (filezise) images

2022-02-25 10:19:18 +01:00

Philipp Oeser changed title from ~~Blender Browser not generating previews for large (filezise) images~~ to Blender File Browser not generating previews for large (filezise) images

2022-02-25 10:42:28 +01:00

Philipp Oeser commented

2022-02-25 10:58:52 +01:00

Changed status from 'Needs User Info' to: 'Confirmed'

Julian Eisel was assigned by Bastien Montagne

2022-02-28 11:54:45 +01:00

Julian Eisel commented

2022-02-28 12:24:51 +01:00

I'm not aware of any code that intentionally prevents reading of such files.

Without having investigated this, my suspicion of what's going on is:

The queue of previews to be loaded (in a background job) is constantly getting cleared and refilled as the File Browser is awaiting currently visible previews.
This makes sense on the face of it, e.g. this is used to only load previews that are actually (close to being) in view. But I have found a number of corner cases where it's causing significant issues. Just not clearing the queue is problematic too.
Now for big images, I fear that the time to load a single image is longer than the time it takes from triggering the loading to cancelling the queue of previews to be loaded. So the preview loading is given no chance to actually get done, it's always cancelled before.
This would be quite nasty and difficult to fix - same for some other issues I've found with the preview loading. However for the Asset Browser UI rewrite (in asset-browser-grid-view), I'm now using the more recently added UI code to (lazy-)load custom previews, which should handle this much better (with some improvements I've worked on). The File Browser can be changed to do the same, seems like an easy thing to do. Plus we can remove all the custom preview loading code for the File Browser.

I'm not aware of any code that intentionally prevents reading of such files. Without having investigated this, my suspicion of what's going on is: - The queue of previews to be loaded (in a background job) is constantly getting cleared and refilled as the File Browser is awaiting currently visible previews. - This makes sense on the face of it, e.g. this is used to only load previews that are actually (close to being) in view. But I have found a number of corner cases where it's causing significant issues. Just not clearing the queue is problematic too. - Now for big images, I fear that the time to load a single image is longer than the time it takes from triggering the loading to cancelling the queue of previews to be loaded. So the preview loading is given no chance to actually get done, it's always cancelled before. This would be quite nasty and difficult to fix - same for some other issues I've found with the preview loading. However for the Asset Browser UI rewrite (in `asset-browser-grid-view`), I'm now using the more recently added UI code to (lazy-)load custom previews, which should handle this much better (with some improvements I've worked on). The File Browser can be changed to do the same, seems like an easy thing to do. Plus we can remove all the custom preview loading code for the File Browser.

Harley Acheson commented

2022-03-01 17:01:39 +01:00

Added subscriber: @Harley

Harley Acheson commented

2022-03-01 17:01:39 +01:00

@JulianEisel - I'm not aware of any code that intentionally prevents reading of such files.

IMB_thumbs.h defines THUMB_SIZE_MAX as 100 Megabytes, which thumb_create_ex uses as the largest file size that will allow image thumbnails.

I'd just remove this entirely, as this just seems to be trying to save users from themselves. Not sure we'd actually get error reports like "I have a folder containing multi-gigabyte images and it takes a long time to show the contents the first time when I'm in thumbnail view".

> @JulianEisel - I'm not aware of any code that intentionally prevents reading of such files. `IMB_thumbs.h` defines `THUMB_SIZE_MAX` as 100 Megabytes, which `thumb_create_ex` uses as the largest file size that will allow image thumbnails. I'd just remove this entirely, as this just seems to be trying to save users from themselves. Not sure we'd actually get error reports like "I have a folder containing multi-gigabyte images and it takes a long time to show the contents the first time when I'm in thumbnail view".

Colin Basnett commented

2022-03-03 00:31:30 +01:00

Added subscriber: @cmbasnett

Colin Basnett commented

2022-03-03 00:31:30 +01:00

Many file formats (EXR included) support mip-maps, so this might be a good solution to think about for actually displaying content for very large files in the future.

Bastien Montagne commented

2022-03-08 18:18:05 +01:00

Added subscriber: @mont29

Bastien Montagne commented

2022-03-08 18:18:05 +01:00

For the record, the THUMB_SIZE_MAX limitation is almost 11 years old (02fbaede8f, fixing #27203 (Crashes with some high-res image thumbnail generation))...

For the record, the `THUMB_SIZE_MAX` limitation is almost 11 years old (02fbaede8f, fixing #27203 (Crashes with some high-res image thumbnail generation))...

Shady Hamza commented

2022-03-31 03:22:52 +02:00

Now when loading 4k Thumbs in browser crashes Blender

Shady Hamza commented

2022-03-31 03:23:31 +02:00

crash in latest 3.2 master

Harley Acheson commented

2022-03-31 04:22:27 +02:00

Your original description includes two images, one 73 MB and the other 328, with a complaint that the smaller shows thumbnail while the larger does not. You’ll see comments later in this thread that this is by current design, that thumbnails are not created for images larger than 100 MB.

So how do your new comments relate to this complaint? Do your included images now crash in 3.2 where they didnt in 3.1? Or are you talking about other specific images, or all images? I assume these are smaller than 100 MB? If not related to your original samples then include some new “crashing” samples here. We’d need to be able to recreate the crash to investigate the cause.

Your original description includes two images, one 73 MB and the other 328, with a complaint that the smaller shows thumbnail while the larger does not. You’ll see comments later in this thread that this is by current design, that thumbnails are not created for images larger than 100 MB. So how do your new comments relate to this complaint? Do your included images now crash in 3.2 where they didnt in 3.1? Or are you talking about other specific images, or all images? I assume these are smaller than 100 MB? If not related to your original samples then include some new “crashing” samples here. We’d need to be able to recreate the crash to investigate the cause.

Shady Hamza commented

2022-03-31 05:02:36 +02:00

So the original description was about not generating thumbs for large images yes, and would some time freezes blender for a while, the problem now is the same thumb issue, plus when opening a folder in browser that contains for example many 4k HDRIs maybe 30+ images in thumbnail view, it doesn't load thumbs(old issue) then it freezes blender then crash, so try opening a folder which contains 40 4k HDRIs in EXR format in thumb view, you might be able to reproduce it

Shady Hamza commented

2022-03-31 05:10:10 +02:00

I don't know but i noticed other softwares like 3dcoat for example when I try to import images it opens the windows browser instead of built in browser, I don't know if that could be possible in blender or not but what if blender builtin browser is used only when importing assets that require additional setup like , and for images and stuff that imports directly we may use the windows browser, I'm no expert

Harley Acheson commented

2022-03-31 05:13:13 +02:00

We don't need more complaint of the old issue, where files that are larger than 100 Megabytes are not thumbnailed.

So all these files that freeze and crash blender are under 100 MB? Do any freeze or crash when alone in a folder or does this only happen with a bunch of them? I think that blender is thumbnailing multiple files in parallel so it might be that you are just running out of RAM. Can you confirm this is happening by looking at Task Manager memory usage while it is happening?

We don't need more complaint of the old issue, where files that are larger than 100 Megabytes are not thumbnailed. So all these files that freeze and crash blender are under 100 MB? Do any freeze or crash when alone in a folder or does this only happen with a bunch of them? I _think_ that blender is thumbnailing multiple files in parallel so it might be that you are just running out of RAM. Can you confirm this is happening by looking at Task Manager memory usage while it is happening?

Shady Hamza commented

2022-03-31 05:19:06 +02:00

Yes under 100mb, and yes with a bunch of them, I have folder that have 2 4ks with other folders didn't crash, will check the ram and get back to you

Shady Hamza commented

2022-03-31 05:26:27 +02:00

My old complaint wasn't about 100mb+ not showing thumbs my complaint was 4k+ images don't load thumbs I have 4ks that are 15mb and 30mb not loading, the 100+mb example was an extreme example to reproduce the issue and I didn't add this example some of the developers did for clarification

Harley Acheson commented

2022-03-31 05:32:21 +02:00

Yes, but your only two example files had one less than 100 MB and one over. This is a specific bug report about that design decision.

For reporting that a folder-full of large images will cause you to run out of RAM, then that is a separate known issue here: https://developer.blender.org/T70584

I'm just making sure that you aren't reporting something we don't already know about.

Yes, but your only two example files had one less than 100 MB and one over. This is a specific bug report about that design decision. For reporting that a folder-full of large images will cause you to run out of RAM, then that is a separate known issue here: https://developer.blender.org/T70584 I'm just making sure that you aren't reporting something we don't already know about.

Shady Hamza commented

2022-03-31 05:56:42 +02:00

Update* latest 3.2 release March 31, doesn't crash, ram ok