Blender can't load (any) Alembic file beyond 1+GB #118839

New Issue

Open

opened 2024-02-28 09:18:46 +01:00 by Pierre_Schiller · 16 comments

Pierre_Schiller commented

2024-02-28 09:18:46 +01:00

System Information
Operating system: Windows-10-10.0.19045-SP0 64 Bits
Graphics card: NVIDIA GeForce RTX 3060/PCIe/SSE2 NVIDIA Corporation 4.6.0 NVIDIA 551.52

Blender Version
Broken: version: 4.1.0 Beta, branch: blender-v4.1-release, commit date: 2024-02-12 20:16, hash: d45175c919f3
Worked: 3.6

Short description of error
Any Alembic file bigger than 900Mb will not load, and freeze Blender 4.0, 4.1.

I generated a liquid simulation in Blender 4.0.
I generated the liquid sim data
Generated the foam and spray data
Then, generated the mesh, in a liquid domain at 192 res.

Exported the result as an Alembic file. Resulting in a 5+ GB file. I'm attaching the file in the link below (Mega)

Exact steps for others to reproduce the error
File > Import > Alembic.
Target to this (5.5 GB file)
Blender will start to load until 14% , then it will freeze itself.

**System Information** Operating system: Windows-10-10.0.19045-SP0 64 Bits Graphics card: NVIDIA GeForce RTX 3060/PCIe/SSE2 NVIDIA Corporation 4.6.0 NVIDIA 551.52 **Blender Version** Broken: version: 4.1.0 Beta, branch: blender-v4.1-release, commit date: 2024-02-12 20:16, hash: `d45175c919f3` Worked: 3.6 **Short description of error** Any Alembic file bigger than 900Mb will not load, and freeze Blender 4.0, 4.1. I generated a liquid simulation in Blender 4.0. I generated the liquid sim data Generated the foam and spray data Then, generated the mesh, in a liquid domain at 192 res. Exported the result as an Alembic file. Resulting in a 5+ GB file. I'm attaching the file in the link below (Mega) **Exact steps for others to reproduce the error** File > Import > Alembic. Target to this ([5.5 GB file](https://mega.nz/file/TchFCISZ#e7UQpOOXMJfuZ_omQ8uXJf8mf5xafIc6dxowVYXzp3Q)) Blender will start to load until 14% , then it will freeze itself.

Pierre_Schiller added the

labels 2024-02-28 09:18:47 +01:00

Iliya Katushenock added the

Pipeline, Assets & IO

labels 2024-02-28 10:17:53 +01:00

Richard Antalik commented

2024-02-28 19:35:01 +01:00

Member

Can you check if your machine has enough memory during loading process?

Can you check if your machine has enough memory during loading process?

Richard Antalik added

Needs Information from User

and removed

labels 2024-02-28 19:35:10 +01:00

Jesse Yurkovich commented

2024-02-28 20:39:51 +01:00

Member

Things will load, it just takes a very long time (close to 3 hours right now). There is a simple change to reduce that to about 30 minutes (55gb memory used) but Blender doesn't scale to scenes like this. The alembic contains >365000 individual objects and things will be unusably slow at this point. I'll look into addressing the slow load time as it's somewhat simple to do but this file will mostly remain unusable after that.

Things will load, it just takes a very long time (close to 3 hours right now). There is a simple change to reduce that to about 30 minutes (55gb memory used) but Blender doesn't scale to scenes like this. The alembic contains >365000 individual objects and things will be unusably slow at this point. I'll look into addressing the slow load time as it's somewhat simple to do but this file will mostly remain unusable after that.

Pierre_Schiller commented

2024-02-28 21:01:44 +01:00

Author

My machine has 56 GB of Ram. 12 Vram using RTX 3060 nvidia card.
Windows 10 OS.
Other than that, the file will not load (I've spent 1h30 mins watching the loader and it freezes on 15%).

My machine has 56 GB of Ram. 12 Vram using RTX 3060 nvidia card. Windows 10 OS. Other than that, the file will not load (I've spent 1h30 mins watching the loader and it freezes on 15%).

Pierre_Schiller closed this issue

2024-02-28 21:01:47 +01:00

Blender Bot added

and removed

Needs Information from User

labels 2024-02-28 21:01:48 +01:00

Iliya Katushenock reopened this issue

2024-02-28 21:02:34 +01:00

Blender Bot added

and removed

labels 2024-02-28 21:02:35 +01:00

Pierre_Schiller commented

2024-02-28 21:08:33 +01:00

Author

Correct: the file will be slow and unusable.
The file contains a liquid sim and spray+foam.
I separated the liquid until I got a 900MB .ABC file (this loads, and works normally when imported)
Then I baked the spray and foam, (tried to lower it as much as possible) and I sized it down from 10 GB to 5.5GB which is the file I am sharing with this.
Caching the foam and spray as a vdb point cloud or particle cache could be useful, but the data is not generated when the liquid particle simulation is removed.
If the spray particle simulation is the only one in the stack (I deleted liquid and foam particle operators), then at the time of exporting the alembic file, it only outputs data in space. There's no way to remap (connect/assign?) this to an object.
I'm attaching the resulting .alembic file screenshot. (2.2 GB loaded in 8 minutes)

Correct: the file will be slow and unusable. The file contains a liquid sim and spray+foam. I separated the liquid until I got a 900MB .ABC file (this loads, and works normally when imported) Then I baked the spray and foam, (tried to lower it as much as possible) and I sized it down from 10 GB to 5.5GB which is the file I am sharing with this. Caching the foam and spray as a vdb point cloud or particle cache could be useful, but the data is not generated when the liquid particle simulation is removed. If the spray particle simulation is the only one in the stack (I deleted liquid and foam particle operators), then at the time of exporting the alembic file, it only outputs data in space. There's no way to remap (connect/assign?) this to an object. I'm attaching the resulting .alembic file screenshot. (2.2 GB loaded in 8 minutes)

particles.png

790 KiB

Pierre_Schiller commented

2024-02-28 21:13:40 +01:00

Author

Also "vector" attributes (speed for motion blur), caused the .alembic file to be frozen when loading.
Adding attributes to a 1+GB alembic file seems to slow down or freeze all together the Blender import.

Also "vector" attributes (speed for motion blur), caused the .alembic file to be frozen when loading. Adding attributes to a 1+GB alembic file seems to slow down or freeze all together the Blender import.

Jesse Yurkovich referenced this issue

2024-02-28 22:23:02 +01:00

Alembic: Alleviate long loading times in large files #118869

Jesse Yurkovich commented

2024-02-28 22:26:34 +01:00

Member

I posted a PR to alleviate some of the problem. The 5.5gb file will load in about 30 minutes for me:
Alembic import of 'T:\SPLASH_foam.abc' took 1690.5 s (0H:28m:10s)

I posted a PR to alleviate some of the problem. The 5.5gb file will load in about 30 minutes for me: `Alembic import of 'T:\SPLASH_foam.abc' took 1690.5 s (0H:28m:10s)`

large-alembic2.png

165 KiB

large-alembic.png

767 KiB

Jesse Yurkovich referenced this issue from a commit

2024-02-29 18:30:34 +01:00

Alembic: Alleviate long loading times in large files

Jesse Yurkovich commented

2024-03-01 09:47:11 +01:00

Member

Please give the most recent 4.2 Alpha build a try and see if you're able to load it too now: https://builder.blender.org/download/daily/

Please give the most recent 4.2 Alpha build a try and see if you're able to load it too now: https://builder.blender.org/download/daily/

Jesse Yurkovich added

Needs Information from User

and removed

labels 2024-03-01 09:47:22 +01:00

Pierre_Schiller commented

2024-03-02 21:43:54 +01:00

Author

I regenerated the (original) .abc fluid simulation file (which is 11 GB).
I downloaded Blender 4.2
I opened and imported the 11 GB .abc sim, Blender 4.2 crashed (again) when loading reached 12%.
I only had Blender Open (because we want to discard that the OS has no Ram conflicts: My system has 52 GB ram)
Perplexity agrees in that unused attributes also add "weight" to the alembic file.
.

Also, for common users, we don't know how to open and load from command line, so I guess that's the only way you did load the previous 5.5 GB file.

Blender 4.0, 4.1, 4.2 confirmed CRASHING when loading 1GB+ .alembic file.

I regenerated the (original) .abc fluid simulation file (which is 11 GB). I downloaded Blender 4.2 I opened and imported the 11 GB .abc sim, Blender 4.2 crashed (again) when loading reached 12%. I only had Blender Open (because we want to discard that the OS has no Ram conflicts: My system has 52 GB ram) Perplexity agrees in that unused attributes also add "weight" to the alembic file. ![image](/attachments/2d287a48-84f0-4749-9b12-e8cfd3b21233). Also, for common users, we don't know how to open and load from command line, so I guess that's the only way you did load the previous 5.5 GB file. Blender 4.0, 4.1, 4.2 confirmed CRASHING when loading 1GB+ .alembic file.

194 KiB

Pierre_Schiller closed this issue

2024-03-02 21:43:58 +01:00

Blender Bot added

and removed

Needs Information from User

labels 2024-03-02 21:44:00 +01:00

Pierre_Schiller commented

2024-03-02 21:45:15 +01:00

Author

I mistakenly clicked on "Comment and Close".
The issue is still open.

I mistakenly clicked on "Comment and Close". The issue is still open.

Pierre_Schiller reopened this issue

2024-03-02 21:45:19 +01:00

Blender Bot added

and removed

labels 2024-03-02 21:45:21 +01:00

Jesse Yurkovich commented

2024-03-02 21:56:01 +01:00

Member

No, I didn't load from the command line. I only started the regular blender UI from there so I could easily show you the time it took to load. All I did was drag and drop the alembic file into the viewport to load it normally.

This will probably just remain open unless you can do more troubleshooting yourself as to what the issue might be as I cannot reproduce your issue here. You could very well be running out of memory since the 5.5 GB file required 54GB to load.

Using 4.2 run the blender_factory_startup.cmd batch file (just double click it) located right next to blender.exe and try loading it again. Upload the log files to this report if it crashes again.

No, I didn't load from the command line. I only started the regular blender UI from there so I could easily show you the time it took to load. All I did was drag and drop the alembic file into the viewport to load it normally. This will probably just remain open unless you can do more troubleshooting yourself as to what the issue might be as I cannot reproduce your issue here. You could very well be running out of memory since the 5.5 GB file required 54GB to load. Using 4.2 run the `blender_factory_startup.cmd` batch file (just double click it) located right next to `blender.exe` and try loading it again. Upload the log files to this report if it crashes again.

Jesse Yurkovich added

Needs Information from User

and removed

labels 2024-03-02 21:56:11 +01:00

Pierre_Schiller commented

2024-03-03 05:26:20 +01:00

Author

yeah, that's the issue: there's no .log file. Blender instant crashes when reaching 15%
Also, you may be getting different results, we need another (common) user to load the 5.5GB alembic file.
I'm loading the 11 GB .abc file on my machine, virgin blender 4.2. Nothing. Crashes automagically at 15%.

yeah, that's the issue: there's no .log file. Blender instant crashes when reaching 15% Also, you may be getting different results, we need another (common) user to load the 5.5GB alembic file. I'm loading the 11 GB .abc file on my machine, virgin blender 4.2. Nothing. Crashes automagically at 15%.

Pierre_Schiller commented

2024-03-03 05:53:35 +01:00

Author

Default cmd blender launch . Crashed at 23% with the 11.5 GB file I re-generated for the alembic fluid sim file, locally.

Default cmd blender launch . Crashed at 23% with the 11.5 GB file I re-generated for the alembic fluid sim file, locally.

blender_system_info.txt

21 KiB

blender_debug_output.txt

726 B

Pierre_Schiller commented

2024-03-03 06:01:57 +01:00

Author

"see console".
Can't. Because it's closed.
What's the next step?
Even with CMD default blender, it crashes.
The only logic explanation is that I'm running Blender from the E: drive and user data is on C:

"see console". Can't. Because it's closed. What's the next step? Even with CMD default blender, it crashes. The only logic explanation is that I'm running Blender from the E: drive and user data is on C:

Jesse Yurkovich added

and removed

Needs Information from User

labels 2024-03-05 02:15:15 +01:00

Pratik Borhade commented

2024-03-05 07:35:00 +01:00

Member

Hi, maybe launch blender from command line and see if anything is captured in console after crash?: https://docs.blender.org/manual/en/latest/advanced/command_line/launch/windows.html

Check eventviewer or reliability monitor for errors, they might have some relevant info.

You could very well be running out of memory since the 5.5 GB file required 54GB to load.

Re "out of memory case": monitor memory usage when file is loading.

Hi, maybe launch blender from command line and see if anything is captured in console after crash?: https://docs.blender.org/manual/en/latest/advanced/command_line/launch/windows.html Check eventviewer or reliability monitor for errors, they might have some relevant info. > You could very well be running out of memory since the 5.5 GB file required 54GB to load. Re "out of memory case": monitor memory usage when file is loading.

Pratik Borhade added

Needs Information from User

and removed

labels 2024-03-05 07:35:39 +01:00

Pierre_Schiller commented

2024-03-20 12:49:08 +01:00

Author

Hello. I did this. Nothing appears from CMD GPU or the other .cmd Blender launch. The text file appears blank.
However. Blender, in the end, is mismanaging the use of the GPU, as soon as it gets "filled", Blender crashes. It prevents from launching itself in Windows system tasks again.
After crashing, no other blender version opens (3.6, 4.0, 4.1, 4.2) in my system.
This is also happening whenever an EEVEE scene renders, and fills up the Vram. Blender auto-quits. Blender restarts and flushes the Vram, and then it can render again as mentioned by many users with simple EEVEE render scenes.
There are a lot of issues mentioning this.

In addition, normal Blender launch and rendering of a 1GB+ .alembic fluid cache, in a "regular user" machine, will break Blender.

Hello. I did this. Nothing appears from CMD GPU or the other .cmd Blender launch. The text file appears blank. However. Blender, in the end, is mismanaging the use of the GPU, as soon as it gets "filled", Blender crashes. It prevents from launching itself in Windows system tasks again. After crashing, no other blender version opens (3.6, 4.0, 4.1, 4.2) in my system. This is also happening whenever an EEVEE scene renders, and fills up the Vram. Blender auto-quits. Blender restarts and flushes the Vram, and then it can render again as mentioned by many users with simple EEVEE render scenes. There are a lot of issues [mentioning this.](https://projects.blender.org/blender/blender/issues/108789) In addition, normal Blender launch and rendering of a 1GB+ .alembic fluid cache, in a "regular user" machine, will break Blender.

blender gpu debug.png

50 KiB

blender gpu debug.png

Pratik Borhade commented

2024-03-21 04:40:39 +01:00

Member

After crashing, no other blender version opens (3.6, 4.0, 4.1, 4.2) in my system.

Mostly OS or other third part software like antivirus may prevent blender from launching.

fills up the Vram. Blender auto-quits

Crash due to "out of memory" are not considered as bug

> After crashing, no other blender version opens (3.6, 4.0, 4.1, 4.2) in my system. Mostly OS or other third part software like antivirus may prevent blender from launching. > fills up the Vram. Blender auto-quits Crash due to "out of memory" are not considered as bug

Sign in to join this conversation.

No Label

Animation & Rigging

Asset Browser Project (Legacy)

Automated Testing

Blender Asset Bundle

Dependency Graph

Development Management

EEVEE & Viewport

Images & Movies

Motion Tracking

Nodes & Physics

Pipeline, Assets & IO

Platforms, Builds & Tests

Render & Cycles

Render Pipeline

Sculpt, Paint & Texture

Video Sequencer

Virtual Reality

Blender 2.8 Project

Milestone 1: Basic, Local Asset Browser

Good First Issue

Animation & Rigging

Development Management

EEVEE & Viewport

Nodes & Physics

Pipeline, Assets & IO

Platforms, Builds & Tests

Render & Cycles

Sculpt, Paint & Texture

Needs Info from Developers

Needs Information from User

No Milestone

No project

No Assignees

4 Participants

Notifications

Due Date

The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: blender/blender#118839

No description provided.