Merging/joining into single object can massively increase file size #96926

Closed
opened 2022-03-31 16:45:28 +02:00 by Steve Warner · 26 comments

NOTE: Other triagers: please dont close this after a week, providing a repro case might take a bit.

System Information
Operating system: Windows-10-10.0.19044-SP0 64 Bits
Graphics card: NVIDIA GeForce GTX 1080/PCIe/SSE2 NVIDIA Corporation 4.5.0 NVIDIA 497.29

Blender Version
Broken: version: 3.1.0, branch: master, commit date: 2022-03-08 18:16, hash: c77597cd0e
Worked: Not sure.

Short description of error
Single object massively increases file size vs separating the mesh into multiple objects

Exact steps for others to reproduce the error
Start Blender and delete all objects in the scene.
Create a new collection called "Suzanne."
Add a Suzanne to this collection.
Select Suzanne and go to Edit mode
Select All
Go to Mesh > Split > Faces and Edges by Vertices
You now have a single object that consists of separate faces.
Go to Object Mode
Save the file as "Suzanne Whole"
Go back to Edit mode
Select All
Separate by Loose Parts
Go back to Object Mode
Save the file as "Suzanne Parts"

You will now see that the Suzanne Whole file is substantially larger than the Suzanne Parts file.

Contrary to what would be expected, having hundreds of objects drastically reduces the file size. However having the same amount of geometry in a single object causes the file size to balloon up, not by a small amount, but by a massive amount.

When this is done with a more complicated model (in the tens or hundreds of thousands of polygons), it can result in a difference of a file going from 200MB to 5GB.

Suzanne - Whole.blend

Suzanne - Parts.blend

NOTE: Other triagers: please dont close this after a week, providing a repro case might take a bit. **System Information** Operating system: Windows-10-10.0.19044-SP0 64 Bits Graphics card: NVIDIA GeForce GTX 1080/PCIe/SSE2 NVIDIA Corporation 4.5.0 NVIDIA 497.29 **Blender Version** Broken: version: 3.1.0, branch: master, commit date: 2022-03-08 18:16, hash: `c77597cd0e` Worked: Not sure. **Short description of error** Single object massively increases file size vs separating the mesh into multiple objects **Exact steps for others to reproduce the error** Start Blender and delete all objects in the scene. Create a new collection called "Suzanne." Add a Suzanne to this collection. Select Suzanne and go to Edit mode Select All Go to Mesh > Split > Faces and Edges by Vertices You now have a single object that consists of separate faces. Go to Object Mode Save the file as "Suzanne Whole" Go back to Edit mode Select All Separate by Loose Parts Go back to Object Mode Save the file as "Suzanne Parts" You will now see that the Suzanne Whole file is substantially larger than the Suzanne Parts file. Contrary to what would be expected, having hundreds of objects drastically *reduces* the file size. However having the same amount of geometry in a single object causes the file size to balloon up, not by a small amount, but by a massive amount. When this is done with a more complicated model (in the tens or hundreds of thousands of polygons), it can result in a difference of a file going from 200MB to 5GB. [Suzanne - Whole.blend](https://archive.blender.org/developer/F12959594/Suzanne_-_Whole.blend) [Suzanne - Parts.blend](https://archive.blender.org/developer/F12959595/Suzanne_-_Parts.blend)
Author

Added subscriber: @stevewarner

Added subscriber: @stevewarner

#71823 was marked as duplicate of this issue

#71823 was marked as duplicate of this issue
Steve Warner changed title from Many mesh objects balloons file size vs many meshes in a single object to Single object massively increases file size vs separating the mesh into multiple objects 2022-03-31 16:47:11 +02:00
Member

Added subscriber: @PratikPB2123

Added subscriber: @PratikPB2123
Member

Changed status from 'Needs Triage' to: 'Needs Developer To Reproduce'

Changed status from 'Needs Triage' to: 'Needs Developer To Reproduce'
Member

Thanks for the report.
I don't think this is a bug. New object gets created for the every loose face you've.
Also, shared vertices gets duplicated for every face next to it so I expect the file size to increase.
I'll defer this report to developers just in case I'm missing something.

Thanks for the report. I don't think this is a bug. New object gets created for the every loose face you've. Also, shared vertices gets duplicated for every face next to it so I expect the file size to increase. I'll defer this report to developers just in case I'm missing something.
Member

Added subscriber: @lichtwerk

Added subscriber: @lichtwerk
Member

Changed status from 'Needs Developer To Reproduce' to: 'Needs User Info'

Changed status from 'Needs Developer To Reproduce' to: 'Needs User Info'
Member

Not sure I understand correctly:
Suzanne - Parts is a lot bigger in filesize than Suzanne - Whole (3MB vs 328kB).
This is to be expected, but it seems you are observing the exact opposite?

Not sure I understand correctly: `Suzanne - Parts` is a lot bigger in filesize than `Suzanne - Whole` (3MB vs 328kB). This is to be expected, but it seems you are observing the exact opposite?
Member

@lichtwerk , I think I did not read last few lines of the report carefully :p
What I read is Mesh converted to loose part is more in file size than the mesh as single object
Which is expected I believe... (because each face is now a separate object)

@lichtwerk , I think I did not read last few lines of the report carefully :p What I read is Mesh converted to loose part is more in file size than the mesh as single object Which is expected I believe... (because each face is now a separate object)
Author

Sorry for the confusion. I think using Suzanne is muddying the waters. Let me give the real-world example that alerted me to this as I think that might help.

image.png

This is a proposed theme park land that we're working on. It has over 21,000 objects and over 2 million polygons.

The file size on disk for this model is 368 MB

image.png

As you can see, the environment has all kinds of detailed models in it.

image.png

image.png

I don't need most of these for my current task, so the models circled here I'm going to delete.

image.png

Now I have just the show building as seen here:

image.png

The scene is notably lighter as it now only has 333,000 polys. But for some strange reason, my number of objects has increased.

image.png

I put the roof of the building in a separate collection because I'll need to show the building with the roof on, and then with the roof off with the rides in it.

image.png

The file size on disk is now smaller as expected. It's down to 251 MB. Great.

image.png

I'm going to be sending this to a co-worker, so I want to simplify the scene. I'm going to take all those objects that make up the rest of the building and simply join them into a single object. That way, there will only be two objects in the scene. The building, and the roof. Easy. Select all the objects in the Building collection and press CTRL+J. Then wait.

image.png

Great! It worked. I now have two objects in my scene. The poly count hasn't changed. Still at 333,000 polys. And my outliner is nice and clean. Just two objects. However Blender sometimes stores junk data. Let's purge the Orphan data.

image.png

Yep. Looks like there's a bunch of meshes left over. Purge away!

image.png

Great! Now my scene is nice and clean. Time to save and send to my co-worker.

image.png

Wait! WTF? My file size on disk has gone to a whopping 4.89 GB.

The entire model with all the stuff in it was only 368 MB. But by deleting files and simply joining thousands of objects into one single object, my file size has gone to nearly 5GB!

I've inspected the model. Deleted unnecessary materials. Cleared out any cached images. How does a scene go from 300MB to 4GB when I've deleted things?

So now to try this on a simple scene, because I can't send you this model.

Create a Suzanne. Unweld all the verts and separate the faces into their own meshes. This will represent the city. Lots of tiny individual objects. Save the file and name it Parts. There are 500 objects (each a single face) and 1,968 verts. It's 3.12 MB on disk. Now take those 500 objects and join them into a single object. Don't merge verts. Now you have a single object, but you still have the same number of faces and the same number of verts. So the only thing that's changed is the number of objects. from 500 down to 1. Save the file. Name it whole. Now you get a file that's 321K on disk.

Well, hell. That's now the OPPOSITE of the problem that I'm having, where I'm getting a small file size with LOTS of objects but a HUGE file size with only 2 objects.

So I file the bug report, because either way you slice it, it doesn't make a lot of sense. You have a scene with the same amount of polys. The same amount of verts. The only difference is the number of container objects for the mesh parts. And the file size is wildly different.

I hope that makes sense. Something seems really off here.

Sorry for the confusion. I think using Suzanne is muddying the waters. Let me give the real-world example that alerted me to this as I think that might help. ![image.png](https://archive.blender.org/developer/F12961873/image.png) This is a proposed theme park land that we're working on. It has over 21,000 objects and over 2 million polygons. The file size on disk for this model is 368 MB ![image.png](https://archive.blender.org/developer/F12961881/image.png) As you can see, the environment has all kinds of detailed models in it. ![image.png](https://archive.blender.org/developer/F12961884/image.png) ![image.png](https://archive.blender.org/developer/F12961886/image.png) I don't need most of these for my current task, so the models circled here I'm going to delete. ![image.png](https://archive.blender.org/developer/F12961889/image.png) Now I have just the show building as seen here: ![image.png](https://archive.blender.org/developer/F12961894/image.png) The scene is notably lighter as it now only has 333,000 polys. But for some strange reason, my number of objects has increased. ![image.png](https://archive.blender.org/developer/F12961898/image.png) I put the roof of the building in a separate collection because I'll need to show the building with the roof on, and then with the roof off with the rides in it. ![image.png](https://archive.blender.org/developer/F12961901/image.png) The file size on disk is now smaller as expected. It's down to 251 MB. Great. ![image.png](https://archive.blender.org/developer/F12961903/image.png) I'm going to be sending this to a co-worker, so I want to simplify the scene. I'm going to take all those objects that make up the rest of the building and simply join them into a single object. That way, there will only be two objects in the scene. The building, and the roof. Easy. Select all the objects in the Building collection and press CTRL+J. Then wait. ![image.png](https://archive.blender.org/developer/F12961918/image.png) Great! It worked. I now have two objects in my scene. The poly count hasn't changed. Still at 333,000 polys. And my outliner is nice and clean. Just two objects. However Blender sometimes stores junk data. Let's purge the Orphan data. ![image.png](https://archive.blender.org/developer/F12961920/image.png) Yep. Looks like there's a bunch of meshes left over. Purge away! ![image.png](https://archive.blender.org/developer/F12961922/image.png) Great! Now my scene is nice and clean. Time to save and send to my co-worker. ![image.png](https://archive.blender.org/developer/F12961945/image.png) Wait! WTF? My file size on disk has gone to a whopping 4.89 GB. The entire model with all the stuff in it was only 368 MB. But by deleting files and simply joining thousands of objects into one single object, my file size has gone to nearly 5GB! I've inspected the model. Deleted unnecessary materials. Cleared out any cached images. How does a scene go from 300MB to 4GB when I've deleted things? So now to try this on a simple scene, because I can't send you this model. Create a Suzanne. Unweld all the verts and separate the faces into their own meshes. This will represent the city. Lots of tiny individual objects. Save the file and name it Parts. There are 500 objects (each a single face) and 1,968 verts. It's 3.12 MB on disk. Now take those 500 objects and join them into a single object. Don't merge verts. Now you have a single object, but you still have the same number of faces and the same number of verts. So the only thing that's changed is the number of objects. from 500 down to 1. Save the file. Name it whole. Now you get a file that's 321K on disk. Well, hell. That's now the OPPOSITE of the problem that I'm having, where I'm getting a small file size with LOTS of objects but a HUGE file size with only 2 objects. So I file the bug report, because either way you slice it, it doesn't make a lot of sense. You have a scene with the same amount of polys. The same amount of verts. The only difference is the number of container objects for the mesh parts. And the file size is wildly different. I hope that makes sense. Something seems really off here.
Member

Changed status from 'Needs User Info' to: 'Needs Triage'

Changed status from 'Needs User Info' to: 'Needs Triage'
Member

Hi, thanks for the information.
Appears to be same as said here: #71823
Disable Global Undo and see if that makes any difference when joining objects

Hi, thanks for the information. Appears to be same as said here: #71823 Disable Global Undo and see if that makes any difference when joining objects
Member

Changed status from 'Needs Triage' to: 'Needs User Info'

Changed status from 'Needs Triage' to: 'Needs User Info'
Member

Changed status from 'Needs User Info' to: 'Needs Developer To Reproduce'

Changed status from 'Needs User Info' to: 'Needs Developer To Reproduce'
Member

Added subscribers: @mont29, @ideasman42

Added subscribers: @mont29, @ideasman42
Member

Hm, I dont understand why this or #71823 (Merging objects that use meshes with users count of 1 increases the file size massively) are not bugs tbh.

@ideasman42 , @mont29 : could you comment?

Hm, I dont understand why this or #71823 (Merging objects that use meshes with users count of 1 increases the file size massively) are not bugs tbh. @ideasman42 , @mont29 : could you comment?
Member

So I file the bug report, because either way you slice it, it doesn't make a lot of sense

Well, having many "container" objects increasing the file size does make a bit of sense I think

In any case, there seems to be something wrong when {key Ctrl J} joining in certain cases, will try to come up with a repro case, @stevewarner : it would still help a lot if you could share one, too

> So I file the bug report, because either way you slice it, it doesn't make a lot of sense Well, having many "container" objects increasing the file size does make a bit of sense I think In any case, there seems to be something wrong when {key Ctrl J} joining in certain cases, will try to come up with a repro case, @stevewarner : it would still help a lot if you could share one, too
Member

Added subscribers: @roxlu, @AnthonyEdlin

Added subscribers: @roxlu, @AnthonyEdlin
Philipp Oeser changed title from Single object massively increases file size vs separating the mesh into multiple objects to Merging/joining into single object can massively increase file size 2022-04-26 11:55:18 +02:00
Member

Changed status from 'Needs Developer To Reproduce' to: 'Needs User Info'

Changed status from 'Needs Developer To Reproduce' to: 'Needs User Info'
Member

Until we have found such a repro case, will set status to "Needs Information from User" though.

Other triagers: please dont close this after a week, providing a repro case might take a bit.

Until we have found such a repro case, will set status to "Needs Information from User" though. Other triagers: please dont close this after a week, providing a repro case might take a bit.
Member

Just wondering if this could possibly be related to #98249?

Just wondering if this could possibly be related to #98249?

Changed status from 'Needs User Info' to: 'Archived'

Changed status from 'Needs User Info' to: 'Archived'

We cannot keep an incomplete report open forever, will close it for now. Can easily be re-opened if/when we get a reliable reproducible case.

We cannot keep an incomplete report open forever, will close it for now. Can easily be re-opened if/when we get a reliable reproducible case.

#71823 (comment)

Same issue, screen record supplied.

https://projects.blender.org/blender/blender/issues/71823#issuecomment-1006812 Same issue, screen record supplied.
Member

So this was reported again in #111564

And the reason seems to be that joining does not limit the number of UV layers to 8 (as done elsewhere in blender).
So you might end up with hundreds or thousands of UV layers (and each of them occupying memory for a -- now very large -- mesh).

I think this is probably what was the issue in #71823 as well.

I think the situation should still be improved:

  • either limiting the amount of UV layers upon joining to 8 (as we elsewhere in the UI)
  • merging the UV layers based upon index (so merge all first uv layers of all sources into one on the destination, merge all second uv layers of all sources into a second uv layer on destination, ...) -- this could be an option on the operator
  • if the above doesnt work out for some reason, at least spit out a warning in the UI that more than 8 UV layers have been created

I will open another issue for this shortly...

So this was reported again in #111564 And the reason seems to be that joining does not limit the number of UV layers to 8 (as done elsewhere in blender). So you might end up with hundreds or thousands of UV layers (and each of them occupying memory for a -- now very large -- mesh). I think this is probably what was the issue in #71823 as well. I think the situation should still be improved: - either limiting the amount of UV layers upon joining to 8 (as we elsewhere in the UI) - merging the UV layers based upon index (so merge all first uv layers of all sources into one on the destination, merge all second uv layers of all sources into a second uv layer on destination, ...) -- this could be an option on the operator - if the above doesnt work out for some reason, at least spit out a warning in the UI that more than 8 UV layers have been created I will open another issue for this shortly...
Member

I opened #111608 for this.

I opened #111608 for this.
Sign in to join this conversation.
No Label
Interest
Alembic
Interest
Animation & Rigging
Interest
Asset System
Interest
Audio
Interest
Automated Testing
Interest
Blender Asset Bundle
Interest
BlendFile
Interest
Code Documentation
Interest
Collada
Interest
Compatibility
Interest
Compositing
Interest
Core
Interest
Cycles
Interest
Dependency Graph
Interest
Development Management
Interest
EEVEE
Interest
FBX
Interest
Freestyle
Interest
Geometry Nodes
Interest
glTF
Interest
Grease Pencil
Interest
ID Management
Interest
Images & Movies
Interest
Import Export
Interest
Line Art
Interest
Masking
Interest
Metal
Interest
Modeling
Interest
Modifiers
Interest
Motion Tracking
Interest
Nodes & Physics
Interest
OpenGL
Interest
Overlay
Interest
Overrides
Interest
Performance
Interest
Physics
Interest
Pipeline & IO
Interest
Platforms, Builds & Tests
Interest
Python API
Interest
Render & Cycles
Interest
Render Pipeline
Interest
Sculpt, Paint & Texture
Interest
Text Editor
Interest
Translations
Interest
Triaging
Interest
Undo
Interest
USD
Interest
User Interface
Interest
UV Editing
Interest
VFX & Video
Interest
Video Sequencer
Interest
Viewport & EEVEE
Interest
Virtual Reality
Interest
Vulkan
Interest
Wayland
Interest
Workbench
Interest: X11
Legacy
Asset Browser Project
Legacy
Blender 2.8 Project
Legacy
Milestone 1: Basic, Local Asset Browser
Legacy
OpenGL Error
Meta
Good First Issue
Meta
Papercut
Meta
Retrospective
Meta
Security
Module
Animation & Rigging
Module
Asset System
Module
Core
Module
Development Management
Module
Grease Pencil
Module
Modeling
Module
Nodes & Physics
Module
Pipeline & IO
Module
Platforms, Builds & Tests
Module
Python API
Module
Render & Cycles
Module
Sculpt, Paint & Texture
Module
Triaging
Module
User Interface
Module
VFX & Video
Module
Viewport & EEVEE
Platform
FreeBSD
Platform
Linux
Platform
macOS
Platform
Windows
Severity
High
Severity
Low
Severity
Normal
Severity
Unbreak Now!
Status
Archived
Status
Confirmed
Status
Duplicate
Status
Needs Info from Developers
Status
Needs Information from User
Status
Needs Triage
Status
Resolved
Type
Bug
Type
Design
Type
Known Issue
Type
Patch
Type
Report
Type
To Do
No Milestone
No project
No Assignees
6 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: blender/blender#96926
No description provided.