Efficient storage of icons in git #37989

Git is storing a full copy of icons for every change (almost 1mb increase to the repo size for every icon edit).

Specifically:
./release/datafiles/blender_icons16.png
./release/datafiles/blender_icons32.png
./release/datafiles/prvicons.png

From SVG source:
./release/freedesktop/icons/blender_icon_source.svg
./release/datafiles/prvicons.svg

Ideally we could use RSVG and render the SVG as a build step, but unfortunately RSVG doesn't support mask's in a way thats compatible with inkscape.

http://sourceforge.net/mailarchive/message.php?msg_id=31778009

So for now I would propose to store each icon file be moved into its own directory and split into uncompressed bitmap files (as an automated python script), Then on build the icons can be re-combined into a single PNG.

This way we can make a change to one icon without having to store a new copy of the entire file each time.

Workflow for creating icons

Basically the same as it is now...

Edit the SVG
Run release/datafiles/blender_icons.sh

Rather then committing the png, commit ./release/datafiles/blender_icons(16/32)/ directory which contains pixmaps

Workflow for building

Basically the same as it is now...

... internally there would be a program like datatoc that would take files from ./release/datafiles/blender_icons(16/32)/ and create a PNG in the build path.

Changed status to: 'Open'

Added subscribers: @brecht, @Sergey

Added subscriber: @ThomasDinges

We would also need the 32x ones of course. (./release/datafiles/blender_icons32).

Notes so far:

Apart blender_icons.sh you would also need blender_icons.bat for Windows. And also make it so OSX works fine (it'll use .sh guess). Wouldn't want that some internal optimization change makes someone's life more difficult.
We've got other binary binary files which are much bigger even (fonts for example, splash) which are being modified on the same time basis as icons.

Ok, cambo says .blend are not compressed so this is not so big issue, but why using 0 compression level for PNG doesn't work for us?

Tweaking icons are gonna to become PITA. Imagine you've tweaked few icons in SVG, generated split PNG. Now what?
All the files might be different from what they where originally (different inkscape versions, different platforms or so).

How would you stage files which you indeed changed? Unless files does have meaningful name, that would end up in rather annoying calculation of icon coords in a grid. Afraid we'll have people who'll commit all the files instead and all our attempts to optimize anything would mean nothing.

Based on previous note, would really recommend using rather natural names for icons (which is easy to do based UI_icons.h i would think) rather than cryptic names like 001.png and so. Would help seeing what's going on after icons modification.
Did someone checked other projects? From quick glance linux/gimp/calligra doesn't do anything specific against binary files in their repo. As in, we're developers and tools are to make our life easier, not we're to make tools life easier! :)

What is the longterm prognosis, lets say if we keep modifying the source as usual and keep the binary files as is in 1 year, 2 years?.

It shouldn't matter if its 300 or 400MB in total in a year. IMHO. ;)

@ThomasDinges - disagree, if we add 80mb per year because of binary files this is going to need cleaning up later.

I doubt it would be so much, but rather avoid this if files need updating frequently and it can be avoided without too much trouble.

@Sergey, you raise valid points...

The other files we have in our repo are not changing that much (or we can't help it - as with splash),

I've submitted a patch which stored icons in git as small uncompressed bitmaps.

You do raise a good point that some small change in rasterization would update all icons. so I'll add a check for color difference so very minor changes dont cause all pixmaps to regenerate.

Removed subscriber: @ideasman42

Added subscriber: @Blendify

Is there a need for this task anymore?

Changed status from 'Open' to: 'Resolved'

This has been implemented, closing.

Download

What's New

Blender Studio

Manual

Developers Blog

Documentation

Benchmark

Blender Conference

Development Fund

One-time Donations

Efficient storage of icons in git #37989