Buildbot: Migrate package archive format for Linux from tar.bz2 to tar.xz

xz compresses 25% better than bz2, reducing download times and server load.
The numbers:

blender-2.80-linux-glibc217-x86_64.tar.bz2 (release): 134 886 174 bytes
with xz:    96 181 604 bytes (-28.7%)
with xz -9: 93 871 548 bytes (-30.4%)

blender-2.81-7c1fbe24ca33-linux-glibc217-x86_64.tar.bz2 (beta): 173 600 363 bytes
with xz:    133 100 664 bytes (-23.3%)
with xz -9: 129 534 124 bytes (-25.4%)

xz also decompresses more than twice as fast as bz2, however compression needs
four times as long (on my 7-year-old laptop 3-4 minutes instead of <1).

Also xz has become more common than bz2, e.g. Debian/Ubuntu deb packages have
been xz-compressed for years, so the dpkg package manager as well as systemd
and grub all depend on liblzma being present, whereas bz2 is becoming more and
more optional.

Current Linux archives also include the UID/GID of whatever user account
happens to be used for building by the blender.org infrastructure. If someone
then installs these archives as root e.g. to /usr/local/... and doesn't pay
full attention the files remain owned by a regular user, which is a serious
security issue. This patch fixes that by setting the UID/GID to 0.

Differential Revision: https://developer.blender.org/D6138
This commit is contained in:
Jens
2019-11-28 11:40:59 +01:00
committed by Sergey Sharybin
parent 320d8ab155
commit d4835b88b2
3 changed files with 22 additions and 10 deletions

View File

@@ -51,8 +51,9 @@ try:
if extension == 'zip':
archive_cmd = ['zip', '-9', '-r', package_archive, package_dir]
elif extension == 'tar.bz2':
archive_cmd = ['tar', 'cjf', package_archive, package_dir]
elif extension == 'tar.xz':
archive_cmd = ['tar', '-cf', package_archive, '--owner=0', '--group=0',
'--use-compress-program=xz -9', package_dir]
else:
sys.stderr.write('Unknown archive extension: ' + extension)
sys.exit(-1)