archive
/
blender-buildbot
Archived
1
0
Fork 0
Code Issues 3 Pull Requests Projects Releases Wiki Activity

Buildbot creates nightly builds of same hash code but different content #88496

New Issue
Closed
opened 2021-05-23 06:12:54 +02:00 by Meino Cramer · 22 comments
Meino Cramer commented 2021-05-23 06:12:54 +02:00
Copy Link

Operating system: GENTOO Linux, updated daily
Graphics card: NVIDIA Corporation TU106 [GeForce RTX 2060 SUPER] (rev a1)
Graphic driver: x11-drivers/nvidia-drivers-465.31

Blender Version
Broken: adding info would create more confusion than without it :) (see below)
Worked: adding info would create more confusion than without it :) (see below)

Short description
Buildbot creates nightly builds with same hash code but different content

Exact steps for others to reproduce the error
Unpack blender 3.0 / 2.93 nightly build ( available 22.5.2021 morning ) and
blender 3.0 / 2.93 nightly build ( available 23.5.2021 morning ). Compare
directory contents of all four directories accordingly. Despite of the same
hash code the contents differs (Linux builds).
If the difference doesn't matter - why to include and rebuild in first place?
If the difference matters - why using the same hash code?

Operating system: GENTOO Linux, updated daily Graphics card: NVIDIA Corporation TU106 [GeForce RTX 2060 SUPER] (rev a1) Graphic driver: x11-drivers/nvidia-drivers-465.31 **Blender Version** Broken: adding info would create more confusion than without it :) (see below) Worked: adding info would create more confusion than without it :) (see below) **Short description** Buildbot creates nightly builds with same hash code but different content **Exact steps for others to reproduce the error** Unpack blender 3.0 / 2.93 nightly build ( available 22.5.2021 morning ) and blender 3.0 / 2.93 nightly build ( available 23.5.2021 morning ). Compare directory contents of all four directories accordingly. Despite of the same hash code the contents differs (Linux builds). If the difference doesn't matter - why to include and rebuild in first place? If the difference matters - why using the same hash code?
Meino Cramer commented 2021-05-23 06:12:54 +02:00
Author
Copy Link

Added subscriber: @mcq

Added subscriber: @mcq
Anton Raves commented 2021-05-23 09:27:46 +02:00
Copy Link

Added subscriber: @Memento

Added subscriber: @Memento
Anton Raves commented 2021-05-23 09:33:29 +02:00
Copy Link

The downloaded archive filesize is different too, not just on the Linux version, the Mac version too.

The downloaded archive filesize is different too, not just on the Linux version, the Mac version too.
Alaska commented 2021-05-23 10:19:30 +02:00
Copy Link

Added subscriber: @Alaska

Added subscriber: @Alaska
Alaska commented 2021-05-23 10:19:30 +02:00
Copy Link

The build bot was recently shifted to a new system and is currently being tweaked a little bit. The change in file contents and file size may be down to one of these changes with the build bot (The new build bot when initially deployed did exclude some CUDA related files, so it's possible others were either missing or things that weren't supposed to be added were added).

I also believe the buildbot does a build of Blender everyday even if there isn't a change. But don't quote me on that.

The build bot was recently shifted to a new system and is currently being tweaked a little bit. The change in file contents and file size may be down to one of these changes with the build bot (The new build bot when initially deployed did exclude some CUDA related files, so it's possible others were either missing or things that weren't supposed to be added were added). I also believe the buildbot does a build of Blender everyday even if there isn't a change. But don't quote me on that.
Meino Cramer commented 2021-05-23 14:07:41 +02:00
Author
Copy Link

The building of experimantal builds every night independantly from changes is known to me - that's since the genesis :)
Identical checksum reflects indentical content then - no problem at all.
But under "normal circumstances" (that is without bots being swapping around) I would say it is a little annoying being urged to compare
archives contents by hand when checksums are identical (I mean...indication of differnces is the reason behind the existence of checksums, isn't?)
But if currently 'everything is shifting"...ok, then the checksums are more of an decorative form of mathemitcal art... :) for a shorter period of time, I think.
So no problem for now.
.....and: What are "normal circumstances" in the field of software development anyhow ? :) :)
Cheers!
mcc

The building of experimantal builds every night independantly from changes is known to me - that's since the genesis :) Identical checksum reflects indentical content then - no problem at all. But under "normal circumstances" (that is without bots being swapping around) I would say it is a little annoying being urged to compare archives contents by hand when checksums are identical (I mean...indication of differnces is the reason behind the existence of checksums, isn't?) But if currently 'everything is shifting"...ok, then the checksums are more of an decorative form of mathemitcal art... :) for a shorter period of time, I think. So no problem for now. .....and: What are "normal circumstances" in the field of software development anyhow ? :) :) Cheers! mcc
Evan Wilson commented 2021-05-24 16:46:00 +02:00
Copy Link

Added subscriber: @EAW

Added subscriber: @EAW
Evan Wilson commented 2021-05-24 16:46:00 +02:00
Copy Link

This comment was removed by @EAW

*This comment was removed by @EAW*
Ermins Ermanis commented 2021-05-24 17:05:05 +02:00
Copy Link

Added subscriber: @erminos

Added subscriber: @erminos
Ermins Ermanis commented 2021-05-24 17:05:05 +02:00
Copy Link

What do you mean by "hash code" and by "checksum"? The same source code version (represented by Git commit hash) when recompiled produces slightly different binaries, which should have the same functionality however... You can use SHA256 checksums (by adding ".sha256" to daily build URLs) before downloading the full archive - to compare with a previous (or incomplete) download :)

What do you mean by "hash code" and by "checksum"? The same source code version (represented by Git commit *hash*) when recompiled produces slightly different binaries, which should have the same functionality however... You can use SHA256 *checksums* (by adding ".sha256" to daily build URLs) before downloading the full archive - to compare with a previous (or incomplete) download :)
Evan Wilson commented 2021-05-24 18:30:20 +02:00
Copy Link

To add onto @erminos comment:

According to https://builder.blender.org/download/daily/archive/

May 22 2021: https://builder.blender.org/download/daily/archive/blender-3.0.0-alpha+master.7adbe62a09ca-linux.x86_64-release.tar.xz.sha256
May 23 2021: https://builder.blender.org/download/daily/archive/blender-3.0.0-alpha+master.f7b22fc3d271-linux.x86_64-release.tar.xz.sha256
To add onto @erminos comment: According to https://builder.blender.org/download/daily/archive/ | May 22 2021: | https://builder.blender.org/download/daily/archive/blender-3.0.0-alpha+master.7adbe62a09ca-linux.x86_64-release.tar.xz.sha256 | | -- | -- | | May 23 2021: | https://builder.blender.org/download/daily/archive/blender-3.0.0-alpha+master.f7b22fc3d271-linux.x86_64-release.tar.xz.sha256 |
Anton Raves commented 2021-05-24 18:37:14 +02:00
Copy Link

@EAW That is not what we are seeing at https:*builder.blender.org/download/daily/ , there the hash code has been d5c3bff6e774 for three days in a row now, with changing archive file sizes for 2.93 beta.

@EAW That is not what we are seeing at [https:*builder.blender.org/download/daily/ ](https:*builder.blender.org/download/daily/), there the hash code has been d5c3bff6e774 for three days in a row now, with changing archive file sizes for 2.93 beta.
Ermins Ermanis commented 2021-05-24 19:21:26 +02:00
Copy Link

Also, as pointed out by LazyDodo - the Pyton addons are under Git submodules of Blender main Git repo, whose commit hash isn't affected by changes in addons.

Also, as pointed out by LazyDodo - the Pyton addons are under Git submodules of Blender main Git repo, whose commit hash isn't affected by changes in addons.
James Monteath commented 2021-05-25 09:25:19 +02:00
Copy Link

Added subscriber: @ponderz

Added subscriber: @ponderz
James Monteath commented 2021-05-25 09:25:19 +02:00
Copy Link

The update-code step does not follow normal git processing.
I will add details on how this works in the wiki.

In general, we always sync to HEAD all submodules of the branch we are building.
All release* branches will sync the same named git branch in each submodule.
All other other branches (experimental) will sync to HEAD the submodule's master branch.

However, there was an update to the pipeline code to add missing kernel files into the package, this was fixed on May 20th.

The `update-code` step does **not** follow normal git processing. I will add details on how this works in the wiki. In general, we always sync to HEAD all submodules of the branch we are building. All `release*` branches will sync the same named git branch in each submodule. All other other branches `(experimental)` will sync to HEAD the submodule's `master` branch. However, there was an update to the pipeline code to add missing kernel files into the package, this was fixed on May 20th.
James Monteath commented 2021-05-25 09:37:30 +02:00
Copy Link

To investigate further you can look at the output of the update-code step.
https://builder.blender.org/admin/#/builders/20/builds/163/steps/1/logs/stdio

The step will display SVN dependency info and git submodule info.

SVN info

svn info /home/blender/git/blender-v293/lib/linux_centos7_x86_64
Path: /home/blender/git/blender-v293/lib/linux_centos7_x86_64
Working Copy Root Path: /home/blender/git/blender-v293/lib/linux_centos7_x86_64
URL: https://svn.blender.org/svnroot/bf-blender/tags/blender-2.93-release/lib/linux_centos7_x86_64
Repository Root: https://svn.blender.org/svnroot/bf-blender
Repository UUID: 954f8c5b-7b00-dc11-b283-0030488c597c
Revision: 62627
Node Kind: directory
Schedule: normal
Last Changed Author: sybren
Last Changed Rev: 62583
Last Changed Date: 2021-02-26 12:31:33 +0100 (Fri, 26 Feb 2021)
svn info /home/blender/git/blender-v293/lib/tests
Path: /home/blender/git/blender-v293/lib/tests
Working Copy Root Path: /home/blender/git/blender-v293/lib/tests
URL: https://svn.blender.org/svnroot/bf-blender/tags/blender-2.93-release/lib/tests
Repository Root: https://svn.blender.org/svnroot/bf-blender
Repository UUID: 954f8c5b-7b00-dc11-b283-0030488c597c
Revision: 62627
Node Kind: directory
Schedule: normal
Last Changed Author: jbakker
Last Changed Rev: 62626
Last Changed Date: 2021-05-10 12:17:44 +0200 (Mon, 10 May 2021)

Git Info

pipenv run python /home/blender/git/bdr-devops-core/cmd/build/worker/make_update.py --no-blender --use-tests --no-libraries --use-centos-libraries --branch blender-v2.93-release
Updating Submodules
git pull --rebase origin blender-v2.93-release
From https://git.blender.org/blender-addons
 * branch              blender-v2.93-release -> FETCH_HEAD
Already up to date.
git pull --rebase origin blender-v2.93-release
From https://git.blender.org/blender-addons-contrib
 * branch            blender-v2.93-release -> FETCH_HEAD
Already up to date.
git pull --rebase origin blender-v2.93-release
From https://git.blender.org/blender-translations
 * branch            blender-v2.93-release -> FETCH_HEAD
Already up to date.
git pull --rebase origin blender-v2.93-release
From https://git.blender.org/blender-dev-tools
 * branch            blender-v2.93-release -> FETCH_HEAD
Already up to date.
Git submodules list
git submodule foreach git rev-parse --abbrev-ref HEAD
release/datafiles/locale=blender-v2.93-release
release/scripts/addons=blender-v2.93-release
release/scripts/addons_contrib=blender-v2.93-release
source/tools=blender-v2.93-release

To make this easier to investigate in the future, we will summarize all the update details at the end of the log.

To investigate further you can look at the output of the `update-code` step. https://builder.blender.org/admin/#/builders/20/builds/163/steps/1/logs/stdio The step will display SVN dependency info and git submodule info. SVN info ``` svn info /home/blender/git/blender-v293/lib/linux_centos7_x86_64 Path: /home/blender/git/blender-v293/lib/linux_centos7_x86_64 Working Copy Root Path: /home/blender/git/blender-v293/lib/linux_centos7_x86_64 URL: https://svn.blender.org/svnroot/bf-blender/tags/blender-2.93-release/lib/linux_centos7_x86_64 Repository Root: https://svn.blender.org/svnroot/bf-blender Repository UUID: 954f8c5b-7b00-dc11-b283-0030488c597c Revision: 62627 Node Kind: directory Schedule: normal Last Changed Author: sybren Last Changed Rev: 62583 Last Changed Date: 2021-02-26 12:31:33 +0100 (Fri, 26 Feb 2021) svn info /home/blender/git/blender-v293/lib/tests Path: /home/blender/git/blender-v293/lib/tests Working Copy Root Path: /home/blender/git/blender-v293/lib/tests URL: https://svn.blender.org/svnroot/bf-blender/tags/blender-2.93-release/lib/tests Repository Root: https://svn.blender.org/svnroot/bf-blender Repository UUID: 954f8c5b-7b00-dc11-b283-0030488c597c Revision: 62627 Node Kind: directory Schedule: normal Last Changed Author: jbakker Last Changed Rev: 62626 Last Changed Date: 2021-05-10 12:17:44 +0200 (Mon, 10 May 2021) ``` Git Info ``` pipenv run python /home/blender/git/bdr-devops-core/cmd/build/worker/make_update.py --no-blender --use-tests --no-libraries --use-centos-libraries --branch blender-v2.93-release Updating Submodules git pull --rebase origin blender-v2.93-release From https://git.blender.org/blender-addons * branch blender-v2.93-release -> FETCH_HEAD Already up to date. git pull --rebase origin blender-v2.93-release From https://git.blender.org/blender-addons-contrib * branch blender-v2.93-release -> FETCH_HEAD Already up to date. git pull --rebase origin blender-v2.93-release From https://git.blender.org/blender-translations * branch blender-v2.93-release -> FETCH_HEAD Already up to date. git pull --rebase origin blender-v2.93-release From https://git.blender.org/blender-dev-tools * branch blender-v2.93-release -> FETCH_HEAD Already up to date. Git submodules list git submodule foreach git rev-parse --abbrev-ref HEAD release/datafiles/locale=blender-v2.93-release release/scripts/addons=blender-v2.93-release release/scripts/addons_contrib=blender-v2.93-release source/tools=blender-v2.93-release ``` To make this easier to investigate in the future, we will summarize all the update details at the end of the log.
Ermins Ermanis commented 2021-05-25 10:02:22 +02:00
Copy Link

In #88496#1165139, @Memento wrote:
... with changing archive file sizes for 2.93 beta.

Btw, file size on that page seems rounded to 0.01 MB, so it may miss smaller changes.

> In #88496#1165139, @Memento wrote: > ... with changing archive file sizes for 2.93 beta. Btw, file size on that page seems rounded to 0.01 MB, so it may miss smaller changes.
James Monteath commented 2021-05-25 11:42:01 +02:00
Copy Link

However, we can make this work with a configuration file the branch that overrides the submodule's preferred branch.
If a new branch was created from release branch, it then contains the same config and the pipeline will work as expected.

However, we can make this work with a configuration file the branch that overrides the submodule's preferred branch. If a new branch was created from release branch, it then contains the same config and the pipeline will work as expected.
Philipp Oeser commented 2022-02-11 14:19:33 +01:00
Copy Link

Added subscribers: @justa, @lichtwerk

Added subscribers: @justa, @lichtwerk
Philipp Oeser commented 2022-02-11 14:19:33 +01:00
Copy Link

Changed status from 'Needs Triage' to: 'Needs Developer To Reproduce'

Changed status from 'Needs Triage' to: 'Needs Developer To Reproduce'
Philipp Oeser commented 2022-02-11 14:19:33 +01:00
Copy Link

@justa, @mcq : is this still a thing?

@justa, @mcq : is this still a thing?
Brecht Van Lommel commented 2023-02-16 19:13:53 +01:00
Copy Link

This is still a thing but not planned to be resolved.

This is still a thing but not planned to be resolved.
This repo is archived. You cannot comment on issues.
No Branch/Tag Specified
Branches Tags
master
tmp-release-builds
Labels
Clear labels   
legacy module
Development Management

  
legacy module
Platforms, Builds, Tests & Devices

  
legacy module
Python API

  
legacy module
Rendering & Cycles

  
legacy project
3.3

  
legacy project
Development Management

  
legacy project
Documentation

  
legacy project
Infrastructure: Blender Buildbot

  
legacy project
Infrastructure: Websites

  
legacy project
Platform: macOS

  
legacy project
Platforms, Builds, Tests & Devices

  
legacy project
Platform: Windows

  
legacy project
Python API

  
legacy project
Render & Cycles

  
Priority::Low

   
Priority::Normal

   
Priority::Unbreak Now!

   
Status::Archived

   
Status::Confirmed

   
Status::Duplicate

   
Status::Needs Information from Developers

   
Status::Needs Triage

   
Status::Resolved

   
Type::Bug

   
Type::Report

   
Type::To Do
No Label
legacy module
Development Management
legacy module
Platforms, Builds, Tests & Devices
legacy module
Python API
legacy module
Rendering & Cycles
legacy project
3.3
legacy project
Development Management
legacy project
Documentation
legacy project
Infrastructure: Blender Buildbot
legacy project
Infrastructure: Websites
legacy project
Platform: macOS
legacy project
Platforms, Builds, Tests & Devices
legacy project
Platform: Windows
legacy project
Python API
legacy project
Render & Cycles
Priority::Low
Priority::Normal
Priority::Unbreak Now!
Status::Archived
Status::Confirmed
Status::Duplicate
Status::Needs Information from Developers
Status::Needs Triage
Status::Resolved
Type::Bug
Type::Report
Type::To Do
Milestone
Clear milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
No Assignees
8 Participants
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: archive/blender-buildbot#88496
No description provided.
Delete Branch "%!s(<nil>)"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?