infrastructure/blender-buildbot-www
1
0
Fork 2
Code Issues 4 Pull Requests 1 Projects Releases Wiki Activity

Cache: Add cache-control headers with revalidation to ensure checksum consistency #9

Merged
Bart van der Braak merged 2 commits from cache-revalidation into main 2024-07-25 16:01:33 +02:00
Conversation 2 Commits 2 Files Changed 3 +11 -3
Bart van der Braak commented 2024-07-24 13:28:10 +02:00
Owner
Copy Link

This should fix the issue described in infrastructure/blender-projects-platform#89:

The checksums are stored in a separate file (<build_name>.sha256). They are served via CDN, similar to the actual Blender builds.

The issue comes from the fact that sometimes builds do change without change in the file name. For example, LTS tracks keeps being build every night, or a vdev build can be re-triggered with the new libraries.

This leads to situation when CDN cache is in inconsistent state due to build and checksum being cached separately.
Perhaps there is a way to provide proper cache-control from our side, so that CDN can more immediately react to the changes to the content.

This should fix the issue described in infrastructure/blender-projects-platform#89: > > The checksums are stored in a separate file (<build_name>.sha256). They are served via CDN, similar to the actual Blender builds. > > The issue comes from the fact that sometimes builds do change without change in the file name. For example, LTS tracks keeps being build every night, or a vdev build can be re-triggered with the new libraries. > > This leads to situation when CDN cache is in inconsistent state due to build and checksum being cached separately. > Perhaps there is a way to provide proper cache-control from our side, so that CDN can more immediately react to the changes to the content.
Bart van der Braak added 4 commits 2024-07-24 13:28:11 +02:00
Bart van der Braak changed title from Cache: Add `cache-control` headers with revalidation to ensure consistency between builds and their shasums to WIP: Cache: Add `cache-control` headers with revalidation to ensure consistency between builds and their shasums 2024-07-24 13:35:29 +02:00
Sergey Sharybin commented 2024-07-24 14:20:39 +02:00
Owner
Copy Link

master is PROD, develop is UATEST. So the PR needs to target develop branch.

We should probably just switch the meaning, and have main for development/UATEST and production for things deployed to PROD. With a bit of coordination at the buildbot-master side it should be doable quite easily.

P.S. The buildbot config does is managed manually. That is something to keep in mind when we'll be deploying this.

`master` is PROD, `develop` is UATEST. So the PR needs to target `develop` branch. We should probably just switch the meaning, and have `main` for development/UATEST and `production` for things deployed to PROD. With a bit of coordination at the buildbot-master side it should be doable quite easily. P.S. The buildbot config does is managed manually. That is something to keep in mind when we'll be deploying this.
👍 1
Bart van der Braak changed target branch from master to develop 2024-07-24 14:21:09 +02:00
Bart van der Braak changed title from WIP: Cache: Add `cache-control` headers with revalidation to ensure consistency between builds and their shasums to Cache: Add `cache-control` headers with revalidation to ensure consistency between builds and their shasums 2024-07-25 09:47:51 +02:00
Bart van der Braak force-pushed cache-revalidation from 528f4a8795 to 02896cb071 2024-07-25 10:34:31 +02:00 Compare
Bart van der Braak changed target branch from develop to main 2024-07-25 10:43:30 +02:00
Bart van der Braak changed title from Cache: Add `cache-control` headers with revalidation to ensure consistency between builds and their shasums to Cache: Add `cache-control` headers with revalidation to ensure checksum consistency 2024-07-25 10:49:51 +02:00
Bart van der Braak commented 2024-07-25 14:37:45 +02:00
Author
Owner
Copy Link

@Sergey

I changed the branching as follows (and changed the target to main on this PR).

  • The default branch is now main (previously develop), which is deployed to builder-uatest.blender.org:
blender@pvep-lvm-buildbot-master-01:~/git/blender-buildbot-www$ git checkout main
Branch 'main' set up to track remote branch 'main' from 'origin'.
Switched to a new branch 'main'
  • The production branch (previously master) is now deployed to builder.blender.org:
blender@pvep-lvm-buildbot-master-02:~/git/blender-buildbot-www$ git checkout production 
Branch 'production' set up to track remote branch 'production' from 'origin'.
Switched to a new branch 'production'
@Sergey I changed the branching as follows (and changed the target to `main` on this PR). - The default branch is now `main` (previously `develop`), which is deployed to `builder-uatest.blender.org`: ``` blender@pvep-lvm-buildbot-master-01:~/git/blender-buildbot-www$ git checkout main Branch 'main' set up to track remote branch 'main' from 'origin'. Switched to a new branch 'main' ``` - The `production` branch (previously `master`) is now deployed to `builder.blender.org`: ``` blender@pvep-lvm-buildbot-master-02:~/git/blender-buildbot-www$ git checkout production Branch 'production' set up to track remote branch 'production' from 'origin'. Switched to a new branch 'production' ```
Sergey Sharybin approved these changes 2024-07-25 15:36:43 +02:00
Sergey Sharybin left a comment
Owner
Copy Link

@bartvdbraak Lovely!

Deployment to the UATEST and PROD would need some manual work with the 000-default.conf, but surely you'll figure it out :)

As for the change, it can be landed.

P.S. Maybe keep an eye after deployment, to see if we need to increase the interval if CPU or network usage goes too much up.

@bartvdbraak Lovely! Deployment to the UATEST and PROD would need some manual work with the `000-default.conf`, but surely you'll figure it out :) As for the change, it can be landed. P.S. Maybe keep an eye after deployment, to see if we need to increase the interval if CPU or network usage goes too much up.
👍 1
Bart van der Braak merged commit 1198e95b6c into main 2024-07-25 16:01:33 +02:00
Sign in to join this conversation.
Reviewers
No reviewers
Sergey Sharybin
Labels
Clear labels
No items
No Label
Milestone
Clear milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
No Assignees
2 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: infrastructure/blender-buildbot-www#9
No description provided.
Delete Branch "cache-revalidation"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?