Buildbot: Use shared compilation cache #86

New Issue

Sergey Sharybin · 2024-06-11T10:40:16+02:00

Sergey Sharybin commented

2024-06-11 10:40:16 +02:00

Currently buildbot relies on a build system like Ninja and incremental builds to avoid compilation when unnecessary. It has the following downsides:

Even for vdev branch the coherency can be improved by sharing more between different builders. Especially if it is possible to share the result of GPU compile step.
For the patch builders coherency is quite bad, as there is a single type of worker (vexp) which builds patches. This makes it so sequential build of a patch after addressing feedback could take same long as initial/clean build.

One of possible solutions is to use sccache. Need to check how well it handles cases when different workers require using different toolchains.

Having #85 implemented first would allow measuring the impact of this change more scientifically.

Currently buildbot relies on a build system like Ninja and incremental builds to avoid compilation when unnecessary. It has the following downsides: - Even for vdev branch the coherency can be improved by sharing more between different builders. Especially if it is possible to share the result of GPU compile step. - For the patch builders coherency is quite bad, as there is a single type of worker (vexp) which builds patches. This makes it so sequential build of a patch after addressing feedback could take same long as initial/clean build. One of possible solutions is to use [sccache](https://github.com/mozilla/sccache). Need to check how well it handles cases when different workers require using different toolchains. Having #85 implemented first would allow measuring the impact of this change more scientifically.

🚀 3

Sergey Sharybin added the

Service

Buildbot

label 2024-06-11 10:40:16 +02:00

Sergey Sharybin referenced this issue

2024-06-11 10:41:02 +02:00

Buildbot: automated build for every pull request #44

Bart van der Braak added this to the DevOps Progress Board project 2024-07-16 12:52:45 +02:00

Bart van der Braak added the

Type

Feature