Check for impossible to execute tasks #99392

New Issue

Sybren A. Stüvel · 2022-07-04T14:27:56+02:00

Sybren A. Stüvel commented

2022-07-04 14:27:56 +02:00

At Manager startup, it should check for impossible to execute tasks.

For example, consider the scenario where all but one worker were blocklisted for a certain task type, and the last worker that could run it, failed it. Now if that failure was that Worker's first one, it would not get blocklisted and still counts as "can execute this task type on this job". However, since it failed the task, it won't be allowed to retry it, and thus the task will get stuck in soft-failed status.

This could be implemented on the persistence layer, by breaking up the task scheduler function into several parts, which then can be called by both the task scheduler and the "unfinishable jobs" checker.

After this is implemented, it could be interesting to run this as a background goroutine every once in a while.

At Manager startup, it should check for impossible to execute tasks. For example, consider the scenario where all but one worker were blocklisted for a certain task type, and the last worker that could run it, failed it. Now if that failure was that Worker's first one, it would *not* get blocklisted and still counts as "can execute this task type on this job". However, since it failed the task, it won't be allowed to retry it, and thus the task will get stuck in `soft-failed` status. This could be implemented on the persistence layer, by breaking up the task scheduler function into several parts, which then can be called by both the task scheduler and the "unfinishable jobs" checker. --------------- After this is implemented, it could be interesting to run this as a background goroutine every once in a while.

Sybren A. Stüvel commented

2022-07-04 14:27:56 +02:00

Changed status from 'Needs Triage' to: 'Confirmed'