Job compilation should survive server restarts #62534

New Issue

Sybren A. Stüvel · 2019-03-13T10:16:05+01:00

Sybren A. Stüvel commented

2019-03-13 10:16:05 +01:00

Currently any running job compilation is aborted when the server restarts. Since job compilation also can take more than a second, we should probably move it to a background process with Celery. Celery is a bit better in finishing the task before shutting down, and we can have a background monitoring operation and re-create any jobts that are stuck in 'under-construction' status.

This actually happened to a Cloud subscriber (https://cloud.blender.org/flamenco/p-5c73c4680ab12b4c191cf8d0/jobs/5c8673670e2ba003b9e4b33b).

Currently any running job compilation is aborted when the server restarts. Since job compilation also can take more than a second, we should probably move it to a background process with Celery. Celery is a bit better in finishing the task before shutting down, and we can have a background monitoring operation and re-create any jobts that are stuck in 'under-construction' status. This actually happened to a Cloud subscriber (https://cloud.blender.org/flamenco/p-5c73c4680ab12b4c191cf8d0/jobs/5c8673670e2ba003b9e4b33b).

Sybren A. Stüvel commented

2019-03-13 10:16:05 +01:00

Added subscribers: @dr.sybren, @fsiddi

Michael Soluyanov commented

2019-03-13 10:47:19 +01:00

Added subscriber: @crantisz

Michael Soluyanov commented

2019-03-13 10:47:19 +01:00

I think it's need to have ability to remove jobs in 'under-construction' status, anyway, even if it will be fixed.

Sybren A. Stüvel commented

2019-03-13 11:38:41 +01:00

Nope, deletion of those jobs is impossible for a reason. When the job is in that status, its tasks are being created by Flamenco Server, and nobody else should be touching the job.

Sybren A. Stüvel commented

2022-02-28 13:17:07 +01:00

Changed status from 'Confirmed' to: 'Archived'

Sybren A. Stüvel closed this issue

2022-02-28 13:17:07 +01:00

Sybren A. Stüvel commented

2022-02-28 13:17:07 +01:00

A new generation of Flamenco (version 3.0) is in development (in the main branch at https://developer.blender.org/diffusion/F/), so all issues with Flamenco version 2 or older will be archived.

A new generation of Flamenco (version 3.0) is in development (in the `main` branch at https://developer.blender.org/diffusion/F/), so all issues with Flamenco version 2 or older will be archived.

Sign in to join this conversation.