Last-Rendered Image: refactor queue to be per job #99394

New Issue

Sybren A. Stüvel · 2022-07-04T14:36:32+02:00

Sybren A. Stüvel commented

2022-07-04 14:36:32 +02:00

The "last rendered image" processing queue is currently a single, limited-size queue. If the queue is full, images sent in by Workers are discarded until there is space in the queue again. This means that a spammy job can starve the other jobs from queueing images.

It would be nice if there was not only a limit on the total number of queued images, but also a limit on the number of queued images per job. This per-job limit should be of the order of G / J, where G is the global limit, and J is the expected number of jobs to run in parallel.

The "last rendered image" processing queue is currently a single, limited-size queue. If the queue is full, images sent in by Workers are discarded until there is space in the queue again. This means that a spammy job can starve the other jobs from queueing images. It would be nice if there was not only a limit on the total number of queued images, but also a limit on the number of queued images per job. This per-job limit should be of the order of `G / J`, where `G` is the global limit, and `J` is the expected number of jobs to run in parallel.

Sybren A. Stüvel commented

2022-07-04 14:36:32 +02:00

Changed status from 'Needs Triage' to: 'Confirmed'