WIP: Add Tasks and Jobs Index Column for Improved Sorting and Progress Tracking #104301

MrJW · 2024-05-06T01:15:31+02:00

MrJW commented

2024-05-06 01:15:31 +02:00

This pull request introduces a new index column to both the "Jobs" and "Tasks" tables to enhance sorting and progress tracking. Previously large amount of jobs and / or tasks would jumble up which was hard to visually see what is going on.

Behavior before:

Behavior after:

Experimented with aligning content to center, however It does not look good:

It can be changed by adding hozAlign: 'center' param to column definition. Unfortunately headerHozAlign does nothing in this case.
I like that order starts from 0, however changing it to 1 is also a feasible solution. Would like to hear feedback.

This pull request introduces a new index column to both the "Jobs" and "Tasks" tables to enhance sorting and progress tracking. Previously large amount of jobs and / or tasks would jumble up which was hard to visually see what is going on. Behavior before: ![image](/attachments/85aeca2b-c8e2-4135-9316-015b4ac015df) Behavior after: ![image](/attachments/35de17a1-9bf0-4738-b422-c4b428a543f5) Experimented with aligning content to center, however It does not look good: ![image](/attachments/3bc07ea3-d9c7-41a4-9589-3844fd4668d5) It can be changed by adding `hozAlign: 'center'` param to column definition. Unfortunately `headerHozAlign` does nothing in this case. I like that order starts from 0, however changing it to 1 is also a feasible solution. Would like to hear feedback.

image.png

49 KiB

image.png

69 KiB

image.png

21 KiB

MrJW added 4 commits 2024-05-06 01:15:35 +02:00

refactor(README): added development section b1e8c19127

feat: added curent task log to Task Details View bb6d2d9701

feat(tasks): added Task Index Column for Improved Sorting and Progress Tracking 1a653a9341

feat(jobs): added Jobs Index Column for Improved Sorting and Progress Tracking 78afec9f2e

Sybren A. Stüvel commented

2024-05-06 11:48:50 +02:00

Thanks, this is an interesting approach.

@Andy_Goralczyk @SimonThommes do you have feedback / ideas / opinions?

Thanks, this is an interesting approach. @Andy_Goralczyk @SimonThommes do you have feedback / ideas / opinions?

Simon Thommes commented

2024-05-06 15:08:31 +02:00

I have run into the issue that tasks are jumping around in order a couple of times, so introducing something like this sounds like a solid idea to me.

I'm a little bit confused what the index means exactly though. Is it just a sequential index that is not manually editable? Then I feel like a Created column would achieve the same result while holding some additional useful information.

I have run into the issue that tasks are jumping around in order a couple of times, so introducing something like this sounds like a solid idea to me. I'm a little bit confused what the index means exactly though. Is it just a sequential index that is not manually editable? Then I feel like a `Created` column would achieve the same result while holding some additional useful information.

Sybren A. Stüvel requested changes 2024-05-06 15:57:10 +02:00

Sybren A. Stüvel left a comment

I have run into the issue that tasks are jumping around in order a couple of times, so introducing something like this sounds like a solid idea to me.

👍

I'm a little bit confused what the index means exactly though. Is it just a sequential index that is not manually editable? Then I feel like a Created column would achieve the same result while holding some additional useful information.

All tasks are created pretty much at once, so unless we can get sub-millisecond granularity in the timestamps I wouldn't mind some explicit index.

The question is what to do with job indices, though. At some point they're going to become prohibitively large. For jobs it's probably, as Simon also said, good enough to be able to sort by creation timestamp.

The task indices should probably be generated by Flamenco Manager, and not by the web interface. There is no guarantee in which order the tasks are reported by the API. It's probably better to create task indices when the job gets compiled.

About the PR itself: please do not include unrelated formatting changes. If your editor auto-formats differently from my editor, we'd have a constant battle between formatters.

> I have run into the issue that tasks are jumping around in order a couple of times, so introducing something like this sounds like a solid idea to me. :+1: > I'm a little bit confused what the index means exactly though. Is it just a sequential index that is not manually editable? Then I feel like a `Created` column would achieve the same result while holding some additional useful information. All tasks are created pretty much at once, so unless we can get sub-millisecond granularity in the timestamps I wouldn't mind some explicit index. The question is what to do with job indices, though. At some point they're going to become prohibitively large. For jobs it's probably, as Simon also said, good enough to be able to sort by creation timestamp. The task indices should probably be generated by Flamenco Manager, and not by the web interface. There is no guarantee in which order the tasks are reported by the API. It's probably better to create task indices when the job gets compiled. ------------ About the PR itself: please do not include unrelated formatting changes. If your editor auto-formats differently from my editor, we'd have a constant battle between formatters.

👍 2

MrJW commented

2024-05-06 23:02:59 +02:00

I have run into the issue that tasks are jumping around in order a couple of times, so introducing something like this sounds like a solid idea to me.

I'm a little bit confused what the index means exactly though. Is it just a sequential index that is not manually editable? Then I feel like a Created column would achieve the same result while holding some additional useful information.

I didn't have better name for that as there is no direct translation from my language. But the idea was to sort jobs by their frame range chronology. So the first jobs would be for frames at the begging of the animation and similarly the end frame jobs - their actual order of operation. I checked the API output, in my cases the order it gave me was the one I was looking for. I tried that with current release and my version with multiple jobs.

> I have run into the issue that tasks are jumping around in order a couple of times, so introducing something like this sounds like a solid idea to me. > > I'm a little bit confused what the index means exactly though. Is it just a sequential index that is not manually editable? Then I feel like a `Created` column would achieve the same result while holding some additional useful information. I didn't have better name for that as there is no direct translation from my language. But the idea was to sort jobs by their frame range chronology. So the first jobs would be for frames at the begging of the animation and similarly the end frame jobs - their actual order of operation. I checked the API output, in my cases the order it gave me was the one I was looking for. I tried that with current release and my version with multiple jobs.

MrJW commented

2024-05-06 23:06:32 +02:00

The task indices should probably be generated by Flamenco Manager, and not by the web interface. There is no guarantee in which order the tasks are reported by the API. It's probably better to create task indices when the job gets compiled.

I agree, I'm not familiar with go backend (yet).

> The task indices should probably be generated by Flamenco Manager, and not by the web interface. There is no guarantee in which order the tasks are reported by the API. It's probably better to create task indices when the job gets compiled. I agree, I'm not familiar with go backend (yet).

MrJW added 1 commit 2024-05-07 00:36:55 +02:00

fix(Taks): new tasks are now at the top of the list c33140a7da

MrJW added 1 commit 2024-05-07 00:43:30 +02:00

refactor(*): revert unintended formatting 07a50061fb

MrJW added 1 commit 2024-05-07 00:45:17 +02:00

refactor(*): revert unintended formatting 85079ca376

Simon Thommes commented

2024-05-07 10:50:17 +02:00

I didn't have better name for that as there is no direct translation from my language. But the idea was to sort jobs by their frame range chronology. So the first jobs would be for frames at the begging of the animation and similarly the end frame jobs - their actual order of operation. I checked the API output, in my cases the order it gave me was the one I was looking for. I tried that with current release and my version with multiple jobs.

I see. For the tasks within a job I think that works and indeed it's probably best to do explicit indexing when the tasks are being created, but for the job level I don't think that works, since the frame range doesn't necessarily have the same meaning. So I'd propose to rely on the creation date there instead.

> I didn't have better name for that as there is no direct translation from my language. But the idea was to sort jobs by their frame range chronology. So the first jobs would be for frames at the begging of the animation and similarly the end frame jobs - their actual order of operation. I checked the API output, in my cases the order it gave me was the one I was looking for. I tried that with current release and my version with multiple jobs. I see. For the tasks within a job I think that works and indeed it's probably best to do explicit indexing when the tasks are being created, but for the job level I don't think that works, since the frame range doesn't necessarily have the same meaning. So I'd propose to rely on the creation date there instead.

MrJW commented

2024-05-12 09:34:23 +02:00

I didn't have better name for that as there is no direct translation from my language. But the idea was to sort jobs by their frame range chronology. So the first jobs would be for frames at the begging of the animation and similarly the end frame jobs - their actual order of operation. I checked the API output, in my cases the order it gave me was the one I was looking for. I tried that with current release and my version with multiple jobs.

I see. For the tasks within a job I think that works and indeed it's probably best to do explicit indexing when the tasks are being created, but for the job level I don't think that works, since the frame range doesn't necessarily have the same meaning. So I'd propose to rely on the creation date there instead.

Ok, I'll see If I can do that on frontend and report back.

> > I didn't have better name for that as there is no direct translation from my language. But the idea was to sort jobs by their frame range chronology. So the first jobs would be for frames at the begging of the animation and similarly the end frame jobs - their actual order of operation. I checked the API output, in my cases the order it gave me was the one I was looking for. I tried that with current release and my version with multiple jobs. > > I see. For the tasks within a job I think that works and indeed it's probably best to do explicit indexing when the tasks are being created, but for the job level I don't think that works, since the frame range doesn't necessarily have the same meaning. So I'd propose to rely on the creation date there instead. Ok, I'll see If I can do that on frontend and report back.

Sybren A. Stüvel commented

2024-05-12 11:41:00 +02:00

I didn't have better name for that as there is no direct translation from my language.

Which language is that, and which word would you use in that language?

But the idea was to sort jobs by their frame range chronology.

That would be tasks, right? Not jobs?

So the first jobs would be for frames at the begging of the animation and similarly the end frame jobs - their actual order of operation.

The order of operation is not linear. When there are multiple Workers, they can execute tasks in any order. The only thing that's fixed is that a task X that depends on tasks Y1, Y2, etc. will be executed after those dependencies are completed.

I checked the API output, in my cases the order it gave me was the one I was looking for. I tried that with current release and my version with multiple jobs.

If the API doesn't guarantee any ordering, I wouldn't rely on what it happens to do right now.

For the tasks within a job I think that works and indeed it's probably best to do explicit indexing when the tasks are being created, but for the job level I don't think that works, since the frame range doesn't necessarily have the same meaning. So I'd propose to rely on the creation date there instead.

Ok, I'll see If I can do that on frontend and report back.

You can't do the task part on the frontend, that has to be done on the backend. The job part should be doable on the frontend though.

> I didn't have better name for that as there is no direct translation from my language. Which language is that, and which word would you use in that language? > But the idea was to sort jobs by their frame range chronology. That would be tasks, right? Not jobs? > So the first jobs would be for frames at the begging of the animation and similarly the end frame jobs - their actual order of operation. The order of operation is not linear. When there are multiple Workers, they can execute tasks in any order. The only thing that's fixed is that a task `X` that depends on tasks `Y1`, `Y2`, etc. will be executed after those dependencies are completed. > I checked the API output, in my cases the order it gave me was the one I was looking for. I tried that with current release and my version with multiple jobs. If the API doesn't guarantee any ordering, I wouldn't rely on what it happens to do right now. >> For the tasks within a job I think that works and indeed it's probably best to do explicit indexing when the tasks are being created, but for the job level I don't think that works, since the frame range doesn't necessarily have the same meaning. So I'd propose to rely on the creation date there instead. > Ok, I'll see If I can do that on frontend and report back. You can't do the task part on the frontend, that has to be done on the backend. The job part should be doable on the frontend though.

MrJW changed title from ~~Add Tasks and Jobs Index Column for Improved Sorting and Progress Tracking~~ to WIP: Add Tasks and Jobs Index Column for Improved Sorting and Progress Tracking

2024-05-15 12:43:17 +02:00

Sybren A. Stüvel commented

2024-07-05 08:36:41 +02:00

@MrJW are you still working on this?

This pull request is marked as a work in progress.

View command line instructions.