TaskScheduler Improvements #75134

New Issue

Jeroen Bakker · 2020-03-27T16:22:01+01:00

Jeroen Bakker commented

2020-03-27 16:22:01 +01:00

The current task scheduler was implemented for running heavy tasks. Currently it is also used to run small and tiny tasks (sculpting, draw manager and other areas).
There are some improvements that could be done to optimize the task scheduler for small/tiny tasks.

Subtasks can depend on parent tasks. The sub tasks will only be executed when the parent task has finished. This would improve the CPU utilization in some areas
** DrawManager can schedule all the task in a single go.
Should we introduce concepts like WorkGroups and EvaluationGraphs

In stead of implementing the low level bits we could reuse existing libraries like TTB, Which was already been experimenting with.

#70709 (Switch Blender task scheduler to use TBB), https://developer.blender.org/diffusion/B/history/temp-tbb-task-scheduler/
TBB supports flow graphs (https://www.threadingbuildingblocks.org/tutorial-intel-tbb-flow-graph).

The current task scheduler was implemented for running heavy tasks. Currently it is also used to run small and tiny tasks (sculpting, draw manager and other areas). There are some improvements that could be done to optimize the task scheduler for small/tiny tasks. 1. Subtasks can depend on parent tasks. The sub tasks will only be executed when the parent task has finished. This would improve the CPU utilization in some areas ** DrawManager can schedule all the task in a single go. 2. Should we introduce concepts like WorkGroups and EvaluationGraphs In stead of implementing the low level bits we could reuse existing libraries like TTB, Which was already been experimenting with. * #70709 (Switch Blender task scheduler to use TBB), https://developer.blender.org/diffusion/B/history/temp-tbb-task-scheduler/ * TBB supports flow graphs (https://www.threadingbuildingblocks.org/tutorial-intel-tbb-flow-graph).

Jeroen Bakker commented

2020-03-27 16:22:01 +01:00

Changed status from 'Needs Triage' to: 'Confirmed'

Jeroen Bakker commented

2020-03-27 16:22:01 +01:00

Added subscriber: @Jeroen-Bakker

Jeroen Bakker changed title from ~~[WIP] TaskScheduler Improvements~~ to TaskScheduler Improvements

2020-03-30 09:30:31 +02:00

Paul Szajner commented

2020-04-03 03:45:17 +02:00

Added subscriber: @sozap

Sergey Sharybin commented

2020-04-14 10:46:22 +02:00

Added subscriber: @Sergey

Sergey Sharybin commented

2020-04-14 10:46:22 +02:00

Subtasks can depend on parent tasks. The sub tasks will only be executed when the parent task has finished. This would improve the CPU utilization in some areas.

This is already possible. Could be streamlined a bit, but all the required fundamentals are there.

Should we introduce concepts like WorkGroups and EvaluationGraphs

Not in the current scheduler.

Priority here should be to get TBB to replace our own naive implementation. In my tests from a while back it solved a lot of threading overhead, simplified API (no need to have such a fine-grained functions).

And as you've mentioned, TBB supports flow graphs, which can also be used in #75208.

> Subtasks can depend on parent tasks. The sub tasks will only be executed when the parent task has finished. This would improve the CPU utilization in some areas. This is already possible. Could be streamlined a bit, but all the required fundamentals are there. > Should we introduce concepts like WorkGroups and EvaluationGraphs Not in the current scheduler. Priority here should be to get TBB to replace our own naive implementation. In my tests from a while back it solved a lot of threading overhead, simplified API (no need to have such a fine-grained functions). And as you've mentioned, TBB supports flow graphs, which can also be used in #75208.

Garek commented

2021-12-21 07:21:55 +01:00

Added subscriber: @Garek