Taskflow
2.7.0
|
Parallel workloads often require making control-flow decisions across dependent tasks. Taskflow supports an very efficient interface of conditional tasking for users to implement general control flow such as dynamic flows, cycles, and conditionals that are otherwise difficult to do with existing frameworks.
A condition task evalutes a set of instructions and returns an integer index of the next immediate successor to execute. The index is defined with respect to the order of its successor construction.
The example above implements a simple yet commonly used feedback loop through a condition task (line 7-10) that returns a random binary value. If the return value from cond
is 0, it loops back to itself, or otherwise to stop
. Our conditional tasking interface is very neat and expressive. A complex flow control often just takes a few lines of code to implement. The code below creates another taskflow with three condition tasks.
Debrief:
You can use condition tasks to create cycles as long as the graph does not introduce task race during execution. However, cycles are not allowed in non-condition tasks.
In order to understand how an executor schedules condition tasks, we define two dependency types, strong dependency and weak dependency. A strong dependency is a preceding link from a non-condition task to another task. A weak dependency is a preceding link from a condition task to another task. The number of dependents of a task is the sum of strong dependency and weak dependency. The table below lists the strong dependency and weak dependency numbers of each task in the previous example.
task | strong dependency | weak dependency | dependents |
---|---|---|---|
A | 0 | 0 | 0 |
B | 1 | 1 | 2 |
C | 1 | 0 | 1 |
D | 1 | 0 | 1 |
E | 0 | 1 | 1 |
F | 1 | 0 | 1 |
G | 0 | 1 | 1 |
H | 0 | 1 | 1 |
I | 1 | 0 | 1 |
K | 1 | 0 | 1 |
L | 0 | 1 | 1 |
M | 1 | 0 | 1 |
cond_1 | 1 | 0 | 1 |
cond_2 | 1 | 0 | 1 |
cond_3 | 1 | 1 | 2 |
You can query the number of strong dependents, the number of weak dependents, and the number of dependents of a task.
When you submit a task to an executor, the scheduler starts with tasks of zero dependents (both zero strong and weak dependencies) and continues to execute successive tasks whenever their strong dependencies are met. However, the scheduler skips this rule when executing a condition task and jumps directly to its successors indexed by the return value.
Each task has an atomic join counter to keep track of strong dependents that are met at runtime. When a task completes, the join counter is restored to the task's strong dependency number in the graph, such that the subsequent execution can reuse the counter again.
Condition tasks are handy in creasing dynamic and cyclic control flows, but they are also easy to make mistakes. It is your responsibility to ensure a taskflow is properly conditioned. Top things to avoid include no source tasks to start with and task race. The figure below shows common pitfalls and their remedies.
In the error1 scenario, there is no source task for the scheduler to start with, and the simplest fix is to add a task S that has no dependents. In the error2 scenario, D might be scheduled twice by E through the strong dependency and C through the weak dependency (on return 1). To fix this problem, you can add an auxiliary task D-aux to break the mixed use of strong dependency and weak dependency. In the risky scenario, task X may not be raced if P and M is exclusively branching to X.
A good practice for avoiding mistakes of conditional tasking is to infer the execution flow of your graphs based on our scheduling rules. Make sure there is no task race.