How to wait for execution of another electron

Let’s say we need to wait for a task (task_A) to finish execution before we start a different task (task_B) but none of the outputs of task_A are inputs to task_B.

For this purpose, there exists a method called wait_for() in the electron which takes as input the tasks to wait for before executing this one. It can be used as follows:

import covalent as ct

@ct.electron
def task_1(a):
    import time
    time.sleep(3)
    return a ** 2

@ct.electron
def task_2(x, y):
    return x * y

@ct.electron
def task_3(b):
    return b ** 3

@ct.lattice
def workflow():
    res_1 = task_1(2)
    res_2 = task_2(res_1, 3)
    res_3 = task_3(5).wait_for(res_1)

    return task_2(res_2, res_3)

result = ct.dispatch_sync(workflow)()
print(result.result)

1500

Here, task_3 is going to wait for completion of task_1 even though task_3’s execution does not directly depend on task_1’s output.

You can also pass in a list of tasks to wait for:

import covalent as ct
import time


@ct.electron
def task_1a(a):
    time.sleep(1)
    return a ** 2

@ct.electron
def task_1b(a):
    time.sleep(1)
    return a ** 3

@ct.electron
def task_1c(a):
    time.sleep(1)
    return a ** 4

@ct.electron
def task_2(x, y):
    return x * y

@ct.electron
def task_3(b):
    return b ** 3

@ct.lattice
def workflow():
    res_1a = task_1a(2)
    res_1b = task_1b(2)
    res_1c = task_1c(2)
    res_2 = task_2(res_1a, 3)
    res_3 = task_3(5).wait_for([res_1a, res_1b, res_1c])

    return task_2(res_2, res_3)

result = ct.dispatch_sync(workflow)()
print(result.result)

1500

That said, this is one way to explicity create a dependency between tasks. One can always connect the tasks using output-input dependency connection.

wait_for is especially useful when the dependent task does not take any inputs but still needs to wait for some other tasks, which may not return an output, to finish before starting its own execution.