How to construct tasks from C code

Often we find workflows consist of different functions written in different languages, each most suitable for the task being executed. This example demonstrates how to call a compiled C function as an electron.

import covalent as ct
from ctypes import POINTER, c_int32
import os

First, let’s write a simple program in C:

c_source = """
#include "test.h"

void test_entry(int x, int *y, int *z)
{
        *y += x;
        *z = 5;
}
"""

c_header = """
void test_entry(int x, int *y, int *z);
"""

with open("test.c", "w") as f:
    f.write(c_source)

with open("test.h", "w") as f:
    f.write(c_header)

Next, compile it into a shared library:

!gcc -shared -fPIC -o libtest.so test.c

Optionally, confirm the entrypoint function is exposed in the library:

!nm -D libtest.so | grep 'T test_entry'

0000000000001129 T test_entry

Now, we are ready to construct a task which interfaces with the compiled function using a Lepton object. Note the last argument which helps the Lepton understand how to convert types.

library_path = os.path.join(os.getcwd(),"libtest.so")
task = ct.Lepton(
    language = "C",
    library_name = library_path,
    function_name = "test_entry",
    argtypes = [
        (c_int32, ct.Lepton.INPUT),
        (POINTER(c_int32), ct.Lepton.INPUT_OUTPUT),
        (POINTER(c_int32), ct.Lepton.OUTPUT)
    ]
)

Finally, use the Lepton in the context of a lattice:

@ct.lattice
def workflow(x: int, y: int) -> int:
    return task(x, y)

result = ct.dispatch_sync(workflow)(1, 2)
print(result.result)

(3, 5)

Note the return values consist of all input-output and output-only variables. Output-only variables can only be scalars, since the length is not otherwise known. When you want to return an array, declare it as an input-output variable and initialize it appropriately before passing it to the lepton.