Synchronous Base Executor Class#
- class covalent.executor.base.BaseExecutor(*args, **kwargs)[source]#
Base executor class to be used for defining any executor plugin. Subclassing this class will allow you to define your own executor plugin which can be used in covalent.
- log_stdout#
The path to the file to be used for redirecting stdout.
- log_stderr#
The path to the file to be used for redirecting stderr.
- cache_dir#
The location used for cached files in the executor.
- time_limit#
time limit for the task
- retries#
Number of times to retry execution upon failure
Methods:
cancel
(task_metadata, job_handle)Method to cancel the job identified uniquely by the job_handle (base class)
execute
(function, args, kwargs, dispatch_id, …)Execute the function with the given arguments.
from_dict
(object_dict)Rehydrate a dictionary representation
Check if the task was requested to be cancelled by the user
get_dispatch_context
(dispatch_info)Start a context manager that will be used to access the dispatch info for the executor.
run
(function, args, kwargs, task_metadata)Abstract method to run a function in the executor.
set_job_handle
(handle)Save the job_id/handle returned by the backend executing the task
setup
(task_metadata)Placeholder to run any executor specific tasks
teardown
(task_metadata)Placeholder to run any executor specific cleanup/teardown actions
to_dict
()Return a JSON-serializable dictionary representation of self
write_streams_to_file
(stream_strings, …)Write the contents of stdout and stderr to respective files.
- cancel(task_metadata, job_handle)[source]#
Method to cancel the job identified uniquely by the job_handle (base class)
- Arg(s)
task_metadata: Metadata of the task to be cancelled job_handle: Unique ID of the job assigned by the backend
- Return(s)
False by default
- Return type
Literal
[False]
- execute(function, args, kwargs, dispatch_id, results_dir, node_id=- 1)[source]#
Execute the function with the given arguments.
This calls the executor-specific run() method.
- Parameters
function (
Callable
) – The input python function which will be executed and whose result is ultimately returned by this function.args (
List
) – List of positional arguments to be used by the function.kwargs (
Dict
) – Dictionary of keyword arguments to be used by the function.dispatch_id (
str
) – The unique identifier of the external lattice process which is calling this function.results_dir (
str
) – The location of the results directory.node_id (
int
) – ID of the node in the transport graph which is using this executor.
- Returns
The result of the function execution.
- Return type
output
- from_dict(object_dict)#
Rehydrate a dictionary representation
- Parameters
object_dict (
dict
) – a dictionary representation returned by to_dict- Return type
- Returns
self
Instance attributes will be overwritten.
- get_cancel_requested()[source]#
Check if the task was requested to be cancelled by the user
- Arg(s)
None
- Return(s)
True/False whether task cancellation was requested
- Return type
bool
- get_dispatch_context(dispatch_info)#
Start a context manager that will be used to access the dispatch info for the executor.
- Parameters
dispatch_info (
DispatchInfo
) – The dispatch info to be used inside current context.- Return type
AbstractContextManager
[DispatchInfo
]- Returns
A context manager object that handles the dispatch info.
- abstract run(function, args, kwargs, task_metadata)[source]#
Abstract method to run a function in the executor.
- Parameters
function (
Callable
) – The function to run in the executorargs (
List
) – List of positional arguments to be used by the functionkwargs (
Dict
) – Dictionary of keyword arguments to be used by the function.task_metadata (
Dict
) – Dictionary of metadata for the task. Current keys are dispatch_id and node_id
- Returns
The result of the function execution
- Return type
output
- set_job_handle(handle)[source]#
Save the job_id/handle returned by the backend executing the task
- Arg(s)
handle: Any JSONable type to identifying the task being executed by the backend
- Return(s)
Response from saving the job handle to database
- Return type
Any
- teardown(task_metadata)[source]#
Placeholder to run any executor specific cleanup/teardown actions
- Return type
Any
- to_dict()#
Return a JSON-serializable dictionary representation of self
- Return type
dict
- write_streams_to_file(stream_strings, filepaths, dispatch_id, results_dir)[source]#
Write the contents of stdout and stderr to respective files.
- Parameters
stream_strings (
Iterable
[str
]) – The stream_strings to be written to files.filepaths (
Iterable
[str
]) – The filepaths to be used for writing the streams.dispatch_id (
str
) – The ID of the dispatch which initiated the request.results_dir (
str
) – The location of the results directory.
- Return type
None