mlos_bench.services.local.local_exec

Helper functions to run scripts and commands locally on the scheduler side.

Classes

LocalExecService

Collection of methods to run scripts and commands in an external process on the node

Functions

split_cmdline(→ Iterable[List[str]])

A single command line may contain multiple commands separated by special characters

Module Contents

class mlos_bench.services.local.local_exec.LocalExecService(config: Dict[str, Any] | None = None, global_config: Dict[str, Any] | None = None, parent: mlos_bench.services.base_service.Service | None = None, methods: Dict[str, Callable] | List[Callable] | None = None)

Bases: mlos_bench.services.local.temp_dir_context.TempDirContextService, mlos_bench.services.types.local_exec_type.SupportsLocalExec

Collection of methods to run scripts and commands in an external process on the node acting as the scheduler.

Can be useful for data processing due to reduced dependency management complications vs the target environment.

Create a new instance of a service to run scripts locally.

Parameters:

• config (dict) – Free-format dictionary that contains parameters for the service. (E.g., root path for config files, etc.)

• global_config (dict) – Free-format dictionary of global parameters.

• parent (Service) – An optional parent service that can provide mixin functions.

• methods (Union[Dict[str, Callable], List[Callable], None]) – New methods to register with the service.

local_exec(script_lines: Iterable[str], env: Mapping[str, mlos_bench.tunables.tunable.TunableValue] | None = None, cwd: str | None = None) → Tuple[int, str, str]

Execute the script lines from script_lines in a local process.

Parameters:

• script_lines (Iterable[str]) – Lines of the script to run locally. Treat every line as a separate command to run.

• env (Mapping[str, Union[int, float, str]]) – Environment variables (optional).

• cwd (str) – Work directory to run the script at. If omitted, use temp_dir or create a temporary dir.

Returns:

(return_code, stdout, stderr) – A 3-tuple of return code, stdout, and stderr of the script process.

Return type:

(int, str, str)

abort_on_error

mlos_bench.services.local.local_exec.split_cmdline(cmdline: str) → Iterable[List[str]]

A single command line may contain multiple commands separated by special characters (e.g., &&, ||, etc.) so further split the commandline into an array of subcommand arrays.

Parameters:

cmdline (str) – The commandline to split.

Yields:

Iterable[List[str]] – A list of subcommands or separators, each one a list of tokens. Can be rejoined as a flattened array.

Return type:

Iterable[List[str]]