ablator.modules.storage package

Submodules

ablator.modules.storage.remote module

File is obsolete to be replaced with rclone. TODO

class ablator.modules.storage.remote.RemoteConfig(*args: Any, debug: bool = False, **kwargs: Any)

Bases: ConfigBase

Configuration for a remote storage.

Attributes:

remote_pathstr

The path to the remote storage.

usernamestr

The username to use for the remote storage.

hostnamestr

The hostname of the remote storage.

portNone | int

The port to use for the remote storage.

exclude_globNone | str

A glob to exclude from the rsync.

exclude_chkptsbool

Whether to exclude checkpoints from the rsync.

config_class

alias of RemoteConfig

exclude_chkpts: bool = False

exclude_glob: str = None

hostname: str

port: int = None

remote_path: str

rsync_down(local_path: Path, destination: str, timeout_s: int | None = None, run_async: bool = False)

start a new process and download files from the remote storage.

Parameters:

local_pathPath

The local path to download to.

destinationstr

The destination path in the remote storage.

timeout_sint | None

The timeout in seconds, by default None.

run_asyncbool

Whether to run the command asynchronously, by default False.

rsync_up(local_path: Path, destination: str, timeout_s: int | None = None, run_async: bool = False)

start a new process and upload files to the remote storage.

Parameters:

local_pathPath

The local path to upload.

destinationstr

The destination path in the remote storage.

timeout_sint | None

The timeout in seconds, by default None.

run_asyncbool

Whether to run the command asynchronously, by default False.

username: str

ablator.modules.storage.remote.run_cmd_wait(cmd: str, timeout: int = 300, raise_errors: bool = False) → str

Run a command and wait for it to finish. If the command takes longer than timeout seconds, kill it. If raise_errors is True, raise a TimeoutExpired exception.

Parameters:

cmdstr

The command to run.

timeoutint

The timeout in seconds, by default 300.

raise_errorsbool

Whether to raise errors, by default False.

Returns:

Optional[str]

The output of the command.

Raises:

subprocess.TimeoutExpired

If the command takes longer than timeout seconds and raise_errors is True.

Module contents