ablator.modules.storage package#
Submodules#
ablator.modules.storage.remote module#
File is obsolete to be replaced with rclone. TODO
- class ablator.modules.storage.remote.RemoteConfig(*args, **kwargs)[source]#
Bases:
ConfigBaseConfiguration for a remote storage.
- Attributes:
- remote_pathstr
The path to the remote storage.
- usernamestr
The username to use for the remote storage.
- hostnamestr
The hostname of the remote storage.
- portNone | int
The port to use for the remote storage.
- exclude_globNone | str
A glob to exclude from the rsync.
- exclude_chkptsbool
Whether to exclude checkpoints from the rsync.
- config_class#
alias of
RemoteConfig
- exclude_chkpts: bool = False#
- exclude_glob: str = None#
- hostname: str#
- port: int = None#
- remote_path: str#
- rsync_down(local_path: Path, destination: str, timeout_s: int | None = None, run_async=False)[source]#
start a new process and download files from the remote storage.
- Parameters:
- local_pathPath
The local path to download to.
- destinationstr
The destination path in the remote storage.
- timeout_sint | None
The timeout in seconds.
- run_asyncbool
Whether to run the command asynchronously.
- rsync_up(local_path: Path, destination: str, timeout_s: int | None = None, run_async=False)[source]#
start a new process and upload files to the remote storage.
- Parameters:
- local_pathPath
The local path to upload.
- destinationstr
The destination path in the remote storage.
- timeout_sint | None
The timeout in seconds.
- run_asyncbool
Whether to run the command asynchronously.
- username: str#
- ablator.modules.storage.remote.run_cmd_wait(cmd, timeout=300, raise_errors=False) str[source]#
Run a command and wait for it to finish. If the command takes longer than
timeoutseconds, kill it. Ifraise_errorsis True, raise aTimeoutExpiredexception.- Parameters:
- cmdstr
The command to run.
- timeoutint
The timeout in seconds.
- raise_errorsbool
Whether to raise errors.
- Returns:
- str
The output of the command.