ablator.modules.loggers package

Submodules

ablator.modules.loggers.file module

class ablator.modules.loggers.file.FileLogger(path: str | Path | None = None, verbose: bool = True, prefix: str | None = None)

Bases: object

A logger that writes messages to a file and prints them to the console.

Parameters:

pathstr | Path | None

Path to the log file, by default None.

verbosebool

Whether to print messages to the console, by default True.

prefixstr | None

A prefix to add to each logged message, by default None.

Attributes:

WARNINGstr

ANSI escape code for the warning text color.

FAILstr

ANSI escape code for the error text color.

ENDCstr

ANSI escape code for resetting the text color.

ENDC: str = '\x1b[0m'

FAIL: str = '\x1b[91m'

WARNING: str = '\x1b[93m'

error(msg: str) → str

Log an error message.

Parameters:

msgstr

The message to log.

Returns:

str

the formated string message

info(msg: str, verbose: bool = False) → str

Log an info message.

Parameters:

msgstr

The message to log.

verbosebool

Whether to print messages to the console, by default False.

Returns:

str

the formated string message

set_path(path: str | Path)

Set the path to the log file.

Parameters:

pathstr | Path

The path to the log file.

set_prefix(prefix: str | None = None)

Set the prefix for the logger.

Parameters:

prefixstr | None

The prefix to add to each logged message, by default None.

warn(msg: str, verbose: bool = True) → str

Log a warning message.

Parameters:

msgstr

The message to log.

verbosebool

Whether to print messages to the console, by default True.

Returns:

str

the formated string message

class ablator.modules.loggers.file.RemoteFileLogger(path: str | Path | None = None, verbose: bool = True, prefix: str | None = None)

Bases: FileLogger

set_path(path: str | Path)

Set the path to the log file.

Parameters:

pathstr | Path

The path to the log file.

set_prefix(prefix: str | None = None)

Set the prefix for the logger.

Parameters:

prefixstr | None

The prefix to add to each logged message, by default None.

to_remote(address: str | None = None)

ablator.modules.loggers.main module

class ablator.modules.loggers.main.SummaryLogger(run_config: RunConfig, experiment_dir: str | Path | None = None, resume: bool = False, keep_n_checkpoints: int | None = None, verbose: bool = True)

Bases: object

A logger for training and evaluation summary.

Parameters:

run_configRunConfig

The run configuration.

experiment_dirstr | None | Path

Path to the trial directory, by default None.

resumebool

Whether to resume from an existing model directory, by default False.

keep_n_checkpointsint | None

Number of checkpoints to keep, by default None.

verbosebool

Whether to print messages to the console, by default True.

Raises:

FileExistsError

If resume is set to False but the experiment directory already exists.

Attributes:

SUMMARY_DIR_NAMEstr

Name of the summary directory.

RESULTS_JSON_NAMEstr

Name of the results JSON file.

LOG_FILE_NAMEstr

Name of the log file.

CONFIG_FILE_NAMEstr

Name of the configuration file.

METADATA_JSONstr

Name of the metadata JSON file.

CHKPT_DIR_NAMESlist[str]

List of checkpoint directory names.

CHKPT_DIR_VALUESlist[str]

List of checkpoint directory values.

CHKPT_DIRSdict[str, Path]

Dictionary containing checkpoint directories.

keep_n_checkpointsint

Number of checkpoints to keep.

log_iterationint

Current log iteration.

checkpoint_iterationdict[str, dict[str, int]]

checkpoint_iteration is a dictionary that keeps track of the checkpoint iterations for each directory. It is used in the checkpoint() method to determine the appropriate iteration number for the saved checkpoint.

log_file_pathPath | None

Path to the log file.

dashboardLoggerBase | None

Dashboard logger.

experiment_dirPath | None

the trial directory.

result_json_pathPath | None

Path to the results JSON file.

BACKUP_CONFIG_FILE_NAME: str = 'config_backup_{i}.yaml'

CHKPT_DIRS: dict[str, pathlib.Path]

CHKPT_DIR_NAMES: list[str] = ['best', 'recent']

CHKPT_DIR_VALUES: list[str] = ['best_checkpoints', 'checkpoints']

CONFIG_FILE_NAME: str = 'config.yaml'

LOG_FILE_NAME: str = 'train.log'

METADATA_JSON: str = 'metadata.json'

RESULTS_JSON_NAME: str = 'results.json'

SUMMARY_DIR_NAME: str = 'dashboard'

checkpoint(save_dict: object, file_name: str, itr: int | None = None, is_best: bool = False)

Save a checkpoint and update the checkpoint iteration

Saves the model checkpoint in the appropriate directory based on the is_best parameter. If is_best==True, the checkpoint is saved in the "best" directory, indicating the best performing model so far. Otherwise, the checkpoint is saved in the "recent" directory, representing the most recent checkpoint.

The file path for the checkpoint is constructed using the selected directory name ("best" or "recent"), and the file name with the format "{file_name}_{itr:010}.pt", where itr is the iteration number.

The checkpoint_iteration dictionary is updated with the current iteration number for each directory. If itr is not provided, the iteration number is increased by 1 each time a checkpoint is saved. Otherwise, the iteration number is set to the provided itr.

Parameters:

save_dictobject

The object to save.

file_namestr

The file name.

itrint | None

The iteration. If not provided, the current iteration is incremented by 1, by default None.

is_bestbool

Whether this is the best checkpoint, by default False.

clean_checkpoints(keep_n_checkpoints: int)

Clean up checkpoints and keep only the specified number of checkpoints.

Parameters:

keep_n_checkpointsint

Number of checkpoints to keep.

error(*args: Any, **kwargs: Any)

Log an error message to files and to console using the logger. Here you can use positional or keyword arguments. Possible parameters are shown in the Parameters section.

Parameters:

msgstr

The message to log.

info(*args: Any, **kwargs: Any)

Log an info to files and to console message using the logger. Here you can use positional or keyword arguments. Possible parameters are shown in the Parameters section.

Parameters:

msgstr

The message to log,

verbosebool

Whether to print messages to the console, by default False.

update(metrics: Metrics | dict, itr: int | None = None)

Update the dashboard with the given metrics. write some metrics to json files and update the current metadata (log_iteration)

Parameters:

metricsUnion[Metrics, dict]

The metrics to update.

itrOptional[int]

The iteration, by default None.

Raises:

AssertionError

If the iteration is not greater than the current iteration.

Notes

Attribute log_iteration is increased by 1 every time update() is called while training models.

warn(*args: Any, **kwargs: Any)

Log a warning message to files and to console using the logger. Here you can use positional or keyword arguments. Possible parameters are shown in the Parameters section.

Parameters:

msgstr

The message to log,

verbosebool

Whether to print messages to the console, by default True.

ablator.modules.loggers.tensor module

class ablator.modules.loggers.tensor.TensorboardLogger(summary_dir: str | Path)

Bases: LoggerBase

A logger class for Tensorboard visualization.

Parameters:

summary_dirUnion[str, Path]

The directory to store the Tensorboard summary files.

Attributes:

summary_dirUnion[str, Path]

The directory to store the Tensorboard summary files.

backend_loggerSummaryWriter

The PyTorch Tensorboard SummaryWriter object used to log data.

add_image(k: str, v: ndarray, itr: int, dataformats: str | None = 'CHW')

Add an image to the TensorBoard dashboard.

Parameters:

kstr

The tag associated with the image.

vnp.ndarray

The image data.

itrint

The iteration number.

dataformatsty.Optional[str]

The format of the image data, by default "CHW".

add_scalar(k: str, v: float | int, itr: int)

Add a scalar to the TensorBoard dashboard.

Parameters:

kstr

The tag associated with the scalar.

vfloat | int

The scalar value.

itrint

The iteration number.

add_scalars(k: str, v: dict[str, float | int], itr: int)

Add multiple scalars to the TensorBoard dashboard.

Parameters:

kstr

The main tag associated with the scalars.

vdict[str, float | int]

A dictionary of scalar tags and values.

itrint

The iteration number.

add_table(k: str, v: DataFrame, itr: int)

Add a table to the TensorBoard dashboard.

Parameters:

kstr

The tag associated with the table.

vpd.DataFrame

The table data.

itrint

The iteration number.

add_text(k: str, v: str, itr: int)

Add a text to the TensorBoard dashboard.

Parameters:

kstr

The tag associated with the text.

vstr

The text data.

itrint

The iteration number.

write_config(config: ConfigBase)

Write the configuration to the TensorBoard dashboard.

Parameters:

configConfigBase

The configuration object.

Module contents

class ablator.modules.loggers.LoggerBase

Bases: ABC

abstract add_image(k, v, itr, dataformats='CHW')

abstract add_scalar(k, v, itr)

abstract add_scalars(k, v, itr)

abstract add_table(k, v, itr)

abstract add_text(k, v, itr)

abstract write_config(config: ConfigBase)

class ablator.modules.loggers.LoggerConfig(*args: Any, debug: bool = False, **kwargs: Any)

Bases: ConfigBase

config_class

alias of LoggerConfig

abstract make_logger()