Plotting analysis#

class ablator.analysis.plot.main.PlotAnalysis(results: DataFrame | Results, categorical_attributes: list[str] | None = None, numerical_attributes: list[str] | None = None, optim_metrics: dict[str, ablator.config.proto.Optim] | None = None, save_dir: str | None = None, cache: bool = False)[source]

Class for plotting experiment results. You can use this class and Results class to visualize the relationship between the result metrics and any hyperparameter you run ablation study on. This valuable insight offers an intuitive understanding of how these parameters may influence your model’s performance.

Plots supported are linear plots for numerical data and violin plots for categorical data.

Parameters:

resultspd.DataFrame | Results: The result dataframe.
categorical_attributeslist[str] | None: The list of all the categorical hyperparameter names
numerical_attributeslist[str] | None: The list of all the numerical hyperparameter names
optim_metricsdict[str, Optim] | None: A dictionary mapping metric names to optimization directions.
save_dirstr | None: The directory to save analysis results to.
cachebool: Whether to cache results.

Examples

Data frame to be used:

>>> df = pd.DataFrame({'val_accuracy': np.random.uniform(0.8,0.9,10),
...       'train_config.optimizer_config.arguments.lr': np.random.uniform(0.001, 0.1,10),
...       "index": range(10),
...       "path": range(10)})

Creating dictionaries that map the configuration parameters [categorical + numerical] to custom labels for plots:

>>> numerical_name_remap = {
...     "train_config.optimizer_config.arguments.lr": "Learning Rate",
... }
... categorical_name_remap = {}
... attribute_name_remap = {**categorical_name_remap, **numerical_name_remap}

Initalize the PlotAnalysis and plot the figures:

>>> analysis = PlotAnalysis(
...     df,
...     save_dir="./plots",
...     cache=True,
...     optim_metrics={"val_accuracy": Optim.max},
...     numerical_attributes=list(numerical_name_remap.keys()),
...     categorical_attributes=list(categorical_name_remap.keys()),
... )
>>> analysis.make_figures(
...    metric_name_remap={
...        "val_accuracy": "Validation Accuracy",
...    },
...    attribute_name_remap= attribute_name_remap
... )

The directory "plots" contains all the plots of the HPO experiments

make_figures(metric_name_remap: dict[str, str] | None = None, attribute_name_remap: dict[str, str] | None = None, save_dir: str | Path | None = None, **plt_kwargs: Any)[source]

Generate violin plots for categorical values and linear plots for numerical values. Plots are created as metrics vs. attributes. Additional keyword arguments to pass to the plot method are ax and append:

Parameters:

metric_name_remapdict[str, str] | None: mappings for config’s metrics keys to user defined names, by default None.
attribute_name_remapdict[str, str] | None: mappings for config’s searchspace names to user defined names for attributes, by default None.
save_dirstr | Path | None: optional directory of where to save the results, when unspecified, it expects one set during class initialization, by default None.
axAxes | None: A matplotlib.axes.Axes object representing the axis to plot on),
appendbool: A boolean indicating whether to append plots to an existing axes object) and extra arguments for creating the plots.

make_linearplot(attribute_names: list[str], metrics: list[str], save_dir: Path | str, **plt_kwargs) → dict[source]

To make linear plots for the given attribute names (data type: numerical) v.s. the metrics and save the plots to the save_dir directory.

Parameters:

attribute_nameslist[str]: list of attributes to plot against the metrics.
metricslist[str]: list of metrics to plot against the given attributes.
save_dirty.Union[Path, str]: directory to save results.
**plt_kwargs: Additional keyword arguments to pass to the plot method.

Returns:

dict

make_violinplot(attribute_names: list[str], metrics: list[str], save_dir: Path | str, **plt_kwargs)[source]

Make violin plots for the given attribute names (data type: discrete) v.s. the metrics and save the plots to the save_dir directory.

Parameters:

attribute_nameslist[str]: list of attributes to plot against the metrics.
metricslist[str]: list of metrics to plot against the given attributes.
save_dirty.Union[Path, str]: directory to save results.
**plt_kwargs: Additional keyword arguments to pass to the plot.