{ "cells": [ { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "# Quick start with Ablator" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "Welcome to the Ablator tutorial! In this chapter, you will learn how to use Ablator from scratch. We will provide you a simple demo where you can see what it looks like to run Ablator and also play around with Ablator with your own ideas. You are also welcome to download this demo @[Colab](https://colab.research.google.com/drive/127l02PicoLxAZ3b_JL9eVpMxQ_UQKgiG?usp=sharing) or [Github](https://github.com/SeanXiaoby/ablator-fork/tree/tutorial-demos/examples/demo-basics-usage-vscode)\n", "\n", "Let's get started!" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "## Installing" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "We assume that you have already installed Python and pip on your local machine. Please use the following command to install Ablator:\n", "\n", "```bash\n", "pip install ablator\n", "```" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "## Preparations\n", "\n", "To use Ablator in your own projects, there are some minimum codes you need to write. We can identify them as follows:\n", "\n", "- Set up configurations\n", "- Define a model and datasets\n", "- Launch Ablator\n", "\n", "Before started, let's import the necessary packages:" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "```python\n", "import shutil\n", "import argparse\n", "from typing import Any, Callable, Dict\n", "import torch\n", "import torch.nn as nn\n", "import torch.optim as optim\n", "import torchvision\n", "import torchvision.transforms as transforms\n", "from sklearn.metrics import accuracy_score\n", "```" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "### Set up configurations\n", "\n", "There are multiple ways to set up configurations for Ablator. In this chapter, we will use codes and parameters to set up the configurations for Ablator. The following code shows how to set up the configurations for Ablator:" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "```python\n", "from ablator import ModelConfig, TrainConfig, OptimizerConfig, RunConfig,\n", " configclass, Literal\n", "\n", "@configclass\n", "class SimpleConfig(ModelConfig):\n", " name: Literal[\"simplenet\"]\n", "\n", "@configclass\n", "class SimpleRunConfig(RunConfig):\n", " model_config: SimpleConfig\n", "\n", "run_config = SimpleRunConfig(\n", " experiment_dir = \"/tmp/dir\",\n", " train_config = TrainConfig(\n", " dataset = \"mnist\",\n", " batch_size = 64,\n", " epochs = 10,\n", " scheduler_config = None,\n", " rand_weights_init = False,\n", " optimizer_config = OptimizerConfig(\n", " name = \"sgd\",\n", " arguments = {\n", " \"lr\": 0.001,\n", " \"momentum\": 0.1\n", " }\n", " )\n", " ),\n", " model_config = SimpleConfig(name = \"simplenet\"),\n", " metrics_n_batches = 200,\n", " device= \"cpu\",\n", " amp=False\n", ")\n", "```" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "### Define a model and datasets\n", "\n", "The core parts of a single experiment in Ablator are actually your customized models and datasets. In this demo, we will use a simple LeNet-5 model and classic MNIST dataset to run a training experiment with Ablator. The following code shows how to define a model and datasets:" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "```python\n", "# Define a simple CNN model using components from PyTorch packages\n", "# And then we wrap up the CNN model in a wrapper class, which defines the loss function,\n", "# forward pass and indicated output formats\n", "\n", "class SimpleCNN(nn.Module):\n", " def __init__(self):\n", " super(SimpleCNN, self).__init__()\n", " self.conv1 = nn.Conv2d(1, 6, 5)\n", " self.relu1 = nn.ReLU()\n", " self.pool1 = nn.MaxPool2d(2, 2)\n", " self.conv2 = nn.Conv2d(6, 16, 5)\n", " self.relu2 = nn.ReLU()\n", " self.pool2 = nn.MaxPool2d(2, 2)\n", " self.fc1 = nn.Linear(16 * 4 * 4, 120)\n", " self.relu3 = nn.ReLU()\n", " self.fc2 = nn.Linear(120, 84)\n", " self.relu4 = nn.ReLU()\n", " self.fc3 = nn.Linear(84, 10)\n", "\n", " def forward(self, x):\n", " x = self.pool1(self.relu1(self.conv1(x)))\n", " x = self.pool2(self.relu2(self.conv2(x)))\n", " x = x.view(-1, 16 * 4 * 4)\n", " x = self.relu3(self.fc1(x))\n", " x = self.relu4(self.fc2(x))\n", " x = self.fc3(x)\n", " return x\n", "\n", "\n", "class MyModel(nn.Module):\n", " def __init__(self, config: SimpleConfig) -> None:\n", " super().__init__()\n", " self.model = SimpleCNN()\n", " self.loss = nn.CrossEntropyLoss()\n", " # self.optimizer = optim.SGD(self.model.parameters(), lr=0.001, momentum=0.9)\n", "\n", " def forward(self, x, labels, custom_input=None):\n", " # custom_input is for demo purposes only, defined in the dataset wrapper\n", " out = self.model(x)\n", " loss = self.loss(out, labels)\n", " if labels is not None:\n", " loss = self.loss(out, labels)\n", "\n", " out = out.argmax(dim=-1)\n", " return {\"y_pred\": out, \"y_true\": labels}, loss\n", "\n", "\n", "# Create the training & validation dataloaders from the MNIST dataset.\n", "# Also, data preprocessing is defined here, including normalization and other transformations\n", "\n", "transform = transforms.Compose([\n", " transforms.ToTensor(),\n", " transforms.Normalize((0.5,), (0.5,))\n", "])\n", "\n", "trainset = torchvision.datasets.MNIST(root='./datasets', train=True, download=True, transform=transform)\n", "trainloader = torch.utils.data.DataLoader(trainset, batch_size=64, shuffle=True, num_workers=2)\n", "\n", "testset = torchvision.datasets.MNIST(root='./datasets', train=False, download=True, transform=transform)\n", "testloader = torch.utils.data.DataLoader(testset, batch_size=64, shuffle=False, num_workers=2)\n", "\n", "\n", "# A evaluation function is definded here for Ablator to evaluate the model and training process.\n", "\n", "def my_accuracy(y_true, y_pred):\n", " return accuracy_score(y_true.flatten(), y_pred.flatten())\n", "\n", "```" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "### Launch Ablator\n", "\n", "As a final step, we can launch wrap up all we have done before and launch Ablator.\n", "\n", "Before launching Ablator, we have to make sure the temporary directory to cache the results are created and empty. Please use this line of codes to do it:\n", "\n", "```shell\n", "mkdir /tmp/dir\n", "```\n", "\n", "Then, we can launch Ablator with the following codes:" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "```python\n", "class MyModelWrapper(ModelWrapper):\n", " def __init__(self, *args, **kwargs):\n", " super().__init__(*args, **kwargs)\n", "\n", " def make_dataloader_train(self, run_config: SimpleRunConfig): # type: ignore\n", " return trainloader\n", "\n", " def make_dataloader_val(self, run_config: SimpleRunConfig): # type: ignore\n", " return testloader\n", "\n", " def evaluation_functions(self) -> Dict[str, Callable]:\n", " return {\"accuracy_score\": my_accuracy}\n", "\n", "\n", "if __name__ == \"__main__\":\n", " wrapper = MyModelWrapper(model_class=MyModel)\n", " # run_config = SimpleRunConfig.load(config)\n", " # shutil.rmtree(run_config.experiment_dir)\n", " ablator = ProtoTrainer(\n", " wrapper=wrapper,\n", " run_config=run_config,\n", " )\n", " ablator.launch()\n", "```" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "If you are using Juypter Notebook, you can directly run the above codes in the notebook. If you are using a Python script, you can save the above codes in a Python script and run it with the following command:\n", "\n", "```shell\n", "python .py\n", "```\n", "\n", "If Ablator is successfully launched, you should see information printed on the console!" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "## Access the results\n", "\n", "The training process should have saved in the temporary directory you specified in the `run_config`. To retrieve the training process, you can access the results by using the following codes:\n", "\n", "```shell\n", "cd /tmp/dir/\n", "cat results.json\n", "```\n", "\n", "You should see the training results from each epoch.\n", "\n", "If you are using the Jupyter Notebook, you can also visualize the results by using Tensorboard:\n", "\n", "```python\n", "# Load the TensorBoard extension\n", "import tensorflow as tf\n", "%load_ext tensorboard\n", "\n", "# Start TensorBoard\n", "%tensorboard --logdir /tmp/dir//dashboard/tensorboard\n", "```" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "## Next steps\n", "\n", "Ablator is far beyond what we show you in this tutorial. Please refer to the following chapters for more features and functionalies of Ablator!" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [] } ], "metadata": { "kernelspec": { "display_name": "env", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.10.6" }, "orig_nbformat": 4 }, "nbformat": 4, "nbformat_minor": 2 }