sdk/python/jobs/single-step/pytorch/train-hyperparameter-tune-deploy-with-pytorch/train-hyperparameter-tune-deploy-with-pytorch.ipynb

{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Train, hyperparameter tune, and deploy with PyTorch\n", "In this article, you will train, hyperparameter tune, and deploy a [PyTorch](https://pytorch.org/) model using the Azure Machine Learning (Azure ML) Python SDK v2.\n", "\n", "The example scripts in this article are used to classify chicken and turkey images to build a deep learning neural network (DNN) based on [PyTorch's transfer learning tutorial](https://pytorch.org/tutorials/beginner/transfer_learning_tutorial.html). Transfer learning is a technique that applies knowledge gained from solving one problem to a different but related problem. Transfer learning shortens the training process by requiring less data, time, and compute resources than training from scratch. To learn more about transfer learning, see the [deep learning vs machine learning](https://docs.microsoft.com/azure/machine-learning/concept-deep-learning-vs-machine-learning#what-is-transfer-learning) article.\n", "\n", "Whether you're training a deep learning PyTorch model from the ground-up or you're bringing an existing model into the cloud, you can use Azure Machine Learning to scale out open-source training jobs using elastic cloud compute resources. You can build, deploy, version, and monitor production-grade models with Azure Machine Learning.\n", "\n", "## Requirements\n", "In order to benefit from this article, you need to have:\n", "* an Azure subscription. If you don't have an Azure subscription, [create a free account](https://aka.ms/AMLFree) before you begin.\n", "* Run this code on either of these environments:\n", " 1. an Azure Machine Learning compute instance - no downloads or installation necessary\n", " * Complete the [Quickstart: Get started with Azure Machine Learning](https://docs.microsoft.com/azure/machine-learning/quickstart-create-resources) to create a dedicated notebook server pre-loaded with the SDK and the sample repository.\n", " * In the samples deep learning folder on the notebook server, find a completed and expanded notebook by navigating to this directory: * v2 > jobs > single-step > pytorch > train-hyperparameter-tune-deploy-with-pytorch* folder.\n", " 1. your own Jupyter Notebook server\n", " * [Install the Azure Machine Learning SDK v2](https://docs.microsoft.com/python/api/overview/azure/ml/installv2?view=azure-ml-py)\n", " * [Create a workspace configuration file](https://docs.microsoft.com/azure/machine-learning/how-to-configure-environment#workspace)\n", " * [Download the sample script files](TBD) `pytorch_train.py`\n", " \n", " You can also find a completed [Jupyter Notebook version](TBD) of this guide on the GitHub samples page.\n", "\n", "Before you can run the code in this article to create a GPU cluster, you'll need to [request a quota increase](https://docs.microsoft.com/azure/machine-learning/how-to-manage-quotas) for your workspace." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Connect to the workspace\n", "\n", "First, you'll need to connect to your Azure ML workspace. The workspace is the top-level resource for Azure Machine Learning, providing a centralized place to work with all the artifacts you create when you use Azure Machine Learning.\n", "\n", "We are using `DefaultAzureCredential` to get access to workspace. \n", "`DefaultAzureCredential` should be capable of handling most Azure SDK authentication scenarios. \n", "\n", "Reference for more available credentials if it does not work for you: [configure credential example](../../configuration.ipynb), [azure-identity reference doc](https://docs.microsoft.com/python/api/azure-identity/azure.identity?view=azure-python)." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "name": "credential" }, "outputs": [], "source": [ "# Handle to the workspace\n", "from azure.ai.ml import MLClient\n", "\n", "# Authentication package\n", "from azure.identity import DefaultAzureCredential\n", "\n", "credential = DefaultAzureCredential()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "If you want to use a browser instead to login and authenticate, you can use the following code instead. " ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# Handle to the workspace\n", "# from azure.ai.ml import MLClient\n", "\n", "# Authentication package\n", "# from azure.identity import InteractiveBrowserCredential\n", "# credential = InteractiveBrowserCredential()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "In the next cell, enter your Subscription ID, Resource Group name and Workspace name. To find subscription ID and resource group:\n", "\n", "1. In the upper right Azure Machine Learning Studio toolbar, select your workspace name.\n", "1. Copy the value for Resource group and subsccription ID into the code." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "name": "ml_client" }, "outputs": [], "source": [ "# Get a handle to the workspace\n", "ml_client = MLClient(\n", " credential=credential,\n", " subscription_id=\"<SUBSCRIPTION_ID>\",\n", " resource_group_name=\"<RESOURCE_GROUP>\",\n", " workspace_name=\"<AML_WORKSPACE_NAME>\",\n", ")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The result is a handler to the workspace that you'll use to manage other resources and jobs.\n", "\n", "> [!IMPORTANT]\n", "> Creating MLClient will not connect to the workspace. The client initialization is lazy, it will wait for the first time it needs to make a call (in the notebook below, that will happen during compute creation)." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Create a Compute Resource to run our job\n", "\n", "AzureML needs a compute resource for running a job. It can be single or multi-node machines with Linux or Windows OS, or a specific compute fabric like Spark.\n", "\n", "In this example, we provision a Linux [compute cluster](https://docs.microsoft.com/azure/machine-learning/how-to-create-attach-compute-cluster?tabs=python). See the [full list on VM sizes and prices](https://azure.microsoft.com/pricing/details/machine-learning/) .\n", "\n", "For this example we need a gpu cluster, let's pick a STANDARD_NC6 model and create an Azure ML Compute" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "name": "gpu_compute_target" }, "outputs": [], "source": [ "from azure.ai.ml.entities import AmlCompute\n", "\n", "gpu_compute_target = \"gpu-cluster\"\n", "\n", "try:\n", " # let's see if the compute target already exists\n", " gpu_cluster = ml_client.compute.get(gpu_compute_target)\n", " print(\n", " f\"You already have a cluster named {gpu_compute_target}, we'll reuse it as is.\"\n", " )\n", "\n", "except Exception:\n", " print(\"Creating a new gpu compute target...\")\n", "\n", " # Let's create the Azure ML compute object with the intended parameters\n", " gpu_cluster = AmlCompute(\n", " # Name assigned to the compute cluster\n", " name=\"gpu-cluster\",\n", " # Azure ML Compute is the on-demand VM service\n", " type=\"amlcompute\",\n", " # VM Family\n", " size=\"STANDARD_NC6s_v3\",\n", " # Minimum running nodes when there is no job running\n", " min_instances=0,\n", " # Nodes in cluster\n", " max_instances=4,\n", " # How many seconds will the node running after the job termination\n", " idle_time_before_scale_down=180,\n", " # Dedicated or LowPriority. The latter is cheaper but there is a chance of job termination\n", " tier=\"Dedicated\",\n", " )\n", "\n", " # Now, we pass the object to MLClient's create_or_update method\n", " gpu_cluster = ml_client.begin_create_or_update(gpu_cluster).result()\n", "\n", "print(\n", " f\"AMLCompute with name {gpu_cluster.name} is created, the compute size is {gpu_cluster.size}\"\n", ")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Create an environment for the job\n", "\n", "To run an AzureML job, you'll need an [environment](https://docs.microsoft.com/azure/machine-learning/concept-environments). An environment is the software runtime and libraries that you want installed on the compute where you’ll be training. It is similar to your python emvironment on your local machine.\n", "\n", "AzureML provides many curated or readymade environments which are useful for common training and inference scenarios. You can also create your own “custom” environments using a docker image, or a conda configuration \n", "\n", "In this example, you'll reuse the curated AzureML environment `AzureML-pytorch-1.9-ubuntu18.04-py37-cuda11-gp`. You will use the latest version of this environment using the `@latest` directive." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "name": "curated_env_name" }, "outputs": [], "source": [ "curated_env_name = \"AzureML-acpt-pytorch-2.2-cuda12.1@latest\"" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Data for training\n", "The dataset we will use (located on a public blob [here](https://azuremlexamples.blob.core.windows.net/datasets/fowl_data.zip) as a zip file) consists of about 120 training images each for turkeys and chickens, with 100 validation images for each class. The images are a subset of the [Open Images v5 Dataset](https://storage.googleapis.com/openimages/web/index.html). We will download and extract the dataset as part of our training script `pytorch_train.py`" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Build the command job to train\n", "\n", "Now that you have all assets required to run your job, it's time to build the job itself, using the Azure ML Python SDK v2. We will be creating a `command` job.\n", "\n", "An AzureML `command` job is a resource that specifies all the details needed to execute your training code in the cloud: inputs and outputs, the type of hardware to use, software to install, and how to run your code. the `command` job contains information to execute a single command.\n", "\n", "## The training script\n", "\n", "We will use the training script - *pytorch_train.py* python file. This script dowloads data, trains a model and registers the model too.\n", "\n", "## Configure the Command\n", "Now that you have a script that can perform the desired tasks, you'll use the general purpose **command** to run this script. \n", "\n", "* The inputs used in this command are number of epochs, learning rate, momentum and output directory\n", "* You will use the compute created earlier to run this command.\n", "* You will use the curated environment `AzureML-pytorch-1.9-ubuntu18.04-py37-cuda11-gpu` which was intialized earlier.\n", "* You will configure the command line action itself - in this case, the command is `python pytorch_train.py`. You can access the inputs/outputs in the command via the `${{ ... }}` notation.\n", "* You will configure some metadata like display name, experiment name etc. An experiment is a container for all the iterations one does on a certain project. All the jobs submitted under the same experiment name would be listed next to each other in Azure ML studio." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "name": "job" }, "outputs": [], "source": [ "from azure.ai.ml import command\n", "from azure.ai.ml import Input\n", "\n", "job = command(\n", " inputs=dict(\n", " num_epochs=30, learning_rate=0.001, momentum=0.9, output_dir=\"./outputs\"\n", " ),\n", " compute=gpu_compute_target,\n", " environment=curated_env_name,\n", " code=\"./src/\", # location of source code\n", " command=\"python pytorch_train.py --num_epochs ${{inputs.num_epochs}} --output_dir ${{inputs.output_dir}}\",\n", " experiment_name=\"pytorch-birds\",\n", " display_name=\"pytorch-birds-image\",\n", ")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Submit the job \n", "\n", "It's now time to submit the job to run in AzureML. This time you'll use `create_or_update` on `ml_client.jobs`.\n", "\n", "Once completed, the job will register a model in your workspace as a result of training. You can view the job in AzureML studio by clicking on the link in the output of the next cell." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "name": "create_job" }, "outputs": [], "source": [ "ml_client.jobs.create_or_update(job)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## What happens during job execution\n", "As the job is executed, it goes through the following stages:\n", "\n", "* *Preparing*: A docker image is created according to the environment defined. The image is uploaded to the workspace's container registry and cached for later runs. Logs are also streamed to the job history and can be viewed to monitor progress. If a curated environment is used, the cached image backing that curated environment will be used.\n", "* *Scaling*: The cluster attempts to scale up if the cluster requires more nodes to execute the run than are currently available.\n", "* *Running*: All scripts in the `src` folder are uploaded to the compute target, data stores are mounted or copied, and the script is executed. Outputs from stdout and the ./logs folder are streamed to the job history and can be used to monitor the job." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Tune model hyperparameters\n", "Now that we've seen how to do a simple PyTorch training run using the SDK, let's see if we can further improve the accuracy of our model. We can optimize our model's hyperparameters using Azure Machine Learning's sweep capabilities.\n", "\n", "You will replace some of the parameters passed to the training job with special inputs from the `azure.ml.sweep` package – that way, you are defining the parameter space in which to search.\n", "\n", "Since the training script uses a learning rate schedule to decay the learning rate every several epochs, you can tune the initial learning rate and the momentum parameters. " ] }, { "cell_type": "code", "execution_count": null, "metadata": { "name": "job_for_sweep" }, "outputs": [], "source": [ "from azure.ai.ml.sweep import Uniform\n", "\n", "# we will reuse the command_job created before. we call it as a function so that we can apply inputs\n", "job_for_sweep = job(\n", " learning_rate=Uniform(min_value=0.0005, max_value=0.005),\n", " momentum=Uniform(min_value=0.9, max_value=0.99),\n", ")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Then you configure sweep on the command job, with some sweep-specific parameters like the primary metric to watch and the sampling algorithm to use. \n", "* You can use random sampling to try different configuration sets of hyperparameters to maximize the primary metric, the best validation accuracy (best_val_acc).\n", "* You can specify the early termination policy to use to early terminate poorly performing runs. Here you use the BanditPolicy, which will terminate any run that doesn't fall within the slack factor of our primary evaluation metric. You will apply this policy every epoch (since we report our `best_val_acc` metric every epoch and `evaluation_interval`=1). Notice we will delay the first policy evaluation until after the first 10 epochs (`delay_evaluation`=10). " ] }, { "cell_type": "code", "execution_count": null, "metadata": { "name": "sweep_job" }, "outputs": [], "source": [ "from azure.ai.ml.sweep import BanditPolicy\n", "\n", "sweep_job = job_for_sweep.sweep(\n", " compute=\"gpu-cluster\",\n", " sampling_algorithm=\"random\",\n", " primary_metric=\"best_val_acc\",\n", " goal=\"Maximize\",\n", " max_total_trials=8,\n", " max_concurrent_trials=4,\n", " early_termination_policy=BanditPolicy(\n", " slack_factor=0.15, evaluation_interval=1, delay_evaluation=10\n", " ),\n", ")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now you can submit this job as before. This will now run a sweep job that sweeps over our train job." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "name": "create_sweep_job" }, "outputs": [], "source": [ "returned_sweep_job = ml_client.create_or_update(sweep_job)\n", "\n", "# stream the output and wait until the job is finished\n", "ml_client.jobs.stream(returned_sweep_job.name)\n", "\n", "# refresh the latest status of the job after streaming\n", "returned_sweep_job = ml_client.jobs.get(name=returned_sweep_job.name)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "You can monitor the job using the studio UI link presented when you run the job.\n", "\n", "## Find the best model\n", "**Once all the runs complete**, you can find the run that produced the model with the highest accuracy." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "name": "model" }, "outputs": [], "source": [ "from azure.ai.ml.entities import Model\n", "\n", "if returned_sweep_job.status == \"Completed\":\n", "\n", " # First let us get the run which gave us the best result\n", " best_run = returned_sweep_job.properties[\"best_child_run_id\"]\n", "\n", " # lets get the model from this run\n", " model = Model(\n", " # the script stores the model as \"outputs\"\n", " path=\"azureml://jobs/{}/outputs/artifacts/paths/outputs/\".format(best_run),\n", " name=\"run-model-example\",\n", " description=\"Model created from run.\",\n", " type=\"custom_model\",\n", " )\n", "\n", "else:\n", " print(\n", " \"Sweep job status: {}. Please wait until it completes\".format(\n", " returned_sweep_job.status\n", " )\n", " )" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Deploy the model as an online endpoint\n", "\n", "Now deploy your machine learning model as a web service in the Azure cloud, an [`online endpoint`](https://docs.microsoft.com/azure/machine-learning/concept-endpoints).\n", "\n", "To deploy a machine learning service, you usually need:\n", "\n", "* The model assets (file, metadata) that you want to deploy. You've already registered these assets in your training job.\n", "* Some code to run as a service. The code executes the model on a given input request. This entry script receives data submitted to a deployed web service and passes it to the model, then returns the model's response to the client. The script is specific to your model. The entry script must understand the data that the model expects and returns. When using a MLFlow model, this script is automatically created for you." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Create a new online endpoint\n", "\n", "As a firsdt step, you need to create your online endpoint. The endpoint name needs to be unique in the entire Azure region. For this article, you'll create a unique name using [`UUID`](https://en.wikipedia.org/wiki/Universally_unique_identifier#:~:text=A%20universally%20unique%20identifier%20(UUID,%2C%20for%20practical%20purposes%2C%20unique.)." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "name": "online_endpoint_name" }, "outputs": [], "source": [ "import uuid\n", "\n", "# Creating a unique name for the endpoint\n", "online_endpoint_name = \"aci-birds-endpoint-\" + str(uuid.uuid4())[:8]" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "name": "endpoint" }, "outputs": [], "source": [ "from azure.ai.ml.entities import ManagedOnlineEndpoint\n", "\n", "# create an online endpoint\n", "endpoint = ManagedOnlineEndpoint(\n", " name=online_endpoint_name,\n", " description=\"Classify turkey/chickens using transfer learning with PyTorch\",\n", " auth_mode=\"key\",\n", " tags={\"data\": \"birds\", \"method\": \"transfer learning\", \"framework\": \"pytorch\"},\n", ")\n", "\n", "endpoint = ml_client.begin_create_or_update(endpoint).result()\n", "\n", "print(f\"Endpoint {endpoint.name} provisioning state: {endpoint.provisioning_state}\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Once you've created an endpoint, you can retrieve it as below:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "name": "get_endpoint" }, "outputs": [], "source": [ "endpoint = ml_client.online_endpoints.get(name=online_endpoint_name)\n", "\n", "print(\n", " f'Endpint \"{endpoint.name}\" with provisioning state \"{endpoint.provisioning_state}\" is retrieved'\n", ")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Deploy the model to the endpoint\n", "\n", "Once the endpoint is created, deploy the model with the entry script. Each endpoint can have multiple deployments and direct traffic to these deployments can be specified using rules. Here you'll create a single deployment that handles 100% of the incoming traffic. We have chosen a color name for the deployment, for example, *aci-blue*, *aci-green*, *aci-red* deployments, which is arbitrary.\n", "\n", "Deploy the best version of the model. \n", "\n", "> [!NOTE]\n", "> Expect this deployment to take approximately 6 to 8 minutes." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "name": "blue_deployment" }, "outputs": [], "source": [ "from azure.ai.ml.entities import (\n", " ManagedOnlineDeployment,\n", " Model,\n", " Environment,\n", " CodeConfiguration,\n", ")\n", "\n", "online_deployment_name = \"aci-blue\"\n", "\n", "# create an online deployment.\n", "blue_deployment = ManagedOnlineDeployment(\n", " name=online_deployment_name,\n", " endpoint_name=online_endpoint_name,\n", " model=model,\n", " environment=curated_env_name,\n", " code_configuration=CodeConfiguration(code=\"./score/\", scoring_script=\"score.py\"),\n", " instance_type=\"Standard_NC6s_v3\",\n", " instance_count=1,\n", ")\n", "\n", "blue_deployment = ml_client.begin_create_or_update(blue_deployment).result()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Test with a sample image\n", "\n", "Now that the model is deployed to the endpoint, you can run inference with it. Let us take a sample image to predict on. Lets dispaly this iamge." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "name": "display_image" }, "outputs": [], "source": [ "# install pillow if PIL cannot imported\n", "%pip install pillow\n", "import json\n", "from PIL import Image\n", "import matplotlib.pyplot as plt\n", "\n", "%matplotlib inline\n", "plt.imshow(Image.open(\"test_img.jpg\"))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Create a function to format and resize the iamge" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "name": "process_image" }, "outputs": [], "source": [ "# install torch and torchvision if needed\n", "%pip install torch\n", "%pip install torchvision\n", "\n", "import torch\n", "from torchvision import transforms\n", "\n", "\n", "def preprocess(image_file):\n", " \"\"\"Preprocess the input image.\"\"\"\n", " data_transforms = transforms.Compose(\n", " [\n", " transforms.Resize(256),\n", " transforms.CenterCrop(224),\n", " transforms.ToTensor(),\n", " transforms.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225]),\n", " ]\n", " )\n", "\n", " image = Image.open(image_file)\n", " image = data_transforms(image).float()\n", " image = torch.tensor(image)\n", " image = image.unsqueeze(0)\n", " return image.numpy()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Format the iamge and convert it to a json file" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "name": "test_json" }, "outputs": [], "source": [ "image_data = preprocess(\"test_img.jpg\")\n", "input_data = json.dumps({\"data\": image_data.tolist()})\n", "with open(\"request.json\", \"w\") as outfile:\n", " outfile.write(input_data)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Invoke the endpoint with this json and print the result" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "name": "test_deployment" }, "outputs": [], "source": [ "# test the blue deployment\n", "result = ml_client.online_endpoints.invoke(\n", " endpoint_name=online_endpoint_name,\n", " request_file=\"request.json\",\n", " deployment_name=online_deployment_name,\n", ")\n", "\n", "print(result)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Clean up resources\n", "\n", "If you're not going to use the endpoint, delete it to stop using the resource. Make sure no other deployments are using an endpoint before you delete it.\n", "\n", "\n", "> [!NOTE]\n", "> Expect this step to take approximately 6 to 8 minutes." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "name": "delete_endpoint" }, "outputs": [], "source": [ "ml_client.online_endpoints.begin_delete(name=online_endpoint_name)" ] } ], "metadata": { "description": { "description": "Train, hyperparameter tune, and deploy a PyTorch model to classify chicken and turkey images to build a deep learning neural network (DNN) based on PyTorch's transfer learning tutorial." }, "kernelspec": { "display_name": "Python 3.10 - SDK V2", "language": "python", "name": "python310-sdkv2" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.7.12" }, "orig_nbformat": 4, "vscode": { "interpreter": { "hash": "27e3d0e72e4fca658b4cea21737d79da5e68f90d3ccf7f33207fcc73892eee38" } } }, "nbformat": 4, "nbformat_minor": 2 }

sdk/python/jobs/single-step/pytorch/train-hyperparameter-tune-deploy-with-pytorch/train-hyperparameter-tune-deploy-with-pytorch.ipynb (718 lines of code) (raw):