From 0abd786f2290955f2ffa82e85a0a9aff1ca73164 Mon Sep 17 00:00:00 2001 From: Hongshan Li Date: Tue, 24 Nov 2020 00:33:47 -0800 Subject: [PATCH] MNIST offline mode with MXNet/PyTorch/TensorFlow (#1787) * mnist train and test notebooks tested on local mode * added config.json for global config * training notebook requires user to be in the same region as the public s3 bucket * default to non-local mode * Website preview (#1764) * mnist train and test notebooks tested on local mode * added config.json for global config * training notebook requires user to be in the same region as the public s3 bucket * default to non-local mode * cleared output / added sym link to global config.json * minor fix * added swp file to gitignore; * added updated mxnet examples * train entry point tested * train notebook ready * train / inference tested * inference.py not needed * removed zombie cells / changed public model addr * cleared outputs * default to non-local mode * default to non-local mode * removed downloaded model * default to nonlocal mode * small bug fix * Pytorch vpc (#1780) * deleted training notebook * train / deployment notebook tested in local mode * default to non-local mode * added utils * changed rst * removed a trained model * Website preview (#1785) * Notebook cleaned and data on S3 - XGBoost (#1713) * Notebook cleaned and data on S3 * Cleared all cell outputs * formatting a cell * PR comments addressed * PR comments addressed * Instance type updated * Bucket changed to prod regions bucket and citation added * Deleted install instructions * Cleaned up Linear Learner notebook (#1709) * Cleaned up Linear Learner notebook * directory name changed on S3 * PR comments addressed * Updated instance type and kernel * Bucket changed to prod bucket and citation added * Changed parameter name as in SageMaker v2 * Update introduction_to_amazon_algorithms/linear_learner_abalone/Linear_Learner_Regression_csv_format.ipynb Co-authored-by: Aaron Markham * Update introduction_to_amazon_algorithms/linear_learner_abalone/Linear_Learner_Regression_csv_format.ipynb Co-authored-by: Aaron Markham * Update introduction_to_amazon_algorithms/linear_learner_abalone/Linear_Learner_Regression_csv_format.ipynb Co-authored-by: Aaron Markham * Update introduction_to_amazon_algorithms/linear_learner_abalone/Linear_Learner_Regression_csv_format.ipynb Co-authored-by: Aaron Markham * Update introduction_to_amazon_algorithms/linear_learner_abalone/Linear_Learner_Regression_csv_format.ipynb Co-authored-by: Aaron Markham * Update introduction_to_amazon_algorithms/linear_learner_abalone/Linear_Learner_Regression_csv_format.ipynb Co-authored-by: Aaron Markham Co-authored-by: Aaron Markham * Image classification notebook fix + data source on S3 (#1700) * Fixing notebooks * Cleared all outputs * PR comments addressed and code cleaned * Typo fix * Added kernel type in description * Fixed instance type to studio * Added instance type * Data bucket changed to prod bucket * Download links added * estimator parameter changed to be compatible with SageMaker v2 * Update introduction_to_amazon_algorithms/imageclassification_caltech/Image-classification-transfer-learning-highlevel.ipynb Co-authored-by: Aaron Markham Co-authored-by: Aaron Markham * Small fix, notebook formatting, data on S3 - PCA (#1715) * Small fix and notebook formatting * Variable name changed * Updated instance type and kernel * Bucket changed to prod bucket * Changed parameter name in Estimator as in SageMaker v2 * Added missing import Co-authored-by: Aaron Markham * website: add getting started videos; rename featured examples to studio (#1758) * add getting started videos; rename featured examples to studio * vidoes for getting started; combine to same page * refactor byo algo with pipe mode to be python3 and sdk v2 (#1690) * Docs: Deleting working_with_redshift_data.ipynb New notebook will cover this topic in more depth. Deleting to remove duplication. https://github.com/aws/amazon-sagemaker-examples/issues/1447 * add GT video, fix links, update copyright notice (#1763) * train entry point tested * train notebook ready * train / inference tested * inference.py not needed * default to non-local mode * default to non-local mode * removed downloaded model * removed a trained model Co-authored-by: vivekmadan2 <53404938+vivekmadan2@users.noreply.github.com> Co-authored-by: Aaron Markham Co-authored-by: Talia <31782251+TEChopra1000@users.noreply.github.com> * add a line break * editorial fix par style guide * removed swp file * fix json * fixed typos * bug fix * better way to upload data * deleted empty cell * tensorflow / pytorch tested * removed mxnet for more testing * cleared output * removed mxnet from rst * minor fixes Co-authored-by: vivekmadan2 <53404938+vivekmadan2@users.noreply.github.com> Co-authored-by: Aaron Markham Co-authored-by: Talia <31782251+TEChopra1000@users.noreply.github.com> --- .gitignore | 1 + config.json | 4 + frameworks/pytorch/code/config.json | 1 + frameworks/pytorch/code/inference.py | 57 ++ frameworks/pytorch/code/test_inference.py | 91 +++ frameworks/pytorch/code/test_train.py | 60 ++ frameworks/pytorch/code/train.py | 203 ++++++ .../pytorch/get_started_mnist_deploy.ipynb | 601 ++++++++++++++++++ .../pytorch/get_started_mnist_train.ipynb | 379 +++++++++++ frameworks/pytorch/utils | 1 + frameworks/tensorflow/code/config.json | 1 + frameworks/tensorflow/code/test_train.py | 59 ++ frameworks/tensorflow/code/train.py | 200 ++++++ .../tensorflow/get_started_mnist_deploy.ipynb | 304 +++++++++ .../tensorflow/get_started_mnist_train.ipynb | 387 +++++++++++ frameworks/tensorflow/utils | 1 + frameworks/utils/config.json | 4 + frameworks/utils/mnist.py | 101 +++ .../{ => code}/horovod_mnist.py | 16 +- .../mxnet_horovod_mnist/code/test_train.py | 59 ++ .../tensorflow_serving_container.ipynb | 4 +- training/frameworks.rst | 8 +- 22 files changed, 2532 insertions(+), 10 deletions(-) create mode 100644 config.json create mode 120000 frameworks/pytorch/code/config.json create mode 100644 frameworks/pytorch/code/inference.py create mode 100644 frameworks/pytorch/code/test_inference.py create mode 100644 frameworks/pytorch/code/test_train.py create mode 100644 frameworks/pytorch/code/train.py create mode 100644 frameworks/pytorch/get_started_mnist_deploy.ipynb create mode 100644 frameworks/pytorch/get_started_mnist_train.ipynb create mode 120000 frameworks/pytorch/utils create mode 120000 frameworks/tensorflow/code/config.json create mode 100644 frameworks/tensorflow/code/test_train.py create mode 100644 frameworks/tensorflow/code/train.py create mode 100644 frameworks/tensorflow/get_started_mnist_deploy.ipynb create mode 100644 frameworks/tensorflow/get_started_mnist_train.ipynb create mode 120000 frameworks/tensorflow/utils create mode 100644 frameworks/utils/config.json create mode 100644 frameworks/utils/mnist.py rename sagemaker-python-sdk/mxnet_horovod_mnist/{ => code}/horovod_mnist.py (97%) create mode 100644 sagemaker-python-sdk/mxnet_horovod_mnist/code/test_train.py diff --git a/.gitignore b/.gitignore index 482c392c..b84193c2 100644 --- a/.gitignore +++ b/.gitignore @@ -3,4 +3,5 @@ **/__pycache__ **/.aws-sam .DS_Store + **/_build diff --git a/config.json b/config.json new file mode 100644 index 00000000..1db24021 --- /dev/null +++ b/config.json @@ -0,0 +1,4 @@ +{ + "public_bucket": "sagemaker-sample-files" +} + diff --git a/frameworks/pytorch/code/config.json b/frameworks/pytorch/code/config.json new file mode 120000 index 00000000..5d347dc6 --- /dev/null +++ b/frameworks/pytorch/code/config.json @@ -0,0 +1 @@ +../../../config.json \ No newline at end of file diff --git a/frameworks/pytorch/code/inference.py b/frameworks/pytorch/code/inference.py new file mode 100644 index 00000000..e02a40c9 --- /dev/null +++ b/frameworks/pytorch/code/inference.py @@ -0,0 +1,57 @@ +import json +import logging +import sys + +import torch +import torch.nn as nn +import torch.nn.functional as F + +logger = logging.getLogger(__name__) +logger.setLevel(logging.DEBUG) +logger.addHandler(logging.StreamHandler(sys.stdout)) + +# Based on https://github.com/pytorch/examples/blob/master/mnist/main.py +class Net(nn.Module): + def __init__(self): + super(Net, self).__init__() + self.conv1 = nn.Conv2d(1, 10, kernel_size=5) + self.conv2 = nn.Conv2d(10, 20, kernel_size=5) + self.conv2_drop = nn.Dropout2d() + self.fc1 = nn.Linear(320, 50) + self.fc2 = nn.Linear(50, 10) + + def forward(self, x): + x = F.relu(F.max_pool2d(self.conv1(x), 2)) + x = F.relu(F.max_pool2d(self.conv2_drop(self.conv2(x)), 2)) + x = x.view(-1, 320) + x = F.relu(self.fc1(x)) + x = F.dropout(x, training=self.training) + x = self.fc2(x) + return F.log_softmax(x, dim=1) + +device = torch.device('cuda' if torch.cuda.is_available() else 'cpu') + +def model_fn(model_dir): + model = Net().to(device) + model.eval() + return model + +# data preprocessing +def input_fn(request_body, request_content_type): + assert request_content_type=='application/json' + data = json.loads(request_body)['inputs'] + data = torch.tensor(data, dtype=torch.float32, device=device) + return data + +# inference +def predict_fn(input_object, model): + with torch.no_grad(): + prediction = model(input_object) + return prediction + +# postprocess +def output_fn(predictions, content_type): + assert content_type == 'application/json' + res = predictions.cpu().numpy().tolist() + return json.dumps(res) + diff --git a/frameworks/pytorch/code/test_inference.py b/frameworks/pytorch/code/test_inference.py new file mode 100644 index 00000000..59824c7a --- /dev/null +++ b/frameworks/pytorch/code/test_inference.py @@ -0,0 +1,91 @@ +from inference import model_fn, input_fn, predict_fn, output_fn +import os +import json +import shutil +import boto3 +import botocore +import tarfile +import numpy as np +import sagemaker + +def fetch_model(model_data): + """ Untar the model.tar.gz object either from local file system + or a S3 location + + Args: + model_data (str): either a path to local file system starts with + file:/// that points to the `model.tar.gz` file or an S3 link + starts with s3:// that points to the `model.tar.gz` file + + Returns: + model_dir (str): the directory that contains the uncompress model + checkpoint files + """ + + model_dir = "/tmp/model" + if not os.path.exists(model_dir): + os.makedirs(model_dir) + + if model_data.startswith("file"): + _check_model(model_data) + shutil.copy2(os.path.join(model_dir, "model.tar.gz"), + os.path.join(model_dir, "model.tar.gz")) + elif model_data.startswith("s3"): + # get bucket name and object key + bucket_name = model_data.split("/")[2] + key = "/".join(model_data.split("/")[3:]) + + s3 = boto3.resource("s3") + try: + s3.Bucket(bucket_name).download_file( + key, os.path.join(model_dir, 'model.tar.gz')) + except botocore.exceptions.ClientError as e: + if e.response['Error']['Code'] == '404': + print("the object does not exist.") + else: + raise + + # untar the model + tar = tarfile.open(os.path.join(model_dir, 'model.tar.gz')) + tar.extractall(model_dir) + tar.close() + + return model_dir + + +def test(model_data): + # decompress the model.tar.gz file + model_dir = fetch_model(model_data) + + # load the model + net = model_fn(model_dir) + + # simulate some input data to test transform_fn + + data = { + "inputs": np.random.rand(16, 1, 28, 28).tolist() + } + + # encode numpy array to binary stream + serializer = sagemaker.serializers.JSONSerializer() + + jstr = serializer.serialize(data) + jstr = json.dumps(data) + + # "send" the bin_stream to the endpoint for inference + # inference container calls transform_fn to make an inference + # and get the response body for the caller + + content_type='application/json' + input_object = input_fn(jstr, content_type) + predictions = predict_fn(input_object, net) + res = output_fn(predictions, content_type) + print(res) + return + + +if __name__ == '__main__': + model_data = "s3://sagemaker-us-west-2-688520471316/mxnet/mnist/pytorch-training-2020-11-21-22-02-56-203/model.tar.gz" + test(model_data) + + diff --git a/frameworks/pytorch/code/test_train.py b/frameworks/pytorch/code/test_train.py new file mode 100644 index 00000000..61f9d844 --- /dev/null +++ b/frameworks/pytorch/code/test_train.py @@ -0,0 +1,60 @@ +from train import train, parse_args + +import sys +import os +import boto3 +import json + +dirname = os.path.dirname(os.path.abspath(__file__)) + +with open(os.path.join(dirname, "config.json"), "r") as f: + CONFIG = json.load(f) + +def download_from_s3(data_dir='/tmp/data', train=True): + """Download MNIST dataset and convert it to numpy array + + Args: + data_dir (str): directory to save the data + train (bool): download training set + + Returns: + tuple of images and labels as numpy arrays + """ + + if not os.path.exists(data_dir): + os.makedirs(data_dir) + + + if train: + images_file = "train-images-idx3-ubyte.gz" + labels_file = "train-labels-idx1-ubyte.gz" + else: + images_file = "t10k-images-idx3-ubyte.gz" + labels_file = "t10k-labels-idx1-ubyte.gz" + + # download objects + s3 = boto3.client('s3') + bucket = CONFIG["public_bucket"] + for obj in [images_file, labels_file]: + key = os.path.join("datasets/image/MNIST", obj) + dest = os.path.join(data_dir, obj) + if not os.path.exists(dest): + s3.download_file(bucket, key, dest) + return + +class Env: + def __init__(self): + # simulate container env + os.environ["SM_MODEL_DIR"] = "/tmp/model" + os.environ["SM_CHANNEL_TRAINING"]="/tmp/data" + os.environ["SM_CHANNEL_TESTING"]="/tmp/data" + os.environ["SM_HOSTS"] = '["algo-1"]' + os.environ["SM_CURRENT_HOST"]="algo-1" + os.environ["SM_NUM_GPUS"] = "0" + + +if __name__=='__main__': + Env() + args = parse_args() + train(args) + diff --git a/frameworks/pytorch/code/train.py b/frameworks/pytorch/code/train.py new file mode 100644 index 00000000..a964ac13 --- /dev/null +++ b/frameworks/pytorch/code/train.py @@ -0,0 +1,203 @@ +import argparse +import gzip +import json +import logging +import os +import sys + +import numpy as np +import torch +import torch.nn as nn +import torch.nn.functional as F +import torch.optim as optim +from torch.utils.data import Dataset, DataLoader + + +logger = logging.getLogger(__name__) +logger.setLevel(logging.DEBUG) +logger.addHandler(logging.StreamHandler(sys.stdout)) + +# Based on https://github.com/pytorch/examples/blob/master/mnist/main.py +class Net(nn.Module): + def __init__(self): + super(Net, self).__init__() + self.conv1 = nn.Conv2d(1, 10, kernel_size=5) + self.conv2 = nn.Conv2d(10, 20, kernel_size=5) + self.conv2_drop = nn.Dropout2d() + self.fc1 = nn.Linear(320, 50) + self.fc2 = nn.Linear(50, 10) + + def forward(self, x): + x = F.relu(F.max_pool2d(self.conv1(x), 2)) + x = F.relu(F.max_pool2d(self.conv2_drop(self.conv2(x)), 2)) + x = x.view(-1, 320) + x = F.relu(self.fc1(x)) + x = F.dropout(x, training=self.training) + x = self.fc2(x) + return F.log_softmax(x, dim=1) + + +# Decode binary data from SM_CHANNEL_TRAINING +# Decode and preprocess data +# Create map dataset + + +def normalize(x, axis): + eps = np.finfo(float).eps + mean = np.mean(x, axis=axis, keepdims=True) + # avoid division by zero + std = np.std(x, axis=axis, keepdims=True) + eps + return (x - mean) / std + +def convert_to_tensor(data_dir, images_file, labels_file): + """Byte string to torch tensor + """ + with gzip.open(os.path.join(data_dir, images_file), 'rb') as f: + images = np.frombuffer(f.read(), + np.uint8, offset=16).reshape(-1, 28, 28).astype(np.float32) + + with gzip.open(os.path.join(data_dir, labels_file), 'rb') as f: + labels = np.frombuffer(f.read(), np.uint8, offset=8).astype( + np.int64) + + # normalize the images + images = normalize(images, axis=(1,2)) + + # add channel dimension (depth-major) + images = np.expand_dims(images, axis=1) + + # to torch tensor + images = torch.tensor(images, dtype=torch.float32) + labels = torch.tensor(labels, dtype=torch.int64) + return images, labels + + +class MNIST(Dataset): + def __init__(self, data_dir, train=True): + + if train: + images_file="train-images-idx3-ubyte.gz" + labels_file="train-labels-idx1-ubyte.gz" + else: + images_file="t10k-images-idx3-ubyte.gz" + labels_file="t10k-labels-idx1-ubyte.gz" + + self.images, self.labels = convert_to_tensor( + data_dir, images_file, labels_file) + + + def __len__(self): + return len(self.labels) + + def __getitem__(self, idx): + return self.images[idx], self.labels[idx] + + +def train(args): + use_cuda = args.num_gpus > 0 + device = torch.device("cuda" if use_cuda > 0 else "cpu") + + torch.manual_seed(args.seed) + if use_cuda: + torch.cuda.manual_seed(args.seed) + + train_loader = DataLoader(MNIST(args.train, train=True), + batch_size=args.batch_size, shuffle=True) + test_loader = DataLoader(MNIST(args.test, train=False), + batch_size=args.test_batch_size, shuffle=False) + + net = Net().to(device) + loss_fn = nn.CrossEntropyLoss() + optimizer = optim.Adam(net.parameters(), + betas=(args.beta_1, args.beta_2), + weight_decay=args.weight_decay) + + logger.info("Start training ...") + for epoch in range(1, args.epochs+1): + net.train() + for batch_idx, (imgs, labels) in enumerate(train_loader, 1): + imgs, labels = imgs.to(device), labels.to(device) + output = net(imgs) + loss = loss_fn(output, labels) + + optimizer.zero_grad() + loss.backward() + optimizer.step() + + if batch_idx % args.log_interval == 0: + print('Train Epoch: {} [{}/{} ({:.0f}%)] Loss: {:.6f}'.format( + epoch, batch_idx * len(imgs), len(train_loader.sampler), + 100. * batch_idx / len(train_loader), loss.item())) + + # test the model + test(net, test_loader, device) + + # save model checkpoint + save_model(net, args.model_dir) + return + +def test(model, test_loader, device): + model.eval() + test_loss = 0 + correct = 0 + with torch.no_grad(): + for imgs, labels in test_loader: + imgs, labels = imgs.to(device), labels.to(device) + output = model(imgs) + test_loss+=F.cross_entropy(output, labels, reduction='sum').item() + + pred = output.max(1, keepdim=True)[1] + correct+=pred.eq(labels.view_as(pred)).sum().item() + + test_loss /= len(test_loader.dataset) + logger.info('Test set: Average loss: {:.4f}, Accuracy: {}/{}, {})\n'.format( + test_loss, correct, len(test_loader.dataset), + 100.0 * correct / len(test_loader.dataset) + )) + return + +def save_model(model, model_dir): + logger.info('Saving the model') + path = os.path.join(model_dir, 'model.pth') + torch.save(model.cpu().state_dict(), path) + return + +def parse_args(): + parser = argparse.ArgumentParser() + + # Data and model checkpoints directories + parser.add_argument('--batch-size', type=int, default=64, metavar='N', + help='input batch size for training (default: 64)') + parser.add_argument('--test-batch-size', type=int, default=1000, metavar='N', + help='input batch size for testing (default: 1000)') + parser.add_argument('--epochs', type=int, default=1, metavar='N', + help='number of epochs to train (default: 1)') + parser.add_argument('--learning-rate', type=float, default=0.001, metavar='LR', + help='learning rate (default: 0.01)') + parser.add_argument('--beta_1', type=float, default=0.9, metavar='BETA1', + help='beta1 (default: 0.9)') + parser.add_argument('--beta_2', type=float, default=0.999, metavar='BETA2', + help='beta2 (default: 0.999)') + parser.add_argument('--weight-decay', type=float, default=1e-4, metavar='WD', + help='L2 weight decay (default: 1e-4)') + parser.add_argument('--seed', type=int, default=1, metavar='S', + help='random seed (default: 1)') + parser.add_argument('--log-interval', type=int, default=100, metavar='N', + help='how many batches to wait before logging training status') + parser.add_argument('--backend', type=str, default=None, + help='backend for distributed training (tcp, gloo on cpu and gloo, nccl on gpu)') + + # Container environment + parser.add_argument('--hosts', type=list, default=json.loads(os.environ['SM_HOSTS'])) + parser.add_argument('--current-host', type=str, default=os.environ['SM_CURRENT_HOST']) + parser.add_argument('--model-dir', type=str, default=os.environ['SM_MODEL_DIR']) + parser.add_argument('--train', type=str, default=os.environ['SM_CHANNEL_TRAINING']) + parser.add_argument('--test', type=str, default=os.environ['SM_CHANNEL_TESTING']) + parser.add_argument('--num-gpus', type=int, default=os.environ['SM_NUM_GPUS']) + + return parser.parse_args() + + +if __name__ == "__main__": + args = parse_args() + train(args) diff --git a/frameworks/pytorch/get_started_mnist_deploy.ipynb b/frameworks/pytorch/get_started_mnist_deploy.ipynb new file mode 100644 index 00000000..d929363a --- /dev/null +++ b/frameworks/pytorch/get_started_mnist_deploy.ipynb @@ -0,0 +1,601 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Deploy a Trained PyTorch Model\n", + "\n", + "In this notebook, we walk through the process of deploying a trained model to a SageMaker endpoint. If you recently ran [the notebook for training](get_started_mnist_deploy.ipynb) with %store% magic, the `model_data` can be restored. Otherwise, we retrieve the \n", + "model artifact from a public S3 bucket." + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "no stored variable or alias pt_mnist_model_data\n" + ] + } + ], + "source": [ + "# setups\n", + "\n", + "import os\n", + "import json\n", + "\n", + "import boto3\n", + "import sagemaker\n", + "from sagemaker.pytorch import PyTorchModel\n", + "from sagemaker import get_execution_role, Session\n", + "\n", + "# Get global config\n", + "with open('code/config.json', 'r') as f:\n", + " CONFIG=json.load(f)\n", + "\n", + "sess = Session()\n", + "\n", + "role = get_execution_role()\n", + "\n", + "%store -r pt_mnist_model_data\n", + "\n", + "try: \n", + " pt_mnist_model_data\n", + "except NameError:\n", + " import json\n", + " \n", + " # copy a pretrained model from a public public to your default bucket\n", + " s3 = boto3.client('s3')\n", + " bucket = CONFIG['public_bucket']\n", + " key = 'datasets/image/MNIST/model/pytorch-training-2020-11-21-22-02-56-203/model.tar.gz'\n", + " s3.download_file(bucket, key, 'model.tar.gz')\n", + " \n", + " # upload to default bucket\n", + " pt_mnist_model_data = sess.upload_data(\n", + " path='model.tar.gz',bucket=sess.default_bucket(),key_prefix='model/pytorch')\n", + " " + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "s3://sagemaker-us-west-2-688520471316/model/pytorch/model.tar.gz\n" + ] + } + ], + "source": [ + "print(pt_mnist_model_data)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## PyTorch Model Object\n", + "\n", + "The `PyTorchModel` class allows you to define an environment for making inference using your\n", + "model artifact. Like `PyTorch` class we discussed \n", + "[in this notebook for training an PyTorch model](\n", + "get_started_mnist_train.ipynb), it is high level API used to set up a docker image for your model hosting service.\n", + "\n", + "Once it is properly configured, it can be used to create a SageMaker\n", + "endpoint on an EC2 instance. The SageMaker endpoint is a containerized environment that uses your trained model \n", + "to make inference on incoming data via RESTful API calls. \n", + "\n", + "Some common parameters used to initiate the `PyTorchModel` class are:\n", + "- entry_point: A user defined python file to be used by the inference image as handlers of incoming requests\n", + "- source_dir: The directory of the `entry_point`\n", + "- role: An IAM role to make AWS service requests\n", + "- model_data: the S3 location of the compressed model artifact. It can be a path to a local file if the endpoint \n", + "is to be deployed on the SageMaker instance you are using to run this notebook (local mode)\n", + "- framework_version: version of the PyTorch package to be used\n", + "- py_version: python version to be used\n", + "\n", + "We elaborate on the `entry_point` below.\n", + "\n" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": {}, + "outputs": [], + "source": [ + "model = PyTorchModel(\n", + " entry_point='inference.py',\n", + " source_dir='code',\n", + " role=role,\n", + " model_data=pt_mnist_model_data,\n", + " framework_version='1.5.0',\n", + " py_version='py3'\n", + ")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Entry Point for the Inference Image\n", + "\n", + "Your model artifacts pointed by `model_data` is pulled by the `PyTorchModel` and it is decompressed and saved in\n", + "in the docker image it defines. They become regular model checkpoint files that you would produce outside SageMaker. This means in order to use your trained model for serving, \n", + "you need to tell `PyTorchModel` class how to a recover a PyTorch model from the static checkpoint.\n", + "\n", + "Also, the deployed endpoint interacts with RESTful API calls, you need to tell it how to parse an incoming \n", + "request to your model. \n", + "\n", + "These two instructions needs to be defined as two functions in the python file pointed by `entry_point`.\n", + "\n", + "By convention, we name this entry point file `inference.py` and we put it in the `code` directory.\n", + "\n", + "To tell the inference image how to load the model checkpoint, you need to implement a function called \n", + "`model_fn`. This function takes one positional argument \n", + "\n", + "- `model_dir`: the directory of the static model checkpoints in the inference image.\n", + "\n", + "The return of `model_fn` is an PyTorch model. In this example, the `model_fn`\n", + "looks like:\n", + "\n", + "```python\n", + "def model_fn(model_dir):\n", + " model = Net().to(device)\n", + " model.eval() \n", + " return model\n", + "```\n", + "\n", + "Next, you need to tell the hosting service how to handle the incoming data. This includes:\n", + "\n", + "* How to parse the incoming request\n", + "* How to use the trained model to make inference\n", + "* How to return the prediction to the caller of the service\n", + "\n", + "\n", + "You do it by implementing 3 functions:\n", + "\n", + "#### `input_fn` function\n", + "\n", + "The SageMaker PyTorch model server will invoke an `input_fn` function in your inferece\n", + "entry point. This function handles data decoding. The `input_fn` have the following signature:\n", + "```python\n", + "def input_fn(request_body, request_content_type)\n", + "```\n", + "The two positional arguments are:\n", + "- `request_body`: the payload of the incoming request\n", + "- `request_content_type`: the content type of the incoming request\n", + "\n", + "The return of `input_fn` is an object that can be passed to `predict_fn`\n", + "\n", + "In this example, the `input_fn` looks like:\n", + "```python\n", + "def input_fn(request_body, request_content_type):\n", + " assert request_content_type=='application/json'\n", + " data = json.loads(request_body)['inputs']\n", + " data = torch.tensor(data, dtype=torch.float32, device=device)\n", + " return data\n", + "```\n", + "It requires the request payload is encoded as a json string and\n", + "it assumes the decoded payload contains a key `inputs`\n", + "that maps to the input data to be consumed by the model.\n", + "\n", + "\n", + "\n", + "#### `predict_fn` \n", + "After the inference request has been deserialzed by `input_fn`, the SageMaker PyTorch\n", + "model server invokes `predict_fn` on the return value of `input_fn`.\n", + "\n", + "The `predict_fn` function has the following signature:\n", + "```python\n", + "def predict_fn(input_object, model)\n", + "```\n", + "The two positional arguments are:\n", + "- `input_object`: the return value from `input_fn`\n", + "- `model`: the return value from `model_fn`\n", + "\n", + "The return of `predict_fn` is the first argument to be passed to `output_fn`\n", + "\n", + "In this example, the `predict_fn` function looks like\n", + "\n", + "```python\n", + "def predict_fn(input_object, model):\n", + " with torch.no_grad():\n", + " prediction = model(input_object)\n", + " return prediction\n", + "```\n", + "\n", + "Note that we directly feed the return of `input_fn` to `predict_fn`.\n", + "This means you should invoke the SageMaker PyTorch model server with data that\n", + "can be readily consumed by the model, i.e. normalized and has batch and channel dimension. \n", + "\n", + "\n", + "#### `output_fn` \n", + "After invoking `predict_fn`, the model server invokes `output_fn` for data post-process.\n", + "The `output_fn` has the following signature:\n", + "\n", + "```python\n", + "def output_fn(prediction, content_type)\n", + "```\n", + "\n", + "The two positional arguments are:\n", + "- `prediction`: the return value from `predict_fn`\n", + "- `content_type`: the content type of the response\n", + "\n", + "The return of `output_fn` should be a byte array of data serialized to `content_type`.\n", + "\n", + "In this exampe, the `output_fn` function looks like\n", + "\n", + "```python\n", + "def output_fn(predictions, content_type):\n", + " assert content_type == 'application/json'\n", + " res = predictions.cpu().numpy().tolist()\n", + " return json.dumps(res)\n", + "```\n", + "\n", + "After the inference, the function uses `content_type` to encode the \n", + "prediction into the content type of the response. In this example,\n", + "the function requires the caller of the service to accept json string. \n", + "\n", + "For more info on handler functions, check the [SageMaker Python SDK document](https://sagemaker.readthedocs.io/en/stable/frameworks/pytorch/using_pytorch.html#process-model-output)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Execute the inference container\n", + "Once the `PyTorchModel` class is initiated, we can call its `deploy` method to run the container for the hosting\n", + "service. Some common parameters needed to call `deploy` methods are:\n", + "\n", + "- initial_instance_count: the number of SageMaker instances to be used to run the hosting service.\n", + "- instance_type: the type of SageMaker instance to run the hosting service. Set it to `local` if you want run the hosting service on the local SageMaker instance. Local mode are typically used for debugging. \n", + "- serializer: A python callable used to serialize (encode) the request data.\n", + "- deserializer: A python callable used to deserialize (decode) the response data.\n", + "\n", + "Commonly used serializers and deserialzers are implemented in `sagemaker.serializers` and `sagemaker.deserializer`\n", + "submodules of the SageMaker Python SDK. \n", + "\n", + "Since in the `transform_fn` we declared that the incoming requests are json-encoded, we need use a json serializer,\n", + "to encode the incoming data into a json string. Also, we declared the return content type to be json string, we\n", + "need to use a json deserializer to parse the response into a an (in this case, an \n", + "integer represeting the predicted hand-written digit). \n", + "\n", + " Note: local mode is not supported in SageMaker Studio " + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "---------------!" + ] + } + ], + "source": [ + "from sagemaker.serializers import JSONSerializer\n", + "from sagemaker.deserializers import JSONDeserializer\n", + "\n", + "# set local_mode to False if you want to deploy on a remote\n", + "# SageMaker instance\n", + "\n", + "local_mode=False\n", + "\n", + "if local_mode:\n", + " instance_type='local'\n", + "else:\n", + " instance_type='ml.c4.xlarge'\n", + "\n", + "predictor = model.deploy(\n", + " initial_instance_count=1,\n", + " instance_type=instance_type,\n", + " serializer=JSONSerializer(),\n", + " deserializer=JSONDeserializer()\n", + " )" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The `predictor` we get above can be used to make prediction requests agaist a SageMaker endpoint. For more\n", + "information, check [the api reference for SageMaker Predictor](\n", + "https://sagemaker.readthedocs.io/en/stable/api/inference/predictors.html#sagemaker.predictor.Predictor)\n", + "\n", + "Now, let's test the endpoint with some dummy data. " + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "metadata": {}, + "outputs": [], + "source": [ + "import random\n", + "import numpy as np\n", + "\n", + "dummy_data = {\n", + " 'inputs': np.random.rand(16, 1, 28, 28).tolist()\n", + "}" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "In `transform_fn`, we declared that the parsed data is a python dictionary with a key `inputs` and its value should \n", + "be a 1D array of length 784. Hence, the definition of `dummy_data`. " + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "metadata": {}, + "outputs": [], + "source": [ + "res = predictor.predict(dummy_data)" + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Predictions: [[-2.365556001663208, -2.2061409950256348, -2.3553152084350586, -2.2513961791992188, -2.3498764038085938, -2.3511288166046143, -2.1809515953063965, -2.376275062561035, -2.4135451316833496, -2.2082533836364746], [-2.356170892715454, -2.193467378616333, -2.3679440021514893, -2.2636070251464844, -2.3376224040985107, -2.347769260406494, -2.1841466426849365, -2.388836622238159, -2.4109153747558594, -2.208566665649414], [-2.3748226165771484, -2.1906440258026123, -2.371753692626953, -2.260221242904663, -2.3479442596435547, -2.3560948371887207, -2.1920952796936035, -2.3820199966430664, -2.387697458267212, -2.1960866451263428], [-2.35616397857666, -2.201260566711426, -2.369040012359619, -2.24025297164917, -2.339461088180542, -2.350977897644043, -2.185832977294922, -2.4039688110351562, -2.405621290206909, -2.2077877521514893], [-2.3606762886047363, -2.204914093017578, -2.3792786598205566, -2.2498393058776855, -2.3379690647125244, -2.3598997592926025, -2.1787753105163574, -2.378344774246216, -2.4168739318847656, -2.195408582687378], [-2.352180004119873, -2.1964755058288574, -2.374354600906372, -2.2453770637512207, -2.3465771675109863, -2.3567099571228027, -2.186619281768799, -2.3881094455718994, -2.4104809761047363, -2.2037932872772217], [-2.3627235889434814, -2.2082910537719727, -2.3625595569610596, -2.236499309539795, -2.343416929244995, -2.350337028503418, -2.185572862625122, -2.3872437477111816, -2.413639783859253, -2.2089436054229736], [-2.3638532161712646, -2.202801465988159, -2.356856346130371, -2.2474398612976074, -2.359022855758667, -2.3567655086517334, -2.1831603050231934, -2.383793354034424, -2.401374340057373, -2.204136610031128], [-2.3584203720092773, -2.203669309616089, -2.356856107711792, -2.242065191268921, -2.3388164043426514, -2.353959321975708, -2.1987273693084717, -2.3955893516540527, -2.4062514305114746, -2.203401803970337], [-2.374253034591675, -2.2048423290252686, -2.37831974029541, -2.2443180084228516, -2.3376474380493164, -2.35740327835083, -2.1803739070892334, -2.3850717544555664, -2.4088034629821777, -2.1917836666107178], [-2.361870288848877, -2.2027125358581543, -2.373116970062256, -2.2477071285247803, -2.341639518737793, -2.3631412982940674, -2.184450149536133, -2.37685227394104, -2.4156301021575928, -2.1943106651306152], [-2.3658766746520996, -2.1951119899749756, -2.372037887573242, -2.2629857063293457, -2.332758903503418, -2.352918863296509, -2.1863441467285156, -2.390299081802368, -2.404448986053467, -2.197378396987915], [-2.364487886428833, -2.1919915676116943, -2.3680667877197266, -2.2561895847320557, -2.3555908203125, -2.353691816329956, -2.1823601722717285, -2.389714479446411, -2.4032039642333984, -2.196568489074707], [-2.3647568225860596, -2.198798894882202, -2.367218255996704, -2.2393057346343994, -2.353621006011963, -2.3591856956481934, -2.1855597496032715, -2.393371105194092, -2.4103264808654785, -2.191272258758545], [-2.357618808746338, -2.1956608295440674, -2.3811066150665283, -2.237548828125, -2.352346658706665, -2.3478593826293945, -2.184941053390503, -2.4012489318847656, -2.4046220779418945, -2.2000842094421387], [-2.363558769226074, -2.1945061683654785, -2.3820648193359375, -2.2576940059661865, -2.3389761447906494, -2.3584983348846436, -2.1834285259246826, -2.385425329208374, -2.4102983474731445, -2.188638687133789]]\n" + ] + } + ], + "source": [ + "print(\"Predictions:\", res)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "If the input data does not look exactly like `dummy_data`, the endpoint will raise an exception. This is because \n", + "of the stringent way we defined the `transform_fn`. Let's test the following example." + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "metadata": {}, + "outputs": [], + "source": [ + "dummy_data = [random.random() for _ in range(784)]" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "When the `dummy_data` is parsed in `transform_fn`, it does not have an `inputs` field, so `transform_fn` will crush. " + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# uncomment the following line to make inference on incorrectly formated input data\n", + "# res = predictor.predict(dummy_data)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Now, let's use real MNIST test to test the endpoint. We use helper functions defined in `code.utils` to \n", + "download MNIST data set and normalize the input data." + ] + }, + { + "cell_type": "code", + "execution_count": 16, + "metadata": {}, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAA6EAAABRCAYAAAAjIaCuAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nO2dd3wUZf7H35NNBRJI6CWQQBKaCqI0ATlFpUgRGwiKeooigjTbef487+wVECwURbzDcgciYEPhTkGlSe+9E0AQCBgIKfP747uTQtom2ezODt/365VXNrMzk+ezz7Mz8zzfZpimiaIoiqIoiqIoiqL4giB/N0BRFEVRFEVRFEW5eNBJqKIoiqIoiqIoiuIzdBKqKIqiKIqiKIqi+AydhCqKoiiKoiiKoig+QyehiqIoiqIoiqIois/QSaiiKIqiKIqiKIriM8o0CTUMo5thGFsNw9hhGMaT3mqUnXC6RtUX+Dhdo9P1gfM1qr7Ax+kaVV/g43SNTtcHztfodH0lxjTNUv0ALmAn0BAIBdYCzUp7Pjv+OF2j6gv8H6drdLq+i0Gj6gv8H6drVH2B/+N0jU7XdzFodLq+0vyUxRLaBthhmuYu0zTPA58CfcpwPjvidI2qL/Bxukan6wPna1R9gY/TNaq+wMfpGp2uD5yv0en6SkxwGY6tC+zP9fcBoG1RB4QaYWY4FcvwL31LOBXJJJ0oI8Y8zYljwGiK0Oh0fRBYGnPrc286DXxU1DGBpA90jBaE0zU6XR8Elka9zuTH6fogsDTqGM2P0/WB8zU6XR8EnsbcnObEMdM0qxe1T1kmoUYB28x8OxnGA8ADAOFUoK3RpQz/0rccMQ9wnMM0M65kgTlzr3tzHo1O1weBqzG3PoAF5sxjOEgf6Bi1cLpGp+uDwNWo1xnB6fogcDXqGBWcrg+cr9Hp+iCwNeYml8ZCKYs77gEgNtff9YBDF+5kmuZk0zSvNE3zyhDCyvDvfE8YEZzjbO5N+TQ6XR8ErsYC9IXiIH2gY9TC6Rqdrg8CV6NeZwSn64PA1ahjVHC6PnC+Rqfrg8DWWFLKYgldASQahhEPHAT6AwO80iqbEEU0ZznDWfMPEMuvozReTPrCiACIAeb6uVle5WLqQxyoD+yp8cTd7QFY/tK7ABzL/IM7E64FwExLK9G57KjPm9jtOpPyTSMAlrSYBUDSokEAxPdfV+pz+rsPjZBQAEIXxDAr4SsAugx9CICIOcvLfH5/6ytv7DZGy4OLqQ9xoD5wvkan6ysNpbaEmqaZAQwD5gObgX+bprnRWw2zA0FGEI1pyWoWAzTHYRovJn1LmA/wu5P0wcXVhzhQHzhf48WkT68zgcnFpE/HaGDidH3gfI1O11caymIJxTTNr4GvvdQWW1LNqE01arPAnLnBNM0X/N0eb3Ox6ANYYM487OfmlAsXSx86VR/YR2P6dVcAUOfPu+RvMxOA3o+OITJtaanPaxd95YWdrjMD6v8K5PSdt/BnHx4eIrGMyxLGk+XelhFRpjLn+dAxGvjYrQ+brpRH7HG15Tv5UUo1ZjSpV+rz2U1feeB0jU7XV1K8exVXFEVRFEVRFEVRlCIokyVUUbxFUMtm7OteBYAX/iyZ42+qeAaA+C8eIGlo2eN+FEUpnKDISOKf3wrAe7E/ArDSHf4ZvWgPGf5qmBdI7StZ8Be/PSl7W6eHHwSgwuxlfmlTebD7xfY8UOUt91+yxhy+opL/GlRGgipKaYLJo8dnb7t08X0AxH9aesu8nXBFRQGwe9QlANS/eh9fNpmTZ58QwwWIdbvlxOEAxL4u1jUz/byvmqp4gKtKZYyYaADG1f4CgExT7PehRiZBkZEAZJ0+7Z8GKoqN0Emo4hdc1aoCcGBQYwCmDh/PFaGuPPtkuhNXL+75JvcN7ejT9inKxUJQhQoA7Joaz9zYaQCMONQBgJ0PJ8pOyev90jZvcejq/BXFrG0Js3O27RjbDoA6i+TiEygTVFe0PPRO7f8uQe7JZ59tvQCoN10WFrzrnFv+uKpU5sQn1QBoIXmJ2Hw+ixr/ifBjq7yDESYZL/c+eQVj73ofgGsiFma/n3XB/ummtT2LVcNkQt780sEANBqwpnwbW0aCY8X9dPurcs+f1nYag1dJsixWywQ87r0tAGQe/933DfQyW//WlG23v+P+K6+z4W2VjvN6f1lsqDpliY9bVr6cvakNAIvemZwvFKDT2n5U7rHDH80CILiBFPLYeV892t+wAYCp7oXWLEyePNwagGUvye+KMwPjuu8E1B1XURRFURRFURRF8RmOsIS6GiewbXC1At8b3HUh3yQ3B+DAulrZ2+v9V1Zqwr5eUf4NVPIQXK8uzeZKaaR5NSe6t7oK3T/cMHA1SwIgc9O28m6ecgGn7hTrUI3BewBIrHSUb2e2y7NPuz7rWDrnsmLPVXvJOQBcP6zybiOVUnO6+6UAbOj4Trb77ca/S1+GL3eGG/zOfu8V+p5l/ezQbhPzG7j36ye/us5uWd5N8wrb/9IEgPZhC7K3Hf53AwCqHwtMi0tG0zh+uGxqnm19fxhK0qzAt1JktJdnkjUPjC/w/c3nxRa6/Fw8AOGGuNz2i0zO3mfoZYsAWFAllsyTp8qtrWXBVaUyB26uD8B/O7wKQG1XBda3l5AbpBIUiY3vB6DJsAwyU1J83k5vcHToVQAsu/U1oHBrfaXbpA9ds8UynHnseLm3rTxxNRdvtonjJAwg3Qwh6wJb/o8tPqE3rX3eNot948QFem2bt7K3ZbltcFlk8WItuaZsff0XAG7q+TCJ96z0cSu9hxEsUzvL4xBg9/1Suqv+tXs9Ose+hXL/qP9q+br9qyVUURRFURRFURRF8RkBYwk910v8zffdCGt65l09dLGUCEOCRlyGzKutQHCA26NWA3DtwREA7LhhChl3iCW077Y+sv81h8qx9Z7japzAnhfCAdh01b88Oqbhd5KoYdG18rmMP9aJDVdJzEnWuXPl0Mqy0ear3TxdbUOebUmfDSXxCbGO7X5WykRsuudtAKKDInhgjlQCmtxeEoz4e/UwOF5WiXa8XJmYORWK3f/w1VkY5yUG7fHrvgRgcOX9+fazxm/D2Q+S+LBvV/xd1auTemUcAGHf5HgIRN57EID/JMzL3vbisLxtCyKIrGH/K/Z//HiffFZ//8ufqfw/iRHxd18WhlWuZG/PEABW3TqWSoZ8r6x+arTwXgAaj9wbcPFM57vJyvT0N99wb4ngoTck6UmNL3/xU6t8R1HW0UF7r3a/srdVxoq3+/DWt7O3zTgtZTiqvxuYFlCL3X0rZMe3ppnpANT+NsSfTfIawX+Inh3pGSSE5H8Me2SUfA8jvhBPBCNEnm8+nt+WOY0l2c3D0RLrO/2ebtQaZ8/v6+4Rzbn9Zom9u3egaDpbM5TTseL5dP/9XwGwvYtYvJtPvZsGtwdW/Ln1HXxptMT2RgcVHbO8sPnnAHS8figAUZ/Y8/7nKakNJK63cUjh3mz+4rchYmpf1tqaM+TkBrh2uPvzX3GQI10lZnTeM68BsPn697i+z8MARMwJPG8go0kCAF/Mn1H6kzR2/6ojn1P85xkEL/S+ddg2k9CMLldwqJM84E0bNAGASUf/xK3V5GG4Y7jcUMONYIKR/dqt7g/AsSNRhB6Wm1OUO/a5+hdbck7unpAmnZJJTu/4m5m7WC4E85JkQtCDVuWiqzis7H+HBrcAYNSQmdwVKSW8rMQ8RzNTOed+fe13owAIPSpdlxUM3/Z/HRA3F4CXa66kV0QXOcCGk9AHopcD0tYpp+TLn/R/68lym/urbM1/TK8K8jA4JdKd6dHPE5fMKtKODR2ngTtnUpD7ApeFWeAx1sPhqjMygR2VWgeABXsa07aeuEhYwfI9266igI+hXNk5sTaLrxoHwPWvPwZArQnLOJ0Wlm/fvRnSV+dMufG4yCCT/MlfLOq4ZNGns/v+/N9xE3OyXPa330142+TWrOouN66ooHD31pzPwVrk2natPHgkvjGYxHsCZxJ6vltrpk8eC0Btl3TKhBOJ1Jzsdr3xW8v8izX5PNLe3pNPAIJcbBklD8BtwnJ67KNhkpAohMB1JwPYNHBitltf22Vyraj3mTMy4porZKL1yH3DSK0hzy4nG8uEu1239UQu2weQnZHacoVLe6MOTM57rlb913NoXPm3uTTEj93AipmSiCdooxgDKrp/AD4+3B2APq/IM0xCjWOk+7qRZWT7w/IMc33E2RIdV33IHgAyf6hFRnIAlm015H5f4bGDxe7acfVAYvB9KFWzQZsBcBk5zybZzx2fy0J6BlB16gEAeiHPPf/5v9eYNF6+VCOPDJEDl67zRZNLTVCLpgA0mrqLKiEbvXberTdJkq0k10M03SvhAZk7dnvt/OqOqyiKoiiKoiiKovgMv1tCT30tZuPZl4ynhstya5RVi9ZuqxDAlb+K21tKSgSN3pNV36q/yipHTFpavvMWlI7eqs+UOTVnre3xw1e6X12YFL38cSU25MY5YukdUnkxAGlmBq8cl0Qh/5wl1sz4jw+TuX0XAEnI/lZZhaOf1aNRcF73j+NZZ7Otv3Zn+9maAJgZgVWF0Ni8E4DuAwdz+7vfAjD9b70BiNqWQkqijLWo7Tm1wIz9bgv3BW6bsWxgw5dJ1h8AzFvVMruvfcX5E+FUDhK3r+WPixWwadIw4l+Xvhn1SicAxtZZTPfPxwCQMMozy8ThkZK0ocPAVdnnGNRM3FwWE17ocb5m2zvi9r+xx0TCjLztarOqf/br5a0+zfPehuvepd2YkQDUfsOernG5GTVhRrYFdHeGeEvMv78jpNt7tbekWEmHIH8ZC6tOaHapllFLsbv7bW4OP9KWLf0m5Nk2cPcNhK/ZAwReSRaL5DFXuV/lWHLjRkrincC6SxRP8MKVRLlfW78PPQuQt4ak5Y677/b8vbpkXxwNsKcLa2ZKCmws/DsV85NYoDadlwQq7zX8D/1vkXtLxQBIQHWuZxt+HPCa+6/8YTlv/t4QgCn/6QZA/J/2ZHvfzU6QEKOWdw2jzquBZwlNuUNCo35ImlDoPlaCrYgpVXzSJk+IL8JroOpU8bjsl/EYP78gSTMjX5NQvT962LO+q9lBkuf1nfodAPdG5Q/z8gbber3L3GulFNg7D90G4BX3XLWEKoqiKIqiKIqiKD7D75bQrnXFmpljBc2h9co7qP6ixGHVXCkxnjVypQkuadzSqRslLfrixu9wxhTr6eYbrBUa38VzuRJldezGOSsYUlliAbekS3v6/Gc0jR6T1Zj6iEUl99qnFQR/aKLEJP7a6uN852/3+RgST9o3dqb3uj+z9HKxJL1aS2LQEsYNoekTW4o6zFZYCZ9cP6xiVtMaAFRCPvMsoJLb8OKJPTr9hiv58BJZTZz1h5QRavroVp9bMpqM3kCzc5I8YtNt0p7NN01kWXeJWRqy6k4Abro3jtBbS7Z+VfGwfBIPV5fkRfNTq/L+yg4AJNkgdu3AX8T6sqWPWICDCeaWHRKvdPQdiYOo9u+clfmelw4AoPE0CUJ/o9ZyVoyWY3stlSLyxs/2KyJ/8i5J1NAy7CesMgI9lzwEQLzNY15KQ0HJhywLaIXZ0p8Js33apLITJHHY0T1ykulZ1uwzN9k30ZennInLufItS5Nrj3m+fMoDBApBUXK/33zdpOxtlpUp9m2/P8aVmvRYsYDmxFNW4FyM3FsqFnKMnUju6Crw2dVi4XWJANQ/LM9yWXObw5d594ntvofMV8utieXGFaNWF7vPI9vEe8hfyX2CDPmOBOWyt+3qKyOrYRGPyNEfLuH6AbcA8H3zWQC07zcMyLGW2oXdfeQ+XhoL6OZ08Qr998n85XPaV9oOwA0Rf2Rv613xBABVJ08DYNSG26kz5CRAqeOa/X71WnFDXQB6Vbsk33s1tu3ODsgvS6KM0/3FJWuGO/gdKvCPI/IA7LOMlkEuuLIZADd++AMAQyrv5f0UmVTO7icJMRqtyz/AgyIjSekmx771qkwOWoYW3nVNX95ja7elanckM36JuGGPiJaH+B293+Pq+FsBOLbp4kqLcrhtKElud6tei+TCl5ji+zqaWampJD4hN5bOK2Qy2vXxxdwXLQ/rq9vLheeWib2p+VpksedzVa8OwP7J1ZlwmWTTSDflZjB2+ECSvrVPjd4RgyTjZLC7Xu3lKwYSO1zcbiL3579bZa2TBZOtneQB5PVljXk0RlJJ7eklN4X4n8u3zSUhuJa4vT/0V7mh1nZFMOuM1FZu9KJcYwPDgb/sWJPPQOXAE+IGt6Z5jhvc4lSpAefpBNQK50i7qmn2ttCTshBq/rqhwGPKG2tx9r3uH2RvG/OcLJDEHLHXg5+v2T+1Vr5tM09JKFHQj8VPBuzK2Zr5QzEijtn/SrT/aVm0XHHnG+ROWGeRMP8BAJIO/1rsubrX2Mi3sRKClbH/gPcaWY6cuKc9Y2pYbsj59VtkTKvpfuW9RDYl4ecl8tx8ot43AFQOCmXDnVIrdPDVEu624ttLiP1OJlrGkrXZx1a4XSZcz/8oNbOjt9ovySfA9jvfBSC9FI/ND24eCEBU95353lvSRd5zTZIqHV0iUrPf6xAuk9flV86gTV93Rv13SjcJVXdcRVEURVEURVEUxWf43RKaeeSovLB+exlXdDQNHxGrRf1gWf09a55n8VuymhyNb1ZYXdGVmTd7ep5t76fUy7aAWpaV3KT1EBN5jad28WW8rHbsc5fGuGZDPwBSvqrN+6Mk0roo66idyDp9WpKgAC1miDvyn8LTWXTpTNnh0vzH/N9RCb7OOvKbT9roCzKvkbJAH/55PP87K2MzcZDvLaC5Md1JvirPEOvf0hkhfP7Y4wD8OlLcTWclzmXZBHGTu6eHuDbmrmn620Pi8tnkThnTs+Omc9lPkha9zjRZNQ21kRW0IDKXRZOxf3Ox+xnuEku1Q3yffr4k/H6NuBQPjPw6e9tfv5FrSMI6z1z3Dz0uq/9X3ZpjeVk+/XIAarxtv2RMkpQov0v0/EOyLaDKsZDjWTDkzq+yt32TKh4J/xl0vXtL4QlqgmvVZN+dYjHtNfAnAP5eI6feh+XS+9rhG9jX9o/8Jyhnjo0TL4RrIqQdb59sRMy00t2fg2uL5TA9ribBm/cAkHnyVNkb6WP2PSPfuXVtxOqdRY4b7k9/FQ+vMB8nsPMmB7rnt3pGrRGLip29uRKul0SRVt3o3Dx1tBVNR4sr44UhNebaLXQZLPfMhVPEtXpold28d9eNANR7MTAsoU0e3Eid4MItoG+daAJAlXXiaeivJGkJo+Xe1jZcShtu6ZNTT/n9+hIalPXAQg79WZ57eq2Uvqn3okFqdfFoGh4j98xPusv9ImFLdYwKhSdTNM/JuTLLaU5zIVa5uFRT5gafpCRwX+V9he6/8bx8s1482IOYwXKtLei7ZiUdmtBZLMaj/xxHr5vlPv98De+FUKklVFEURVEURVEURfEZgWE6KwWuqjEAHPuoKvMafAZASpbM+q9+61HqTPftyr1Zt0b267PuFYvJr/ehqjsG1NVYYiRPXVaV83eLL/q4ZrJS1i4MPj0jq+BjX7sdyAmOjqyVyukR1qqMndcOL8CdBOXVOyTBS+bHn9IlIn+pHYDkzFTWdJYEUlmpgWG18IRdd0tpiMtDg2i1XFJe12GTP5tUIHXeECtn32mSZj7lTwm8+KqMzc03SRrzvT3PZ8d71nRJMOT7J92pw68fSPx2sYqa6YGRYKRK58METSw+JfvOibUBGBg53yftKi2/NzfybWs8SeIHPVmlNtu3oNr1UpR8Yt2fsrcnP/E9APevfhgA45e1+Q+2KR81WCQv3Pl9utZp6b/GeEDCN5IAYkiVXdnbxnx+NwANVxRuMUzrLh41V738M19Uk1X9M1lyrX37ZLPs/WJDxGrxTr1FtB4ucT41J/jewp3ljk5+a2FXEvE8fjeoYkX2PNYCgFH9JMb73qj9dForFv/oWyWOKSs1teAT+Ikjj4i1M+SMBHXFfLAk2wL65X1WxpqcMmx3vStWnTpf2c/7oKTUis2bk+Nvv7WwtbeTESyPzOEuGUsuI8eOk5ol97b/TmhPzMlCvo9ZmVTcLBayf54Wa/09UUeJ7Ogbq1lZ2TbtCgDmN3g/+35/ISGGiynzbgAgfpM9YrmTPpJn/0uj72f+VWINrZertKFl1V3Z9kPZMCcnmVEWkq9jcG8pgdL4tmS6VhCvipx9ciz69+4R7cc7lIeS/Fz+piRMMsWRhM63rYQCLKHtVt0BQOjHMjeK+sQzD6iMg3KDjH3uEMtWuhMYTfGeJdRxk9CMa+VL8vtoeXBc2vJTTmRJ5rXua6XWaJ3XfH/x3jI0Kvt1qimPfZX6JZPWLw6AJxvOA3Jnicu5SN3/4a00nCIPHlWT836ptz/SkE7hATT5vABjrbgxTj18NV3ivy90v8wU50w+rYeOlV0kUdaRzCxqjrdPrcx8ZMl4tZKeVJx5nOeT7wFgx2C58m29fnL2hfhIpjzgfvtoZwBCNxWfnMHfnMjIm4tx0aUz6TLrZgBOz5SkabW+3geh4oa86WlJ6rO+w0T3EaHZxwal5Z/w+Zsrrikg83Ry8Q8+2ybJTWdBt7HUv6AeMUDn+fJA3HilLCoFckqx+YfW2HYiGnRJE56p+b77L7lWjDp0FUlvSThDQXcAKxPyfU/NAWRCNvcPqfP21mjJWhn+ZU7WyjO3dQWg97h3rFLdPsPVOIEvL/vQ/Zd8lyIOuzw6NqOL3PNHT5rBdRE/XPBuED+2+ASAm6v2Bfw8CXVnNt7znNQj3nDPRLggO/grI5szsLJMPq0H5SB3h7Rceg/1Xg38yefZm0T/vEvGureIzjUn65GVmuynVhVPRkdJUvNJ/FQAMnNd8Fr9IiEnDT4oeuKVsVu+s58fkXCcuyK/Jsu03z0jN64qlQFoUFeeAdLNzDwTr9w0++VuGj4rYUW2uR+4jR7xS2FoC8lev3W0jLlmDZKZlTi32FM8Em3P6g1WXXIr+eDYEfmvD88fu4zqfWUOYWZ4N3ToZCtZfKlRzH6Foe64iqIoiqIoiqIois9wjCXUVU3qTSW+Iunlx9fJWY1qO3MMAAmj/Fc7M/xQzkddNUhWYBY2/zzffu+n1GPsJzcB0PADWTGrf+CXQh1tn7p5VvbrHlt6AxB04pg3mlyuBIXLav6Jz6VEzVfx//Znc3xK3C2SDrtSkLiAdF5xJ7V/8G9CopJi1cBMtMqQHMx5r6ZLdO3tKav+ifb2VAVgwUPiO9P+I0ko0SEsK+f72dy9098KOlKsNmfN87SZNBqA+Dfks7FLoYHghnF0q/pTnm1tXhhOjVP5V+yPDBcr/ciHJUlYpwixVNQrIAlFuplJ3fmyjmkltLITCaOW0ogh+bal9pWkdIvfnlTQYbbkUJcYKgfl9ZZY+NUV1D+Yf9XbVVPWpPs89l8gp37cU0euZOOfxMU8PCXHAuqKFuto72dk/zNZadT41beJibY+WI3KQaF5ttX/6kSR36HzXaVEyfPvSnKlK8P8lf7Ecw48KWNv3T2S5K0gfY9VXY9V9sKyNs04LaXs6t2ysdzbWN6k33AlNz4nSWGig/J6V+yeH0897GsJ3dMztND3Ks2v5NE5rPJI1cPOZG/7/ZR44kSXoW3lSepVSQDMb/6Oe0vh9quzKeG2vB9YZK2VhIOJEslAOtAb8fhJHiP3v7FDJ9ElQq4nBZU+sZL7DJg6AgDDhJrLRXPIAv/UPd/yRHz2a+u68bvbK23ulM7UyCilB4Xbe+Pw8La06F9A+a4yWvHVEqooiqIoiqIoiqL4DEdYQs0OLWk5USxJz9UQK8TRTIn7uPaDx2nykQTW+jNyMn7cBu7v0Tnf9p2nJLYsc6qsXldZfoj6e2XFoqTt3XFIkhclnLN3mm9XVBStF0tswTPVciygaaYoPpApQf+N3PEwNV0RHB4lK1S1xgZuPIxVjH1s3EcA7M6QFaTo9yP91qaysuOfUqIjxFjDd6liqdmfLl4JW2+WVdM3/9SEH/vJfpmb7FnKJOgnuW48P1CWR9P/cZIFzWZ7fPyP56oQ+5yMTbtYQC0ydu3h2+NS9+iOyCMAZEYApizxWsk29j/Whr/dOwOATw9LzNZ7kyQuduZLr1PTJd9HaxX4vpdHUm2mPRJPFEZB3i8VZruT3byd762AouHUvQXeIzY/3wCAOVW/ybN90bh2VEnJ319bn2ks+8csAOCq1fcQY4MEU5bF4kLMDhK3O/Y96cCmoTlr6Zculri8p1p+C8DAyGT+lRIrx53xfdmZ3Lhq1mDEXV/k2XYkM41v/0jKt++4f4k3VIg7fDWQ73sXcu3rP/FozNY822acluefBrOO+K2chydE7ircblN9pSSrKe76f7KPxJV+GSv3x50ZZ0l4U77JtomhzE27yxgydqbHu8fOCVzblhVf+UjYg6x52CqLlL9HD2ZKjGzsC/b5Xia22J/9euYZySfzUWO59tWg5O20rrO7bpL7/uYBEwrcr8rqkBKfOzeBO1oURVEURVEURVGUgCOgLaFWHOiRJ1OzLaAWc87I6m79Z/PHUxrBwbjqSyxixq495d1MQLK7HmqXf3sEVvmH3dIeD8/3x60SW9Kj4pvkTt9uZ4zWYo3p/MHSfCuhKVnnaPOxxO5GHBEL4eoxknk0CIMaPd2rPGMJWLYMF0t1/WCJCUn6bCgACfP8F6tcWowrJFDyk44Sj/VdagVeHHEPAOGHZPk+/FOxRoyM2cT0W64HINamllALY4lYgCJuieLKQZL6PLKXxCgNjfuBvy4XC8W2a9/Pc9ywH+4kCftnAbYI+dMxXHMkhmT7A7Jqumlgzkpn30bSd1gVIojgqSMSg/fjeLmQVZtuTyto7pjPgrLdWu/DmnzvBRI7H6hPg2cO5tnmSohn2gVj85KfJCt83D+XElRRYs9+v0WsMe1HrGB2zbcAePukWOSqj8nyvTXKyCl3YLHztfY0eizvGDvf9Uri/i73jpZhEjeZbua0dmOnaXn2P5V1jvFTxJpf+4R/rBaWp8G2xxtyd9RXed7r+sHj1H82f7tiS2G5sDtn+4h3xeiYiViPnt229AEgZLg8w2Rus/f9oeqmc4W+t0BksC4AABVjSURBVHWkaGg8JJyscwXvd75ba1554b0820buug3z1wJi7WzCtkHh3FLJyjMi39EQw5UdK7k5XTzXho14BICIOcsvPEVAY8VVnnPrrRMcxnUR8tz+yL/EwyvhztV+aZu3sDLrHrq5EQBNB27mhqpSzmtgZOEx2ieyzhF8rmz2+4CchBqXywPwkX/I4F/e6tPs98a43ch+fVHStldkWXYNztSGEvYd9mgyB76sA0CtsXt80mZvc7itXAyq5grsj5tmT8O2lSjj8U/F1a+gkjLX/30MDd21Tw/85ap87//+b1k0qMb+fO8FAsH16rK875sA7MmQL23tX2zpfOMRW4fIRLqFO09D97sHEbZgBZDjUvTcp1Kfr899bzLvfpnN3H7kMQCqTVma7Q5qRzJTUqgx0f0g6K7CMo0GNK7udnW/YP7S8BP7agFY+2VTeTFMXC6XtvqEWV9JKEDOA0bhJH09hKQHJeQhOsuek0+LQ1fnJErYMVYmzLndcgtLSNTp4QepUIK6lL6k7vfHOPWoPNhaCYp69VzKpinuxdT9Mi4zd+xm/TlxweoQLin5600SdynjiuacfVGSofzUfGL2ua3J53c3ScmIzO07ylVLQTT8Io2s2/O6vf2332t0PfM4ABWS5fvVfvAq3qgjSbasOoWFlYoAuG3LAGq/6d8JXVBlKc+2sX/OIs+BDHcSk9NwrlebPPv/dlkwcdftyXsOQ/RnmQYHTokrYO4wjrDjcj471ukNaiHXnqfemA5AmBGcnUQxe/Jp88VJT9h+/RQAHv/5Ss5kiq5V78oiWKuH5IYxqNok2l2Q423LvlokWsWKbcjT18zN9x1LN3O+d7+kysTFaZNPizH7ewGQ/JzMI/7+zhTahsnc48uOEhIwst2Q7DIwgYRlTLj6Q+m70TFfl+j4a955jHoflO36Wuwk1DCMWOAjoBbi7j7ZNM3xhmHEAJ8BccAe4HbTNE+UqTV+4JyZykZWkMY5DAzqEk99I5F08zzrWcpZUomgAoBnRctsiCcagUTDMKK1D+2H0/WBjlHtQ/vj9D48m57C+sNfkWaedKQ+0DGqfWh/VJ+O0YsJTyyhGcAY0zRXGYYRCaw0DON74B5goWmaLxuG8STwJPBE+TVVSL/uCiZMlVXcJiGypHQ0M5WrPxYLS8K/pD9dcbJyeOrrBIY0XATA338QV7pq/4ql1jSZvadhkMhlRBnRZJjpLGchMWZNktlDDDWIM5qwx9zC7xytVd7aSsKwHjlJJ774owoA4ZtkNTyf+7EHGheYM09TTn14/AZZKesU/m32NisJUednJMV11Q+XExxXH4ARg/Imb1h8Lpga/5JVpoLWvD3R5+8+PPpexex09O0X/xmAhjM9s7rYTZ/ZvgUvXZ03UUFBackb/E2+Y5cnDmFDZ1kl/uVv4vrX9/NuZB47nr2vv8doeePvPmwwVUrP9LhOroFfN/miSAvo3XuuA2D3RAlraPz5asysop00A6EPxTJasBtudsKiQvBnH2Zu2sbVk+Uet3aIWNRerPkrj30hz2nffSHeI3ETNhJi/Jbn2BGTPgGgUchxkkLEdeFMlljOWn05kmb/2AeAeWg3iST5RV/I2l08c1TKJPyjhnhU1HZFsOaB8SU6z450ua/0nyihHfUmrc9zz7DbGJ02bByXhBZf4sByVc5jkWqd83LDeXneueXrO4g4GUerr+uQ+dMKv98raHMp8W/Ltef6iLOAJOKhi9tyX4pT+rMPQ1eJl8DwQ/J9m1AnvxXo1Vo5YRmuF8RrJNPM/+Sy8ryoT5qYnichkd/HqCHj8dCY9gBcW+FVrJJBubGs+ROni0t1XQ9dyP19L/SEqhtznqKnxX0HwOWtxaL/6ObbWNzyYwAS3HOQs7XC8wTF+bsPu1aQa/q2tfIRLkxuzKFtEg52V+eccm1Nw8Xy6Yk31JI0F09v7wtA2Isy54j9eXmZk2kV679pmmayaZqr3K9PA5uBukAfYLp7t+nATWVsi18IMyKIMsRNN9gIoQKRpHGW3zhEbSTLoPu3XUs4FYsnGoHjaB/aEqfrAx2joH1od5zeh07XB84fo8HRkYTFS6iR9mFg9qHq0zF6MVGimFDDMOKAy4FlQE3TNJNBJqqGYdQo5JgHgAcAwsXEXipcSWJNOzEqJdsCOvSgFJhf/0YLkpaLT/2mv0gzdtwowd8r0kwGfCkJYJIeKXql+6z5B6c5SWViOE8aYYasbYQZEWAW/Fl5S5+nWJ9DYlhOgoOnP74TgPqHi1+JKkwjUrO3fkHHlFXj783zb+v0nFhAq30gK4Xnerah0/Py+r4oWSXNcq+x/OX/HiDqD8+S99itD4NjJWbrw0umY60mhq0t/f+wg77fWlWkb6WjALzmLv1RFAkP7uSub7oB8M/4b4vZ2z9j1Jf4ow8zfxPr2LFZ7njrv+a8dyRTLBQ3P/MYVVeflI07ZCXV+t6VdLXTLn24s5/cBxoxJM/fuWn0mbyXgOcJwvzRh3HvSkKeCf0SARgevZ3Xask97bUh7nvbkPzHda9gJb8LZdwJd9H54VIuLOl/ywtMhudrfZknT7Ght1wrr50ipaz+L+Errok4U+gxye5x+6Q7ZmvTp02pM19KENXZVny5JF+N0d3Dmrhfzc/eVi9Y7gX1PDpD8VjW1I1uj5O17WH4fXdw+ucFVD7n+3uF5dW06d4I5teV79Uxd391+2IMiSX4rhVFSfuwrPqMqjKxuKJS2WPHH3xNnoFqrCj8uc2f19FfR1leCPmtoABDd/QHoO4rpY8JtMPzTEFEzFnO5vES99k0RGLqLQ8UISdBE8DZGFeh6UH90YdW3oCnq63L+V38o1o2uzPOceOnj+bZVndRBhFfrcizzRuZMDyehBqGUQmYBYw0TTPFMIp3IQEwTXMyMBkgyogpcZuD42XVYMC8HwDoXynH1Wj1OxL0/Xtrk2dekIfbjuFSq+mV4y0AWHxXKxLXFn/ByDAzWMcSGtOSYCPE40+3rPpKym8dZZLdLSI1e1v8Z/KZFOfaYieN5yNl/Gx/WzJVru4zjkpG3otdv50ycYn62LMblp30WZyaIu5vSSGhtF8jiXrqjRPX1ZL+Izvq+/Tja4GiXXGyTp/mRFqsR+ezo8bcZDaqXabj/aUv6DJ5EH5ixCfZ24Yd7AjAhtckU2r0f5Z4pcapHfuwoMnnoL1XAwXXEi0Kf+mz3Ne/HyCJbLZOrcnEuj8VdQgAndbKdSfylUhC1u0BwHViVaH7+0uflVypklz2GfOX+xh4x0J5XVWyhy44G8kTUyWcIfYbCb2x6onW5BeP3Tt9qTGtWuGt+ulcOA+tGAiAsUMyFzececqjtuy6VRIUmQl/8G5rSfrXMVySVzVKz+KPI2/wwrgovn+iKpzyzfPamdvkfv7SK5L8q0NYFmeyzgPQ5XVxJ08c751EUaXpwzKP0d17AXjzI8m4HHf/JP4Unl6ic1zyi9SijvtQwgIKu+b663t4to/4eQcVke09xHARNFoSY5X2nmHH+0RuHnp8JAD/HVtwfUwgO0PwiaYQU8D7PtU4Rlxk+dKz/5GbyafiAFh3RpbF1r/Rgoaf+SYBoUfpVA3DCEEmoDNM0/zcvfmIYRi13e/XBo6WTxPLnywzi3UsoRb1qWHUBSCUMNJMWblz//a0eootKU4jEIL2oW1xuj7QMap9aH+c3odO1wfOH6Pp6SYD7v+N7jdVoEt3sbqEBoVrHwYQF7s+HaMXD55kxzWA94HNpmm+meutucDdwMvu33O83bi0G1vz3jtSGNKqrZib5S++C0jQ984M6diWn0lCgkZjrNXtzUX+D9M02cSvVCSSBkZS9vbq1CGZvcTRhGT2ApwsmxrvcLxFyRd+PNEIVAU+9lpDc5H4z98BSB4g1tvargqsGTXxgr3CWJEm2h58azgAdf+5xaPz27EPM6+RcgdvNpYV4f+drUDVJ8R1IystrUTnsqM+i7huUt82/eWyncffY9RTkq+qVKrj/N2HWevkuzStsXiWTKMBINfMSl4qS2KXPqyzyH2N7Ff0frtflUQTnpZl8XcfWlh9uacN9OSKYvevTE7JlaIshXbRZ1HvpV/48SWZRP2YKwuP5XVRGguMXcboZT/dB0DCyKPEJ+ct7eCprrhc+bXerC11mIeOq8aRibMZlHSCbzL+xjfvQt3TS6ieVavc+9CVEM/xS+Qe1yFMVHRefysVu0mpoFpeqn1qhz6s95JoeeXHOxk8QFwfv+gp7qvN3cm/cmMlMlq0vxENBohbfVb6+QLP7W99+3tL3xVV+ujfZ2IIOn3WvV/JsNt1pjAiZ4u3yMDh3QGY0fCbQveNbno8z9/+6ENjqzyLtXx7OE/d/RkAt1cqeo77/DHxglp+l/jqWveWSC+5y3uCJ+64HYC7gPWGYViXvaeQyee/DcO4D9gH3FY+TSxfTnGcw+yjEpVZan4PQAKX0IDGrGcpB809uPNeFV6x1eZ4ohGIQvo04HB6HzpdH+gY1T60P07vQ6frA+eP0XNb9nF60Vp+/C2EXcdfByDMbEgDownrzSXahwGA6tMxejFhmD4sGB9lxJhtjS4e73/o0asKsJjlkJwplrVO34+k6RspQPkVPV5gzlxpmuaVRe1TUn2l4aq1snr2dDWJkZlyKpZ5XWQVIyP5cKnP64k+KJtGs4PE8H786dvZ5Uosem3riTlSYlysGB9v46s+dEVH03/JegDuiJREGZdNHk79v5dv0XRf6Uvr0Zp/TJQEGFbR5nYvjSg03uGJRz6ha4WDAFQIkiD/vi3ylmjxBF+M0eJwVZc05/PWzM+z/YZBgwlemL9MTUmxy3WmvPB1H84/lL8cS6eHHwSKL8lSWrQPna8PPNcYXFvKJHz+6zxaLb0HgNj+UrbELMQaVloMtxUuqGIEmafkmYgCnvHKow/3/99VJFwnVs8e1eX+N/e2jmRu3OrxObyFjlGhNBq3fSCn3dL13XzvLUuT+/cL/e+C5etLdN6SYpc+tJKBbn5cklJt6Z7zuVy7XtxtQsdVJfTbFfkPLoLy7ENXtDuxcHAxdka3V15mSkqJzu8pnmgsUXZcX1NraSob3Rdpy8Uhg0zePiG16xb0lUlN0vZfS1Vvygn0i9zO3Mh28ofN142Mn+WBcGBshwLePeT+CXx2jWzKHZEL8mwL8u6zhl8J+3oF049KUpu2sf8DYPlfJhTpvjPzjCQmeuZrcZhIPFV4YhQ7kxVnq3LBSjF0rdMy3zZP3W8VxRtYi8O967amHhsB72SVLAhrUpt50vc3HNMF24/KIt3cB6TGZOZW309AlbIRO8edKqZr/veGfCCVJmKXl++Cup3I3LYTgMYPyRykyYSHCT4pbucNn7SS9+zyR9MKJfPECX83wWM8SkykKIqiKIqiKIqiKN7A1pbQoJ/W0P/90QCcj5K1w8o7oNoke64++ILpa8Xq+XQXcceNCgpn8oLpAPR9VlKgx3zgm9TKSsHMGDQOkJWy5WmSGr/ewtNFHBF4rJskLuBX1JTA9tXD86cx77utDwD75sdRf46UEUrYXLq6k3Yh6ETBdQujn93L6YU+boyiKIpNqP9sjnXsYvVMcwIRc5YD0HtO63zvxXopuVQgYnkZJA1Z7ueWOAu1hCqKoiiKoiiKoig+w9aWUIDY5y7elZeCSLxnLQCN/yUFu7d2/oApJ6RAdPV5kopfVyH9y8ObB/DbtmoANH7Jba0/Ur5B/L4mZlpea3vPlwsqFSExvnU55JgxaR45BkC/XTcA8FnD7wA4eKYyURzzW7sURVEURVECCdtPQpULyJLH+UYDJMlPD1rlevM3PzRIuZDKPXZk1+ZzyuRLEbJOi1v16U7y2/r+RbHTb21SFEVRFEUJNNQdV1EURVEURVEURfEZPq0TahjGb8AfEBB+a9XI284GpmlWL+oAp+sDMAzjNBAIedcv1Afah47XB87X6HR9oNcZm6FjtAACSKOO0QJwuj5wvkan64OL4F7oy0kogGEYv3pSoNXflLadqs8elKWdTtfodH1lPdaXaB969zhfo2PU+8f5Gqf3odP1gY7R8jrWl2gfevc4X1Padqo7rqIoiqIoiqIoiuIzdBKqKIqiKIqiKIqi+Ax/TEIn++F/lobStlP12YOytNPpGp2ur6zH+hLtQ+8e52t0jHr/OF/j9D50uj7QMVpex/oS7UPvHudrStVOn8eEKoqiKIqiKIqiKBcv6o6rKIqiKIqiKIqi+AyfTUINw+hmGMZWwzB2GIbxpK/+b3EYhhFrGMb/DMPYbBjGRsMwRri3P2sYxkHDMNa4f3p4cC7baXS6PvCeRqfrcx/jaI2qzz/oGNU+vOBcttPodH2gY1T7MM95HK3PfYztNDpdH3hXI6ZplvsP4AJ2Ag2BUGAt0MwX/9uDttUGWrlfRwLbgGbAs8Cjga7R6fq8pdHp+i4GjaovsPVdDBqdrs/OGp2uz1sana7vYtDodH121uh0fd7UaJqmzyyhbYAdpmnuMk3zPPAp0MdH/7tITNNMNk1zlfv1aWAzULcUp7KlRqfrA69pdLo+cL5G1ecndIx6jNP1gU01Ol0f6BgtAU7X6HR9YFONTtcHXtXos0loXWB/rr8PUMoGlyeGYcQBlwPL3JuGGYaxzjCMDwzDiC7mcNtrdLo+KJNGp+sD52tUfTZAx6j2ITbX6HR9oGO0mMOdrtHp+iAANDpdH5RZo88moUYB22yVltcwjErALGCkaZopwLtAI6AlkAy8UdwpCthmG41O1wdl1uh0feB8jarPz+gY1T7E5hqdrg90jKJ96HR9YHONTtcHXtHos0noASA219/1gEM++t/FYhhGCPJBzjBN83MA0zSPmKaZaZpmFjAFMY0XhW01Ol0feEWj0/WB8zWqPj+iY1T70I1tNTpdH+gYRfsQnK8PbKzR6frAaxp9NgldASQahhFvGEYo0B+Y66P/XSSGYRjA+8Bm0zTfzLW9dq7d+gIbijmVLTU6XR94TaPT9YHzNao+P6FjNBvtQ5tqdLo+0DHqRvvQ+frAphqdrg+8qtE32XFNyaDUA8mgtBP4q6/+rwft6oiYuNcBa9w/PYB/Auvd2+cCtQNRo9P1eVOj0/VdDBpVX2Druxg0Ol2fXTU6XZ+OUe3Di0mfXTU6XZ+3NRruEyqKoiiKoiiKoihKueMrd1xFURRFURRFURRF0UmooiiKoiiKoiiK4jt0EqooiqIoiqIoiqL4DJ2EKoqiKIqiKIqiKD5DJ6GKoiiKoiiKoiiKz9BJqKIoiqIoiqIoiuIzdBKqKIqiKIqiKIqi+AydhCqKoiiKoiiKoig+4/8BkGZUSH8hGTQAAAAASUVORK5CYII=\n", + "text/plain": [ + "
" + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + } + ], + "source": [ + "from utils.mnist import mnist_to_numpy, normalize\n", + "import random\n", + "import matplotlib.pyplot as plt\n", + "%matplotlib inline\n", + "\n", + "data_dir = '/tmp/data'\n", + "X, _ = mnist_to_numpy(data_dir, train=False)\n", + "\n", + "# randomly sample 16 images to inspect\n", + "mask = random.sample(range(X.shape[0]), 16)\n", + "samples = X[mask]\n", + "\n", + "# plot the images \n", + "fig, axs = plt.subplots(nrows=1, ncols=16, figsize=(16, 1))\n", + "\n", + "for i, splt in enumerate(axs):\n", + " splt.imshow(samples[i])" + ] + }, + { + "cell_type": "code", + "execution_count": 13, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "(16, 28, 28) uint8\n" + ] + } + ], + "source": [ + "print(samples.shape, samples.dtype)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Before we invoke the SageMaker PyTorch model server with `samples`, we need to do\n", + "some pre-processing\n", + "- convert its data type to 32 bit floating point\n", + "- normalize each channel (only one channel for MNIST)\n", + "- add a channel dimension" + ] + }, + { + "cell_type": "code", + "execution_count": 21, + "metadata": {}, + "outputs": [], + "source": [ + "samples = normalize(samples.astype(np.float32), axis=(1, 2))\n", + "\n", + "res = predictor.predict(\n", + " {\n", + " 'inputs': np.expand_dims(samples, axis=1).tolist()\n", + " }\n", + ")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The response is a list of probablity vector of each sample." + ] + }, + { + "cell_type": "code", + "execution_count": 22, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Predicted digits: [3, 6, 9, 3, 3, 6, 1, 6, 6, 9, 3, 9, 6, 3, 1, 6]\n" + ] + } + ], + "source": [ + "predictions = np.argmax(np.array(res, dtype=np.float32), axis=1).tolist()\n", + "print(\"Predicted digits: \", predictions)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Test and debug the entry point before deployment\n", + "\n", + "When deploying a model to a SageMaker endpoint, it is a good practice to test the entry \n", + "point. The following snippet shows you how you can test and debug the `model_fn` and \n", + "`transform_fn` you implemented in the entry point for the inference image. " + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "!pygmentize code/test_inference.py" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The `test` function simulates how the inference container works. It pulls the model\n", + "artifact and loads the model into \n", + "memory by calling `model_fn` and parse `model_dir` to it. \n", + "When it receives a request,\n", + "it calls `input_fn`, `predict_fn` and `output_fn` consecutively. \n", + "\n", + "Implementing such a test function helps you debugging the entry point before put it into\n", + "the production. If `test` runs correctly, then you can be certain that if the incoming\n", + "data and its content type are what they suppose to be, then the endpoint point is going\n", + "to work as expected. " + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## (Optional) Clean up \n", + "\n", + "If you do not plan to use the endpoint, you should delete it to free up some computation \n", + "resource. If you use local, you will need to manually delete the docker container bounded\n", + "at port 8080 (the port that listens to the incoming request).\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "import os\n", + "\n", + "if not local_mode:\n", + " predictor.delete_endpoint()\n", + "else:\n", + " os.system(\"docker container ls | grep 8080 | awk '{print $1}' | xargs docker container rm -f\")" + ] + } + ], + "metadata": { + "instance_type": "ml.t3.medium", + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.6" + } + }, + "nbformat": 4, + "nbformat_minor": 4 +} diff --git a/frameworks/pytorch/get_started_mnist_train.ipynb b/frameworks/pytorch/get_started_mnist_train.ipynb new file mode 100644 index 00000000..81e0db0a --- /dev/null +++ b/frameworks/pytorch/get_started_mnist_train.ipynb @@ -0,0 +1,379 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# MNIST training with PyTorch\n", + "\n", + "MNIST is a widely used dataset for handwritten digit classification. It consists of 70,000 labeled 28x28 pixel grayscale images of hand-written digits. The dataset is split into 60,000 training images and 10,000 test images. There are 10 classes (one for each of the 10 digits). This tutorial will show how to train and test an MNIST model on SageMaker using PyTorch. \n", + "\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "import os\n", + "import json\n", + "\n", + "import sagemaker\n", + "from sagemaker.pytorch import PyTorch\n", + "from sagemaker import get_execution_role\n", + "\n", + "\n", + "sess = sagemaker.Session()\n", + "\n", + "role = get_execution_role()\n", + "\n", + "output_path='s3://' + sess.default_bucket() + '/mnist'" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## PyTorch Estimator\n", + "\n", + "The `PyTorch` class allows you to run your training script on SageMaker\n", + "infrastracture in a containerized environment. In this notebook, we\n", + "refer to this container as *training container*. \n", + "\n", + "You need to configure\n", + "it with the following parameters to set up the environment:\n", + "\n", + "- entry_point: A user defined python file to be used by the training container as the \n", + "instructions for training. We further discuss this file in the next subsection.\n", + "\n", + "- role: An IAM role to make AWS service requests\n", + "\n", + "- instance_type: The type of SageMaker instance to run your training script. \n", + "Set it to `local` if you want to run the training job on \n", + "the SageMaker instance you are using to run this notebook\n", + "\n", + "- instance count: The number of instances you need to run your training job. \n", + "Multiple instances are needed for distributed training.\n", + "\n", + "- output_path: \n", + "S3 bucket URI to save training output (model artifacts and output files)\n", + "\n", + "- framework_version: The version of PyTorch you need to use.\n", + "\n", + "- py_version: The python version you need to use\n", + "\n", + "For more information, see [the API reference](https://sagemaker.readthedocs.io/en/stable/api/training/estimators.html#sagemaker.estimator.EstimatorBase)\n", + "\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Implement the entry point for training\n", + "\n", + "The entry point for training is a python script that provides all \n", + "the code for training a PyTorch model. It is used by the SageMaker \n", + "PyTorch Estimator (`PyTorch` class above) as the entry point for running the training job.\n", + "\n", + "Under the hood, SageMaker PyTorch Estimator creates a docker image\n", + "with runtime environemnts \n", + "specified by the parameters you used to initiated the\n", + "estimator class and it injects the training script into the \n", + "docker image to be used as the entry point to run the container.\n", + "\n", + "In the rest of the notebook, we use *training image* to refer to the \n", + "docker image specified by the PyTorch Estimator and *training container*\n", + "to refer to the container that runs the training image. \n", + "\n", + "This means your training script is very similar to a training script\n", + "you might run outside Amazon SageMaker, but it can access the useful environment \n", + "variables provided by the training image. Checkout [the short list of environment variables provided by the SageMaker service](https://sagemaker.readthedocs.io/en/stable/frameworks/mxnet/using_mxnet.html?highlight=entry%20point) to see some common environment \n", + "variables you might used. Checkout [the complete list of environment variables](https://github.com/aws/sagemaker-training-toolkit/blob/master/ENVIRONMENT_VARIABLES.md) for a complete \n", + "description of all environment variables your training script\n", + "can access to. \n", + "\n", + "In this example, we use the training script `code/train.py`\n", + "as the entry point for our PyTorch Estimator.\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "!pygmentize 'code/train.py'" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Set hyperparameters\n", + "\n", + "In addition, PyTorch estimator allows you to parse command line arguments\n", + "to your training script via `hyperparameters`.\n", + "\n", + " Note: local mode is not supported in SageMaker Studio " + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# set local_mode to be True if you want to run the training script\n", + "# on the machine that runs this notebook\n", + "\n", + "local_mode=True\n", + "\n", + "if local_mode:\n", + " instance_type='local'\n", + "else:\n", + " instance_type='ml.c4.xlarge'\n", + " \n", + "est = PyTorch(\n", + " entry_point='train.py',\n", + " source_dir='code', # directory of your training script\n", + " role=role,\n", + " framework_version='1.5.0',\n", + " py_version='py3',\n", + " instance_type=instance_type,\n", + " instance_count=1,\n", + " output_path=output_path,\n", + " hyperparameters={\n", + " 'batch-size':128,\n", + " 'epochs':20,\n", + " 'learning-rate': 1e-3,\n", + " 'log-interval':100\n", + " }\n", + ")\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The training container executes your training script like\n", + "\n", + "```\n", + "python train.py --batch-size 100 --epochs 10 --learning-rate 1e-3 \\\n", + " --log-interval 100\n", + "```" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Set up channels for training and testing data\n", + "\n", + "You need to tell `PyTorch` estimator where to find your training and \n", + "testing data. It can be a link to an S3 bucket or it can be a path\n", + "in your local file system if you use local mode. In this example,\n", + "we download the MNIST data from a public S3 bucket and upload it \n", + "to your default bucket. " + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "import logging\n", + "import boto3\n", + "from botocore.exceptions import ClientError\n", + "\n", + "\n", + "# Download training and testing data from a public S3 bucket\n", + "\n", + "\n", + "def download_from_s3(data_dir='/tmp/data', train=True):\n", + " \"\"\"Download MNIST dataset and convert it to numpy array\n", + " \n", + " Args:\n", + " data_dir (str): directory to save the data\n", + " train (bool): download training set\n", + " \n", + " Returns:\n", + " None\n", + " \"\"\"\n", + " \n", + " # Get global config\n", + " with open('code/config.json', 'r') as f:\n", + " CONFIG=json.load(f)\n", + " \n", + " if not os.path.exists(data_dir):\n", + " os.makedirs(data_dir)\n", + " \n", + " if train:\n", + " images_file = \"train-images-idx3-ubyte.gz\"\n", + " labels_file = \"train-labels-idx1-ubyte.gz\"\n", + " else:\n", + " images_file = \"t10k-images-idx3-ubyte.gz\"\n", + " labels_file = \"t10k-labels-idx1-ubyte.gz\"\n", + "\n", + " # download objects\n", + " s3 = boto3.client('s3')\n", + " bucket = CONFIG['public_bucket']\n", + " for obj in [images_file, labels_file]:\n", + " key = os.path.join(\"datasets/image/MNIST\", obj)\n", + " dest = os.path.join(data_dir, obj)\n", + " if not os.path.exists(dest):\n", + " s3.download_file(bucket, key, dest)\n", + " return\n", + "\n", + "\n", + "download_from_s3('/tmp/data', True)\n", + "download_from_s3('/tmp/data', False)\n", + "\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# upload to the default bucket\n", + "\n", + "prefix = 'mnist'\n", + "bucket = sess.default_bucket()\n", + "loc = sess.upload_data(path='/tmp/data', bucket=bucket, key_prefix=prefix)\n", + "\n", + "channels = {\n", + " \"training\": loc,\n", + " \"testing\": loc\n", + "}\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The keys of the dictionary `channels` are parsed to the training image\n", + "and it creates the environment variable `SM_CHANNEL_`. \n", + "\n", + "In this example, `SM_CHANNEL_TRAINING` and `SM_CHANNEL_TESTING` are created in the training image (checkout \n", + "how `code/train.py` access these variables). For more information,\n", + "see: [SM_CHANNEL_{channel_name}](https://github.com/aws/sagemaker-training-toolkit/blob/master/ENVIRONMENT_VARIABLES.md#sm_channel_channel_name)\n", + "\n", + "If you want, you can create a channel for validation:\n", + "```\n", + "channels = {\n", + " 'training': train_data_loc,\n", + " 'validation': val_data_loc,\n", + " 'test': test_data_loc\n", + " }\n", + "```\n", + "You can then access this channel within your training script via\n", + "`SM_CHANNEL_VALIDATION`\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Run the training script on SageMaker\n", + "Now, the training container has everything to execute your training\n", + "script. You can start the container by calling `fit` method." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "scrolled": true + }, + "outputs": [], + "source": [ + "est.fit(inputs=channels)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Inspect and store model data\n", + "\n", + "Now, the training is finished, the model artifact has been saved in \n", + "the `output_path`. We " + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "pt_mnist_model_data = est.model_data\n", + "print(\"Model artifact saved at:\\n\", pt_mnist_model_data)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We store the variable `model_data` in the current notebook kernel. \n", + "In the [next notebook](get_started_with_mnist_deploy.ipynb), you will learn how to retrieve the model artifact and deploy to a SageMaker\n", + "endpoint." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "%store pt_mnist_model_data" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Test and debug the entry point before executing the training container\n", + "\n", + "The entry point `code/train.py` provided here has been tested and it can be executed in the training container. \n", + "When you do develop your own training script, it is a good practice to simulate the container environment \n", + "in the local shell and test it before sending it to SageMaker, because debugging in a containerized environment\n", + "is rather cumbersome. The following script shows how you can test your training script:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "!pygmentize code/test_train.py" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.6" + }, + "notice": "Copyright 2017 Amazon.com, Inc. or its affiliates. All Rights Reserved. Licensed under the Apache License, Version 2.0 (the \"License\"). You may not use this file except in compliance with the License. A copy of the License is located at http://aws.amazon.com/apache2.0/ or in the \"license\" file accompanying this file. This file is distributed on an \"AS IS\" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License." + }, + "nbformat": 4, + "nbformat_minor": 4 +} diff --git a/frameworks/pytorch/utils b/frameworks/pytorch/utils new file mode 120000 index 00000000..19985ba5 --- /dev/null +++ b/frameworks/pytorch/utils @@ -0,0 +1 @@ +../utils/ \ No newline at end of file diff --git a/frameworks/tensorflow/code/config.json b/frameworks/tensorflow/code/config.json new file mode 120000 index 00000000..5d347dc6 --- /dev/null +++ b/frameworks/tensorflow/code/config.json @@ -0,0 +1 @@ +../../../config.json \ No newline at end of file diff --git a/frameworks/tensorflow/code/test_train.py b/frameworks/tensorflow/code/test_train.py new file mode 100644 index 00000000..38b28c68 --- /dev/null +++ b/frameworks/tensorflow/code/test_train.py @@ -0,0 +1,59 @@ +from train import train, parse_args + +import sys +import os +import boto3 +import json + +dirname = os.path.dirname(os.path.abspath(__file__)) + +with open(os.path.join(dirname, "config.json"), "r") as f: + CONFIG = json.load(f) + +def download_from_s3(data_dir='/tmp/data', train=True): + """Download MNIST dataset and convert it to numpy array + Args: + data_dir (str): directory to save the data + train (bool): download training set + Returns: + tuple of images and labels as numpy arrays + """ + + if not os.path.exists(data_dir): + os.makedirs(data_dir) + + if train: + images_file = "train-images-idx3-ubyte.gz" + labels_file = "train-labels-idx1-ubyte.gz" + else: + images_file = "t10k-images-idx3-ubyte.gz" + labels_file = "t10k-labels-idx1-ubyte.gz" + + # download objects + s3 = boto3.client('s3') + bucket = CONFIG["public_bucket"] + for obj in [images_file, labels_file]: + key = os.path.join("datasets/image/MNIST", obj) + dest = os.path.join(data_dir, obj) + if not os.path.exists(dest): + s3.download_file(bucket, key, dest) + return + +class Env: + def __init__(self): + # simulate container env + os.environ["SM_MODEL_DIR"] = "/tmp/tf/model" + os.environ["SM_CHANNEL_TRAINING"]="/tmp/data" + os.environ["SM_CHANNEL_TESTING"]="/tmp/data" + os.environ["SM_HOSTS"] = '["algo-1"]' + os.environ["SM_CURRENT_HOST"]="algo-1" + os.environ["SM_NUM_GPUS"] = "0" + + +if __name__=='__main__': + Env() + download_from_s3() + download_from_s3(train=False) + args = parse_args() + train(args) + diff --git a/frameworks/tensorflow/code/train.py b/frameworks/tensorflow/code/train.py new file mode 100644 index 00000000..f2feb178 --- /dev/null +++ b/frameworks/tensorflow/code/train.py @@ -0,0 +1,200 @@ +from __future__ import print_function + +import argparse +import logging +import os +import json +import gzip +import numpy as np +import traceback + +import tensorflow as tf +from tensorflow.keras.layers import Dense, Flatten, Conv2D +from tensorflow.keras import Model + + +logging.basicConfig(level=logging.DEBUG) + +# Define the model object + +class SmallConv(Model): + def __init__(self): + super(SmallConv, self).__init__() + self.conv1 = Conv2D(32, 3, activation='relu') + self.flatten = Flatten() + self.d1 = Dense(128, activation='relu') + self.d2 = Dense(10) + + def call(self, x): + x = self.conv1(x) + x = self.flatten(x) + x = self.d1(x) + return self.d2(x) + + +# Decode and preprocess data +def convert_to_numpy(data_dir, images_file, labels_file): + """Byte string to numpy arrays""" + with gzip.open(os.path.join(data_dir, images_file), 'rb') as f: + images = np.frombuffer(f.read(), np.uint8, offset=16).reshape(-1, 28, 28) + + with gzip.open(os.path.join(data_dir, labels_file), 'rb') as f: + labels = np.frombuffer(f.read(), np.uint8, offset=8) + + return (images, labels) + +def mnist_to_numpy(data_dir, train): + """Load raw MNIST data into numpy array + + Args: + data_dir (str): directory of MNIST raw data. + This argument can be accessed via SM_CHANNEL_TRAINING + + train (bool): use training data + + Returns: + tuple of images and labels as numpy array + """ + + if train: + images_file = "train-images-idx3-ubyte.gz" + labels_file = "train-labels-idx1-ubyte.gz" + else: + images_file = "t10k-images-idx3-ubyte.gz" + labels_file = "t10k-labels-idx1-ubyte.gz" + + return convert_to_numpy(data_dir, images_file, labels_file) + + +def normalize(x, axis): + eps = np.finfo(float).eps + + mean = np.mean(x, axis=axis, keepdims=True) + # avoid division by zero + std = np.std(x, axis=axis, keepdims=True) + eps + return (x - mean) / std + +# Training logic + +def train(args): + # create data loader from the train / test channels + x_train, y_train = mnist_to_numpy(data_dir=args.train, train=True) + x_test, y_test = mnist_to_numpy(data_dir=args.test, train=False) + + x_train, x_test = x_train.astype(np.float32), x_test.astype(np.float32) + + # normalize the inputs to mean 0 and std 1 + x_train, x_test = normalize(x_train, (1, 2)), normalize(x_test, (1, 2)) + + # expand channel axis + # tf uses depth minor convention + x_train, x_test = np.expand_dims(x_train, axis=3), np.expand_dims(x_test, axis=3) + + # normalize the data to mean 0 and std 1 + train_loader = tf.data.Dataset.from_tensor_slices( + (x_train, y_train)).shuffle(len(x_train)).batch(args.batch_size) + + test_loader = tf.data.Dataset.from_tensor_slices( + (x_test, y_test)).batch(args.batch_size) + + model = SmallConv() + model.compile() + loss_fn = tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True) + optimizer = tf.keras.optimizers.Adam( + learning_rate=args.learning_rate, + beta_1=args.beta_1, + beta_2=args.beta_2 + ) + + + train_loss = tf.keras.metrics.Mean(name='train_loss') + train_accuracy = tf.keras.metrics.SparseCategoricalAccuracy(name='train_accuracy') + + test_loss = tf.keras.metrics.Mean(name='test_loss') + test_accuracy = tf.keras.metrics.SparseCategoricalAccuracy(name='test_accuracy') + + + @tf.function + def train_step(images, labels): + with tf.GradientTape() as tape: + predictions = model(images, training=True) + loss = loss_fn(labels, predictions) + grad = tape.gradient(loss, model.trainable_variables) + optimizer.apply_gradients(zip(grad, model.trainable_variables)) + + train_loss(loss) + train_accuracy(labels, predictions) + return + + @tf.function + def test_step(images, labels): + predictions = model(images, training=False) + t_loss = loss_fn(labels, predictions) + test_loss(t_loss) + test_accuracy(labels, predictions) + return + + print("Training starts ...") + for epoch in range(args.epochs): + train_loss.reset_states() + train_accuracy.reset_states() + test_loss.reset_states() + test_accuracy.reset_states() + + for batch, (images, labels) in enumerate(train_loader): + train_step(images, labels) + + for images, labels in test_loader: + test_step(images, labels) + + print( + f'Epoch {epoch + 1}, ' + f'Loss: {train_loss.result()}, ' + f'Accuracy: {train_accuracy.result() * 100}, ' + f'Test Loss: {test_loss.result()}, ' + f'Test Accuracy: {test_accuracy.result() * 100}' + ) + + # Save the model + # A version number is needed for the serving container + # to load the model + version = '00000000' + ckpt_dir = os.path.join(args.model_dir, version) + if not os.path.exists(ckpt_dir): + os.makedirs(ckpt_dir) + model.save(ckpt_dir) + return + + +def parse_args(): + parser = argparse.ArgumentParser() + + parser.add_argument('--batch-size', type=int, default=32) + parser.add_argument('--epochs', type=int, default=1) + parser.add_argument('--learning-rate', type=float, default=1e-3) + parser.add_argument('--beta_1', type=float, default=0.9) + parser.add_argument('--beta_2', type=float, default=0.999) + + # Environment variables given by the training image + parser.add_argument('--model-dir', type=str, default=os.environ['SM_MODEL_DIR']) + parser.add_argument('--train', type=str, default=os.environ['SM_CHANNEL_TRAINING']) + parser.add_argument('--test', type=str, default=os.environ['SM_CHANNEL_TESTING']) + + parser.add_argument('--current-host', type=str, default=os.environ['SM_CURRENT_HOST']) + parser.add_argument('--hosts', type=list, default=json.loads(os.environ['SM_HOSTS'])) + + return parser.parse_args() + + + +if __name__ == '__main__': + args = parse_args() + train(args) + + + + + + + + diff --git a/frameworks/tensorflow/get_started_mnist_deploy.ipynb b/frameworks/tensorflow/get_started_mnist_deploy.ipynb new file mode 100644 index 00000000..dc47419f --- /dev/null +++ b/frameworks/tensorflow/get_started_mnist_deploy.ipynb @@ -0,0 +1,304 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Deploy a Trained TensorFlow V2 Model\n", + "\n", + "In this notebook, we walk through the process of deploying a trained model to a SageMaker endpoint. If you recently ran [the notebook for training](get_started_mnist_deploy.ipynb) with %store% magic, the `model_data` can be restored. Otherwise, we retrieve the \n", + "model artifact from a public S3 bucket." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# setups\n", + "\n", + "import os\n", + "import json\n", + "\n", + "import sagemaker\n", + "from sagemaker.tensorflow import TensorFlowModel\n", + "from sagemaker import get_execution_role, Session\n", + "import boto3\n", + "\n", + "# Get global config\n", + "with open('code/config.json', 'r') as f:\n", + " CONFIG=json.load(f)\n", + "\n", + "sess = Session()\n", + "role = get_execution_role()\n", + "\n", + "%store -r tf_mnist_model_data\n", + "\n", + "\n", + "\n", + "try: \n", + " tf_mnist_model_data\n", + "except NameError:\n", + " import json\n", + " # copy a pretrained model from a public public to your default bucket\n", + " s3 = boto3.client('s3')\n", + " bucket = CONFIG['public_bucket']\n", + " key = 'datasets/image/MNIST/model/tensorflow-training-2020-11-20-23-57-13-077/model.tar.gz'\n", + " s3.download_file(bucket, key, 'model.tar.gz')\n", + " tf_mnist_model_data = sess.upload_data(\n", + " path='model.tar.gz', bucket=sess.default_bucket(), key_prefix='model/tensorflow')\n", + " os.remove('model.tar.gz')\n", + "\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "print(tf_mnist_model_data)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## TensorFlow Model Object\n", + "\n", + "The `TensorFlowModel` class allows you to define an environment for making inference using your\n", + "model artifact. Like `TensorFlow` estimator class we discussed \n", + "[in this notebook for training an Tensorflow model](\n", + "get_started_mnist_train.ipynb), it is high level API used to set up a docker image for your model hosting service.\n", + "\n", + "Once it is properly configured, it can be used to create a SageMaker\n", + "endpoint on an EC2 instance. The SageMaker endpoint is a containerized environment that uses your trained model \n", + "to make inference on incoming data via RESTful API calls. \n", + "\n", + "Some common parameters used to initiate the `TensorFlowModel` class are:\n", + "- role: An IAM role to make AWS service requests\n", + "- model_data: the S3 bucket URI of the compressed model artifact. It can be a path to a local file if the endpoint \n", + "is to be deployed on the SageMaker instance you are using to run this notebook (local mode)\n", + "- framework_version: version of the MXNet package to be used\n", + "- py_version: python version to be used" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "\n", + "model = TensorFlowModel(\n", + " role=role,\n", + " model_data=tf_mnist_model_data,\n", + " framework_version='2.3.0',\n", + ")\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Execute the Inference Container\n", + "Once the `TensorFlowModel` class is initiated, we can call its `deploy` method to run the container for the hosting\n", + "service. Some common parameters needed to call `deploy` methods are:\n", + "\n", + "- initial_instance_count: the number of SageMaker instances to be used to run the hosting service.\n", + "- instance_type: the type of SageMaker instance to run the hosting service. Set it to `local` if you want run the hosting service on the local SageMaker instance. Local mode are typically used for debugging. \n", + "\n", + " Note: local mode is not supported in SageMaker Studio " + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "from sagemaker.serializers import JSONSerializer\n", + "from sagemaker.deserializers import JSONDeserializer\n", + "\n", + "# set local_mode to False if you want to deploy on a remote\n", + "# SageMaker instance\n", + "\n", + "local_mode=False\n", + "\n", + "if local_mode:\n", + " instance_type='local'\n", + "else:\n", + " instance_type='ml.c4.xlarge'\n", + "\n", + "predictor = model.deploy(\n", + " initial_instance_count=1,\n", + " instance_type=instance_type,\n", + " )" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Making Predictions Against a SageMaker endpoint\n", + "\n", + "Once you have the `Predictor` instance returned by `model.deploy(...)`, you can send prediction requests to your endpoints. In this case, the model accepts normalized \n", + "batch images in depth-minor convention. " + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# use some dummy inputs\n", + "import numpy as np\n", + "\n", + "dummy_inputs = {\n", + " 'instances': np.random.rand(4, 28, 28, 1)\n", + "}\n", + "\n", + "res = predictor.predict(dummy_inputs)\n", + "print(res)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The formats of the input and output data correspond directly to the request and response\n", + "format of the `Predict` method in [TensorFlow Serving REST API](https://www.tensorflow.org/tfx/serving/api_rest), for example, the key of the array to be \n", + "parsed to the model in the `dummy_inputs` needs to be called `instances`. Moreover, the input data needs to have a batch dimension. " + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Uncomment the following lines to see an example that cannot be processed by the endpoint\n", + "\n", + "#dummy_data = {\n", + "# 'instances': np.random.rand(28, 28, 1).tolist()\n", + "#}\n", + "#print(predictor.predict(inputs))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Now, let's use real MNIST test to test the endpoint. We use helper functions defined in `code.utils` to \n", + "download MNIST data set and normalize the input data." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "from utils.mnist import mnist_to_numpy, normalize\n", + "import random\n", + "import matplotlib.pyplot as plt\n", + "%matplotlib inline\n", + "\n", + "data_dir = '/tmp/data'\n", + "X, _ = mnist_to_numpy(data_dir, train=False)\n", + "\n", + "# randomly sample 16 images to inspect\n", + "mask = random.sample(range(X.shape[0]), 16)\n", + "samples = X[mask]\n", + "\n", + "# plot the images \n", + "fig, axs = plt.subplots(nrows=1, ncols=16, figsize=(16, 1))\n", + "\n", + "for i, splt in enumerate(axs):\n", + " splt.imshow(samples[i])" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Since the model accepts normalized input, you will need to normalize the samples before \n", + "sending it to the endpoint. " + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "samples = normalize(samples, axis=(1, 2))\n", + "predictions = predictor.predict(\n", + " np.expand_dims(samples, 3) # add channel dim\n", + ")['predictions'] \n", + "\n", + "# softmax to logit\n", + "predictions = np.array(predictions, dtype=np.float32)\n", + "predictions = np.argmax(predictions, axis=1)\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "print(\"Predictions: \", predictions.tolist())" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## (Optional) Clean up \n", + "\n", + "If you do not plan to use the endpoint, you should delete it to free up some computation \n", + "resource. If you use local, you will need to manually delete the docker container bounded\n", + "at port 8080 (the port that listens to the incoming request).\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "import os\n", + "\n", + "if not local_mode:\n", + " predictor.delete_endpoint()\n", + "else:\n", + " os.system(\"docker container ls | grep 8080 | awk '{print $1}' | xargs docker container rm -f\")" + ] + } + ], + "metadata": { + "instance_type": "ml.t3.medium", + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.6" + } + }, + "nbformat": 4, + "nbformat_minor": 4 +} diff --git a/frameworks/tensorflow/get_started_mnist_train.ipynb b/frameworks/tensorflow/get_started_mnist_train.ipynb new file mode 100644 index 00000000..71ba19d2 --- /dev/null +++ b/frameworks/tensorflow/get_started_mnist_train.ipynb @@ -0,0 +1,387 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Training a Tensorflow Model on MNIST\n", + "\n", + "MNIST is a widely used dataset for handwritten digit classification. It consists of 70,000 labeled 28x28 pixel grayscale images of hand-written digits. The dataset is split into 60,000 training images and 10,000 test images. There are 10 classes (one for each of the 10 digits). This tutorial will show how to train a Tensorflow V2 model on MNIST model on SageMaker.\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "import os\n", + "import json\n", + "\n", + "import sagemaker\n", + "from sagemaker.tensorflow import TensorFlow\n", + "from sagemaker import get_execution_role\n", + "\n", + "sess = sagemaker.Session()\n", + "\n", + "role = get_execution_role()\n", + "\n", + "output_path='s3://' + sess.default_bucket() + '/tensorflow/mnist'" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## TensorFlow Estimator\n", + "\n", + "The `TensorFlow` class allows you to run your training script on SageMaker\n", + "infrastracture in a containerized environment. In this notebook, we\n", + "refer to this container as *training container*. \n", + "\n", + "You need to configure\n", + "it with the following parameters to set up the environment:\n", + "\n", + "- entry_point: A user defined python file to be used by the training container as the \n", + "instructions for training. We will further discuss this file in the next subsection\n", + "\n", + "- role: An IAM role to make AWS service requests\n", + "\n", + "- instance_type: The type of SageMaker instance to run your training script. \n", + "Set it to `local` if you want to run the training job on \n", + "the SageMaker instance you are using to run this notebook\n", + "\n", + "- model_dir: S3 bucket URI where the checkpoint data and models can be exported to during training (default: None). \n", + "To disable having model_dir passed to your training script, set `model_dir`=False\n", + "\n", + "- instance count: The number of instances you need to run your training job. \n", + "Multiple instances are needed for distributed training\n", + "\n", + "- output_path: \n", + "S3 bucket URI to save training output (model artifacts and output files)\n", + "\n", + "- framework_version: The version of TensorFlow you need to use.\n", + "\n", + "- py_version: The python version you need to use\n", + "\n", + "For more information, see [the API reference](https://sagemaker.readthedocs.io/en/stable/api/training/estimators.html#sagemaker.estimator.EstimatorBase)\n", + "\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Implement the entry point for training\n", + "\n", + "The entry point for training is a python script that provides all \n", + "the code for training a TensorFlow model. It is used by the SageMaker \n", + "TensorFlow Estimator (`TensorFlow` class above) as the entry point for running the training job.\n", + "\n", + "Under the hood, SageMaker TensorFlow Estimator downloads a docker image\n", + "with runtime environemnts \n", + "specified by the parameters you used to initiated the\n", + "estimator class and it injects the training script into the \n", + "docker image to be used as the entry point to run the container.\n", + "\n", + "In the rest of the notebook, we use *training image* to refer to the \n", + "docker image specified by the TensorFlow Estimator and *training container*\n", + "to refer to the container that runs the training image. \n", + "\n", + "This means your training script is very similar to a training script\n", + "you might run outside Amazon SageMaker, but it can access the useful environment \n", + "variables provided by the training image. Checkout [the short list of environment variables provided by the SageMaker service](https://sagemaker.readthedocs.io/en/stable/frameworks/mxnet/using_mxnet.html?highlight=entry%20point) to see some common environment \n", + "variables you might used. Checkout [the complete list of environment variables](https://github.com/aws/sagemaker-training-toolkit/blob/master/ENVIRONMENT_VARIABLES.md) for a complete \n", + "description of all environment variables your training script\n", + "can access to. \n", + "\n", + "In this example, we use the training script `code/train.py`\n", + "as the entry point for our TensorFlow Estimator. " + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "!pygmentize 'code/train.py'" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Set hyperparameters\n", + "\n", + "In addition, TensorFlow estimator allows you to parse command line arguments\n", + "to your training script via `hyperparameters`.\n", + "\n", + " Note: local mode is not supported in SageMaker Studio. " + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# set local_mode to be True if you want to run the training script\n", + "# on the machine that runs this notebook\n", + "\n", + "local_mode=True\n", + "\n", + "if local_mode:\n", + " instance_type='local'\n", + "else:\n", + " instance_type='ml.c4.xlarge'\n", + " \n", + "est = TensorFlow(\n", + " entry_point='train.py',\n", + " source_dir='code', # directory of your training script\n", + " role=role,\n", + " framework_version='2.3.0',\n", + " model_dir=False, # don't pass --model_dir to your training script\n", + " py_version='py37',\n", + " instance_type=instance_type,\n", + " instance_count=1,\n", + " output_path=output_path,\n", + " hyperparameters={\n", + " 'batch-size':512,\n", + " 'epochs':10,\n", + " 'learning-rate': 1e-3,\n", + " 'beta_1' : 0.9,\n", + " 'beta_2' : 0.999\n", + " \n", + " }\n", + ")\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The training container executes your training script like\n", + "\n", + "```\n", + "python train.py --batch-size 32 --epochs 10 --learning-rate 0.001\n", + " --beta_1 0.9 --beta_2 0.999\n", + "```" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Set up channels for training and testing data\n", + "\n", + "You need to tell `TensorFlow` estimator where to find your training and \n", + "testing data. It can be a link to an S3 bucket or it can be a path\n", + "in your local file system if you use local mode. In this example,\n", + "we download the MNIST data from a public S3 bucket and upload it \n", + "to your default bucket. " + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "import logging\n", + "import boto3\n", + "from botocore.exceptions import ClientError\n", + "# Download training and testing data from a public S3 bucket\n", + "\n", + "def download_from_s3(data_dir='/tmp/data', train=True):\n", + " \"\"\"Download MNIST dataset and convert it to numpy array\n", + " \n", + " Args:\n", + " data_dir (str): directory to save the data\n", + " train (bool): download training set\n", + " \n", + " Returns:\n", + " None\n", + " \"\"\"\n", + " \n", + " if not os.path.exists(data_dir):\n", + " os.makedirs(data_dir)\n", + " \n", + " if train:\n", + " images_file = \"train-images-idx3-ubyte.gz\"\n", + " labels_file = \"train-labels-idx1-ubyte.gz\"\n", + " else:\n", + " images_file = \"t10k-images-idx3-ubyte.gz\"\n", + " labels_file = \"t10k-labels-idx1-ubyte.gz\"\n", + " \n", + " with open('code/config.json', 'r') as f:\n", + " config = json.load(f)\n", + "\n", + " # download objects\n", + " s3 = boto3.client('s3')\n", + " bucket = config['public_bucket']\n", + " for obj in [images_file, labels_file]:\n", + " key = os.path.join(\"datasets/image/MNIST\", obj)\n", + " dest = os.path.join(data_dir, obj)\n", + " if not os.path.exists(dest):\n", + " s3.download_file(bucket, key, dest)\n", + " return\n", + "\n", + "\n", + "download_from_s3('/tmp/data', True)\n", + "download_from_s3('/tmp/data', False)\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# upload to the default bucket\n", + "\n", + "prefix = 'mnist'\n", + "bucket = sess.default_bucket()\n", + "loc = sess.upload_data(path='/tmp/data', bucket=bucket, key_prefix=prefix)\n", + "\n", + "channels = {\n", + " \"training\": loc,\n", + " \"testing\": loc\n", + "}\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The keys of the dictionary `channels` are parsed to the training image\n", + "and it creates the environment variable `SM_CHANNEL_`. \n", + "\n", + "In this example, `SM_CHANNEL_TRAINING` and `SM_CHANNEL_TESTING` are created in the training image (checkout \n", + "how `code/train.py` access these variables). For more information,\n", + "see: [SM_CHANNEL_{channel_name}](https://github.com/aws/sagemaker-training-toolkit/blob/master/ENVIRONMENT_VARIABLES.md#sm_channel_channel_name)\n", + "\n", + "If you want, you can create a channel for validation:\n", + "```\n", + "channels = {\n", + " 'training': train_data_loc,\n", + " 'validation': val_data_loc,\n", + " 'test': test_data_loc\n", + " }\n", + "```\n", + "You can then access this channel within your training script via\n", + "`SM_CHANNEL_VALIDATION`." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Run the training script on SageMaker\n", + "Now, the training container has everything to execute your training\n", + "script. You can start the container by calling `fit` method." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "scrolled": true + }, + "outputs": [], + "source": [ + "est.fit(inputs=channels)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Inspect and store model data\n", + "\n", + "Now, the training is finished, the model artifact has been saved in \n", + "the `output_path`. We " + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "tf_mnist_model_data = est.model_data\n", + "print(\"Model artifact saved at:\\n\", tf_mnist_model_data)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We will store the variable `model_data` in the current notebook kernel. \n", + "In the [next notebook](get_started_with_mnist_deploy.ipynb), you will learn how to retrieve the model artifact and deploy to a SageMaker\n", + "endpoint." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "%store tf_mnist_model_data" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Test and debug the entry point before executing the training container\n", + "\n", + "The entry point `code/train.py` provided here has been tested and it can be executed in the training container. \n", + "When you develop your own training script, it is a good practice to simulate the container environment \n", + "in the local shell and test it before sending it to SageMaker, because debugging in a containerized environment\n", + "is rather cumbersome. The following script shows how you can test your training script:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "!pygmentize code/test_train.py" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "In [the next notebook](get_started_mnist_deploy.ipynb) you will see how to deploy your \n", + "trained model artifacts to a SageMaker endpoint. " + ] + } + ], + "metadata": { + "instance_type": "ml.t3.medium", + "kernelspec": { + "display_name": "Python 3 (Data Science)", + "language": "python", + "name": "python3__SAGEMAKER_INTERNAL__arn:aws:sagemaker:us-west-2:236514542706:image/datascience-1.0" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.6" + }, + "notice": "Copyright 2017 Amazon.com, Inc. or its affiliates. All Rights Reserved. Licensed under the Apache License, Version 2.0 (the \"License\"). You may not use this file except in compliance with the License. A copy of the License is located at http://aws.amazon.com/apache2.0/ or in the \"license\" file accompanying this file. This file is distributed on an \"AS IS\" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License." + }, + "nbformat": 4, + "nbformat_minor": 4 +} diff --git a/frameworks/tensorflow/utils b/frameworks/tensorflow/utils new file mode 120000 index 00000000..19985ba5 --- /dev/null +++ b/frameworks/tensorflow/utils @@ -0,0 +1 @@ +../utils/ \ No newline at end of file diff --git a/frameworks/utils/config.json b/frameworks/utils/config.json new file mode 100644 index 00000000..1db24021 --- /dev/null +++ b/frameworks/utils/config.json @@ -0,0 +1,4 @@ +{ + "public_bucket": "sagemaker-sample-files" +} + diff --git a/frameworks/utils/mnist.py b/frameworks/utils/mnist.py new file mode 100644 index 00000000..43f2b6bd --- /dev/null +++ b/frameworks/utils/mnist.py @@ -0,0 +1,101 @@ +import numpy as np +from urllib import request +import gzip +import os +import boto3 +import json + +dirname = os.path.dirname(os.path.abspath(__file__)) + +with open(os.path.join(dirname, "config.json"), "r") as f: + CONFIG = json.load(f) + +def mnist_to_numpy(data_dir='/tmp/data', train=True): + """Download MNIST dataset and convert it to numpy array + + Args: + data_dir (str): directory to save the data + train (bool): download training set + + Returns: + tuple of images and labels as numpy arrays + """ + + if not os.path.exists(data_dir): + os.makedirs(data_dir) + + + if train: + images_file = "train-images-idx3-ubyte.gz" + labels_file = "train-labels-idx1-ubyte.gz" + else: + images_file = "t10k-images-idx3-ubyte.gz" + labels_file = "t10k-labels-idx1-ubyte.gz" + + # download objects + s3 = boto3.client('s3') + bucket = CONFIG["public_bucket"] + for obj in [images_file, labels_file]: + key = os.path.join("datasets/image/MNIST", obj) + dest = os.path.join(data_dir, obj) + if not os.path.exists(dest): + s3.download_file(bucket, key, dest) + + return _convert_to_numpy(data_dir, images_file, labels_file) + +def _convert_to_numpy(data_dir, images_file, labels_file): + """Byte string to numpy arrays""" + with gzip.open(os.path.join(data_dir, images_file), 'rb') as f: + images = np.frombuffer(f.read(), np.uint8, offset=16).reshape(-1, 28, 28) + + with gzip.open(os.path.join(data_dir, labels_file), 'rb') as f: + labels = np.frombuffer(f.read(), np.uint8, offset=8) + + return (images, labels) + +def normalize(x, axis): + eps = np.finfo(float).eps + + mean = np.mean(x, axis=axis, keepdims=True) + # avoid division by zero + std = np.std(x, axis=axis, keepdims=True) + eps + return (x - mean) / std + +def adjust_to_framework(x, framework='pytorch'): + """Adjust a ``numpy.ndarray`` to be used as input for specified framework + + Args: + x (numpy.ndarray): Batch of images to be adjusted + to follow the convention in pytorch / tensorflow / mxnet + + framework (str): Framework to use. Takes value in + ``pytorch``, ``tensorflow`` or ``mxnet`` + Return: + numpy.ndarray following the convention of tensors in the given + framework + """ + + if x.ndim == 3: + # input is gray-scale + x = np.expand_dims(x, 1) + + if framework in ['pytorch', 'mxnet']: + # depth-major + return x + elif framework == 'tensorlfow': + # depth-minor + return np.transpose(x, (0, 2, 3, 1)) + elif framework == 'mxnet': + return x + else: + raise ValueError('framework must be one of ' + \ + '[pytorch, tensorflow, mxnet], got {}'.format(framework)) + +if __name__ == '__main__': + X, Y = mnist_to_numpy() + X, Y = X.astype(np.float32), Y.astype(np.int8) + + + + + diff --git a/sagemaker-python-sdk/mxnet_horovod_mnist/horovod_mnist.py b/sagemaker-python-sdk/mxnet_horovod_mnist/code/horovod_mnist.py similarity index 97% rename from sagemaker-python-sdk/mxnet_horovod_mnist/horovod_mnist.py rename to sagemaker-python-sdk/mxnet_horovod_mnist/code/horovod_mnist.py index f92ccb3b..d3ba2962 100644 --- a/sagemaker-python-sdk/mxnet_horovod_mnist/horovod_mnist.py +++ b/sagemaker-python-sdk/mxnet_horovod_mnist/code/horovod_mnist.py @@ -22,8 +22,7 @@ from mxnet import autograd, gluon, nd from mxnet.test_utils import download -def main(): - +def main(args): # Function to get mnist iterator given a rank def get_mnist_iterator(rank): data_dir = "data-%d" % rank @@ -175,7 +174,7 @@ def main(): device = context.device_type + str(num_workers) logging.info('Device info: %s', device) -if __name__ == "__main__": +def parse_args(): # Handling script arguments parser = argparse.ArgumentParser(description='MXNet MNIST Distributed Example') parser.add_argument('--batch-size', type=int, default=64, @@ -188,7 +187,8 @@ if __name__ == "__main__": help='learning rate (default: 0.01)') parser.add_argument('--momentum', type=float, default=0.9, help='SGD momentum (default: 0.9)') - parser.add_argument('--no-cuda', action='store_true', help='disable training on GPU (default: False)') + parser.add_argument('--no-cuda', type=bool, default=False, + help='disable training on GPU (default: False)') # Container Environment parser.add_argument('--hosts', type=list, default=json.loads(os.environ['SM_HOSTS'])) @@ -203,8 +203,12 @@ if __name__ == "__main__": # Disable CUDA if there are no GPUs. if mx.context.num_gpus() == 0: args.no_cuda = True + return args +if __name__ == "__main__": + + args = parse_args() logging.basicConfig(level=logging.INFO) logging.info(args) - - main() + main(args) + diff --git a/sagemaker-python-sdk/mxnet_horovod_mnist/code/test_train.py b/sagemaker-python-sdk/mxnet_horovod_mnist/code/test_train.py new file mode 100644 index 00000000..38b28c68 --- /dev/null +++ b/sagemaker-python-sdk/mxnet_horovod_mnist/code/test_train.py @@ -0,0 +1,59 @@ +from train import train, parse_args + +import sys +import os +import boto3 +import json + +dirname = os.path.dirname(os.path.abspath(__file__)) + +with open(os.path.join(dirname, "config.json"), "r") as f: + CONFIG = json.load(f) + +def download_from_s3(data_dir='/tmp/data', train=True): + """Download MNIST dataset and convert it to numpy array + Args: + data_dir (str): directory to save the data + train (bool): download training set + Returns: + tuple of images and labels as numpy arrays + """ + + if not os.path.exists(data_dir): + os.makedirs(data_dir) + + if train: + images_file = "train-images-idx3-ubyte.gz" + labels_file = "train-labels-idx1-ubyte.gz" + else: + images_file = "t10k-images-idx3-ubyte.gz" + labels_file = "t10k-labels-idx1-ubyte.gz" + + # download objects + s3 = boto3.client('s3') + bucket = CONFIG["public_bucket"] + for obj in [images_file, labels_file]: + key = os.path.join("datasets/image/MNIST", obj) + dest = os.path.join(data_dir, obj) + if not os.path.exists(dest): + s3.download_file(bucket, key, dest) + return + +class Env: + def __init__(self): + # simulate container env + os.environ["SM_MODEL_DIR"] = "/tmp/tf/model" + os.environ["SM_CHANNEL_TRAINING"]="/tmp/data" + os.environ["SM_CHANNEL_TESTING"]="/tmp/data" + os.environ["SM_HOSTS"] = '["algo-1"]' + os.environ["SM_CURRENT_HOST"]="algo-1" + os.environ["SM_NUM_GPUS"] = "0" + + +if __name__=='__main__': + Env() + download_from_s3() + download_from_s3(train=False) + args = parse_args() + train(args) + diff --git a/sagemaker-python-sdk/tensorflow_serving_container/tensorflow_serving_container.ipynb b/sagemaker-python-sdk/tensorflow_serving_container/tensorflow_serving_container.ipynb index 84328ebe..7438960d 100644 --- a/sagemaker-python-sdk/tensorflow_serving_container/tensorflow_serving_container.ipynb +++ b/sagemaker-python-sdk/tensorflow_serving_container/tensorflow_serving_container.ipynb @@ -317,7 +317,7 @@ ], "metadata": { "kernelspec": { - "display_name": "conda_tensorflow_p36", + "display_name": "Environment (conda_tensorflow_p36)", "language": "python", "name": "conda_tensorflow_p36" }, @@ -331,7 +331,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.6.5" + "version": "3.6.10" } }, "nbformat": 4, diff --git a/training/frameworks.rst b/training/frameworks.rst index 83e990bb..f9896917 100644 --- a/training/frameworks.rst +++ b/training/frameworks.rst @@ -10,7 +10,6 @@ Apache MXNet .. toctree:: :maxdepth: 1 - ../sagemaker-python-sdk/mxnet_gluon_mnist/mxnet_mnist_with_gluon ../sagemaker-python-sdk/mxnet_gluon_embedding_server/mxnet_embedding_server ../sagemaker-python-sdk/mxnet_gluon_sentiment/mxnet_sentiment_analysis_with_gluon ../introduction_to_applying_machine_learning/gluon_recommender_system/gluon_recommender_system @@ -45,7 +44,9 @@ PyTorch .. toctree:: :maxdepth: 1 - ../sagemaker-python-sdk/pytorch_mnist/pytorch_mnist + ../frameworks/pytorch/get_started_mnist_train + ../frameworks/pytorch/get_started_mnist_deploy + ../sagemaker-python-sdk/pytorch_lstm_word_language_model/pytorch_rnn ../sagemaker-python-sdk/pytorch_lstm_word_language_model/pytorch_rnn @@ -86,6 +87,9 @@ TensorFlow .. toctree:: :maxdepth: 1 + ../frameworks/tensorflow/get_started_mnist_train + ../frameworks/tensorflow/get_started_mnist_deploy + ../sagemaker-python-sdk/tensorflow_moving_from_framework_mode_to_script_mode/tensorflow_moving_from_framework_mode_to_script_mode ../sagemaker-python-sdk/tensorflow_moving_from_framework_mode_to_script_mode/tensorflow_moving_from_framework_mode_to_script_mode ../sagemaker-python-sdk/tensorflow_script_mode_horovod/tensorflow_script_mode_horovod ../sagemaker-python-sdk/tensorflow_script_mode_pipe_mode/tensorflow_script_mode_pipe_mode