[AI from Scratch] Episode 236: Running Models in Cloud Environments — Using AWS and GCP

Recap and Today’s Theme

Hello! In the previous episode, we explored creating and publishing APIs, using Flask to turn a trained model into an API and deploying it with Heroku. This enabled us to make prediction functions accessible from other applications and systems.

This time, we will dive into running models in cloud environments using AWS and GCP. By leveraging cloud services, you can operate models in a scalable manner, handle a large volume of requests, and efficiently manage computational resources. This episode will guide you through the steps of deploying and managing models in cloud environments using AWS and GCP.

Advantages of Running Models in Cloud Environments

Using cloud environments offers the following benefits:

1. Scalability: Automatically adjusts resources based on traffic, accommodating a large number of requests.
2. Cost Efficiency: Charges are based only on actual usage, reducing unnecessary costs.
3. High Availability: Redundant data centers ensure service continuity even if issues occur, with automatic server switching.
4. Security: Cloud services provide robust security measures, protecting your data and services.

Deploying Models with AWS (Amazon Web Services)

First, let’s discuss how to deploy models using AWS. AWS provides Amazon SageMaker, a service that supports model deployment, training, and tuning in an integrated manner. You can also use AWS Lambda to deploy a model as a serverless API.

1. Model Deployment with Amazon SageMaker

Amazon SageMaker is a managed service that supports the development, training, and deployment of machine learning models. Below are the steps to deploy a model using SageMaker.

(1) Creating and Configuring an AWS Account

1. Access AWS and create an account.
2. In the AWS Management Console, use IAM (Identity and Access Management) to create a new user and grant them SageMaker management permissions.

(2) Creating a SageMaker Notebook Instance

Create a Notebook instance in SageMaker for model training and deployment.

1. Access SageMaker in the AWS Management Console.
2. From Notebook instances, create a new instance. Choose the instance type (compute capacity) and storage size as needed.
3. Once the Notebook instance is launched, you can access the Jupyter Notebook environment.

(3) Training and Deploying the Model

You can train and deploy models directly within the SageMaker Jupyter Notebook. Below is a simple example code for deploying a pre-trained model using SageMaker:

import sagemaker
from sagemaker.tensorflow import TensorFlow

# Setting up the SageMaker session and role
sagemaker_session = sagemaker.Session()
role = 'your-sagemaker-execution-role'

# Training the model (or using an existing model)
tensorflow_estimator = TensorFlow(entry_point='train.py',  # Training script
                                  role=role,
                                  instance_count=1,
                                  instance_type='ml.m5.large',
                                  framework_version='2.4.1')

# Deploying the model
predictor = tensorflow_estimator.deploy(initial_instance_count=1,
                                        instance_type='ml.m5.large')

• entry_point: Specifies the path to the training script (e.g., train.py).
• instance_type: Defines the type of instance for training.
• deploy: Deploys the model and publishes it as an API endpoint.

With this setup, the model deployed on SageMaker is accessible through an API endpoint, allowing other applications to interact with it.

2. Serverless Deployment with AWS Lambda

You can deploy an API without managing servers using AWS Lambda, a serverless and event-driven service that charges based on the execution of code.

(1) Setting Up AWS Lambda

1. Access Lambda in the AWS Management Console and create a new function.
2. Choose Python as the runtime and specify a function name.

(2) Lambda Function Code Example

Below is a simple example of a Lambda function using a TensorFlow model for predictions:

import json
import tensorflow as tf
import numpy as np

def lambda_handler(event, context):
    # Loading the model
    model = tf.keras.models.load_model('/mnt/data/model.h5')

    # Extracting input data from the request
    input_data = np.array(event['input']).reshape(1, -1)

    # Making a prediction
    prediction = model.predict(input_data)
    predicted_class = int(np.argmax(prediction))

    # Returning the result
    return {
        'statusCode': 200,
        'body': json.dumps({'prediction': predicted_class})
    }

Deploy this function on Lambda and integrate it with API Gateway to publish it as an API.

Deploying Models with GCP (Google Cloud Platform)

Google Cloud Platform (GCP) also offers services for deploying machine learning models. We will use Google AI Platform for deployment.

1. Setting Up Google Cloud AI Platform

GCP provides Google Cloud AI Platform for managing model deployment and operations.

(1) Creating and Configuring a GCP Account

1. Create a GCP account and access the Google Cloud Console.
2. Create a project and enable the AI Platform service.

(2) Preparing for AI Platform Deployment

Next, install the Google Cloud SDK and set up your local environment to connect with GCP:

gcloud init

(3) Deploying the Model

Use the following commands to upload your model to Google Cloud Storage and deploy it on the AI Platform:

# Upload the model file to Google Cloud Storage
gsutil cp model.h5 gs://your-bucket-name/models/model.h5

# Deploy the model on AI Platform
gcloud ai-platform models create my_model --regions us-central1

gcloud ai-platform versions create v1 \
    --model my_model \
    --origin gs://your-bucket-name/models/ \
    --runtime-version 2.5 \
    --framework tensorflow \
    --python-version 3.7

• gsutil: Tool for uploading files to Google Cloud Storage.
• gcloud ai-platform: Command to deploy the model on the AI Platform.

Once deployed, an API endpoint is created on GCP, allowing you to use the model for predictions via HTTP requests.

Best Practices for Cloud Model Deployment

1. Ensure Scalability: Configure auto-scaling settings to automatically expand resources based on traffic.
2. Enhance Security: Implement authentication features and access restrictions to protect against unauthorized access.
3. Monitoring and Logging: Utilize cloud monitoring tools (e.g., AWS CloudWatch, GCP Monitoring) to check API performance and errors in real-time.
4. Cost Management: As cloud resources are pay-per-use, manage costs according to usage to avoid unnecessary expenses.

Summary

This episode explained how to deploy models in cloud environments using AWS and GCP. Cloud services offer scalable and high-availability model deployment, laying the foundation for more advanced machine learning model deployment and system development.

Next Episode Preview

Next time, we will discuss environment setup using Docker, a technology for ensuring reproducibility and efficiency in development and deployment. Learn how Docker can help unify development environments and streamline deployment!

---

Notes

• Amazon SageMaker: AWS’s managed service supporting machine learning model development, training, and deployment.
• Google AI Platform: GCP’s service for managing model deployment, operation, and monitoring in one place.