AI Deploy - Tutoriel - Créer et utiliser une image Docker personnalisée (EN)

Objective

This tutorial covers the process of building your own Docker image for AI Deploy. After detailing major guidelines, we will cover a quick example.

Requirements

• Access to the OVHcloud Control Panel
• A Public Cloud project
• A user for AI Deploy
• Docker installed on a personal computer or a virtual machine
• Knowledge about building a Docker image (see the official Getting Started guide)

Quick overview

AI Deploy's main goal is to simplify AI models or applications deployment and release them in production, with resiliency and security, in a few seconds. Each application is linked to compute resources such as CPUs or GPUs, and can be accessed through an HTTP Endpoint provided by AI Deploy for each app.

In order to be deployed, your model or application has to be containerised inside a Docker image. Containers provide isolation but also flexibility for your deployments. The Docker images that you build can be deployed locally, with OVHcloud AI Deploy but also with cloud competitors such as AWS or GCP.

Inside your Docker image, you are free to install almost anything and everything as long as you follow the guidelines below.

AI Deploy accepts images from public or private repositories. In short, we can summarize AI deploy with the following schema:

Guidelines to follow

Start from an existing Docker image

Instead of starting from scratch, feel free to start from an existing Docker image, as long as it is compliant with the following guidelines. For example, you can start from 'python', from 'alpine' or equivalent.

If you need to work with GPUs, please read the next paragraph.

Use specific images with CUDA drivers for GPUs

Here is a potential list of official base images (featuring cuda drivers) that you can use:

• pytorch/pytorch:latest
• tensorflow/tensorflow:latest-gpu
• huggingface/transformers-pytorch-gpu:latest (docker pull huggingface/transformers-pytorch-gpu:latest)
• mxnet/python
• nvidia/cuda

For example, if you want to start from the base image tensorflow/tensorflow:latest-gpu:

FROM tensorflow/tensorflow:latest-gpu

Use the linux/amd64 architecture

Your Docker image has to support at least the linux/amd64 platform to be deployed correctly. Otherwise deployment will fail.

When you invoke a build, you can set the --platform flag to specify the target platform for the build output, linux/amd64. This is especially relevant if you use newer Apple computers (M1/M2/... chipsets) or ARM-based computers.

docker buildx build --platform linux/amd64,linux/arm64 ...

More information can be found in the official Docker documentation.

Create an OVHcloud user and a working directory

Deployed containers are not run as root, but by an “OVHcloud” user with UID 42420. It means that if you want to be able to write in a specific directory at runtime, you will have to give it specific rights.

This is the case in the vast majority of use cases.

You can do it with the following instructions:

# Define a working directory called workspace
WORKDIR /workspace

# Copy some files from your computer to the Docker image
COPY my_app.py /workspace/
COPY my_models /workspace/my_models/

# Create a HOME dedicated to the OVHcloud user (42420:42420)
RUN chown -R 42420:42420 /workspace
ENV HOME=/workspace

# Change the ownership of any other useful directory to the OVHcloud user (42420:42420)
RUN chown -R 42420:42420 <another_useful_directory>

# Run your app
CMD [ "python3" , "/workspace/my_app.py" ]

Install dependencies via apt or pip

Usually, Python or Linux packages will be required for your application. You can follow Docker best practices for that, meaning install dependencies with apt or pip.

Here is a compliant example:

RUN apt-get update && apt-get install -y \
  git \
  other_linux_packages \
  ... \

# Install few Python requirements, such as pandas, scikit-learn, taipy...
COPY requirements.txt /workspace/requirements.txt
RUN pip install -r requirements.txt

Manage output data effectively (Swift or S3* compatible Object Storage)

Just like AI Notebooks and AI Training, AI Deploy is easily connected to remote storage such as Swift or S3 compatible Object Storage containers at launch. However, unlike AI Notebooks and AI Training, AI Deploy does NOT* synchronise data back to your remote storage.

If you need to write data somewhere, for example output from your AI model (generated images), your code application should include storage connection. For example, you can use the Python Boto3 library when using Python and S3 compatible Object Storage.

Be careful, if you write data directly in your working directory, it will be lost when you stop your application.

Use environment variables for dynamic values

Sometimes, instead of hardcoding a variable inside a Dockerfile, it is much more powerful to pass variables during deployment. Docker provides this option natively through the --env argument, and OVHcloud AI tools follow the same logic.

During AI Deploy app creation, you will be able to pass environment variables via CLI, API or UI in the control panel. In your Dockerfile, you can gather these variables with the ENV value.

For example, you can launch a new app with two variables like this:

ovhai run app <my_docker_image> -e LANGUAGE=english TOKEN=12345678

In your Dockerfile, you can easily reuse the variables:

# no default value
ENV LANGUAGE
ENV TOKEN

# a default value
ENV foo /bar
# or ENV foo=/bar

# ENV values can be used during the build
ADD . $foo
# or ADD . ${foo}
# translates to: ADD . /bar

Exposing your model or application with an API

Inside your Dockerfile, you will need to expose your model or application so anyone can use it. The easiest way is to expose API via REST endpoint. The most popular open source frameworks for exposing APIs are Flask and Fast API.

You can find a basic example in the section below, and more advanced tutorials in our AI Deploy documentation.

Exposing your application with a web frontend

While an API is useful for automation and code, sometimes you will need to expose your application or model through a web interface.

AI Deploy is fully compliant with multiple frontend frameworks, such as Streamlit, Gradio or Taipy. You can of course also build your own frontend with your favourite tools, such as HTML/CSS.

You can find a basic example in the section below, and more advanced tutorials in our AI Deploy documentation.

Basic example: Write your own Dockerfile and build your image

Here we will build a basic Docker image, following the guidelines.

Prepare the Dockerfile

Create a new file and name it Dockerfile, following the guidelines.

1. First you need to choose a base image to start from.
2. Install what you need as dependencies with apt or pip. Bash command instructions on your Dockerfile should begin with the RUN prefix.
3. Copy files from your local directory inside the Docker image with the COPY prefix.
4. Allow user "OVHcloud UID 42420" to get specific rights.
5. Run your script.

A basic example can be summarised like this:

# Start from official Python image since we don't need GPU
FROM python:3.9

# Create a working directory
WORKDIR /workspace

# Install a few requirements, such as vim and git
RUN apt-get update && apt-get install -y vim git

# Add your files to your Docker image. NB: best practice is to put data outside, using Object Storage
ADD example.py /workspace/
ADD dataset.csv /workspace/

# Create a HOME dedicated to the OVHcloud user (42420:42420). Mandatory step
RUN chown -R 42420:42420 /workspace
ENV HOME=/workspace

# Run your script
CMD [ "python3" , "/workspace/example.py" ]

Build your Docker image

Once your Dockerfile is complete and matches your needs, you have to choose a name and build the image using one of the following commands:

# Build the image using your machine's default architecture
docker build . -t <image-identifier>

# Build image targeting the linux/amd64 architecture
docker buildx build --platform linux/amd64 -t <image-identifier> .

• The first command builds the image using your system’s default architecture. This may work if your machine already uses the linux/amd64 architecture, which is required to run containers with our AI products. However, on systems with a different architecture (e.g. ARM64 on Apple Silicon), the resulting image will not be compatible and cannot be deployed.

• The second command explicitly targets the linux/AMD64 architecture to ensure compatibility with our AI services. This requires buildx, which is not installed by default. If you haven’t used buildx before, you can install it by running: docker buildx install

Test it locally (optional)

If you want to verify that your built image is working properly, run the following command:

docker run --rm -it --user=42420:42420 <image-identifier>

Push the image to the registry of your choice

Pushing your image to a registry is needed in order for AI Deploy to pull it.

AI Deploy provides a default registry called Shared registry where users are able to push their custom images. It is linked with every project by default.

The basic commands to push a Docker image to a registry is:

# Add a new registry into OVHcloud AI Tools
ovhai registry add <url>

# Push your image
docker login -u <registry-user> -p <registry-password> <registry>
docker tag <image-identifier> <registry>/<image-identifier>
docker push <registry>/<image-identifier>

Example: If you want to push an image named custom-image inside a registry registry.gra.ai.cloud.ovh.net:

# Add a new registry into OVHcloud AI Tools
ovhai registry add my-registry.ai.cloud.ovh.net

docker login -u <registry-user> -p <registry-password> my-registry.ai.cloud.ovh.net
docker tag custom-image:latest my-registry.ai.cloud.ovh.net/custom-image:latest
docker push my-registry.ai.cloud.ovh.net/custom-image:latest

If you want to know the exact commands to push on the shared registry, please consult the Details button of the Shared Docker Registry section in the Home panel of AI Training.

Go further

• Discover some AI Deploy apps we built with API or Web frontend via our Apps portfolio.

If you need training or technical assistance to implement our solutions, contact your sales representative or click on this link to get a quote and ask our Professional Services experts for a custom analysis of your project.

Feedback

Please send us your questions, feedback and suggestions to improve the service:

• On the OVHcloud Discord server

*: S3 is a trademark of Amazon Technologies, Inc. OVHcloud’s service is not sponsored by, endorsed by, or otherwise affiliated with Amazon Technologies, Inc.