A machine learning model aims to solve a real problem and the solution as a service should reach the consumer who can easily use it. This is the essence of putting your model into production. As such, model deployment becomes an integral part of the ML lifecycle. In this article, we will go through the model deployment process for deploying a container containing a simple flask app on the Heroku cloud platform. I have tried to keep the process simple so that it provides a basic understanding of the sequence involved.
The scope of the substance is broadly categorized into;
- Develop a simple API using the Flask microframework for Python
- Put a container API into a microservice using Docker
- Deploy the container to the cloud, Heroku
Flask app development
Flask is a web framework for Python. A web framework is a set of libraries and modules that enable web application developers to write applications without worrying about low-level complexities such as protocol, thread management, etc. Flask is often preferred because it is highly Pythonic and has an easy learning curve.
To start the project, it is a good idea to choose a working environment for the development of your project. Visual Studio Code provides an excellent platform for meaningful interaction and application development. There are many alternatives and feel free to choose the one that suits you best. It is a good practice to develop our app in a virtual environment. Suppose we are concurrently working on two projects, one that requires TensorFlow v1.5 and one that requires TensorFlow v2.0. The virtual environment provides us with a tool to keep the project and its dependencies isolated from each other. The Virtual Environment is recommended when you are working on any Python based project. Virtualenv, pipenv, hair, etc. are popular tools for creating a virtual environment. To start, I used WSL (Windows Subsystem for Linux) with ubuntu 18.04 as the environment for the project. I found the Linux environment in WSL a good and easy interface to develop the intended application. The following code will get a virtual environment for our project,
$ pip install virtualenv
$ mkdir demo
$ virtualenv demo
$ source activate demo/bin/activate
$ which python
The above code creates a virtual environment called experimental The thing that python shows is that we are using python from the virtual environment. (It will display the current directory plus/demo/bin/python). To deactivate the env type you can write our simple flask now and name it main.py.
From the vial import the vial
app = vial (__ name__)
Returns “Hello World”
if __name__ == “__main__”:
app.run (host = ‘0.0.0.0′, port = 5000, debug = true)
The code main.py imports the Flask object from the Flask package and creates an instance of the Flask application called app and passes variable (__name__) to it. The @app.route(‘/) is a python decorator which turns the python function into a view function that converts the function return value to HTTP response which can be displayed by a web browser. “/” indicates the function will respond at the main URL. Our index function returns a string “Hello World”. By default, main.py runs on localhost, host= ‘0.0.0.0’ tells to run on all your machine’s IP addresses and port tell the app will run port 5000 of the machine. Let’s run this app on the local machine,
$ pip install flask $ python3 main.py
We see an output that looks like this.
We can see the main.py output by going to the browser and typing the displayed IP address or by using the command $ curl localhost: 5000 on terminal
Our app appears to be running on the local machine. The station threw a warning “Warning: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead.”. Let’s understand what this means…
In short, if we want to run our app, we must choose a production server like unicorn, waitress etc. production because it does not scale well and by default only serves one request at a time.” We will use gonicorn, WSGI production server to publish our app and if you are using windows gonicorn may not work and Waitress It is the option to move. Uses Ctrl + c To stop the app from running earlier and execute the following commands on the device,
$ pip install gunicorn $ gunicorn --bind 0.0.0.0:5000 main:app
This indicates that our app is running and we can check the way we did before using the browser or curl command. amazing! Let’s go ahead.
Our app is in the Heroku container
Let’s move on to our app container. While Docker containers are a topic on their own, I will limit my words to a brief introduction. Docker is one of the Dev-Ops tools required to deploy applications. We can package and ship the container with the app and all of its dependencies without worrying about compatibility or device dependency issues. With Docker, applications can run no matter where they are.
The docker image is similar to a container diagram. The above diagram shows our process, we are writing a Docker file to create a custom image and run our app with the help of a container. Docker was first developed in the Linux environment but was later made available for windows and mac devices. We need WSL2 to use docker on windows and there are many resources online to help you download the docker environment for your device. Assuming we have a docker environment in our devices, let’s move forward.
$ docker --version
This will tell us that the docker is installed correctly or not. Before creating our image and running our container requirements. txt.
$ pip freeze > requirements.txt
This creates a text file with the required packages in the current directory. Create a file with the following content in the current directory and name it DockerfileAnd
FROM docker.io/python:3.7 WORKDIR /app COPY requirements.txt ./ RUN pip install --no-cache-dir -r requirements.txt COPY . . ENTRYPOINT ["gunicorn",""--bind","0.0.0.0:5000",main:app"]
The file uses a python 3.7 image as the base image to build our app. The commands to be executed are WORKDIR, COPY, RUN to create a working directory, copy files, and run pip install packages in requirements.txt. I specifically unicorn == 19.9.0 in requirements.txt where higher versions were giving errors. In the end ENTRYPOINT tells us how the container will run, here it will run the application in gunicorn server on localhost on port 5000. Once the Dockerfile is ready we can build and run our container, (make sure you are in the project directory!)
$ docker build -t demo-app .
This should keep the image building process going and the following command should show our image,
$ docker images
Run the container in localhost using the following commands in sequence,
$ docker run -dit -p 5000:5000 demo-app
$ docker ps
The above commands should show that our container ran successfully,
Docker sets the container’s id and name (if we don’t explicitly name it) and we set container port 5000 to device port 5000 during the run command. Now our container is running and can be checked again using the browser or via the curl command.
After our app is successfully running on localhost, let’s publish it to the cloud for others to use. I used Heroku to publish my app. Heroku is a platform as a service (PaaS) that enables developers to build, run, and run apps entirely in the cloud. They also offer a free account to host some of the apps. So sign up for an account and also download Heroku CLI to post.
This will confirm that the Heroku CLI is installed. The following sequence of commands will build the container on the Heroku cloud platform.
$ heroku login
This will give the message “Signed in…Done”
$ heroku container:login
$ heroku create
This will create an app and Heroku will give a name (if we don’t specify a specific name) and the corresponding link to the app. Note this app name for more commands,
$ heroku container:push web --app
This creates the container and pushes it to the cloud and gives a message that “Your image has been pushed successfully. You can now edit it with the “container: release” command.
$ heroku container:release web --app
Upon successful release, the message “Editing Images on the Web… Done” is displayed.
Let’s check that our app is working by visiting the link provided by Heroku…
there he is! congratulations! Our app is deployed to the cloud…
This article was an attempt to learn about developing a simple flask app, putting the app into a docker container and finally deploying it to the cloud using platform services. Hiroko. After deploying a very basic application, the next logical step is to scale the application to Flask to deploy a machine learning model. To be honest, while building the app, I stumbled upon several bugs and resources from the online community were instrumental in resolving the bugs and publishing the app.
“You don’t learn to walk by following the rules. You learn by doing and by falling.” – Richard Branson
About the author
Subramanian Hariharan is a marine engineer with over 30 years of experience who is passionate about leveraging data in business solutions.