Deploy a WebApp to interact with your Python scripts on the Cloud

You have created a python script, it crunches data, interacts with inputs… now you want to make it work inside a website. This is the shortest path to have a WebApp running on the cloud to interface your scripts. Django is chosen because it’s a batteries-included framework, it includes most of the tools needed out of the box. With some additional tools, you can run a top-notch application in no time.

Django

An awesome framework, used by many big websites, usable in minutes.

Install Django

pip install django

Create a project

django-admin startproject projectname

Test with the built-in server

python manage.py runserver

This is not suitable for deployment, but it’s very convenient for development. If it works, it will be available at http://127.0.0.1:8000/

As recommended by the command prompt, you should now:

python manage.py migrate

 

Django apps get data from the model, then a view uses a template to show you the information the way you want.

Add an app of your own to the / (home) directory

python manage.py startapp myownapp

Open urls.py from the main folder and add:

from django.conf.urls import url, include
from django.contrib import admin

urlpatterns = [
 url(r'^admin/', admin.site.urls),
 url(r'^/', include('myownapp.urls')),
]

In the same folder, add the app name to the settings.py app list:

Now, go to the /myownapp folder and create urls.py like;

from django.conf.urls import url
from myownapp import views

urlpatterns = [
    url(r'^$', views.TestView.as_view()),
]

Create a View

In myownapp/views.py add the TestView view.

from django.shortcuts import render
from django.views.generic import TemplateView

# Create your views here.
class TestView(TemplateView):
 def get(self, request, **kwargs):
 return render(request, 'index.html', context=None)

According to the paradigm explained before, here we see that:

A get request, returns a response, which is a render of the template index.html .

Create a myownapp/templates folder and a new file index.htm :

<!-- howdy/templates/index.html -->
<!DOCTYPE html>
<html>
 <head>
 <meta charset="utf-8">
 <title>My Own App</title>
 </head>
 <body>
 <h1>Hello from the other side.</h1>
 </body>
</html>

 

Go to  http://127.0.0.1:8000/  and now you should see your website running!

 

 

 

 

 

Django REST Framework

The current trend tends to make completely isolated User Interfaces which interface the backend only through an API.

pip install django djangorestframework

 

Create a new project

django-admin startproject todo_api

Add DRF (and any new app you create) to the installed apps in settings.py

INSTALLED_APPS = (
    ...
    'rest_framework',
)

Run the test server (on localhost:8000 )

python src/manage.py runserver

 

Related models and views should have a separate app:

python manage.py startapp candles

Once you define new models inside candles/models.py the DB is migrated

python manage.py makemigrations candles
python manage.py migrate

 

It’s a good practice to create tests for the models in /test , this will create a test database and perform the tests on it.

python manage.py test

 

Serializers

Translate data between models and JSON which requests work with. They should be in appname/serializers.py and have their own tests.

 

Views

Define what each API endpoint does. They are basically functions that you define to update or retrieve data. In appname/views.py you define classes and their methods to interact with models through serializers.

 

URLs

Now it’s time to map those views to URLs using regex. This is done in projectname/urls.py

In Django, named capturing groups are passed to your view as keyword arguments. Unnamed capturing groups (just a parenthesis) are passed to your view as arguments.

 

 

 

Add safe user authentication to DRF

We are adding some packages for enhanced auth:

pip install django-rest-auth djangorestframework-jwt django-allauth
INSTALLED_APPS = (
    ...
    'rest_framework',
    'rest_auth',
    'django.contrib.sites',
    'allauth',
    'allauth.account',
    'rest_auth.registration',
)

Add the settings in settings.py

# Configure the JWTs to expire after 1 hour, and allow users to refresh near-expiration tokens
JWT_AUTH = {
 'JWT_EXPIRATION_DELTA': datetime.timedelta(hours=1),
 'JWT_ALLOW_REFRESH': True,
}

# Make JWT Auth the default authentication mechanism for Django
REST_FRAMEWORK = {
 'DEFAULT_AUTHENTICATION_CLASSES': (
 'rest_framework_jwt.authentication.JSONWebTokenAuthentication',
 ),
}

# Enables django-rest-auth to use JWT tokens instead of regular tokens.
REST_USE_JWT = True


# required for the django.contrib.sites dependency.
SITE_ID = 1

Add the following URLs

from rest_framework_jwt.views import refresh_jwt_token
...
# authentication
 url(r'^', include('rest_auth.urls')),
 url(r'^registration/', include('rest_auth.registration.urls')),
 url(r'^refresh-token/', refresh_jwt_token),
...

 

Using bcrypt

In settings.py use this order of hashers

PASSWORD_HASHERS = [
 'django.contrib.auth.hashers.BCryptSHA256PasswordHasher',
 'django.contrib.auth.hashers.BCryptPasswordHasher',
 'django.contrib.auth.hashers.PBKDF2PasswordHasher',
 'django.contrib.auth.hashers.PBKDF2SHA1PasswordHasher',
 'django.contrib.auth.hashers.Argon2PasswordHasher',
]

 

Use authentication in Views

To take advantage of authentication, Views will behave differently if authenticated. The most common things you want to know are:

  • request.auth   Will be None if not authenticated, or the token if authenticated.
  • request.user Will be AnonymousUser if not authenticated, or the username if authenticated.

Use permissions in Views

You can easily restrict a View with predefined or custom permissions that depend on auth status.

class ExampleView(APIView):
    permission_classes = (IsAuthenticated,)

Typical permissions are:

IsAuthenticated , IsAuthenticatedOrReadOnly

They can be checked programmatically

if request.method in permissions.SAFE_METHODS:
    # Check permissions for read-only request
else:
    # Check permissions for write request

And you can build Custom Permissions deriving the base class .has_permission(self, request, view) method:

class TestCustomPermission(permissions.BasePermission):
 """
 Global permission check.
 """

def has_permission(self, request, view):
 return request.user.get_username() == 'myuser'
permission_classes = (permissions.IsAuthenticated, TestCustomPermission)

 

 

 

 

 

 

Frontend: VueJS

 

Vue js is an awesome javascript framework to build reactive front-ends. It is very well-thought and gaining popularity quickly.

 

Some troubleshooting when deploying your webapp:

  • You might need to create a symlink node -> nodejs (in Ubuntu and Debian).
  • Then, run sudo npm install

 

If you plan to run it on an ARM board, such as a Raspberry-Pi, you must download and install the tarballs directly:

sudo wget https://nodejs.org/dist/v8.11.3/node-v8.11.3-linux-armv6l.tar.xz

sudo tar -xf node-v8.11.3-linux-armv6l.tar.xz

cd node...
sudo cp -R * /usr/local/

 

 

 

Deploying on AWS

 

These are the guides that will provide step-by-step guidance for each part of the process.

 

Additionally, the following tips might save you some head-scratching:

Login to AWS EC2 instance (also valid for SFTP connection)

1- Download the private key file (don’t lose it, it can not be downloaded again).

2- Setup your Mobaxterm or Putty terminal like this:

The host can be found in the EC2 Management Console, under Public DNS.

3- You will need a username, these are the default ones:

  • For Amazon Linux 2 or the Amazon Linux AMI, the user name is ec2-user.
  • For a Centos AMI, the user name is centos.
  • For a Debian AMI, the user name is admin or root.
  • For a Fedora AMI, the user name is ec2-user or fedora.
  • For a RHEL AMI, the user name is ec2-user or root.
  • For a SUSE AMI, the user name is ec2-user or root.
  • For an Ubuntu AMI, the user name is ubuntu.

 

Python versions

Cloud providers have the nasty habit of providing Python 2.7 by default, which is deprecated, instead of Python 3.

Note that you may need to run your scripts with python3 filename.py and install packages with pip3 install –user packagename.

 

Test Django servers

You will need to provide Inbound rules in AWS to allow traffic in ports 80, 443 and the port where your API is listening (8000 by default). Then you have to run it specifying IP 0.0.0.0 (not localhost), so that it is reachable from anywhere.

manage.py runserver 0.0.0.0:8000

 

Use Supervisor to auto-start and auto-restart processes

This is an easy way of handling the multiple commands that you have to run every time your machine starts. Here is an example working with Gunicorn:

http://rahmonov.me/posts/run-a-django-app-with-nginx-gunicorn-and-supervisor/

 

Install MongoDB

First, install mongo and it’s packages. Then, create the admin database and the root user. Once created, edit /etc/mongod.conf and add:

security:
  authorization: enabled

Now, you can login with the root user, and create other users in/for other databases:

mongo -u rootuser -p rootpassword --authenticationDatabase admin
use test
db.createUser(
  {
    user: "myTester",
    pwd: "xyz123",
    roles: [ { role: "readWrite", db: "test" },
             { role: "read", db: "reporting" } ]
  }
)

 

 

 

Easy and Free HTTPS

Cloudflare provides a lot of free services, including https encryption. You just have to add your site, and point the nameservers in your domain provider to Cloudflare’s nameservers.

If you are using AWS, you can find the IP where your Cloudfare records should point in the dashboard:

Adding https redirection is very easy.

Finally, you can generate origin certificates, and there are instructions for each type of server.

 

 

 

Mirror your databases for offline testing

 

Install mongodb server on your development machine.

On the deployment machine, run:

mongodump -u rootuser -p rootpassword --authenticationDatabase admin

 

Download the generated dump folder to the development machine. In the dev machine run:

mongod

And in a separate terminal:

mongorestore path/to/dump/

 

Now you can connect to the local instance, which will have an exact replica of the deployed databases, and test using real data.