IEMAP RESTful/GraphQL API

General Informations

This API allow you to do some CRUD operations with RESTful methods over the Mission Innovation mongoDB.

This API can run in development mode in you local machine or deployed as containerized production-ready service on your server and/or on common public cloud providers.

Official site

You can find the last working API version on the official site: iemap.enea.it

Official documentation

All routes are described on:

• iemap.enea.it/docs with Swagger
• iemap.enea.it/redoc with ReDoc.

A complete documentation is available on ReadTheDocs.

Project structure

All files related to the application are in the app directory into the following subfolders:

• models: pydantic models that used in crud or handlers
• crud: CRUD for types from models (create new user/article/comment, check if user is followed by another, etc)
• db: db specific utils
• core: some general components (jwt, security, configuration)
• api: handlers for routes
• main.py: FastAPI application instance, CORS configuration and api router including

Installation

Get the code

First of all you need to get the code:

git clone https://github.com/ai4mat/iemap-api.git

and jump to its folder:

cd iemap-api

Make some configurations

You first need to setup configurations into the environments file. Copy the env.sample into .env and edit this file for each variable.

Start for development

1 - Export the variables

First of all you need to export variables into the environment with:

export $(xargs < .env)

After that you need to create the python environment. You can choose to use pip or poetry.

2a - Setup python environment with pip

Create the virtualenv (assuming you have python 3.X) and activate it:

python3 -m venv <your-virtual-env>
source <your-virtual-env>/bin/activate

Then install requirements:

pip install -r requirements

2b - Setup python environment with poetry

Install poetry:

curl -sSL https://install.python-poetry.org | python3 -

Run the following commands to bootstrap your environment with poetry:

poetry install
poetry shell

3 - Run the server

Now you're ready to start the API just with:

cd app/
uvicorn main:app --reload

Note to run on SSL with a self-signed certificate use:
uvicorn main:app --host 0.0.0.0 --port 8001 --reload --ssl-keyfile ~/Downloads/nginx-selfsigned.key --ssl-certfile ~/Downloads/nginx-selfsigned.crt --log-level debug --log-config log_conf.yml --proxy-headers

Run as container (Production)

0 - Prerequisites

In the following we are assuming that you can manage docker with a non-root user. To do so, run the following commands:

sudo groupadd docker
sudo usermod -aG docker $USER

You had created the docker group first and then added your user to it. This way now you can build, run and stop containers with your user, without worrying about sudo.

1 - Configuration

Into the .env file, you need to set:

FILESDIR=data

That is the default folder inside the container to store files. That folder is created automatically when you build the container, and mounted automatically when you run it with the external host folder specified as the following:

export HOST_FILESDIR=<absoloute path where uploaded files are stored>

You may prefer to store this into the .bashrc or .profile.

2 - Build image and run container

Run the following command to build the image and run the container:

make all

You can also run multiple containers from the same builded image. You need to build first and then run each container on different port. To do so, run the following command:

make build

And then run each container:

make HOST_PORT=<port> run

In the following a complete list of commands defined into the Makefile, to simplify container managment:

Action	command
Build and run	make all
Build image	make build
Run container	make run
Stop container	make stop
Start container	make start
Kill (stop & remove) container)	make kill
Clean (remove eventually dead containers and remove images))	make clean

Remeber: to get the list of running containers (with their IDs), run:

docker ps

3 - Configure NGINX as reverse proxy

Create a new virtual host in your /etc/nginx/sites-available folder and add the following configuration (supposing you are running with SSL/TLS encryption):

server {
    listen 80;
    server_name <your-domain-name>;
    return 301 https://$server_name$request_uri;
}

server {
    listen 443 ssl;
    server_name <your-domain-name>;

    ssl_certificate /etc/letsencrypt/live/<your-domain-name>/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/<your-domain-name>/privkey.pem;

    location / {
        proxy_pass http://127.0.0.1:8000;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_set_header X-NginX-Proxy true;
        proxy_redirect off;
        proxy_pass_request_headers on;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
    }
}

3 Bis - Configure NGINX as load balancer

If you're running multiple containers on the same server, you can configure NGINX as load balancer. To do so, you need to create a new virtual host in your /etc/nginx/sites-available folder and add the following configuration:

upstream backend {
    least_conn;
    server 127.0.0.1:<port1>;
    server 127.0.0.1:<port2>;
    ...
}

server {
    listen 80;
    server_name <your-domain-name>;
    return 301 https://$server_name$request_uri;
}

server {
    listen 443 ssl;
    server_name <your-domain-name>;

    ssl_certificate /etc/letsencrypt/live/<your-domain-name>/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/<your-domain-name>/privkey.pem;

    location / {
        proxy_pass http://backend;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_set_header X-NginX-Proxy true;
        proxy_redirect off;
        proxy_pass_request_headers on;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
    }
}

Please note that we have configured the load balancer with the Least connections algorithm. This means that the server with the least connections will be used. If you want to use the Round-Robin algorithm, you can change the least_conn in the upstream definition to round_robin.

4 - Check and restart NGINX

Check the configuration and activate the new virtual host:

sudo nginx -t

If the check is ok, then create the symbolic link into the /etc/nginx/sites-enabled folder:

ln -s /etc/nginx/sites-available/<your-vhost-name> /etc/nginx/sites-enabled/<your-vhost-name>

Then restart the server:

systemctl restart nginx

Check API

1a - Check if the API is running locally

curl http://localhost:8000

1b - Check if the API is running on the server (with SSL/TLS encryption)

curl https://<server-hostname>

2 - Expected behavior

If all is working properly, you'll get this output:

{
  "request_method": "GET",
  "path_name": "",
  "message": "Reply from IEMAP API at <current time and date>"
}

Credits

• Sergio Ferlito ([email protected]) for the development and optimization of the API.
• Marco Puccini ([email protected]) for the initial idea, the first implementation and the DevOps activities.
• Claudio Ronchetti ([email protected]) for data model and general support.