+This [application](./resources) is composed of: +- a [PostgreSQL database](https://www.postgresql.org) (to run in a container) +- a [NestJS](https://nestjs.com) API +- a [React](https://reactjs.org) web application + +Here's a simple schema of the application architecture: + + + +### Lay the foundations + +In the directory `step2`, copy the [frontend](./resources/frontend.zip) +and [backend](./resources/backend.zip) zips and extract them. + +You should end up with the following architecture: + +```shell +tree -d +# . +# ├── backend +# │ ├── src +# │ │ └── notes +# │ └── test +# └── frontend +# ├── cypress +# ├── public +# └── src +# ├── components +# │ ├── List +# │ ├── Task +# │ └── TaskForm +# ├── dto +# └── services +# +# 13 directories +``` + +### Dockerfiles + +As you may notice, there is no `Dockerfile` in `frontend` or `backend`.
+You figured out, you will need to create images for the `frontend` and the `backend`.
+In each directory, create a `Dockerfile` and write a set of instructions to +make it work. + +> 💡 You will need to test and understand how each program work to containerize it.
+> Google is also your best friend, as a DevOps engineer it's common to +> work on multiple stack without necessarily being an expert on them. + +### Compose + +[Docker compose](https://docs.docker.com/compose/compose-file/) aims to +orchestrate services. It will help you a lot when you need to deploy +multiple [microservices](https://aws.amazon.com/microservices/). + +Your `docker-compose` file must be composed of: + +3 services: +- backend: run the `backend` image +- frontend: run the `frontend` image +- database: run the `postgres` image + +1 network: +- backend: link database & backend in a private network + +1 volume: +- db-data: store database permanent data + +To complete this step, you'll need to use all the knowledge acquired during +previous day. It will be important to set an environment to correctly +configure the database, backend and frontend in the `docker-compose`. + +> Don't forget to exposed ports and link your services 😄 + +> This step is voluntarily complex to make you search and understand +> by yourself, this is what real DevOps engineers do 🚀 + +## Additional resources + +- [Optimize your Dockerfile](https://medium.com/@tonistiigi/faster-multi-platform-builds-dockerfile-cross-compilation-guide-part-1-ec087c719eaf) +- [Buildkit, the future of docker build](https://github.com/moby/buildkit) +- [Scale to the moon with Kubernetes](https://kubernetes.io) +- [Create a Swarm of container with DockerSwarm](https://docs.docker.com/engine/swarm/) +- [Improve your stack deployment with Helm](https://helm.sh) +- [A GUI to manage your container: LazyDocker](https://github.com/jesseduffield/lazydocker) +- [Infra as code with Terraform](https://registry.terraform.io) + +
+Organization +
++ +
+ 
+ 
+ 
+
+
+ 
+> If you use another OS, please, follow this [documentation](https://docs.docker.com/engine/install/) + +#### Clean up + +Let's start by removing old unused dependencies + +```shell +sudo dnf remove docker \ + docker-client \ + docker-client-latest \ + docker-common \ + docker-latest \ + docker-latest-logrotate \ + docker-logrotate \ + docker-selinux \ + docker-engine-selinux \ + docker-engine +``` + +#### Installation + +We can now install docker engine and the compose plugin: + +```shell +# Add docker repository to your package manager +sudo dnf -y install dnf-plugins-core +sudo dnf config-manager --add-repo https://download.docker.com/linux/fedora/docker-ce.repo + +# Install Docker +sudo dnf install docker-ce docker-ce-cli containerd.io + +# Install docker compose +sudo dnf install docker-compose-plugin + +### Start Docker + +After installing all the dependencies, start docker engine: + +```shell +# Start docker service +sudo systemctl start docker + +# Optional: Automatically start it when on computer start +sudo systemctl enable docker +``` + +The last thing to do for a seamless experience is to [enable the use of Docker without a root user](https://docs.docker.com/engine/install/linux-postinstall/#manage-docker-as-a-non-root-user): + +```shell +# Add current user to docker group to use docker CLI without sudo +sudo groupadd docker +sudo usermod -aG docker $USER + +# Logout from your user session or restart your computer +``` + +#### Troubleshooting + +If you are on Fedora 31+, you will certainly meet this issue: + +```shell +docker: Error response from daemon: cgroups: cannot found cgroup mount destination: unknown. +``` + +If that happens, read this [issue](https://github.com/docker/for-linux/issues/219) +and try the fixes in it. + +> If you still struggle, don't hesitate to ask the staff to help you 😄 + +## Check your installation + +You can verify that everything goes well by running a [`hello-world`](https://hub.docker.com/_/hello-world) +container. + +You should have the following output: + +```shell +docker run hello-world +# Unable to find image 'hello-world:latest' locally +# latest: Pulling from library/hello-world +# 2db29710123e: Pull complete +# Digest: sha256:18a657d0cc1c7d0678a3fbea8b7eb4918bba25968d3e1b0adebfa71caddbc346 +# Status: Downloaded newer image for hello-world:latest +# +# Hello from Docker! +# This message shows that your installation appears to be working correctly. +# +# To generate this message, Docker took the following steps: +# 1. The Docker client contacted the Docker daemon. +# 2. The Docker daemon pulled the "hello-world" image from the Docker Hub. +# (amd64) +# 3. The Docker daemon created a new container from that image which runs the +# executable that produces the output you are currently reading. +# 4. The Docker daemon streamed that output to the Docker client, which sent it +# to your terminal. +# +# To try something more ambitious, you can run an Ubuntu container with: +# $ docker run -it ubuntu bash +# +# Share images, automate workflows, and more with a free Docker ID: +# https://hub.docker.com/ +# +# For more examples and ideas, visit: +# https://docs.docker.com/get-started/ +``` + + +## Back to the workshop + +[Jump !](./README.md) \ No newline at end of file diff --git a/Software/day04/1 - Docker/resources/backend.zip b/Software/day04/1 - Docker/resources/backend.zip new file mode 100644 index 0000000..fd75315 Binary files /dev/null and b/Software/day04/1 - Docker/resources/backend.zip differ diff --git a/Software/day04/1 - Docker/resources/frontend.zip b/Software/day04/1 - Docker/resources/frontend.zip new file mode 100644 index 0000000..f8cab60 Binary files /dev/null and b/Software/day04/1 - Docker/resources/frontend.zip differ diff --git a/Software/day04/2 - Deployment/README.md b/Software/day04/2 - Deployment/README.md new file mode 100644 index 0000000..91cd9dd --- /dev/null +++ b/Software/day04/2 - Deployment/README.md @@ -0,0 +1,215 @@ +# PoC Software Pool 2024 - Day 04 - Deployment + +**Day purposes** + +✔️ Deploy an entire application + +✔️ Use a cloud instance + +✔️ Discover GitHub Actions + +✔️ Setup a CI/CD pipeline + +## Introduction + +[This morning](../1%20-%20Docker/) you discovered Docker, a very useful and popular tool to package your application into containers. That's great, but you still need to start these containers *somewhere* reachable by someone who wants to access your app. + +This can be quite complicated if you want to do it by yourself on your computer: +- It should be up and running 24/7 +- You should make sure it's secured and up to date in case of vulnerabilities +- It can impact the performances of your other apps +- Dozens of other problems... + +That's where cloud providers like [AWS](https://aws.amazon.com/) or [Scaleway](https://www.scaleway.com/) 🇫🇷 come into play: they offer cloud-based services like data storage or computing power. + +They handle the hardware, maintenance and security concerns to let you focus on what's really important for you as a developer: shipping your application to users 🚀 + +> 💡 If you are curious, check [this link](https://www.bmc.com/blogs/saas-vs-paas-vs-iaas-whats-the-difference-and-how-to-choose/) to learn about the different types of services. + +Today you will use [Scaleway Instances](https://www.scaleway.com/en/virtual-instances/) which are computing units installed with an OS of your choice, on which you can then connect via SSH and use as you want 🚀 + +Then you will discover [GitHub Actions](https://github.com/features/actions) to run tests and update your deployed application whenever you make change to your code 🔥 + + +## Step 0 - Setup + +Please refer to the [SETUP.md](./SETUP.md) file. + +## Step 1 - Copy app files & run it 📁 + +### 📑 **Description** + +Now that you know how to connect to your virtual instance, you have to add your application files and run it 🔥
+Here you can use the same application as [this morning](../1%20-%20Docker/resources/) for which you already created a `docker-compose.yml` file for an easy startup 👍 + +### 📌 **Tasks** + +- Find a command to copy a local folder to a remote computer via SSH +- Move your application code to the instance, for example in the `/home/ubuntu/myapp` folder +- Connect to the instance and run your application + - You may need to install docker, but now you know how to do it 🚀 + +> Don't forget to update the necessary variables in the `.env` files 😉 + +### 📚 **Documentation** + +- [Secure Copy Protocol](https://en.wikipedia.org/wiki/Secure_copy_protocol) + +### ✔️ **Validation** + +If everything went well, you can open your browser at `http://
+Unlike you run it on `localhost`, here it's accessible to anyone 😉 + + +## Step 2 - Your first GitHub Action ⚙️ + +### 📑 **Description** + +Deploying an application manually is cool, but DevOps is also about automating processes.
+Several tools exists to automate all kinds of workflows, from testing to deployment, and today you'll use [GitHub Actions](https://github.com/features/actions). + +An Action is basically composed of several jobs (like `docker compose` services 😉) that can be triggered on events happening in your repository (a push, a PR opened...). These jobs will run in a virtual machine provided by GitHub with several OS choices. + +No better way to discover it than to practice, let's create your first GitHub Action 🚀 + +### 📌 **Tasks** + +- Create a `.github/workflows` folder at the root of your repository +> Commonly, the `.github` folder can also contain an `assets` directory that stores your images used in `README`s, or the `ISSUE_TEMPLATE` folder that is useful to setup templates for features, bug reports... + +- Create a `hello.yaml` file in `.github/workflows` with the following properties: + - It should be named `Hello world` + - It should be triggered: + - When there is a push on any branch + - When an issue is opened on the repository + - It should have one job called `hello-world` that: + - Is executed on a Ubuntu 22.04 machine + - Has a step named `Display a greeting` that displays (with the `echo` command) the following message:
+ `Hi from GitHub Action, I'm running on
+Make a push and open an issue to check if the action is correctly triggered and if the message displayed is correct 😄 + + +## Step 3 - Run tests in Action 🧪 + +### 📑 **Description** + +Good job creating and running your first GitHub Action 🥳\ +Now that you are familiar with it's syntax, let's go further by implementing a Continuous Integration: the objective is to make sure our project keeps working as intended while we keep developing it. + +That's why here we will create an Action to run some tests on every commits, and retrieve the results for a simplified debugging if they fail 😉 + +For this exercise, you have to use the [Cypress](https://cypress.io/) tests contained in the frontend of your application. + +> Cypress is a tool to run all kind of tests on your frontend, you'll discover it tomorrow if you choose the React option 😉 + +### 📌 **Tasks** + +- Create a `ci-tests.yaml` action with the following parameters: + - It should be triggered on every push and pull request + - It should contain a single job named `front-cypress` + - Run the frontend and the tests + - Upload the screenshots and videos as artifacts if the tests fail + +### 📚 **Documentation** + +- [Continuous Integration with Cypress](https://docs.cypress.io/guides/continuous-integration/introduction) +- [GitHub Actions Marketplace](https://github.com/marketplace/actions) +- [Workflow Artifacts](https://docs.github.com/en/actions/using-workflows/storing-workflow-data-as-artifacts) + +### ✔️ **Validation** + +To validate your work, you can check the logs of your action to see if they contain the tests output. + +Then, change the title of the application in `src/App.tsx` to make the tests fail and make sure the artifacts are present in the Action run. + +
+
+
+Example of success output
+ +  +
+
+Example of failure output
+ +  ++ + +## Step 4 - CD on the Scaleway instance 🔀 + +### 📑 **Description** + +Well done, now you have a way to automatically keep your app stable 🎉\ +But if it's working correctly you can probably deploy it (directly in production to your users, or in a development environment for a future version release for example). + +### 📌 **Tasks** + +Your objective for this step is to combine the knowledge you've acquired from the previous steps to create a GitHub Action that will deploy the application (frontend and backend) to your Scaleway instance when there's a commit on your `main` branch + +This means that you need to: +- Have some credentials to connect to the instance +- Upload the code +- Set environment variables if needed +- Rebuild the app with docker + +> This step is intentionally less guided as you should have all the basic knowledge to be able to complete it, but don't hesitate to ask the staff for help if you struggle 😉 + +### 📚 **Documentation** + +- [Using Secrets with Actions](https://docs.github.com/en/actions/security-guides/security-hardening-for-github-actions#using-secrets) + +### ✔️ **Validation** + +Just like in the first step, you can access your app at `http://
+Organization +
++ +
+
+
+We've already done this for you today, so you only have to connect to it 🔥 + +### 📌 **Tasks** + +Go see a supervisor to have: + - An IP address + - A private SSH key + +Connect to the instance through SSH using the above information + +### 📚 **Documentation** + +- [Scaleway Instances Quickstart](https://www.scaleway.com/en/docs/compute/instances/quickstart/) +- [What is SSH and how to use it](https://www.ucl.ac.uk/isd/what-ssh-and-how-do-i-use-it) + +### ✔️ **Validation** + +Once you are connected to the instance, you can check the machine information with the following command: +```sh +hostnamectl +# Operating System: Ubuntu 22.04.1 LTS +# Kernel: Linux 5.15.0-53-generic +# Architecture: x86-64 +# Hardware Vendor: Scaleway +# Hardware Model: SCW-DEV1-S +``` + +You can then `exit` to go back to your terminal 😄 + +## Back to the workshop + +[Jump !](./README.md) + + +