From 14acfe64fddf3068c6b2658b258280292f733dc5 Mon Sep 17 00:00:00 2001 From: Mateus Vieira <68292695+mateusvrs@users.noreply.github.com> Date: Fri, 1 Dec 2023 10:50:08 -0300 Subject: [PATCH] devops(deploy): configura o ambiente de deploy para o heroku (#131) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * devops(docker): muda entrypoint de pasta * devops(requirements): adiciona deps de produção Co-authored-by: HenrikhKenino <110349110+HenrikhKenino@users.noreply.github.com> * devops(make): arruma caminho do entrypoint * devops(heroku): arquivos de config do deploy * devops(heroku): separa arquivos de config dev/prod Co-authored-by: HenrikhKenino <110349110+HenrikhKenino@users.noreply.github.com> * devops(heroku): configura domínios permitidos Co-authored-by: HenrikhKenino <110349110+HenrikhKenino@users.noreply.github.com> * devops(core): add redirect ssl to django Co-authored-by: HenrikhKenino <110349110+HenrikhKenino@users.noreply.github.com> * devops(updatedb): cria melhor descrição de erro * docs(executing): docs seguir novo padrão de pastas * git(workflows): adiciona nomes aos jobs do actions --------- Co-authored-by: HenrikhKenino <110349110+HenrikhKenino@users.noreply.github.com> --- .github/workflows/docs.yml | 1 + .github/workflows/test.yml | 1 + Makefile | 4 +- README.md | 19 ++++---- api/.env.example | 2 +- api/Procfile | 2 + api/api/management/commands/updatedb.py | 17 +++++-- api/{ => config}/entrypoint.sh | 0 api/core/settings/base.py | 21 --------- api/core/settings/dev.py | 26 +++++++++-- api/core/settings/prod.py | 56 +++++++++++++++++++++++- requirements.txt => api/requirements.txt | 15 +++---- api/runtime.txt | 1 + docker/Dockerfile | 10 ++--- docs/executing.md | 56 ++++++++++-------------- 15 files changed, 141 insertions(+), 90 deletions(-) create mode 100644 api/Procfile rename api/{ => config}/entrypoint.sh (100%) rename requirements.txt => api/requirements.txt (82%) create mode 100644 api/runtime.txt diff --git a/.github/workflows/docs.yml b/.github/workflows/docs.yml index 95960b61..58224f66 100644 --- a/.github/workflows/docs.yml +++ b/.github/workflows/docs.yml @@ -10,6 +10,7 @@ permissions: jobs: docs: + name: mkdocs deploy runs-on: ubuntu-latest strategy: max-parallel: 4 diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml index ac537ba6..c906bd87 100644 --- a/.github/workflows/test.yml +++ b/.github/workflows/test.yml @@ -11,6 +11,7 @@ permissions: jobs: test: + name: api test coverage runs-on: ubuntu-latest steps: diff --git a/Makefile b/Makefile index 5285e03a..1fbaea4a 100644 --- a/Makefile +++ b/Makefile @@ -13,11 +13,11 @@ config-mock: bash scripts/config.sh entrypoint-chmod: - chmod +x ./api/entrypoint.sh + chmod +x ./api/config/entrypoint.sh # Install Dependencies install: - pip install -r requirements.txt + pip install -r ./api/requirements.txt npm install --prefix ./web diff --git a/README.md b/README.md index 1922d474..36b7bcb9 100644 --- a/README.md +++ b/README.md @@ -2,9 +2,9 @@ [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](./LICENSE) [![codecov](https://codecov.io/gh/unb-mds/2023-2-Squad11/branch/main/graph/badge.svg?token=ZQZQZQZQZQ)](https://codecov.io/gh/unb-mds/2023-2-Squad11) -[![GitHub issues](https://img.shields.io/github/issues/unb-mds/2023-2-Squad11)]() -[![GitHub contributors](https://img.shields.io/github/contributors/unb-mds/2023-2-Squad11)]() -[![GitHub stars](https://img.shields.io/github/stars/unb-mds/2023-2-Squad11)]() +[![GitHub issues](https://img.shields.io/github/issues/unb-mds/2023-2-Squad11)](https://img.shields.io/github/issues/unb-mds/2023-2-Squad11) +[![GitHub contributors](https://img.shields.io/github/contributors/unb-mds/2023-2-Squad11)](https://img.shields.io/github/contributors/unb-mds/2023-2-Squad11) +[![GitHub stars](https://img.shields.io/github/stars/unb-mds/2023-2-Squad11)](https://img.shields.io/github/stars/unb-mds/2023-2-Squad11) [![Hit Counter](https://views.whatilearened.today/views/github/unb-mds/2023-2-Squad11.svg)](https://views.whatilearened.today/views/github/unb-mds/2023-2-Squad11.svg)
@@ -31,11 +31,15 @@ O projeto é software livre e está sob a licença [MIT](./LICENSE). - [💻 Ambiente](#-ambiente) - [📁 Dependências do projeto](#-dependências-do-projeto) - [💾 Execução](#-execução) + - [Observações do Docker](#observações-do-docker) - [✅ Autenticação do Google OAuth](#-autenticação-do-google-oauth) - - [🖱️ Acesso aos serviços](#-acesso-aos-serviços) + - [🖱️ Acesso aos serviços](#️-acesso-aos-serviços) - [📍 Migrations](#-migrations) - [📚 Documentação](#-documentação) - [📎 Extra](#-extra) + - [Story Map e Activity Flow](#story-map-e-activity-flow) + - [Arquitetura](#arquitetura) + - [Protótipo](#protótipo) ## 👥 Equipe @@ -79,10 +83,10 @@ Para instalar as dependências do projeto, você pode rodar os seguintes comando ```bash # Crie um ambiente virtual Python -python3 -m venv env +python3 -m venv api/env # Ative o ambiente virtual -source env/bin/activate +source api/env/bin/activate # Instale os pacotes do Python e Node make install @@ -136,7 +140,6 @@ Adicionando serviços: - IAM Service Account Credentials API - Identity and Access Management (IAM) API - ### 🖱️ Acesso aos serviços | Serviço | URL | @@ -174,4 +177,4 @@ A documentação do projeto pode ser encontrada clicando [aqui](https://unb-mds. ### Protótipo -- Para acessar o protótipo do projeto, clique [aqui](https://www.figma.com/proto/o5Ffh1fWmmQz7KcDGuHrVP/Sua-grade-UNB?type=design&node-id=16-2775&scaling=scale-down&page-id=0%3A1&mode=design&t=L5JwoVdZsjyLBGdb-1). \ No newline at end of file +- Para acessar o protótipo do projeto, clique [aqui](https://www.figma.com/proto/o5Ffh1fWmmQz7KcDGuHrVP/Sua-grade-UNB?type=design&node-id=16-2775&scaling=scale-down&page-id=0%3A1&mode=design&t=L5JwoVdZsjyLBGdb-1). diff --git a/api/.env.example b/api/.env.example index eba198d9..430ea204 100644 --- a/api/.env.example +++ b/api/.env.example @@ -2,7 +2,7 @@ # Gerar secret key: # python3 -c 'from django.core.management.utils import get_random_secret_key; print(get_random_secret_key())' DJANGO_SECRET_KEY="your_secret_key" -SETTINGS_FILE_PATH="core.settings.base" +SETTINGS_FILE_PATH="core.settings.dev" # Banco de Dados DB_ENGINE="django.db.backends.postgresql" diff --git a/api/Procfile b/api/Procfile new file mode 100644 index 00000000..37593051 --- /dev/null +++ b/api/Procfile @@ -0,0 +1,2 @@ +release: python3 manage.py migrate +web: gunicorn core.wsgi \ No newline at end of file diff --git a/api/api/management/commands/updatedb.py b/api/api/management/commands/updatedb.py index 7e59bcd6..72368c82 100644 --- a/api/api/management/commands/updatedb.py +++ b/api/api/management/commands/updatedb.py @@ -17,7 +17,8 @@ def add_arguments(self, parser: CommandParser) -> None: help="Atualiza o banco de dados com as disciplinas dos períodos atual e seguinte.") parser.add_argument('-p', '--period', action='store', default=None, - choices=[".".join(sessions.get_current_year_and_period()), ".".join(sessions.get_next_period())], + choices=[".".join(sessions.get_current_year_and_period()), ".".join( + sessions.get_next_period())], dest='period', help="Atualiza o banco de dados com as disciplinas do período especificado.") parser.add_argument('-d', '--delete', action='store_true', dest='delete', default=False, @@ -57,18 +58,26 @@ def handle(self, *args: Any, **options: Any): print("Atualizando o banco de dados...") for year, period in choices: - start_time = time() - self.update_departments(departments_ids=departments_ids, year=year, period=period) - self.display_success_update_message(operation=f"{year}/{period}", start_time=start_time) + try: + start_time = time() + self.update_departments( + departments_ids=departments_ids, year=year, period=period) + self.display_success_update_message( + operation=f"{year}/{period}", start_time=start_time) + except Exception as exception: + print("Houve um erro na atualização do bando de dados.") + print(f"Error: {exception}") def update_departments(self, departments_ids: list, year: str, period: str) -> None: """Atualiza os departamentos do banco de dados e suas respectivas disciplinas.""" for department_id in departments_ids: + print(f"WebScraping do departamento: {department_id}") disciplines_list = web_scraping.get_department_disciplines( department_id=department_id, current_year=year, current_period=period) department = get_or_create_department( code=department_id, year=year, period=period) + print("Atualizando disciplinas do departamento...") # Para cada disciplina do período atual, deleta as turmas previamente cadastradas e cadastra novas turmas no banco de dados for discipline_code in disciplines_list: classes_info = disciplines_list[discipline_code] diff --git a/api/entrypoint.sh b/api/config/entrypoint.sh similarity index 100% rename from api/entrypoint.sh rename to api/config/entrypoint.sh diff --git a/api/core/settings/base.py b/api/core/settings/base.py index 968e54b4..2defeaa7 100644 --- a/api/core/settings/base.py +++ b/api/core/settings/base.py @@ -25,13 +25,6 @@ # SECURITY WARNING: keep the secret key used in production secret! SECRET_KEY = config("DJANGO_SECRET_KEY") -# SECURITY WARNING: don't run with debug turned on in production! -DEBUG = True - -ALLOWED_HOSTS = ["*"] -CORS_ALLOW_ALL_ORIGINS = True -CORS_ALLOW_CREDENTIALS = True - # Application definition INSTALLED_APPS = [ @@ -119,20 +112,6 @@ WSGI_APPLICATION = 'core.wsgi.application' -# Database -# https://docs.djangoproject.com/en/4.2/ref/settings/#databases - -DATABASES = { - "default": { - "ENGINE": "django.db.backends.postgresql", - "NAME": config("DB_NAME"), - "USER": config("DB_USERNAME"), - "PASSWORD": config("DB_PASSWORD"), - "HOST": config("DB_HOSTNAME"), - "PORT": config("DB_PORT", cast=int), - } -} - # Password validation # https://docs.djangoproject.com/en/4.2/ref/settings/#auth-password-validators diff --git a/api/core/settings/dev.py b/api/core/settings/dev.py index 98dd39a5..71cd7d63 100644 --- a/api/core/settings/dev.py +++ b/api/core/settings/dev.py @@ -1,11 +1,31 @@ from .base import * +# SECURITY WARNING: don't run with debug turned on in production! +DEBUG = True + +ALLOWED_HOSTS = [ + "localhost", + "0.0.0.0", + "127.0.0.1" +] + +# django-cors-headers +# https://github.com/adamchainz/django-cors-headers + +CORS_ALLOW_ALL_ORIGINS = True +CORS_ALLOW_CREDENTIALS = True + + # Database # https://docs.djangoproject.com/en/4.2/ref/settings/#databases DATABASES = { - 'default': { - 'ENGINE': 'django.db.backends.sqlite3', - 'NAME': BASE_DIR / 'db.sqlite3', + "default": { + "ENGINE": "django.db.backends.postgresql", + "NAME": config("DB_NAME"), + "USER": config("DB_USERNAME"), + "PASSWORD": config("DB_PASSWORD"), + "HOST": config("DB_HOSTNAME"), + "PORT": config("DB_PORT", cast=int), } } diff --git a/api/core/settings/prod.py b/api/core/settings/prod.py index dd7be20e..773e7540 100644 --- a/api/core/settings/prod.py +++ b/api/core/settings/prod.py @@ -1,6 +1,60 @@ from .base import * +import dj_database_url + + # SECURITY WARNING: don't run with debug turned on in production! DEBUG = False -ALLOWED_HOSTS = ["0.0.0.0:8000", "localhost:8000", "127.0.0.1:8000"] \ No newline at end of file +ALLOWED_HOSTS = [ + ".herokuapp.com", + ".suagradeunb.com.br" +] + + +# django-cors-headers +# https://github.com/adamchainz/django-cors-headers + +CORS_ALLOWED_ORIGINS = [ + "https://suagradeunb.com.br", + "https://api.suagradeunb.com.br" +] +# CORS_ALLOW_ALL_ORIGINS = True +CORS_ALLOW_CREDENTIALS = True + + +# SSL Redirect + +SECURE_PROXY_SSL_HEADER = ('HTTP_X_FORWARDED_PROTO', 'https') +SECURE_SSL_REDIRECT = True + + +# Application definition + +INSTALLED_APPS.insert(0, "whitenoise.runserver_nostatic") + +MIDDLEWARE.insert(1, "whitenoise.middleware.WhiteNoiseMiddleware") + + +# Database +# https://docs.djangoproject.com/en/4.2/ref/settings/#databases + +DATABASES = { + "default": dj_database_url.config( + conn_max_age=600, + conn_health_checks=True, + ssl_require=True, + ), +} + +STORAGES = { + # Enable WhiteNoise's GZip and Brotli compression of static assets: + # https://whitenoise.readthedocs.io/en/latest/django.html#add-compression-and-caching-support + "staticfiles": { + "BACKEND": "whitenoise.storage.CompressedManifestStaticFilesStorage", + }, +} + +# Don't store the original (un-hashed filename) version of static files, to reduce slug size: +# https://whitenoise.readthedocs.io/en/latest/django.html#WHITENOISE_KEEP_ONLY_HASHED_FILES +WHITENOISE_KEEP_ONLY_HASHED_FILES = True diff --git a/requirements.txt b/api/requirements.txt similarity index 82% rename from requirements.txt rename to api/requirements.txt index 788fd78a..50081e95 100644 --- a/requirements.txt +++ b/api/requirements.txt @@ -2,6 +2,7 @@ asgiref==3.7.2 autopep8==2.0.4 Babel==2.12.1 beautifulsoup4==4.12.2 +Brotli==1.1.0 cachetools==5.3.1 certifi==2023.7.22 cffi==1.16.0 @@ -11,18 +12,14 @@ colorama==0.4.6 coverage==7.3.2 cryptography==41.0.4 defusedxml==0.7.1 +dj-database-url==2.1.0 Django==4.2.5 -django-allauth==0.57.0 django-cors-headers==4.3.0 djangorestframework==3.14.0 djangorestframework-simplejwt==5.3.0 drf-yasg==1.21.7 ghp-import==2.1.0 -google-api-core==2.12.0 -google-api-python-client==2.102.0 -google-auth==2.23.2 -google-auth-httplib2==0.1.1 -googleapis-common-protos==1.60.0 +gunicorn==21.2.0 httplib2==0.22.0 idna==3.4 inflection==0.5.1 @@ -33,13 +30,13 @@ mergedeep==1.3.4 mkdocs==1.5.3 mkdocs-material==9.4.2 mkdocs-material-extensions==1.2 -oauthlib==3.2.2 packaging==23.1 paginate==0.5.6 pathspec==0.11.2 platformdirs==3.10.0 protobuf==4.24.4 -psycopg2-binary==2.9.7 +psycopg==3.1.13 +psycopg-binary==3.1.13 pyasn1==0.5.0 pyasn1-modules==0.3.0 pycodestyle==2.11.1 @@ -56,7 +53,6 @@ PyYAML==6.0.1 pyyaml_env_tag==0.1 regex==2023.8.8 requests==2.31.0 -requests-oauthlib==1.3.1 rsa==4.9 six==1.16.0 soupsieve==2.5 @@ -66,3 +62,4 @@ Unidecode==1.3.7 uritemplate==4.1.1 urllib3==2.0.5 watchdog==3.0.0 +whitenoise==6.6.0 diff --git a/api/runtime.txt b/api/runtime.txt new file mode 100644 index 00000000..dfe813b8 --- /dev/null +++ b/api/runtime.txt @@ -0,0 +1 @@ +python-3.11.6 \ No newline at end of file diff --git a/docker/Dockerfile b/docker/Dockerfile index 9bce0af1..f4afb994 100644 --- a/docker/Dockerfile +++ b/docker/Dockerfile @@ -7,22 +7,18 @@ WORKDIR /usr/src/api ENV PYTHONDONTWRITEBYTECODE 1 ENV PYTHONUNBUFFERED 1 -# instalar dependências do poastgres +# instalar dependências do postgres RUN apk update && apk add postgresql-dev gcc python3-dev musl-dev # copiar os requisitos python -COPY ./requirements.txt /usr/src/api/requirements.txt +COPY ./api/requirements.txt /usr/src/api/requirements.txt # instalação das dependências RUN pip install --upgrade pip && \ pip install -r /usr/src/api/requirements.txt -# copiar entrypoint.sh e dar permissão de execução -COPY ./api/entrypoint.sh /usr/src/api/entrypoint.sh -RUN chmod +x /usr/src/api/entrypoint.sh - # copiar o projeto COPY ./api /usr/src/api/ # executar o arquivo de inicialização -ENTRYPOINT ["/usr/src/api/entrypoint.sh"] \ No newline at end of file +ENTRYPOINT ["/usr/src/api/config/entrypoint.sh"] \ No newline at end of file diff --git a/docs/executing.md b/docs/executing.md index 36cce330..51b528b9 100644 --- a/docs/executing.md +++ b/docs/executing.md @@ -3,16 +3,18 @@ hide: - navigation --- -## 📝 Sumário +# Como executar o projeto? -- [✨ Início](#inicio) - - [📋 Pré-requisitos](#pre-requisitos) - - [💻 Ambiente](#ambiente) - - [📁 Dependências do projeto](#dependencias-do-projeto) - - [💾 Execução](#execucao) - - [✅ Autenticação do Google OAuth](#autenticacao-do-google-oauth) - - [🖱️ Acesso aos serviços](#acesso-aos-servicos) - - [📍 Migrations](#migrations) +## 👥 Equipe + +| Nome | GitHub | +| :--- | :----: | +| Arthur Ribeiro e Sousa | [@artrsousa1](https://github.com/artrsousa1) | +| Caio Falcão Habibe Costa | [@CaioHabibe](https://github.com/CaioHabibe) | +| Caio Felipe Rocha Rodrigues| [@caio-felipee](https://github.com/caio-felipee) | +| Gabriel Henrique Castelo Costa | [@GabrielCastelo-31](https://github.com/GabrielCastelo-31) | +| Henrique Camelo Quenino | [@henriquecq](https://github.com/henriquecq) | +| Mateus Vieira Rocha da Silva | [@mateusvrs](https://github.com/mateusvrs) | ## ✨ Início @@ -26,11 +28,10 @@ git clone https://github.com/unb-mds/2023-2-Squad11.git Para rodar o projeto, você precisa instalar as dependências globais, que são: -- [GNU Make 4.3 (ou superior)](https://www.incredibuild.com/integrations/gnu-make) -- [Python v3.11.6 e Pip v22.0.2 (ou superior)](https://www.python.org/downloads/release/python-3116/) -- [Django v4.2.5 (ou superior)](https://www.djangoproject.com/download/) -- [Node v20.9.0](https://nodejs.org/en/download/) e [NPM v10.1.0 (ou superior)](https://nodejs.org/en/download/) -- [Docker Engine v24.0.6](<(https://docs.docker.com/engine/install/)>) [ e Docker Compose v2.21.0 (ou superior)](https://docs.docker.com/compose/install/) +- GNU Make 4.3 (ou superior) +- Python v3.11.6 e Pip v22.0.2 (ou superior) +- Node v20.9.0 e NPM v10.1.0 (ou superior) +- Docker Engine v24.0.6 e Docker Compose v2.21.0 (ou superior) ### 💻 Ambiente @@ -46,10 +47,10 @@ Para instalar as dependências do projeto, você pode rodar os seguintes comando ```bash # Crie um ambiente virtual Python -python3 -m venv env +python3 -m venv api/env # Ative o ambiente virtual -source env/bin/activate +source api/env/bin/activate # Instale os pacotes do Python e Node make install @@ -76,19 +77,6 @@ docker compose up --build docker compose down -v ``` -**Você também pode executar os comandos acima utilizando o make:** - -```bash -# Se você quiser rodar em segundo plano -make start - -# Se alterações foram feitas no Dockerfile ou no docker-compose.yml -make start-b - -# Se for necessário deletar os volumes -make stop-v -``` - ### ✅ Autenticação do Google OAuth Para que o login com o Google funcione, é necessário trocar o `your_client_id` no arquivo `web/.env.local` pelo **Client ID** do projeto no Google Cloud. @@ -113,15 +101,15 @@ Adicionando serviços: 1. Entre na aba **APIs e Serviços**. 2. Clique em **Ativar APIs e Serviços**. 3. Ative os seguintes serviços: - - IAM Service Account Credentials API - - Identity and Access Management (IAM) API + - IAM Service Account Credentials API + - Identity and Access Management (IAM) API ### 🖱️ Acesso aos serviços -| Serviço | URL | -| :------- | :--------------------------------------------: | +| Serviço | URL | +| :--- | :----: | | Frontend | [http://localhost:3000](http://localhost:3000) | -| Backend | [http://localhost:8000](http://localhost:8000) | +| Backend | [http://localhost:8000](http://localhost:8000) | ### 📍 Migrations