feat: add all database models, pydantic models for tasks, fixtures and tests for database #16

raf-nr · 2023-11-27T21:18:36Z

Add pydantic models for tasks config and result
Add database models for tasks, files and user information
Add empty migration and migration with tables
Add fixtures for tests, that interact with database
Add parametrized test for testing database queries
Add goal for up of docker container in CI and other small work

toadharvard

Намного лучше, чем раньше, но делать еще оченб много.

toadharvard · 2023-12-12T16:50:29Z

.env.example


 POSTGRES_USER=admin
 POSTGRES_PASSWORD=admin
 POSTGRES_HOST=localhost
 POSTGRES_DB=desbordante
 POSTGRES_PORT=5432

+POSTGRES_USER_TEST=admin


не дублируй, лучше подмени в тестах настройки

готово

toadharvard · 2023-12-12T16:50:37Z

.github/workflows/run-linter-and-tests.yaml

@@ -18,11 +18,21 @@ jobs:
        with:
          python-version: '3.11'

+      - name: Set up docker


Эта генерация от чат-гпт совсем не подходит по контексту.

я конечно не чат-гпт, но согласен

toadharvard · 2023-12-12T16:50:57Z

.github/workflows/run-linter-and-tests.yaml

@@ -18,11 +18,21 @@ jobs:
        with:
          python-version: '3.11'

+      - name: Set up docker
+        uses: docker/login-action@v1


ты не пушишь ничего же, зачем?

действительно зачем

.github/workflows/run-linter-and-tests.yaml

toadharvard · 2023-12-12T16:52:11Z

.pre-commit-config.yaml

@@ -26,6 +26,7 @@ repos:
    hooks:
      - id: ruff
        args: [ --fix, --exit-non-zero-on-fix ]
+        exclude: tests/


Добавь опцию отключащую конкретную мешающую проверку, а не линтер на всех тестах.

toadharvard · 2023-12-12T18:08:22Z

tests/fixtures/insert_data_in_db.py

+                session.add(new_task)
+                session.commit()
+
+    return add_task


Весь файл очень большой, а каждая функция заменяется 2-3 строчками.

удалил

toadharvard · 2023-12-12T18:10:00Z

tests/tests_data/code.py

@@ -0,0 +1,21 @@
+from datetime import datetime


Я бы предпочел испоьзование билблиотеки fake

Они умеет генерить по схеме тестовые данные

toadharvard · 2023-12-12T18:11:26Z

tests/tests_data/file_info.py

+        "renamed_header": "header",
+        "rows_count": 4,
+        "count_of_columns": 54,
+        "path": "2///",


Плохие примерв. Напиши нормальные типы в схеме и генерь себе тестовые данные.

поправил

toadharvard · 2023-12-12T18:17:05Z

tests/conftest.py

+@pytest.fixture(scope="function", autouse=True)
+def clean_tables(request, session_test):
+    if "fixture_name" in request.fixturenames:
+        yield


Для чего тут yield?

мы с помощью этого не перезапускаем наши фикстуры(у которых scope=function) между тестами одного параметризованного теста

toadharvard · 2023-12-12T18:17:56Z

tests/fixtures/fetch_data_from_db.py

+    def fetch_first(model_name: str):
+        with session_test() as session:
+            with session.connection():
+                data = session.query(tables[model_name]).first()


select вместо query используют

…d pydantic-validators

… containers up

toadharvard · 2023-12-22T09:31:05Z

.pre-commit-config.yaml

@@ -26,6 +26,7 @@ repos:
    hooks:
      - id: ruff
        args: [ --fix, --exit-non-zero-on-fix ]
+        files: ^env\.py$


Кажется ты полностью убил линтер, просто добавь правило в исключение в pyproject.toml

toadharvard · 2023-12-22T12:09:21Z

.github/workflows/run-linter-and-tests.yaml

@@ -24,5 +24,8 @@ jobs:
      - name: Run all linters and formatters
        run: make lint

+      - name: Up all containers
+        run: make up -d &


Подумать как убрать необходимость в &

Возможно это ломает CI

toadharvard · 2023-12-22T12:11:01Z

app/db/migrations/versions/08c0492d6748_.py

@@ -0,0 +1,29 @@
+"""


Файлы миграций назваются нечитаемо. В спецификации alembic есть возмоность генерировать нормальные имена.

toadharvard · 2023-12-22T12:31:10Z

app/tasks/schema/ar/config.py

+
+
+class ARTaskConfig(BaseTaskConfig):
+    min_support_ar: Annotated[float, Field(ge=0, le=1, metadata={"decimal_places": 4})]


Поля metadata не существует у Field. Поправь везде, где встречается

toadharvard · 2023-12-22T12:34:52Z

tests/db_tests/test_database_queries.py

+        ("insert_task_data", "task", task_data),
+    ],
+)
+def test_database_queries(


Все еще большой

chore: rename module with background tasks

db14a26

toadharvard requested changes Dec 12, 2023

View reviewed changes

raf-nr added 2 commits December 22, 2023 04:32

chore: add in dependencies psycopg, fastapi-user, sqlaclhemy-utils an…

b6f11f2

…d pydantic-validators

chore: add instructions for rolling back migrations and arguments for…

24a1988

… containers up

raf-nr changed the title ~~feat: add database models for user information, files and tasks~~ feat: add all database models, pydantic models for tasks, fixtures and tests for database Dec 22, 2023

toadharvard requested changes Dec 22, 2023

View reviewed changes

raf-nr added 15 commits December 22, 2023 16:05

chore: disable check for unused imports in ruff

0317cb9

chore: add empty migration

0b8d4f7

feat: add database models for user information

bef1019

feat: add database models for files information

6be72f1

feat: add pydantic models for tasks

b20341d

feat: add database model for task

02315f2

chore: import database models to env file for migrations

3fade7b

chore: add migration with all models

4cf7728

chore: add psycopg to postgres dialect driver

f11bfd0

feat: add data for tests

6ecc6d8

chore: initialize settings for the test database

f57776a

feat: add fixtures for tests

df7a19e

feat: add parametrized test to check database queries

6efa79e

chore: delete unnecessary file

500032c

chore: add goal for up of docker container in CI

a1e300c

toadharvard approved these changes Dec 22, 2023

View reviewed changes

toadharvard merged commit 96fd16c into Desbordante:main Dec 22, 2023
2 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: add all database models, pydantic models for tasks, fixtures and tests for database #16

feat: add all database models, pydantic models for tasks, fixtures and tests for database #16

raf-nr commented Nov 27, 2023 •

edited

Loading

toadharvard left a comment

toadharvard Dec 12, 2023

raf-nr Dec 21, 2023

toadharvard Dec 12, 2023 •

edited

Loading

raf-nr Dec 21, 2023

toadharvard Dec 12, 2023

raf-nr Dec 21, 2023

toadharvard Dec 12, 2023

toadharvard Dec 12, 2023 •

edited

Loading

raf-nr Dec 22, 2023

toadharvard Dec 12, 2023

toadharvard Dec 12, 2023

toadharvard Dec 12, 2023 •

edited

Loading

raf-nr Dec 22, 2023

toadharvard Dec 12, 2023 •

edited

Loading

raf-nr Dec 16, 2023

toadharvard Dec 12, 2023

raf-nr Dec 16, 2023

toadharvard Dec 22, 2023

toadharvard Dec 22, 2023

toadharvard Dec 22, 2023

toadharvard Dec 22, 2023

toadharvard Dec 22, 2023

toadharvard Dec 22, 2023



		class ARTaskConfig(BaseTaskConfig):
		min_support_ar: Annotated[float, Field(ge=0, le=1, metadata={"decimal_places": 4})]

feat: add all database models, pydantic models for tasks, fixtures and tests for database #16

feat: add all database models, pydantic models for tasks, fixtures and tests for database #16

Conversation

raf-nr commented Nov 27, 2023 • edited Loading

toadharvard left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

toadharvard Dec 12, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

toadharvard Dec 12, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

toadharvard Dec 12, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

toadharvard Dec 12, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

raf-nr commented Nov 27, 2023 •

edited

Loading

toadharvard Dec 12, 2023 •

edited

Loading

toadharvard Dec 12, 2023 •

edited

Loading

toadharvard Dec 12, 2023 •

edited

Loading

toadharvard Dec 12, 2023 •

edited

Loading