Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

remove flask upper pin #35

Merged
51 changes: 5 additions & 46 deletions .github/workflows/tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -15,55 +15,14 @@ on:
branches: master
schedule:
# * is a special character in YAML so you have to quote this string
- cron: '0 3 * * 6'
- cron: "0 3 * * 6"
workflow_dispatch:
inputs:
reason:
description: 'Reason'
description: "Reason"
required: false
default: 'Manual trigger'
default: "Manual trigger"

jobs:
Tests:
runs-on: ubuntu-20.04
strategy:
matrix:
python-version: [3.8, 3.9]
requirements-level: [pypi]
db-service: [postgresql14]
search-service: [opensearch2]

env:
DB: ${{ matrix.db-service }}
EXTRAS: tests,${{ matrix.search-service }}
steps:
- name: Checkout
uses: actions/checkout@v2

- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v2
with:
python-version: ${{ matrix.python-version }}

- name: Generate dependencies
run: |
python -m pip install --upgrade pip setuptools py wheel requirements-builder
requirements-builder -e "$EXTRAS" --level=${{ matrix.requirements-level }} setup.py > .${{ matrix.requirements-level }}-${{ matrix.python-version }}-requirements.txt

- name: Cache pip
uses: actions/cache@v2
with:
path: ~/.cache/pip
key: ${{ runner.os }}-pip-${{ hashFiles('.${{ matrix.requirements-level }}-${{ matrix.python-version }}-requirements.txt') }}

- name: Install dependencies
run: |
pip install -r .${{ matrix.requirements-level }}-${{ matrix.python-version }}-requirements.txt
pip install ".[$EXTRAS]"
pip freeze
docker --version
docker-compose --version

- name: Run tests
run: |
./run-tests.sh
Python:
uses: inveniosoftware/workflows/.github/workflows/tests-python.yml@master
37 changes: 19 additions & 18 deletions invenio_banners/records/models.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
# -*- coding: utf-8 -*-
#
# Copyright (C) 2020-2023 CERN.
# Copyright (C) 2024 Graz University of Technology.
#
# Invenio-Banners is free software; you can redistribute it and/or modify it
# under the terms of the MIT License; see LICENSE file for more details.
Expand All @@ -24,7 +25,6 @@ class BannerModel(db.Model, Timestamp):
"""Defines a message to show to users."""

__tablename__ = "banners"
__versioned__ = {"versioning": False}

id = db.Column(db.Integer, primary_key=True)

Expand All @@ -51,6 +51,7 @@ def create(cls, data):
"""Create a new banner."""
_categories = [t[0] for t in current_app.config["BANNERS_CATEGORIES"]]
assert data.get("category") in _categories

with db.session.begin_nested():
obj = cls(
message=data.get("message"),
Expand All @@ -62,22 +63,21 @@ def create(cls, data):
)
db.session.add(obj)

db.session.commit()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to add the unit of work decorator in each service method that is modifying a record, and therefore calling these create/update/delete.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this db.session.commit is not necessary. it is also not necessary to change something in the services.

tried in a instance. the banner is stored in the database also without that db.session.commit()

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It cannot be magically happen... a decorator must be added here
If it happens, we need to understand why.
Maybe flask-sqlalchemy is auto-committing and the end of the HTTP request?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To add a bit to the confusion: Here it seems that they move towards explicit db.session.commit's.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ntarocco you are totally right, it cannot magically happen. it happened the first time i tested it but i couldn't reproduce it. so i added now the unit_of_work decorator to all service methods which create/update/delete models

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice thanks! In case you want to be 100% sure, you could change the resources tests and add something like this:

id = client.put(...)  # create banner
client.get(.../<id>)  # get the created banner

This should fail without the uow, and pass after adding it ☺️

return obj

@classmethod
def update(cls, data, id):
"""Update an existing banner."""
with db.session.begin_nested():
cls.query.filter_by(id=id).update(data)
db.session.query(cls).filter_by(id=id).update(data)

db.session.commit()

@classmethod
def get(cls, id):
"""Get banner by its id."""
try:
return cls.query.filter_by(id=id).one()
return db.session.query(cls).filter_by(id=id).one()
except NoResultFound:
raise BannerNotExistsError(id)

Expand All @@ -87,15 +87,14 @@ def delete(cls, banner):
with db.session.begin_nested():
db.session.delete(banner)

db.session.commit()

@classmethod
def get_active(cls, url_path):
"""Return active banners."""
now = datetime.utcnow()

query = (
cls.query.filter(cls.active.is_(True))
db.session.query(cls)
.filter(cls.active.is_(True))
.filter(cls.start_datetime <= now)
.filter((cls.end_datetime.is_(None)) | (now <= cls.end_datetime))
)
Expand All @@ -113,16 +112,17 @@ def get_active(cls, url_path):
@classmethod
def search(cls, search_params, filters):
"""Filter banners accordingly to query params."""
banners = (
BannerModel.query.filter(or_(*filters))
.order_by(
search_params["sort_direction"](text(",".join(search_params["sort"])))
)
.paginate(
page=search_params["page"],
per_page=search_params["size"],
error_out=False,
)
if filters == []:
filtered = db.session.query(BannerModel).filter()
else:
filtered = db.session.query(BannerModel).filter(or_(*filters))

banners = filtered.order_by(
search_params["sort_direction"](text(",".join(search_params["sort"])))
).paginate(
page=search_params["page"],
per_page=search_params["size"],
error_out=False,
)

return banners
Expand All @@ -133,7 +133,8 @@ def disable_expired(cls):
now = datetime.utcnow()

query = (
cls.query.filter(cls.active.is_(True))
db.session.query(cls)
.filter(cls.active.is_(True))
.filter(cls.end_datetime.isnot(None))
.filter(cls.end_datetime < now)
)
Expand Down
8 changes: 4 additions & 4 deletions invenio_banners/resources/errors.py
Original file line number Diff line number Diff line change
@@ -1,19 +1,19 @@
# -*- coding: utf-8 -*-
#
# Copyright (C) 2023 CERN.
# Copyright (C) 2024 Graz University of Technology.
#
# Invenio-Banners is free software; you can redistribute it and/or modify it
# under the terms of the MIT License; see LICENSE file for more details.

"""Errors."""

from flask_resources import HTTPJSONException, create_error_handler

from ..services.errors import BannerNotExistsError
import marshmallow as ma
from flask_resources import HTTPJSONException, create_error_handler
from invenio_records_resources.errors import validation_error_to_list_errors

from ..services.errors import BannerNotExistsError


class HTTPJSONValidationException(HTTPJSONException):
"""HTTP exception serializing to JSON and reflecting Marshmallow errors."""
Expand All @@ -25,7 +25,7 @@ def __init__(self, exception):
super().__init__(code=400, errors=validation_error_to_list_errors(exception))


class ErrorHandlersMixin():
class ErrorHandlersMixin:
"""Mixin to define error handlers."""

error_handlers = {
Expand Down
10 changes: 9 additions & 1 deletion invenio_banners/services/results.py
Original file line number Diff line number Diff line change
@@ -1,14 +1,22 @@
# -*- coding: utf-8 -*-
#
# Copyright (C) 2022-2023 CERN.
# Copyright (C) 2024 Graz University of Technology.
#
# Invenio-Banners is free software; you can redistribute it and/or modify it
# under the terms of the MIT License; see LICENSE file for more details.

"""Service results."""
from flask_sqlalchemy import Pagination

from invenio_records_resources.services.records.results import RecordItem, RecordList

try:
# flask_sqlalchemy<3.0.0
from flask_sqlalchemy import Pagination
except ImportError:
# flask_sqlalchemy>=3.0.0
from flask_sqlalchemy.pagination import Pagination


class BannerItem(RecordItem):
"""Single banner result."""
Expand Down
62 changes: 36 additions & 26 deletions tests/resources/test_resources.py
Original file line number Diff line number Diff line change
@@ -1,78 +1,88 @@
# -*- coding: utf-8 -*-
#
# Copyright (C) 2022 CERN.
# Copyright (C) 2024 Graz University of Technology.
#
# Invenio-Banners is free software; you can redistribute it and/or modify it
# under the terms of the MIT License; see LICENSE file for more details.

"""Banner resource tests."""
from datetime import date, datetime
from datetime import date, datetime, timedelta

import pytest
from invenio_db import db
from invenio_records_resources.services.errors import PermissionDeniedError

from invenio_banners.records import BannerModel

banners = {
"banner0": {
"message": "banner0",
"url_path": "/banner0",
"category": "info",
"active": True,
"start_datetime": date(2022, 7, 20).strftime("%Y-%m-%d %H:%M:%S"),
"end_datetime": (datetime.utcnow() - timedelta(days=20)).strftime(
"%Y-%m-%d %H:%M:%S"
),
},
"banner1": {
"message": "banner1",
"url_path": "/banner1",
"category": "info",
"active": True,
"start_datetime": date(2022, 7, 20),
"end_datetime": date(2023, 1, 29),
"start_datetime": date(2022, 7, 20).strftime("%Y-%m-%d %H:%M:%S"),
"end_datetime": (datetime.utcnow() + timedelta(days=20)).strftime(
"%Y-%m-%d %H:%M:%S"
),
},
"banner2": {
"message": "banner2",
"url_path": "/banner2",
"category": "other",
"active": False,
"start_datetime": date(2022, 12, 15),
"end_datetime": date(2023, 1, 5),
"start_datetime": date(2022, 12, 15).strftime("%Y-%m-%d %H:%M:%S"),
"end_datetime": (datetime.utcnow() + timedelta(days=10)).strftime(
"%Y-%m-%d %H:%M:%S"
),
},
"banner3": {
"message": "banner3",
"url_path": "/banner3",
"category": "warning",
"active": True,
"start_datetime": date(2023, 1, 20),
"end_datetime": date(2023, 2, 25),
"start_datetime": date(2023, 1, 20).strftime("%Y-%m-%d %H:%M:%S"),
"end_datetime": (datetime.utcnow() + timedelta(days=30)).strftime(
"%Y-%m-%d %H:%M:%S"
),
},
}


def _create_banner(client, data, headers, status_code=None):
"""Send POST request."""
result = client.post(
"/banners/",
headers=headers,
json=data,
)
result = client.post("/banners/", headers=headers, json=data)
assert result.status_code == status_code
return result


def _update_banner(client, id, data, headers, status_code=None):
"""Send PUT request."""
result = client.put(
"/banners/{0}".format(id),
headers=headers,
json=data,
)
result = client.put(f"/banners/{id}", headers=headers, json=data)
assert result.status_code == status_code
return result


def _delete_banner(client, id, headers, status_code=None):
"""Send DELETE request."""
result = client.delete("/banners/{0}".format(id), headers=headers)
result = client.delete(f"/banners/{id}", headers=headers)
assert result.status_code == status_code
return result


def _get_banner(client, id, status_code=None):
"""Send GET request."""
result = client.get("/banners/{0}".format(id))
result = client.get(f"/banners/{id}")
assert result.status_code == status_code
return result

Expand Down Expand Up @@ -108,15 +118,15 @@ def test_create_banner(client, admin, headers):
def test_disable_expired_after_create_action(client, admin, headers):
"""Disable expired banners after a create a banner action."""
# create banner first
banner1 = BannerModel.create(banners["banner1"])
assert banner1.active is True
banner0 = BannerModel.create(banners["banner0"])
assert banner0.active is True
banner_data = banners["banner2"]

admin.login(client)

_create_banner(client, banner_data, headers, 201).json

expired_banner = BannerModel.get(banner1.id)
expired_banner = BannerModel.get(banner0.id)
assert expired_banner.active is False


Expand Down Expand Up @@ -145,9 +155,9 @@ def test_update_banner(client, admin, headers):
def test_disable_expired_after_update_action(client, admin, headers):
"""Disable expired banners after an update a banner action."""
# create banner first
banner1 = BannerModel.create(banners["banner1"])
banner0 = BannerModel.create(banners["banner0"])
banner2 = BannerModel.create(banners["banner2"])
assert banner1.active is True
assert banner0.active is True

admin.login(client)

Expand All @@ -160,7 +170,7 @@ def test_disable_expired_after_update_action(client, admin, headers):

_update_banner(client, banner2.id, new_data, headers, 200).json

expired_banner = BannerModel.get(banner1.id)
expired_banner = BannerModel.get(banner0.id)
assert expired_banner.active is False


Expand Down Expand Up @@ -200,7 +210,7 @@ def test_delete_banner(client, admin, headers):
_delete_banner(client, banner.id, headers, 204)

# check that it's not present in db
assert BannerModel.query.filter_by(id=banner.id).one_or_none() is None
assert db.session.query(BannerModel).filter_by(id=banner.id).one_or_none() is None


def test_delete_is_forbidden(client, user, headers):
Expand Down
Loading