Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature: HTTP DELETE node #293

Merged
merged 70 commits into from
Sep 1, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
70 commits
Select commit Hold shift + click to select a range
6a6ca71
started working on delete
nh916 Aug 25, 2023
c4d0d3f
setup `_set_logger` method for `cript.API`
nh916 Aug 25, 2023
688e296
added `Notes` to delete docstrings
nh916 Aug 25, 2023
125eedd
refactored `logger` and `verbose`
nh916 Aug 25, 2023
1531c0c
removed unused imports and formatted with black
nh916 Aug 25, 2023
f91e467
ignoring mypy error
nh916 Aug 25, 2023
0e910ab
formatting with black and removing unused imports
nh916 Aug 25, 2023
53d7c36
removing `cript.API.refresh()` method for now
nh916 Aug 25, 2023
cb4480d
format with trunk
nh916 Aug 25, 2023
421ddc7
changed logging format of `cript.API.delete`
nh916 Aug 26, 2023
0ae36d8
changed the error for paginator from generic `Exception` to `APIError`
nh916 Aug 26, 2023
0498d84
wrote `DELETE` for test_project.py
nh916 Aug 26, 2023
6f6d6bd
wrote out logic for `integrate_delete_node_helper`
nh916 Aug 28, 2023
2541d1e
updated `test_project.py` comment
nh916 Aug 28, 2023
df43358
added type hinting to paginator.py `response`
nh916 Aug 28, 2023
a2edf83
wrote out `delete_integration_node_helper`
nh916 Aug 28, 2023
791b473
updated comments
nh916 Aug 29, 2023
5f3f50c
updated `test_integration_collection`
nh916 Aug 29, 2023
14c4692
wrote Delete integration test for collection
nh916 Aug 29, 2023
1e38b22
wrote Delete integration test for experiment
nh916 Aug 29, 2023
e261ed4
wrote Delete integration test for process
nh916 Aug 29, 2023
a42d2d0
wrote Delete integration test for computation
nh916 Aug 29, 2023
2c545d8
fixed `delete integration function` imports for tests since it is not…
nh916 Aug 29, 2023
69cc8a2
wrote Delete integration test for data
nh916 Aug 29, 2023
7ae4cfe
wrote Delete integration test for inventory
nh916 Aug 29, 2023
980d5b8
wrote Delete integration test for file
nh916 Aug 29, 2023
3478647
wrote Delete integration test for algorithm
nh916 Aug 29, 2023
7f504f3
updated docstrings for clarity in `delete_integration_node_helper`
nh916 Aug 29, 2023
0940e21
wrote Delete integration test for citation
nh916 Aug 29, 2023
3e58dc1
wrote Delete integration test for condition
nh916 Aug 29, 2023
1fabb5b
wrote Delete integration test for equipment
nh916 Aug 29, 2023
0bcaf7d
wrote Delete integration test for ingredient
nh916 Aug 29, 2023
e15f0a4
wrote Delete integration test for parameter
nh916 Aug 29, 2023
ee9bfc9
wrote Delete integration test for quantity
nh916 Aug 29, 2023
c17df00
wrote Delete integration test for Software
nh916 Aug 29, 2023
be23182
wrote Delete integration test for Software
nh916 Aug 29, 2023
6663501
wrote Delete integration test for Software
nh916 Aug 29, 2023
844cbab
wrote Delete integration test for Reference
nh916 Aug 29, 2023
5a01f5a
upgraded/refactored `APIError`
nh916 Aug 29, 2023
8c69cde
bug fix: changed delete URL to `node_type_snake_case` instead of node…
nh916 Aug 29, 2023
5dfd394
wrote and successfully passing computational_forcefield DELETE
nh916 Aug 29, 2023
f699a93
wrote and successfully passing `computational_process` DELETE
nh916 Aug 29, 2023
a1fd669
upgraded paginator errors
nh916 Aug 29, 2023
55f71ef
wrote DELETE tests for `material`, `project`, `property`
nh916 Aug 29, 2023
54eb340
fixed mypy static typing errors
nh916 Aug 29, 2023
01d5cd4
formatted with black
nh916 Aug 29, 2023
76140de
removed unused import
nh916 Aug 29, 2023
baf1bb0
formatted with trunk
nh916 Aug 29, 2023
cf7fd43
formatted with trunk
nh916 Aug 29, 2023
3b413dd
fixed imports with isort
nh916 Aug 29, 2023
9dc819e
formatted with trunk
nh916 Aug 29, 2023
b6fcdf3
working on fixing CI issues
nh916 Aug 30, 2023
37519a1
optimizing imports
nh916 Aug 30, 2023
f5c8cb6
upgraded documentation for `cript.API.delete()`
nh916 Aug 30, 2023
5349cb2
trunk fmt
InnocentBug Aug 30, 2023
1cdfb21
updated `cript.API.delete()` documentation
nh916 Aug 30, 2023
95f0399
updated host documentation in `cript.API`
nh916 Aug 30, 2023
7857e7b
upgraded documentation for `cript.API.verobose`
nh916 Aug 30, 2023
eb47dd2
fixed `BIGSMILES` search test
nh916 Aug 31, 2023
9cefd2e
removed `Navid log style` from integration_test_helper.py
nh916 Aug 31, 2023
998dfcd
Merge branch 'develop' into feature-delete
nh916 Aug 31, 2023
5fcdef5
Merge branch 'develop' into feature-delete
nh916 Aug 31, 2023
7b2e0cb
formatted with trunk
nh916 Aug 31, 2023
22eb336
importing `Optional` to cript/api/exceptions.py
nh916 Aug 31, 2023
caea71f
formatted with trunk
nh916 Aug 31, 2023
3544320
added `timeout` to `requests.delete()`
nh916 Aug 31, 2023
3137cfd
fixed import error
nh916 Aug 31, 2023
4b4ae7c
formatted with trunk
nh916 Aug 31, 2023
96bc812
Merge branch 'develop' into feature-delete
nh916 Sep 1, 2023
c1e3395
making trunk happy
InnocentBug Sep 1, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
180 changes: 155 additions & 25 deletions src/cript/api/api.py
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@

from cript.api.api_config import _API_TIMEOUT
from cript.api.exceptions import (
APIError,
CRIPTAPIRequiredError,
CRIPTAPISaveError,
CRIPTConnectionError,
Expand Down Expand Up @@ -55,27 +56,11 @@ class API:
"""
## Definition
API Client class to communicate with the CRIPT API

Attributes
----------
verbose : bool
A boolean flag that controls whether verbose logging is enabled or not.

When `verbose` is set to `True`, the class will provide additional detailed logging
to the terminal. This can be useful for debugging and understanding the internal
workings of the class.

When `verbose` is set to `False`, the class will only provide essential and concise
logging information, making the terminal output less cluttered and more user-friendly.

```python
# turn off the terminal logs
api.verbose = False
```
"""

# dictates whether the user wants to see terminal log statements or not
verbose: bool = True
_verbose: bool = True
logger: logging.Logger = None # type: ignore

_host: str = ""
_api_token: str = ""
Expand Down Expand Up @@ -242,6 +227,9 @@ def __init__(self, host: Union[str, None] = None, api_token: Union[str, None] =

self._get_db_schema()

# set a logger instance to use for the class logs
self._set_logger()

def __str__(self) -> str:
"""
States the host of the CRIPT API client
Expand All @@ -252,6 +240,93 @@ def __str__(self) -> str:
"""
return f"CRIPT API Client - Host URL: '{self.host}'"

def _set_logger(self, verbose: bool = True) -> None:
"""
Prepare and configure the logger for the API class.

This function creates and configures a logger instance associated with the current module (class).

Parameters
----------
verbose: bool default True
set if you want `cript.API` to give logs to console or not

Returns
-------
logging.Logger
The configured logger instance.
"""
# Create a logger instance associated with the current module
logger = logging.getLogger(__name__)

# Set the logger's level based on the verbose flag
if verbose:
logger.setLevel(logging.INFO) # Display INFO logs
else:
logger.setLevel(logging.CRITICAL) # Display no logs

# Create a console handler
console_handler = logging.StreamHandler()

# Create a formatter for log messages (customize the format as desired)
formatter = logging.Formatter("%(levelname)s: %(message)s")

# Associate the formatter with the console handler
console_handler.setFormatter(formatter)

# Add the console handler to the logger
logger.addHandler(console_handler)

# set logger for the class
self.logger = logger

@property
def verbose(self) -> bool:
"""
A boolean flag that controls whether verbose logging is enabled or not.

When `verbose` is set to `True`, the class will provide additional detailed logging
to the terminal. This can be useful for debugging and understanding the internal
workings of the class.

```bash
INFO: Validating Project graph...
```

When `verbose` is set to `False`, the class will only provide essential logging information,
making the terminal output less cluttered and more user-friendly.

Examples
--------
```python
# turn off the terminal logs
api.verbose = False
```

Returns
-------
bool
verbose boolean value
"""
return self._verbose

@verbose.setter
def verbose(self, new_verbose_value: bool) -> None:
"""
sets the verbose value and then sets a new logger for the class

Parameters
----------
new_verbose_value: bool
new verbose value to turn the logging ON or OFF

Returns
-------
None
"""
self._verbose = new_verbose_value
self._set_logger(verbose=new_verbose_value)

@beartype
def _prepare_host(self, host: str) -> str:
# strip ending slash to make host always uniform
Expand Down Expand Up @@ -349,10 +424,10 @@ def host(self):

The term "host" designates the specific CRIPT instance to which you intend to upload your data.

For most users, the host will be `criptapp.org`
For most users, the host will be `api.criptapp.org`

```yaml
host: criptapp.org
host: api.criptapp.org
```

Examples
Expand Down Expand Up @@ -572,11 +647,9 @@ def _is_node_schema_valid(self, node_json: str, is_patch: bool = False) -> bool:

node_dict = json.loads(node_json)

if self.verbose:
# logging out info to the terminal for the user feedback
# (improve UX because the program is currently slow)
logging.basicConfig(format="%(levelname)s: %(message)s", level=logging.INFO)
logging.info(f"Validating {node_type} graph...")
# logging out info to the terminal for the user feedback
# (improve UX because the program is currently slow)
self.logger.info(f"Validating {node_type} graph...")

# set the schema to test against http POST or PATCH of DB Schema
schema_http_method: str
Expand Down Expand Up @@ -974,3 +1047,60 @@ def search(
raise RuntimeError("Internal Error: Failed to recognize any search modes. Please report this bug on https://github.com/C-Accel-CRIPT/Python-SDK/issues.")

return Paginator(http_headers=self._http_headers, api_endpoint=api_endpoint, query=value_to_search, current_page_number=page_number)

def delete(self, node) -> None:
"""
Simply deletes the desired node from the CRIPT API and writes a log in the terminal that the node has been
successfully deleted.

Examples
--------
```python
api.delete(node=my_material_node)
```

Notes
-----
After the node has been successfully deleted, a log is written to the terminal if `cript.API.verbose = True`

```bash
INFO: Deleted `Data` with UUID of `80bfc642-157e-4692-a547-97c470725397` from CRIPT API.
```

Warnings
--------
After successfully deleting a node from the API, keep in mind that your local Project node in your script
may still contain outdated data as it has not been synced with the API.

To ensure you have the latest data, follow these steps:

1. Fetch the newest Project node from the API using the [`cript.API.search()`](./#cript.api.api.API.search) provided by the SDK.
1. Deserialize the retrieved data into a new Project node using the [`load_nodes_from_json`](../../utility_functions/#cript.nodes.util.load_nodes_from_json) utility function.
1. Replace your old Project node with the new one in your script for accurate and up-to-date information.

Parameters
----------
node: UUIDBaseNode
The node that you want to delete

Raises
------
APIError
If the API responds with anything other than HTTP status 200, then the CRIPT Python SDK raises `APIError`
`APIError` is raised in case the API cannot delete the specified node.
Such cases can happen if you do not have permission to delete the node
or if the node is actively being used elsewhere in CRIPT platform and the API cannot delete it.

Returns
-------
None
"""

delete_node_api_url: str = f"{self._host}/{node.node_type_snake_case}/{node.uuid}/"

response: Dict = requests.delete(headers=self._http_headers, url=delete_node_api_url, timeout=_API_TIMEOUT).json()

if response["code"] != 200:
raise APIError(api_error=str(response), http_method="DELETE", api_url=delete_node_api_url)

self.logger.info(f"Deleted `{node.node_type}` with UUID of `{node.uuid}` from CRIPT API.")
25 changes: 20 additions & 5 deletions src/cript/api/exceptions.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
import json
from typing import List, Set
from typing import List, Optional, Set

from cript.exceptions import CRIPTException

Expand Down Expand Up @@ -214,13 +214,28 @@ class APIError(CRIPTException):

api_error: str = ""

def __init__(self, api_error: str) -> None:
# having the URL that the API gave an error for helps in debugging
api_url: Optional[str] = None
http_method: Optional[str] = None

def __init__(self, api_error: str, http_method: Optional[str] = None, api_url: Optional[str] = None) -> None:
self.api_error = api_error

def __str__(self) -> str:
error_message: str = f"The API responded with {self.api_error}"
self.api_url = api_url

return error_message
# TODO consider having an enum for all the HTTP methods so they are easily entered and disallows anything
# that would be not make sense
self.http_method = http_method

def __str__(self) -> str:
# TODO refactor all of SDK using this error to be used the same to avoid this logic and Optional attributes
# this logic currently exists to make previous SDK work

# if you have the URL then display it, otherwise just show the error message
if self.api_url and self.http_method:
return f"CRIPT Python SDK sent HTTP `{self.http_method.upper()}` request to URL: `{self.api_url}` and API responded with {self.api_error}"
InnocentBug marked this conversation as resolved.
Show resolved Hide resolved
else:
return f"The API responded with: {self.api_error}"


class FileDownloadError(CRIPTException):
Expand Down
4 changes: 2 additions & 2 deletions src/cript/api/paginator.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
from beartype import beartype

from cript.api.api_config import _API_TIMEOUT
from cript.api.exceptions import APIError


class Paginator:
Expand Down Expand Up @@ -225,8 +226,7 @@ def fetch_page_from_api(self) -> List[dict]:
self.current_page_results = []
return self.current_page_results

# TODO give a CRIPT error if HTTP response is anything other than 200
if api_response["code"] != 200:
raise Exception(f"API responded with: {api_response['error']}")
raise APIError(api_error=str(response), http_method="GET", api_url=temp_api_endpoint)

return self.current_page_results
45 changes: 45 additions & 0 deletions tests/integration_test_helper.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@
from deepdiff import DeepDiff

import cript
from cript.nodes.uuid_base import UUIDBaseNode


def integrate_nodes_helper(cript_api: cript.API, project_node: cript.Project):
Expand Down Expand Up @@ -96,3 +97,47 @@ def integrate_nodes_helper(cript_api: cript.API, project_node: cript.Project):
print(my_project_from_api.get_json(sort_keys=False, condense_to_uuid={}, indent=2).json)
print("==============================================================")
print("\n\n\n######################################## TEST Passed ########################################\n\n\n")


def delete_integration_node_helper(cript_api: cript.API, node_to_delete: UUIDBaseNode) -> None:
"""
1. takes the node/sub-object that needs to be deleted
1. checks that it first exists on the API before deleting
1. sends an HTTP DELETE to API
1. asserts that the API response is 200
> This is done under the hood in the `cript.API.delete()`
method where at the end it checks that the response is 200, else raises an error
1. tries to get the same node via UUID from the API and expects that it should fail

Notes
------
> for future this test should also take the project that the node was in, get the project again,
> and compare that the node was successfully deleted/missing from the project

> within search we can also add assert statements to be sure that the API gave the same node
> that we searched for. Not needed at the time of writing this test
"""

if not HAS_INTEGRATION_TESTS_ENABLED:
pytest.skip("DELETE integration tests with API requires real API and Storage token")
return

# check that the node we want to delete first exists on the API
my_node_paginator = cript_api.search(
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I understand the logic.
But would it be better to just send delete and handle the error from API if the node doesn't exist?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that could streamline the process, but the only thing that we would lose from that would be that we did not prove that the node was there before we went to delete it, so if we start getting bugs we might be asking ourselves,

"is it having issues because the API can't delete the node? or is it because the node never existed on the API?"

I was thinking that writing it this way could be a bit more explicit, but removing the first search would honestly not take away much as far as I can see and would be pretty easy too.

You think remove it?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think, I would insist on getting rid of it in production code, but this is tests only and should be OK.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would honestly be happy getting rid of it too either now or in a future PR. I think both should work and be okay

# passing in the full node to get the node type from
node_type=node_to_delete,
search_mode=cript.SearchModes.UUID,
value_to_search=str(node_to_delete.uuid),
)

# be sure API returned at least one result for the node that we searched for
assert len(my_node_paginator.current_page_results) == 1

# DELETE the node from the API
cript_api.delete(node=node_to_delete)

# should not be able to get node by UUID anymore because it is deleted
deleted_node_paginator = cript_api.search(node_type=node_to_delete, search_mode=cript.SearchModes.UUID, value_to_search=str(node_to_delete.uuid))

# be sure API responded with empty results
assert len(deleted_node_paginator.current_page_results) == 0
8 changes: 7 additions & 1 deletion tests/nodes/primary_nodes/test_collection.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,10 @@
import json
import uuid

from integration_test_helper import integrate_nodes_helper
from integration_test_helper import (
delete_integration_node_helper,
integrate_nodes_helper,
)
from util import strip_uid_from_dict

import cript
Expand Down Expand Up @@ -182,3 +185,6 @@ def test_integration_collection(cript_api, simple_project_node, simple_collectio
# simple_project_node.collection[0].notes = "my collection notes UPDATED"

integrate_nodes_helper(cript_api=cript_api, project_node=simple_project_node)

# ========= test delete =========
delete_integration_node_helper(cript_api=cript_api, node_to_delete=simple_collection_node)
8 changes: 7 additions & 1 deletion tests/nodes/primary_nodes/test_computation.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,10 @@
import json
import uuid

from integration_test_helper import integrate_nodes_helper
from integration_test_helper import (
delete_integration_node_helper,
integrate_nodes_helper,
)
from util import strip_uid_from_dict

import cript
Expand Down Expand Up @@ -143,3 +146,6 @@ def test_integration_computation(cript_api, simple_project_node, simple_computat
simple_project_node.collection[0].experiment[0].computation[0].type = "data_fit"

integrate_nodes_helper(cript_api=cript_api, project_node=simple_project_node)

# ========= test delete =========
delete_integration_node_helper(cript_api=cript_api, node_to_delete=simple_computation_node)
Loading
Loading