Skip to content

Commit

Permalink
#149 Addressed review comments
Browse files Browse the repository at this point in the history
  • Loading branch information
ahsimb committed Jul 8, 2024
1 parent f5561b0 commit 12d1899
Show file tree
Hide file tree
Showing 11 changed files with 77 additions and 70 deletions.
21 changes: 21 additions & 0 deletions doc/api.rst
Original file line number Diff line number Diff line change
Expand Up @@ -9,13 +9,34 @@ exasol.bucketfs.Service
:undoc-members:
:show-inheritance:

exasol.bucketfs.BucketLike
--------------------------
.. autoclass:: exasol.bucketfs.BucketLike
:members:
:undoc-members:
:show-inheritance:

exasol.bucketfs.Bucket
-----------------------
.. autoclass:: exasol.bucketfs.Bucket
:members:
:undoc-members:
:show-inheritance:

exasol.bucketfs.SaaSBucket
--------------------------
.. autoclass:: exasol.bucketfs.SaaSBucket
:members:
:undoc-members:
:show-inheritance:

exasol.bucketfs.MountedBucket
-----------------------------
.. autoclass:: exasol.bucketfs.MountedBucket
:members:
:undoc-members:
:show-inheritance:

exasol.bucketfs.path.PathLike
-----------------------------
.. autoclass:: exasol.bucketfs._path.PathLike
Expand Down
2 changes: 1 addition & 1 deletion doc/examples/bucket_saas.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
"""
This tutorial is relevant for the Exasol SaaS database.
This example is relevant for the Exasol SaaS database.
It demonstrates the creation of a bucket object for a SaaS database.
"""
import os
Expand Down
2 changes: 1 addition & 1 deletion doc/examples/delete.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
"""
This tutorial is relevant for the On-Prem Exasol database.
These examples are relevant for the On-Prem Exasol database.
"""
from exasol.bucketfs import Service

Expand Down
2 changes: 1 addition & 1 deletion doc/examples/download.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
"""
This tutorial is relevant for the On-Prem Exasol database.
These examples are relevant for the On-Prem Exasol database.
"""
from exasol.bucketfs import (
Service,
Expand Down
2 changes: 1 addition & 1 deletion doc/examples/list.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
"""
This tutorial is relevant for the On-Prem Exasol database.
These examples are relevant for the On-Prem Exasol database.
"""
from exasol.bucketfs import Service

Expand Down
14 changes: 8 additions & 6 deletions doc/examples/path_like.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
"""
We will demonstrate the usage of the PathLike interface with an example of handling
customer reviews.
In this tutorial we will demonstrate the usage of the PathLike interface
with an example of handling customer reviews.
"""
from typing import ByteString
import tempfile
Expand Down Expand Up @@ -69,7 +69,7 @@
b'I will definitely be using their services again in the future.'
)

# Now let's write same bad reviews into a different subdirectory.
# Now let's write some bad reviews in a different subdirectory.
bad_reviews = reviews / 'bad'

# Previously we provided content as a ByteString. But we can also use a file object,
Expand Down Expand Up @@ -124,7 +124,7 @@ def read_content(bfs_path: bfs.path.PathLike) -> ByteString:

# A file can be deleted using the rm() method. Please note that once the file is
# deleted it won't be possible to write another file to the same path for a certain
# period of time, due to internal internode synchronisation procedure.
# time, due to internal internode synchronisation procedure.
mike_s_review.rm()

# A directory can be deleted using the rmdir() method. If it is not empty we need
Expand All @@ -134,5 +134,7 @@ def read_content(bfs_path: bfs.path.PathLike) -> ByteString:
# Now all reviews should be deleted.
print('Are any reviews left?', reviews.exists())

# In BucketFS a directory doesn't exist as a physical object. Therefore, the
# exists() function called on a path for an empty directory returns False.
# It may look surprising why a call to the review.exists() returns False, since we
# have not deleted the base directory. In BucketFS a directory doesn't exist as a
# distinct entity. Therefore, the exists() function called on a path for an empty
# directory returns False.
2 changes: 1 addition & 1 deletion doc/examples/quickstart.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
"""
This tutorial is relevant for the On-Prem Exasol database.
These examples are relevant for the On-Prem Exasol database.
"""

from exasol.bucketfs import (
Expand Down
2 changes: 1 addition & 1 deletion doc/examples/service.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
"""
This tutorial is relevant for the On-Prem Exasol database.
These examples are relevant for the On-Prem Exasol database.
"""

# List buckets
Expand Down
2 changes: 1 addition & 1 deletion doc/examples/upload.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
"""
This tutorial is relevant for the On-Prem Exasol database.
These examples are relevant for the On-Prem Exasol database.
"""
import io

Expand Down
90 changes: 37 additions & 53 deletions exasol/bucketfs/_buckets.py
Original file line number Diff line number Diff line change
Expand Up @@ -78,7 +78,7 @@ def delete(self, path: str) -> None:
Q. What happens if the path points to a directory?
A. Same. There are no directories as such in the BucketFS, hence
a directory path is just a non-existent file.
a directory path is just a non-existent file.
"""

def upload(self, path: str, data: ByteString | BinaryIO) -> None:
Expand All @@ -90,20 +90,20 @@ def upload(self, path: str, data: ByteString | BinaryIO) -> None:
Q. What happens if the parent is missing?
A. The bucket doesn't care about the structure of the file's path. Looking from the prospective
of a file system, the bucket will create the missing parent, but in reality it will just
store the data indexed by the provided path.
of a file system, the bucket will create the missing parent, but in reality it will just
store the data indexed by the provided path.
Q. What happens if the path points to an existing file?
A. That's fine, the file will be updated.
Q. What happens if the path points to an existing directory?
A. The bucket doesn't care about the structure of the file's path. Looking from the prospective
of a file system, there will exist a file and directory with the same name.
of a file system, there will exist a file and directory with the same name.
Q. How should the path look like?
A. It should look like a POSIX path, but it should not contain any of the NTFS invalid characters.
It can have the leading and/or ending backslashes, which will be subsequently removed.
If the path doesn't conform to this format an BucketFsError will be raised.
It can have the leading and/or ending backslashes, which will be subsequently removed.
If the path doesn't conform to this format an BucketFsError will be raised.
"""

def download(self, path: str, chunk_size: int = 8192) -> Iterable[ByteString]:
Expand All @@ -126,7 +126,23 @@ def download(self, path: str, chunk_size: int = 8192) -> Iterable[ByteString]:

class Bucket:
"""
Implementation of the On-Premises bucket.
Implementation of the BucketLike interface for the BucketFS in Exasol On-Premises database.
Args:
name:
Name of the bucket.
service:
Url where this bucket is hosted on.
username:
Username used for authentication.
password:
Password used for authentication.
verify:
Either a boolean, in which case it controls whether we verify
the server's TLS certificate, or a string, in which case it must be a path
to a CA bundle to use. Defaults to ``True``.
service_name:
Optional name of the BucketFS service.
"""

def __init__(
Expand All @@ -138,25 +154,6 @@ def __init__(
verify: bool | str = True,
service_name: Optional[str] = None
):
"""
Create a new bucket instance.
Args:
name:
Name of the bucket.
service:
Url where this bucket is hosted on.
username:
Username used for authentication.
password:
Password used for authentication.
verify:
Either a boolean, in which case it controls whether we verify
the server's TLS certificate, or a string, in which case it must be a path
to a CA bundle to use. Defaults to ``True``.
service_name:
Optional name of the BucketFS service.
"""
self._name = name
self._service = _parse_service_url(service)
self._username = username
Expand Down Expand Up @@ -201,13 +198,6 @@ def __iter__(self) -> Iterator[str]:
def upload(
self, path: str, data: ByteString | BinaryIO | Iterable[ByteString]
) -> None:
"""
Uploads a file onto this bucket
Args:
path: in the bucket the file shall be associated with.
data: raw content of the file.
"""
url = _build_url(service_url=self._service, bucket=self.name, path=path)
LOGGER.info("Uploading %s to bucket %s.", path, self.name)
response = requests.put(url, data=data, auth=self._auth, verify=self._verify)
Expand All @@ -217,15 +207,6 @@ def upload(
raise BucketFsError(f"Couldn't upload file: {path}") from ex

def delete(self, path) -> None:
"""
Deletes a specific file in this bucket.
Args:
path: points to the file which shall be deleted.
Raises:
A BucketFsError if the operation couldn't be executed successfully.
"""
url = _build_url(service_url=self._service, bucket=self.name, path=path)
LOGGER.info("Deleting %s from bucket %s.", path, self.name)
response = requests.delete(url, auth=self._auth, verify=self._verify)
Expand All @@ -236,16 +217,6 @@ def delete(self, path) -> None:
raise BucketFsError(f"Couldn't delete: {path}") from ex

def download(self, path: str, chunk_size: int = 8192) -> Iterable[ByteString]:
"""
Downloads a specific file of this bucket.
Args:
path: which shall be downloaded.
chunk_size: which shall be used for downloading.
Returns:
An iterable of binary chunks representing the downloaded file.
"""
url = _build_url(service_url=self._service, bucket=self.name, path=path)
LOGGER.info(
"Downloading %s using a chunk size of %d bytes from bucket %s.",
Expand All @@ -263,6 +234,19 @@ def download(self, path: str, chunk_size: int = 8192) -> Iterable[ByteString]:


class SaaSBucket:
"""
Implementation of the BucketLike interface for the BucketFS in Exasol SaaS.
Arguments:
url:
Url of the Exasol SaaS service.
account_id:
SaaS user account ID.
database_id:
SaaS database ID.
pat:
Personal Access Token
"""

def __init__(self, url: str, account_id: str, database_id: str, pat: str) -> None:
self._url = url
Expand Down Expand Up @@ -362,7 +346,7 @@ def __str__(self):

class MountedBucket:
"""
Implementation of the Bucket interface backed by a normal file system.
Implementation of the BucketLike interface backed by a normal file system.
The targeted use case is the access to the BucketFS files from a UDF.
Arguments:
Expand Down
8 changes: 4 additions & 4 deletions exasol/bucketfs/_path.py
Original file line number Diff line number Diff line change
Expand Up @@ -469,9 +469,9 @@ def build_path(**kwargs) -> PathLike:
directory is as in the code below.
path = build_path(...) / "the_desired_path"
The rest of the arguments are backend specific.
The rest of the arguments are backend specific.
On-prem arguments:
On-prem arguments:
url:
Url of the BucketFS service, e.g. `http(s)://127.0.0.1:2580`.
username:
Expand All @@ -487,7 +487,7 @@ def build_path(**kwargs) -> PathLike:
service_name:
Optional name of the BucketFS service.
SaaS arguments:
SaaS arguments:
url:
Url of the Exasol SaaS. Defaults to 'https://cloud.exasol.com'.
account_id:
Expand All @@ -500,7 +500,7 @@ def build_path(**kwargs) -> PathLike:
Personal Access Token, e.g. 'exa_pat_aj39AsM3bYR9bQ4qk2wiG8SWHXbRUGNCThnep5YV73az6A'
(given example is not a valid PAT).
Mounted BucketFS directory arguments:
Mounted BucketFS directory arguments:
service_name:
Name of the BucketFS service (not a service url). Defaults to 'bfsdefault'.
bucket_name:
Expand Down

0 comments on commit 12d1899

Please sign in to comment.