[BUG]: PermissionError: Access Denied when downloading DFP example data

[dagardner-nv]

Version

24.10

Which installation method(s) does this occur on?

Source

Describe the bug.

Running fetch_example_data.py in a fresh git checkout results in an access denied error.

The script will skip over any files already downloaded, so this cannot be reproduced if the dataset has already been downloaded.

Minimum reproducible example

python examples/digital_fingerprinting/fetch_example_data.py all

Relevant log output

Click here to see error details

$ python examples/digital_fingerprinting/fetch_example_data.py all

Downloading AZUREAD_2022-08-01T00_03_56.207Z.json

Traceback (most recent call last):

File "/home/dagardner/work/conda/envs/morpheus-2410-reltest2/lib/python3.10/site-packages/s3fs/core.py", line 394, in _lsdir

for i in it:

File "/home/dagardner/work/conda/envs/morpheus-2410-reltest2/lib/python3.10/site-packages/botocore/paginate.py", line 269, in iter

response = self._make_request(current_kwargs)

File "/home/dagardner/work/conda/envs/morpheus-2410-reltest2/lib/python3.10/site-packages/botocore/paginate.py", line 357, in _make_request

return self._method(**current_kwargs)

File "/home/dagardner/work/conda/envs/morpheus-2410-reltest2/lib/python3.10/site-packages/botocore/client.py", line 569, in _api_call

return self._make_api_call(operation_name, kwargs)

File "/home/dagardner/work/conda/envs/morpheus-2410-reltest2/lib/python3.10/site-packages/botocore/client.py", line 1023, in _make_api_call

raise error_class(parsed_response, operation_name)

botocore.exceptions.ClientError: An error occurred (AccessDenied) when calling the ListObjectsV2 operation: Access Denied
During handling of the above exception, another exception occurred:
Traceback (most recent call last):

File "/home/dagardner/work/morpheus-2410-reltest2/examples/digital_fingerprinting/fetch_example_data.py", line 588, in 

main()

File "/home/dagardner/work/morpheus-2410-reltest2/examples/digital_fingerprinting/fetch_example_data.py", line 584, in main

fetch_dataset(dataset)

File "/home/dagardner/work/morpheus-2410-reltest2/examples/digital_fingerprinting/fetch_example_data.py", line 554, in fetch_dataset

fs_hndl.get_file(os.path.join(s3_base_path, f), train_dir + f)

File "/home/dagardner/work/conda/envs/morpheus-2410-reltest2/lib/python3.10/site-packages/fsspec/spec.py", line 890, in get_file

elif self.isdir(rpath):

File "/home/dagardner/work/conda/envs/morpheus-2410-reltest2/lib/python3.10/site-packages/s3fs/core.py", line 601, in isdir

return bool(self._lsdir(path))

File "/home/dagardner/work/conda/envs/morpheus-2410-reltest2/lib/python3.10/site-packages/s3fs/core.py", line 409, in _lsdir

raise translate_boto_error(e)

PermissionError: Access Denied

Full env printout

Click here to see environment details

[Paste the results of print_env.sh here, it will be hidden by default]

Other/Misc.

No response

Code of Conduct

• I agree to follow Morpheus' Code of Conduct
• I have searched the open bugs and have found no duplicates for this bug report