Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use of Swift storage #19

Open
caida-ricky opened this issue Apr 27, 2022 · 0 comments
Open

Use of Swift storage #19

caida-ricky opened this issue Apr 27, 2022 · 0 comments

Comments

@caida-ricky
Copy link

caida-ricky commented Apr 27, 2022

I tried to use netacq-edge provider from CAIDA's network, and have swift credential setup. However, I got the following error when I tried to run the pyipmeta_test.py code in the test folder.

Testing netacq-edge with latest db...
Traceback (most recent call last):
  File "./pyipmeta_test.py", line 44, in <module>
    ipm = pyipmeta.IpMeta(providers=["netacq-edge"])
  File "build/bdist.linux-x86_64/egg/pyipmeta/pyipmeta.py", line 72, in __init__
  File "build/bdist.linux-x86_64/egg/pyipmeta/pyipmeta.py", line 130, in _reload
  File "build/bdist.linux-x86_64/egg/pyipmeta/dbidx.py", line 116, in __init__
  File "build/bdist.linux-x86_64/egg/pyipmeta/dbidx.py", line 137, in _load_index
swiftclient.exceptions.ClientException:
Auth versions 2.0 and 3 require python-keystoneclient, install it or use Auth
version 1.0 which requires ST_AUTH, ST_USER, and ST_KEY environment
variables to be set or overridden with -A, -U, or -K.

I had the keystoneclient installed. I can correctly list the objects with this code.

import os
from swiftclient.service import SwiftService, SwiftError
swift_opts = {
            # Apparently SwiftService by default checks only ST_AUTH_VERSION.
            # We emulate the swift CLI, and check three different variables.
            "auth_version": os.environ.get('ST_AUTH_VERSION',
                os.environ.get('OS_AUTH_VERSION',
                os.environ.get('OS_IDENTITY_API_VERSION', '1.0'))),
            }
with SwiftService(options=swift_opts) as swift:
    list_parts_gen = swift.list(container="datasets-external-netacq-edge-processed")
    for page in list_parts_gen:
        print(page["listing"])

But page["success"] was empty and and triggered line 136-137 in dbidx.py to throw an (unnecessary) error.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant