Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add check-local-portal-creds #244

Open
wants to merge 4 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions CHANGELOG.rst
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,15 @@ Change Log
----------


7.2.0
=====

* Add command ``check-local-portal-creds``.
* In ``Makefile``:
* Add ``test-full`` (like ``test``, but not instafail)
* Enable compatibility warnings about SQLAlchemy 2.0 transition.


7.1.3
=====

Expand Down
13 changes: 10 additions & 3 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -29,15 +29,22 @@ build:
test:
@git log -1 --decorate | head -1
@date
pytest -xvv --instafail --timeout=200
SQLALCHEMY_WARN_20=1 pytest -xvv --instafail --timeout=200
@git log -1 --decorate | head -1
@date

test-full:
@git log -1 --decorate | head -1
@date
SQLALCHEMY_WARN_20=1 pytest -vv --timeout=200
@git log -1 --decorate | head -1
@date

remote-test-npm:
poetry run pytest -xvvv --timeout=400 --durations=100 --aws-auth --es search-fourfront-testing-opensearch-kqm7pliix4wgiu4druk2indorq.us-east-1.es.amazonaws.com:443 -m "indexing"
SQLALCHEMY_WARN_20=1 poetry run pytest -xvvv --timeout=400 --durations=100 --aws-auth --es search-fourfront-testing-opensearch-kqm7pliix4wgiu4druk2indorq.us-east-1.es.amazonaws.com:443 -m "indexing"

remote-test-unit:
poetry run pytest -xvvv --timeout=400 --durations=100 --aws-auth --es search-fourfront-testing-opensearch-kqm7pliix4wgiu4druk2indorq.us-east-1.es.amazonaws.com:443 -m "not indexing"
SQLALCHEMY_WARN_20=1 poetry run pytest -xvvv --timeout=400 --durations=100 --aws-auth --es search-fourfront-testing-opensearch-kqm7pliix4wgiu4druk2indorq.us-east-1.es.amazonaws.com:443 -m "not indexing"

update:
poetry update
Expand Down
3 changes: 2 additions & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[tool.poetry]
name = "dcicsnovault"
version = "7.1.3"
version = "7.2.0"
description = "Storage support for 4DN Data Portals."
authors = ["4DN-DCIC Team <[email protected]>"]
license = "MIT"
Expand Down Expand Up @@ -144,6 +144,7 @@ PyYAML = ">=5.1,<5.5"
wheel = ">=0.29.0"

[tool.poetry.scripts]
check-local-portal-creds = "snovault.commands.check_local_portal_creds:main"
wipe-test-indices = "snovault.commands.wipe_test_indices:main"

[build-system]
Expand Down
78 changes: 78 additions & 0 deletions snovault/commands/check_local_portal_creds.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,78 @@
import os
import argparse

from dcicutils.misc_utils import PRINT
from dcicutils.command_utils import script_catch_errors


REPO_NAME = os.path.basename(os.path.abspath(os.curdir))


def check_local_creds(appname):

if not appname:
if 'cgap' in REPO_NAME:
appname = 'cgap'
elif 'ff' in REPO_NAME or 'ffourfront' in REPO_NAME:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you mean 'fourfront' not 'ffourfront'?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep. Thanks for spotting that.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dmichaels-harvard, I re-implemented some details of this PR in a more data-driven style to make it harder for such typos to creep in. The whole thing was too dependent on incidental string constants. See commit de10c89 for details.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK and commit b1c291d adds a bit more change.

appname = 'fourfront'
else:
PRINT("Can't figure out if this is cgap or fourfront.")
exit(1)

class WarningMaker:

count = 0

@classmethod
def warn(cls, *args, **kwargs):
cls.count += 1
return PRINT(*args, **kwargs)

warn = WarningMaker.warn

if appname not in ['fourfront', 'ff', 'cgap']:
raise RuntimeError(f"Unknown appname {appname!r}. Expected 'cgap' or 'fourfront' (or 'ff').")

global_env_bucket = os.environ.get('GLOBAL_ENV_BUCKET')
cgap_env_bucket = 'cgap-devtest-main-foursight-envs'
fourfront_env_bucket = 'foursight-prod-envs'

def check_global_env_bucket(var, val):
if appname == 'cgap' and val != cgap_env_bucket:
warn(f"{var} is {val!r}, but should be {cgap_env_bucket}.")
elif appname == 'fourfront' and val != fourfront_env_bucket:
warn(f"{var} is {val!r}, but should be {fourfront_env_bucket}.")

check_global_env_bucket('GLOBAL_ENV_BUCKET', global_env_bucket)
global_bucket_env = os.environ.get('GLOBAL_BUCKET_ENV')
if global_bucket_env:
if global_bucket_env == global_env_bucket:
warn("GLOBAL_BUCKET_ENV is the same as GLOBAL_ENV_BUCKET,"
" but you can just get rid of GLOBAL_BUCKET_ENV now.")
elif not global_env_bucket:
warn("You need to set GLOBAL_ENV_BUCKET, not GLOBAL_BUCKET_ENV.")
check_global_env_bucket('GLOBAL_BUCKET_ENV', global_bucket_env)
for var in ['CHECK_RUNNER', 'ACCOUNT_NUMBER', 'ENV_NAME']:
if os.environ.get(var):
warn(f"The variable {var} has a non-null value but should be unset.")
for var in ['Auth0Client', 'Auth0Secret']:
if not os.environ.get(var):
warn(f"The variable {var} has no value but should be set.")
if WarningMaker.count == 0:
PRINT("Things look good.")


def main():
parser = argparse.ArgumentParser(
description='Echos version information from ~/.cgap-keys.json or override file.')
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This description seems lifted from somewhere else

parser.add_argument('--appname', help='Name of app to check for (cgap or ff/fourfront)', type=str, default=None)
args = parser.parse_args()

appname = args.appname

with script_catch_errors():
check_local_creds(appname=appname)


if __name__ == '__main__':
main()