Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SMaHT ingestion related work. #293

Merged
merged 50 commits into from
Nov 30, 2023
Merged

Conversation

dmichaels-harvard
Copy link
Contributor

No description provided.

@coveralls
Copy link

coveralls commented Nov 20, 2023

Pull Request Test Coverage Report for Build 7045798624

Warning: This coverage report may be inaccurate.

We've detected an issue with your CI configuration that might affect the accuracy of this pull request's coverage report.
To ensure accuracy in future PRs, please see these guidelines.
A quick fix for this PR: rebase it; your next report should be accurate.

  • 195 of 455 (42.86%) changed or added relevant lines in 8 files are covered.
  • 7 unchanged lines in 2 files lost coverage.
  • Overall coverage decreased (-1.5%) to 77.125%

Changes Missing Coverage Covered Lines Changed/Added Lines %
dcicutils/sheet_utils.py 65 83 78.31%
dcicutils/bundle_utils.py 66 100 66.0%
dcicutils/misc_utils.py 50 86 58.14%
dcicutils/zip_utils.py 0 60 0.0%
dcicutils/data_readers.py 0 112 0.0%
Files with Coverage Reduction New Missed Lines %
dcicutils/bundle_utils.py 2 83.98%
dcicutils/sheet_utils.py 5 85.87%
Totals Coverage Status
Change from base Build 6908989375: -1.5%
Covered Lines: 9471
Relevant Lines: 12280

💛 - Coveralls

Copy link
Member

@willronchetti willronchetti left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some small comments, nothing major

Comment on lines +12 to +25
def unpack_zip_file_to_temporary_directory(file: str) -> str:
with temporary_directory() as tmp_directory_name:
with zipfile.ZipFile(file, "r") as zipf:
zipf.extractall(tmp_directory_name)
yield tmp_directory_name


@contextmanager
def unpack_tar_file_to_temporary_directory(file: str) -> str:
with temporary_directory() as tmp_directory_name:
with tarfile.open(file, "r") as tarf:
tarf.extractall(tmp_directory_name)
yield tmp_directory_name

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You might want to allow optional args to these functions in case additional options are needed/desired

Comment on lines +54 to +57
with tempfile.TemporaryDirectory() as tmp_directory_name:
yield tmp_directory_name
finally:
remove_temporary_directory(tmp_directory_name)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought temp directories were removed automatically?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh yeah, wait a minute, I think you should be right; I think I did this for a reason, need to remind myself.

Comment on lines +62 to +63
def __init__(self):
super().__init__()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this not the default behavior?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No it's not. A derived init must explicitly call its base init, if it wants that behavior.

Comment on lines +566 to +571
found = True
for item in item_ref:
if not lookup_table.get(item):
found = False
break
return True if found else None
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why mixed return value (None + bool, why not just bool). Also, why not just

for item in item_ref:
     if not lookup_table.get(item):
           return None
return True 

Comment on lines +600 to +606
found = True
for item in item_ref:
info = get_metadata(f"/{to_camel_case(item_type)}/{item}",
ff_env=self.portal_env, vapp=self.portal_vapp)
if not isinstance(info, dict) or 'uuid' not in info:
found = False
break
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same pattern here as above can be simplified

@dmichaels-harvard dmichaels-harvard merged commit 9474417 into master Nov 30, 2023
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants