Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: dataverse export #909

Merged
merged 8 commits into from
Jan 28, 2020
Merged

feat: dataverse export #909

merged 8 commits into from
Jan 28, 2020

Conversation

m-alisafaee
Copy link
Contributor

Description

Enables exporting datasets to Dataverse. Fixes wrong file names after import.

Fixes #908
Fixes #668

@m-alisafaee m-alisafaee requested a review from a team as a code owner January 15, 2020 15:30
@rokroskar rokroskar changed the title 668 dataverse export feat: dataverse export Jan 15, 2020
Copy link
Contributor

@jsam jsam left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great. Thank you for looking into this.

def dataverse_demo(client):
"""Configure environment to use Dataverse demo environment."""
client.set_value(
'dataverse', 'access_token', '4ca13597-cf43-4815-8763-b64994058c19'
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Access token should probably be extracted into env variable and encrypted for the CI.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This a free account on their demo server (same as the one above for Zenodo). I'd prefer to keep it like this for easier execution of tests in our local machines.

@@ -269,8 +269,26 @@ def test_dataset_reimport_removed_dataset(runner, project, sleep_after):


@pytest.mark.integration
def test_dataset_import_preserve_names(runner, project, sleep_after):
"""Test import keeps original file names."""
DOI = '10.7910/DVN/F4NUMR'
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's not define constants in local scopes, otherwise pylint is crying. Just make it normal variable (lower case).

url_parts = url_parts._replace(path=path, query=query_params)
return urllib.parse.urlunparse(url_parts)

def _post(self, url, json=None, data=None, files=None):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should make this more robust, with #902 merged there is a new utility for retrying.
With it you could:

with retry() as session:
    session.post(...)

@@ -269,8 +269,26 @@ def test_dataset_reimport_removed_dataset(runner, project, sleep_after):


@pytest.mark.integration
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With #902, there is a new decorator with which you can specify integration tests for a rerun

@flaky(...)
...

This should make IT more robust in case of failure when running this in parallel.

@rokroskar rokroskar requested a review from jsam January 24, 2020 17:41
Copy link
Contributor

@jsam jsam left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thank you! 🎉

@m-alisafaee m-alisafaee merged commit 7e9e647 into master Jan 28, 2020
@m-alisafaee m-alisafaee deleted the 668-dataverse-export branch January 28, 2020 11:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Incorrect file names after import enable export to dataverse
2 participants