Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Exporting variables from root, hidden and secure #361

Open
jamesrkg opened this issue Jul 1, 2019 · 0 comments
Open

Exporting variables from root, hidden and secure #361

jamesrkg opened this issue Jul 1, 2019 · 0 comments
Milestone

Comments

@jamesrkg
Copy link

jamesrkg commented Jul 1, 2019

This ticket is partially influenced the the conversation here:

https://yougov.slack.com/messages/C2MLN3SNS/convo/C2MLN3SNS-1561587638.058400/

Currently, ds.export's hidden and variables kwargs are mutually exclusive and trigger this code:

scrunch/scrunch/datasets.py

Lines 2217 to 2231 in c857398

# hidden is mutually exclusive with
# variables to include in the download
if hidden and not variables:
if not self.resource.body.permissions.edit:
raise AttributeError(
"Only Dataset editors can export hidden variables")
payload['where'] = {
'function': 'select',
'args': [{
'map': {
x: {'variable': x}
for x in self.resource.variables.index.keys()
}
}]
}

With the addition of the secure tree this is now outdated, since hidden=True currently also exports all secure variables as well though based on the above linked conversation that is likely to change on the server side. We should update this in so doing improve flexibility and control overall.

Current behavior:

  • variables
    • None - as controlled by the API (this will export non-hidden and non-secure variables)
    • list - aliases of variables to export, these can be any variables be they hidden, secure or not. Passing a list here will also supersede hidden, which will be ignored.
  • hidden
    • True - export all hidden variables, will be ignored if a list is passed to variables.
    • False - do not export the hidden variables, will be ignored if a list is passed to variables.

Goals:

  • Remove mutual exclusivity between variables and hidden kwargs
  • Add secure kwarg (or private??)
  • Support simple way to export some, none or all non-hidden, non-secure variables
  • Support simple way to export some, none or all hidden variables
  • Support simple way to export some, none or all secure variables
  • With a combination of the above, throw clear error if no variables have been requested
  • Implementing the above will be simplified by the recent addition of hidden and secure attributes to the variables catalog.
  • Safeguard the export of hidden and/or secure variables in principle, separate to whether or not they are explicitly requested (since the location of these variables can change over time).

Proposal:

  • variables
    • None - as controlled by the API (this will export non-hidden and non-secure variables)
    • list - aliases of variables to export, these can be any variables be they hidden, secure or neither.
  • hidden
    • None - no hidden variables will be exported, even if they are in a list passed to variables.
    • True - automatically export all hidden variables, in addition to any other variables passed as a list passed to variables
    • False - do not automatically export all hidden variables but hidden variables in a list passed to variables will be exported.
  • secure
    • None - no secure variables will be exported, even if they are in a list passed to variables.
    • True - automatically export all secure variables, in addition to any other variables passed as a list passed to variables
    • False - do not automatically export all secure variables but secure variables in a list passed to variables will be exported.
@jamesrkg jamesrkg added this to the Wishlist milestone Jul 1, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant