Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

update_visit_data: intelligently filter out configured responses? #372

Open
shippy opened this issue Mar 10, 2020 · 0 comments
Open

update_visit_data: intelligently filter out configured responses? #372

shippy opened this issue Mar 10, 2020 · 0 comments

Comments

@shippy
Copy link
Contributor

shippy commented Mar 10, 2020

With the latest influx of variables into Data Entry

  • lssaga1_youth_dm15d_dm15d_1_dm15d_y and "lssaga1_youth_dm15d_dm15d_2_dm15d_y have a bunch of In progress responses, some . responses
  • mrireport_mri_set*_*_mrilu6_mricns90[dh] have a bunch of . responses.

This is clocking at about 83 issues. The standard SOP is to ask the sites to go fix it in the Import project, but given the homogeneity of the errors, it might not be a bad idea to implement some filtering in update_visit_data. Implementation sketch:

  1. a config section in sibis_sys_config.yml of the form
    update_visit_data:
      ignored_values:
        lssaga1_youth_dm15d_dm15d_1_dm15d_y:
          - "."
          - "In progress"
  2. a method in update_visit_data that goes through ignored values per field and empties them out with something like data.loc[data[varname] == value, varname] = np.nan
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant