-
Notifications
You must be signed in to change notification settings - Fork 51
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Dq analytics level #622
Merged
Merged
Dq analytics level #622
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
sarah-tuva
approved these changes
Oct 21, 2024
bradmontierth
added a commit
that referenced
this pull request
Oct 22, 2024
commit 4ec9e4b Merge: b56bef4 055f119 Author: bradmontierth <[email protected]> Date: Tue Oct 22 09:12:50 2024 -0600 Merge pull request #626 from tuva-health/gitignore_updates updating gitignore for .envrc commit 055f119 Author: Tom Sherwood <[email protected]> Date: Tue Oct 22 09:33:56 2024 -0400 updating gitignore for .envrc commit b56bef4 Merge: 560bee9 6c7eb36 Author: bradmontierth <[email protected]> Date: Mon Oct 21 15:45:52 2024 -0600 Merge pull request #625 from tuva-health/patch-0.12.1 Patch v0.12.1 commit 6c7eb36 Author: sarah-tuva <[email protected]> Date: Mon Oct 21 14:57:14 2024 -0600 update dbt docs commit 560bee9 Author: Forrest Beckwith <[email protected]> Date: Mon Oct 21 16:16:16 2024 -0400 fixed schema typo (#624) fix schema typo fixed normalize__unmapped_observation schema suffix commit 264512d Author: bradmontierth <[email protected]> Date: Mon Oct 21 12:39:06 2024 -0600 Dq analytics level (#622) * Chronic condition prevalence check * Switching total_patients to use member_months * Updating final sql to get reference values * adding readmission checks * creating 2 union output tables for analytics checks * Chronic condition check updates * adding cms hcc checks * Finishing chronic condition checks * adding concept and measure to dq tables * Encounters start * adding inpatient checks * Updated encounters * adding inpatient drg * encounter pkpy and paid per trend * adding ed classification * service categories pmpm and trend * updates to chronic conditions and reference data * aligning column names between models for union * fix ci testing syntax issues * fixing syntax issues * fixing data type for bigquery * fixing chronic conditions and adding financial pmpm * fixing union all * fixing data type for big query * moving seed to s3 --------- Co-authored-by: Tom Sherwood <[email protected]> commit 34fc393 Author: thutuva <[email protected]> Date: Thu Oct 17 13:25:06 2024 -0400 Ci outside contributor fixes (#621) * added step to check out pr code to run through CI * troubleshooting * reverting ref syntax * edited BQ * save spot * added macro for limit 0 to fix fabric bug; added prov attribution testing; * added macro for limit 0 to fix fabric bug; added prov attribution testing; * added missing comma * fixed dependencies * added checkout step to BQ * upped threads in profiles.yml * updated job flow/dependencies * updated job flow/dependencies * set default in integration_tests dbt_project.yml; added testing of provider_attribution_enabled var only * fixed dupe job name * fixed job name * edited vars being executed in each job commit 11b92b8 Author: thutuva <[email protected]> Date: Tue Oct 15 16:01:43 2024 -0400 Refactor CI testing (#615) * removed old CI workflows; testing snowflake workflow * resolving conflicts * workflow edit * testing * edited Slack message * edited Slack message * updated all workflows to: latest dbt, post slack message, add fabric * fixed snowflake to run serial; code to parse BQ json creds * fixed snowflake to run serial; code to parse BQ json creds * edit BQ creds * edit BQ creds * edit BQ creds * edit BQ creds * edit BQ creds * edit BQ creds * edit BQ creds * edit BQ creds * edit BQ creds * edit BQ creds * troubleshooting * troubleshooting * troubleshooting * troubleshooting * troubleshooting * troubleshooting * troubleshooting * troubleshooting * troubleshooting * troubleshooting * troubleshooting * troubleshooting * troubleshooting * troubleshooting * troubleshooting * troubleshooting * troubleshooting * commented out debugging steps; uncommented dbt build * testing * testing * testing * troubleshooting * troubleshooting * troubleshooting * troubleshooting * troubleshooting * troubleshooting * troubleshooting * troubleshooting * troubleshooting * troubleshooting * troubleshooting * troubleshooting * archived old workflows; updated outside contributor with fabric; new workflow to consolidate old ones * archived old workflows; updated outside contributor with fabric; new workflow to consolidate old ones * fixed formatting * troubleshooting * troubleshooting * troubleshooting * troubleshooting * troubleshooting * uncommented out build for production * fixing data type bug commit faa38f9 Author: bradmontierth <[email protected]> Date: Tue Oct 15 13:32:53 2024 -0600 Align data quality column and table names with docs (#616) * align column and table names with docs * fix redshift syntax * update redshift syntax commit 0f498b3 Merge: 9196539 e44412c Author: eldon-tuva <[email protected]> Date: Fri Oct 11 18:21:07 2024 -0600 Merge pull request #614 from tuva-health/patch_0.12.0 branch for new release commit e44412c Author: Coco Zuloaga <[email protected]> Date: Fri Oct 11 17:56:00 2024 -0600 branch for new release commit 9196539 Merge: c81b536 7b61736 Author: eldon-tuva <[email protected]> Date: Fri Oct 11 16:52:48 2024 -0600 Merge pull request #607 from tuva-health/provider_attribution provider attribution input layer table added commit 7b61736 Merge: 408446a c81b536 Author: eldon-tuva <[email protected]> Date: Fri Oct 11 16:27:54 2024 -0600 Merge branch 'main' into provider_attribution commit c81b536 Author: bradmontierth <[email protected]> Date: Fri Oct 11 15:10:32 2024 -0600 Data Quality Checks (#613) * Adding initial eligibility dq checks * adding pharmacy and medical claim checks * added demographics query and edited model name * added demographics query and edited model name * renaming models per naming convention * edited models to count distinct eligibility spans; updated YAML; added new models * edited model name and table to match design; added data loss query * edited data_loss to add counts for eligibility spans and member months * moving dqi to own folder and data quality formatting * revert dbt_project back to default * changing tag for new models * removed data loss model * changed try_cast * fixed getdate() syntax * fix syntax on inst header fields * adding pharmacy and medical claim checks * renaming models per naming convention * moving dqi to own folder and data quality formatting * revert dbt_project back to default * changing tag for new models * fix syntax on inst header fields * fix big query data type error and add eligibility to data loss * moved dq audit script to final folder; edited getdate syntax; update YAML to include new models * moved dq audit script to final folder; edited getdate syntax; update YAML to include new models * fix syntax issue on data loss check * adding tables to test summary table * fixing redshift syntax * added syntax for redshift * testing redshift fix * fixing syntax for fabric --------- Co-authored-by: Tom Sherwood <[email protected]> Co-authored-by: thutuva <[email protected]> Co-authored-by: tuvaforrest <[email protected]> commit b8c8119 Author: bradmontierth <[email protected]> Date: Fri Oct 11 06:34:29 2024 -0600 Fix missing clinical paid amounts (#610) * fix encounter flag bug * fix clinical encounter paid amount bug * change rehab to rehabilitation commit 864ea4e Author: BijayaShres <[email protected]> Date: Fri Oct 11 01:16:11 2024 +0545 Quality Measure: Medication Adherence for Hypertension (ADH-RAS) (#597) * feat: performance period implementation * chore: denom case1 implementation * feat: denominator implemented * feat: adh-ras exclusion model implementation * fix: performance period reviews and comments fixed * fix: denominator comments applied * fix: minor issues denominator solved * fix: adhras denominator fixed * fix: denominator fixed * fix:denominator death patient data fixed * chore: final and yml modified * feat: adh_ras features added * Update models/quality_measures/intermediate/adh-ras_medication_adherence_for_hypertension/quality_measures__int_adhras_long.sql Co-authored-by: Utsav Paudel <[email protected]> * Update models/quality_measures/intermediate/adh-ras_medication_adherence_for_hypertension/quality_measures__int_adhras_numerator.sql Co-authored-by: Utsav Paudel <[email protected]> * fix: minor issues fixed * fix: cte name fixed in denominator * fix: cte with row_number updated in denominator * fix: numerator row number naming fixed * fix: bigquery typecast bug * fix: denominator cross join result fix * fix: naming in denominator and pdc logic in numerator updated * fix: treatment_period cte name changed in denominator * fix: redshift supportable feature added --------- Co-authored-by: Utsav Paudel <[email protected]> commit 0b471b0 Merge: 22b7a7d f9e8959 Author: bradmontierth <[email protected]> Date: Thu Oct 10 13:11:22 2024 -0600 Merge pull request #605 from tuva-health/data_quality_feature_604 added new data quality models commit f9e8959 Author: thutuva <[email protected]> Date: Thu Oct 10 09:13:02 2024 -0400 Update dbt_project.yml commit 408446a Author: Coco Zuloaga <[email protected]> Date: Wed Oct 9 17:24:56 2024 -0600 Update dbt_project.yml commit 3c1ca54 Author: Coco Zuloaga <[email protected]> Date: Wed Oct 9 13:56:23 2024 -0600 Update _integration_tests__models.yml commit 2ab99dd Author: thutuva <[email protected]> Date: Wed Oct 9 15:52:41 2024 -0400 fixed alias commit 83da5ee Author: thutuva <[email protected]> Date: Wed Oct 9 15:30:06 2024 -0400 added alias to fix fabric error; changed column name to result_count; update YAML with descriptions and reorganized commit 321c687 Author: thutuva <[email protected]> Date: Wed Oct 9 14:33:32 2024 -0400 fix ref syntax commit c565edd Merge: dcf756c 3c8f5e1 Author: thutuva <[email protected]> Date: Wed Oct 9 14:28:39 2024 -0400 Merge remote-tracking branch 'origin/data_quality_feature_604' into data_quality_feature_604 commit dcf756c Merge: e1d8c43 22b7a7d Author: thutuva <[email protected]> Date: Wed Oct 9 14:28:23 2024 -0400 Merge branch 'main' of https://github.com/tuva-health/tuva into data_quality_feature_604 commit a7ce2ae Merge: 809facf 22b7a7d Author: tuvaforrest <[email protected]> Date: Wed Oct 9 11:57:24 2024 -0400 Merge branch 'main' of https://github.com/tuva-health/tuva into provider_attribution commit 809facf Author: tuvaforrest <[email protected]> Date: Wed Oct 9 11:55:18 2024 -0400 changed seed to point to versioned synthetic version commit 3c8f5e1 Merge: e1d8c43 22b7a7d Author: thutuva <[email protected]> Date: Wed Oct 9 11:50:36 2024 -0400 Merge branch 'main' into data_quality_feature_604 commit e1d8c43 Author: thutuva <[email protected]> Date: Wed Oct 9 11:45:00 2024 -0400 updated queries to pull from input_layer commit 22b7a7d Author: Utsav Paudel <[email protected]> Date: Wed Oct 9 00:59:53 2024 +0545 New Quality Measure: Medication Adherence for Diabetes (#603) * feat: add days_supply field in pharmacy_claims staging * feat: performance period model * fix: days_supply in pharmacy claim for fabric * feat: qm adh diabetes intermediate models * fix: numerator cte reference * fix: pdc rounding bug fix * fix: cte names change * feat: final tables update * docs: adh_diabetes measure documentation * fix: aesthetic fixes * fix: rolling sum redshift syntax commit f0eb954 Author: thutuva <[email protected]> Date: Mon Oct 7 11:11:36 2024 -0400 syntax changes for BQ commit 287d0cd Author: tuvaforrest <[email protected]> Date: Mon Oct 7 09:36:55 2024 -0400 removed bad config commit 7bf4190 Author: thutuva <[email protected]> Date: Mon Oct 7 09:32:38 2024 -0400 reverting profile commit 3bd8aa2 Merge: f198a6f f854740 Author: thutuva <[email protected]> Date: Mon Oct 7 09:32:07 2024 -0400 Merge branch 'main' of https://github.com/tuva-health/tuva into data_quality_feature_604 commit f198a6f Author: thutuva <[email protected]> Date: Mon Oct 7 09:30:22 2024 -0400 syntax changes for fabric commit 7a03a73 Author: thutuva <[email protected]> Date: Mon Oct 7 09:30:21 2024 -0400 syntax changes for fabric commit e1389cd Author: Forrest Beckwith <[email protected]> Date: Fri Oct 4 17:08:08 2024 -0400 removed provider attribution ~ file commit 08ceb11 Author: tuvaforrest <[email protected]> Date: Fri Oct 4 16:58:48 2024 -0400 added seed yml, changed default for provider_attribution_enabled in integration_tests commit 142b306 Author: Coco Zuloaga <[email protected]> Date: Fri Oct 4 14:19:25 2024 -0600 fix provider attribution in ci testing commit 1ff52b4 Author: Coco Zuloaga <[email protected]> Date: Fri Oct 4 14:12:45 2024 -0600 fix redshift yml commit b16d726 Author: Coco Zuloaga <[email protected]> Date: Fri Oct 4 14:08:11 2024 -0600 changes to medical claim in ci testing commit 57423cb Author: Coco Zuloaga <[email protected]> Date: Fri Oct 4 14:04:37 2024 -0600 fix ci testing model commit 91a9cbf Merge: 32a8883 87773df Author: Coco Zuloaga <[email protected]> Date: Fri Oct 4 13:58:43 2024 -0600 Merge branch 'provider_attribution' of https://github.com/tuva-health/tuva into provider_attribution commit 32a8883 Author: Coco Zuloaga <[email protected]> Date: Fri Oct 4 13:55:31 2024 -0600 add ci testing model commit 87773df Merge: aee4e26 f854740 Author: Coco Zuloaga <[email protected]> Date: Fri Oct 4 13:48:36 2024 -0600 Merge branch 'main' into provider_attribution commit aee4e26 Author: Coco Zuloaga <[email protected]> Date: Fri Oct 4 13:40:03 2024 -0600 remove unnecessary files commit f01a97b Author: Coco Zuloaga <[email protected]> Date: Fri Oct 4 13:39:30 2024 -0600 changes to pmpm prep commit a5b962b Author: Coco Zuloaga <[email protected]> Date: Fri Oct 4 13:16:44 2024 -0600 added columns to financial_pmpm__pmpm_prep commit 18351d0 Author: Coco Zuloaga <[email protected]> Date: Fri Oct 4 12:33:26 2024 -0600 provider attribution input layer table added commit f854740 Author: tom-tuva <[email protected]> Date: Fri Oct 4 13:32:27 2024 -0400 Doc blocks fix (#606) * Moving column_descriptions.md to its own folder. * Updating docs for doc blocks. commit 1a04b3a Merge: 8d9e63b fa273ca Author: thutuva <[email protected]> Date: Fri Oct 4 11:03:28 2024 -0400 Merge branch 'main' into data_quality_feature_604 commit 8d9e63b Author: thutuva <[email protected]> Date: Fri Oct 4 10:34:42 2024 -0400 edited query for BQ commit fa273ca Author: bradmontierth <[email protected]> Date: Fri Oct 4 08:17:03 2024 -0600 adding service category 3 to medical_claim (#602) commit 1cf562a Author: thutuva <[email protected]> Date: Fri Oct 4 08:34:59 2024 -0400 added tuva_last_run commit 7ba7bea Author: thutuva <[email protected]> Date: Fri Oct 4 08:30:15 2024 -0400 added new data quality models commit e27e9da Author: Addy Naik <[email protected]> Date: Thu Oct 3 15:35:45 2024 -0400 fix schemas template for claims preprocessing (#598) fix schemas commit 72d8db0 Merge: 6253700 ffbf02e Author: eldon-tuva <[email protected]> Date: Wed Oct 2 15:31:12 2024 -0600 Merge pull request #599 from tuva-health/doc_blocks Doc blocks commit ffbf02e Merge: 9207317 6253700 Author: tom-tuva <[email protected]> Date: Wed Oct 2 16:23:11 2024 -0400 Merge branch 'main' into doc_blocks commit 6253700 Author: eldon-tuva <[email protected]> Date: Wed Oct 2 13:46:42 2024 -0600 Update access key (#600) commit 9207317 Merge: ebb4522 d85099e Author: Tom Sherwood <[email protected]> Date: Wed Oct 2 12:44:08 2024 -0400 Merge branch 'doc_blocks' of https://github.com/tuva-health/tuva into doc_blocks commit ebb4522 Author: Tom Sherwood <[email protected]> Date: Wed Oct 2 12:43:50 2024 -0400 Removing unique tests on encounter_id. commit d85099e Merge: e22ee63 c48bad4 Author: tom-tuva <[email protected]> Date: Wed Oct 2 12:00:56 2024 -0400 Merge branch 'main' into doc_blocks commit e22ee63 Author: Tom Sherwood <[email protected]> Date: Wed Oct 2 11:34:41 2024 -0400 Adding remaining doc block column definitions commit c48bad4 Author: mgoldhirsh <[email protected]> Date: Wed Oct 2 11:16:36 2024 -0400 noticed a bug with institutional procedures, this fixes it (#596) commit caf82b1 Author: Tom Sherwood <[email protected]> Date: Wed Oct 2 10:02:26 2024 -0400 Adding doc blocks for column definitions in encounter models commit e4908b7 Author: Addy Naik <[email protected]> Date: Tue Oct 1 13:06:29 2024 -0400 fix the space in the generated schema name template (#594) Update pharmacy_models.yml commit 2563687 Author: tom-tuva <[email protected]> Date: Fri Sep 27 18:49:01 2024 -0400 Release for version 0.11.0 (#595) commit 9873562 Merge: af81e97 3a52e89 Author: Coco Zuloaga <[email protected]> Date: Tue Sep 24 15:56:42 2024 -0600 Merge pull request #592 from tuva-health/add_schema_extract_script added helper scripts folder commit af81e97 Author: bradmontierth <[email protected]> Date: Tue Sep 24 14:27:29 2024 -0600 Encounters and Service Categories (#585) Updating service categories and adding new encounter types. commit 3a52e89 Author: tuvaforrest <[email protected]> Date: Tue Sep 24 16:15:09 2024 -0400 created helper scripts folder, added script to extract tuva project schema and readme. commit 85f9fd2 Merge: 4e055eb 2a5dd60 Author: Coco Zuloaga <[email protected]> Date: Thu Sep 19 10:21:14 2024 -0600 Merge pull request #590 from tuva-health/patch_0.10.8 patch 0.10.8 commit 2a5dd60 Author: thutuva <[email protected]> Date: Thu Sep 19 10:39:56 2024 -0400 updated docs commit a9b8873 Author: thutuva <[email protected]> Date: Thu Sep 19 09:38:59 2024 -0400 updated dbt_project.yml commit 4e055eb Merge: b2fb6f3 0c93b70 Author: Coco Zuloaga <[email protected]> Date: Wed Sep 18 12:09:24 2024 -0600 Merge pull request #589 from tuva-health/ed_encounter_age_bug_fix removed column admit_age commit 0c93b70 Author: thutuva <[email protected]> Date: Wed Sep 18 13:32:01 2024 -0400 removed column admit_age commit b2fb6f3 Author: Utsav Paudel <[email protected]> Date: Tue Sep 17 03:18:07 2024 +0545 New Quality Measure NQF0420: Pain Assessment and Follow-Up (#586) * feat: denominator raw implementation * feat: exclusions and numerator raw implementation * docs: nqf0420 qm models documentation * feat: nqf0420 final tables * fix: tuva_last_run added in numerator * fix: POS 02 logic implementation commit 0825261 Merge: 6db8904 8442d07 Author: Coco Zuloaga <[email protected]> Date: Tue Sep 10 14:49:07 2024 -0600 Merge pull request #583 from tuva-health/cocozuloaga-integration_tests_input_layer_schema Update _integration_tests__models.yml to use input_layer schema commit 8442d07 Author: Coco Zuloaga <[email protected]> Date: Fri Sep 6 18:49:53 2024 -0600 Update _integration_tests__models.yml to use input_layer schema Replace _tuva_claims and _tuva_clinical schemas with an input_layer schema for input layer tables inside integration_tests.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Describe your changes
Please include a summary of any changes.
adding analytics data quality checks
How has this been tested?
Please describe the tests you ran to verify your changes. Provide instructions or code to reproduce output.
ran on real data and viewed output of union tables
ci testing
Reviewer focus
Please summarize the specific items you’d like the reviewer(s) to look into.
Checklist before requesting a review
tuva_last_run
to the final outputPackage release checklist
dbt_project.yml
(Optional) Gif of how this PR makes you feel
Loom link