Skip to content

Component ‐ Data Integrity Checks

Jan Bernitt edited this page Dec 11, 2024 · 10 revisions

Important

🔮 Vision

Changes or the general direction the feature is likely to take in the future (affecting 🧾: API 🖥️: app) ...

  1. 💭 🧾🖥️ Unify the issue refs so they always hold UIDs and add "metadata" so the type is either known from the YAML file or computed dynamically as part of the DB query. This is so we can support links to view/edit the conflicting object in the maintenance app
  2. ✨ 🧾🖥️ Persistence for the results so the last N results can be stored, viewed and compared to analyse trends and regressions
  3. ✨ 🧾 A job that uses the results history to find regressions and notify users
  4. ✨ 🧾 A job to run checks periodically based on their costs which can be learned from the new history
  5. ✨ 🧾🖥️ An action to auto-repair identified issues - the repair would be a SQL template in the YAML together with a description of what it does. The IDs of the identified issues are the parameters to the template to run the repair. Good examples would be remove trailing spaces from names
  6. ✨ 🧾(🖥️) Add JSON + CSV download for details

Usage Analysis for Actions

Tables that show which existing checks can already make use of actions.

The * indicates a situation where the action is clear but we might not know if consistency is maintained when executing it.

Fix individually

name id field template comment
dashboards_no_items id DELETE /api/dashboards/{id}
dashboards_not_viewed_one_year id DELETE /api/dashboards/{id}
maps_not_viewed_one_year id DELETE /api/maps/{id} might require to unlink from dashboards first
visualizations_not_viewed_one_year id DELETE /api/visualizations/{id} might require to unlink from dashboards first
categories_dimensions_no_visualizations id PATCH /api/categories/{id} + [{"op": "replace", "path": "/dataDimension", "value": "false"}] or use in visualizations
category_combos_unused* id ... delete if unused
cocs_wrong_cardinality* id ... delete if unused
category_option_combos_disjoint* id ... delete if unused
category_option_combos_have_duplicates* id + refs ... merge
category_options_excess_groupset_membership id + refs? ... edit: assign options to group
category_option_group_sets_incomplete id + refs? ... edit: assign options to group
category_options_no_categories id ... delete (or assign to category)
invalid_category_combos id ... delete (or add options to category)
data_elements_aggregate_abandoned id ... delete
data_elements_cannot_aggregate_operator_not_none id ... patch aggregationType to NONE
data_elements_violating_exclusive_group_sets id ... edit to adjust groups

** It depends** (multi action cases)

name id field templates comment
push_analysis_no_recipients id DELETE /api/pushAnalysis/{id}, add user group: GET dhis-web-maintenance/index.html#/edit/otherSection/pushAnalysis/{id}
categories_no_options id DELETE /api/category/{id}, add options: GET /dhis-web-maintenance/index.html#/edit/categorySection/category/{id}
categories_one_default_category id ... delete or rename
categories_one_default_category_combo id ... delete or rename
categories_one_default_category_option id ... delete or rename
categories_one_default_category_option_combo id ... delete or rename
categories_same_category_options* id ... delete category (if unused), or merge?
categories_unique_category_combo id ... delete if unused, or merge?
category_option_combos_no_names id ... delete or rename
data_elements_can_aggregate_with_none_operator id ... edit and set type or set type directly via PATCH to most common

Too complicated (no actions)

  • category_options_shared_within_category_combo
  • catoptioncombos_no_catcombo
  • data_elements_aggregate_with_different_period_types