Load data dictionary case properties in chunks

[kaapstorm]

Product Description

Improves the performance of the Data Dictionary page for projects with high numbers of case types and case properties.

Technical Summary

Context:

• SC-3698
• Spec

This change allows the data dictionary to support tens of thousands of case properties. It does this by fetching only the case properties of the selected case type, and by chunking the case properties over multiple requests if there are a lot.

This change builds on work currently in open PRs. Commits are rebased as follows:

1. master
2. ze/dd-only-save-changed-props (PR Only Save Changed Case Properties in Data Dictionary #34781 )
3. ay/data-dict-chunk-load (PR New view for Data Dict JSON to support chunk load #34787 )
4. These new commits, starting with "propertyListItem and groupsViewModel subscribe"

Feature Flag

N/A. Available on all plans with Data Dictionary.

Safety Assurance

Safety story

• Tested locally
• Tested on Staging

Automated test coverage

Code changes are in JavaScript viewmodels and not covered by tests.

QA Plan

QA ticket: QA-6727

Rollback instructions

• This PR can be reverted after deploy with no further considerations

Labels & Review

• Risk label is set correctly
• The set of people pinged as reviewers is appropriate for the level of risk of the change

[kaapstorm]

I am sorry for the rebase, but I hope it makes it easier to review now that the base branches are merged.

[zandre-eng]

The data that we pass as prop to this model has the field for is safe to delete stored as is_safe_to_delete and not isSafeToDelete. We'll need to change that here to correctly fetch the value. This should resolve the bug where the delete button is not showing.

[zandre-eng]

After looking at the code, it seems that groups with no case properties won't be loaded to the front-end because of what's happening here in the view. Specifically, a query is done on case properties, fetching related groups. Any groups without case properties will therefore not be in the query.

[zandre-eng]

Happy to approve once this has been given a final test pass on staging.

[kaapstorm]

Deployed to staging.

@zandre-eng do you mean that you or I can check this on Staging, or that the QA Team should give this another pass?

The QA Team's domain is here: https://staging.commcarehq.org/a/qateam/data_dictionary/

The QA ticket is here: https://dimagi.atlassian.net/browse/QA-6727

[zandre-eng]

@kaapstorm I think if QA is able to do a final pass then that should be fine.

[Charl1996]

I admittedly don't have a lot of context around the data dictionary, hence me asking this: what is the group in this context and why do we check whether there is a group here?

[kaapstorm]

When the user selects a case type, the viewmodel fetches its case properties.

The server returns the CaseProperty instances nested in their CasePropertyGroup instances ("data.groups" on line 96). (The CaseProperty instances that don't have a CasePropertyGroup are nested in a group with an empty name.)

So if self.groups() does not have a value, it means that the viewmodel has not fetched its case properties already, so it should do that now.

[zandre-eng]

QA has passed on this PR, and so all looks good from my end to approve.