feat(DHIS2-16132): transpose section forms #367

kabaros · 2024-02-20T17:01:04Z

This PR implements the ability to display sections transposed, and grouped by category in the data entry app. Implements DHIS2-16132 .

transpose-different-options.webm

Default Section form - without pivoting (existing implementation)

Form pivoted (data elements moved to columns)

Single Category pivoted

Multiple categories (with one pivoted)

Data elements with different category combos

Notes about implementation

As an general approach, I aimed to decouple the display details (table, rows, columns, column and row spans) from our internal concepts (data elements, categories, category options, cocs etc...), to provide a middle layer (a matrix) that represents a form in our context. This middle layer can become the base of a form builder, for implementations that are cross-platform and other improvements in the future.

With the previous implementation, we were more than half-way there, but the data elements were tightly coupled to rows, and in turn the rows/data elements are tightly coupled to functionality like search, filters, highlighting current cell in the table etc... The next step should be to consolidate this implementation with the "normal" section form - there is a followup ticket for this: https://dhis2.atlassian.net/browse/LIBS-584

The only concept that remains metadata-aware is the date entry field - this was the existing design and it works well, as a self-contained component responsible of its own state, validation etc... regardless of where it appears in a form.

I avoided changing the original CategoryComboTable and added a new PivotedCategoryComboTable to handle the new use cases for transposing. I didn't want to change something that already works and well-tested for what's essentially an experimental new feature (i.e. not part of achieving parity with the existing app). Next step is to consolidate theses two components and the underlying matrix generation.
In the grouped matrix logic, I added a special case for when there is one category and another for 1+ categories. This is needed because we need a "shadow" row in the case of more categories, this is unnecessary for one category. See the photo below, the first one has a "shadow" row (location fixed/outreach) and the categories are displayed underneath each other - this is necessary for multiple categories. In the case of one category (second table), we can do with the category being on top of the options.

Background

We are aiming to add more form configuration options as part of an initiative to provide configurations natively to data entry forms to reduce the necessity for custom forms. Users are currently building custom forms as a workaround for shortcomings of the configuration options (ability to transpose, or customise a cell design) or implementation (such to avoid issues with RTL issues). This is an RFC that describes the approach and the priorities for form configuration options. This is based on a thorough investigation by the functional design team for custom form use cases in real-life implementations. Based on that investigation, the ability to transpose forms and adding free-text descriptions were one of the main reasons people choose to go the custom forms route so we're tackling these first.

Known limitations

Transpose (not grouping) doesn't work for a dataset with more than one category combo
Support multiple transposed categories (right now, we transpose only one)
Search and filtering doesn't work as these were tightly couple with data elements - we will need to design and rethink that for transposed sections.

ToDos (future)

Consolidate CategoryComboTable and PivotedCategoryComboTable, as well as underlying "matrix" logic: https://dhis2.atlassian.net/browse/LIBS-584

Related PRs:

dhis2-bot · 2024-02-20T17:03:00Z

🚀 Deployed on https://pr-367--dhis2-data-entry.netlify.app

feat: add ability to transpose/pivot a section form refactor: change file structure to separate transposed from grouped test: add missed tests

kabaros · 2024-03-05T09:06:41Z

src/shared/use-context-selection/use-context-selection.js

@@ -27,6 +27,20 @@ export function usePeriodId() {
    return useQueryParam('periodId', PARAMS_SCHEMA.periodId)
 }

+export function useFeature(feature) {


I added this "feature toggle" in order to be able to test the functionality without the backend and maintenance app deployed.

This makes it easy for testing 😅, but I'm not sure we actually would want to expose a query parameter like this? I think people who design data sets would not want their users to arbitrarily transpose things (I assume we're not going to advertise this, but I guess it's possible that people share links if we expose it?)

There aren't really checks for this, so providing invalid categories causes some issues, but maybe that's not of concern if it's just for testing?

kabaros · 2024-03-05T09:08:02Z

...rkspace/category-combo-table-body-pivoted/generate-form-matrix/__fixtures/pivot-test-data.js

I needed a test that mimics what we do before (based on the whole metadata) - this file can/should be significantly reduced to only include what's relevant to the test but that wasn't easy to do. I have it in my list for the refactors after (as well as getting rid of snapshot tests)

kabaros · 2024-03-05T09:09:39Z

src/data-workspace/category-combo-table-body-pivoted/category-combo-table-body-pivoted.js

+                                    )
+                                }
+
+                                if (fieldInRow.type === 'de') {


one thing about my implementation of the matrix generation is that it also generates the data element entry fields. The previous implementation had this as a component concern, and I think that was cleaner - so as part of the consolidating these components, I will also follow the previous approach.

Sounds good. Yeah I think the data-structure could just return the Ids, and the component would get the dataElement in component-level? But it doesn't matter too much I think.

But I think it makes sense that it generates the structure for them - since now everything is intertwined. It's not as simple as looping over dataElements and categoryOptionsCombos anymore. One row is not always equal to the cocs.length anylonger, so I'm not sure how easy it is to do this on component level?
I guess the structure would have the rows (dataElements and categories), and the headers, and could loop from that info. But I think this is fine for now.

...ategory-combo-table-body-pivoted/generate-form-matrix/generate-matrix-grouped-by-category.js

tomzemp

Looks cool as a new feature, and seems like it's working for the most part (with some issues in how the transposition is actually being rendered).

I'm approving now given the time frame, but I'm not sure if it is really necessary to update this app in time for the soft freeze, or if would be better to resolve the existing issues before merging in?

...rkspace/category-combo-table-body-pivoted/generate-form-matrix/generate-matrix-transposed.js

src/data-workspace/category-combo-table-body-pivoted/category-combo-table-body-pivoted.js

tomzemp · 2024-03-05T15:41:07Z

src/shared/use-context-selection/use-context-selection.js

@@ -27,6 +27,20 @@ export function usePeriodId() {
    return useQueryParam('periodId', PARAMS_SCHEMA.periodId)
 }

+export function useFeature(feature) {


This makes it easy for testing 😅, but I'm not sure we actually would want to expose a query parameter like this? I think people who design data sets would not want their users to arbitrarily transpose things (I assume we're not going to advertise this, but I guess it's possible that people share links if we expose it?)

There aren't really checks for this, so providing invalid categories causes some issues, but maybe that's not of concern if it's just for testing?

kabaros · 2024-03-05T17:10:56Z

@tomzemp thanks for testing - I fix the issue with the transposed form with one default combo or one category. Below is how it looks. Also removed the cool 🕶️ feature toggle.

# [100.4.0](v100.3.10...v100.4.0) (2024-03-05) ### Features * **DHIS2-16132:** add ability to transpose/pivot a section form ([#367](#367)) ([1d841ab](1d841ab))

dhis2-bot · 2024-03-05T18:47:04Z

🎉 This PR is included in version 100.4.0 🎉

The release is available on:

Your semantic-release bot 📦🚀

This was referenced Feb 20, 2024

feat(DHIS2-16132): add ability to transpose/pivot a section form dhis2/maintenance-app#2802

Merged

feat(DHIS2-16132): add ability to customise the display of a section form dhis2/dhis2-core#16562

Merged

dhis2-bot temporarily deployed to netlify February 20, 2024 17:03 Inactive

dhis2-bot temporarily deployed to netlify February 24, 2024 07:23 Inactive

kabaros force-pushed the DHIS2-16132/data-entry-pivot-table branch from a0b968b to 019f2e6 Compare February 29, 2024 07:25

dhis2-bot temporarily deployed to netlify February 29, 2024 07:26 Inactive

kabaros force-pushed the DHIS2-16132/data-entry-pivot-table branch 3 times, most recently from 63252c7 to 30f0481 Compare March 4, 2024 20:45

dhis2-bot temporarily deployed to netlify March 4, 2024 20:47 Inactive

kabaros force-pushed the DHIS2-16132/data-entry-pivot-table branch from 30f0481 to b33b9e8 Compare March 4, 2024 20:51

dhis2-bot temporarily deployed to netlify March 4, 2024 20:53 Inactive

kabaros force-pushed the DHIS2-16132/data-entry-pivot-table branch from b33b9e8 to ade9622 Compare March 4, 2024 20:54

dhis2-bot temporarily deployed to netlify March 4, 2024 20:56 Inactive

kabaros force-pushed the DHIS2-16132/data-entry-pivot-table branch from ade9622 to 1993cdd Compare March 4, 2024 21:08

dhis2-bot temporarily deployed to netlify March 4, 2024 21:10 Inactive

feat(DHIS2-16132): add ability to transpose/pivot a section form

14b1b94

feat: add ability to transpose/pivot a section form refactor: change file structure to separate transposed from grouped test: add missed tests

kabaros force-pushed the DHIS2-16132/data-entry-pivot-table branch from 1993cdd to 14b1b94 Compare March 5, 2024 08:53

dhis2-bot temporarily deployed to netlify March 5, 2024 09:04 Inactive

kabaros commented Mar 5, 2024

View reviewed changes

kabaros marked this pull request as ready for review March 5, 2024 09:10

kabaros requested a review from a team March 5, 2024 09:10

Birkbjo reviewed Mar 5, 2024

View reviewed changes

...ategory-combo-table-body-pivoted/generate-form-matrix/generate-matrix-grouped-by-category.js Show resolved Hide resolved

tomzemp approved these changes Mar 5, 2024

View reviewed changes

dhis2-bot temporarily deployed to netlify March 5, 2024 17:09 Inactive

refactor: apply code review comments

a939679

kabaros force-pushed the DHIS2-16132/data-entry-pivot-table branch from 4e0a2f9 to a939679 Compare March 5, 2024 17:10

dhis2-bot temporarily deployed to netlify March 5, 2024 17:12 Inactive

kabaros merged commit 1d841ab into master Mar 5, 2024
12 checks passed

dhis2-bot added a commit that referenced this pull request Mar 5, 2024

chore(release): cut 100.4.0 [skip release]

72c96bf

# [100.4.0](v100.3.10...v100.4.0) (2024-03-05) ### Features * **DHIS2-16132:** add ability to transpose/pivot a section form ([#367](#367)) ([1d841ab](1d841ab))

dhis2-bot added the released label Mar 5, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(DHIS2-16132): transpose section forms #367

feat(DHIS2-16132): transpose section forms #367

kabaros commented Feb 20, 2024 •

edited

Loading

dhis2-bot commented Feb 20, 2024 •

edited

Loading

kabaros Mar 5, 2024

tomzemp Mar 5, 2024

kabaros Mar 5, 2024

kabaros Mar 5, 2024

Birkbjo Mar 5, 2024

tomzemp left a comment

tomzemp Mar 5, 2024

kabaros commented Mar 5, 2024

dhis2-bot commented Mar 5, 2024

feat(DHIS2-16132): transpose section forms #367

feat(DHIS2-16132): transpose section forms #367

Conversation

kabaros commented Feb 20, 2024 • edited Loading

Default Section form - without pivoting (existing implementation)

Form pivoted (data elements moved to columns)

Single Category pivoted

Multiple categories (with one pivoted)

Data elements with different category combos

Notes about implementation

Background

Known limitations

ToDos (future)

Related PRs:

dhis2-bot commented Feb 20, 2024 • edited Loading

kabaros Mar 5, 2024

Choose a reason for hiding this comment

tomzemp Mar 5, 2024

Choose a reason for hiding this comment

kabaros Mar 5, 2024

Choose a reason for hiding this comment

kabaros Mar 5, 2024

Choose a reason for hiding this comment

Birkbjo Mar 5, 2024

Choose a reason for hiding this comment

tomzemp left a comment

Choose a reason for hiding this comment

tomzemp Mar 5, 2024

Choose a reason for hiding this comment

kabaros commented Mar 5, 2024

dhis2-bot commented Mar 5, 2024

kabaros commented Feb 20, 2024 •

edited

Loading

dhis2-bot commented Feb 20, 2024 •

edited

Loading