Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sorter: Extract validation out of scoring #1876

Merged

Conversation

Myranae
Copy link
Contributor

@Myranae Myranae commented Nov 18, 2024

Summary:

To complete server-side scoring, we are separating out validation logic from scoring logic. This separates that logic for the sorter widget and updates associated tests.

Issue: LEMS-2605

Test plan:

  • Confirm all checks pass
  • Confirm widget still works as expected via Storybook

@Myranae Myranae self-assigned this Nov 18, 2024
Copy link
Contributor

github-actions bot commented Nov 18, 2024

npm Snapshot: Published

Good news!! We've packaged up the latest commit from this PR (61de24e) and published it to npm. You
can install it using the tag PR1876.

Example:

yarn add @khanacademy/perseus@PR1876

If you are working in Khan Academy's webapp, you can run:

./dev/tools/bump_perseus_version.sh -t PR1876

Copy link
Contributor

github-actions bot commented Nov 18, 2024

Size Change: +157 B (+0.01%)

Total Size: 1.29 MB

Filename Size Change
packages/perseus/dist/es/index.js 421 kB +157 B (+0.04%)
ℹ️ View Unchanged
Filename Size
packages/kas/dist/es/index.js 39 kB
packages/keypad-context/dist/es/index.js 760 B
packages/kmath/dist/es/index.js 4.27 kB
packages/math-input/dist/es/index.js 77.9 kB
packages/math-input/dist/es/strings.js 1.79 kB
packages/perseus-core/dist/es/index.js 1.48 kB
packages/perseus-editor/dist/es/index.js 698 kB
packages/perseus-linter/dist/es/index.js 22.2 kB
packages/perseus/dist/es/strings.js 3.68 kB
packages/pure-markdown/dist/es/index.js 3.66 kB
packages/simple-markdown/dist/es/index.js 12.5 kB

compressed-size-action

@Myranae Myranae marked this pull request as ready for review November 18, 2024 17:32
@khan-actions-bot khan-actions-bot requested a review from a team November 18, 2024 17:32
@khan-actions-bot
Copy link
Contributor

Gerald

Required Reviewers
  • @Khan/perseus for changes to .changeset/sweet-trainers-drop.md, packages/perseus/src/widgets/sorter/score-sorter.test.ts, packages/perseus/src/widgets/sorter/score-sorter.ts, packages/perseus/src/widgets/sorter/validate-sorter.test.ts, packages/perseus/src/widgets/sorter/validate-sorter.ts

Don't want to be involved in this pull request? Comment #removeme and we won't notify you of further changes.

Comment on lines 32 to 43

it("is invalid when the user has not made any changes", () => {
const userInput: PerseusSorterUserInput = {
options: ["$15$ grams", "$55$ grams", "$0.005$ kilograms"],
changed: false,
};
const rubric: PerseusSorterRubric = {
correct: ["$0.005$ kilograms", "$15$ grams", "$55$ grams"],
};
const result = scoreSorter(userInput, rubric);
expect(result).toHaveInvalidInput();
});
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we could keep a variant of this test in each scorer's test suite. But, we can use mocking to ensure that validate is called first... something like:

I think we could keep a variant of this test in each scorer's test suite. But, we can use mocking to ensure that validate is called first... something like:

// Top of file 
import * as SorterValidator from './validate-sorter'

    // Inside describe() block...
    it("should score if validator passes", () => {
        // Mock validator saying "all good" 👍 
        const mockValidate = jest
            .spyOn(SorterValidator, "default")
            .mockReturnValue(null);


        const score = scoreSorter(userInput, rubric);

        // Assert
        expect(mockValidate).toHaveBeenCalled();
        expect(score).toHaveBeenAnsweredCorrectly();
    });

    it("should abort if validator returns invalid", () => {
        // Mock validator saying "all good" 👍 
        const mockValidate = jest
            .spyOn(SorterValidator, "default")
            .mockReturnValue({type: "invalid", message: null});

        const score = scoreSorter(userInput, rubric);

        // Assert
        expect(mockValidate).toHaveBeenCalled();
        expect(score).toHaveInvalidInput()
    });

expect(result).toHaveBeenAnsweredIncorrectly();
});

it("is invalid when the user has not made any changes", () => {
it("should abort if validator returns invalid", () => {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jeremywiebe Do the below added tests look as expected? Thanks!

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, this looks correct! Thanks for adding these!

Matthew has an idea to refactor the scoring so that we don't have to guarantee the scorer calls validation first: https://khanacademy.atlassian.net/browse/LEMS-2658

But for now, let's leave these tests and logic in place.

@jeremywiebe jeremywiebe changed the base branch from main to release/server-side-scoring November 26, 2024 20:19
@jeremywiebe jeremywiebe merged commit 0bd4270 into release/server-side-scoring Nov 26, 2024
14 of 15 checks passed
@jeremywiebe jeremywiebe deleted the tb/LEMS-2605/sorter-split-out-validation branch November 26, 2024 20:31
jeremywiebe added a commit that referenced this pull request Nov 27, 2024
…ng` to `main` (#1926)

## Summary:
This PR includes the following commits:
- Numeric-Input: Extract validation from scorer (#1882)
- Orderer: Extract validation out of scoring (#1869)
- Sorter: Extract validation out of scoring (#1876)
- Dropdown: Extract validation out of scoring (#1898)
- Plotter: Extract validation out of scoring (#1899)
- Radio: Extract validation out of scoring (#1902)
- Categorizer: Extract validation out of scoring (#1862)

Issue: <none>

## Test plan:

Author: jeremywiebe

Reviewers: lillialexis, #perseus

Required Reviewers:

Approved By: lillialexis

Checks: ✅ Publish npm snapshot (ubuntu-latest, 20.x), ✅ Lint, Typecheck, Format, and Test (ubuntu-latest, 20.x), ✅ Check builds for changes in size (ubuntu-latest, 20.x), ✅ Cypress (ubuntu-latest, 20.x), ✅ Check for .changeset entries for all changed files (ubuntu-latest, 20.x), ✅ gerald, ✅ Publish npm snapshot (ubuntu-latest, 20.x), 🚫 Lint, Typecheck, Format, and Test (ubuntu-latest, 20.x), ✅ Cypress (ubuntu-latest, 20.x), 🚫 Check builds for changes in size (ubuntu-latest, 20.x), ✅ Check for .changeset entries for all changed files (ubuntu-latest, 20.x), ✅ Publish Storybook to Chromatic (ubuntu-latest, 20.x), ✅ gerald, ✅ .github/dependabot.yml

Pull Request URL: #1926
jeremywiebe pushed a commit that referenced this pull request Nov 27, 2024
This PR was opened by the [Changesets release](https://github.com/changesets/action) GitHub action. When you're ready to do a release, you can merge this and the packages will be published to npm automatically. If you're not ready to do a release yet, that's fine, whenever you add more changesets to main, this PR will be updated.


# Releases
## @khanacademy/[email protected]

### Minor Changes

-   [#1898](#1898) [`3a9b5921b`](3a9b592) Thanks [@Myranae](https://github.com/Myranae)! - Introduces a validation function for the dropdown widget (extracted from dropdown scoring function).


-   [#1862](#1862) [`451de899f`](451de89) Thanks [@Myranae](https://github.com/Myranae)! - Split out validation function for the `categorizer` widget. This can be used to check if the user selected an answer for every row, confirming the question is ready to be scored.


-   [#1882](#1882) [`40d2ebb75`](40d2ebb) Thanks [@jeremywiebe](https://github.com/jeremywiebe)! - Introduces a validation function for the numeric-input widget (extracted from numeric-input scoring function).


-   [#1899](#1899) [`2437ce61b`](2437ce6) Thanks [@Myranae](https://github.com/Myranae)! - Introduces a validation function for the plotter widget (extracted from the scoring function).


-   [#1869](#1869) [`f43edd42c`](f43edd4) Thanks [@Myranae](https://github.com/Myranae)! - Split out validation function for the `orderer` widget. This can be used to check if the user has ordered at least one option, confirming the question is ready to be scored.


-   [#1902](#1902) [`0cec7628c`](0cec762) Thanks [@Myranae](https://github.com/Myranae)! - Introduces a validation function for the radio widget (extracted from the scoring function), though not all validation logic can be extracted.


-   [#1876](#1876) [`0bd4270ad`](0bd4270) Thanks [@Myranae](https://github.com/Myranae)! - Split out validation function for the `sorter` widget. This can be used to check if the user has made any changes to the sorting order, confirming whether or not the question can be scored.

### Patch Changes

-   [#1907](#1907) [`3dbca965a`](3dbca96) Thanks [@benchristel](https://github.com/benchristel)! - Internal: add and pass regression tests for `PerseusItem` parser.


-   [#1915](#1915) [`ee09e9fc0`](ee09e9f) Thanks [@handeyeco](https://github.com/handeyeco)! - Remove use of findDOMNode in number-input component


-   [#1920](#1920) [`88ba71bef`](88ba71b) Thanks [@handeyeco](https://github.com/handeyeco)! - Fix some file-wide error suppressions


-   [#1919](#1919) [`64ea2ee86`](64ea2ee) Thanks [@handeyeco](https://github.com/handeyeco)! - Remove usage of findDOMNode in text-input component


-   [#1906](#1906) [`799ffe4a5`](799ffe4) Thanks [@anakaren-rojas](https://github.com/anakaren-rojas)! - update moveable point component and use control point method to have optional params


-   [#1895](#1895) [`841551a65`](841551a) Thanks [@benchristel](https://github.com/benchristel)! - Internal: remove unused fields from `answerArea` when parsing `PerseusItem`s.


-   [#1921](#1921) [`8ec06f444`](8ec06f4) Thanks [@benchristel](https://github.com/benchristel)! - Internal: Inline widget version into Expression widget parser.


-   [#1914](#1914) [`3e98b7cd3`](3e98b7c) Thanks [@handeyeco](https://github.com/handeyeco)! - Add tests for propUpgrades functions (and remove underscore usage)


-   [#1908](#1908) [`7f2866cf4`](7f2866c) Thanks [@benchristel](https://github.com/benchristel)! - Internal: Migrate expression widget options to the latest version in parseAndTypecheckPerseusItem (not yet used in production).

-   Updated dependencies \[[`88ba71bef`](88ba71b)]:
    -   @khanacademy/[email protected]
    -   @khanacademy/[email protected]
    -   @khanacademy/[email protected]

## @khanacademy/[email protected]

### Patch Changes

-   [#1920](#1920) [`88ba71bef`](88ba71b) Thanks [@handeyeco](https://github.com/handeyeco)! - Fix some file-wide error suppressions

## @khanacademy/[email protected]

### Patch Changes

-   [#1920](#1920) [`88ba71bef`](88ba71b) Thanks [@handeyeco](https://github.com/handeyeco)! - Fix some file-wide error suppressions


-   [#1863](#1863) [`584153588`](5841535) Thanks [@jeremywiebe](https://github.com/jeremywiebe)! - Switch two corner usages of deprecated @khanacademy/wonder-blocks-spacing to @khanacademy/wonder-blocks-tokens

-   Updated dependencies \[[`3dbca965a`](3dbca96), [`ee09e9fc0`](ee09e9f), [`3a9b5921b`](3a9b592), [`88ba71bef`](88ba71b), [`64ea2ee86`](64ea2ee), [`451de899f`](451de89), [`40d2ebb75`](40d2ebb), [`799ffe4a5`](799ffe4), [`2437ce61b`](2437ce6), [`841551a65`](841551a), [`8ec06f444`](8ec06f4), [`f43edd42c`](f43edd4), [`0cec7628c`](0cec762), [`0bd4270ad`](0bd4270), [`3e98b7cd3`](3e98b7c), [`7f2866cf4`](7f2866c)]:
    -   @khanacademy/[email protected]
    -   @khanacademy/[email protected]
    -   @khanacademy/[email protected]

## @khanacademy/[email protected]

### Patch Changes

-   Updated dependencies \[[`88ba71bef`](88ba71b)]:
    -   @khanacademy/[email protected]

## @khanacademy/[email protected]

### Patch Changes

-   [#1920](#1920) [`88ba71bef`](88ba71b) Thanks [@handeyeco](https://github.com/handeyeco)! - Fix some file-wide error suppressions

## @khanacademy/[email protected]

### Patch Changes

-   Updated dependencies \[[`88ba71bef`](88ba71b)]:
    -   @khanacademy/[email protected]
    -   @khanacademy/[email protected]
    -   @khanacademy/[email protected]

Author: khan-actions-bot

Reviewers: jeremywiebe

Required Reviewers:

Approved By: jeremywiebe

Checks: ✅ Cypress (ubuntu-latest, 20.x), ⏭️  Publish npm snapshot, ✅ Check builds for changes in size (ubuntu-latest, 20.x), ✅ Lint, Typecheck, Format, and Test (ubuntu-latest, 20.x), ✅ Check for .changeset entries for all changed files (ubuntu-latest, 20.x), ✅ gerald

Pull Request URL: #1918
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants