-
Notifications
You must be signed in to change notification settings - Fork 351
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Sorter: Extract validation out of scoring #1876
Sorter: Extract validation out of scoring #1876
Conversation
npm Snapshot: PublishedGood news!! We've packaged up the latest commit from this PR (61de24e) and published it to npm. You Example: yarn add @khanacademy/perseus@PR1876 If you are working in Khan Academy's webapp, you can run: ./dev/tools/bump_perseus_version.sh -t PR1876 |
Size Change: +157 B (+0.01%) Total Size: 1.29 MB
ℹ️ View Unchanged
|
GeraldRequired Reviewers
Don't want to be involved in this pull request? Comment |
|
||
it("is invalid when the user has not made any changes", () => { | ||
const userInput: PerseusSorterUserInput = { | ||
options: ["$15$ grams", "$55$ grams", "$0.005$ kilograms"], | ||
changed: false, | ||
}; | ||
const rubric: PerseusSorterRubric = { | ||
correct: ["$0.005$ kilograms", "$15$ grams", "$55$ grams"], | ||
}; | ||
const result = scoreSorter(userInput, rubric); | ||
expect(result).toHaveInvalidInput(); | ||
}); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think we could keep a variant of this test in each scorer's test suite. But, we can use mocking to ensure that validate
is called first... something like:
I think we could keep a variant of this test in each scorer's test suite. But, we can use mocking to ensure that validate
is called first... something like:
// Top of file
import * as SorterValidator from './validate-sorter'
// Inside describe() block...
it("should score if validator passes", () => {
// Mock validator saying "all good" 👍
const mockValidate = jest
.spyOn(SorterValidator, "default")
.mockReturnValue(null);
const score = scoreSorter(userInput, rubric);
// Assert
expect(mockValidate).toHaveBeenCalled();
expect(score).toHaveBeenAnsweredCorrectly();
});
it("should abort if validator returns invalid", () => {
// Mock validator saying "all good" 👍
const mockValidate = jest
.spyOn(SorterValidator, "default")
.mockReturnValue({type: "invalid", message: null});
const score = scoreSorter(userInput, rubric);
// Assert
expect(mockValidate).toHaveBeenCalled();
expect(score).toHaveInvalidInput()
});
Unnecessary value since the validator is being mocked, but want to keep it consistent
expect(result).toHaveBeenAnsweredIncorrectly(); | ||
}); | ||
|
||
it("is invalid when the user has not made any changes", () => { | ||
it("should abort if validator returns invalid", () => { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@jeremywiebe Do the below added tests look as expected? Thanks!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, this looks correct! Thanks for adding these!
Matthew has an idea to refactor the scoring so that we don't have to guarantee the scorer calls validation first: https://khanacademy.atlassian.net/browse/LEMS-2658
But for now, let's leave these tests and logic in place.
0bd4270
into
release/server-side-scoring
…ng` to `main` (#1926) ## Summary: This PR includes the following commits: - Numeric-Input: Extract validation from scorer (#1882) - Orderer: Extract validation out of scoring (#1869) - Sorter: Extract validation out of scoring (#1876) - Dropdown: Extract validation out of scoring (#1898) - Plotter: Extract validation out of scoring (#1899) - Radio: Extract validation out of scoring (#1902) - Categorizer: Extract validation out of scoring (#1862) Issue: <none> ## Test plan: Author: jeremywiebe Reviewers: lillialexis, #perseus Required Reviewers: Approved By: lillialexis Checks: ✅ Publish npm snapshot (ubuntu-latest, 20.x), ✅ Lint, Typecheck, Format, and Test (ubuntu-latest, 20.x), ✅ Check builds for changes in size (ubuntu-latest, 20.x), ✅ Cypress (ubuntu-latest, 20.x), ✅ Check for .changeset entries for all changed files (ubuntu-latest, 20.x), ✅ gerald, ✅ Publish npm snapshot (ubuntu-latest, 20.x), 🚫 Lint, Typecheck, Format, and Test (ubuntu-latest, 20.x), ✅ Cypress (ubuntu-latest, 20.x), 🚫 Check builds for changes in size (ubuntu-latest, 20.x), ✅ Check for .changeset entries for all changed files (ubuntu-latest, 20.x), ✅ Publish Storybook to Chromatic (ubuntu-latest, 20.x), ✅ gerald, ✅ .github/dependabot.yml Pull Request URL: #1926
This PR was opened by the [Changesets release](https://github.com/changesets/action) GitHub action. When you're ready to do a release, you can merge this and the packages will be published to npm automatically. If you're not ready to do a release yet, that's fine, whenever you add more changesets to main, this PR will be updated. # Releases ## @khanacademy/[email protected] ### Minor Changes - [#1898](#1898) [`3a9b5921b`](3a9b592) Thanks [@Myranae](https://github.com/Myranae)! - Introduces a validation function for the dropdown widget (extracted from dropdown scoring function). - [#1862](#1862) [`451de899f`](451de89) Thanks [@Myranae](https://github.com/Myranae)! - Split out validation function for the `categorizer` widget. This can be used to check if the user selected an answer for every row, confirming the question is ready to be scored. - [#1882](#1882) [`40d2ebb75`](40d2ebb) Thanks [@jeremywiebe](https://github.com/jeremywiebe)! - Introduces a validation function for the numeric-input widget (extracted from numeric-input scoring function). - [#1899](#1899) [`2437ce61b`](2437ce6) Thanks [@Myranae](https://github.com/Myranae)! - Introduces a validation function for the plotter widget (extracted from the scoring function). - [#1869](#1869) [`f43edd42c`](f43edd4) Thanks [@Myranae](https://github.com/Myranae)! - Split out validation function for the `orderer` widget. This can be used to check if the user has ordered at least one option, confirming the question is ready to be scored. - [#1902](#1902) [`0cec7628c`](0cec762) Thanks [@Myranae](https://github.com/Myranae)! - Introduces a validation function for the radio widget (extracted from the scoring function), though not all validation logic can be extracted. - [#1876](#1876) [`0bd4270ad`](0bd4270) Thanks [@Myranae](https://github.com/Myranae)! - Split out validation function for the `sorter` widget. This can be used to check if the user has made any changes to the sorting order, confirming whether or not the question can be scored. ### Patch Changes - [#1907](#1907) [`3dbca965a`](3dbca96) Thanks [@benchristel](https://github.com/benchristel)! - Internal: add and pass regression tests for `PerseusItem` parser. - [#1915](#1915) [`ee09e9fc0`](ee09e9f) Thanks [@handeyeco](https://github.com/handeyeco)! - Remove use of findDOMNode in number-input component - [#1920](#1920) [`88ba71bef`](88ba71b) Thanks [@handeyeco](https://github.com/handeyeco)! - Fix some file-wide error suppressions - [#1919](#1919) [`64ea2ee86`](64ea2ee) Thanks [@handeyeco](https://github.com/handeyeco)! - Remove usage of findDOMNode in text-input component - [#1906](#1906) [`799ffe4a5`](799ffe4) Thanks [@anakaren-rojas](https://github.com/anakaren-rojas)! - update moveable point component and use control point method to have optional params - [#1895](#1895) [`841551a65`](841551a) Thanks [@benchristel](https://github.com/benchristel)! - Internal: remove unused fields from `answerArea` when parsing `PerseusItem`s. - [#1921](#1921) [`8ec06f444`](8ec06f4) Thanks [@benchristel](https://github.com/benchristel)! - Internal: Inline widget version into Expression widget parser. - [#1914](#1914) [`3e98b7cd3`](3e98b7c) Thanks [@handeyeco](https://github.com/handeyeco)! - Add tests for propUpgrades functions (and remove underscore usage) - [#1908](#1908) [`7f2866cf4`](7f2866c) Thanks [@benchristel](https://github.com/benchristel)! - Internal: Migrate expression widget options to the latest version in parseAndTypecheckPerseusItem (not yet used in production). - Updated dependencies \[[`88ba71bef`](88ba71b)]: - @khanacademy/[email protected] - @khanacademy/[email protected] - @khanacademy/[email protected] ## @khanacademy/[email protected] ### Patch Changes - [#1920](#1920) [`88ba71bef`](88ba71b) Thanks [@handeyeco](https://github.com/handeyeco)! - Fix some file-wide error suppressions ## @khanacademy/[email protected] ### Patch Changes - [#1920](#1920) [`88ba71bef`](88ba71b) Thanks [@handeyeco](https://github.com/handeyeco)! - Fix some file-wide error suppressions - [#1863](#1863) [`584153588`](5841535) Thanks [@jeremywiebe](https://github.com/jeremywiebe)! - Switch two corner usages of deprecated @khanacademy/wonder-blocks-spacing to @khanacademy/wonder-blocks-tokens - Updated dependencies \[[`3dbca965a`](3dbca96), [`ee09e9fc0`](ee09e9f), [`3a9b5921b`](3a9b592), [`88ba71bef`](88ba71b), [`64ea2ee86`](64ea2ee), [`451de899f`](451de89), [`40d2ebb75`](40d2ebb), [`799ffe4a5`](799ffe4), [`2437ce61b`](2437ce6), [`841551a65`](841551a), [`8ec06f444`](8ec06f4), [`f43edd42c`](f43edd4), [`0cec7628c`](0cec762), [`0bd4270ad`](0bd4270), [`3e98b7cd3`](3e98b7c), [`7f2866cf4`](7f2866c)]: - @khanacademy/[email protected] - @khanacademy/[email protected] - @khanacademy/[email protected] ## @khanacademy/[email protected] ### Patch Changes - Updated dependencies \[[`88ba71bef`](88ba71b)]: - @khanacademy/[email protected] ## @khanacademy/[email protected] ### Patch Changes - [#1920](#1920) [`88ba71bef`](88ba71b) Thanks [@handeyeco](https://github.com/handeyeco)! - Fix some file-wide error suppressions ## @khanacademy/[email protected] ### Patch Changes - Updated dependencies \[[`88ba71bef`](88ba71b)]: - @khanacademy/[email protected] - @khanacademy/[email protected] - @khanacademy/[email protected] Author: khan-actions-bot Reviewers: jeremywiebe Required Reviewers: Approved By: jeremywiebe Checks: ✅ Cypress (ubuntu-latest, 20.x), ⏭️ Publish npm snapshot, ✅ Check builds for changes in size (ubuntu-latest, 20.x), ✅ Lint, Typecheck, Format, and Test (ubuntu-latest, 20.x), ✅ Check for .changeset entries for all changed files (ubuntu-latest, 20.x), ✅ gerald Pull Request URL: #1918
Summary:
To complete server-side scoring, we are separating out validation logic from scoring logic. This separates that logic for the sorter widget and updates associated tests.
Issue: LEMS-2605
Test plan: