Enhancement on the Score calculation logic when there are multiple correct answers for a question

[PoojaJManjooran]

Hi all, requesting comments regarding the enhancements planned on the score calculation, when multiple correct answers are given for a question. Sharing some context below - please use this doc (confluence page) for comments & suggestions.

Context:

Imagine a question, of any interaction type i.e. Choice, Text, Pairs, etc, having multiple correct answers. For instance, imagine a multiple choice question, a fill in the blank, or a match the following having multiple correct answers.

To provide better flexibility and choice on evaluation - we are proposing we enable following scoring modes - ALL, ANY, PARTIAL.

Current Implementation:

When there are multiple correct answers present for a question, as per the current implementation all responses marked as correct will get equal marks. The total marks of the question will be divided equally amongst all responses.

---

Example (MCQ question):

Q: Which among the following is an even number?
a) 4 //Correct answer
b) 1
c) 2 //Correct answer
d) 3

Total score for the question is 1
Player will get 1/2 marks on responding with a
Player will get 1/2 marks on responding with c
Player will get 1 mark on responding with a & c

---

Proposed Implementation

A Scoring Logic will be implemented which will be set for each question, which can have the following values :

ALL

Full marks assigned only when ALL the correct options are selected
No score will be awarded for entering any one (or more ) of the multiple correct answers until ALL correct options are selected.
In case of Match the Following, this would mean - when ALL correct pairs are provided.
In case of a Fill in the Blank with multiple blanks, this would mean - when ALL blanks are provided correctly.

ANY

Full marks assigned when ANY one of the correct answers is selected.

PARTIAL

Partial scores assigned for any one or more correct answers entered.The total marks of the question will be divided equally amongst all responses.
The current implementation calculates the score based on this logic

Further notes on the scoring logic:

Scoring Logic can be present at each question, section (child question set) and (parent) question set level. So that,

A question when used individually needs scoring logic.
A question set will need to override/drive scoring logic for its children questions. E.g. Quiz with partial scoring, Test with no partial scoring.
A question set with multiple levels of hierarchy (i.e. nested question set) may/will need scoring logic for each of the sections.

Questions which are part of any question set will have the effective Scoring Logic same as the one that is set at the immediate parent level.

If questions are present in a Section (within a Question set), the Scoring Logic of the section will be applied for all the questions in the section.
If the question set does not contain any sections and the questions are the immediate children, the Scoring Logic of the question set will be inherited to the questions within.

This allows us to use questions and question sets in a variety of use-cases by combining & composing them as required.

Read many more examples and further details in the RFC here: https://project-sunbird.atlassian.net/wiki/spaces/QB/pages/2346123290/Multiple+Choice+Question

Looking forward to comments and suggestions on the RFC.

Tagging some of the active community members:
@vinukumar-vs @pallakartheekreddy @kiranharidas187 @Prateek-slokam @aks30 @pkranga @kameshbhr @Krishnaj20 @coolbung @parthlawate @vrayulu @rajnishdargan @itsvick @krgauraw @mathewjpallan @vaibhavbhuva

[vrayulu]

@PoojaJManjooran - I think having predefined scoring types like "ALL", "ANY" and "PARTIAL" will reduce the flexibility to have different score combinations. This approach will not work in the above example where there are two correct answers and if the scores assigned have to be "1/3" and "2/3" instead of equal scores.

QuML already has a provision to define scores like this by mapping response values to specific scores. Please see if existing QuML capabilities can be used to fulfil the above requirement.

cc: @pallakartheekreddy