Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Add a skill to better understand bullet lists of items (#730)
This is a re-submission of [**PR 652**](#652). **Describe the contribution to the taxonomy** Understand better when a bullet list of items means that the items are needed jointly rather than separately. - Documents often contain bullet lists of items that one must provide, have available, complete, etc. - Sometimes Merlinite 7b gets confused whether the listed items must be provided jointly or separately - Merlinite 7b tends to think that the items are needed separately, even though they are needed jointly - Here we add a few of such examples, with the correct answers **Input given at the prompt** In one example, Merlinite 7b was given the following (self-authored) context document: ``` To apply for public benefit XYZ, provide the following: * Your full name and date of birth * Your vehicle license plate number ``` Then the following question was asked: ``` Which of the following is true: (A) To apply for public benefit XYZ, you must provide both your full name and vehicle license plate number (B) To apply for public benefit XYZ, you must provide either your full name or vehicle license plate number ``` Most humans would choose (A), but Merlinite 7b chose (B), responding as follows: **Response from the original model** ``` The correct answer is (B) To apply for public benefit XYZ, you must provide either your full name or vehicle license plate number. The statement is true because the application process requires one of these pieces of information, but not necessarily both. ``` **Response from the fine-tuned model** (did not have an opportunity to fine tune the model) **Contribution checklist** - [x] The contribution was tested with `ilab generate` - [x] No errors or warnings were produced by `ilab generate` - [x] All [commits are signed off](https://github.com/instructlab/taxonomy/blob/main/CONTRIBUTING.md#legal) (DCO) - [x] The `qna.yaml` file contains at least 5 `seed_examples` - [x] The `qna.yaml` file was [linted](https://yamllint.com) and [prettified](https://onlineyamltools.com/prettify-yaml) ([yaml-validator](https://jsonformatter.org/yaml-validator) can do both) - [x] An `attribution.txt` file in the same folder as the `qna.yaml` file. --------- Signed-off-by: Alexandre Evfimievski <[email protected]> Signed-off-by: BJ Hargrave <[email protected]> Signed-off-by: Alexandre Evfimievski <[email protected]>
- Loading branch information