Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Few-shot tests on GLM-10B #203

Open
Vispstar-V opened this issue Mar 25, 2024 · 0 comments
Open

Few-shot tests on GLM-10B #203

Vispstar-V opened this issue Mar 25, 2024 · 0 comments

Comments

@Vispstar-V
Copy link

Hi, I'm trying to conduct a multiple-choice test on GLM-10B using few-shot prompting. I have tried to start a service using the following command:

bash scripts/generate_block.sh \
     config_tasks/model_blocklm_10B.sh

Then I input my few-shot prompts, such as:

Please answer this multiple-choice question.\nQuestion: Which instrument measures atmospheric pressure?\nOptions: A: Thermometer. B: Anemometer. C: Barometer. D: Hydrometer.\nAnswer: C.\n\nQuestion: Which gas is most commonly used in light bulbs?\nOptions: A: Nitrogen. B: Oxygen. C: Helium. D: Argon.\nAnswer: D.\n\nQuestion: What is the largest organ in the human body?\nOptions: A: Heart. B: Liver. C: Skin. D: Brain.\nAnswer: C.\n\nQuestion: What is the pH level of pure water?\nOptions: A: 5. B: 7. C: 9. D: 11.\nAnswer:[gMask]

But on most tests, the model fails to provide a correct option. I wonder if there is a flaw in my prompt design, or it is a common issue.
Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant