Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

added qwen1.5-7b to model list #316

Merged
merged 2 commits into from
Mar 22, 2024

Conversation

mengbingrock
Copy link
Contributor

I've verified support of Qwen1.5-7B by OpenVINO and then added it to the github workflow and readme.md

(base) root@8tvt:~/openvino.genai/llm_bench/python# ../../text_generation/causal_lm/cpp/build/greedy_causal_lm qwen/pytorch/dldt/FP32/ "Why is the Sun yellow?"

 

The Sun does not actually appear yellow to us when we look at it. In fact, it appears white because it emits light across a wide range of wavelengths, including all the colors of the visible spectrum. When this light reaches our eyes, our eyes combine the different colors to create the perception of white.

@p-wysocki
Copy link
Collaborator

cc @Wovchena @pavel-esir

@pavel-esir pavel-esir self-requested a review March 22, 2024 10:33
Copy link
Contributor

@pavel-esir pavel-esir left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good to me. thx @mengbingrock

@pavel-esir pavel-esir merged commit a9ab37e into openvinotoolkit:master Mar 22, 2024
9 checks passed
run: |
source ./ov/setupvars.sh
convert_tokenizer ./Qwen1.5-7B-Chat/pytorch/dldt/FP16/ --output ./Qwen1.5-7B-Chat/pytorch/dldt/FP16/ --with-detokenizer --trust-remote-code
timeout 50s ./build/beam_search_causal_lm ./Qwen1.5-7B-Chat/pytorch/dldt/FP16/ "你好!" > ./pred_qwen15.txt
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could you please also add comparison with Hugging Face?
Example is

python -c "
import transformers
with open('pred.txt', 'r') as file:
predictions = file.read()
tokenizer = transformers.LlamaTokenizer.from_pretrained('TinyLlama/TinyLlama-1.1B-Chat-v1.0')
tokenized = tokenizer('69', return_tensors='pt')
for beam in transformers.LlamaForCausalLM.from_pretrained('TinyLlama/TinyLlama-1.1B-Chat-v1.0').generate(**tokenized, num_beam_groups=3, num_beams=15, num_return_sequences=15, diversity_penalty=1.0, max_new_tokens=20, early_stopping=False, length_penalty=1.0, no_repeat_ngram_size=9**9, do_sample=False):
ref = ': ' + tokenizer.decode(beam[tokenized['input_ids'].numel():], skip_special_tokens=True) + '\n'
idx = predictions.find(ref)
if -1 == idx:
raise RuntimeError(f'Missing "{ref=}" from predictions')
predictions = predictions[:idx] + predictions[idx + len(ref):]
"
echo 69 passed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Good First Issue]: Verify qwen1.5-7b-chat with GenAI text_generation
4 participants