Skip to content
This repository has been archived by the owner on Aug 30, 2024. It is now read-only.

Commit

Permalink
clean example
Browse files Browse the repository at this point in the history
Signed-off-by: Yu, Zhentao <[email protected]>
  • Loading branch information
zhentaoyu committed Jan 24, 2024
1 parent b9a7211 commit 3bb4a3b
Showing 1 changed file with 2 additions and 2 deletions.
4 changes: 2 additions & 2 deletions scripts/python_api_example_for_model_server.py
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@
"I want to learn how to play the piano.",
]

model_name = "/home/zhentao/gpt-j-6b" # "EleutherAI/gpt-j-6b" # model_name from huggingface or local model path
model_name = "EleutherAI/gpt-j-6b" # model_name from huggingface or local model path
tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)

res_collect = []
Expand All @@ -34,7 +34,7 @@ def f_response(res, working):
print(a)
print("=====================================")

model_path = "/home/zhentao/ils/ns/q4_j.bin" # please set your corresponding local neural_speed low-bits model file
model_path = "gptj-q4.bin" # please set your corresponding local neural_speed low-bits model file
added_count = 0
s = cpp.ModelServer(f_response, # reponse function (deliver generation results and current reamin working size in server)
model_path, # model_path
Expand Down

0 comments on commit 3bb4a3b

Please sign in to comment.