Prompt template format for ibm-granite/granite-7b-instruct

[jcalvoch]

I see this as an example on the model card:
sys_prompt = "You are an AI language model developed by IBM Research. You are a cautious assistant. You carefully follow instructions. You are helpful and harmless and you follow ethical guidelines and promote positive behavior."

prompt = f'<|system|>\n{sys_prompt}\n<|user|>\n{inputs}\n<|assistant|>\n'
stop_token = '<|endoftext|>'

Are there more details or examples on the prompt templates that work well with this model?

[PrimosK]

I am after the same information...

The closest thing I've found is this:
https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/fm-prompt-samples.html?context=wx

[aldopareja]

from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("instructlab/granite-7b-lab")
model = AutoModelForCausalLM.from_pretrained("instructlab/granite-7b-lab")

sys_prompt = "You are an AI language model developed by IBM Research. You are a cautious assistant. You carefully follow instructions. You are helpful and harmless and you follow ethical guidelines and promote positive behavior."
prompt = """<|system|>\n{sys_prompt}\n<|user|>\n{inputs}\n<|assistant|>\n"""
stop_tokens = ["<|endoftext|>"]

output = model.generate(
    tokenizer.encode(prompt, return_tensors="pt"),
    max_length=50,
    stop_strings=stop_tokens,
)

print(tokenizer.decode(output[0]))

hopefully this helps

[jcalvoch]

Why is this even an answer? It is referencing the same thing I put on my initial message and it doesn't answer my question.

[mayank31398]

Why does this not answer the question?
Its pretty clear to me on how to use the template

[jcalvoch]

1. It is just an example
2. No explanations on how to get the most out of the model in terms of templating, or at least something that explains if this prompting scheme is based on Llama's, Mistral's or if it is IBMs own flavor. Other models have very good documentation on this and I can use those to optimize my prompts.
3. I could infer what |system|, |user| means, but why not have documentation that explains that? Is there a |context| for example? Again, other models are good on explaining this.

[mayank31398]

@aldopareja tagging you again ^^

[AnaVictoriaMartin127]

Hi! @jcalvoch I think this provides explanations on how to get the most out of the model in terms of templating overall :) it has worked for me!! https://learn.deeplearning.ai/courses/chatgpt-prompt-eng/lesson/1/introduction