Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Jinja template support #11016

Draft
wants to merge 7 commits into
base: master
Choose a base branch
from
Draft

Add Jinja template support #11016

wants to merge 7 commits into from

Conversation

ochafik
Copy link
Collaborator

@ochafik ochafik commented Dec 30, 2024

Subset of #9639 with just the Jinja templating support (proper tool support to come as follow ups).

  • Copies minja.hpp & chat-template.hpp from google/minja at this commit
  • Adds --jinja and --chat-template-file flags to llama-server
  • Loads tokenizer.chat_template (or tokenizer.chat_template.tool_use if defined, only when the request has tools), and defaults to a simple chatml template.

TODO:

  • Add some instructions here
  • Add more server tests to exercise the template overrides.
  • Add cross-testing in test-chat-template.cpp (note that minja is tested against a lot of templates in its own repo)

@github-actions github-actions bot added script Script related examples python python script changes server labels Dec 30, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
examples python python script changes script Script related server
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant