Transmuter.ChatTemplate (transmuter v2.0.0)

Copy Markdown View Source

Renders the chat template of a model repository, like apply_chat_template in Python transformers.

Needs the optional otera dependency, which renders Jinja templates in pure Elixir.

repo = Transmuter.Repo.load!({:hf, "HuggingFaceTB/SmolLM2-135M-Instruct"})
Transmuter.ChatTemplate.render(repo, [%{role: "user", content: "Hello"}])
#=> "<|im_start|>system\nYou are a helpful AI assistant named SmolLM...<|im_start|>assistant\n"

Map keys can be atoms or strings. A plain Elixir map gives its keys to tojson in alphabetical order. To get the key order of transformers for tool definitions or messages that come from JSON, decode the JSON with Otera.JSON.decode!/1.

Summary

Functions

Renders messages, a list of maps with :role and :content, with the chat template of repo.

Functions

render(repo, messages, opts \\ [])

Renders messages, a list of maps with :role and :content, with the chat template of repo.

Options

  • :continue_final_message - ends the prompt inside the final message, so that the model continues it. The default is false.

  • :add_generation_prompt - ends the prompt with the start of an assistant message. The default is true, or false with :continue_final_message.

  • :tools - a list of tool definitions in JSON schema form. A repository with a template named "tool_use" renders tools with it.

  • :template_options - other variables of the template, such as enable_thinking: false for Qwen3.

  • :template - the template to render in place of that of the tokenizer, such as the :processor_chat_template of Transmuter.Repo.