r/LocalLLaMA • u/returnity • 12h ago

Resources Improved Qwen3.8-Flash Chat Template (w/ Sharp Template terse option)

https://huggingface.co/nazeshinjite/Flash-Next-Chat-Templates

I've been maintaining my own chat template for Qwen3.8-Flash-Next since it released, and after a discussion in another thread I decided to share it. I made a repo with my v1.4 template, the original Unsloth fixed stock template, and now a "Sharp" terseness mode version (based on /u/peculiar-ragdoll's original). On top of Unsloth's improved template I fixed the requests that still fail or render badly, and imported the relevant features/improvements from froggerics, as well as several of my own: - Inline tags like <|think_low|> set the effort level for clients that can't send template kwargs. - Repeated tool failures get escalating "diagnose first, then change course" notes. - Tool arguments passed as a JSON string render instead of killing the request. - Old turns no longer replay an empty <think></think> block. - A stray </think> with no opening tag gets split out instead of leaking into the answer. - Effort values like max, minimal and none no longer throw.

Normal thinking-on history renders byte-for-byte what Unsloth's template renders, so the KV cache is reused and nothing changes when things are going well.

For the Sharp version, I ported 27B Sharp's terseness block directly. This is what makes Tiel/Dirk/other models w/ the Sharp template respond less verbosely. It's appended after your system prompt, never replacing it. I kept Qwen's xhigh default instead of Sharp's medium. Terseness is a terse chat-template kwarg, on by default; {"terse": false} renders the v1.4 template normally, so you can A/B it on one server with no reload.

Quick test: Flash-Next UD-Q5_K_XL on llama.cpp, xhigh effort, temp 1.0 / top-p 0.95 / top-k 20, 8 everyday prompts (explanations, advice, a coding request, a vague "can you help with my project?"):

Answer tokens Reasoning tokens Wall time
Thinking on (16 pairs) −40% (shorter in 13/16) −8% (mixed)
Thinking off (8 pairs) −61% (shorter in 7/8) n/a

Note: I personally leave the terseness setting off, as I found it does reduce accuracy somewhat, and just use a medium effort or switch to Swift-1.5-Flash-Next if I need a quicker response. However, I know a lot of people love Tiel/Dirk/Sharp, so since I'd already put up the repo, I figured I'd offer that option too. Any feedback or improvement suggestions are welcomed!

EDIT: Updated to v1.5 with a fix that I think will fix that occasional weird "empty conversation" glitch that sometimes happens pre-tool call when the response intends going to go straight into a tool call, rather than saying something. I think its due to the template dropping the initial <|im_start|> special token.

24 Upvotes

82% Upvoted

21 comments sorted by

7

u/leonbollerup 12h ago

Will test directly.. I use a combination of froggerics and chromics.. along with my own fixes - we use it for a business model offered to customers

Will share then we can compare notes

4

u/No_Algae1753 12h ago

Would love to know your results

5

u/leonbollerup 11h ago

here is mine: https://raw.githubusercontent.com/leonbollerup/ai/refs/heads/main/qwen-3.8-27b/jinja/froggeric%2Bchromix/qwen-3.8-chromix%2Bfroggerick.v2-medium-default.jinja

5

u/returnity 11h ago

I just updated to v1.5 with a fix inspired by yours for that dropped <|im_start|> tag that happens when the model goes straight to a tool call. I think it will fix a common glitch I see with the model thinking its context is empty and mentioning the system prompt, then calling the tool it intended as if nothing happened. Thanks for sharing.

3

u/leonbollerup 11h ago

your welcome.. a smart move here might be to merge the best parts .. btw.. "mine" works GREAT with qwen 3.5/3.6/3.8 models ..

1

u/enn36 4h ago

Hi both, I'm gonna try naze's 1.5 and chromix frogg.v2

u/leonbollerup The link you gave says v2, but it goes to a github raw that says v1. Just wanna check if it's correct

2

u/returnity 11h ago

Looking forward to your results. Thanks for sharing your template too -- will check it out. What kind of fixes did you find necessary and why?

1

u/leonbollerup 11h ago

here is mine: https://raw.githubusercontent.com/leonbollerup/ai/refs/heads/main/qwen-3.8-27b/jinja/froggeric%2Bchromix/qwen-3.8-chromix%2Bfroggerick.v2-medium-default.jinja

The merge between chromix and froggericks was mostly because chromix's was orginal with some fixes while froggerics keeps the old style..

Swift original (reference) chromix v1 (now) chromix v2 Sharp-naze v1.4
Built for Swift/Flash-Next Qwen 3.8 27B (dense) 27B Flash-Next (from Unsloth’s fixed original)
Works in vLLM ✅ ✅ ✅ ✅ (written for llama.cpp, renders fine in Jinja2)
Tool-call format XML <tool_call><function=…> identical identical identical
Tool instructions in the prompt original rewritten: new rules and a different example (385 vs 377 tokens) same as v1 original, word for word + 2 rules (no XML/JSON debris in parameters; don’t resend a failing call)
Earlier turns’ reasoning in history dropped → 95% cache reuse kept → 100% kept → 100% kept → 100%
Reasoning levels xhigh, medium, low high/xhigh, medium, low + none everything: none/off, minimal, low, medium, high, max… Unknown values → xhigh
Unknown level (e.g. max) ❌ error (HTTP 400) ❌ error ❌ error ✅ accepted
Tool arguments as a string ❌ crashes ✅ ✅ ✅
Reasoning read from reasoning_content reasoning, reasoning_content, thinking (Anthropic)
Injected brevity block (“terse”) – – – 155 tokens, on by default: “answer directly, no preamble, no filler…” (can be turned off with {"terse": false})
Notes in tool results – – – yes: after a failed command it appends “[system note: this tool call failed. Diagnose…]”, escalating on repeated failures
Extra prompt per request with tools ref. +46 tokens +8 +223 (rules + terse)

analysis:

1

u/leonbollerup 11h ago

Pros and cons

chromix v2:

  • ➕ A drop-in replacement for v1, with no downside. “none” now works, where v1 threw an error.
  • ➕ No changes to the model’s behaviour (no injected text).
  • ➖ The tool instructions are rewritten for the 27B model. The format is right, but the text differs from what Swift/Flash-Next was trained with.
  • ➖ Rejects unusual levels (max, minimal) with an error.

Sharp-naze v1.4:

  • ➕ Closest to the model’s own training: the original tool instructions word for word, plus two sensible guard rules.
  • ➕ The most robust: accepts every level, every reasoning field name and odd arguments.
  • ➕ Help in agent loops: notes after failed tool calls, “don’t resend the same call”.
  • ➖ It changes behaviour: the terse block gives shorter, more direct answers. That’s good for agents, but noticeable in chat.
  • ➖ ~220 extra prompt tokens per request, which costs only ~0.04 s at our speed.
  • ➖ The failure notes can misfire: the author reports 88% precision.
  • ➖ Built and calibrated on llama.cpp with the “pi” agent, not tested on vLLM.

My assessment

  • Short term: chromix v2 is better than v1 with no risk, and can go in now with a ~4 min restart.
  • Overall I’m leaning towards Sharp-naze for agent work. One reason weighs especially heavily: in last night’s tests Swift was significantly weaker at tool calls than Intel (79% vs 82%), and chromix replaces the tool instructions the model was trained with. Part of that gap may be the template.

1

u/returnity 11h ago

Note that the terseness can be turned off, either when loading the server as a kwarg, or per-message in the prompt (i.e. <|terse_off|>). the same way thinking can be changed per-message using <|think_low|> etc.

1

u/No_Algae1753 10h ago

Can you also try https://huggingface.co/froggeric/Qwen-Fixed-Chat-Templates ?

1

u/leonbollerup 10h ago

thats the one mine is based on :)

1

u/returnity 10h ago

What did chromix add to the mix?

2

u/leonbollerup 10h ago

Hes was more "orginal" i think.

i used merged them with Opus .. and around 600 tests and until i had the best verified combination.. we are running 4200..ish.. tests and verification now to merge .. "mine" and yours.. if it plays out we might get something good out of it.. per the test.. yours have some really good stuff..

1

u/No_Algae1753 10h ago

Damn didn't know they could have such an impact that you made that many tests

1

u/No_Algae1753 10h ago

Oh got it wrong thought you were only using the chromix one. However, I dont understand your assessment. You write short term (ig this is your personal benchmark) that chromix is better but you still decide to use Sharp for agent work? Btw im gonna try out yours now

1

u/leonbollerup 10h ago

Chromix's... was.. if memory serves me right - based on the orginal.. while froggerics contains fixes targeted the 27b/35b models.. i wanted one that "could do it all" - and do it will.

1

u/returnity 10h ago

My template is originated as a port of froggeric's and includes essentially all of his improvements, just FYI. I originally built it when FN was new and he hadn't updated for compatibility.

3

u/No_Algae1753 12h ago

Gonna try that out and compare it to frogerric template

1

u/returnity 11h ago edited 10h ago

Please let me know. I included his (applicable) fixes in mine when I built it, which was before he updated to v22.5 with Flash compatibility, so mine is basically a superset of his.

2

u/leonbollerup 2h ago

Morning,

The horde of AI's worked thru the night.. 4700+ tests run (still things to fix).. i decided to just have claude dump everything here: https://github.com/leonbollerup/qwen-asgard-jinja

The tests was run against QFN by Claude Opus 5.5, verified by Fable and Astra.
Configuration was handeld by "skyclaw" (our own sysops ai)

So.. hopefully.. this is the best of all worlds..

Edit:
Take whatever is usefull, throw it away, join the repo.. whatever you do with this is up to you :)