r/Cloud • u/Latter-Actuary4301 • 7h ago
The agent carries unused tool schemas through every turn
Every agent turn is carrying tool_count=28 even after the router narrows the task to 4 relevant tools, so 24 schemas stay in the context window for no reason. We’re a 3 person team running 1800 sessions a week and each turn logs input_tokens=7600 from the schema block. The average session lasts 9 turns, which leaves 68400 schema tokens riding through each session. Our average session costs < $1 and the weekly < $1000. I checked the provider accounting and our request logs twice before blaming tool schema inflation, the numbers kept pointing back to the same block.
The pruning estimate gets us to about $0.19 per session and $342 per week, but I don’t want to save money by making tool selection less reliable. We’re comparing Datadog and Braintrust for span level token attribution and trace filtering on tool_count, plus a clean task quality comparison before and after pruning so the input token cost doesn’t become the only metric that matters.
p95 cost should tell me whether the long sessions improve too and I’m testing whether the router can expose only the 4 selected schemas without breaking recovery when the first choice is wrong. Any suggestions on pruning tool definitions without making the agent worse at selecting a fallback appreciated