Our AI Had Tools Declared And Switched Off. For Months.
This is my favourite kind of bug, in that it is entirely our own fault and completely invisible from the outside.
The Aiva AI layer had a proper tool-calling interface. IAIService declared a tools parameter. AIResponse had a ToolCalls collection. CustomerServiceAiva had an ExecuteToolCallsAsync method with handlers for search_orders, get_order_details and get_tracking_info.
Underneath, in ClaudeAiService, sat this:
// TODO: Add tool support when needed
// For now, tools functionality is disabled to simplify initial implementation
It always returned ToolCalls = null. And every caller was passing null for tools anyway, with a cheerful comment saying // No tools for now.
So the tool executor was dead code. Nothing could ever call it. Meanwhile the system prompt was confidently instructing the model to "use the available tools" that it was never given.
The result is an AI that sounds like it looked something up and did not. It tells the customer it will check their order, then produces a plausible sentence from nothing. That is the worst possible failure mode, because it looks like it worked.
What fixing it actually took
Not much, once someone sat down and did it. Convert our tool definitions into the Anthropic SDK's shape, set ToolChoice to auto, and read the tool_use blocks back out of the response instead of throwing them away.
The bit worth writing up is the loop.
A single call is not enough. The model asks for a tool, you run it, you hand back the result, and it might ask for another one. So there is a bounded agentic loop: call the model, execute whatever it asked for, append the results as tool_result blocks, go round again, stop when it returns plain text or when you hit the iteration ceiling.
Three things in there that matter more than the happy path:
Echo the assistant turn back verbatim. You have to append the model's own message, tool_use blocks intact, before you append the results. Reconstruct it from your own data structures and the ids stop matching.
Hand tool failures to the model, not the caller. If a tool throws, we catch it and send back a tool_result marked as an error. The model then apologises or tries something else. If you let the exception bubble out, one flaky lookup kills the whole conversation.
Cap the loop. Five iterations. An agent that can loop forever will find a way to, usually at three in the morning against a paid API.
The thing I would tell you to check
If your codebase has an AI integration that somebody set up in a hurry, go and look at whether tools are actually being passed. Not whether the interface supports them. Whether the call site passes them and whether the implementation forwards them.
The failure is silent, the output is fluent, and fluent nonsense is much harder to notice than a stack trace.
We also took the chance to move the model ids off their 2024 defaults, which had been sitting in config for the best part of a year, and to sort out the use-case routing table that named a provider we had never implemented. It had been quietly falling back to the default model every time. Same species of bug: configuration that described an intention rather than the behaviour.