Tool use

Chapter 20 ended with reasoning models that can think for minutes before answering, but they can only describe what they would do, not do it. A reasoning model that cannot act on the world is limited. Tool use removes that limit. Modern LLMs are trained to emit structured calls to external functions; the surrounding system executes those calls; the results flow back into the model’s context. The model decides when to invoke a tool and which tool to invoke. This is the bridge from chat to agency.

The conceptual pattern is ReAct (Yao 2022), which Chapter 20 §4 introduced: think → act → observe → repeat. This chapter is about turning that pattern into production engineering. Function-calling APIs from OpenAI, Anthropic, and Google standardized the structured-call interface. JSON Schema specifies what arguments each tool takes. Constrained decoding from Chapter 19 guarantees that the model’s tool calls are well-formed. The agent loop iterates think-act-observe until the model produces a final answer. Multi-tool routing, observation handling, and error recovery turn the basic loop into a robust system. Anthropic’s Model Context Protocol (MCP) standardizes how tools and clients communicate. Computer use generalizes tool calls to any application’s UI.

Every production LLM agent runs on this stack, customer support bots, autonomous coding agents, retrieval pipelines, Claude with tool use, GPT with function calling, AI assistants embedded in IDEs and operating systems. By the end of this chapter you should know how to design and deploy a tool-using system, where the production failure modes are, and which protocols are likely to matter in 2025 and beyond. Tool use is reasoning made effectual, the capability that turns a model into an agent.

From ReAct to tools

The bridge from Chapter 20

Chapter 20 §4 introduced the ReAct pattern: the model emits a Thought, then an Action, receives an Observation, and continues with the next Thought. The loop runs until the model emits a final answer. That pattern is the oldest idea in this chapter, yet it has held up unusually well: every modern agent system still runs some form of Thought → Action → Observation underneath, regardless of which model or framework is in use, even as the API surface around it has churned repeatedly.

What has changed is everything under the pattern. In 2022 (the ReAct era), actions were emitted as natural-language strings, Action: search("population of France"), and the surrounding system parsed them with regex or heuristics. Brittle on a good day, broken on a bad one. In modern production (2024 – 2025), actions are emitted as structured JSON matching a declared schema; the system parses with a JSON parser; constrained decoding (Chapter 19 §8) guarantees that the call is well-formed token by token. The structural reliability is no longer a tuning concern; it is a property of the decoder.

What changed

The conceptual loop, think → act → observe → repeat, is identical. The grounding effect is identical: observations are real, not fabricated, so the model’s reasoning is anchored in actual tool outputs rather than its own priors. The model’s autonomy in choosing when and which tool to invoke is identical. What is new is production-grade infrastructure: schemas, validators, dispatchers, retry logic, monitoring, idempotency guards, audit logs, and the interoperability protocols this chapter closes with.

The empirical scale tells the same story. Toolformer (Schick 2023) demonstrated self-supervised tool-use training across a small fixed set of five tools, a calculator, a question-answering system, a search engine, a translation system, and a calendar, improving downstream task accuracy without degrading the model’s core language ability. GPT-4 with function calling (June 2023) reached $\sim 85$ – $95\%$ on standard tool benchmarks, with user-supplied tools rather than a fixed catalog. The modern frontier in 2025 routinely runs agentic loops across hundreds of tools and dozens of steps, hitting $\sim 70$ – $85\%$ on complex multi-step tasks (TaskBench, ToolBench), lower than single-call benchmarks because the loop has more chances to fail. The benchmark built specifically for this multi-turn regime is τ-bench (Yao et al. 2024, arxiv.org/abs/2406.12045), which drops an agent into a simulated customer-service environment with a scripted user simulator and a policy the agent must follow, then scores whether the end state is policy-compliant rather than whether any individual call was well-formed. Single-call tool selection has its own standard, the Berkeley Function-Calling Leaderboard (gorilla.cs.berkeley.edu/leaderboard.html). Chapter 30 returns to tool-use evaluation directly.

Training tool use

Tool-calling accuracy is not a property of clever prompting; it is trained in with the same SFT machinery Chapter 13 covered. A base model handed a tool schema for the first time in its life produces malformed calls, ignores tools it should use, and invents tools it should not, no matter how well the schema is written. Reliable tool use requires the model to have already seen many examples of exactly this shape: a conversation, a set of schemas, a well-formed call, an observation, and a continuation that uses the observation correctly.

The training data for this is built, not collected. Synthetic generation produces the raw supply of traces: a model, often a stronger one, or the base model itself in a self-supervised loop, is sampled against tool schemas to produce candidate call-observation-continuation sequences. Toolformer’s (Schick 2023) mechanism is a clean example: insert a candidate API call into existing text, run it, and keep the insertion only if it lowers the perplexity of the text that follows. Filtering is where the real work happens: a candidate trace is kept only if the call is well-formed and the tool actually helped, the calculator call lowered the perplexity of the correct continuation, the retrieved document contained the fact the answer needed. Traces where the tool was unnecessary, wrong, or actively hurt the final answer get discarded, not “corrected,” bad traces teach the wrong lesson no matter how they are patched. SFT then trains on the filtered set exactly as Chapter 13 describes: response-only loss masking over (context, response) pairs, except here the “response” alternates between a structured tool call and a text continuation conditioned on the tool’s observation. The model is not reasoning its way into tool use fresh at every inference call; it has seen thousands of instances of “when the context looks like this, the right move is a call, not text.”

Function calling APIs

The interface convention

The convention was established by OpenAI’s June 2023 function-calling release and the rest of the industry converged on it within a year. The model is given a list of available tools, each with a name, a description, and a JSON-Schema parameter spec, along with the user’s request and the conversation history. The model emits either a regular text response (no tool needed) or a structured tool call that names one of the available tools and includes a JSON-encoded argument object. The system parses the call, executes the corresponding function, and returns the result as a tool message in the conversation. The model continues with the result in context, possibly emitting more tool calls, until it produces a final text answer.

Provider variations

The differences across providers are small and shrinking. OpenAI uses a tools array on the request, a tool_choice parameter to bias selection, and a tool_calls field on the response. Anthropic uses a tools array as well, but the response contains tool_use content blocks rather than a separate field, and parallel tool calls, multiple tool invocations in a single response, are supported natively. Google Gemini uses the term “function declarations” but the shape is the same. All three converged on JSON Schema as the parameter specification language, which means a tool definition written for one provider is mechanically convertible to the others. The cross-provider portability story was different even a year ago; today it is real.

Why structured beats natural-language

The structural shift from ReAct-era string parsing to modern JSON tool calls is not cosmetic. Reliability is the headline: a JSON parser either succeeds or fails cleanly, while regex parsing of natural language is a long tail of edge cases that breaks in production six months after launch. Programmatic dispatch collapses from many steps (parse → extract intent → match function → marshal arguments → call) into one: parse the JSON, look up the function by name, splat the arguments. Type safety at the parameter level means bad calls fail at the API boundary with a clear error, not somewhere deep inside the tool. Composability is the most underrated benefit, when calls are structured uniformly, the agent loop can iterate over them without special-casing each tool’s parsing quirks.

The Chapter 19 bridge is what makes the structural guarantee bite. FSM masking (Ch 19 §8) constrains the decoder at each step to emit only tokens that continue a valid JSON object matching the declared schema. The model cannot emit a "location" field where an integer is required, cannot emit unbalanced braces, cannot emit a string that does not match a declared enum. Structural validity is guaranteed at the token level; the model never gets the chance to produce malformed output. Semantic correctness is still on the model: it can still pass location="Atlantis" to a weather tool, but the bytes are always well-formed.

# Anthropic-style tool call structure (schematic, no API calls).
# The pattern: define schemas, send query, parse response, execute, send result back.

tools = [
    {
        "name": "get_weather",
        "description": "Get current weather for a location.",
        "input_schema": {
            "type": "object",
            "properties": {
                "location": {"type": "string", "description": "City, e.g., 'San Francisco'"},
                "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]},
            },
            "required": ["location"],
        },
    }
]

# What the model emits (a real API call would produce this structured response).
mock_model_response = {
    "stop_reason": "tool_use",
    "content": [
        {"type": "text", "text": "I'll check the weather for you."},
        {
            "type": "tool_use",
            "id": "call_abc",
            "name": "get_weather",
            "input": {"location": "San Francisco", "unit": "fahrenheit"},
        },
    ],
}

# System extracts the tool call.
tool_call = next(b for b in mock_model_response["content"] if b["type"] == "tool_use")
print(f"Model wants to call: {tool_call['name']}({tool_call['input']})")

# System executes the function.
def get_weather(location, unit="fahrenheit"):
    return f"72°{'F' if unit == 'fahrenheit' else 'C'}, sunny"

result = get_weather(**tool_call["input"])
print(f"Tool result: {result}")

# System sends the result back as a tool_result message.
tool_result_message = {
    "role": "user",
    "content": [
        {"type": "tool_result", "tool_use_id": tool_call["id"], "content": result}
    ],
}
print(f"\nSent back to model: {tool_result_message}")
print("\n(The model would now produce a final text response using the weather data.)")

The OpenAI shape

The example above is Anthropic’s wire format. OpenAI encodes the same round trip differently in two places. The call rides inside the assistant message’s tool_calls array, and the arguments are a JSON-encoded string, not a parsed object, the caller has to json.loads it before use. The result is not a content block nested inside a user message; it is its own message with role: "tool", addressed back to the call it answers via tool_call_id.

# OpenAI-style tool call structure for the same get_weather round trip.

def get_weather(location, unit="fahrenheit"):
    return f"72°{'F' if unit == 'fahrenheit' else 'C'}, sunny"

# What the model emits: the call lives in tool_calls, arguments as a JSON string.
mock_model_response = {
    "role": "assistant",
    "content": None,
    "tool_calls": [
        {
            "id": "call_abc",
            "type": "function",
            "function": {
                "name": "get_weather",
                "arguments": '{"location": "San Francisco", "unit": "fahrenheit"}',
            },
        }
    ],
}

# System parses the JSON-string arguments, then executes.
import json
call = mock_model_response["tool_calls"][0]
fn_name = call["function"]["name"]
fn_args = json.loads(call["function"]["arguments"])   # string -> dict
print(f"Model wants to call: {fn_name}({fn_args})")

result = get_weather(**fn_args)
print(f"Tool result: {result}")

# The result becomes its own "tool" message, addressed by tool_call_id.
tool_message = {
    "role": "tool",
    "tool_call_id": call["id"],
    "content": result,
}
print(f"\nSent back to model: {tool_message}")

The field names and nesting differ; the principle underneath does not, a structured call goes out, a structured result comes back, and the model conditions on the result. Code written against one shape ports to the other by relabeling fields, not by rethinking the pattern.

Tool schemas

Anatomy of a schema

A tool schema has three parts. Name is a stable identifier, get_weather, search_database, send_email, used by the dispatcher to match the model’s call to the underlying function. Description is natural-language guidance: a short paragraph that tells the model what the tool does, when to use it, and what it returns. The description is the model’s primary signal for tool selection and the place where schema quality lives or dies. Parameters is a JSON Schema object describing the arguments, types, enums, required vs optional, ranges, and per-parameter descriptions.

A well-formed schema for a knowledge-base search tool:

name: search_database
description: |
  Search the company knowledge base for documents matching a query.
  Returns up to 5 most relevant documents with titles and excerpts.
parameters:
  type: object
  properties:
    query:
      type: string
      description: "The search query. Be specific."
    max_results:
      type: integer
      default: 5
      minimum: 1
      maximum: 10
  required: ["query"]

Schema best practices

The model picks tools by reading their descriptions; vague descriptions produce bad tool selection. Be concrete in the first sentence, what the tool does and what it returns. Use enums wherever a string parameter has a known set of valid values; unit: ["celsius", "fahrenheit"] is strictly better than unit: string because the constraint propagates into the schema and the model gets a stronger signal. Mark only truly-required parameters as required; the agent loop has to fill in every required field, so over-marking creates failure modes. Include short examples in the description when the input format is non-obvious, “Examples: ‘pricing changes’, ‘API authentication’” outperforms a parameter description that says only “the query.” Avoid overlapping tools: two tools whose descriptions are interchangeable will produce inconsistent selection, and the model will pick whichever one is mentioned first in context.

The production sweet spot for tool descriptions is 50 to 200 characters, plus one or two example inputs when helpful. Below 50 characters the description is usually too thin to disambiguate from neighboring tools; above 200 characters the description bloats the context window without proportionate selection accuracy. Each tool’s description is paid for at every model call in the conversation, so the per-tool budget compounds when the catalog grows.

{
  "name": "get_weather",
  "description": "Get current weather for a location.",
  "input_schema": {
    "type": "object",
    "properties": {
      "location": {
        "type": "string",
        "description": "City name, e.g., \"San Francisco\" or \"Tokyo\"."
      },
      "unit": {
        "type": "string",
        "description": "Temperature unit.",
        "enum": [
          "celsius",
          "fahrenheit"
        ],
        "default": "fahrenheit"
      }
    },
    "required": [
      "location"
    ]
  }
}

{
  "name": "get_weather",
  "input": {
    "location": "Tokyo",
    "unit": "celsius"
  }
}

The agent loop

Single tool call

The simplest production pattern is one tool call per query. The user asks a question requiring one piece of external information; the model emits a tool call; the system executes; the result is returned; the model produces its final answer. This is enough for most weather, calculator, and database-lookup scenarios, a large fraction of real production tool use is exactly this shape. The rest needs the loop.

Multi-step loop

When a single tool call cannot answer the question, multi-step math, a research task with sequential lookups, a workflow that branches based on intermediate results, the agent loop iterates. The structure is short and surprisingly general:

while not done:
    response = model.generate(context, tools)
    if response.is_text():
        return response.text   # final answer; terminal
    tool_call = response.tool_call
    result = execute(tool_call)
    context.append(tool_call)
    context.append(result)

Termination comes from one of three places. Natural termination: the model emits a text response with no tool call, signaling that it has enough information to answer. This is the desired outcome. Limit termination: the loop hits a max_iterations cap, typically $10$ to $50$ steps, to prevent runaway loops on poorly-formed queries. Error termination: an unrecoverable error in tool execution, or a permission denial the model cannot pivot around. The loop layer owns retries and backoff for transient failures; the tool layer owns its own correctness.

Parallel tool calls

Anthropic and OpenAI both support emitting multiple tool calls in a single response. When the calls are independent, three different searches for three different facts, they execute in parallel and the results come back together in the next message. This is a meaningful speedup for the right shape of query: a five-step sequential loop with $500$ms per call is $2.5$s of latency, while the same five calls in parallel are $500$ms. The model is usually good at distinguishing, it emits parallel calls when the calls are independent and serial calls when one depends on another. Do not force parallelism: if tool B needs tool A’s output, parallelizing them just wastes B’s call.

The agent loop with a mock model and two mock tools is short enough to read end-to-end:

# Schematic agent loop with mock model + mock tools.

def mock_model(messages, tools):
    """
    Mock model that produces tool calls based on the conversation state.
    In production this would be a real LLM API call.
    """
    n_tools_called = sum(
        1 for m in messages
        if m["role"] == "assistant" and "tool_use" in str(m.get("content", ""))
    )

    # First call: search the web.
    if n_tools_called == 0:
        return {"stop_reason": "tool_use",
                "tool_use": {"name": "web_search",
                             "input": {"query": "current weather San Francisco"}}}

    # Second call: convert the result.
    if n_tools_called == 1:
        return {"stop_reason": "tool_use",
                "tool_use": {"name": "calculator",
                             "input": {"expression": "(72 - 32) * 5 / 9"}}}

    # Final: produce a text answer.
    return {"stop_reason": "end_turn",
            "text": "It's 72°F (about 22°C) in San Francisco today."}

def web_search(query):    return "72°F, sunny in SF"
def calculator(expression): return f"{eval(expression):.1f}"

TOOLS = {"web_search": web_search, "calculator": calculator}

def agent_loop(user_message, max_iterations=10):
    messages = [{"role": "user", "content": user_message}]

    for step in range(max_iterations):
        response = mock_model(messages, TOOLS)
        print(f"\n--- Step {step + 1} ---")

        if response["stop_reason"] == "end_turn":
            print(f"Final answer: {response['text']}")
            return response["text"]

        call = response["tool_use"]
        print(f"Tool call: {call['name']}({call['input']})")
        result = TOOLS[call["name"]](**call["input"])
        print(f"Observation: {result}")

        messages.append({"role": "assistant",
                         "content": [{"type": "tool_use", **call}]})
        messages.append({"role": "user",
                         "content": [{"type": "tool_result", "content": result}]})

    return None   # hit max_iterations

agent_loop("What's the weather in San Francisco in Celsius?")

Multi-tool routing

The selection problem

At 5 to 10 tools, the model reads every description and picks the right one. Selection accuracy is high; this is the default mode. At 50 or more tools, the descriptions consume too much context, even at 200 characters each, $50$ tools is $10{,}000$ characters of schema per call, every call, in addition to the conversation. At 500 or more tools, the model gets confused; similar tools collide; accuracy drops below the threshold where the agent loop can recover. Production agents with large tool catalogs need a routing strategy.

Production patterns

Description-based routing is the default and works at $5$ – $50$ tools. The model sees all descriptions and picks; no preprocessing needed. Hierarchical routing classifies the request into a tool category first, then picks a specific tool within that category, a useful structure at $50$ – $500$ tools, especially when the catalog has natural groupings (search tools, write tools, communication tools). Tool retrieval, RAG for tools, embeds each tool’s description; at each step the system retrieves the top-$N$ most-relevant tools and presents only those. This is the required pattern at $100+$ tools, and the same vector-store machinery that Chapter 22 covers for document RAG applies directly. RBAC and permissions hide tools the user or agent does not have access to; this is non-negotiable for production safety, regardless of catalog size.

The failure modes are predictable. Overlapping descriptions, search_documents vs find_files, produce inconsistent selection, with the model picking whichever description matches the query’s surface phrasing rather than its intent. Vague descriptions, “fetch data”, give the model no signal about which data, so the routing falls through to whichever tool was listed first. Hallucinated tools: the model emits a call for a tool that does not exist, happen rarely but reliably; the recovery pattern is to return a structured error listing valid tool names, and the model almost always picks one of those on the next turn.

Observation handling and error recovery

Observations vary in size

A web search might return $50$KB of HTML. A database query might return ten thousand rows. A calculator returns one number. Feeding all of that back into the model’s context unmodified would burn the context window in two or three steps. Truncation caps the observation at $N$ tokens and appends ...truncated; cheap and good enough for many cases. Summarization uses a smaller model to compress the observation before feeding it back, useful when the raw output is structured (search results, log lines) and the summary preserves the parts that matter. Structured extraction returns only the relevant fields, {title, snippet} per search result rather than full HTML; {rows_returned, top_5} rather than the full result set. The right strategy depends on the tool; the discipline is to not let observation size grow without limit.

Errors are observations too

When a tool fails, return the error message as the observation. The model often recovers gracefully:

Action: get_weather(location="Atlantis")
Observation: Error: location not found.
Thought: Atlantis is not a real location. Let me ask the user to clarify.
Final answer: I couldn't find weather for "Atlantis". Could you specify a different city?

The common error patterns are tractable in roughly this order of difficulty. Bad parameters are caught at schema validation and rarely escape to the tool; constrained decoding from Chapter 19 prevents most of them. Missing data: the tool returns empty, is handled by the model, which either rephrases the query or asks the user. Service unavailable transient failures are handled at the loop layer with retry and backoff; the model never sees them unless the retries are exhausted. Permission denied errors are surfaced to the model, which can pivot to a different approach or ask the user for elevated access. Rate limiting is similar, the model can decide to wait, pivot to a less-rate-limited tool, or give up and report.

The structural pattern is the same in every case: an error becomes an observation, the observation enters the loop, and the model decides what to do next. This is agent resilience handled at the loop layer, not the tool layer. Tools should fail informatively; the loop should treat failure as data.

MCP and modern protocols

The interoperability problem

Through 2023 and most of 2024, every LLM integration was a custom build. Connecting Claude to a Postgres database meant writing custom function-calling glue for Claude’s tool API; connecting GPT to the same database meant writing the same glue for OpenAI’s API; the work did not transfer. Multiply across providers, across tools, across teams, and the duplicate effort across the ecosystem was substantial. The conditions were exactly right for a standard.

Anthropic’s Model Context Protocol

MCP (Anthropic, late 2024) is a standard interface between LLM clients and tool/data servers. Servers expose tools, resources, and prompts through the protocol; any MCP-compatible client can talk to any MCP-compatible server. A database team writes one MCP server; any MCP-compatible client, Claude Desktop, a VS Code plugin, a third-party agent framework, can use it without bespoke integration. The transferability is the breakthrough; one tool server, many LLM clients.

The protocol mechanics are deliberately small. MCP runs JSON-RPC over stdio or HTTP. Capabilities negotiation lets a client ask the server “what tools, resources, and prompts do you offer?” and the server replies with a manifest. Tool invocation uses a tools/call method with a name and an argument object; the server executes and returns the result. Resource access exposes read-only data via URIs, useful for context (files, documents, database snapshots). Prompt templates let the server provide reusable prompt scaffolding that the client can fetch and present to the model.

Adoption in early $2025$ is broad enough that engineers building tool-using systems will encounter MCP regularly. Claude Desktop ships MCP clients out of the box. VS Code, Cursor, and other IDEs include MCP plugins. Third-party MCP servers exist for Git, GitHub, Slack, Postgres, Notion, Linear, Asana, and dozens more. The trajectory is the trajectory of LSP (Language Server Protocol) for code editing, an emerging standard whose ecosystem effect is the point.

Exercises

Four exercises that lock in the production tool-use stack. Each is a self-contained problem with a starting template; hints are collapsed by default; try the problem first.

The exercises trace the chapter’s arc: define a schema (Ex 1) → run the agent loop (Ex 2) → route across many tools (Ex 3) → handle side effects safely (Ex 4).

Exercise 1 (easy): Define a tool schema

Convert a Python function signature into a JSON-Schema-style tool schema. Cover name, description, required vs optional parameters, and at least one enum.

schedule_meeting_schema = {
    "name": "schedule_meeting",
    "description": "Schedule a meeting on the user's calendar. Use when the user asks to book, set up, or add a meeting.",
    "input_schema": {
        "type": "object",
        "properties": {
            "title": {"type": "string", "description": "The meeting's title or topic."},
            "duration_minutes": {
                "type": "integer",
                "description": "Length in minutes.",
                "minimum": 5,
                "maximum": 480,
                "default": 30,
            },
            "urgency": {
                "type": "string",
                "description": "How urgent the meeting is.",
                "enum": ["low", "normal", "high"],
                "default": "normal",
            },
        },
        "required": ["title"],
    },
}

Exercise 2 (medium): Implement the agent loop

Build the agent loop pattern: a function that takes a mock model, a tool registry, and a user message; iterates think → act → observe → repeat until the model returns a final answer or hits max_iterations.

def agent_loop(model, tools, user_message, max_iterations=10):
    messages = [{"role": "user", "content": user_message}]

    for iteration in range(max_iterations):
        response = model(messages, list(tools.keys()))
        messages.append({"role": "assistant", "content": response["content"]})

        if response["stop_reason"] == "end_turn":
            for block in response["content"]:
                if block.get("type") == "text":
                    return block["text"]
            return None

        tool_results = []
        for block in response["content"]:
            if block.get("type") == "tool_use":
                fn = tools[block["name"]]
                result = fn(**block["input"])
                tool_results.append({
                    "type": "tool_result",
                    "tool_use_id": block["id"],
                    "content": str(result),
                })
        messages.append({"role": "user", "content": tool_results})

    return None   # hit max_iterations

Exercise 3 (medium): Multi-tool routing via embeddings

Implement embedding-based tool retrieval for a large tool catalog. Given a user query and a catalog of 20 tools (with mock embeddings), return the top-5 most relevant tools to present to the model.

def cosine_similarity(a, b):
    return np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b))

def retrieve_tools(query, tools, top_k=5):
    q = embed_query(query)
    scored = [(tool, cosine_similarity(q, tool["embedding"])) for tool in tools]
    scored.sort(key=lambda pair: pair[1], reverse=True)
    return scored[:top_k]

Exercise 4 (hard): Idempotent tool dispatch

Implement a tool dispatcher that handles side-effect-causing tools safely. Use idempotency keys to prevent duplicate executions on retry, and an audit log to track every call.

def dispatch(self, tool_name, args, idem_key=None):
    if idem_key is None:
        idem_key = str(uuid.uuid4())

    cache_key = (tool_name, idem_key)
    if cache_key in self.cache:
        self.audit.append({"tool": tool_name, "idem_key": idem_key, "status": "cached", "ts": datetime.now()})
        return self.cache[cache_key]

    if tool_name not in self.tools:
        error = {"status": "error", "message": f"Unknown tool: {tool_name}"}
        self.audit.append({"tool": tool_name, "idem_key": idem_key, "status": "error", "ts": datetime.now()})
        return error

    fn = self.tools[tool_name]
    result = fn(**args)
    self.cache[cache_key] = result
    self.audit.append({"tool": tool_name, "idem_key": idem_key, "status": "executed", "ts": datetime.now()})
    return result

The full picture

Computer use: the frontier

Anthropic’s computer use (late 2024) generalizes tool use to any application’s UI. The “tools” become a small primitive set: Screenshot to see the screen, Mouse to click and drag, Keyboard to type and press, Wait to let animations settle. The model becomes a “generalist computer operator”: it can drive any application by looking at the screen and operating mouse and keyboard. The agent loop pattern is identical; only the tools have changed.

The implications are substantial and the caveats are equally substantial. No custom integration needed for legacy apps: the model can use anything a human can use, which opens up a long tail of internal tools and enterprise software that never had a public API. Slow and error-prone today: each step requires a screenshot, a model call, and an action, so a five-step UI workflow that a human does in $10$ seconds takes a computer-use agent several minutes. High-stakes by default: a misaligned model with computer use can delete files, send emails, and click “buy now,” so the production deployment needs careful guardrails. Improving rapidly: the model’s UI-understanding accuracy has climbed quickly across $2024$ and into $2025$, and the latency tax has dropped as serving stacks specialized for the workload.

What’s next in Part VII

Tool use is the capability that turns a reasoning model into an agent. The Thought-Action-Observation loop from ReAct (Yao 2022) became production engineering across $2023$ – $2024$: structured tool calls with JSON Schema, constrained decoding for reliability, parallel calls for speed, multi-tool routing for scale, MCP for interoperability, computer use for legacy applications. Every commercial agent built today rests on this stack. The engineering disciplines this chapter covered, schema design, loop control, observation handling, error recovery, and idempotency, are what separate a production system from a demo.

Chapter 22 opens next: retrieval-augmented generation. Retrieval is one of the most useful tools: the model looks up information it does not have memorized. RAG has its own engineering depth (chunking, embedding, vector search, reranking), so it gets its own chapter. Chapter 23 extends the capability stack to multimodal inputs and outputs: vision, audio, video, the input modalities that make computer use practical and that unlock the next set of agent applications. Then Part VIII covers safety, interpretability, and evaluation, the disciplines that turn capable systems into trustworthy ones. Then Part IX assembles everything into full agent architectures, where reasoning, tool use, retrieval, and multimodal compose into the complete systems that this chapter has been quietly building toward.

The capability stack is being assembled. Reasoning gave the model time to think; tool use gives it agency; retrieval will give it knowledge; multimodal will extend it beyond text. The chapter that turns reasoning into agency is done; the chapter that gives that agency a knowledge base opens next.