---
title: "Using the Claude API in Your Product: Tool Calls, Live Text and Lower Bills"
url: https://phatdev.net/en/blog/claude-api-tool-use-streaming-caching
lang: en
updated: 2026-10-21
alternate:
  vi: https://phatdev.net/vi/blog/claude-api-tool-use-streaming-caching
markdown: https://phatdev.net/en/blog/claude-api-tool-use-streaming-caching.md
---

# Using the Claude API in Your Product: Tool Calls, Live Text and Lower Bills

> Let the AI call your functions, show text as it arrives instead of a blank screen, and stop paying repeatedly for the same text. Correct models for October 2026, with Python and TypeScript.

Categories: AI | Published: 2026-10-19 | Level: Intermediate | 7 min read

Tags: claude, anthropic, tool-use, streaming, prompt-caching

Say you want to add an AI assistant to a shop website: the customer types an order number and the assistant looks up the delivery status. The demo works nicely. Then it goes live and three things happen. Customers stare at a blank screen for ten seconds or more before any text shows up. The assistant can't look up orders by itself, because it knows nothing about your data. And the API bill at month end is higher than you expected, because every call re-sends a long instruction manual.

Those are the three jobs of this post: how to let the AI call your functions, how to show text as it appears, and how to stop paying repeatedly for the same block of text. I use the Claude API. Model names, parameters and syntax below were checked against the current official SDK docs (October 2026), not written from memory.

## How the API works

Everything goes through one door. You send a list of messages (alternating between the user and the AI), optionally with system instructions and a list of tools. You get back a list of content blocks: text, tool requests, thinking summaries. The API remembers nothing between calls, so each time you send the whole conversation again.

Install the official library with `pip install anthropic` (Python) or `npm install @anthropic-ai/sdk` (JavaScript). Keep the API key in the `ANTHROPIC_API_KEY` environment variable, never in code. The library reads it automatically.

## Which model

| Model | ID | Good for | Price in / out per million tokens |
|---|---|---|---|
| Claude Opus 5.5 | `claude-opus-5-5` | hard tasks, code, agents; the default choice | $4 / $20 |
| Claude Sonnet 5.5 | `claude-sonnet-5-5` | high volume, faster and cheaper | $2 / $10 |
| Claude Haiku 5.5 | `claude-haiku-5-5` | classification, simple extraction | $0.10 / $0.50 (prompts up to 100K tokens) |

Write the ID string exactly as in the table, with no date suffix. Prices can change, so check the official pricing page when you budget.

Three things differ from older models and often cause 400 errors. On Opus 5.5 the AI always has a "thinking" step that can't be turned off; leave the `thinking` parameter out. Sending `{"type": "disabled"}` or `budget_tokens` is rejected. To go faster and cheaper, lower `output_config={"effort": "low"}`; the default for this generation is `medium`. Second, don't send a `temperature`, `top_p` or `top_k` other than the default. Third, forcing the AI to call one specific tool with `tool_choice` set to `any` or `tool` is also rejected on Opus 5.5 and Sonnet 5.5; use `auto` and say so in the instructions.

## Letting the AI call your functions (tool use)

Think of it this way. The AI is a capable employee sitting in a closed room. To look up an order, it writes you a note: "please check order HD-1001". You check, then slide the result under the door. It reads it and answers the customer. The AI never runs your function itself. It asks, you run.

Technically, you describe a tool with a name, a description and a data shape. The AI returns a `tool_use` block with parameters, you run the function, and you send the result back in a `tool_result` block. This repeats until the AI stops asking.

![A loop of a model, a tool and a result joined by arrows, with a row of small blocks flowing out on the right like a stream](https://phatdev.net/uploads/blog/claude-api-tool-use-streaming-caching/tool-loop-1920.webp)

Three things to remember. Always validate what the AI sends before running the function, and never assume it matches the shape. If the function fails, return `is_error: true` with an explanation so the AI can correct itself, and don't silently skip it. And if the AI asks for several things in one turn, run them all and send every result in a single message; splitting them across messages gradually teaches the AI to stop asking in parallel.

## Showing text as it appears (streaming)

Normally the API finishes the whole answer before sending it. For a long answer the customer sees a blank screen. Streaming sends pieces of text the moment they're produced, like watching someone type. The library also requires streaming when you allow very long answers.

One detail: on the newer models the "thinking" part isn't returned by default. If you want to show customers a summary, set `thinking={"type": "adaptive", "display": "summarized"}`.

## Not paying twice for the same text (prompt caching)

If every call re-sends the same long manual, mark it so the API remembers it. On later calls the remembered part costs a small fraction of the normal price (around a tenth, or less, depending on the model). The first write costs a little extra, about 1.25 times.

The rule to remember: the cache matches from the start. Change one word near the top and the cache is lost from that point on. Requests are assembled as tools, then system instructions, then messages, so put what rarely changes first and what often changes last. The remembered part also has to be long enough (the docs list a minimum of roughly 512 tokens depending on the model, so double-check); if it's too short the API raises no error, it just doesn't cache. The default memory lasts 5 minutes, and you can set `"ttl": "1h"`.

![A stack of isometric slabs: the three lower stable slabs glow like the cached prefix, the thin top slab has a different colour like the part that changes](https://phatdev.net/uploads/blog/claude-api-tool-use-streaming-caching/cache-prefix-1920.webp)

To know whether caching works, read the usage numbers (`usage`). On the first call `cache_creation_input_tokens` is large; on later calls `cache_read_input_tokens` is large. If the read number is always 0, something is quietly breaking the cache: the current time inside the instructions, a random ID, or a tool list that changes order every time.

## For those who write code

The tool-calling loop in Python (the TypeScript version has the same structure):

```python
TOOLS = [{
    "name": "get_order_status",
    "description": "Look up an order by its id. Use only when the user gives an order id.",
    "input_schema": {
        "type": "object",
        "properties": {"order_id": {"type": "string", "description": "Order id like HD-12345"}},
        "required": ["order_id"],
        "additionalProperties": False,
    },
}]

def run_with_tools(client, user_input, max_turns=8):
    messages = [{"role": "user", "content": user_input}]
    for _ in range(max_turns):
        response = client.messages.create(
            model="claude-opus-5-5", max_tokens=16000, tools=TOOLS, messages=messages)
        if response.stop_reason in ("refusal", "max_tokens"):
            raise RuntimeError(f"Stopped early: {response.stop_reason}")
        if response.stop_reason == "pause_turn":          # only with server-side tools
            messages.append({"role": "assistant", "content": response.content})
            continue
        if response.stop_reason != "tool_use":
            return "".join(b.text for b in response.content if b.type == "text")

        messages.append({"role": "assistant", "content": response.content})
        results = []
        for block in response.content:
            if block.type == "tool_use":
                content, is_error = execute_tool(block.name, block.input)
                results.append({"type": "tool_result", "tool_use_id": block.id,
                                "content": content, "is_error": is_error})
        messages.append({"role": "user", "content": results})   # ALL results in ONE message
    raise RuntimeError("Too many tool turns")
```

Streaming and caching, the same idea in both languages:

```python
with client.messages.stream(
    model="claude-sonnet-5-5", max_tokens=64000,
    messages=[{"role": "user", "content": prompt}],
) as stream:
    for text in stream.text_stream:
        print(text, end="", flush=True)
    final = stream.get_final_message()     # full Message, including usage and stop_reason

response = client.messages.create(
    model="claude-sonnet-5-5", max_tokens=16000,
    system=[{"type": "text", "text": guidelines,                 # stable part first
             "cache_control": {"type": "ephemeral"}}],
    messages=[{"role": "user", "content": f"Review this diff:\n{diff}"}],  # changing part last
)
u = response.usage
print(u.cache_creation_input_tokens, u.cache_read_input_tokens, u.input_tokens)
```

```typescript
const stream = client.messages
  .stream({ model: "claude-sonnet-5-5", max_tokens: 64000,
            messages: [{ role: "user", content: prompt }] })
  .on("text", (text) => process.stdout.write(text));
const final = await stream.finalMessage();
```

Opus 5.5's safety filter sometimes declines a request (`stop_reason: "refusal"`). There is a beta parameter `fallbacks: "default"` with the header `server-side-fallback-2026-07-01` that makes the API re-run the declined request on a substitute model. I only checked that function with the TypeScript compiler and a fake server, not against the real API.

## Common mistakes

The most common one is ignoring `stop_reason`. If it's `max_tokens` the answer was cut off midway; if it's `refusal` the filter declined and `content` may be empty. Check it before reading the content. Another is catching every error in one generic handler. Separate them: `RateLimitError` and 5xx errors deserve a retry (the library already retries twice), while 4xx errors don't, because sending the same thing again fails again. And never leave the API key in the repo or in logs.

## What I checked and what I didn't

My machine has no API key, so no real call was made. On 21 Oct 2026 I re-ran three things. First, compiling the Python file (`python -m py_compile`, with the latest `anthropic` 1.17.0 installed in a scratch folder). Second, `tsc --noEmit` with `@anthropic-ai/sdk` 0.135.0, which accepted the types and parameters, including `fallbacks: "default"`. Third, running both versions against a local fake Messages server: the tool loop sent exactly one `tool_use` and then exactly one `tool_result` in a single message, streaming assembled the text correctly, and `usage` returned the right cache numbers. That checks how my code and the library handle responses; it proves nothing about the real API's behaviour. Before using this for anything billed, run it once with your own key.

## What to do next

Start small: one plain call with `claude-opus-5-5` or `claude-sonnet-5-5`, turn on streaming, and only then add tools. Once tools work, move the long instructions to the top, mark them for caching, and read `usage` to see whether you really save money.

If you need a whole team of AIs rather than a single call, see [Handing a Big Job to Several AIs Without Breaking Things](https://phatdev.net/en/blog/agentic-workflow-orchestrator). How to word the instructions for each call is in [How to Ask an AI So It Gets It Right and Stops Making Things Up](https://phatdev.net/en/blog/prompt-engineering-for-developers).

## References

- [Claude docs: Models overview](https://platform.claude.com/docs/en/about-claude/models/overview)
- [Claude docs: Tool use overview](https://platform.claude.com/docs/en/agents-and-tools/tool-use/overview)
- [Claude docs: Streaming Messages](https://platform.claude.com/docs/en/build-with-claude/streaming)
- [Claude docs: Prompt caching](https://platform.claude.com/docs/en/build-with-claude/prompt-caching)
- [Claude docs: Pricing](https://platform.claude.com/docs/en/about-claude/pricing)
- [Claude docs: Errors](https://platform.claude.com/docs/en/api/errors)
- [anthropic-sdk-python (GitHub)](https://github.com/anthropics/anthropic-sdk-python)
- [anthropic-sdk-typescript (GitHub)](https://github.com/anthropics/anthropic-sdk-typescript)

## Related posts

- [How to Ask an AI So It Gets It Right and Stops Making Things Up](https://phatdev.net/en/blog/prompt-engineering-for-developers.md)
- [Handing a Big Job to Several AIs Without Breaking Things](https://phatdev.net/en/blog/agentic-workflow-orchestrator.md)
- [Claude, Codex and Gemini: Which AI Coding Assistant for Which Job](https://phatdev.net/en/blog/claude-codex-gemini-for-coding.md)