The async execute_tool_calls() silently dropped calls to tools that
were not in tools= (or had no implementation): output and exception
were assigned but no ToolResult was ever appended, so the next
provider call carried an assistant tool_call with no matching result
- which OpenAI and Anthropic reject. The sync executor already
returned an 'Error: tool ... does not exist' result.
The async path now mirrors the sync one: before_call fires (and can
CancelToolCall) even though the tool is unavailable, and an error
ToolResult is appended in call order. Also removes the now-unreachable
tool-is-None branch from the inline sync-implementation path.
This matters more since chain resume landed: a pending call whose
tool is no longer registered would otherwise never resolve.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* PauseChain primitive + chain resume from pending tool calls
Two features that together give chains a first-class suspend/resume
story for human-in-the-loop tools:
llm.PauseChain: raise inside a tool implementation to stop the chain
cleanly. Unlike other exceptions it is not converted into an error
ToolResult - it propagates to the caller with .tool_call (the paused
call) and .tool_results (completed sibling results) attached, and no
provider call is made with a placeholder result. Failure semantics
for concurrent tool execution are now defined: async sibling tasks
always run to completion before a pause or hook exception propagates
(gather with return_exceptions, raised after collection), so nothing
is orphaned mid-flight; sync execution stops at the paused call,
leaving later calls unstarted so they can safely run on resume.
Chain resume: chain(messages=history, tools=...) now detects a
history ending in an assistant message with unresolved tool calls -
e.g. one persisted when a previous run paused or crashed - executes
those calls through the normal before_call/after_call machinery
(skipping any that already have results), then sends the results to
the model as a standard tool-result turn. A resumed call may pause
again, enabling multi-question flows. Histories where a user or
assistant message follows the calls are left alone. Also adds
execute_tool_calls(tool_calls_list=) for executing an explicit list.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Guarantee every tool call has a unique tool_call_id
add_tool_call() now synthesizes a unique tc_-prefixed id (monotonic
ULID) whenever the provider did not supply one. Previously consumers
correlating tool calls with results - or keying external state on a
specific invocation - had to invent fallback matching schemes for
id-less providers, and test models like llm-echo exercised different
code paths than production providers.
Provider-supplied ids are preserved untouched, and responses
rehydrated from the logs database keep their stored ids (synthesis
only happens at add_tool_call time). Existing tests that asserted
tool_call_id None now normalize or mask the synthesized ids.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Tool implementations can receive the ToolCall via llm_tool_call parameter
Tool functions (sync or async, including Toolbox methods) that declare
a parameter named llm_tool_call are now passed the llm.ToolCall object
for the current invocation. The parameter is reserved: it is excluded
from the input schema exposed to the model and is only injected when
declared explicitly - a **kwargs catch-all does not receive it.
This lets tool implementations key external state against the unique
tool_call_id, e.g. for human-in-the-loop approval flows that need to
resume a specific tool call after the answer arrives.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Ran Black
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
When building the `input` list for the OpenAI Responses API from prior
conversation turns, an assistant text-only turn was being serialized as:
{"role": "assistant",
"content": [{"type": "output_text", "text": "..."}]}
The openai-python SDK's EasyInputMessage shape uses a plain string for
this case, matching what a direct OpenAI Responses call would send. Use
the same shape so our history matches the SDK exactly, and add tests
covering both _build_responses_input and a two-turn response.reply()
flow.
Response keeps two parallel stores: _stream_events (read by to_dict /
response.messages) and _tool_calls (read by execute_tool_calls). Only
the messages were correctly serialized, leaving _tool_calls unrecorded.
Part assembly now also walks _tool_calls and appends a ToolCallPart
for any tool_call_id not already represented by a StreamEvent-derived
Part.
Closes#1433
The previous commit wired up encrypted_content round-trip but only
tested that the data flows through correctly on a single tool round-
trip. This adds a multi-turn cassette test that proves the full
interleaved-reasoning capability:
- Each turn produces fresh reasoning_tokens (not just the first)
- Every prior reasoning block is round-tripped on every subsequent
turn (the Nth turn echoes >= N-1 reasoning items)
- ReasoningParts persisted on the assistant messages carry the same
encrypted_content + id that gets sent back on the wire
The puzzle is shaped so the model can't parallelize tool calls -
each db_lookup result tells it the next key to use, forcing the
model to think between calls. The recorded 4-turn chain shows
reasoning_tokens of 45/98/196/17 across turns with reasoning items
accumulating in every outgoing input.
This is the GPT-5-class capability that Chat Completions can't
deliver because it discards reasoning between turns.
When the Responses API returns a reasoning item alongside function
calls, capture its opaque id + encrypted_content as provider_metadata
on the resulting ReasoningPart. _build_responses_input already echoed
that metadata back as a reasoning input item on the next turn - now
the output side actually populates it.
This preserves the model's hidden chain of thought across the tool
round-trip. Without it, GPT-5-class models silently lose ~3% on
SWE-bench (per OpenAI) when used with tools.
Adds a dedicated VCR test that asserts the encrypted_content captured
on the first turn appears verbatim in the second turn's outgoing
request body.
Adds Responses and AsyncResponses classes that drive the OpenAI
/v1/responses endpoint. The existing Chat / AsyncChat classes are
unchanged because other plugins import them.
gpt-5.5 (and gpt-5.5-2026-04-23) is now registered against Responses
by default. Pass `-o chat_completions 1` to fall back to the older
/v1/chat/completions code path.
This is feature parity with the Chat path (text, tools, streaming,
schema, reasoning_effort, verbosity, attachments, system prompts).
Interleaved reasoning across tool round-trips is not exercised yet -
encrypted reasoning items are accepted on the input side, but the
plugin doesn't yet stash them on outgoing ReasoningParts.
Accepts model options as an explicit dict alongside the existing
**kwargs form. The kwargs form continues to work unchanged for
backwards compatibility but is no longer documented. Mixing the two
forms with overlapping keys raises TypeError.
Applies to Model.prompt, Conversation.prompt, Response.reply and
their async equivalents. .chain() already used this pattern.
Co-authored-by: Claude <noreply@anthropic.com>
Each row stores only its current-turn inputs, so a loaded tool-result
response began with an orphan tool_result. `llm -c` then sent a request
with an unexpected tool_use_id. Stitch each response's messages onto the
previous response's chain plus its assistant output during load.
Closes#1426
Refs https://github.com/simonw/llm-anthropic/issues/68
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Providers (e.g. Gemini) often emit thought text with trailing newlines.
Concatenated and combined with click.echo's own newline, that produced
several blank lines before the `## Response` heading. rstrip() at render
time tightens the gap to a single blank line; the stored reasoning column
keeps the provider's text verbatim.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- Drop the placeholder messages() declaration on _BaseResponse so
AsyncResponse.messages() (an async coroutine) no longer trips
the mypy override check. text/json/tool_calls already follow
this pattern.
- Remove three unused imports flagged by ruff in test_parts.py.
- Apply pending black reformats across the tree.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds a `reasoning` column to the responses table (migration m022)
populated from concatenated visible-reasoning text in the assembled
message. `llm logs --md` renders it under a `## Reasoning` heading
above the response when present.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sync: response.messages() forces execution if not drained, so callers
no longer have to remember to call .text() first. Async: `await
response.messages()` awaits the force.
Internal sync paths (_response_to_dict, _chain_for_tool_results,
_build_full_chain, Response.reply, AsyncResponse.reply) use a new
private _messages_now() helper that assumes the response is already
drained, so they don't have to await on async responses.
Drops the now-obsolete "accessing .messages on un-awaited
AsyncResponse raises" parity test — that constraint goes away with
the method form.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Zero-arg sugar: when a response made tool calls and tool_results= is
not passed, reply() runs self.execute_tool_calls() and threads the
results into the next turn. Pass tool_results= explicitly to skip
the auto-execute path (e.g. for mutated or synthetic results). Also
forwards self.prompt.tools to the next turn so the model can call
the same tools again, mirroring Conversation.prompt's tools-or-self
rule.
AsyncResponse.reply() is now an awaitable coroutine — `await
response.reply(...)` — so the auto-execute path can `await
self.execute_tool_calls()` internally. This is a non-shipped API
break: existing async-reply callers in the test suite updated.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
ReasoningPart.token_count duplicated info already on response.token_details
(reasoning_tokens), and the side-channel `response._reasoning_token_count`
attribute with its set_usage ordering footgun was the wrong shape. Replaced
with a clean StreamEvent.redacted=True marker that plugins yield like any
other event. The framework hoists redacted reasoning Parts to the start of
the assembled message so UIs render them before content, even though the
opaque count typically arrives at the end of the stream.
Also fix parallel tool calls emitted without tool_call_id (e.g. Gemini):
a fresh tool_call_name now always allocates a new index instead of falling
through to the prior tool-call group, so N parallel calls produce N
distinct ToolCallParts instead of one with concatenated names and args.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Bug: ChainResponse.responses() and AsyncChainResponse.responses()
built the follow-up Prompt (for tool-result turns inside the chain
loop) without propagating system= or system_fragments= from the
initial prompt. Adapters that read prompt.system directly — OpenAI's
Chat, for example, which sends system as its own message — saw an
empty system on every turn after the first, silently losing the
caller's instruction.
Fix: pass system=self.prompt._system and
system_fragments=self.prompt.system_fragments when constructing the
next Prompt. Same change on sync and async paths.
_chain_for_tool_results keeps building messages= from the prior
response's prompt.messages + messages, so adapters that read
prompt.messages (the Phase 7 canonical input) continue to work too —
this fix is specifically for the adapters that still use the
prompt.system legacy field.
Three regression tests (sync, sync+system_fragments, async) pin the
new behavior.
670 tests in llm core; llm-anthropic (32) and llm-gemini (50) still
green.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Parity with prompt(): all four chain() methods —
Conversation.chain, AsyncConversation.chain, _Model.chain,
_AsyncModel.chain — now accept a messages= kwarg and pre-bake the
full chain via _build_full_chain so the first response of the chain
loop satisfies the invariant response.prompt.messages == what was
sent.
Semantics match prompt() exactly: when messages= is passed, it's
authoritative for the first turn. The prompt= kwarg is ignored for
chain construction (it stays available via prompt.prompt / .system /
.attachments for any legacy plugin code). Subsequent tool-result
turns inside the chain loop still extend the chain via
_chain_for_tool_results, which reads from the prior response's
prompt.messages + messages.
Six new tests cover: conv.chain(messages=), model.chain(messages=),
messages= authoritative over prompt= kwarg, explicit messages=
replaces conversation history, and the async variants of all of the
above.
667 tests in llm core (661 before + 6 new); llm-anthropic (32) and
llm-gemini (50) still green.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds a dedicated module describing the exact JSON-safe shape returned
by Part.to_dict() / Message.to_dict() / Response.to_dict() and accepted
by the matching from_dict methods. Every consumer that reads or writes
serialized llm data can now import a specific TypedDict and get proper
autocomplete, static type-checking, and schema generation support.
Module: llm/serialization.py (deliberately not "schema" — that name is
taken by the structured-output feature).
TextPartDict, ReasoningPartDict, ToolCallPartDict,
ToolResultPartDict, AttachmentPartDict
— one per Part subclass, each discriminated by a
Literal["<type>"] on the `type` field so pydantic/type-checkers
can narrow cleanly.
PartDict = Union[...]
— the discriminated-union form of all Part dicts.
AttachmentDict — the nested attachment payload (base64 content when
bytes were supplied).
MessageDict — {role, parts: list[PartDict], provider_metadata?}
PromptDict, UsageDict, ResponseDict — full Response.to_dict() shape
including the input chain, options, messages, and audit fields.
TypedDicts use typing_extensions.NotRequired (available for 3.10+ via
a transitive pydantic dep) so Python 3.10 consumers work.
Type annotations on every .to_dict() / .from_dict() method across
parts.py and models.py now reference the specific TypedDict rather
than Dict[str, Any]. Consumers writing
def save_messages(msgs: list[MessageDict]) -> None: ...
get autocomplete on msgs[i]["role"], type-errors on typos, and pydantic
TypeAdapter-based validation works out of the box:
from pydantic import TypeAdapter
from llm.serialization import MessageDict
TypeAdapter(MessageDict).validate_python(incoming) # validate
TypeAdapter(MessageDict).json_schema() # export
Also tidied _response_to_dict to omit usage.details when None so the
serialized UsageDict doesn't carry a null field where pydantic would
reject it during validation.
New test_serialization.py (41 tests):
- required/optional key sets on every TypedDict
- actual .to_dict() output conforms to its TypedDict via TypeAdapter
- PartDict discriminated union accepts all 5 Part variants and
rejects unknown types
- Literal discriminator values are correct
- method annotations point at the right TypedDicts
- JSON round-trip of Response.to_dict() validates
661 total tests pass (620 before + 41 new). llm-anthropic (32) and
llm-gemini (50) still green against the editable llm.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
New tests/test_async_parity.py (18 tests) exercises every new API on
the async path via llm-echo (+ async_mock_model where relevant):
- AsyncResponse.to_dict() captures chain, output, datetime_utc
- AsyncResponse.to_dict() raises before await (guard parity)
- AsyncResponse.from_dict() rehydrates and matches original
- AsyncResponse.from_dict() + reply() continues correctly
- model= override on AsyncResponse.from_dict
- AsyncResponse.from_row fallback (SQLite rehydrate) populates
response.messages from _chunks so llm -c --async preserves the
assistant turn
- load_conversation(async_=True).prompt(...) builds full chain
- AsyncConversation.chain tool-result turn pre-bakes chain
- astream_events matches stream_events for text-only output
- reply chains across 3 async turns
- Full three-turn save→restore→reply loop under async
- reply(messages=[...]) kwarg appends to async chain
- response.messages raises on un-awaited AsyncResponse
- usage round-trips through async to_dict/from_dict
- sync/async structurally identical output for same prompts
Plus one test asserting Echo + EchoAsync are both registered.
The tests all passed on first run — Phase 7 async implementation was
already correct. These pin the invariants against future regressions.
620 tests pass (602 before + 18 new).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Simon Willison