- Drop the placeholder messages() declaration on _BaseResponse so
AsyncResponse.messages() (an async coroutine) no longer trips
the mypy override check. text/json/tool_calls already follow
this pattern.
- Remove three unused imports flagged by ruff in test_parts.py.
- Apply pending black reformats across the tree.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds a `reasoning` column to the responses table (migration m022)
populated from concatenated visible-reasoning text in the assembled
message. `llm logs --md` renders it under a `## Reasoning` heading
above the response when present.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sync: response.messages() forces execution if not drained, so callers
no longer have to remember to call .text() first. Async: `await
response.messages()` awaits the force.
Internal sync paths (_response_to_dict, _chain_for_tool_results,
_build_full_chain, Response.reply, AsyncResponse.reply) use a new
private _messages_now() helper that assumes the response is already
drained, so they don't have to await on async responses.
Drops the now-obsolete "accessing .messages on un-awaited
AsyncResponse raises" parity test — that constraint goes away with
the method form.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Zero-arg sugar: when a response made tool calls and tool_results= is
not passed, reply() runs self.execute_tool_calls() and threads the
results into the next turn. Pass tool_results= explicitly to skip
the auto-execute path (e.g. for mutated or synthetic results). Also
forwards self.prompt.tools to the next turn so the model can call
the same tools again, mirroring Conversation.prompt's tools-or-self
rule.
AsyncResponse.reply() is now an awaitable coroutine — `await
response.reply(...)` — so the auto-execute path can `await
self.execute_tool_calls()` internally. This is a non-shipped API
break: existing async-reply callers in the test suite updated.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
ReasoningPart.token_count duplicated info already on response.token_details
(reasoning_tokens), and the side-channel `response._reasoning_token_count`
attribute with its set_usage ordering footgun was the wrong shape. Replaced
with a clean StreamEvent.redacted=True marker that plugins yield like any
other event. The framework hoists redacted reasoning Parts to the start of
the assembled message so UIs render them before content, even though the
opaque count typically arrives at the end of the stream.
Also fix parallel tool calls emitted without tool_call_id (e.g. Gemini):
a fresh tool_call_name now always allocates a new index instead of falling
through to the prior tool-call group, so N parallel calls produce N
distinct ToolCallParts instead of one with concatenated names and args.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Bug: ChainResponse.responses() and AsyncChainResponse.responses()
built the follow-up Prompt (for tool-result turns inside the chain
loop) without propagating system= or system_fragments= from the
initial prompt. Adapters that read prompt.system directly — OpenAI's
Chat, for example, which sends system as its own message — saw an
empty system on every turn after the first, silently losing the
caller's instruction.
Fix: pass system=self.prompt._system and
system_fragments=self.prompt.system_fragments when constructing the
next Prompt. Same change on sync and async paths.
_chain_for_tool_results keeps building messages= from the prior
response's prompt.messages + messages, so adapters that read
prompt.messages (the Phase 7 canonical input) continue to work too —
this fix is specifically for the adapters that still use the
prompt.system legacy field.
Three regression tests (sync, sync+system_fragments, async) pin the
new behavior.
670 tests in llm core; llm-anthropic (32) and llm-gemini (50) still
green.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Parity with prompt(): all four chain() methods —
Conversation.chain, AsyncConversation.chain, _Model.chain,
_AsyncModel.chain — now accept a messages= kwarg and pre-bake the
full chain via _build_full_chain so the first response of the chain
loop satisfies the invariant response.prompt.messages == what was
sent.
Semantics match prompt() exactly: when messages= is passed, it's
authoritative for the first turn. The prompt= kwarg is ignored for
chain construction (it stays available via prompt.prompt / .system /
.attachments for any legacy plugin code). Subsequent tool-result
turns inside the chain loop still extend the chain via
_chain_for_tool_results, which reads from the prior response's
prompt.messages + messages.
Six new tests cover: conv.chain(messages=), model.chain(messages=),
messages= authoritative over prompt= kwarg, explicit messages=
replaces conversation history, and the async variants of all of the
above.
667 tests in llm core (661 before + 6 new); llm-anthropic (32) and
llm-gemini (50) still green.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds a dedicated module describing the exact JSON-safe shape returned
by Part.to_dict() / Message.to_dict() / Response.to_dict() and accepted
by the matching from_dict methods. Every consumer that reads or writes
serialized llm data can now import a specific TypedDict and get proper
autocomplete, static type-checking, and schema generation support.
Module: llm/serialization.py (deliberately not "schema" — that name is
taken by the structured-output feature).
TextPartDict, ReasoningPartDict, ToolCallPartDict,
ToolResultPartDict, AttachmentPartDict
— one per Part subclass, each discriminated by a
Literal["<type>"] on the `type` field so pydantic/type-checkers
can narrow cleanly.
PartDict = Union[...]
— the discriminated-union form of all Part dicts.
AttachmentDict — the nested attachment payload (base64 content when
bytes were supplied).
MessageDict — {role, parts: list[PartDict], provider_metadata?}
PromptDict, UsageDict, ResponseDict — full Response.to_dict() shape
including the input chain, options, messages, and audit fields.
TypedDicts use typing_extensions.NotRequired (available for 3.10+ via
a transitive pydantic dep) so Python 3.10 consumers work.
Type annotations on every .to_dict() / .from_dict() method across
parts.py and models.py now reference the specific TypedDict rather
than Dict[str, Any]. Consumers writing
def save_messages(msgs: list[MessageDict]) -> None: ...
get autocomplete on msgs[i]["role"], type-errors on typos, and pydantic
TypeAdapter-based validation works out of the box:
from pydantic import TypeAdapter
from llm.serialization import MessageDict
TypeAdapter(MessageDict).validate_python(incoming) # validate
TypeAdapter(MessageDict).json_schema() # export
Also tidied _response_to_dict to omit usage.details when None so the
serialized UsageDict doesn't carry a null field where pydantic would
reject it during validation.
New test_serialization.py (41 tests):
- required/optional key sets on every TypedDict
- actual .to_dict() output conforms to its TypedDict via TypeAdapter
- PartDict discriminated union accepts all 5 Part variants and
rejects unknown types
- Literal discriminator values are correct
- method annotations point at the right TypedDicts
- JSON round-trip of Response.to_dict() validates
661 total tests pass (620 before + 41 new). llm-anthropic (32) and
llm-gemini (50) still green against the editable llm.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
_build_parts now falls back to synthesizing from self._chunks and
self._tool_calls when self._stream_events is empty. That's the shape
of a Response rehydrated via from_row (SQLite doesn't persist
StreamEvents under Phase 1-7 scope).
Without this, Conversation.prompt's full-chain construction on a
follow-up turn (llm -c, load_conversation().prompt(...)) produced
[user(q1), user(q2)] — dropping the assistant turn entirely — because
prev.messages was []. Now prev.messages yields
[assistant(text + tool calls)] and the chain is correct:
[user(q1), assistant(a1), user(q2)].
Reasoning signatures and structured reasoning parts are still lost
on SQLite rehydrate — that requires Phase 8 (structured parts
persistence) or use of response.to_dict() / from_dict() for
structure-preserving serialization. For the common case (text-only
multi-turn), llm -c works again.
Regression tests pin the fallback + the end-to-end load_conversation
follow-up.
602 tests pass.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
_chain_for_tool_results() builds the full chain for a tool-result
turn inside a chain loop: prior response's full input + output +
a tool-role message carrying the new results + any attachments.
ChainResponse.responses() and AsyncChainResponse.responses() now
pass that chain as messages= when constructing the next Response.
Why: under the Phase 7 invariant, response.prompt.messages is what
the model sees. Without this, the tool-result turn's prompt.messages
would only synthesize a single tool-role message from the legacy
tool_results= kwarg — stripping reasoning signatures and tool-call
thoughtSignatures from the prior assistant turn. That breaks
multi-turn Gemini 3 tool loops (thoughtSignature must be echoed)
and Claude extended thinking inside chains.
All 600 tests pass.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Three new Response primitives that make conversation persistence
outside SQLite ergonomic — and a clarified invariant that
response.prompt.messages is always exactly what the model was sent.
response.reply(text, **kwargs) -> Response
Builds next-turn chain = self.prompt.messages + self.messages +
[user(text)] and calls self.model.prompt(messages=chain, ...).
Works from any Response, regardless of origin (conversation,
standalone model.prompt, or from_dict-rehydrated).
response.to_dict() / Response.from_dict(data, *, model=None)
JSON-safe serialization. to_dict captures model_id, input chain
(prompt.messages — full), assembled output (response.messages
including reasoning parts and provider_metadata signatures),
options, and optional audit fields (id, usage, datetime_utc).
from_dict rehydrates a _done=True Response where text() returns
the answer and messages returns the full structured view without
re-running the assembler. Async variants likewise.
Full-chain invariant: response.prompt.messages == what was sent.
- Conversation.prompt / AsyncConversation.prompt now pre-compute
the full chain (last response's prompt.messages + last
response's messages + new user turn, or tool_results for
chain loops) and pass it as messages= into the Prompt.
- Prompt.messages, when _explicit_messages is set, returns that
list verbatim. The previous "combine with prompt= trailing
user" behavior is dropped (was Phase 3 scaffolding, no user
ever wants a partial chain).
- _BaseConversation._build_full_chain is the shared builder.
OpenAI adapter simplified: build_messages now reads only
prompt.messages and ignores conversation.responses. Under the
invariant, prompt.messages already has the full history baked in;
walking conversation would double-emit.
response.messages now short-circuits to _loaded_messages when set
(rehydrated responses don't re-run the assembler).
The persistence pattern becomes:
response = model.prompt("Hi", thinking=True)
response.text()
Path("chat.json").write_text(json.dumps(response.to_dict()))
# Later, any process:
data = json.loads(Path("chat.json").read_text())
response = llm.Response.from_dict(data)
response = response.reply("Follow up")
print(response.text())
Reasoning signatures (Anthropic extended thinking, Gemini
thoughtSignature) round-trip via provider_metadata on the
ReasoningPart / ToolCallPart — so multi-turn extended thinking
works across process boundaries for free.
19 new tests + 2 tightened existing tests; 600 total pass.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Add messages= kwarg to model.prompt(), conversation.prompt(), and
their async counterparts. The list flows through to Prompt.__init__
as _explicit_messages.
Prompt.messages is now a property that returns one uniform
list[Message] regardless of which surface the caller used:
- If messages= was passed explicitly, that list is returned
verbatim (with any prompt= string appended as a trailing user
TextPart, matching how system= sugars into a leading system
Message).
- Otherwise synthesized from system=, prompt=, attachments=, and
tool_results=. Plugins read one representation; callers keep all
the existing ergonomic entry points.
No plugin changes yet — existing adapters keep reading the legacy
prompt.prompt / prompt.system / prompt.attachments. Phase 4 is when
the built-in OpenAI plugin switches to reading prompt.messages.
543 tests passing; no regressions.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Teach Response and AsyncResponse to accept str | StreamEvent from
execute(). Plain-str plugins keep working unchanged — their yields are
wrapped as StreamEvent(type="text", chunk=..., part_index=0) internally.
New capabilities:
- response.stream_events() / response.astream_events() yield every
event (text, reasoning, tool_call_*, tool_result) as the model
produces it. Iteration ("for chunk in response") still yields
only text strings.
- response.messages returns the list of assembled Message objects
once the response is done. AsyncResponse.messages raises if not
yet awaited.
- _BaseResponse._build_parts() groups events by part_index into
typed Parts (TextPart, ReasoningPart, ToolCallPart, ToolResultPart).
Mixing families at the same index raises ValueError.
- Opaque reasoning token counts: plugins set
response._reasoning_token_count = N; _build_parts prepends a
ReasoningPart(redacted=True, token_count=N, text="").
- provider_metadata merges across events for the same part
(last non-None wins per top-level namespace key).
ChainResponse.stream_events and AsyncChainResponse.astream_events
pass through from each underlying response.
530 tests passing; no regressions to the existing suite.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Closes#1014
- llm.ToolOutput(output='...', attachments=[...]) for tools to return attachments
- New table: `tool_results_attachments`
- Table is populated when tools return attachments
- llm --tools-debug shows attachments returned by tools
- llm logs shows attachments returned by tools
* Sync models can now call async tools, refs #987
* Test for async tool functions in sync context, refs #987
* Test for asyncio tools, plus test that they run in parallel
* Docs for async tool usage
Simon Willison