Providers (e.g. Gemini) often emit thought text with trailing newlines.
Concatenated and combined with click.echo's own newline, that produced
several blank lines before the `## Response` heading. rstrip() at render
time tightens the gap to a single blank line; the stored reasoning column
keeps the provider's text verbatim.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- Drop the placeholder messages() declaration on _BaseResponse so
AsyncResponse.messages() (an async coroutine) no longer trips
the mypy override check. text/json/tool_calls already follow
this pattern.
- Remove three unused imports flagged by ruff in test_parts.py.
- Apply pending black reformats across the tree.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds a `reasoning` column to the responses table (migration m022)
populated from concatenated visible-reasoning text in the assembled
message. `llm logs --md` renders it under a `## Reasoning` heading
above the response when present.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sync: response.messages() forces execution if not drained, so callers
no longer have to remember to call .text() first. Async: `await
response.messages()` awaits the force.
Internal sync paths (_response_to_dict, _chain_for_tool_results,
_build_full_chain, Response.reply, AsyncResponse.reply) use a new
private _messages_now() helper that assumes the response is already
drained, so they don't have to await on async responses.
Drops the now-obsolete "accessing .messages on un-awaited
AsyncResponse raises" parity test — that constraint goes away with
the method form.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Zero-arg sugar: when a response made tool calls and tool_results= is
not passed, reply() runs self.execute_tool_calls() and threads the
results into the next turn. Pass tool_results= explicitly to skip
the auto-execute path (e.g. for mutated or synthetic results). Also
forwards self.prompt.tools to the next turn so the model can call
the same tools again, mirroring Conversation.prompt's tools-or-self
rule.
AsyncResponse.reply() is now an awaitable coroutine — `await
response.reply(...)` — so the auto-execute path can `await
self.execute_tool_calls()` internally. This is a non-shipped API
break: existing async-reply callers in the test suite updated.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
ReasoningPart.token_count duplicated info already on response.token_details
(reasoning_tokens), and the side-channel `response._reasoning_token_count`
attribute with its set_usage ordering footgun was the wrong shape. Replaced
with a clean StreamEvent.redacted=True marker that plugins yield like any
other event. The framework hoists redacted reasoning Parts to the start of
the assembled message so UIs render them before content, even though the
opaque count typically arrives at the end of the stream.
Also fix parallel tool calls emitted without tool_call_id (e.g. Gemini):
a fresh tool_call_name now always allocates a new index instead of falling
through to the prior tool-call group, so N parallel calls produce N
distinct ToolCallParts instead of one with concatenated names and args.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Bug: ChainResponse.responses() and AsyncChainResponse.responses()
built the follow-up Prompt (for tool-result turns inside the chain
loop) without propagating system= or system_fragments= from the
initial prompt. Adapters that read prompt.system directly — OpenAI's
Chat, for example, which sends system as its own message — saw an
empty system on every turn after the first, silently losing the
caller's instruction.
Fix: pass system=self.prompt._system and
system_fragments=self.prompt.system_fragments when constructing the
next Prompt. Same change on sync and async paths.
_chain_for_tool_results keeps building messages= from the prior
response's prompt.messages + messages, so adapters that read
prompt.messages (the Phase 7 canonical input) continue to work too —
this fix is specifically for the adapters that still use the
prompt.system legacy field.
Three regression tests (sync, sync+system_fragments, async) pin the
new behavior.
670 tests in llm core; llm-anthropic (32) and llm-gemini (50) still
green.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Parity with prompt(): all four chain() methods —
Conversation.chain, AsyncConversation.chain, _Model.chain,
_AsyncModel.chain — now accept a messages= kwarg and pre-bake the
full chain via _build_full_chain so the first response of the chain
loop satisfies the invariant response.prompt.messages == what was
sent.
Semantics match prompt() exactly: when messages= is passed, it's
authoritative for the first turn. The prompt= kwarg is ignored for
chain construction (it stays available via prompt.prompt / .system /
.attachments for any legacy plugin code). Subsequent tool-result
turns inside the chain loop still extend the chain via
_chain_for_tool_results, which reads from the prior response's
prompt.messages + messages.
Six new tests cover: conv.chain(messages=), model.chain(messages=),
messages= authoritative over prompt= kwarg, explicit messages=
replaces conversation history, and the async variants of all of the
above.
667 tests in llm core (661 before + 6 new); llm-anthropic (32) and
llm-gemini (50) still green.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds a dedicated module describing the exact JSON-safe shape returned
by Part.to_dict() / Message.to_dict() / Response.to_dict() and accepted
by the matching from_dict methods. Every consumer that reads or writes
serialized llm data can now import a specific TypedDict and get proper
autocomplete, static type-checking, and schema generation support.
Module: llm/serialization.py (deliberately not "schema" — that name is
taken by the structured-output feature).
TextPartDict, ReasoningPartDict, ToolCallPartDict,
ToolResultPartDict, AttachmentPartDict
— one per Part subclass, each discriminated by a
Literal["<type>"] on the `type` field so pydantic/type-checkers
can narrow cleanly.
PartDict = Union[...]
— the discriminated-union form of all Part dicts.
AttachmentDict — the nested attachment payload (base64 content when
bytes were supplied).
MessageDict — {role, parts: list[PartDict], provider_metadata?}
PromptDict, UsageDict, ResponseDict — full Response.to_dict() shape
including the input chain, options, messages, and audit fields.
TypedDicts use typing_extensions.NotRequired (available for 3.10+ via
a transitive pydantic dep) so Python 3.10 consumers work.
Type annotations on every .to_dict() / .from_dict() method across
parts.py and models.py now reference the specific TypedDict rather
than Dict[str, Any]. Consumers writing
def save_messages(msgs: list[MessageDict]) -> None: ...
get autocomplete on msgs[i]["role"], type-errors on typos, and pydantic
TypeAdapter-based validation works out of the box:
from pydantic import TypeAdapter
from llm.serialization import MessageDict
TypeAdapter(MessageDict).validate_python(incoming) # validate
TypeAdapter(MessageDict).json_schema() # export
Also tidied _response_to_dict to omit usage.details when None so the
serialized UsageDict doesn't carry a null field where pydantic would
reject it during validation.
New test_serialization.py (41 tests):
- required/optional key sets on every TypedDict
- actual .to_dict() output conforms to its TypedDict via TypeAdapter
- PartDict discriminated union accepts all 5 Part variants and
rejects unknown types
- Literal discriminator values are correct
- method annotations point at the right TypedDicts
- JSON round-trip of Response.to_dict() validates
661 total tests pass (620 before + 41 new). llm-anthropic (32) and
llm-gemini (50) still green against the editable llm.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
New tests/test_async_parity.py (18 tests) exercises every new API on
the async path via llm-echo (+ async_mock_model where relevant):
- AsyncResponse.to_dict() captures chain, output, datetime_utc
- AsyncResponse.to_dict() raises before await (guard parity)
- AsyncResponse.from_dict() rehydrates and matches original
- AsyncResponse.from_dict() + reply() continues correctly
- model= override on AsyncResponse.from_dict
- AsyncResponse.from_row fallback (SQLite rehydrate) populates
response.messages from _chunks so llm -c --async preserves the
assistant turn
- load_conversation(async_=True).prompt(...) builds full chain
- AsyncConversation.chain tool-result turn pre-bakes chain
- astream_events matches stream_events for text-only output
- reply chains across 3 async turns
- Full three-turn save→restore→reply loop under async
- reply(messages=[...]) kwarg appends to async chain
- response.messages raises on un-awaited AsyncResponse
- usage round-trips through async to_dict/from_dict
- sync/async structurally identical output for same prompts
Plus one test asserting Echo + EchoAsync are both registered.
The tests all passed on first run — Phase 7 async implementation was
already correct. These pin the invariants against future regressions.
620 tests pass (602 before + 18 new).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
_build_parts now falls back to synthesizing from self._chunks and
self._tool_calls when self._stream_events is empty. That's the shape
of a Response rehydrated via from_row (SQLite doesn't persist
StreamEvents under Phase 1-7 scope).
Without this, Conversation.prompt's full-chain construction on a
follow-up turn (llm -c, load_conversation().prompt(...)) produced
[user(q1), user(q2)] — dropping the assistant turn entirely — because
prev.messages was []. Now prev.messages yields
[assistant(text + tool calls)] and the chain is correct:
[user(q1), assistant(a1), user(q2)].
Reasoning signatures and structured reasoning parts are still lost
on SQLite rehydrate — that requires Phase 8 (structured parts
persistence) or use of response.to_dict() / from_dict() for
structure-preserving serialization. For the common case (text-only
multi-turn), llm -c works again.
Regression tests pin the fallback + the end-to-end load_conversation
follow-up.
602 tests pass.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Three new Response primitives that make conversation persistence
outside SQLite ergonomic — and a clarified invariant that
response.prompt.messages is always exactly what the model was sent.
response.reply(text, **kwargs) -> Response
Builds next-turn chain = self.prompt.messages + self.messages +
[user(text)] and calls self.model.prompt(messages=chain, ...).
Works from any Response, regardless of origin (conversation,
standalone model.prompt, or from_dict-rehydrated).
response.to_dict() / Response.from_dict(data, *, model=None)
JSON-safe serialization. to_dict captures model_id, input chain
(prompt.messages — full), assembled output (response.messages
including reasoning parts and provider_metadata signatures),
options, and optional audit fields (id, usage, datetime_utc).
from_dict rehydrates a _done=True Response where text() returns
the answer and messages returns the full structured view without
re-running the assembler. Async variants likewise.
Full-chain invariant: response.prompt.messages == what was sent.
- Conversation.prompt / AsyncConversation.prompt now pre-compute
the full chain (last response's prompt.messages + last
response's messages + new user turn, or tool_results for
chain loops) and pass it as messages= into the Prompt.
- Prompt.messages, when _explicit_messages is set, returns that
list verbatim. The previous "combine with prompt= trailing
user" behavior is dropped (was Phase 3 scaffolding, no user
ever wants a partial chain).
- _BaseConversation._build_full_chain is the shared builder.
OpenAI adapter simplified: build_messages now reads only
prompt.messages and ignores conversation.responses. Under the
invariant, prompt.messages already has the full history baked in;
walking conversation would double-emit.
response.messages now short-circuits to _loaded_messages when set
(rehydrated responses don't re-run the assembler).
The persistence pattern becomes:
response = model.prompt("Hi", thinking=True)
response.text()
Path("chat.json").write_text(json.dumps(response.to_dict()))
# Later, any process:
data = json.loads(Path("chat.json").read_text())
response = llm.Response.from_dict(data)
response = response.reply("Follow up")
print(response.text())
Reasoning signatures (Anthropic extended thinking, Gemini
thoughtSignature) round-trip via provider_metadata on the
ReasoningPart / ToolCallPart — so multi-turn extended thinking
works across process boundaries for free.
19 new tests + 2 tightened existing tests; 600 total pass.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Lock in the "application does its own persistence without SQLite"
story with:
- Five integration tests covering: Message.to_dict / from_dict
round-trip, re-inflating messages and continuing a conversation,
tool calls + results round-trip, redacted reasoning Parts
round-trip, and provider_metadata round-trip.
- A new "Structured messages and streaming events" section in
docs/python-api.md walking users through messages=[...],
stream_events(), response.messages, and the JSON round-trip
pattern.
No new code — the machinery landed in Phases 1-3. This phase is
validation + documentation.
580 tests passing overall.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Add a display_stream_events() helper (plus async twin) that writes
text events to stdout and reasoning events to stderr in dim style,
with a newline inserted at reasoning→text transitions so the
assistant's final answer starts on a fresh visual line.
Add -R / --no-reasoning flag to both `llm prompt` and `llm chat` to
suppress the stderr reasoning stream. The sync and async streaming
paths in `llm prompt` and the streaming path in `llm chat` now
consume response.stream_events() / astream_events() through the
helper instead of plain iteration.
Conftest: MockModel and AsyncMockModel now set can_stream = True so
CLI tests exercising event-level behavior actually take the
streaming branch. This matches the fixtures' practical behavior —
they yield chunks, one at a time.
With only the built-in OpenAI plugin upgraded (Phase 4), no model
currently streams live reasoning text (OpenAI's reasoning is
redacted — just a token count). The machinery is ready for
llm-anthropic / llm-gemini upgrades to light up the experience.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Capture usage.completion_tokens_details.reasoning_tokens and store it
on response._reasoning_token_count BEFORE set_usage runs — set_usage
pops top-level keys and then simplify_usage_dict strips zero-valued
entries, both of which would lose the count.
Phase 2's _build_parts() picks up _reasoning_token_count and prepends
a ReasoningPart(redacted=True, token_count=N, text="") to the
assembled output messages. That gives CLI and client code a hook to
render "GPT-5 used N reasoning tokens here" without the model
actually streaming any reasoning text (which OpenAI's reasoning
models don't expose).
Both Chat.execute (sync) and AsyncChat.execute do this.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replace string yields with typed StreamEvent yields in both sync
Chat.execute and async AsyncChat.execute, for both streaming and
non-streaming code paths.
Event shape:
- Text chunks emit StreamEvent(type="text", part_index=0).
Empty-string content (OpenAI's first role=assistant delta) is
now skipped as noise.
- Each tool call gets its own part_index past any text that
preceded it (text is always part_index=0, tool calls start at
1). A new tool call emits tool_call_name with name+id, and each
tool-call-args delta emits tool_call_args with the partial
JSON. Callers see arguments build up live via stream_events().
- Non-streaming path emits one StreamEvent per content block:
tool calls first (each with name + args events), then the final
text if present.
response.add_tool_call() is still invoked for every tool call so
existing code paths (response.tool_calls(), chain execution) keep
working. The new event stream is additive.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Rewrite the OpenAI _Shared.build_messages() to consume prompt.messages
(the canonical structured input) and dispatch per Part subtype into
OpenAI's wire format. Prior-turn input history also comes from
prev_response.prompt.messages; prior-turn output continues to use
the flat text_or_raise() / tool_calls_or_raise() accessors since
those tolerate plugin-specific quirks that _build_parts rejects.
Per-Part translation:
- TextPart → message content (string) or {"type": "text", ...}
entry inside attachment-bearing array content
- AttachmentPart → passed through _attachment() for image/audio/pdf
- ToolCallPart → tool_calls[] entry on an assistant message
(content=null when only tool_calls, no text)
- ToolResultPart → one {"role": "tool", tool_call_id, content} per
result (Message role="tool" can carry several)
System dedup: consecutive identical system prompts emit once. The
legacy prompt=/system=/attachments=/tool_results= path continues to
work unchanged because Prompt.messages synthesizes those into the
same Message list the new code reads.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Add messages= kwarg to model.prompt(), conversation.prompt(), and
their async counterparts. The list flows through to Prompt.__init__
as _explicit_messages.
Prompt.messages is now a property that returns one uniform
list[Message] regardless of which surface the caller used:
- If messages= was passed explicitly, that list is returned
verbatim (with any prompt= string appended as a trailing user
TextPart, matching how system= sugars into a leading system
Message).
- Otherwise synthesized from system=, prompt=, attachments=, and
tool_results=. Plugins read one representation; callers keep all
the existing ergonomic entry points.
No plugin changes yet — existing adapters keep reading the legacy
prompt.prompt / prompt.system / prompt.attachments. Phase 4 is when
the built-in OpenAI plugin switches to reading prompt.messages.
543 tests passing; no regressions.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Teach Response and AsyncResponse to accept str | StreamEvent from
execute(). Plain-str plugins keep working unchanged — their yields are
wrapped as StreamEvent(type="text", chunk=..., part_index=0) internally.
New capabilities:
- response.stream_events() / response.astream_events() yield every
event (text, reasoning, tool_call_*, tool_result) as the model
produces it. Iteration ("for chunk in response") still yields
only text strings.
- response.messages returns the list of assembled Message objects
once the response is done. AsyncResponse.messages raises if not
yet awaited.
- _BaseResponse._build_parts() groups events by part_index into
typed Parts (TextPart, ReasoningPart, ToolCallPart, ToolResultPart).
Mixing families at the same index raises ValueError.
- Opaque reasoning token counts: plugins set
response._reasoning_token_count = N; _build_parts prepends a
ReasoningPart(redacted=True, token_count=N, text="").
- provider_metadata merges across events for the same part
(last non-None wins per top-level namespace key).
ChainResponse.stream_events and AsyncChainResponse.astream_events
pass through from each underlying response.
530 tests passing; no regressions to the existing suite.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Add llm/parts.py with Part, TextPart, ReasoningPart, ToolCallPart,
ToolResultPart, AttachmentPart, Message, and StreamEvent dataclasses.
Parts round-trip through to_dict/from_dict (attachments base64-encoded);
role lives on Message, not on Part. Add user/assistant/system/
tool_message constructor helpers that accept strings, Attachments, Parts,
and nested lists.
Everything is a pure value — identity belongs to storage, which lives
elsewhere. No Response integration yet; that's Phase 2.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* Recorded instance of streaming tool response variant "a".
This is the typical response, where "arguments":"" arrives
first in the stream, followed by "arguments":"{}"
The response data is a real capture from the OpenRouter API,
however some request and header data may be from other test fixtures.
* Recorded instance of streaming tool response variant "b".
This is a streaming response where the first arguments
you get is a fully formed "arguments":"{}"
The response data is a real capture from the OpenRouter API,
however some request and header data may be from other test fixtures.
* Test cases for streaming tool responses.
Note that the replays are marked as "read-only", as they are variants
seen in the wild where the streaming tool call argument fragments
arrive in a specific order.
* Fix streaming tool response variant "b", where "arguments":"{}" is what arrives first.
The previous code erroneously caused the first "arguments" to be duplicated,
by using "+=" even when being initially set.
This went unnoticed as many models stream "arguments":"" first.
When a more fully formed "arguments" fragment arrived first, it was causing
"Error: Extra data: line 1 column 3 (char 2)"
* Recorded instance of streaming tool response variant "c".
This was failing with "Error: unsupported operand type(s) for +=: 'NoneType' and 'str'"
The response data is a real capture from the OpenRouter API,
however some request and header data may be from other test fixtures.
* Test case for streaming tool response variant "c".
* Fix streaming tool response variant "c".
However, I'm not sure why arguments was initially not present or seen as None.
Simon Willison