ReasoningPart.token_count duplicated info already on response.token_details
(reasoning_tokens), and the side-channel `response._reasoning_token_count`
attribute with its set_usage ordering footgun was the wrong shape. Replaced
with a clean StreamEvent.redacted=True marker that plugins yield like any
other event. The framework hoists redacted reasoning Parts to the start of
the assembled message so UIs render them before content, even though the
opaque count typically arrives at the end of the stream.
Also fix parallel tool calls emitted without tool_call_id (e.g. Gemini):
a fresh tool_call_name now always allocates a new index instead of falling
through to the prior tool-call group, so N parallel calls produce N
distinct ToolCallParts instead of one with concatenated names and args.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Three new Response primitives that make conversation persistence
outside SQLite ergonomic — and a clarified invariant that
response.prompt.messages is always exactly what the model was sent.
response.reply(text, **kwargs) -> Response
Builds next-turn chain = self.prompt.messages + self.messages +
[user(text)] and calls self.model.prompt(messages=chain, ...).
Works from any Response, regardless of origin (conversation,
standalone model.prompt, or from_dict-rehydrated).
response.to_dict() / Response.from_dict(data, *, model=None)
JSON-safe serialization. to_dict captures model_id, input chain
(prompt.messages — full), assembled output (response.messages
including reasoning parts and provider_metadata signatures),
options, and optional audit fields (id, usage, datetime_utc).
from_dict rehydrates a _done=True Response where text() returns
the answer and messages returns the full structured view without
re-running the assembler. Async variants likewise.
Full-chain invariant: response.prompt.messages == what was sent.
- Conversation.prompt / AsyncConversation.prompt now pre-compute
the full chain (last response's prompt.messages + last
response's messages + new user turn, or tool_results for
chain loops) and pass it as messages= into the Prompt.
- Prompt.messages, when _explicit_messages is set, returns that
list verbatim. The previous "combine with prompt= trailing
user" behavior is dropped (was Phase 3 scaffolding, no user
ever wants a partial chain).
- _BaseConversation._build_full_chain is the shared builder.
OpenAI adapter simplified: build_messages now reads only
prompt.messages and ignores conversation.responses. Under the
invariant, prompt.messages already has the full history baked in;
walking conversation would double-emit.
response.messages now short-circuits to _loaded_messages when set
(rehydrated responses don't re-run the assembler).
The persistence pattern becomes:
response = model.prompt("Hi", thinking=True)
response.text()
Path("chat.json").write_text(json.dumps(response.to_dict()))
# Later, any process:
data = json.loads(Path("chat.json").read_text())
response = llm.Response.from_dict(data)
response = response.reply("Follow up")
print(response.text())
Reasoning signatures (Anthropic extended thinking, Gemini
thoughtSignature) round-trip via provider_metadata on the
ReasoningPart / ToolCallPart — so multi-turn extended thinking
works across process boundaries for free.
19 new tests + 2 tightened existing tests; 600 total pass.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Capture usage.completion_tokens_details.reasoning_tokens and store it
on response._reasoning_token_count BEFORE set_usage runs — set_usage
pops top-level keys and then simplify_usage_dict strips zero-valued
entries, both of which would lose the count.
Phase 2's _build_parts() picks up _reasoning_token_count and prepends
a ReasoningPart(redacted=True, token_count=N, text="") to the
assembled output messages. That gives CLI and client code a hook to
render "GPT-5 used N reasoning tokens here" without the model
actually streaming any reasoning text (which OpenAI's reasoning
models don't expose).
Both Chat.execute (sync) and AsyncChat.execute do this.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replace string yields with typed StreamEvent yields in both sync
Chat.execute and async AsyncChat.execute, for both streaming and
non-streaming code paths.
Event shape:
- Text chunks emit StreamEvent(type="text", part_index=0).
Empty-string content (OpenAI's first role=assistant delta) is
now skipped as noise.
- Each tool call gets its own part_index past any text that
preceded it (text is always part_index=0, tool calls start at
1). A new tool call emits tool_call_name with name+id, and each
tool-call-args delta emits tool_call_args with the partial
JSON. Callers see arguments build up live via stream_events().
- Non-streaming path emits one StreamEvent per content block:
tool calls first (each with name + args events), then the final
text if present.
response.add_tool_call() is still invoked for every tool call so
existing code paths (response.tool_calls(), chain execution) keep
working. The new event stream is additive.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Rewrite the OpenAI _Shared.build_messages() to consume prompt.messages
(the canonical structured input) and dispatch per Part subtype into
OpenAI's wire format. Prior-turn input history also comes from
prev_response.prompt.messages; prior-turn output continues to use
the flat text_or_raise() / tool_calls_or_raise() accessors since
those tolerate plugin-specific quirks that _build_parts rejects.
Per-Part translation:
- TextPart → message content (string) or {"type": "text", ...}
entry inside attachment-bearing array content
- AttachmentPart → passed through _attachment() for image/audio/pdf
- ToolCallPart → tool_calls[] entry on an assistant message
(content=null when only tool_calls, no text)
- ToolResultPart → one {"role": "tool", tool_call_id, content} per
result (Message role="tool" can carry several)
System dedup: consecutive identical system prompts emit once. The
legacy prompt=/system=/attachments=/tool_results= path continues to
work unchanged because Prompt.messages synthesizes those into the
same Message list the new code reads.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* Add type annotations for OpenAI Chat/AsyncChat/Completion execute methods
* Add type hint for OpenAI _Shared class
* cast(Response) to make mypy happy
Co-authored-by: Simon Willison <swillison@gmail.com>
* Recorded instance of streaming tool response variant "a".
This is the typical response, where "arguments":"" arrives
first in the stream, followed by "arguments":"{}"
The response data is a real capture from the OpenRouter API,
however some request and header data may be from other test fixtures.
* Recorded instance of streaming tool response variant "b".
This is a streaming response where the first arguments
you get is a fully formed "arguments":"{}"
The response data is a real capture from the OpenRouter API,
however some request and header data may be from other test fixtures.
* Test cases for streaming tool responses.
Note that the replays are marked as "read-only", as they are variants
seen in the wild where the streaming tool call argument fragments
arrive in a specific order.
* Fix streaming tool response variant "b", where "arguments":"{}" is what arrives first.
The previous code erroneously caused the first "arguments" to be duplicated,
by using "+=" even when being initially set.
This went unnoticed as many models stream "arguments":"" first.
When a more fully formed "arguments" fragment arrived first, it was causing
"Error: Extra data: line 1 column 3 (char 2)"
* Recorded instance of streaming tool response variant "c".
This was failing with "Error: unsupported operand type(s) for +=: 'NoneType' and 'str'"
The response data is a real capture from the OpenRouter API,
however some request and header data may be from other test fixtures.
* Test case for streaming tool response variant "c".
* Fix streaming tool response variant "c".
However, I'm not sure why arguments was initially not present or seen as None.
Closes#1014
- llm.ToolOutput(output='...', attachments=[...]) for tools to return attachments
- New table: `tool_results_attachments`
- Table is populated when tools return attachments
- llm --tools-debug shows attachments returned by tools
- llm logs shows attachments returned by tools
* Sync models can now call async tools, refs #987
* Test for async tool functions in sync context, refs #987
* Test for asyncio tools, plus test that they run in parallel
* Docs for async tool usage
* llm/default_plugins: add o3 model
This is the newest model released by OpenAI and is available through
the API.
* Ran cog
---------
Co-authored-by: Simon Willison <swillison@gmail.com>
Recently support for structured output was added. But custom
OpenAI-compatible models didn't support the `supports_schema` property
in the config file `extra-openai-models.yaml`.
* Allow "reasoning" for extra-openai-models.yaml
Currently you get an error when trying to use `-o reasoning_effort high` with a model that has been defined in `extra-openai-models.yaml`.
This allows a `reasoning` field.
* Mention reasoning: true in other OpenAI models docs
---------
Co-authored-by: Simon Willison <swillison@gmail.com>
Simon Willison