When building the `input` list for the OpenAI Responses API from prior
conversation turns, an assistant text-only turn was being serialized as:
{"role": "assistant",
"content": [{"type": "output_text", "text": "..."}]}
The openai-python SDK's EasyInputMessage shape uses a plain string for
this case, matching what a direct OpenAI Responses call would send. Use
the same shape so our history matches the SDK exactly, and add tests
covering both _build_responses_input and a two-turn response.reply()
flow.
When the Responses API returns a reasoning item alongside function
calls, capture its opaque id + encrypted_content as provider_metadata
on the resulting ReasoningPart. _build_responses_input already echoed
that metadata back as a reasoning input item on the next turn - now
the output side actually populates it.
This preserves the model's hidden chain of thought across the tool
round-trip. Without it, GPT-5-class models silently lose ~3% on
SWE-bench (per OpenAI) when used with tools.
Adds a dedicated VCR test that asserts the encrypted_content captured
on the first turn appears verbatim in the second turn's outgoing
request body.
Adds Responses and AsyncResponses classes that drive the OpenAI
/v1/responses endpoint. The existing Chat / AsyncChat classes are
unchanged because other plugins import them.
gpt-5.5 (and gpt-5.5-2026-04-23) is now registered against Responses
by default. Pass `-o chat_completions 1` to fall back to the older
/v1/chat/completions code path.
This is feature parity with the Chat path (text, tools, streaming,
schema, reasoning_effort, verbosity, attachments, system prompts).
Interleaved reasoning across tool round-trips is not exercised yet -
encrypted reasoning items are accepted on the input side, but the
plugin doesn't yet stash them on outgoing ReasoningParts.
ReasoningPart.token_count duplicated info already on response.token_details
(reasoning_tokens), and the side-channel `response._reasoning_token_count`
attribute with its set_usage ordering footgun was the wrong shape. Replaced
with a clean StreamEvent.redacted=True marker that plugins yield like any
other event. The framework hoists redacted reasoning Parts to the start of
the assembled message so UIs render them before content, even though the
opaque count typically arrives at the end of the stream.
Also fix parallel tool calls emitted without tool_call_id (e.g. Gemini):
a fresh tool_call_name now always allocates a new index instead of falling
through to the prior tool-call group, so N parallel calls produce N
distinct ToolCallParts instead of one with concatenated names and args.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Three new Response primitives that make conversation persistence
outside SQLite ergonomic — and a clarified invariant that
response.prompt.messages is always exactly what the model was sent.
response.reply(text, **kwargs) -> Response
Builds next-turn chain = self.prompt.messages + self.messages +
[user(text)] and calls self.model.prompt(messages=chain, ...).
Works from any Response, regardless of origin (conversation,
standalone model.prompt, or from_dict-rehydrated).
response.to_dict() / Response.from_dict(data, *, model=None)
JSON-safe serialization. to_dict captures model_id, input chain
(prompt.messages — full), assembled output (response.messages
including reasoning parts and provider_metadata signatures),
options, and optional audit fields (id, usage, datetime_utc).
from_dict rehydrates a _done=True Response where text() returns
the answer and messages returns the full structured view without
re-running the assembler. Async variants likewise.
Full-chain invariant: response.prompt.messages == what was sent.
- Conversation.prompt / AsyncConversation.prompt now pre-compute
the full chain (last response's prompt.messages + last
response's messages + new user turn, or tool_results for
chain loops) and pass it as messages= into the Prompt.
- Prompt.messages, when _explicit_messages is set, returns that
list verbatim. The previous "combine with prompt= trailing
user" behavior is dropped (was Phase 3 scaffolding, no user
ever wants a partial chain).
- _BaseConversation._build_full_chain is the shared builder.
OpenAI adapter simplified: build_messages now reads only
prompt.messages and ignores conversation.responses. Under the
invariant, prompt.messages already has the full history baked in;
walking conversation would double-emit.
response.messages now short-circuits to _loaded_messages when set
(rehydrated responses don't re-run the assembler).
The persistence pattern becomes:
response = model.prompt("Hi", thinking=True)
response.text()
Path("chat.json").write_text(json.dumps(response.to_dict()))
# Later, any process:
data = json.loads(Path("chat.json").read_text())
response = llm.Response.from_dict(data)
response = response.reply("Follow up")
print(response.text())
Reasoning signatures (Anthropic extended thinking, Gemini
thoughtSignature) round-trip via provider_metadata on the
ReasoningPart / ToolCallPart — so multi-turn extended thinking
works across process boundaries for free.
19 new tests + 2 tightened existing tests; 600 total pass.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Capture usage.completion_tokens_details.reasoning_tokens and store it
on response._reasoning_token_count BEFORE set_usage runs — set_usage
pops top-level keys and then simplify_usage_dict strips zero-valued
entries, both of which would lose the count.
Phase 2's _build_parts() picks up _reasoning_token_count and prepends
a ReasoningPart(redacted=True, token_count=N, text="") to the
assembled output messages. That gives CLI and client code a hook to
render "GPT-5 used N reasoning tokens here" without the model
actually streaming any reasoning text (which OpenAI's reasoning
models don't expose).
Both Chat.execute (sync) and AsyncChat.execute do this.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replace string yields with typed StreamEvent yields in both sync
Chat.execute and async AsyncChat.execute, for both streaming and
non-streaming code paths.
Event shape:
- Text chunks emit StreamEvent(type="text", part_index=0).
Empty-string content (OpenAI's first role=assistant delta) is
now skipped as noise.
- Each tool call gets its own part_index past any text that
preceded it (text is always part_index=0, tool calls start at
1). A new tool call emits tool_call_name with name+id, and each
tool-call-args delta emits tool_call_args with the partial
JSON. Callers see arguments build up live via stream_events().
- Non-streaming path emits one StreamEvent per content block:
tool calls first (each with name + args events), then the final
text if present.
response.add_tool_call() is still invoked for every tool call so
existing code paths (response.tool_calls(), chain execution) keep
working. The new event stream is additive.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Rewrite the OpenAI _Shared.build_messages() to consume prompt.messages
(the canonical structured input) and dispatch per Part subtype into
OpenAI's wire format. Prior-turn input history also comes from
prev_response.prompt.messages; prior-turn output continues to use
the flat text_or_raise() / tool_calls_or_raise() accessors since
those tolerate plugin-specific quirks that _build_parts rejects.
Per-Part translation:
- TextPart → message content (string) or {"type": "text", ...}
entry inside attachment-bearing array content
- AttachmentPart → passed through _attachment() for image/audio/pdf
- ToolCallPart → tool_calls[] entry on an assistant message
(content=null when only tool_calls, no text)
- ToolResultPart → one {"role": "tool", tool_call_id, content} per
result (Message role="tool" can carry several)
System dedup: consecutive identical system prompts emit once. The
legacy prompt=/system=/attachments=/tool_results= path continues to
work unchanged because Prompt.messages synthesizes those into the
same Message list the new code reads.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* Add type annotations for OpenAI Chat/AsyncChat/Completion execute methods
* Add type hint for OpenAI _Shared class
* cast(Response) to make mypy happy
Co-authored-by: Simon Willison <swillison@gmail.com>
* Recorded instance of streaming tool response variant "a".
This is the typical response, where "arguments":"" arrives
first in the stream, followed by "arguments":"{}"
The response data is a real capture from the OpenRouter API,
however some request and header data may be from other test fixtures.
* Recorded instance of streaming tool response variant "b".
This is a streaming response where the first arguments
you get is a fully formed "arguments":"{}"
The response data is a real capture from the OpenRouter API,
however some request and header data may be from other test fixtures.
* Test cases for streaming tool responses.
Note that the replays are marked as "read-only", as they are variants
seen in the wild where the streaming tool call argument fragments
arrive in a specific order.
* Fix streaming tool response variant "b", where "arguments":"{}" is what arrives first.
The previous code erroneously caused the first "arguments" to be duplicated,
by using "+=" even when being initially set.
This went unnoticed as many models stream "arguments":"" first.
When a more fully formed "arguments" fragment arrived first, it was causing
"Error: Extra data: line 1 column 3 (char 2)"
* Recorded instance of streaming tool response variant "c".
This was failing with "Error: unsupported operand type(s) for +=: 'NoneType' and 'str'"
The response data is a real capture from the OpenRouter API,
however some request and header data may be from other test fixtures.
* Test case for streaming tool response variant "c".
* Fix streaming tool response variant "c".
However, I'm not sure why arguments was initially not present or seen as None.
Closes#1014
- llm.ToolOutput(output='...', attachments=[...]) for tools to return attachments
- New table: `tool_results_attachments`
- Table is populated when tools return attachments
- llm --tools-debug shows attachments returned by tools
- llm logs shows attachments returned by tools
* Sync models can now call async tools, refs #987
* Test for async tool functions in sync context, refs #987
* Test for asyncio tools, plus test that they run in parallel
* Docs for async tool usage
Simon Willison