* PauseChain primitive + chain resume from pending tool calls
Two features that together give chains a first-class suspend/resume
story for human-in-the-loop tools:
llm.PauseChain: raise inside a tool implementation to stop the chain
cleanly. Unlike other exceptions it is not converted into an error
ToolResult - it propagates to the caller with .tool_call (the paused
call) and .tool_results (completed sibling results) attached, and no
provider call is made with a placeholder result. Failure semantics
for concurrent tool execution are now defined: async sibling tasks
always run to completion before a pause or hook exception propagates
(gather with return_exceptions, raised after collection), so nothing
is orphaned mid-flight; sync execution stops at the paused call,
leaving later calls unstarted so they can safely run on resume.
Chain resume: chain(messages=history, tools=...) now detects a
history ending in an assistant message with unresolved tool calls -
e.g. one persisted when a previous run paused or crashed - executes
those calls through the normal before_call/after_call machinery
(skipping any that already have results), then sends the results to
the model as a standard tool-result turn. A resumed call may pause
again, enabling multi-question flows. Histories where a user or
assistant message follows the calls are left alone. Also adds
execute_tool_calls(tool_calls_list=) for executing an explicit list.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Guarantee every tool call has a unique tool_call_id
add_tool_call() now synthesizes a unique tc_-prefixed id (monotonic
ULID) whenever the provider did not supply one. Previously consumers
correlating tool calls with results - or keying external state on a
specific invocation - had to invent fallback matching schemes for
id-less providers, and test models like llm-echo exercised different
code paths than production providers.
Provider-supplied ids are preserved untouched, and responses
rehydrated from the logs database keep their stored ids (synthesis
only happens at add_tool_call time). Existing tests that asserted
tool_call_id None now normalize or mask the synthesized ids.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Tool implementations can receive the ToolCall via llm_tool_call parameter
Tool functions (sync or async, including Toolbox methods) that declare
a parameter named llm_tool_call are now passed the llm.ToolCall object
for the current invocation. The parameter is reserved: it is excluded
from the input schema exposed to the model and is only injected when
declared explicitly - a **kwargs catch-all does not receive it.
This lets tool implementations key external state against the unique
tool_call_id, e.g. for human-in-the-loop approval flows that need to
resume a specific tool call after the answer arrives.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Ran Black
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Accepts model options as an explicit dict alongside the existing
**kwargs form. The kwargs form continues to work unchanged for
backwards compatibility but is no longer documented. Mixing the two
forms with overlapping keys raises TypeError.
Applies to Model.prompt, Conversation.prompt, Response.reply and
their async equivalents. .chain() already used this pattern.
Co-authored-by: Claude <noreply@anthropic.com>
Sync: response.messages() forces execution if not drained, so callers
no longer have to remember to call .text() first. Async: `await
response.messages()` awaits the force.
Internal sync paths (_response_to_dict, _chain_for_tool_results,
_build_full_chain, Response.reply, AsyncResponse.reply) use a new
private _messages_now() helper that assumes the response is already
drained, so they don't have to await on async responses.
Drops the now-obsolete "accessing .messages on un-awaited
AsyncResponse raises" parity test — that constraint goes away with
the method form.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Zero-arg sugar: when a response made tool calls and tool_results= is
not passed, reply() runs self.execute_tool_calls() and threads the
results into the next turn. Pass tool_results= explicitly to skip
the auto-execute path (e.g. for mutated or synthetic results). Also
forwards self.prompt.tools to the next turn so the model can call
the same tools again, mirroring Conversation.prompt's tools-or-self
rule.
AsyncResponse.reply() is now an awaitable coroutine — `await
response.reply(...)` — so the auto-execute path can `await
self.execute_tool_calls()` internally. This is a non-shipped API
break: existing async-reply callers in the test suite updated.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Lock in the "application does its own persistence without SQLite"
story with:
- Five integration tests covering: Message.to_dict / from_dict
round-trip, re-inflating messages and continuing a conversation,
tool calls + results round-trip, redacted reasoning Parts
round-trip, and provider_metadata round-trip.
- A new "Structured messages and streaming events" section in
docs/python-api.md walking users through messages=[...],
stream_events(), response.messages, and the JSON round-trip
pattern.
No new code — the machinery landed in Phases 1-3. This phase is
validation + documentation.
580 tests passing overall.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Closes#1014
- llm.ToolOutput(output='...', attachments=[...]) for tools to return attachments
- New table: `tool_results_attachments`
- Table is populated when tools return attachments
- llm --tools-debug shows attachments returned by tools
- llm logs shows attachments returned by tools
* Sync models can now call async tools, refs #987
* Test for async tool functions in sync context, refs #987
* Test for asyncio tools, plus test that they run in parallel
* Docs for async tool usage
* WIP fragments: schema plus reading but not yet writing, refs #617
* Unique index on fragments.alias, refs #617
* Fragments are now persisted, added basic CLI commands
* Fragment aliases work now, refs #617
* Improved help for -f/--fragment
* Support fragment hash as well
* Documentation for fragments
* Better non-JSON display of llm fragments list
* llm fragments -q search option
* _truncate_string is now truncate_string
* Use condense_json to avoid duplicate data in JSON in DB, refs #617
* Follow up to 3 redirects for fragments
* Python API docs for fragments= and system_fragments=
* Fragment aliases cannot contain a : - this is to ensure we can add custom fragment loaders later on, refs https://github.com/simonw/llm/pull/859#issuecomment-2761534692
* Use template fragments when running prompts
* llm fragments show command plus llm fragments group tests
* Tests for fragments family of commands
* Test for --save with fragments
* Add fragments tables to docs/logging.md
* Slightly better llm fragments --help
* Handle fragments in past conversations correctly
* Hint at llm prompt --help in llm --help, closes#868
* llm logs -f filter plus show fragments in llm logs --json
* Include prompt and system fragments in llm logs -s
* llm logs markdown fragment output and tests, refs #617
Refs #776
* Implemented new llm prompt --schema and model.prompt(schema=)
* Log schema to responses.schema_id and schemas table
* Include schema in llm logs Markdown output
* Test for schema=pydantic_model
* Initial --schema CLI documentation
* Python docs for schema=
* Advanced plugin docs on schemas
* New KeyModel and AsyncKeyModel classes for models that taken keys - closes#744
* llm prompt --key now uses new mechanism, including for async
* use new key mechanism in llm chat command
* Python API tests for llm.KeyModel and llm.AsyncKeyModel
* Python API docs for for prompt(... key="")
* Mention await model.prompt() takes other parameters, reorg sections
* Better title for the model tutorial
* Docs on writing model plugins that take a key
- https://github.com/simonw/llm/issues/507#issuecomment-2458639308
* register_model is now async aware
Refs https://github.com/simonw/llm/issues/507#issuecomment-2458658134
* Refactor Chat and AsyncChat to use _Shared base class
Refs https://github.com/simonw/llm/issues/507#issuecomment-2458692338
* fixed function name
* Fix for infinite loop
* Applied Black
* Ran cog
* Applied Black
* Add Response.from_row() classmethod back again
It does not matter that this is a blocking call, since it is a classmethod
* Made mypy happy with llm/models.py
* mypy fixes for openai_models.py
I am unhappy with this, had to duplicate some code.
* First test for AsyncModel
* Still have not quite got this working
* Fix for not loading plugins during tests, refs #626
* audio/wav not audio/wave, refs #603
* Black and mypy and ruff all happy
* Refactor to avoid generics
* Removed obsolete response() method
* Support text = await async_mock_model.prompt("hello")
* Initial docs for llm.get_async_model() and await model.prompt()
Refs #507
* Initial async model plugin creation docs
* duration_ms ANY to pass test
* llm models --async option
Refs https://github.com/simonw/llm/pull/613#issuecomment-2474724406
* Removed obsolete TypeVars
* Expanded register_models() docs for async
* await model.prompt() now returns AsyncResponse
Refs https://github.com/simonw/llm/pull/613#issuecomment-2475157822
---------
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Simon Willison