Substantially expanded docs/plugins/advanced-model-plugins.md with
the plugin-author guide to the new machinery. Distilled from the
actual llm-anthropic and llm-gemini implementations so plugin
authors have a recipe that mirrors what real providers do.
New / updated sections (doc grew 317 → 663 lines):
- "Attachments from previous conversations" trimmed to a pointer at
prompt.messages — the old pattern of walking conversation.responses
is replaced by the canonical chain view.
- "Structured messages and streaming events"
- StreamEvent contract, backward compat for plain-str plugins
- Full field reference (type / chunk / part_index / tool_call_id /
provider_metadata / server_executed / tool_name)
- part_index allocation rules with two worked examples:
block-keyed (Anthropic-style content_block_start events) and
kind-tracking (Gemini-style chunk-per-part)
- Reasoning: streamed text + opaque _reasoning_token_count, with
the OpenAI-specific gotcha about reading reasoning_tokens BEFORE
set_usage mutates the dict
- Tool calls — tool_call_name + tool_call_args pattern, reminder
that response.add_tool_call() is separately required for
chain-execution
- Server-side tools — server_executed=True on events, raw payload
in provider_metadata for round-trip, post-stream emission for
providers that don't stream tool-result contents
- Opaque provider_metadata — Anthropic signature, Gemini
thoughtSignature, OpenAI encrypted_content — with namespacing
guidance
- Non-streaming path — one event per content block
- "Consuming prompt.messages in build_messages"
- The invariant: prompt.messages is always the full chain; don't
walk conversation.responses (would double-emit)
- Worked build_messages example that dispatches per Part subtype
- Role mapping across OpenAI / Anthropic / Gemini conventions
- Role-alternation merging
- "Restoring opaque metadata on subsequent requests"
- How to read provider_metadata off prior-turn Parts and fold the
signatures back into the outgoing request body
670 tests still green.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* Docs and shape of register_fragment_loaders hook, refs #863
* Update docs for fragment loaders returning a list of FragmentString
* Support multiple fragments with same content, closes#888
* Call the pm.hook.register_fragment_loaders hook
* Test for register_fragment_loaders hook
* Rename FragmentString to Fragment
Closes#863
Refs #776
* Implemented new llm prompt --schema and model.prompt(schema=)
* Log schema to responses.schema_id and schemas table
* Include schema in llm logs Markdown output
* Test for schema=pydantic_model
* Initial --schema CLI documentation
* Python docs for schema=
* Advanced plugin docs on schemas
* New KeyModel and AsyncKeyModel classes for models that taken keys - closes#744
* llm prompt --key now uses new mechanism, including for async
* use new key mechanism in llm chat command
* Python API tests for llm.KeyModel and llm.AsyncKeyModel
* Python API docs for for prompt(... key="")
* Mention await model.prompt() takes other parameters, reorg sections
* Better title for the model tutorial
* Docs on writing model plugins that take a key
- https://github.com/simonw/llm/issues/507#issuecomment-2458639308
* register_model is now async aware
Refs https://github.com/simonw/llm/issues/507#issuecomment-2458658134
* Refactor Chat and AsyncChat to use _Shared base class
Refs https://github.com/simonw/llm/issues/507#issuecomment-2458692338
* fixed function name
* Fix for infinite loop
* Applied Black
* Ran cog
* Applied Black
* Add Response.from_row() classmethod back again
It does not matter that this is a blocking call, since it is a classmethod
* Made mypy happy with llm/models.py
* mypy fixes for openai_models.py
I am unhappy with this, had to duplicate some code.
* First test for AsyncModel
* Still have not quite got this working
* Fix for not loading plugins during tests, refs #626
* audio/wav not audio/wave, refs #603
* Black and mypy and ruff all happy
* Refactor to avoid generics
* Removed obsolete response() method
* Support text = await async_mock_model.prompt("hello")
* Initial docs for llm.get_async_model() and await model.prompt()
Refs #507
* Initial async model plugin creation docs
* duration_ms ANY to pass test
* llm models --async option
Refs https://github.com/simonw/llm/pull/613#issuecomment-2474724406
* Removed obsolete TypeVars
* Expanded register_models() docs for async
* await model.prompt() now returns AsyncResponse
Refs https://github.com/simonw/llm/pull/613#issuecomment-2475157822
---------
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Simon Willison