elizaos--eliza
426e9eeabd
Voice Workbench / headless workbench (mocked backends) (push) Has been cancelled
Voice Workbench / real acoustic lane (nightly, provisioned only) (push) Has been cancelled
ci / test (push) Has been cancelled
ci / lint-and-format (push) Has been cancelled
ci / build (push) Has been cancelled
ci / dev-startup (push) Has been cancelled
gitleaks / gitleaks (push) Has been cancelled
Markdown Links / Relative Markdown Links (push) Has been cancelled
Quality (Extended) / Homepage Build (PR smoke) (push) Has been cancelled
Quality (Extended) / Comment-only diff guard (push) Has been cancelled
Quality (Extended) / Format + Type Safety Ratchet (push) Has been cancelled
Quality (Extended) / Develop Gate (secret scan + UI determinism) (push) Has been cancelled
Quality (Extended) / Develop Gate (lint) (push) Has been cancelled
Chat shell gestures / Chat shell gesture + parity e2e (push) Has been cancelled
Cloud Gateway Discord / Test (push) Has been cancelled
Benchmark Bridge Tests / benchmark (bunx @biomejs/biome check packages/lifeops-bench/src, benchmark-lint) (push) Has been cancelled
Benchmark Bridge Tests / benchmark (bunx vitest run --config packages/lifeops-bench/vitest.config.ts --root packages/lifeops-bench --passWithNoTests, benchmark-tests) (push) Has been cancelled
Build Agent Image / build-and-push (push) Has been cancelled
Dev Smoke / bun run dev onboarding chat (push) Has been cancelled
Dev Smoke / Vite HMR dependency-level smoke (push) Has been cancelled
Electrobun Submodule Guard / electrobun gitlink is fetchable (push) Has been cancelled
Publish @elizaos/example-code / check_npm (push) Has been cancelled
Publish @elizaos/example-code / publish_npm (push) Has been cancelled
Publish @elizaos/plugin-elizacloud / verify_version (push) Has been cancelled
Publish @elizaos/plugin-elizacloud / publish_npm (push) Has been cancelled
Sandbox Live Smoke / Sandbox live smoke (push) Has been cancelled
Snap Build & Test / Build Snap (amd64) (push) Has been cancelled
Snap Build & Test / Build Snap (arm64) (push) Has been cancelled
Test Packaging / elizaos CLI global-install smoke (node + bun) (push) Has been cancelled
Cloud Gateway Webhook / Test (push) Has been cancelled
Cloud Tests / lint-and-types (push) Has been cancelled
Cloud Tests / unit-tests (push) Has been cancelled
Cloud Tests / integration-tests (push) Has been cancelled
Cloud Tests / e2e-tests (push) Has been cancelled
CodeQL Advanced / Analyze (javascript-typescript) (push) Has been cancelled
Deploy Apps Worker (Product 2) / Determine environment (push) Has been cancelled
Deploy Apps Worker (Product 2) / Deploy apps worker to apps-control host (${{ needs.determine-env.outputs.environment }}) (push) Has been cancelled
Deploy Eliza Provisioning Worker / Determine environment (push) Has been cancelled
Deploy Eliza Provisioning Worker / Deploy worker to Hetzner host (${{ needs.determine-env.outputs.environment }} @ ${{ needs.determine-env.outputs.deployment_sha }}) (push) Has been cancelled
Dev Smoke / Classify changed paths (push) Has been cancelled
supply-chain / sbom (push) Has been cancelled
supply-chain / vulnerability-scan (push) Has been cancelled
Build, Push & Deploy to Phala Cloud / build-and-push (push) Has been cancelled
Test Packaging / Validate Packaging Configs (push) Has been cancelled
Test Packaging / Build & Test PyPI Package (push) Has been cancelled
Test Packaging / PyPI on Python ${{ matrix.python }} (push) Has been cancelled
Test Packaging / Pack & Test JS Tarballs (push) Has been cancelled
UI Fixture E2E / ui-fixture-e2e (push) Has been cancelled
UI Fixture E2E / fixture-e2e (push) Has been cancelled
UI Story Gate / story-gate (push) Has been cancelled
vault-ci / test (macos-latest) (push) Has been cancelled
vault-ci / test (ubuntu-latest) (push) Has been cancelled
vault-ci / test (windows-latest) (push) Has been cancelled
vault-ci / app-core wiring tests (push) Has been cancelled
verify-patches / verify patches/CHECKSUMS.sha256 (push) Has been cancelled
Voice Benchmark Smoke / voice-emotion fixture smoke (push) Has been cancelled
Voice Benchmark Smoke / voiceagentbench fixture smoke (push) Has been cancelled
Voice Benchmark Smoke / voicebench-quality unit smoke (push) Has been cancelled
Voice Benchmark Smoke / voicebench TypeScript unit (no audio) (push) Has been cancelled
Voice Benchmark Smoke / voice bench smoke summary (push) Has been cancelled
Windows CI / windows ([bun run --cwd packages/app-core test bun run --cwd packages/elizaos test bun run --cwd packages/cloud/shared test], app-and-cli) (push) Has been cancelled
Windows CI / windows ([bun run --cwd packages/scenario-runner test bun run --cwd packages/vault test bun run --cwd packages/security test bun run --cwd plugins/plugin-coding-tools test], framework-packages) (push) Has been cancelled
Windows CI / windows ([bun run --cwd plugins/plugin-elizacloud test bun run --cwd plugins/plugin-discord test bun run --cwd plugins/plugin-anthropic test bun run --cwd plugins/plugin-openai test bun run --cwd plugins/plugin-app-control test bun run --cwd plugins/pl… (push) Has been cancelled
Windows CI / windows ([node packages/scripts/run-turbo.mjs run build --filter=@elizaos/core --filter=@elizaos/shared --filter=@elizaos/agent --concurrency=4 node packages/scripts/run-bash-linux-only.mjs scripts/verify-riscv64-buildpaths.sh node packages/scripts/run… (push) Has been cancelled
Windows CI / windows ([node packages/scripts/run-turbo.mjs run typecheck --filter=@elizaos/core --filter=@elizaos/shared --filter=@elizaos/cloud-shared --concurrency=4 bun run --cwd packages/core test bun run --cwd packages/shared test], core-runtime, 75) (push) Has been cancelled
150 行
6.6 KiB
Markdown
150 行
6.6 KiB
Markdown
---
|
|
title: Action callbacks and SSE streaming
|
|
description: Why Eliza replaces (not concatenates) action callback text in dashboard chat, and how it matches Discord-style progressive messages.
|
|
---
|
|
|
|
# Action callbacks and SSE streaming
|
|
|
|
Eliza’s dashboard chat uses **Server-Sent Events (SSE)** to stream the assistant reply. Two different kinds of text arrive on the same stream:
|
|
|
|
1. **LLM tokens** — the model’s streamed reply (`onStreamChunk`).
|
|
2. **Action callbacks** — text returned from `HandlerCallback` while an action runs (e.g. `PLAY_AUDIO`, wallet flows, Binance skill fallbacks).
|
|
|
|
This page explains **how those are merged** and **why** that design matches platforms like Discord and Telegram.
|
|
|
|
---
|
|
|
|
## The problem we solved
|
|
|
|
On **Discord**, `@elizaos/plugin-discord` uses a **progressive message**: one channel message is created, then **edited in place** as status updates arrive (“Looking up track…”, “Searching…”, “Now playing: …”).
|
|
|
|
On **web**, each `callback({ text })` was previously fed through the same merge path as arbitrary streamed chunks. Unrelated status strings do not share a prefix with each other, so the merge heuristic often **concatenated** them:
|
|
|
|
```text
|
|
🔍 Looking up track...🔍 Searching for track...✨ Setting up playback...Now playing: **Song**
|
|
```
|
|
|
|
That is correct for **token deltas** that extend the same answer, but wrong for **successive statuses** that should **replace** the previous status.
|
|
|
|
**Why it matters:** Users expect **live, in-place updates** (web2-style realtime), not a growing pile of status fragments. Plugins should not need a second transport (WebSocket, custom events) just to get Discord parity.
|
|
|
|
---
|
|
|
|
## The Eliza behavior
|
|
|
|
Inside `generateChatResponse` (`eliza/packages/agent/src/api/chat-routes.ts`):
|
|
|
|
- **LLM chunks** still use **append** semantics via `appendIncomingText` → `resolveStreamingUpdate` → `onChunk`.
|
|
- **Action callbacks** use **`replaceCallbackText`**:
|
|
- On the **first** callback in a turn, the server snapshots whatever was already streamed (`preCallbackText` — usually the LLM’s partial or final text).
|
|
- Each **subsequent** callback sets the visible reply to:
|
|
|
|
`preCallbackText + "\n\n" + latestCallbackText`
|
|
|
|
- So the **callback segment** is **replaced** each time; the LLM prefix is preserved.
|
|
|
|
The HTTP layer emits a **snapshot** (`onSnapshot`) so the SSE event carries the **full** new `fullText`. The client already treats `fullText` as authoritative and **replaces** the assistant bubble’s text — no UI change was required.
|
|
|
|
**Why snapshot:** The frontend’s SSE parser uses `fullText` when present; replacing the whole assistant message is O(1) for the UI and matches “edit message body” mentally.
|
|
|
|
**Why separate LLM vs callback paths:** LLM streaming is genuinely incremental (append). Action progress is **state replacement** (latest status wins). Mixing both through one merge function blurred those semantics.
|
|
|
|
---
|
|
|
|
## Plugin contract (unchanged)
|
|
|
|
Plugins should keep using the **elizaOS** `HandlerCallback` shape:
|
|
|
|
```typescript
|
|
await callback({ text: "🔍 Searching…", source: message.content.source });
|
|
await callback({ text: "Now playing: **Track**", source: message.content.source });
|
|
```
|
|
|
|
The default remains **replace** semantics for callback text. Plugins can now opt into explicit merge behavior with `merge?: "append" | "replace"` when they need to be precise:
|
|
|
|
```typescript
|
|
await callback({
|
|
text: "🔍 Searching…",
|
|
source: message.content.source,
|
|
merge: "replace",
|
|
});
|
|
```
|
|
|
|
`eliza/plugins/plugin-music-player` now does this explicitly through its `ProgressiveMessage` helper. Most plugins do not need to set `merge`; omitting it preserves the existing behavior.
|
|
|
|
**Why preserve the contract:** Discord and other connectors already rely on this API; Eliza’s job is to interpret repeated callbacks correctly in the **API chat** path, not to fork the plugin surface.
|
|
|
|
---
|
|
|
|
## Where it applies
|
|
|
|
`replaceCallbackText` is wired for:
|
|
|
|
- The main `messageService.handleMessage` action callback.
|
|
- `executeFallbackParsedActions` (parsed action recovery).
|
|
- Direct Binance skill dispatch (`maybeHandleDirectBinanceSkillRequest`).
|
|
- Wallet execution fallback and similar paths that invoke actions with callbacks.
|
|
|
|
**Not** used for `onStreamChunk` — that stays append-only.
|
|
|
|
---
|
|
|
|
## Persisted callback history
|
|
|
|
Reloading a conversation now preserves the **full progressive callback trail**, not just the final callback text.
|
|
|
|
### Schema decision
|
|
|
|
The persisted assistant content can include:
|
|
|
|
```ts
|
|
{
|
|
text: "Now playing: **Track**",
|
|
actionCallbackHistory: [
|
|
"🔍 Looking up track...",
|
|
"🔍 Searching for track...",
|
|
"✨ Setting up playback...",
|
|
"Now playing: **Track**"
|
|
]
|
|
}
|
|
```
|
|
|
|
**Why this shape:** it keeps the normal `text` field backward-compatible for existing clients while adding one optional field that captures the historical callback states in order.
|
|
|
|
### Write path
|
|
|
|
- `generateChatResponse()` records each callback snapshot into `actionCallbackHistory`.
|
|
- If the turn already created a visible assistant memory during action execution (for example an `action_result` memory), the conversation route updates that recent assistant memory **in place** with the callback history.
|
|
- If no assistant memory exists yet, the normal persisted assistant turn carries the same `actionCallbackHistory` field.
|
|
|
|
**Why update in place:** action callbacks already create the visible assistant turn for many runtime flows; attaching the history there avoids duplicate assistant bubbles.
|
|
|
|
### Read path
|
|
|
|
When `/api/conversations/:id/messages` reloads persisted messages, it reconstructs the visible transcript by:
|
|
|
|
1. taking every historical callback line except a trailing duplicate of the final `text`
|
|
2. appending the final `text` as the last visible paragraph
|
|
|
|
That means a reloaded conversation shows the same **status trail + final outcome** users saw while the callback stream was live.
|
|
|
|
---
|
|
|
|
## Related code and docs
|
|
|
|
- **Implementation:** `eliza/packages/agent/src/api/chat-routes.ts` — `replaceCallbackText`, `preCallbackText`, `actionCallbackHistory`.
|
|
- **Persistence + replay:** `eliza/packages/agent/src/api/conversation-routes.ts`.
|
|
- **Example helper:** `eliza/plugins/plugin-music-player/` (submodule -- check that it is initialized).
|
|
- **UI streaming:** [Dashboard — Chat](/dashboard/chat) (SSE / typing indicator).
|
|
- **Changelog:** [Changelog](/changelog) — search for “action callback” or the ship date.
|
|
|
|
---
|
|
|
|
## Follow-ups
|
|
|
|
Possible follow-ups:
|
|
|
|
- Optional metadata to distinguish **replace** vs **append** callback semantics if a real plugin needs both within one turn.
|
|
|
|
See the repository docs for high-level product direction.
|