langbot-app--langbot
Junyan Chin
e9dd584792
* feat(api): support global API key from config.yaml (api.global_api_key) Accept a config-defined global API key anywhere a web-UI key is accepted (X-API-Key / Bearer), with no login session and no DB record. Useful for automated deployments and AI agents (HTTP API + MCP). Defaults to empty (disabled); does not require the lbk_ prefix. - templates/config.yaml: add api.global_api_key with security notes - service/apikey.py: verify_api_key checks global key first (constant-time) - docs/API_KEY_AUTH.md: document the global key + security guidance - tests: cover global-key match, prefix-free, fallback-to-db, disabled * feat(mcp): expose LangBot management as an MCP server at /mcp Add an MCP (Model Context Protocol) server so external AI agents can manage a LangBot instance. Reuses the same API-key auth as the HTTP API (including the config.yaml global API key). - pkg/api/mcp/server.py: FastMCP server wrapping the service layer; 21 curated tools across system/bots/pipelines/models/knowledge/mcp-servers/skills - pkg/api/mcp/mount.py: ASGI dispatcher fronting Quart; authenticates /mcp requests with an API key, runs the streamable-HTTP session manager lifespan - controller/main.py: serve the wrapped ASGI app via hypercorn (was run_task) - web: new 'MCP' tab in the API integration dialog showing endpoint, auth, and client config; i18n for 8 locales - tests/manual/mcp_smoke.py: e2e check (401 unauth, list tools, call tools) Tool surface is intentionally curated (not all ~25 route groups) to keep the agent surface small, safe, and maintainable. Extend deliberately. * feat(skills): add in-repo skills/ as the single source of truth Migrate the agent skills + QA/e2e test harness from the (now archived) langbot-app/langbot-skills repo into LangBot/skills/, and add four new skills. Migrated: - langbot-plugin-dev, langbot-testing (e2e), langbot-env-setup, langbot-skills-maintenance, langbot-eba-adapter-dev - the bin/lbs CLI (src/, test/, scripts/, schemas/, qa-agent-docs/) New: - langbot-dev core backend + web development - langbot-deploy Docker/K8s deployment + config.yaml + global API key - langbot-mcp-ops operating the LangBot MCP server (/mcp) - langbot-space-ops operating the Space marketplace MCP server - src/cli.ts repoRoot(): recognize the skills assets root (skills.index.json + bin/lbs) so the CLI works when nested inside the LangBot repo - README.md: unified skill catalog; skills.index.json regenerated Parity with source verified: bin/lbs validate + node test suite match the source repo (only the uncommitted .lbpkg build-artifact fixture differs). * docs(agents): document agent-facing surfaces + API/MCP/skills sync rule * docs(readme): add 'Built for AI Agents' section across all locales Highlight MCP server, in-repo skills (single source of truth), AGENTS.md sync rule, and llms.txt. Cross-link LangBot Space MCP marketplace. * style(mcp): fix ruff format + prettier lint in MCP server and API panel * style(web): prettier format MCP i18n locale entries * docs(skills): note MCP instance control in dev/testing skills All development-guidance skills now point to the LangBot instance MCP server (/mcp) and the Space marketplace MCP server, reusing API keys.
118 行
3.2 KiB
Markdown
118 行
3.2 KiB
Markdown
# LangBot Agent Testing 技术选型
|
|
|
|
## 状态
|
|
|
|
这是技术选型背景文档，不是当前路线图。当前黑盒 E2E QA 的实施顺序见：
|
|
|
|
```text
|
|
docs/qa-agent/04-black-box-e2e-roadmap.md
|
|
```
|
|
|
|
## 目标
|
|
|
|
`langbot-skills` 的目标不是替代测试框架，而是沉淀 agent 可复用的测试资产，让开发者 clone 仓库后，可以让 Codex、Claude Code、Computer Use 或 Playwright MCP 复用已有路径完成 LangBot 功能验证。
|
|
|
|
核心原则：
|
|
|
|
- Skill 负责路由和少量规则。
|
|
- Reference 负责可读流程和背景知识。
|
|
- Case 负责结构化测试路径。
|
|
- Troubleshooting 负责结构化故障资产。
|
|
- `lbs` 负责结构校验、索引、资产创建和未来的运行/报告能力。
|
|
- UI/browser 是产品 QA 的主路径；API/curl 只用于诊断。
|
|
|
|
## 浏览器控制层
|
|
|
|
不同开发者可用的浏览器控制能力不同，所以浏览器层必须可替换。
|
|
|
|
| 方案 | 适用场景 | 优点 | 代价 |
|
|
|---|---|---|---|
|
|
| Codex / Claude Computer Use | agent 可以直接控制可见浏览器 | 登录和交互路径最自然，通常不需要额外 MCP 浏览器桥接 | 依赖具体 agent 工具能力 |
|
|
| Playwright MCP | 没有 Computer Use，但有 MCP 浏览器工具 | 稳定、可脚本化、适合回归路径 | OAuth 登录通常需要额外 visible profile |
|
|
| 直接 Playwright 脚本 | 测试路径非常稳定，适合 CI | 可重复性强 | 需要维护脚本和 selector |
|
|
| 商业 AI QA 平台 | 团队希望外包测试运行平台 | 报告和 PR 集成完整 | 成本和平台绑定 |
|
|
|
|
## 当前推荐
|
|
|
|
先采用分层降级：
|
|
|
|
```text
|
|
有 Computer Use？
|
|
是 -> 使用 Computer Use 控制浏览器
|
|
否 -> 使用 Playwright MCP
|
|
|
|
需要 GitHub OAuth？
|
|
是 -> 使用持久浏览器 profile，让用户手动完成登录
|
|
否 -> 直接使用已有登录态或测试账号状态
|
|
```
|
|
|
|
具体选择逻辑沉淀在：
|
|
|
|
```text
|
|
skills/langbot-env-setup/references/browser-access-selection.md
|
|
```
|
|
|
|
测试原则固定在：
|
|
|
|
```text
|
|
docs/qa-agent/03-agent-browser-qa-principles.md
|
|
```
|
|
|
|
## 环境变量层
|
|
|
|
测试文档不应写死端口。共享默认值放在：
|
|
|
|
```text
|
|
skills/.env
|
|
```
|
|
|
|
关键变量：
|
|
|
|
```text
|
|
LANGBOT_FRONTEND_URL
|
|
LANGBOT_BACKEND_URL
|
|
LANGBOT_DEV_FRONTEND_URL
|
|
LANGBOT_REPO
|
|
LANGBOT_WEB_REPO
|
|
LANGBOT_BROWSER_PROFILE
|
|
```
|
|
|
|
Agent 执行测试前应先读取 `skills/.env`，再用用户提供的当前环境或已启动服务覆盖默认值。
|
|
|
|
## 测试资产层
|
|
|
|
测试资产分两类：
|
|
|
|
```text
|
|
skills/<skill>/
|
|
references/ # Markdown 流程说明
|
|
cases/ # 结构化测试用例
|
|
troubleshooting/ # 结构化故障记录
|
|
```
|
|
|
|
当前已实现：
|
|
|
|
- `SKILL.md` 路由
|
|
- `references/*.md`
|
|
- `lbs case new/list/show`
|
|
- `lbs trouble show/search`
|
|
- `lbs test plan`
|
|
- `lbs test report`
|
|
- `lbs list / validate / index`
|
|
|
|
下一步重点：
|
|
|
|
- 日志守卫规则补充
|
|
- 报告产物管理
|
|
|
|
## 关键判断
|
|
|
|
不要强制所有内容只能通过 CLI 修改。更好的模式是：
|
|
|
|
- 新增 case/troubleshooting：优先使用 `lbs`
|
|
- 大段流程说明：允许直接编辑 Markdown
|
|
- 结构性变更后：必须运行 `lbs validate`
|
|
- 任何生成索引的变更后：运行 `lbs index`
|
|
|
|
这样既能沉淀结构化资产，又不会在 schema 未稳定时拖慢迭代。
|