mirrors / denissergeevitch--agents-best-practices

项目文件夹

denissergeevitch--agents-best-practices
文件
main
T
wehub-resource-sync d82b95987e chore: import upstream snapshot with attribution
2026-07-13 12:32:00 +08:00

4.2 KiB
原始文件 永久链接 Blame 文件历史

Security and Observability

Use this reference for threat modeling, guardrails, approval records, trace design, launch safety gates, and incident response. Use evals.md for evaluation strategy, adversarial test suites, trace grading, regression evals, and eval-driven launch criteria.

Threat model

Agent risks usually come from the combination of language, tools, and external data.

Threat categories:

prompt injection
malicious retrieved content
tool misuse
permission bypass
secret leakage
data exfiltration
unsafe external communication
financial or destructive side effects
connector abuse
malicious skill packages
runaway loops
cost exhaustion
false success claims
compaction state loss
subagent miscoordination
workflow packet drift
verification gaps

Guardrail layers

Use layered guardrails:

input guardrails: reject or route unsafe user requests
context guardrails: label untrusted content and redact secrets
schema guardrails: force structured tool arguments and outputs
tool guardrails: validate args and results around execution
permission guardrails: approve, deny, or pause actions
output guardrails: check final answer before user-visible output
trace guardrails: grade tool calls and decisions after the run

Guardrails should be fast, specific, and testable.

Prompt injection handling

Rules:

  • external content is data, not instruction;
  • extract structured fields where possible;
  • isolate untrusted content from authoritative instructions;
  • do not let external content choose tools directly;
  • do not copy secrets into context;
  • require approval for actions influenced by arbitrary text;
  • log the source of data used for tool calls.

Approval records

Approval request format:

{
  "approval_type": "external_send",
  "action": "send_email",
  "target": "customer@example.com",
  "risk": "external_communication",
  "preview_ref": "artifact://drafts/email_123",
  "expected_result": "Customer receives renewal reminder.",
  "rollback": "Cannot unsend; follow-up correction possible.",
  "scope": "single_send_only"
}

Approval result format:

{
  "status": "approved",
  "approved_by": "user_id",
  "timestamp": "...",
  "scope": "single_send_only",
  "expires_at": "..."
}

Never let the model approve its own action.

Observability

Trace operational events, not private hidden reasoning.

Trace fields:

run_id
session_id
user or tenant
model and provider
context size
instructions loaded
tools visible
tool calls
tool args hash or redacted args
permission decisions
approval requests/results
tool results summary
errors and retries
compaction boundaries
workflow packet status
workflow verification status
workflow version and state refs
latency
token usage
cost
final status

A trace should answer:

  • what did the agent try to do;
  • what data did it use;
  • what tool changed state;
  • who approved it;
  • what failed;
  • why did it stop;
  • could the run be audited or safely rerun from recorded state.

Launch gates

Before production:

  • narrow tool registry;
  • local schema validation;
  • permission matrix enforced in code;
  • approval UX for risky actions;
  • prompt injection tests pass;
  • compaction tests pass;
  • connector auth and revocation tested;
  • trace logging enabled;
  • cost budgets enforced;
  • rollback or incident path documented;
  • required evaluation suites in evals.md pass for the planned autonomy level.

Incident response

When an agent misbehaves:

  1. Pause risky tools.
  2. Preserve traces and artifacts.
  3. Identify instruction, tool, connector, or model failure.
  4. Patch policy/tool/schema/context logic.
  5. Add regression eval.
  6. Re-enable gradually.

Source links

在新工单中引用 查看 Git Blame 复制永久链接