项目文件夹

文件
wehub-resource-sync 26382a7ac6
CI / Clippy (push) Failing after 15m13s
CI / Test (ubuntu-latest) (push) Failing after 16m1s
CI / Test (macos-latest) (push) Has been cancelled
CI / Test (windows-latest) (push) Has been cancelled
CI / Build (no embeddings / no ORT) (push) Has been cancelled
CI / Format (push) Has been cancelled
CI / Cookbook (Node) (push) Has been cancelled
CI / Pi Extension (Node) (push) Has been cancelled
CI / Rust SDK (lean-ctx-client) (push) Has been cancelled
CI / Embed SDK (lean-ctx-sdk) (push) Has been cancelled
CI / Python SDK (leanctx) (push) Has been cancelled
CI / Hermes Plugin (Python) (push) Has been cancelled
CI / SDK Conformance Matrix (push) Has been cancelled
CI / Coverage (push) Has been cancelled
CI / cargo-deny (push) Has been cancelled
CI / Adversarial Safety (push) Has been cancelled
CI / Benchmarks (push) Has been cancelled
CI / Output-Quality Gate (eval A/B) (push) Has been cancelled
CI / Documentation (push) Has been cancelled
CI / CI Green (push) Has been cancelled
JetBrains Plugin / Actionlint (push) Has been cancelled
CodeQL / Analyze (actions) (push) Has been cancelled
CodeQL / Analyze (javascript-typescript) (push) Has been cancelled
CodeQL / Analyze (rust) (push) Has been cancelled
JetBrains Plugin / Validation (push) Has been cancelled
JetBrains Plugin / Build (push) Has been cancelled
JetBrains Plugin / Test (push) Has been cancelled
Security Check / Security Scan (push) Has been cancelled
chore: import upstream snapshot with attribution
2026-07-13 12:35:30 +08:00

9.7 KiB

CGB Self-Assessment — LeanCTX

Spec version Context Governance Benchmark v1.0-draft
Product & version LeanCTX 3.7.x, OSS engine @ main (2026-06-10)
Assessment type Self-assessment
Assessor LeanCTX maintainers
Date 2026-06-10
Scope notes Self-hosted CLI + MCP server + team server. Cloud control plane and website out of scope.

Spec: context-governance-benchmark (v1.0-draft, pre-review — this grade is against a draft and will be re-assessed at v1.0-final).

Statuses follow each control's own measurement method. Where we could not hard-verify a claim against main today, the control is graded down (see CGB-3.1, CGB-6.5) — initiator self-assessments must over-prove, not over-claim.

Result

CGB v1.0-draft: C2 — Managed. Basic 96% · Hardened 80% · Audited 50%. Self-assessment, not independently verified.

Per-control findings

Domain 1 — Sensitivity & Redaction

CGB-1.1 (Basic) Credential material never reaches the model — Met. core/redaction.rs + core/sensitivity/ redact private keys, API/bearer tokens, cloud keys, credential assignments on the read path. Evidence: redaction + sensitivity test modules in rust/src/core/.

CGB-1.2 (Basic) Declarative, reviewable rules — Met. Policy packs: named TOML patterns, versioned in-repo, built-ins (baseline, strict-redaction, finance-eu, healthcare, open-source). Evidence: docs/contracts/context-policy-packs-v1.md, lean-ctx policy show strict-redaction --toml.

CGB-1.3 (Hardened) Classification beyond secrets — Met. Sensitivity classes + domain packs: finance-eu (IBAN, payment cards, VAT, BIC), healthcare (SSN, MRN, DOB, NPI). Evidence: built-in pack TOML + pack tests.

CGB-1.4 (Hardened) Fail-closed coverage of content paths — Partial. Read/shell/search share the redaction pipeline, but there is no structural single-enforcement-point proof and no CI gate that fails when a new tool bypasses the stage. Gap: coverage is conventional, not structural.

CGB-1.5 (Hardened) Regression-tested efficacy — Met. Redaction/sensitivity tests run in cargo test in CI; corpus includes split-token and noise variants. Improvement noted: encoding-variant coverage is thin.

CGB-1.6 (Audited) Independent verification — Not met. No red team or external assessor has exercised the redaction stage. Declared gap #1.

Domain 2 — Provenance & Integrity

CGB-2.1 (Basic) Source attribution — Met. Reads/search results carry path + mode metadata; session findings store source and timestamp. Evidence: tool output headers, session store schema.

CGB-2.2 (Basic) Transformations disclosed — Met. Compressed reads are framed with mode banners; the mode is recorded in the ledger. Evidence: read-mode output framing, ledger entries.

CGB-2.3 (Hardened) Diagnostics never lossily transformed — Met. Error-preservation in shell compression keeps compiler/test/stack output verbatim (core/compression_safety.rs). Evidence: shell pattern tests.

CGB-2.4 (Hardened) Staleness detected — Partial. Cache validates via mtime+size and fresh=true forces re-reads; cache hits are disclosed. The control requires validation beyond modification-time heuristics (content hash); same-second double-writes are a blind spot. Gap: no content-hash validation on every hit.

CGB-2.5 (Audited) Tamper-evident assembly — Partial. core/audit_trail.rs hash-chains audit events (verified on main), but context-assembly records are not chained end-to-end. Gap: chain covers audit events, not full assembly.

Domain 3 — Budget & Resource Control

CGB-3.1 (Basic) Consumption measured — Partial. Token accounting per call + session ledger exist, with documented divergence notes. The control requires the same tokenization basis the provider bills on; LeanCTX measures with its own tokenizer and documents divergence. Graded down to Partial on the basis-match requirement.

CGB-3.2 (Basic) Hard budgets bind — Met. core/budgets.rs + core/roles.rs::RoleLimits enforce hard caps with refusal on exhaustion. Evidence: budget tests.

CGB-3.3 (Hardened) Attribution per principal/workload — Met. Per-member drilldown (team server), agent identities (core/agent_identity.rs), per-session ledgers. Evidence: team ROI drilldown API + UI.

CGB-3.4 (Hardened) Self-inflicted overhead disclosed — Met. gain --json reports injected_overhead_tokens_per_turn; rules_injection=off and LEAN_CTX_MINIMAL reduce/disable injection. Evidence: performance-tuning guide; claims verified against main (2026-06-09, tokbench follow-up).

CGB-3.5 (Hardened) Fan-out bounded — Partial. core/agent_budget.rs bounds delegated consumption; recursion-depth and concurrency limits for sub-agent spawning are not uniformly configurable. Gap: depth bound is implicit, not policy.

Domain 4 — Audit & Evidence

CGB-4.1 (Basic) Admin actions logged — Met. Org audit log: membership/role changes, key issuance/revocation, plan and policy events with actor/action/target/timestamp. Evidence: docs/contracts/org-audit-log-v1.md.

CGB-4.2 (Basic) Append-only — Met. No edit/delete surface in UI/API/CLI; retention sweep is system-initiated and itself recorded. Evidence: audit API surface review.

CGB-4.3 (Hardened) Retention policy-driven — Met. Plan-based retention windows, automatic sweeper, effective window visible to org admins. Evidence: retention sweeper + account audit page.

CGB-4.4 (Hardened) Open-format export — Met. CSV export for audit log, JSON for ledgers/ROI, schemas documented. Evidence: /api/account/org/audit/export.csv contract.

CGB-4.5 (Audited) Claims reproducible — Met. Pre-registered protocols, pinned task sets, self-hashing result artifacts; agent-task harness uses the official SWE-bench evaluation. Evidence: bench/agent-task/PROTOCOL.md, tokbench methodology.

CGB-4.6 (Audited) Evidence integrity third-party verifiable — Partial. Benchmark artifacts embed SHA-256 self-hashes; audit/ledger exports are not signed. Gap: no signature on operational evidence exports.

Domain 5 — Access Scoping

CGB-5.1 (Basic) Filesystem jailed — Met. core/pathjail.rs: workspace-rooted canonicalization, symlink/traversal refusal, explicit allow-roots. Evidence: pathjail tests.

CGB-5.2 (Basic) Command allowlist — Met. core/shell_allowlist/: allowlist semantics incl. indirect execution (sh -c, interpreter one-liners, pipe-to-shell) on CLI and MCP surfaces. Evidence: allowlist test suite.

CGB-5.3 (Hardened) Destructive tier — Partial. Dangerous commands are distinguished, but destructive/cloud-mutation tooling is not a separately gated tier bound to roles. Gap: single tier beyond the deny set.

CGB-5.4 (Hardened) Roles bind capabilities — Met. core/roles.rs: ToolPolicy, TeamScope, RoleLimits, deny-by-default for ungranted tools. Evidence: roles tests.

CGB-5.5 (Hardened) Egress governed — Partial. URL tools deniable per policy pack; telemetry consent-based. No single operator command enumerates the full effective egress surface. Gap: enumerability. Declared gap #2.

Domain 6 — Lifecycle & Retention

CGB-6.1 (Basic) Stores enumerable — Met. ~/.lean-ctx/ layout documented; lean-ctx status/doctor list stores and sizes. Evidence: docs + doctor output.

CGB-6.2 (Basic) Complete local erasure — Met. lean-ctx uninstall removes stores, hooks, LaunchAgent, binary; lean-ctx stop halts processes (verified on main, cli/dispatch/network.rs). Evidence: uninstall path + process-management docs.

CGB-6.3 (Hardened) Memory lifecycle semantics — Met. core/memory_lifecycle.rs + core/memory_policy.rs: decay, supersession with history, eviction rules; knowledge timeline exposes history. Evidence: lifecycle tests.

CGB-6.4 (Hardened) Shared state honors boundary rules — Partial. Team sync applies redaction before upload; member/device revocation exists. Recipient-side effect of revocation is not verified end-to-end. Gap: revocation effect verification.

CGB-6.5 (Audited) Local operation without vendor services — Partial. Core engine is offline-by-design (local stores, no required cloud calls, consent-based telemetry), but no documented network-isolation test demonstrates Domains 1–5 under blocked vendor endpoints. Graded down until that test exists.

Declared gaps

  1. No independent verification of the redaction stage (CGB-1.6, Not met) — planned as a paid external engagement once revenue allows.
  2. Egress surface not operator-enumerable in one step (CGB-5.5, Partial) — a doctor-style egress inventory is missing; tracked as a follow-up feature.
  3. Further Partials, tracked for the C3 path: structural fail-closed gate (CGB-1.4), content-hash staleness validation (CGB-2.4), assembly-level chaining (CGB-2.5), billing-basis token accounting (CGB-3.1), fan-out depth policy (CGB-3.5), signed evidence exports (CGB-4.6), destructive tiering (CGB-5.3), revocation verification (CGB-6.4), isolation test (CGB-6.5).

Score calculation

Level Met Partial Not met N/A Score
Basic (12) 11 1 (3.1) 0 0 11.5/12 = 96%
Hardened (15) 9 6 (1.4, 2.4, 3.5, 5.3, 5.5, 6.4) 0 0 12/15 = 80%
Audited (5) 1 (4.5) 3 (2.5, 4.6, 6.5) 1 (1.6) 0 2.5/5 = 50%

Grade per SCORING.md: C1 ✓ (96% ≥ 75%) · C2 ✓ (96% ≥ 90%, 80% ≥ 50%) · C3 ✗ (Basic ≠ 100%, blocked by CGB-3.1) → C2 — Managed.

Reassessment

  • At CGB v1.0-final (after external review of the spec), and
  • after closing CGB-3.1 (provider-tokenizer accounting), the single Basic control blocking the C3 path.