main
2 次代码提交
| 作者 | SHA1 | 备注 | 提交日期 | |
|---|---|---|---|---|
 Gelei Deng
|
ab5fbb4d90 |
feat: ship the durable multi-model autonomous PentestGPT runtime (#493)
* first refactor
* feat: dockerized tool with persistent Claude+Codex login + multi-model benchmark
Run the autonomous CTF/pentest tool in Docker with a one-time, persistent login for
BOTH Claude Code and Codex, and add a multi-model benchmark harness.
Backend (multi-model):
- Add `--backend {claude,codex}` to the CTF pipeline. CodexBackend (pentestgpt/core/
backend.py) wraps unified_agent's Codex backend and translates its events into
AgentMessages, so the same pipeline runs on Claude (opus/sonnet) or Codex
(gpt-5.5/gpt-5.4-mini). Wired through config.backend, pipeline stage construction,
and the CLI (+ PENTESTGPT_CODEX_EFFORT; greppable [CODEX_USAGE] under PENTESTGPT_BENCH=1).
Docker tool (tool-only image; the benchmark stays OUTSIDE the image):
- Extend Dockerfile: Codex CLI (@openai/codex) + openai_codex SDK + unified_agent/
pentestgpt_agent/pentestgpt_legacy packages + gobuster/dirb + socat. Add .dockerignore
(keeps creds/benchmark/workspace out of the build context).
- Persistent dual login (the hard part) — asymmetric by token model:
* Claude: `setup-token` -> token stored in the pentestgpt-claude volume; entrypoint
exports CLAUDE_CODE_OAUTH_TOKEN (setup-token does not write .credentials.json; macOS
host creds live in the Keychain and can't be copied).
* Codex: the container does its OWN `codex login` (NOT seeding -- ChatGPT refresh tokens
are single-use, so a shared/copied login 401s on first refresh). The 127.0.0.1:1455
OAuth callback is forwarded into the container via a socat hop (-p 1455:8455).
* scripts/docker-login.sh is idempotent: checks logins live, logs in only the missing one(s).
- docker-compose codex-config volume (+ pinned names); entrypoint token-export + non-blocking
preflight; scripts/docker-auth-status.sh; Make targets (docker-build/login/auth-status/
run/shell/down/nuke).
- Verified end-to-end: one `make docker-login` -> a fresh container reports claude+codex
logged in with live round-trips; the CTF pipeline (Codex) captured a flag against an
isolated fixture and the pentest pipeline ran cleanly; persists across recreation, no re-login.
Benchmark (multi-model, host-side):
- benchmark/pilot/ harness (run_pilot.py + report.py): builds each xbow challenge, discovers
the loopback port, runs the pipeline across the 4 model combos, judges by the baked
FLAG{sha256(UPPER-dir)}, and renders REPORT.md (infra failures excluded from solve rates).
Includes the partial pilot's results (results.jsonl + REPORT.md).
Docs: docs/docker-dev-plan.md (full plan + implementation status); CLAUDE.md and README
docker quickstart; benchmark/pilot/README.md; design-doc roadmap (docs/redesign).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix: fail controller on backend error messages
* fix: allow listing sessions without target
* docs: add docker xbow benchmark report
* fix: infer concrete backend constructor type
* docs: refresh docker benchmark documentation
* feat(benchmark): add pure single-agent baseline + pipeline comparison
Add a "pure single agent" benchmark variant -- one bare `claude -p` /
`codex exec` call per target (no pipeline) -- to quantify what the 3-stage
PentestGPT pipeline buys over an un-orchestrated agent on the xbow targets.
- pentestgpt/prompts/stages.py: ctf_single_agent_{system,task}_prompt -- the
pipeline's shared fragments collapsed into ONE turn, so prompt content is
held constant and the only variable is the multi-stage decomposition.
- benchmark/pilot/run_docker_bench.py: docker-network runner
(--variant single|pipeline). Brings the target up, discovers the container's
internal IP+network (skips DB side-cars/ports), docker-runs the tool image on
that network, and scores the ground-truth flag against the agent's *assistant
text* only (parity with the pipeline's raw streaming). Reads stdout in chunks
to handle >64KB JSON lines. Resumable; --dry-run supported.
- benchmark/pilot/report_comparison.py -> DOCKER_COMPARISON.md: head-to-head
pipeline-vs-single per model on the common non-infra set.
- tests/unit/test_single_agent_prompt.py: prompt-builder coverage.
- docs: README, CLAUDE.md, benchmark README, DOCKER_REPORT updated.
Recorded result (10 medium/hard targets x 4 models, container-to-container,
same baseline image digest 0c4c0f3e..., commit
|
||
|
Gelei Deng
|
e238d701f2 |
feat: 🎸 version 1.0 agentic workflow (#325)
* feat: 🎸 version 1.0 agentic workflow
Major rewrite of PentestGPT to use an agentic pipeline architecture:
Core Changes: - New event-driven architecture with EventBus for
TUI-agent decoupling - Implemented AgentController with 5-state
lifecycle (IDLE->RUNNING->PAUSED->COMPLETED->ERROR) - Added AgentBackend
interface with ClaudeCodeBackend implementation - Session management
with file-based persistence for resumable pentests - Langfuse
integration for observability and tracing Interface: - New Textual-based
TUI with real-time activity feed - Keyboard shortcuts: F1 help, Ctrl+P
pause, Ctrl+Q quit - Enhanced CLI with --target, --instruction,
--non-interactive, --debug flags Project Structure: - Moved legacy
multi-LLM version (v0.15) to legacy/ directory - New pentestgpt/core/
for agent, controller, events, session modules - New
pentestgpt/interface/ for TUI and CLI components - New
pentestgpt/benchmark/ for xbow benchmark integration - Comprehensive
test suite in tests/ with unit and integration tests DevOps: - Docker
support with Ubuntu 24.04 container - GitHub Actions CI/CD pipeline -
Makefile with dev commands (test, lint, format, typecheck) - Added
xbow-validation-benchmarks as submodule
* style: format code with Black
This commit fixes the style issues introduced in
|