wshobson--agents
be57c0b2e3
* feat(adapters): multi-harness framework + harness_portability eval dimension
Turn this Claude Code plugin marketplace into a generic agentic-harness
marketplace. Adapters under tools/adapters/ emit harness-native artifacts
for OpenAI Codex CLI, Cursor, OpenCode, and Gemini CLI from a single
Markdown source. Source-of-truth stays under plugins/ — Claude Code is
unchanged.
Framework (tools/adapters/):
- base.py — PluginSource parser, HarnessAdapter ABC, write/mirror helpers
(path-traversal guard, UTF-8-safe), inline-list + block-list + block-scalar
YAML-ish parser, _utf8_safe_cut, _split_inline_list, _normalize_author
- capabilities.py — per-harness capability matrix, TOOL_NAME_MAPS,
MODEL_ALIASES, resolve_model() with explicit warnings
- codex.py — emits .codex/{skills,agents}/ + AGENTS.md (≤150-line
table-of-contents). Fence-aware body splitter, _utf8_safe_cut for
multibyte safety, _yaml_scalar with reserved-word + special-char quoting.
Skill/command name collision detection (and second-order __cmd fallback).
- cursor.py — emits .cursor-plugin/{plugin,marketplace}.json + curated
.cursor/rules/*.mdc. _validate_mdc_frontmatter handles YAML block scalars
(no false positives on colons in description body). _normalize_author
handles dict, npm-style strings, and author lists.
- opencode.py — transpiles agents to .opencode/agents/<id>.md with
mode:subagent + permission: deny-everything-else block (skill/task always
allowed as base capabilities — Claude's implicit defaults).
- gemini.py — emits native skills/, agents/, and commands/ at extension
root (April 2026 spec). Tool-allowlist remapped via TOOL_NAME_MAPS.
CLI + tooling:
- tools/generate.py — unified `make generate HARNESS=<x> [PLUGIN=<y>]`,
with --clean (containment-guarded; case-insensitive on Darwin/Win32),
--prune (orphan removal across all per-harness output trees), --strict
(warnings fail), per-plugin error aggregation, refuses --clean --plugin
(would silently wipe other plugins' artifacts).
- tools/validate_generated.py — structural validation across all four
harness outputs. Codex 8KB cap → error. _extract_permission_block
correctly handles nested permission keys (column-0 only).
- tools/doc_gardener.py — recurring drift detection per OpenAI harness-
engineering principle. STALE_ARTIFACT (info), DEAD_LINK (error),
MARKETPLACE_ORPHAN (error), SKILL_OVER_CODEX_CAP (warning), grouped
output sorted by severity.
plugin-eval (extends existing framework):
- New harness_portability dimension (6% weight, rebalanced from existing
static sub-scores). Surfaces non-portable patterns with concrete
remediation hints: SKILL_OVER_CODEX_CAP, CLAUDE_TOOL_REFS,
CLAUDE_TOOL_PROSE, AGENT_NAME_COLLISION, BARE_MODEL_ALIAS.
- _CAMEL_TOOL_PATTERN requires Claude-tool context (no false positives
on Rust's `Task` etc.). _TOOL_PROSE_PATTERN case-sensitive on tool
names, case-insensitive on the leading article.
- Findings do NOT also feed anti_pattern_penalty (no double-counting).
Documentation:
- Top-level guides: CODEX.md, CURSOR.md, OPENCODE.md (≤150 lines each,
table-of-contents pattern per OpenAI harness-engineering post)
- docs/harnesses.md — capability matrix, graceful-degradation table,
generated output paths
- docs/authoring.md — portable-content style guide (tools, models,
collision rules, fence-respect)
- docs/round-trip-results.md — real-CLI verification recipes (OpenCode
discovers 193 subagents, Gemini extensions validate passes, Codex
TOMLs all parse)
- CONTRIBUTING.md — new file pointing at docs/authoring.md
- README.md — rewritten for multi-harness (145 lines, was 460)
- CLAUDE.md — trimmed to 60-line table-of-contents
- GEMINI.md — trimmed from 1500 to 500 tokens (3× over budget previously)
Tests: 181 passing (103 plugin-eval + 78 tools/tests). Real-CLI round-trip
verified for OpenCode, Gemini, and Codex (TOML parses).
Replaces tools/generate_gemini_commands.py with the unified CLI.
* refactor(skills): extract detail to references/details.md (~75 skills)
Apply Anthropic's canonical SKILL.md progressive-disclosure pattern across
the marketplace: SKILL.md body becomes a navigation tier (trigger phrasing
+ quick start), detailed templates and worked examples move to
references/details.md (loaded on demand by the agent).
Motivation: OpenAI Codex CLI hard-truncates skills at 8 KB. Before this
change, ~90 skills exceeded that cap and would silently break on Codex.
The progressive-disclosure pattern is also Anthropic's documented
recommendation for token efficiency — Claude Code reads references/ files
on demand when the body navigation says to.
What's extracted, by pattern:
- Pass 1 (## Templates section): 19 skills — full template libraries
moved to references/details.md
- Pass 2 (## Implementation Patterns / ## Advanced Patterns): 13 skills
- Pass 3 (everything between nav-tier and wrap-tier headings): 53 skills
- Conservative re-extraction for 8 skills that got over-reduced — kept
~6-7 KB inline (most of the quick-start tier) plus references/ overflow
What stays inline (SKILL.md navigation tier):
- description: frontmatter (triggering — unchanged for all skills)
- ## When to Use This Skill / ## Core Concepts / ## Quick Start
- ## Best Practices / ## Troubleshooting / ## See Also wrap-ups
- A pointer note ("see references/details.md") so the agent knows where
to look for detail
What goes to references/details.md (detail tier, on-demand load):
- ## Templates (full code template libraries)
- ## Implementation Patterns / ## Advanced Patterns (deep examples)
- Mid-skill walkthroughs that exceed the inline budget
Also in this commit:
- plugins/brand-landingpage description trimmed from 958→543 chars
(preserves trigger phrasing, drops verbose example-quote list)
Net effect:
- SKILL_OVER_CODEX_CAP findings: 90 → 10 (88% reduction)
- All triggers unchanged — discovery behavior identical across harnesses
- 75 new references/details.md files with the extracted content
- Same depth of guidance, loaded progressively
Remaining 10 oversized skills are complex multi-section docs (e.g.
postgresql, code-review-excellence, evaluation-methodology) that need
per-skill manual judgment — flagged by `make garden` for future work.
* chore: bump all plugin versions (multi-harness release)
Patch-bump every local plugin (81) in both .claude-plugin/marketplace.json
entries and each plugins/<name>/.claude-plugin/plugin.json. Minor-bump the
top-level marketplace metadata.version (1.6.0 → 1.7.0) to signal the
multi-harness adapter framework addition.
The external git-subdir entry (qa-orchestra) is unaffected — its version
is governed by its upstream repo.
* fix(opencode): preserve explicit tools:[] + word-boundary subtask match
Addresses two Codex review findings on PR #541.
## P1 — `tools: []` silently upgraded to permissive (privilege escalation)
Before: `_build_permission_block` returned `{}` for any empty list, which
omits the `permission:` block entirely from the emitted agent. An author
who explicitly wrote `tools: []` to lock down an advisory-only agent got
an UNRESTRICTED agent in OpenCode. Affected agent in this tree:
`plugins/arm-cortex-microcontrollers/agents/arm-cortex-expert.md`.
Fix: `_build_permission_block` now takes a `has_tools_field` flag so the
caller can distinguish "tools: key missing" (Claude default permissive)
from "tools: []" (explicit lock-down). The lock-down case emits a
deny-everything block that allows ONLY the base capabilities (skill, task)
that Claude Code always grants implicitly. Verified against the real
arm-cortex-expert agent — now emits read/edit/write/bash/grep/glob/list:
deny, task/skill: allow.
## P2 — `"agent" in cmd.body.lower()` false-positives on substrings
Before: a command body containing `PerformanceReviewAgent` (class name
in a code snippet) or `useragent` triggered `subtask: true`, changing
runtime behavior based on incidental text.
Fix: switch to a compiled word-boundary regex `\b(agent|subagent)s?\b`
(case-insensitive). Tests confirm the substring `PerformanceReviewAgent`
no longer fires, while a real "spawn a subagent" sentence still does.
## Tests
3 new regression tests in tools/tests/test_adapters.py:
- `test_explicit_empty_tools_yields_locked_permission_block` (P1)
- `test_missing_tools_field_yields_no_permission_block` (P1 boundary)
- `test_subtask_inference_word_boundary` (P2)
184 total tests pass (was 181). OpenCode round-trip still discovers all
193 subagents; arm-cortex-expert agent is now properly locked down.
* test: behavioral verification + CI gates for multi-harness pipeline
Adds three layers of automated verification that pure-Python parser tests
miss, plus the CI jobs that turn them into hard gates. Catches the kinds
of issues that previously only surfaced when a real user installed the
marketplace and tried to use it.
## test_real_world.py — real-source structural tests
Runs against the actual `plugins/` tree (not synthetic fixtures). Catches
issues that only appear on real content:
- every marketplace entry resolves to a plugins/<name>/ dir
- every local plugin dir appears in marketplace.json
- marketplace.json version == per-plugin plugin.json version (catches drift)
- every plugin loads via load_plugin() without error
- no plugin name contains `__` (adapter namespace separator)
- every agent has name + description; every skill has a trigger phrase
(same regex plugin_eval's MISSING_TRIGGER check uses)
- no agent name collides with Codex built-ins
- every refactored skill (with `references/details.md`) has:
- meaningful detail content (>=500 B in details.md)
- a pointer to references/ in the SKILL.md body
- a navigation-tier heading preserved (When to Use, Overview, etc.)
- body >= 600 B (not a stub)
- every plugin.json has name + version matching the dir
This test pass found and fixed three real defects before commit:
- ship-mate/skills/scan: description had no trigger phrase ("Use when…")
- reverse-engineering/skills/memory-forensics: nav-tier section lost
during extraction
- reverse-engineering/skills/binary-analysis-patterns: same
All three are now fixed (preserved trigger phrasing, added When-to-Use
sections back to the skills my extraction over-trimmed).
## test_round_trip.py — generate→parse→verify
CI runs this AFTER `make generate-all`. Catches generation-time regressions:
- OpenCode/Codex/Gemini agent counts match source agent count (no skips)
- every Codex SKILL.md under 8 KB (the cap that would silently truncate)
- every Codex agent TOML has required fields + valid sandbox_mode
- every OpenCode agent has mode in {primary,subagent,all} and
provider-prefixed model
- locked agents (source `tools: []`) emit proper deny-everything permission
block with skill/task allow (regression guard for PR-541 P1)
- every Gemini @{path} injection resolves to a real source file
- every Gemini command TOML has prompt + {{args}} placeholder
- every context file (CLAUDE.md, AGENTS.md, GEMINI.md, etc.) within
150-line cap
- Cursor marketplace + per-plugin manifests cover all local plugins
- .cursor/rules/*.mdc only use the 3 documented frontmatter keys
## test_cli_smoke.py — real-CLI subprocess tests
Invokes the actual harness binaries (OpenCode, Gemini, Codex, Claude Code)
against the generated artifacts. Catches CLI-level issues pure-Python
parsing can't see: schema-loader drift, plugin-discovery bugs, version
incompatibilities.
- `opencode agent list` — must succeed AND discover every source agent
(currently 191 + 2 OpenCode built-ins)
- `gemini extensions validate <repo>` — must return success
- `codex doctor` — must report healthy install
- every Codex agent TOML must parse with stdlib `tomllib`
- `claude --version` — sanity check the Claude Code CLI loads
- marketplace.json must have owner + metadata.version for Claude Code's loader
Per-CLI tests skip gracefully when the binary isn't on PATH, so local
devs only exercise what they have installed. CI installs OpenCode +
Gemini and turns those skips into hard gates.
## Makefile + CI
- `make test` — full pytest suite (plugin-eval + tools/tests/)
- `make smoke-test` — generates if needed, then runs real-CLI smoke tests
- `.github/workflows/validate.yml` extended with:
- `tools-tests` job — runs pytest tools/tests/
- `multi-harness-generate` job — `make generate-all && make validate
STRICT=1 && make garden`, uploads generated artifacts on every run
- `cli-smoke-test` job — installs OpenCode + Gemini, runs test_cli_smoke.py
## Test counts
- Before: 184 tests
- After: 386 tests (parameterized real-source tests over all 82 plugins)
- All passing locally on OpenCode 1.15.7 + Gemini 0.42.0 + Codex 0.133.0
+ Claude Code 2.1.148
352 行
7.7 KiB
Markdown
352 行
7.7 KiB
Markdown
---
|
|
name: binary-analysis-patterns
|
|
description: Master binary analysis patterns including disassembly, decompilation, control flow analysis, and code pattern recognition. Use when analyzing executables, understanding compiled code, or performing static analysis on binaries.
|
|
---
|
|
|
|
# Binary Analysis Patterns
|
|
|
|
Comprehensive patterns and techniques for analyzing compiled binaries, understanding assembly code, and reconstructing program logic.
|
|
|
|
## When to Use This Skill
|
|
|
|
- Reverse-engineering an unknown executable to understand its behavior
|
|
- Analyzing malware or obfuscated binaries with Ghidra / IDA Pro / Binary Ninja
|
|
- Recognizing common assembly idioms (function prologues, switch tables, vtable dispatch)
|
|
- Reconstructing high-level control flow from compiled code
|
|
- Identifying compiler-introduced patterns (stack canaries, PIC trampolines)
|
|
|
|
## Detailed section: Disassembly Fundamentals
|
|
|
|
Originally a 2047-byte section in this SKILL.md. Moved to `references/details.md` to fit Codex's 8 KB skill body cap.
|
|
|
|
## Control Flow Patterns
|
|
|
|
### Conditional Branches
|
|
|
|
```asm
|
|
; if (a == b)
|
|
cmp eax, ebx
|
|
jne skip_block
|
|
; ... if body ...
|
|
skip_block:
|
|
|
|
; if (a < b) - signed
|
|
cmp eax, ebx
|
|
jge skip_block ; Jump if greater or equal
|
|
; ... if body ...
|
|
skip_block:
|
|
|
|
; if (a < b) - unsigned
|
|
cmp eax, ebx
|
|
jae skip_block ; Jump if above or equal
|
|
; ... if body ...
|
|
skip_block:
|
|
```
|
|
|
|
### Loop Patterns
|
|
|
|
```asm
|
|
; for (int i = 0; i < n; i++)
|
|
xor ecx, ecx ; i = 0
|
|
loop_start:
|
|
cmp ecx, [n] ; i < n
|
|
jge loop_end
|
|
; ... loop body ...
|
|
inc ecx ; i++
|
|
jmp loop_start
|
|
loop_end:
|
|
|
|
; while (condition)
|
|
jmp loop_check
|
|
loop_body:
|
|
; ... body ...
|
|
loop_check:
|
|
cmp eax, ebx
|
|
jl loop_body
|
|
|
|
; do-while
|
|
loop_body:
|
|
; ... body ...
|
|
cmp eax, ebx
|
|
jl loop_body
|
|
```
|
|
|
|
### Switch Statement Patterns
|
|
|
|
```asm
|
|
; Jump table pattern
|
|
mov eax, [switch_var]
|
|
cmp eax, max_case
|
|
ja default_case
|
|
jmp [jump_table + eax*8]
|
|
|
|
; Sequential comparison (small switch)
|
|
cmp eax, 1
|
|
je case_1
|
|
cmp eax, 2
|
|
je case_2
|
|
cmp eax, 3
|
|
je case_3
|
|
jmp default_case
|
|
```
|
|
|
|
## Data Structure Patterns
|
|
|
|
### Array Access
|
|
|
|
```asm
|
|
; array[i] - 4-byte elements
|
|
mov eax, [rbx + rcx*4] ; rbx=base, rcx=index
|
|
|
|
; array[i] - 8-byte elements
|
|
mov rax, [rbx + rcx*8]
|
|
|
|
; Multi-dimensional array[i][j]
|
|
; arr[i][j] = base + (i * cols + j) * element_size
|
|
imul eax, [cols]
|
|
add eax, [j]
|
|
mov edx, [rbx + rax*4]
|
|
```
|
|
|
|
### Structure Access
|
|
|
|
```c
|
|
struct Example {
|
|
int a; // offset 0
|
|
char b; // offset 4
|
|
// padding // offset 5-7
|
|
long c; // offset 8
|
|
short d; // offset 16
|
|
};
|
|
```
|
|
|
|
```asm
|
|
; Accessing struct fields
|
|
mov rdi, [struct_ptr]
|
|
mov eax, [rdi] ; s->a (offset 0)
|
|
movzx eax, byte [rdi+4] ; s->b (offset 4)
|
|
mov rax, [rdi+8] ; s->c (offset 8)
|
|
movzx eax, word [rdi+16] ; s->d (offset 16)
|
|
```
|
|
|
|
### Linked List Traversal
|
|
|
|
```asm
|
|
; while (node != NULL)
|
|
list_loop:
|
|
test rdi, rdi ; node == NULL?
|
|
jz list_done
|
|
; ... process node ...
|
|
mov rdi, [rdi+8] ; node = node->next (assuming next at offset 8)
|
|
jmp list_loop
|
|
list_done:
|
|
```
|
|
|
|
## Common Code Patterns
|
|
|
|
### String Operations
|
|
|
|
```asm
|
|
; strlen pattern
|
|
xor ecx, ecx
|
|
strlen_loop:
|
|
cmp byte [rdi + rcx], 0
|
|
je strlen_done
|
|
inc ecx
|
|
jmp strlen_loop
|
|
strlen_done:
|
|
; ecx contains length
|
|
|
|
; strcpy pattern
|
|
strcpy_loop:
|
|
mov al, [rsi]
|
|
mov [rdi], al
|
|
test al, al
|
|
jz strcpy_done
|
|
inc rsi
|
|
inc rdi
|
|
jmp strcpy_loop
|
|
strcpy_done:
|
|
|
|
; memcpy using rep movsb
|
|
mov rdi, dest
|
|
mov rsi, src
|
|
mov rcx, count
|
|
rep movsb
|
|
```
|
|
|
|
### Arithmetic Patterns
|
|
|
|
```asm
|
|
; Multiplication by constant
|
|
; x * 3
|
|
lea eax, [rax + rax*2]
|
|
|
|
; x * 5
|
|
lea eax, [rax + rax*4]
|
|
|
|
; x * 10
|
|
lea eax, [rax + rax*4] ; x * 5
|
|
add eax, eax ; * 2
|
|
|
|
; Division by power of 2 (signed)
|
|
mov eax, [x]
|
|
cdq ; Sign extend to EDX:EAX
|
|
and edx, 7 ; For divide by 8
|
|
add eax, edx ; Adjust for negative
|
|
sar eax, 3 ; Arithmetic shift right
|
|
|
|
; Modulo power of 2
|
|
and eax, 7 ; x % 8
|
|
```
|
|
|
|
### Bit Manipulation
|
|
|
|
```asm
|
|
; Test specific bit
|
|
test eax, 0x80 ; Test bit 7
|
|
jnz bit_set
|
|
|
|
; Set bit
|
|
or eax, 0x10 ; Set bit 4
|
|
|
|
; Clear bit
|
|
and eax, ~0x10 ; Clear bit 4
|
|
|
|
; Toggle bit
|
|
xor eax, 0x10 ; Toggle bit 4
|
|
|
|
; Count leading zeros
|
|
bsr eax, ecx ; Bit scan reverse
|
|
xor eax, 31 ; Convert to leading zeros
|
|
|
|
; Population count (popcnt)
|
|
popcnt eax, ecx ; Count set bits
|
|
```
|
|
|
|
## Decompilation Patterns
|
|
|
|
### Variable Recovery
|
|
|
|
```asm
|
|
; Local variable at rbp-8
|
|
mov qword [rbp-8], rax ; Store to local
|
|
mov rax, [rbp-8] ; Load from local
|
|
|
|
; Stack-allocated array
|
|
lea rax, [rbp-0x40] ; Array starts at rbp-0x40
|
|
mov [rax], edx ; array[0] = value
|
|
mov [rax+4], ecx ; array[1] = value
|
|
```
|
|
|
|
### Function Signature Recovery
|
|
|
|
```asm
|
|
; Identify parameters by register usage
|
|
func:
|
|
; rdi used as first param (System V)
|
|
mov [rbp-8], rdi ; Save param to local
|
|
; rsi used as second param
|
|
mov [rbp-16], rsi
|
|
; Identify return by RAX at end
|
|
mov rax, [result]
|
|
ret
|
|
```
|
|
|
|
### Type Recovery
|
|
|
|
```asm
|
|
; 1-byte operations suggest char/bool
|
|
movzx eax, byte [rdi] ; Zero-extend byte
|
|
movsx eax, byte [rdi] ; Sign-extend byte
|
|
|
|
; 2-byte operations suggest short
|
|
movzx eax, word [rdi]
|
|
movsx eax, word [rdi]
|
|
|
|
; 4-byte operations suggest int/float
|
|
mov eax, [rdi]
|
|
movss xmm0, [rdi] ; Float
|
|
|
|
; 8-byte operations suggest long/double/pointer
|
|
mov rax, [rdi]
|
|
movsd xmm0, [rdi] ; Double
|
|
```
|
|
|
|
## Ghidra Analysis Tips
|
|
|
|
### Improving Decompilation
|
|
|
|
```java
|
|
// In Ghidra scripting
|
|
// Fix function signature
|
|
Function func = getFunctionAt(toAddr(0x401000));
|
|
func.setReturnType(IntegerDataType.dataType, SourceType.USER_DEFINED);
|
|
|
|
// Create structure type
|
|
StructureDataType struct = new StructureDataType("MyStruct", 0);
|
|
struct.add(IntegerDataType.dataType, "field_a", null);
|
|
struct.add(PointerDataType.dataType, "next", null);
|
|
|
|
// Apply to memory
|
|
createData(toAddr(0x601000), struct);
|
|
```
|
|
|
|
### Pattern Matching Scripts
|
|
|
|
```python
|
|
# Find all calls to dangerous functions
|
|
for func in currentProgram.getFunctionManager().getFunctions(True):
|
|
for ref in getReferencesTo(func.getEntryPoint()):
|
|
if func.getName() in ["strcpy", "sprintf", "gets"]:
|
|
print(f"Dangerous call at {ref.getFromAddress()}")
|
|
```
|
|
|
|
## IDA Pro Patterns
|
|
|
|
### IDAPython Analysis
|
|
|
|
```python
|
|
import idaapi
|
|
import idautils
|
|
import idc
|
|
|
|
# Find all function calls
|
|
def find_calls(func_name):
|
|
for func_ea in idautils.Functions():
|
|
for head in idautils.Heads(func_ea, idc.find_func_end(func_ea)):
|
|
if idc.print_insn_mnem(head) == "call":
|
|
target = idc.get_operand_value(head, 0)
|
|
if idc.get_func_name(target) == func_name:
|
|
print(f"Call to {func_name} at {hex(head)}")
|
|
|
|
# Rename functions based on strings
|
|
def auto_rename():
|
|
for s in idautils.Strings():
|
|
for xref in idautils.XrefsTo(s.ea):
|
|
func = idaapi.get_func(xref.frm)
|
|
if func and "sub_" in idc.get_func_name(func.start_ea):
|
|
# Use string as hint for naming
|
|
pass
|
|
```
|
|
|
|
## Best Practices
|
|
|
|
### Analysis Workflow
|
|
|
|
1. **Initial triage**: File type, architecture, imports/exports
|
|
2. **String analysis**: Identify interesting strings, error messages
|
|
3. **Function identification**: Entry points, exports, cross-references
|
|
4. **Control flow mapping**: Understand program structure
|
|
5. **Data structure recovery**: Identify structs, arrays, globals
|
|
6. **Algorithm identification**: Crypto, hashing, compression
|
|
7. **Documentation**: Comments, renamed symbols, type definitions
|
|
|
|
### Common Pitfalls
|
|
|
|
- **Optimizer artifacts**: Code may not match source structure
|
|
- **Inline functions**: Functions may be expanded inline
|
|
- **Tail call optimization**: `jmp` instead of `call` + `ret`
|
|
- **Dead code**: Unreachable code from optimization
|
|
- **Position-independent code**: RIP-relative addressing
|