项目文件夹

文件
Seth Hobson be57c0b2e3 feat: multi-harness plugin marketplace (Codex, Cursor, OpenCode, Gemini) (#541)
* feat(adapters): multi-harness framework + harness_portability eval dimension

Turn this Claude Code plugin marketplace into a generic agentic-harness
marketplace. Adapters under tools/adapters/ emit harness-native artifacts
for OpenAI Codex CLI, Cursor, OpenCode, and Gemini CLI from a single
Markdown source. Source-of-truth stays under plugins/ — Claude Code is
unchanged.

Framework (tools/adapters/):
- base.py — PluginSource parser, HarnessAdapter ABC, write/mirror helpers
  (path-traversal guard, UTF-8-safe), inline-list + block-list + block-scalar
  YAML-ish parser, _utf8_safe_cut, _split_inline_list, _normalize_author
- capabilities.py — per-harness capability matrix, TOOL_NAME_MAPS,
  MODEL_ALIASES, resolve_model() with explicit warnings
- codex.py — emits .codex/{skills,agents}/ + AGENTS.md (≤150-line
  table-of-contents). Fence-aware body splitter, _utf8_safe_cut for
  multibyte safety, _yaml_scalar with reserved-word + special-char quoting.
  Skill/command name collision detection (and second-order __cmd fallback).
- cursor.py — emits .cursor-plugin/{plugin,marketplace}.json + curated
  .cursor/rules/*.mdc. _validate_mdc_frontmatter handles YAML block scalars
  (no false positives on colons in description body). _normalize_author
  handles dict, npm-style strings, and author lists.
- opencode.py — transpiles agents to .opencode/agents/<id>.md with
  mode:subagent + permission: deny-everything-else block (skill/task always
  allowed as base capabilities — Claude's implicit defaults).
- gemini.py — emits native skills/, agents/, and commands/ at extension
  root (April 2026 spec). Tool-allowlist remapped via TOOL_NAME_MAPS.

CLI + tooling:
- tools/generate.py — unified `make generate HARNESS=<x> [PLUGIN=<y>]`,
  with --clean (containment-guarded; case-insensitive on Darwin/Win32),
  --prune (orphan removal across all per-harness output trees), --strict
  (warnings fail), per-plugin error aggregation, refuses --clean --plugin
  (would silently wipe other plugins' artifacts).
- tools/validate_generated.py — structural validation across all four
  harness outputs. Codex 8KB cap → error. _extract_permission_block
  correctly handles nested permission keys (column-0 only).
- tools/doc_gardener.py — recurring drift detection per OpenAI harness-
  engineering principle. STALE_ARTIFACT (info), DEAD_LINK (error),
  MARKETPLACE_ORPHAN (error), SKILL_OVER_CODEX_CAP (warning), grouped
  output sorted by severity.

plugin-eval (extends existing framework):
- New harness_portability dimension (6% weight, rebalanced from existing
  static sub-scores). Surfaces non-portable patterns with concrete
  remediation hints: SKILL_OVER_CODEX_CAP, CLAUDE_TOOL_REFS,
  CLAUDE_TOOL_PROSE, AGENT_NAME_COLLISION, BARE_MODEL_ALIAS.
- _CAMEL_TOOL_PATTERN requires Claude-tool context (no false positives
  on Rust's `Task` etc.). _TOOL_PROSE_PATTERN case-sensitive on tool
  names, case-insensitive on the leading article.
- Findings do NOT also feed anti_pattern_penalty (no double-counting).

Documentation:
- Top-level guides: CODEX.md, CURSOR.md, OPENCODE.md (≤150 lines each,
  table-of-contents pattern per OpenAI harness-engineering post)
- docs/harnesses.md — capability matrix, graceful-degradation table,
  generated output paths
- docs/authoring.md — portable-content style guide (tools, models,
  collision rules, fence-respect)
- docs/round-trip-results.md — real-CLI verification recipes (OpenCode
  discovers 193 subagents, Gemini extensions validate passes, Codex
  TOMLs all parse)
- CONTRIBUTING.md — new file pointing at docs/authoring.md
- README.md — rewritten for multi-harness (145 lines, was 460)
- CLAUDE.md — trimmed to 60-line table-of-contents
- GEMINI.md — trimmed from 1500 to 500 tokens (3× over budget previously)

Tests: 181 passing (103 plugin-eval + 78 tools/tests). Real-CLI round-trip
verified for OpenCode, Gemini, and Codex (TOML parses).

Replaces tools/generate_gemini_commands.py with the unified CLI.

* refactor(skills): extract detail to references/details.md (~75 skills)

Apply Anthropic's canonical SKILL.md progressive-disclosure pattern across
the marketplace: SKILL.md body becomes a navigation tier (trigger phrasing
+ quick start), detailed templates and worked examples move to
references/details.md (loaded on demand by the agent).

Motivation: OpenAI Codex CLI hard-truncates skills at 8 KB. Before this
change, ~90 skills exceeded that cap and would silently break on Codex.
The progressive-disclosure pattern is also Anthropic's documented
recommendation for token efficiency — Claude Code reads references/ files
on demand when the body navigation says to.

What's extracted, by pattern:
- Pass 1 (## Templates section): 19 skills — full template libraries
  moved to references/details.md
- Pass 2 (## Implementation Patterns / ## Advanced Patterns): 13 skills
- Pass 3 (everything between nav-tier and wrap-tier headings): 53 skills
- Conservative re-extraction for 8 skills that got over-reduced — kept
  ~6-7 KB inline (most of the quick-start tier) plus references/ overflow

What stays inline (SKILL.md navigation tier):
- description: frontmatter (triggering — unchanged for all skills)
- ## When to Use This Skill / ## Core Concepts / ## Quick Start
- ## Best Practices / ## Troubleshooting / ## See Also wrap-ups
- A pointer note ("see references/details.md") so the agent knows where
  to look for detail

What goes to references/details.md (detail tier, on-demand load):
- ## Templates (full code template libraries)
- ## Implementation Patterns / ## Advanced Patterns (deep examples)
- Mid-skill walkthroughs that exceed the inline budget

Also in this commit:
- plugins/brand-landingpage description trimmed from 958→543 chars
  (preserves trigger phrasing, drops verbose example-quote list)

Net effect:
- SKILL_OVER_CODEX_CAP findings: 90 → 10 (88% reduction)
- All triggers unchanged — discovery behavior identical across harnesses
- 75 new references/details.md files with the extracted content
- Same depth of guidance, loaded progressively

Remaining 10 oversized skills are complex multi-section docs (e.g.
postgresql, code-review-excellence, evaluation-methodology) that need
per-skill manual judgment — flagged by `make garden` for future work.

* chore: bump all plugin versions (multi-harness release)

Patch-bump every local plugin (81) in both .claude-plugin/marketplace.json
entries and each plugins/<name>/.claude-plugin/plugin.json. Minor-bump the
top-level marketplace metadata.version (1.6.0 → 1.7.0) to signal the
multi-harness adapter framework addition.

The external git-subdir entry (qa-orchestra) is unaffected — its version
is governed by its upstream repo.

* fix(opencode): preserve explicit tools:[] + word-boundary subtask match

Addresses two Codex review findings on PR #541.

## P1 — `tools: []` silently upgraded to permissive (privilege escalation)

Before: `_build_permission_block` returned `{}` for any empty list, which
omits the `permission:` block entirely from the emitted agent. An author
who explicitly wrote `tools: []` to lock down an advisory-only agent got
an UNRESTRICTED agent in OpenCode. Affected agent in this tree:
`plugins/arm-cortex-microcontrollers/agents/arm-cortex-expert.md`.

Fix: `_build_permission_block` now takes a `has_tools_field` flag so the
caller can distinguish "tools: key missing" (Claude default permissive)
from "tools: []" (explicit lock-down). The lock-down case emits a
deny-everything block that allows ONLY the base capabilities (skill, task)
that Claude Code always grants implicitly. Verified against the real
arm-cortex-expert agent — now emits read/edit/write/bash/grep/glob/list:
deny, task/skill: allow.

## P2 — `"agent" in cmd.body.lower()` false-positives on substrings

Before: a command body containing `PerformanceReviewAgent` (class name
in a code snippet) or `useragent` triggered `subtask: true`, changing
runtime behavior based on incidental text.

Fix: switch to a compiled word-boundary regex `\b(agent|subagent)s?\b`
(case-insensitive). Tests confirm the substring `PerformanceReviewAgent`
no longer fires, while a real "spawn a subagent" sentence still does.

## Tests

3 new regression tests in tools/tests/test_adapters.py:
- `test_explicit_empty_tools_yields_locked_permission_block` (P1)
- `test_missing_tools_field_yields_no_permission_block` (P1 boundary)
- `test_subtask_inference_word_boundary` (P2)

184 total tests pass (was 181). OpenCode round-trip still discovers all
193 subagents; arm-cortex-expert agent is now properly locked down.

* test: behavioral verification + CI gates for multi-harness pipeline

Adds three layers of automated verification that pure-Python parser tests
miss, plus the CI jobs that turn them into hard gates. Catches the kinds
of issues that previously only surfaced when a real user installed the
marketplace and tried to use it.

## test_real_world.py — real-source structural tests

Runs against the actual `plugins/` tree (not synthetic fixtures). Catches
issues that only appear on real content:

- every marketplace entry resolves to a plugins/<name>/ dir
- every local plugin dir appears in marketplace.json
- marketplace.json version == per-plugin plugin.json version (catches drift)
- every plugin loads via load_plugin() without error
- no plugin name contains `__` (adapter namespace separator)
- every agent has name + description; every skill has a trigger phrase
  (same regex plugin_eval's MISSING_TRIGGER check uses)
- no agent name collides with Codex built-ins
- every refactored skill (with `references/details.md`) has:
  - meaningful detail content (>=500 B in details.md)
  - a pointer to references/ in the SKILL.md body
  - a navigation-tier heading preserved (When to Use, Overview, etc.)
  - body >= 600 B (not a stub)
- every plugin.json has name + version matching the dir

This test pass found and fixed three real defects before commit:
- ship-mate/skills/scan: description had no trigger phrase ("Use when…")
- reverse-engineering/skills/memory-forensics: nav-tier section lost
  during extraction
- reverse-engineering/skills/binary-analysis-patterns: same

All three are now fixed (preserved trigger phrasing, added When-to-Use
sections back to the skills my extraction over-trimmed).

## test_round_trip.py — generate→parse→verify

CI runs this AFTER `make generate-all`. Catches generation-time regressions:

- OpenCode/Codex/Gemini agent counts match source agent count (no skips)
- every Codex SKILL.md under 8 KB (the cap that would silently truncate)
- every Codex agent TOML has required fields + valid sandbox_mode
- every OpenCode agent has mode in {primary,subagent,all} and
  provider-prefixed model
- locked agents (source `tools: []`) emit proper deny-everything permission
  block with skill/task allow (regression guard for PR-541 P1)
- every Gemini @{path} injection resolves to a real source file
- every Gemini command TOML has prompt + {{args}} placeholder
- every context file (CLAUDE.md, AGENTS.md, GEMINI.md, etc.) within
  150-line cap
- Cursor marketplace + per-plugin manifests cover all local plugins
- .cursor/rules/*.mdc only use the 3 documented frontmatter keys

## test_cli_smoke.py — real-CLI subprocess tests

Invokes the actual harness binaries (OpenCode, Gemini, Codex, Claude Code)
against the generated artifacts. Catches CLI-level issues pure-Python
parsing can't see: schema-loader drift, plugin-discovery bugs, version
incompatibilities.

- `opencode agent list` — must succeed AND discover every source agent
  (currently 191 + 2 OpenCode built-ins)
- `gemini extensions validate <repo>` — must return success
- `codex doctor` — must report healthy install
- every Codex agent TOML must parse with stdlib `tomllib`
- `claude --version` — sanity check the Claude Code CLI loads
- marketplace.json must have owner + metadata.version for Claude Code's loader

Per-CLI tests skip gracefully when the binary isn't on PATH, so local
devs only exercise what they have installed. CI installs OpenCode +
Gemini and turns those skips into hard gates.

## Makefile + CI

- `make test` — full pytest suite (plugin-eval + tools/tests/)
- `make smoke-test` — generates if needed, then runs real-CLI smoke tests
- `.github/workflows/validate.yml` extended with:
  - `tools-tests` job — runs pytest tools/tests/
  - `multi-harness-generate` job — `make generate-all && make validate
    STRICT=1 && make garden`, uploads generated artifacts on every run
  - `cli-smoke-test` job — installs OpenCode + Gemini, runs test_cli_smoke.py

## Test counts

- Before: 184 tests
- After: 386 tests (parameterized real-source tests over all 82 plugins)
- All passing locally on OpenCode 1.15.7 + Gemini 0.42.0 + Codex 0.133.0
  + Claude Code 2.1.148
2026-05-22 08:18:21 -04:00

7.7 KiB

name, description
name description
binary-analysis-patterns Master binary analysis patterns including disassembly, decompilation, control flow analysis, and code pattern recognition. Use when analyzing executables, understanding compiled code, or performing static analysis on binaries.

Binary Analysis Patterns

Comprehensive patterns and techniques for analyzing compiled binaries, understanding assembly code, and reconstructing program logic.

When to Use This Skill

  • Reverse-engineering an unknown executable to understand its behavior
  • Analyzing malware or obfuscated binaries with Ghidra / IDA Pro / Binary Ninja
  • Recognizing common assembly idioms (function prologues, switch tables, vtable dispatch)
  • Reconstructing high-level control flow from compiled code
  • Identifying compiler-introduced patterns (stack canaries, PIC trampolines)

Detailed section: Disassembly Fundamentals

Originally a 2047-byte section in this SKILL.md. Moved to references/details.md to fit Codex's 8 KB skill body cap.

Control Flow Patterns

Conditional Branches

; if (a == b)
cmp eax, ebx
jne skip_block
; ... if body ...
skip_block:

; if (a < b) - signed
cmp eax, ebx
jge skip_block    ; Jump if greater or equal
; ... if body ...
skip_block:

; if (a < b) - unsigned
cmp eax, ebx
jae skip_block    ; Jump if above or equal
; ... if body ...
skip_block:

Loop Patterns

; for (int i = 0; i < n; i++)
xor ecx, ecx           ; i = 0
loop_start:
cmp ecx, [n]           ; i < n
jge loop_end
; ... loop body ...
inc ecx                ; i++
jmp loop_start
loop_end:

; while (condition)
jmp loop_check
loop_body:
; ... body ...
loop_check:
cmp eax, ebx
jl loop_body

; do-while
loop_body:
; ... body ...
cmp eax, ebx
jl loop_body

Switch Statement Patterns

; Jump table pattern
mov eax, [switch_var]
cmp eax, max_case
ja default_case
jmp [jump_table + eax*8]

; Sequential comparison (small switch)
cmp eax, 1
je case_1
cmp eax, 2
je case_2
cmp eax, 3
je case_3
jmp default_case

Data Structure Patterns

Array Access

; array[i] - 4-byte elements
mov eax, [rbx + rcx*4]        ; rbx=base, rcx=index

; array[i] - 8-byte elements
mov rax, [rbx + rcx*8]

; Multi-dimensional array[i][j]
; arr[i][j] = base + (i * cols + j) * element_size
imul eax, [cols]
add eax, [j]
mov edx, [rbx + rax*4]

Structure Access

struct Example {
    int a;      // offset 0
    char b;     // offset 4
    // padding  // offset 5-7
    long c;     // offset 8
    short d;    // offset 16
};
; Accessing struct fields
mov rdi, [struct_ptr]
mov eax, [rdi]         ; s->a (offset 0)
movzx eax, byte [rdi+4] ; s->b (offset 4)
mov rax, [rdi+8]       ; s->c (offset 8)
movzx eax, word [rdi+16] ; s->d (offset 16)

Linked List Traversal

; while (node != NULL)
list_loop:
test rdi, rdi          ; node == NULL?
jz list_done
; ... process node ...
mov rdi, [rdi+8]       ; node = node->next (assuming next at offset 8)
jmp list_loop
list_done:

Common Code Patterns

String Operations

; strlen pattern
xor ecx, ecx
strlen_loop:
cmp byte [rdi + rcx], 0
je strlen_done
inc ecx
jmp strlen_loop
strlen_done:
; ecx contains length

; strcpy pattern
strcpy_loop:
mov al, [rsi]
mov [rdi], al
test al, al
jz strcpy_done
inc rsi
inc rdi
jmp strcpy_loop
strcpy_done:

; memcpy using rep movsb
mov rdi, dest
mov rsi, src
mov rcx, count
rep movsb

Arithmetic Patterns

; Multiplication by constant
; x * 3
lea eax, [rax + rax*2]

; x * 5
lea eax, [rax + rax*4]

; x * 10
lea eax, [rax + rax*4]  ; x * 5
add eax, eax            ; * 2

; Division by power of 2 (signed)
mov eax, [x]
cdq                     ; Sign extend to EDX:EAX
and edx, 7              ; For divide by 8
add eax, edx            ; Adjust for negative
sar eax, 3              ; Arithmetic shift right

; Modulo power of 2
and eax, 7              ; x % 8

Bit Manipulation

; Test specific bit
test eax, 0x80          ; Test bit 7
jnz bit_set

; Set bit
or eax, 0x10            ; Set bit 4

; Clear bit
and eax, ~0x10          ; Clear bit 4

; Toggle bit
xor eax, 0x10           ; Toggle bit 4

; Count leading zeros
bsr eax, ecx            ; Bit scan reverse
xor eax, 31             ; Convert to leading zeros

; Population count (popcnt)
popcnt eax, ecx         ; Count set bits

Decompilation Patterns

Variable Recovery

; Local variable at rbp-8
mov qword [rbp-8], rax  ; Store to local
mov rax, [rbp-8]        ; Load from local

; Stack-allocated array
lea rax, [rbp-0x40]     ; Array starts at rbp-0x40
mov [rax], edx          ; array[0] = value
mov [rax+4], ecx        ; array[1] = value

Function Signature Recovery

; Identify parameters by register usage
func:
    ; rdi used as first param (System V)
    mov [rbp-8], rdi    ; Save param to local
    ; rsi used as second param
    mov [rbp-16], rsi
    ; Identify return by RAX at end
    mov rax, [result]
    ret

Type Recovery

; 1-byte operations suggest char/bool
movzx eax, byte [rdi]   ; Zero-extend byte
movsx eax, byte [rdi]   ; Sign-extend byte

; 2-byte operations suggest short
movzx eax, word [rdi]
movsx eax, word [rdi]

; 4-byte operations suggest int/float
mov eax, [rdi]
movss xmm0, [rdi]       ; Float

; 8-byte operations suggest long/double/pointer
mov rax, [rdi]
movsd xmm0, [rdi]       ; Double

Ghidra Analysis Tips

Improving Decompilation

// In Ghidra scripting
// Fix function signature
Function func = getFunctionAt(toAddr(0x401000));
func.setReturnType(IntegerDataType.dataType, SourceType.USER_DEFINED);

// Create structure type
StructureDataType struct = new StructureDataType("MyStruct", 0);
struct.add(IntegerDataType.dataType, "field_a", null);
struct.add(PointerDataType.dataType, "next", null);

// Apply to memory
createData(toAddr(0x601000), struct);

Pattern Matching Scripts

# Find all calls to dangerous functions
for func in currentProgram.getFunctionManager().getFunctions(True):
    for ref in getReferencesTo(func.getEntryPoint()):
        if func.getName() in ["strcpy", "sprintf", "gets"]:
            print(f"Dangerous call at {ref.getFromAddress()}")

IDA Pro Patterns

IDAPython Analysis

import idaapi
import idautils
import idc

# Find all function calls
def find_calls(func_name):
    for func_ea in idautils.Functions():
        for head in idautils.Heads(func_ea, idc.find_func_end(func_ea)):
            if idc.print_insn_mnem(head) == "call":
                target = idc.get_operand_value(head, 0)
                if idc.get_func_name(target) == func_name:
                    print(f"Call to {func_name} at {hex(head)}")

# Rename functions based on strings
def auto_rename():
    for s in idautils.Strings():
        for xref in idautils.XrefsTo(s.ea):
            func = idaapi.get_func(xref.frm)
            if func and "sub_" in idc.get_func_name(func.start_ea):
                # Use string as hint for naming
                pass

Best Practices

Analysis Workflow

  1. Initial triage: File type, architecture, imports/exports
  2. String analysis: Identify interesting strings, error messages
  3. Function identification: Entry points, exports, cross-references
  4. Control flow mapping: Understand program structure
  5. Data structure recovery: Identify structs, arrays, globals
  6. Algorithm identification: Crypto, hashing, compression
  7. Documentation: Comments, renamed symbols, type definitions

Common Pitfalls

  • Optimizer artifacts: Code may not match source structure
  • Inline functions: Functions may be expanded inline
  • Tail call optimization: jmp instead of call + ret
  • Dead code: Unreachable code from optimization
  • Position-independent code: RIP-relative addressing