* feat(adapters): multi-harness framework + harness_portability eval dimension
Turn this Claude Code plugin marketplace into a generic agentic-harness
marketplace. Adapters under tools/adapters/ emit harness-native artifacts
for OpenAI Codex CLI, Cursor, OpenCode, and Gemini CLI from a single
Markdown source. Source-of-truth stays under plugins/ — Claude Code is
unchanged.
Framework (tools/adapters/):
- base.py — PluginSource parser, HarnessAdapter ABC, write/mirror helpers
(path-traversal guard, UTF-8-safe), inline-list + block-list + block-scalar
YAML-ish parser, _utf8_safe_cut, _split_inline_list, _normalize_author
- capabilities.py — per-harness capability matrix, TOOL_NAME_MAPS,
MODEL_ALIASES, resolve_model() with explicit warnings
- codex.py — emits .codex/{skills,agents}/ + AGENTS.md (≤150-line
table-of-contents). Fence-aware body splitter, _utf8_safe_cut for
multibyte safety, _yaml_scalar with reserved-word + special-char quoting.
Skill/command name collision detection (and second-order __cmd fallback).
- cursor.py — emits .cursor-plugin/{plugin,marketplace}.json + curated
.cursor/rules/*.mdc. _validate_mdc_frontmatter handles YAML block scalars
(no false positives on colons in description body). _normalize_author
handles dict, npm-style strings, and author lists.
- opencode.py — transpiles agents to .opencode/agents/<id>.md with
mode:subagent + permission: deny-everything-else block (skill/task always
allowed as base capabilities — Claude's implicit defaults).
- gemini.py — emits native skills/, agents/, and commands/ at extension
root (April 2026 spec). Tool-allowlist remapped via TOOL_NAME_MAPS.
CLI + tooling:
- tools/generate.py — unified `make generate HARNESS=<x> [PLUGIN=<y>]`,
with --clean (containment-guarded; case-insensitive on Darwin/Win32),
--prune (orphan removal across all per-harness output trees), --strict
(warnings fail), per-plugin error aggregation, refuses --clean --plugin
(would silently wipe other plugins' artifacts).
- tools/validate_generated.py — structural validation across all four
harness outputs. Codex 8KB cap → error. _extract_permission_block
correctly handles nested permission keys (column-0 only).
- tools/doc_gardener.py — recurring drift detection per OpenAI harness-
engineering principle. STALE_ARTIFACT (info), DEAD_LINK (error),
MARKETPLACE_ORPHAN (error), SKILL_OVER_CODEX_CAP (warning), grouped
output sorted by severity.
plugin-eval (extends existing framework):
- New harness_portability dimension (6% weight, rebalanced from existing
static sub-scores). Surfaces non-portable patterns with concrete
remediation hints: SKILL_OVER_CODEX_CAP, CLAUDE_TOOL_REFS,
CLAUDE_TOOL_PROSE, AGENT_NAME_COLLISION, BARE_MODEL_ALIAS.
- _CAMEL_TOOL_PATTERN requires Claude-tool context (no false positives
on Rust's `Task` etc.). _TOOL_PROSE_PATTERN case-sensitive on tool
names, case-insensitive on the leading article.
- Findings do NOT also feed anti_pattern_penalty (no double-counting).
Documentation:
- Top-level guides: CODEX.md, CURSOR.md, OPENCODE.md (≤150 lines each,
table-of-contents pattern per OpenAI harness-engineering post)
- docs/harnesses.md — capability matrix, graceful-degradation table,
generated output paths
- docs/authoring.md — portable-content style guide (tools, models,
collision rules, fence-respect)
- docs/round-trip-results.md — real-CLI verification recipes (OpenCode
discovers 193 subagents, Gemini extensions validate passes, Codex
TOMLs all parse)
- CONTRIBUTING.md — new file pointing at docs/authoring.md
- README.md — rewritten for multi-harness (145 lines, was 460)
- CLAUDE.md — trimmed to 60-line table-of-contents
- GEMINI.md — trimmed from 1500 to 500 tokens (3× over budget previously)
Tests: 181 passing (103 plugin-eval + 78 tools/tests). Real-CLI round-trip
verified for OpenCode, Gemini, and Codex (TOML parses).
Replaces tools/generate_gemini_commands.py with the unified CLI.
* refactor(skills): extract detail to references/details.md (~75 skills)
Apply Anthropic's canonical SKILL.md progressive-disclosure pattern across
the marketplace: SKILL.md body becomes a navigation tier (trigger phrasing
+ quick start), detailed templates and worked examples move to
references/details.md (loaded on demand by the agent).
Motivation: OpenAI Codex CLI hard-truncates skills at 8 KB. Before this
change, ~90 skills exceeded that cap and would silently break on Codex.
The progressive-disclosure pattern is also Anthropic's documented
recommendation for token efficiency — Claude Code reads references/ files
on demand when the body navigation says to.
What's extracted, by pattern:
- Pass 1 (## Templates section): 19 skills — full template libraries
moved to references/details.md
- Pass 2 (## Implementation Patterns / ## Advanced Patterns): 13 skills
- Pass 3 (everything between nav-tier and wrap-tier headings): 53 skills
- Conservative re-extraction for 8 skills that got over-reduced — kept
~6-7 KB inline (most of the quick-start tier) plus references/ overflow
What stays inline (SKILL.md navigation tier):
- description: frontmatter (triggering — unchanged for all skills)
- ## When to Use This Skill / ## Core Concepts / ## Quick Start
- ## Best Practices / ## Troubleshooting / ## See Also wrap-ups
- A pointer note ("see references/details.md") so the agent knows where
to look for detail
What goes to references/details.md (detail tier, on-demand load):
- ## Templates (full code template libraries)
- ## Implementation Patterns / ## Advanced Patterns (deep examples)
- Mid-skill walkthroughs that exceed the inline budget
Also in this commit:
- plugins/brand-landingpage description trimmed from 958→543 chars
(preserves trigger phrasing, drops verbose example-quote list)
Net effect:
- SKILL_OVER_CODEX_CAP findings: 90 → 10 (88% reduction)
- All triggers unchanged — discovery behavior identical across harnesses
- 75 new references/details.md files with the extracted content
- Same depth of guidance, loaded progressively
Remaining 10 oversized skills are complex multi-section docs (e.g.
postgresql, code-review-excellence, evaluation-methodology) that need
per-skill manual judgment — flagged by `make garden` for future work.
* chore: bump all plugin versions (multi-harness release)
Patch-bump every local plugin (81) in both .claude-plugin/marketplace.json
entries and each plugins/<name>/.claude-plugin/plugin.json. Minor-bump the
top-level marketplace metadata.version (1.6.0 → 1.7.0) to signal the
multi-harness adapter framework addition.
The external git-subdir entry (qa-orchestra) is unaffected — its version
is governed by its upstream repo.
* fix(opencode): preserve explicit tools:[] + word-boundary subtask match
Addresses two Codex review findings on PR #541.
## P1 — `tools: []` silently upgraded to permissive (privilege escalation)
Before: `_build_permission_block` returned `{}` for any empty list, which
omits the `permission:` block entirely from the emitted agent. An author
who explicitly wrote `tools: []` to lock down an advisory-only agent got
an UNRESTRICTED agent in OpenCode. Affected agent in this tree:
`plugins/arm-cortex-microcontrollers/agents/arm-cortex-expert.md`.
Fix: `_build_permission_block` now takes a `has_tools_field` flag so the
caller can distinguish "tools: key missing" (Claude default permissive)
from "tools: []" (explicit lock-down). The lock-down case emits a
deny-everything block that allows ONLY the base capabilities (skill, task)
that Claude Code always grants implicitly. Verified against the real
arm-cortex-expert agent — now emits read/edit/write/bash/grep/glob/list:
deny, task/skill: allow.
## P2 — `"agent" in cmd.body.lower()` false-positives on substrings
Before: a command body containing `PerformanceReviewAgent` (class name
in a code snippet) or `useragent` triggered `subtask: true`, changing
runtime behavior based on incidental text.
Fix: switch to a compiled word-boundary regex `\b(agent|subagent)s?\b`
(case-insensitive). Tests confirm the substring `PerformanceReviewAgent`
no longer fires, while a real "spawn a subagent" sentence still does.
## Tests
3 new regression tests in tools/tests/test_adapters.py:
- `test_explicit_empty_tools_yields_locked_permission_block` (P1)
- `test_missing_tools_field_yields_no_permission_block` (P1 boundary)
- `test_subtask_inference_word_boundary` (P2)
184 total tests pass (was 181). OpenCode round-trip still discovers all
193 subagents; arm-cortex-expert agent is now properly locked down.
* test: behavioral verification + CI gates for multi-harness pipeline
Adds three layers of automated verification that pure-Python parser tests
miss, plus the CI jobs that turn them into hard gates. Catches the kinds
of issues that previously only surfaced when a real user installed the
marketplace and tried to use it.
## test_real_world.py — real-source structural tests
Runs against the actual `plugins/` tree (not synthetic fixtures). Catches
issues that only appear on real content:
- every marketplace entry resolves to a plugins/<name>/ dir
- every local plugin dir appears in marketplace.json
- marketplace.json version == per-plugin plugin.json version (catches drift)
- every plugin loads via load_plugin() without error
- no plugin name contains `__` (adapter namespace separator)
- every agent has name + description; every skill has a trigger phrase
(same regex plugin_eval's MISSING_TRIGGER check uses)
- no agent name collides with Codex built-ins
- every refactored skill (with `references/details.md`) has:
- meaningful detail content (>=500 B in details.md)
- a pointer to references/ in the SKILL.md body
- a navigation-tier heading preserved (When to Use, Overview, etc.)
- body >= 600 B (not a stub)
- every plugin.json has name + version matching the dir
This test pass found and fixed three real defects before commit:
- ship-mate/skills/scan: description had no trigger phrase ("Use when…")
- reverse-engineering/skills/memory-forensics: nav-tier section lost
during extraction
- reverse-engineering/skills/binary-analysis-patterns: same
All three are now fixed (preserved trigger phrasing, added When-to-Use
sections back to the skills my extraction over-trimmed).
## test_round_trip.py — generate→parse→verify
CI runs this AFTER `make generate-all`. Catches generation-time regressions:
- OpenCode/Codex/Gemini agent counts match source agent count (no skips)
- every Codex SKILL.md under 8 KB (the cap that would silently truncate)
- every Codex agent TOML has required fields + valid sandbox_mode
- every OpenCode agent has mode in {primary,subagent,all} and
provider-prefixed model
- locked agents (source `tools: []`) emit proper deny-everything permission
block with skill/task allow (regression guard for PR-541 P1)
- every Gemini @{path} injection resolves to a real source file
- every Gemini command TOML has prompt + {{args}} placeholder
- every context file (CLAUDE.md, AGENTS.md, GEMINI.md, etc.) within
150-line cap
- Cursor marketplace + per-plugin manifests cover all local plugins
- .cursor/rules/*.mdc only use the 3 documented frontmatter keys
## test_cli_smoke.py — real-CLI subprocess tests
Invokes the actual harness binaries (OpenCode, Gemini, Codex, Claude Code)
against the generated artifacts. Catches CLI-level issues pure-Python
parsing can't see: schema-loader drift, plugin-discovery bugs, version
incompatibilities.
- `opencode agent list` — must succeed AND discover every source agent
(currently 191 + 2 OpenCode built-ins)
- `gemini extensions validate <repo>` — must return success
- `codex doctor` — must report healthy install
- every Codex agent TOML must parse with stdlib `tomllib`
- `claude --version` — sanity check the Claude Code CLI loads
- marketplace.json must have owner + metadata.version for Claude Code's loader
Per-CLI tests skip gracefully when the binary isn't on PATH, so local
devs only exercise what they have installed. CI installs OpenCode +
Gemini and turns those skips into hard gates.
## Makefile + CI
- `make test` — full pytest suite (plugin-eval + tools/tests/)
- `make smoke-test` — generates if needed, then runs real-CLI smoke tests
- `.github/workflows/validate.yml` extended with:
- `tools-tests` job — runs pytest tools/tests/
- `multi-harness-generate` job — `make generate-all && make validate
STRICT=1 && make garden`, uploads generated artifacts on every run
- `cli-smoke-test` job — installs OpenCode + Gemini, runs test_cli_smoke.py
## Test counts
- Before: 184 tests
- After: 386 tests (parameterized real-source tests over all 82 plugins)
- All passing locally on OpenCode 1.15.7 + Gemini 0.42.0 + Codex 0.133.0
+ Claude Code 2.1.148
12 KiB
workflow-patterns — detailed patterns and worked examples
TDD Task Lifecycle
Follow these 11 steps for each task:
Step 1: Select Next Task
Read plan.md and identify the next pending [ ] task. Select tasks in order within the current phase. Do not skip ahead to later phases.
Step 2: Mark as In Progress
Update plan.md to mark the task as [~]:
- [~] **Task 2.1**: Implement user validation
Commit this status change separately from implementation.
Step 3: RED - Write Failing Tests
Write tests that define the expected behavior before writing implementation:
- Create test file if needed
- Write test cases covering happy path
- Write test cases covering edge cases
- Write test cases covering error conditions
- Run tests - they should FAIL
Example:
def test_validate_user_email_valid():
user = User(email="test@example.com")
assert user.validate_email() is True
def test_validate_user_email_invalid():
user = User(email="invalid")
assert user.validate_email() is False
Step 4: GREEN - Implement Minimum Code
Write the minimum code necessary to make tests pass:
- Focus on making tests green, not perfection
- Avoid premature optimization
- Keep implementation simple
- Run tests - they should PASS
Step 5: REFACTOR - Improve Clarity
With green tests, improve the code:
- Extract common patterns
- Improve naming
- Remove duplication
- Simplify logic
- Run tests after each change - they should remain GREEN
Step 6: Verify Coverage
Check test coverage meets the 80% target:
pytest --cov=module --cov-report=term-missing
If coverage is below 80%:
- Identify uncovered lines
- Add tests for missing paths
- Re-run coverage check
Step 7: Document Deviations
If implementation deviated from plan or introduced new dependencies:
- Update tech-stack.md with new dependencies
- Note deviations in plan.md task comments
- Update spec.md if requirements changed
Step 8: Commit Implementation
Create a focused commit for the task:
git add -A
git commit -m "feat(user): implement email validation
- Add validate_email method to User class
- Handle empty and malformed emails
- Add comprehensive test coverage
Task: 2.1
Track: user-auth_20250115"
Commit message format:
- Type: feat, fix, refactor, test, docs, chore
- Scope: affected module or component
- Summary: imperative, present tense
- Body: bullet points of changes
- Footer: task and track references
Step 9: Attach Git Notes
Add rich task summary as git note:
git notes add -m "Task 2.1: Implement user validation
Summary:
- Added email validation using regex pattern
- Handles edge cases: empty, no @, no domain
- Coverage: 94% on validation module
Files changed:
- src/models/user.py (modified)
- tests/test_user.py (modified)
Decisions:
- Used simple regex over email-validator library
- Reason: No external dependency for basic validation"
Step 10: Update Plan with SHA
Update plan.md to mark task complete with commit SHA:
- [x] **Task 2.1**: Implement user validation `abc1234`
Step 11: Commit Plan Update
Commit the plan status update:
git add conductor/tracks/*/plan.md
git commit -m "docs: update plan - task 2.1 complete
Track: user-auth_20250115"
Phase Completion Protocol
When all tasks in a phase are complete, execute the verification protocol:
Identify Changed Files
List all files modified since the last checkpoint:
git diff --name-only <last-checkpoint-sha>..HEAD
Ensure Test Coverage
For each modified file:
- Identify corresponding test file
- Verify tests exist for new/changed code
- Run coverage for modified modules
- Add tests if coverage < 80%
Run Full Test Suite
Execute complete test suite:
pytest -v --tb=short
All tests must pass before proceeding.
Generate Manual Verification Steps
Create checklist of manual verifications:
## Phase 1 Verification Checklist
- [ ] User can register with valid email
- [ ] Invalid email shows appropriate error
- [ ] Database stores user correctly
- [ ] API returns expected response codes
WAIT for User Approval
Present verification checklist to user:
Phase 1 complete. Please verify:
1. [ ] Test suite passes (automated)
2. [ ] Coverage meets target (automated)
3. [ ] Manual verification items (requires human)
Respond with 'approved' to continue, or note issues.
Do NOT proceed without explicit approval.
Create Checkpoint Commit
After approval, create checkpoint commit:
git add -A
git commit -m "checkpoint: phase 1 complete - user-auth_20250115
Verified:
- All tests passing
- Coverage: 87%
- Manual verification approved
Phase 1 tasks:
- [x] Task 1.1: Setup database schema
- [x] Task 1.2: Implement user model
- [x] Task 1.3: Add validation logic"
Record Checkpoint SHA
Update plan.md checkpoints table:
## Checkpoints
| Phase | Checkpoint SHA | Date | Status |
| ------- | -------------- | ---------- | -------- |
| Phase 1 | def5678 | 2025-01-15 | verified |
| Phase 2 | | | pending |
Quality Assurance Gates
Before marking any task complete, verify these gates:
Passing Tests
- All existing tests pass
- New tests pass
- No test regressions
Coverage >= 80%
- New code has 80%+ coverage
- Overall project coverage maintained
- Critical paths fully covered
Style Compliance
- Code follows style guides
- Linting passes
- Formatting correct
Documentation
- Public APIs documented
- Complex logic explained
- README updated if needed
Type Safety
- Type hints present (if applicable)
- Type checker passes
- No type: ignore without reason
No Linting Errors
- Zero linter errors
- Warnings addressed or justified
- Static analysis clean
Mobile Compatibility
If applicable:
- Responsive design verified
- Touch interactions work
- Performance acceptable
Security Audit
- No secrets in code
- Input validation present
- Authentication/authorization correct
- Dependencies vulnerability-free
Git Integration
Commit Message Format
<type>(<scope>): <subject>
<body>
<footer>
Types:
feat: New featurefix: Bug fixrefactor: Code change without feature/fixtest: Adding testsdocs: Documentationchore: Maintenance
Git Notes for Rich Summaries
Attach detailed notes to commits:
git notes add -m "<detailed summary>"
View notes:
git log --show-notes
Benefits:
- Preserves context without cluttering commit message
- Enables semantic queries across commits
- Supports track-based operations
SHA Recording in plan.md
Always record the commit SHA when completing tasks:
- [x] **Task 1.1**: Setup schema `abc1234`
- [x] **Task 1.2**: Add model `def5678`
This enables:
- Traceability from plan to code
- Semantic revert operations
- Progress auditing
Verification Checkpoints
Why Checkpoints Matter
Checkpoints create restore points for semantic reversion:
- Revert to end of any phase
- Maintain logical code state
- Enable safe experimentation
When to Create Checkpoints
Create checkpoint after:
- All phase tasks complete
- All phase verifications pass
- User approval received
Checkpoint Commit Content
Include in checkpoint commit:
- All uncommitted changes
- Updated plan.md
- Updated metadata.json
- Any documentation updates
How to Use Checkpoints
For reverting:
# Revert to end of Phase 1
git revert --no-commit <phase-2-commits>...
git commit -m "revert: rollback to phase 1 checkpoint"
For review:
# See what changed in Phase 2
git diff <phase-1-sha>..<phase-2-sha>
Handling Deviations
During implementation, deviations from the plan may occur. Handle them systematically:
Types of Deviations
Scope Addition Discovered requirement not in original spec.
- Document in spec.md as new requirement
- Add tasks to plan.md
- Note addition in task comments
Scope Reduction Feature deemed unnecessary during implementation.
- Mark tasks as
[-](skipped) with reason - Update spec.md scope section
- Document decision rationale
Technical Deviation Different implementation approach than planned.
- Note deviation in task completion comment
- Update tech-stack.md if dependencies changed
- Document why original approach was unsuitable
Requirement Change Understanding of requirement changes during work.
- Update spec.md with corrected requirement
- Adjust plan.md tasks if needed
- Re-verify acceptance criteria
Deviation Documentation Format
When completing a task with deviation:
- [x] **Task 2.1**: Implement validation `abc1234`
- DEVIATION: Used library instead of custom code
- Reason: Better edge case handling
- Impact: Added email-validator to dependencies
Error Recovery
Failed Tests After GREEN
If tests fail after reaching GREEN:
- Do NOT proceed to REFACTOR
- Identify which test started failing
- Check if refactoring broke something
- Revert to last known GREEN state
- Re-approach the implementation
Checkpoint Rejection
If user rejects a checkpoint:
- Note rejection reason in plan.md
- Create tasks to address issues
- Complete remediation tasks
- Request checkpoint approval again
Blocked by Dependency
If task cannot proceed:
- Mark task as
[!]with blocker description - Check if other tasks can proceed
- Document expected resolution timeline
- Consider creating dependency resolution track
TDD Variations by Task Type
Data Model Tasks
RED: Write test for model creation and validation
GREEN: Implement model class with fields
REFACTOR: Add computed properties, improve types
API Endpoint Tasks
RED: Write test for request/response contract
GREEN: Implement endpoint handler
REFACTOR: Extract validation, improve error handling
Integration Tasks
RED: Write test for component interaction
GREEN: Wire components together
REFACTOR: Improve error propagation, add logging
Refactoring Tasks
RED: Add characterization tests for current behavior
GREEN: Apply refactoring (tests should stay green)
REFACTOR: Clean up any introduced complexity
Working with Existing Tests
When modifying code with existing tests:
Extend, Don't Replace
- Keep existing tests passing
- Add new tests for new behavior
- Update tests only when requirements change
Test Migration
When refactoring changes test structure:
- Run existing tests (should pass)
- Add new tests for refactored code
- Migrate test cases to new structure
- Remove old tests only after new tests pass
Regression Prevention
After any change:
- Run full test suite
- Check for unexpected failures
- Investigate any new failures
- Fix regressions before proceeding
Checkpoint Verification Details
Automated Verification
Run before requesting approval:
# Test suite
pytest -v --tb=short
# Coverage
pytest --cov=src --cov-report=term-missing
# Linting
ruff check src/ tests/
# Type checking (if applicable)
mypy src/
Manual Verification Guidance
For manual items, provide specific instructions:
## Manual Verification Steps
### User Registration
1. Navigate to /register
2. Enter valid email: test@example.com
3. Enter password meeting requirements
4. Click Submit
5. Verify success message appears
6. Verify user appears in database
### Error Handling
1. Enter invalid email: "notanemail"
2. Verify error message shows
3. Verify form retains other entered data
Performance Considerations
Test Suite Performance
Keep test suite fast:
- Use fixtures to avoid redundant setup
- Mock slow external calls
- Run subset during development, full suite at checkpoints
Commit Performance
Keep commits atomic:
- One logical change per commit
- Complete thought, not work-in-progress
- Tests should pass after every commit