项目文件夹

文件
Seth Hobson baa5bd7997 feat: add llm-finetuning and dgx-spark-ops plugins (eval-gated fine-tuning lifecycle) (#624)
* feat(dgx-spark-ops): scaffold plugin and register in marketplace

Claude-Session: https://claude.ai/code/session_01RsN3Vz5fZRTdVMkNtmxhSf

* feat(dgx-spark-ops): add spark-environment-setup skill

Claude-Session: https://claude.ai/code/session_01RsN3Vz5fZRTdVMkNtmxhSf

* fix(dgx-spark-ops): tighten spark-environment-setup per review

Claude-Session: https://claude.ai/code/session_01RsN3Vz5fZRTdVMkNtmxhSf

* feat(dgx-spark-ops): add spark-training-gotchas skill

Claude-Session: https://claude.ai/code/session_01RsN3Vz5fZRTdVMkNtmxhSf

* fix(dgx-spark-ops): align preflight.sh output contract with docs

Claude-Session: https://claude.ai/code/session_01RsN3Vz5fZRTdVMkNtmxhSf

* feat(dgx-spark-ops): add spark-memory-thermal-ops skill

Claude-Session: https://claude.ai/code/session_01RsN3Vz5fZRTdVMkNtmxhSf

* fix(dgx-spark-ops): trim spark-memory-thermal-ops and source 68GB anchor

Claude-Session: https://claude.ai/code/session_01RsN3Vz5fZRTdVMkNtmxhSf

* feat(dgx-spark-ops): add dgx-spark-ops-engineer agent

Claude-Session: https://claude.ai/code/session_01RsN3Vz5fZRTdVMkNtmxhSf

* fix(dgx-spark-ops): defer agent facts to skills

Claude-Session: https://claude.ai/code/session_01RsN3Vz5fZRTdVMkNtmxhSf

* feat(dgx-spark-ops): add /spark-preflight command

Claude-Session: https://claude.ai/code/session_01RsN3Vz5fZRTdVMkNtmxhSf

* feat(llm-finetuning): scaffold plugin and register in marketplace

Claude-Session: https://claude.ai/code/session_01RsN3Vz5fZRTdVMkNtmxhSf

* feat(llm-finetuning): add finetuning-method-selection skill with dated model catalog

Claude-Session: https://claude.ai/code/session_01RsN3Vz5fZRTdVMkNtmxhSf

* feat(llm-finetuning): add lora-qlora-recipes skill

Claude-Session: https://claude.ai/code/session_01RsN3Vz5fZRTdVMkNtmxhSf

* feat(llm-finetuning): add preference-optimization skill

Claude-Session: https://claude.ai/code/session_01RsN3Vz5fZRTdVMkNtmxhSf

* feat(llm-finetuning): add grpo-rlvr-training skill

Claude-Session: https://claude.ai/code/session_01RsN3Vz5fZRTdVMkNtmxhSf

* fix(llm-finetuning): add isolation warning to execution reward

Claude-Session: https://claude.ai/code/session_01RsN3Vz5fZRTdVMkNtmxhSf

* feat(llm-finetuning): add vision-sft skill

Claude-Session: https://claude.ai/code/session_01RsN3Vz5fZRTdVMkNtmxhSf

* feat(llm-finetuning): add dataset-curation skill

Claude-Session: https://claude.ai/code/session_01RsN3Vz5fZRTdVMkNtmxhSf

* fix(llm-finetuning): correct loss-masking example in dataset-curation

Claude-Session: https://claude.ai/code/session_01RsN3Vz5fZRTdVMkNtmxhSf

* feat(llm-finetuning): add eval-harness-first skill (Phase 0 gate)

Claude-Session: https://claude.ai/code/session_01RsN3Vz5fZRTdVMkNtmxhSf

* fix(llm-finetuning): bring eval-harness-first into line band

Claude-Session: https://claude.ai/code/session_01RsN3Vz5fZRTdVMkNtmxhSf

* fix(llm-finetuning): reclaim byte headroom in eval-harness-first

Claude-Session: https://claude.ai/code/session_01RsN3Vz5fZRTdVMkNtmxhSf

* feat(llm-finetuning): add trace-to-training-data skill

Claude-Session: https://claude.ai/code/session_01RsN3Vz5fZRTdVMkNtmxhSf

* feat(llm-finetuning): add checkpoint-promotion skill

Claude-Session: https://claude.ai/code/session_01RsN3Vz5fZRTdVMkNtmxhSf

* feat(llm-finetuning): add quantized-export skill

Claude-Session: https://claude.ai/code/session_01RsN3Vz5fZRTdVMkNtmxhSf

* feat(llm-finetuning): add llm-finetuning-architect agent

Claude-Session: https://claude.ai/code/session_01RsN3Vz5fZRTdVMkNtmxhSf

* feat(llm-finetuning): add llm-finetuning-training-engineer agent

Claude-Session: https://claude.ai/code/session_01RsN3Vz5fZRTdVMkNtmxhSf

* feat(llm-finetuning): add llm-finetuning-eval-engineer agent

Claude-Session: https://claude.ai/code/session_01RsN3Vz5fZRTdVMkNtmxhSf

* feat(llm-finetuning): add /finetune phase-gated lifecycle command

Claude-Session: https://claude.ai/code/session_01RsN3Vz5fZRTdVMkNtmxhSf

* fix(llm-finetuning): thread checkpoint path into /finetune Phase 6

Claude-Session: https://claude.ai/code/session_01RsN3Vz5fZRTdVMkNtmxhSf

* feat(llm-finetuning): add /promote-checkpoint command

Claude-Session: https://claude.ai/code/session_01RsN3Vz5fZRTdVMkNtmxhSf

* fix(llm-finetuning): thread checkpoint path via phase5.output in /promote-checkpoint

Claude-Session: https://claude.ai/code/session_01RsN3Vz5fZRTdVMkNtmxhSf

* fix(llm-finetuning): robust checkpoint discovery and goldens fingerprint in re-gate

Claude-Session: https://claude.ai/code/session_01RsN3Vz5fZRTdVMkNtmxhSf

* docs: register llm-finetuning and dgx-spark-ops (94 plugins, 203 agents, 175 skills, 109 commands)

Claude-Session: https://claude.ai/code/session_01RsN3Vz5fZRTdVMkNtmxhSf

* docs(agents): add fine-tuning and spark-ops agent entries (203 agents)

Claude-Session: https://claude.ai/code/session_01RsN3Vz5fZRTdVMkNtmxhSf

* chore: generate per-harness artifacts for llm-finetuning and dgx-spark-ops

Claude-Session: https://claude.ai/code/session_01RsN3Vz5fZRTdVMkNtmxhSf

* fix: final-review cleanups for fine-tuning plugins

Claude-Session: https://claude.ai/code/session_01RsN3Vz5fZRTdVMkNtmxhSf

* fix: address PR #624 review feedback (G1 NGC detection, Phase 3 fallback dispatch, registry metadata)

Claude-Session: https://claude.ai/code/session_01RsN3Vz5fZRTdVMkNtmxhSf

* fix: enforce sandbox boundary in execution grader and reward examples

Claude-Session: https://claude.ai/code/session_01RsN3Vz5fZRTdVMkNtmxhSf

* fix: address CodeRabbit review findings on PR #624

Verified and fixed 37 of 43 outstanding CodeRabbit findings across the
llm-finetuning and dgx-spark-ops plugins (skipping 6 confirmed false
positives/already-fixed, with reasons in the disposition report).

Highlights: cross-file contracts (goldens fingerprint persistence,
paired-arena stage numbering, canonical golden-ID field, RERUN
resolution before promotion) now match between finetune.md,
promote-checkpoint.md, and checkpoint-promotion's templates. TRL API
usages (SFTConfig.max_length, trl.experimental ORPO/CPO imports)
verified against live TRL docs rather than blindly renamed. Several
runnable examples hardened against real failure modes: malformed
judge output, empty arena results, non-distinct DPO pairs, unbounded
rejection-sampling fan-out, non-deterministic smoke-test comparisons,
silent FP8-to-bf16 fallback, and orphaned background thermal-sampler
processes. Container detection (G9) and LoRA adapter-size math fixed
in both dgx-spark-ops and llm-finetuning where the same bugs were
independently present in each plugin.

Claude-Session: https://claude.ai/code/session_01RsN3Vz5fZRTdVMkNtmxhSf

* fix: apply dogfood friction-log remediations (F1-F32) from DGX Spark run

Claude-Session: https://claude.ai/code/session_01RsN3Vz5fZRTdVMkNtmxhSf

* fix: address CodeRabbit round-2 findings (digest pins, suite-size math, monkeypatch scoping)

Claude-Session: https://claude.ai/code/session_01RsN3Vz5fZRTdVMkNtmxhSf
2026-07-14 13:18:02 -04:00

13 KiB

Architecture & Design Principles

This marketplace follows industry best practices with a focus on granularity, composability, and minimal token usage.

Core Philosophy

Single Responsibility Principle

  • Each plugin does one thing well (Unix philosophy)
  • Clear, focused purposes (describable in 5-10 words)
  • Average plugin size: 5.5 components (follows Anthropic's 2-8 pattern)
  • Zero bloated plugins - all plugins focused and purposeful

Composability Over Bundling

  • Mix and match plugins based on needs
  • Workflow orchestrators compose focused plugins
  • No forced feature bundling
  • Clear boundaries between plugins

Context Efficiency

  • Smaller tools = faster processing
  • Better fit in LLM context windows
  • More accurate, focused responses
  • Install only what you need

Maintainability

  • Single-purpose = easier updates
  • Clear boundaries = isolated changes
  • Less duplication = simpler maintenance
  • Isolated dependencies

Granular Plugin Architecture

Plugin Distribution

  • 94 marketplace plugins (90 local + 4 external via git-subdir) optimized for specific use cases
  • 26 clear categories with 1-10 plugins each for easy discovery
  • Organized by domain:
    • Development: 6 plugins (debugging, backend, frontend, UI, multi-platform, essentials)
    • Security: 6 plugins (scanning, compliance, API, frontend/mobile, reverse engineering, hook policy)
    • Operations: 4 plugins (incident, diagnostics, distributed, observability)
    • Languages: 10 plugins (Python, JS/TS, systems, JVM, scripting, functional, embedded, and more)
    • Infrastructure: 5 plugins (deployment, validation, K8s, cloud, CI/CD)
    • And 21 more specialized categories

Component Breakdown

203 Local Specialized Agents

  • Domain experts with deep knowledge
  • Organized across architecture, languages, infrastructure, quality, data/AI, documentation, business, and SEO
  • Model-optimized with a five-tier strategy (Fable, Opus, Sonnet, Haiku, Inherit) for performance and cost

16 Workflow Orchestrators

  • Multi-agent coordination systems
  • Complex operations like full-stack development, security hardening, ML pipelines, incident response
  • Pre-configured agent workflows

109 Local Commands

  • Optimized utilities including:
    • Project scaffolding (Python, TypeScript, Rust)
    • Security scanning (SAST, dependency audit, XSS)
    • Test generation (pytest, Jest)
    • Component scaffolding (React, React Native)
    • Infrastructure setup (Terraform, Kubernetes)

175 Local Agent Skills

  • Modular knowledge packages
  • Progressive disclosure architecture
  • Domain-specific expertise across 48 plugins
  • Spec-compliant (Anthropic Agent Skills Specification)

Repository Structure

claude-agents/
├── .claude-plugin/
│   └── marketplace.json          # Marketplace catalog (94 plugins)
├── plugins/                       # Isolated plugin directories
│   ├── python-development/
│   │   ├── agents/               # Python language agents
│   │   │   ├── python-pro.md
│   │   │   ├── django-pro.md
│   │   │   └── fastapi-pro.md
│   │   ├── commands/             # Python tooling
│   │   │   └── python-scaffold.md
│   │   └── skills/               # Python skills (5 total)
│   │       ├── async-python-patterns/
│   │       ├── python-testing-patterns/
│   │       ├── python-packaging/
│   │       ├── python-performance-optimization/
│   │       └── uv-package-manager/
│   ├── backend-development/
│   │   ├── agents/
│   │   │   ├── backend-architect.md
│   │   │   ├── graphql-architect.md
│   │   │   └── tdd-orchestrator.md
│   │   ├── commands/
│   │   │   └── feature-development.md
│   │   └── skills/               # Backend skills (3 total)
│   │       ├── api-design-principles/
│   │       ├── architecture-patterns/
│   │       └── microservices-patterns/
│   ├── security-scanning/
│   │   ├── agents/
│   │   │   └── security-auditor.md
│   │   ├── commands/
│   │   │   ├── security-hardening.md
│   │   │   ├── security-sast.md
│   │   │   └── security-dependencies.md
│   │   └── skills/               # Security skills (1 total)
│   │       └── sast-configuration/
│   ├── c4-architecture/
│   │   ├── agents/               # C4 architecture agents
│   │   │   ├── c4-code.md
│   │   │   ├── c4-component.md
│   │   │   ├── c4-container.md
│   │   │   └── c4-context.md
│   │   └── commands/
│   │       └── c4-architecture.md
│   └── ... (77 more isolated plugins)
├── docs/                          # Documentation
│   ├── agent-skills.md           # Agent Skills guide
│   ├── agents.md                 # Agent reference
│   ├── plugins.md                # Plugin catalog
│   ├── usage.md                  # Usage guide
│   └── architecture.md           # This file
└── README.md                      # Quick start

Plugin Structure

Each plugin contains:

  • agents/ - Specialized agents for that domain (optional)
  • commands/ - Tools and workflows specific to that plugin (optional)
  • skills/ - Modular knowledge packages with progressive disclosure (optional)

Minimum Requirements

  • At least one agent OR one command
  • Clear, focused purpose
  • Proper frontmatter in all files
  • Entry in marketplace.json

Example Plugin

plugins/kubernetes-operations/
├── agents/
│   └── kubernetes-architect.md   # K8s architecture and design
├── commands/
│   └── k8s-deploy.md            # Deployment automation
└── skills/
    ├── k8s-manifest-generator/   # Manifest creation skill
    ├── helm-chart-scaffolding/   # Helm chart skill
    ├── gitops-workflow/          # GitOps automation skill
    └── k8s-security-policies/    # Security policy skill

Agent Skills Architecture

Progressive Disclosure

Skills use a three-tier architecture for token efficiency:

  1. Metadata (Frontmatter): Name and activation criteria (always loaded)
  2. Instructions: Core guidance and patterns (loaded when activated)
  3. Resources: Examples and templates (loaded on demand)

Specification Compliance

All skills follow the Agent Skills Specification:

---
name: skill-name # Required: hyphen-case
description: What the skill does. Use when [trigger]. # Required: < 1024 chars
---
# Skill content with progressive disclosure

Benefits

  • Token Efficiency: Load only relevant knowledge when needed
  • Specialized Expertise: Deep domain knowledge without bloat
  • Clear Activation: Explicit triggers prevent unwanted invocation
  • Composability: Mix and match skills across workflows
  • Maintainability: Isolated updates don't affect other skills

See Agent Skills for complete details on the 175 skills.

Model Configuration Strategy

Five-Tier Architecture

The system uses Claude Fable, Opus, Sonnet, Haiku, and Inherit assignments strategically:

Model Count Use Case
Fable 0 agents Longest-horizon autonomous work (opt-in tier)
Opus 55 agents Critical architecture, security, code review
Sonnet 71 agents Complex tasks, support with intelligence
Haiku 25 agents Fast operational tasks
Inherit 52 agents Defers model choice to the user at runtime

Selection Criteria

Haiku - Fast Execution & Deterministic Tasks

  • Generating code from well-defined specifications
  • Creating tests following established patterns
  • Writing documentation with clear templates
  • Executing infrastructure operations
  • Performing database query optimization
  • Handling customer support responses
  • Processing SEO optimization tasks
  • Managing deployment pipelines

Sonnet - Complex Reasoning & Architecture

  • Designing system architecture
  • Making technology selection decisions
  • Performing security audits
  • Reviewing code for architectural patterns
  • Creating complex AI/ML pipelines
  • Providing language-specific expertise
  • Orchestrating multi-agent workflows
  • Handling business-critical legal/HR matters

Hybrid Orchestration

Combine models for optimal performance and cost:

Planning Phase (Sonnet) → Execution Phase (Haiku) → Review Phase (Sonnet)

Example:
backend-architect (Sonnet) designs API
  ↓
Generate endpoints (Haiku) implements spec
  ↓
test-automator (Haiku) creates tests
  ↓
code-reviewer (Sonnet) validates architecture

Performance & Quality

Optimized Token Usage

  • Isolated plugins load only what you need
  • Granular architecture reduces unnecessary context
  • Progressive disclosure (skills) loads knowledge on demand
  • Clear boundaries prevent context pollution

Component Coverage

  • 100% agent coverage - all plugins include at least one agent
  • 100% component availability - all 203 local agents accessible across plugins
  • Efficient distribution - 5.5 components per plugin average

Discoverability

  • Clear plugin names convey purpose immediately
  • Logical categorization with 26 well-defined categories
  • Searchable documentation with cross-references
  • Easy to find the right tool for the job

Design Patterns

Pattern 1: Single-Purpose Plugin

Each plugin focuses on one domain:

python-development/
├── agents/           # Python language experts
├── commands/         # Python project scaffolding
└── skills/           # Python-specific knowledge

Benefits:

  • Clear responsibility
  • Easy to maintain
  • Minimal token usage
  • Composable with other plugins

Pattern 2: Workflow Orchestration

Orchestrator plugins coordinate multiple agents:

full-stack-orchestration/
└── commands/
    └── full-stack-feature.md    # Coordinates 7+ agents

Orchestration:

  1. backend-architect (design API)
  2. database-architect (design schema)
  3. frontend-developer (build UI)
  4. test-automator (create tests)
  5. security-auditor (security review)
  6. deployment-engineer (CI/CD)
  7. observability-engineer (monitoring)

Pattern 3: Agent + Skill Integration

Agents provide reasoning, skills provide knowledge:

User: "Build FastAPI project with async patterns"
  ↓
fastapi-pro agent (orchestrates)
  ↓
fastapi-templates skill (provides patterns)
  ↓
python-scaffold command (generates project)

Pattern 4: Multi-Plugin Composition

Complex workflows use multiple plugins:

Feature Development Workflow:
1. backend-development:feature-development
2. security-scanning:security-hardening
3. unit-testing:test-generate
4. comprehensive-review:full-review
5. cicd-automation:workflow-automate
6. observability-monitoring:monitor-setup

Versioning & Updates

Marketplace Updates

  • Marketplace catalog in .claude-plugin/marketplace.json
  • Semantic versioning for plugins
  • Backward compatibility maintained
  • Clear migration guides for breaking changes

Plugin Updates

  • Individual plugin updates don't affect others
  • Skills can be updated independently
  • Agents can be added/removed without breaking workflows
  • Commands maintain stable interfaces

Contributing Guidelines

Adding a Plugin

  1. Create plugin directory: plugins/{plugin-name}/
  2. Add agents and/or commands
  3. Optionally add skills
  4. Update marketplace.json
  5. Document in appropriate category

Adding an Agent

  1. Create plugins/{plugin-name}/agents/{agent-name}.md
  2. Add frontmatter (name, description, model)
  3. Write comprehensive system prompt
  4. Update plugin definition

Adding a Skill

  1. Create plugins/{plugin-name}/skills/{skill-name}/SKILL.md
  2. Add YAML frontmatter (name, description with "Use when")
  3. Write skill content with progressive disclosure
  4. Add to plugin's skills array in marketplace.json

Quality Standards

  • Clear naming - Hyphen-case, descriptive
  • Focused scope - Single responsibility
  • Complete documentation - What, when, how
  • Tested functionality - Verify before committing
  • Spec compliance - Follow Anthropic guidelines

See Also