simonw--llm
Simon Willison
117 行
3.6 KiB
Plaintext
117 行
3.6 KiB
Plaintext
================================================================================
|
|
COST ESTIMATION FEATURE - COMPLETE PLANNING PACKAGE
|
|
================================================================================
|
|
|
|
WHAT'S IN THIS DIRECTORY
|
|
-------------------------
|
|
A complete, production-ready plan for adding cost estimation to the LLM project.
|
|
|
|
KEY REVISION: Cost estimates are integrated into the existing -u/--usage flag,
|
|
NOT as separate CLI commands. This is simpler, more intuitive, and easier to
|
|
maintain.
|
|
|
|
|
|
START HERE
|
|
----------
|
|
1. READ: INDEX.md (navigation guide)
|
|
2. READ: REVISED_PLAN_SUMMARY.md (what changed and why)
|
|
3. READ: COMPARISON.md (why this approach is better)
|
|
4. IMPLEMENT: Follow IMPLEMENTATION_CHECKLIST_REVISED.md
|
|
|
|
|
|
QUICK OVERVIEW
|
|
--------------
|
|
Before: llm "Hello" -m gpt-4 -u
|
|
Token usage: 10 input, 5 output
|
|
|
|
After: llm "Hello" -m gpt-4 -u
|
|
Token usage: 10 input, 5 output, Cost: $0.000450 ($0.000300 input, $0.000150 output)
|
|
|
|
|
|
WHAT GETS BUILT
|
|
---------------
|
|
✅ llm/costs.py - CostEstimator class (NEW)
|
|
✅ llm/models.py - Response.cost() method (MODIFIED)
|
|
✅ llm/utils.py - Enhanced token_usage_string() (MODIFIED)
|
|
✅ llm/cli.py - Updated usage display (MODIFIED)
|
|
✅ llm/pricing_data.json - Bundled pricing data (NEW)
|
|
✅ tests/test_costs.py - Comprehensive tests (NEW)
|
|
|
|
|
|
WHAT'S NOT BEING BUILT
|
|
----------------------
|
|
❌ llm logs cost - Use -u flag instead
|
|
❌ llm cost-update - Deferred to future
|
|
❌ llm cost-models - Deferred to future
|
|
❌ Cost aggregation - Deferred to future
|
|
|
|
|
|
DOCUMENTATION FILES
|
|
-------------------
|
|
Essential:
|
|
- INDEX.md [Navigation guide]
|
|
- REVISED_PLAN_SUMMARY.md [Start here - overview]
|
|
- COMPARISON.md [Original vs revised]
|
|
- IMPLEMENTATION_CHECKLIST_REVISED.md [Step-by-step guide]
|
|
|
|
Reference:
|
|
- COST_ESTIMATION_PLAN.md [Technical details]
|
|
- COST_ARCHITECTURE.md [Architecture diagrams]
|
|
- EXAMPLE_TESTS.py [Test examples]
|
|
- README_COST_FEATURE.md [Feature overview]
|
|
|
|
Data:
|
|
- pricing_data.json [77 models, 17KB]
|
|
|
|
Superseded:
|
|
- IMPLEMENTATION_CHECKLIST.md [DO NOT USE - original version]
|
|
|
|
|
|
KEY METRICS
|
|
-----------
|
|
Code reduction: 26% less than original plan
|
|
New commands: 0 (vs 3 in original)
|
|
Test reduction: 16% fewer tests needed
|
|
Implementation: 3-4 days estimated
|
|
Complexity: Medium (was High)
|
|
User experience: 5/5 (was 3/5)
|
|
|
|
|
|
IMPLEMENTATION PHASES
|
|
---------------------
|
|
Phase 1: Core functionality (llm/costs.py) - 1-2 days
|
|
Phase 2: Integration (Response.cost()) - 0.5 day
|
|
Phase 3: CLI enhancement (token_usage_string) - 0.5 day
|
|
Phase 4: Documentation & polish - 1 day
|
|
TOTAL: 3-4 days
|
|
|
|
|
|
SUCCESS CRITERIA
|
|
----------------
|
|
[x] Documentation complete
|
|
[x] Plan revised and approved
|
|
[ ] Cost appears with -u flag
|
|
[ ] Cost calculation accurate
|
|
[ ] No breaking changes
|
|
[ ] Tests >90% coverage
|
|
|
|
|
|
NEXT STEPS
|
|
----------
|
|
1. Review REVISED_PLAN_SUMMARY.md
|
|
2. Review COMPARISON.md
|
|
3. Open IMPLEMENTATION_CHECKLIST_REVISED.md
|
|
4. Start Phase 1: Create llm/costs.py
|
|
|
|
|
|
CONTACT & RESOURCES
|
|
-------------------
|
|
Pricing data: https://www.llm-prices.com/
|
|
LLM project: https://github.com/simonw/llm
|
|
LLM docs: https://llm.datasette.io/
|
|
|
|
|
|
================================================================================
|
|
READY TO IMPLEMENT! 🚀
|
|
================================================================================
|