alishahryar1--free-claude-code
65a342ede4
## Problem FCC preserved named reasoning effort but numeric-budget providers received no intensity unless a client supplied an exact token budget. NIM and llama.cpp therefore treated Low through Max like the same provider default. ## Changes | Before | After | | --- | --- | | Named effort had no FCC-owned numeric meaning. | FCC maps Low, Medium, High, X-High, and Max to 512, 1,024, 2,048, 4,096, and 8,192 tokens. | | NIM received only thinking booleans for named effort. | [NIM](https://docs.nvidia.com/nim/large-language-models/1.15.0/thinking-budget-control.html) receives thinking booleans plus the mapped `reasoning_budget`, retaining its existing retry without rejected budget control. | | llama.cpp forwarded only exact client budgets. | [llama.cpp](https://github.com/ggml-org/llama.cpp/blob/master/tools/server/README.md) receives the mapped `thinking_budget_tokens` value. | | Named, boolean, and provider-default adapters shared no explicit numeric contract. | Named adapters keep words, boolean adapters keep on/off, and only numeric-budget adapters consume the FCC scale. | | Documentation prohibited every named-effort budget conversion. | Documentation defines the product scale, exact-budget precedence, and model-independent ownership boundary. | | Version 4.8.0 exposed tiers that collapsed on numeric providers. | Version 4.8.1 completes the tiers; all five local CI checks pass with 2,383 tests passed and 40 skipped. | <!-- greptile_comment --> <details open><summary><h3>Greptile Summary</h3></summary> This PR gives named reasoning tiers numeric budgets for providers that require token counts. The main changes are: - Adds one shared tier-to-token scale with exact-budget precedence. - Sends mapped budgets to NVIDIA NIM and llama.cpp. - Removes conflicting client-supplied NIM budget fields. - Adds focused provider and policy tests. - Updates documentation and bumps the package to 4.8.1. </details> <h3>Confidence Score: 5/5</h3> This looks safe to merge. The NIM fix removes both conflicting budget locations while preserving unrelated nested options. No blocking issue remains in the changed paths. <details><summary><h3><a href="https://www.greptile.com/trex"><img alt="T-Rex" src="https://greptile-static-assets.s3.amazonaws.com/trex/trex_green.svg" height="20" align="absmiddle"></a> T-Rex Logs</h3></summary> **What T-Rex did** - T-Rex ran a pre-change focused validation against the budget logic and observed 16 failures and 2 passes, establishing the baseline. - T-Rex ran a post-change focused validation (head run) and confirmed all 18 cases passed, including the mocked NIM retry paths. - T-Rex executed the full focused validation after the change and produced a verbose log showing 91/91 passing. <a href="https://app.greptile.com/trex/runs/14800553/artifacts"><picture><source media="(prefers-color-scheme: dark)" srcset="https://greptile-static-assets.s3.amazonaws.com/badges/ViewAllArtifactsDark.svg?v=4"><source media="(prefers-color-scheme: light)" srcset="https://greptile-static-assets.s3.amazonaws.com/badges/ViewAllArtifacts.svg?v=4"><img alt="View all artifacts" src="https://greptile-static-assets.s3.amazonaws.com/badges/ViewAllArtifacts.svg?v=4"></picture></a> <sub><a href="https://www.greptile.com/trex"><img alt="T-Rex" src="https://greptile-static-assets.s3.amazonaws.com/trex/trex_green.svg" height="14" align="absmiddle"></a> Ran code and verified through T-Rex</sub> </details> <details open><summary><h3>Important Files Changed</h3></summary> | Filename | Overview | |----------|----------| | src/free_claude_code/core/reasoning.py | Adds the shared effort-to-token scale and preserves exact client budgets as the higher-priority value. | | src/free_claude_code/providers/nvidia_nim/request_options.py | Removes top-level and nested client budgets before inserting one policy-derived NIM budget. | | src/free_claude_code/providers/openai_chat/reasoning.py | Extends llama.cpp request encoding to send mapped named-effort budgets. | </details> <sub>Reviews (2): Last reviewed commit: ["fix: canonicalize NIM reasoning budgets"](https://github.com/alishahryar1/free-claude-code/commit/368f1a88d3f1404fcab2365703cca89f1077f1ea) | [Re-trigger Greptile](https://app.greptile.com/api/retrigger?id=44998377)</sub> <!-- /greptile_comment -->
170 行
4.9 KiB
Python
170 行
4.9 KiB
Python
import pytest
|
|
|
|
from free_claude_code.application.reasoning import (
|
|
client_reasoning_policy,
|
|
resolve_reasoning_policy,
|
|
)
|
|
from free_claude_code.config.reasoning import ReasoningPreference
|
|
from free_claude_code.core.anthropic.models import MessagesRequest
|
|
from free_claude_code.core.reasoning import (
|
|
ReasoningControl,
|
|
ReasoningEffort,
|
|
ReasoningPolicy,
|
|
)
|
|
|
|
|
|
def _request(**overrides) -> MessagesRequest:
|
|
payload = {
|
|
"model": "provider/model",
|
|
"messages": [{"role": "user", "content": "hello"}],
|
|
}
|
|
payload.update(overrides)
|
|
return MessagesRequest.model_validate(payload)
|
|
|
|
|
|
def test_client_without_reasoning_control_uses_provider_default() -> None:
|
|
assert client_reasoning_policy(_request()) == ReasoningPolicy.provider_default()
|
|
|
|
|
|
def test_client_reasoning_preserves_effort_and_exact_budget() -> None:
|
|
policy = client_reasoning_policy(
|
|
_request(
|
|
thinking={"type": "enabled", "budget_tokens": 4096},
|
|
output_config={"effort": "xhigh"},
|
|
)
|
|
)
|
|
|
|
assert policy == ReasoningPolicy.on(
|
|
effort=ReasoningEffort.XHIGH,
|
|
budget_tokens=4096,
|
|
)
|
|
|
|
|
|
def test_named_effort_preserves_intent_without_exact_client_budget() -> None:
|
|
policy = client_reasoning_policy(_request(output_config={"effort": "high"}))
|
|
|
|
assert policy == ReasoningPolicy(
|
|
control=ReasoningControl.DEFAULT,
|
|
effort=ReasoningEffort.HIGH,
|
|
)
|
|
assert policy.budget_tokens is None
|
|
assert policy.requests_reasoning is True
|
|
|
|
|
|
def test_invalid_budget_does_not_implicitly_enable_reasoning() -> None:
|
|
policy = client_reasoning_policy(_request(thinking={"budget_tokens": 0}))
|
|
|
|
assert policy == ReasoningPolicy.provider_default()
|
|
|
|
|
|
@pytest.mark.parametrize(
|
|
"messages_request",
|
|
[
|
|
_request(thinking={"type": "disabled"}),
|
|
_request(output_config={"effort": "none"}),
|
|
],
|
|
)
|
|
def test_client_disable_is_explicit(messages_request: MessagesRequest) -> None:
|
|
policy = client_reasoning_policy(messages_request)
|
|
|
|
assert policy.control is ReasoningControl.OFF
|
|
assert policy.output_enabled is False
|
|
assert policy.requests_reasoning is False
|
|
|
|
|
|
def test_disabled_thinking_preserves_independent_effort_intent() -> None:
|
|
policy = client_reasoning_policy(
|
|
_request(
|
|
thinking={"type": "disabled"},
|
|
output_config={"effort": "medium"},
|
|
)
|
|
)
|
|
|
|
assert policy == ReasoningPolicy(
|
|
control=ReasoningControl.OFF,
|
|
effort=ReasoningEffort.MEDIUM,
|
|
)
|
|
assert policy.requests_reasoning is False
|
|
|
|
|
|
def test_fixed_route_effort_overrides_client_disable() -> None:
|
|
policy = resolve_reasoning_policy(
|
|
_request(thinking={"type": "disabled"}),
|
|
ReasoningPreference.MAX,
|
|
)
|
|
|
|
assert policy == ReasoningPolicy.on(effort=ReasoningEffort.MAX)
|
|
|
|
|
|
def test_fixed_off_overrides_client_enable() -> None:
|
|
policy = resolve_reasoning_policy(
|
|
_request(thinking={"type": "enabled", "budget_tokens": 1024}),
|
|
ReasoningPreference.OFF,
|
|
)
|
|
|
|
assert policy == ReasoningPolicy.off()
|
|
|
|
|
|
def test_client_preference_preserves_client_policy() -> None:
|
|
request = _request(output_config={"effort": "low"})
|
|
|
|
assert resolve_reasoning_policy(
|
|
request, ReasoningPreference.CLIENT
|
|
) == client_reasoning_policy(request)
|
|
|
|
|
|
def test_unresolved_inherit_is_rejected() -> None:
|
|
with pytest.raises(ValueError, match="must be resolved"):
|
|
resolve_reasoning_policy(_request(), ReasoningPreference.INHERIT)
|
|
|
|
|
|
@pytest.mark.parametrize("budget", [0, -1, True])
|
|
def test_reasoning_budget_requires_a_positive_integer(budget: int) -> None:
|
|
with pytest.raises(ValueError, match="positive integer"):
|
|
ReasoningPolicy.on(budget_tokens=budget)
|
|
|
|
|
|
def test_reasoning_budget_requires_explicit_on_control() -> None:
|
|
with pytest.raises(ValueError, match="control to be on"):
|
|
ReasoningPolicy(budget_tokens=100)
|
|
|
|
|
|
@pytest.mark.parametrize(
|
|
("effort", "expected"),
|
|
(
|
|
(ReasoningEffort.MINIMAL, 512),
|
|
(ReasoningEffort.LOW, 512),
|
|
(ReasoningEffort.MEDIUM, 1_024),
|
|
(ReasoningEffort.HIGH, 2_048),
|
|
(ReasoningEffort.XHIGH, 4_096),
|
|
(ReasoningEffort.MAX, 8_192),
|
|
),
|
|
)
|
|
def test_reasoning_effort_has_one_fcc_numeric_budget(
|
|
effort: ReasoningEffort, expected: int
|
|
) -> None:
|
|
assert ReasoningPolicy.on(effort=effort).numeric_budget_tokens == expected
|
|
|
|
|
|
def test_exact_reasoning_budget_takes_precedence_over_effort_mapping() -> None:
|
|
policy = ReasoningPolicy.on(
|
|
effort=ReasoningEffort.XHIGH,
|
|
budget_tokens=777,
|
|
)
|
|
|
|
assert policy.numeric_budget_tokens == 777
|
|
|
|
|
|
@pytest.mark.parametrize(
|
|
"policy",
|
|
(
|
|
ReasoningPolicy.provider_default(),
|
|
ReasoningPolicy.off(),
|
|
ReasoningPolicy.on(),
|
|
),
|
|
)
|
|
def test_reasoning_without_numeric_intensity_has_no_budget(
|
|
policy: ReasoningPolicy,
|
|
) -> None:
|
|
assert policy.numeric_budget_tokens is None
|