mirrors / sgl-project--sglang

项目文件夹

sgl-project--sglang
文件
main
T
wehub-resource-sync 94057c3d3e
PR Test (NPU) / check-changes (push) Has been cancelled
详情
PR Test (NPU) / pr-gate (push) Has been cancelled
详情
PR Test (NPU) / set-image-config (push) Has been cancelled
详情
PR Test (NPU) / stage-b-test-1-npu-a2 (0) (push) Has been cancelled
详情
PR Test (NPU) / stage-b-test-1-npu-a2 (1) (push) Has been cancelled
详情
PR Test (NPU) / stage-b-test-2-npu-a2 (0) (push) Has been cancelled
详情
PR Test (NPU) / stage-b-test-2-npu-a2 (1) (push) Has been cancelled
详情
PR Test (NPU) / stage-b-test-4-npu-a3 (push) Has been cancelled
详情
PR Test (NPU) / stage-b-test-16-npu-a3 (push) Has been cancelled
详情
PR Test (NPU) / multimodal-gen-test-1-npu-a3 (push) Has been cancelled
详情
PR Test (NPU) / multimodal-gen-test-2-npu-a3 (push) Has been cancelled
详情
PR Test (Arm64) / pr-gate (push) Has been cancelled
详情
PR Test (Arm64) / check-changes (push) Has been cancelled
详情
PR Test (Arm64) / build-test (push) Has been cancelled
详情
PR Test (sgl-router) / gate (push) Has been cancelled
详情
PR Test (sgl-router) / tier-1 — lint (push) Has been cancelled
详情
PR Test (sgl-router) / tier-2 — build + test (push) Has been cancelled
详情
PR Test (sgl-router) / tier-3 — docker (placeholder) (push) Has been cancelled
详情
PR Test (sgl-router) / tier-3 — k8s integration (push) Has been cancelled
详情
PR Test (sgl-router) / tier-3 — e2e (push) Has been cancelled
详情
PR Test (sgl-router) / finish (push) Has been cancelled
详情
PR Test (NPU) / single-node-poc (map[name:qwen3_6_27b_w8a8_1p_in64k_out1k_50ms runner:linux-aarch64-a3-2 test_case:test/registered/ascend/performance/qwen3_6_27b/test_npu_qwen3_6_27b_w8a8_1p_in64k_out1k_50ms.py test_type:perf]) (push) Has been cancelled
详情
PR Test (NPU) / pr-test-npu-finish (push) Has been cancelled
详情
PR Test (Xeon) / pr-gate (push) Has been cancelled
详情
PR Test (Xeon) / check-changes (push) Has been cancelled
详情
PR Test (Xeon) / build-test (, xeon-gnr, base-b-test-cpu) (push) Has been cancelled
详情
PR Test (XPU) / check-changes (push) Has been cancelled
详情
PR Test (XPU) / pr-gate (push) Has been cancelled
详情
PR Test (XPU) / stage-a-test-1-gpu-xpu (push) Has been cancelled
详情
PR Test (XPU) / wait-for-stage-a (push) Has been cancelled
详情
PR Test (XPU) / stage-b-test-1-gpu-xpu (push) Has been cancelled
详情
PR Test (XPU) / finish (push) Has been cancelled
详情
CI Model Inventory / build-inventory (push) Has been cancelled
详情
Lint / lint (push) Has been cancelled
详情
PR Benchmark (SMG Components) / Benchmark Compilation Check (push) Has been cancelled
详情
PR Benchmark (SMG Components) / Benchmark - Manual Policy (push) Has been cancelled
详情
PR Benchmark (SMG Components) / Benchmark - Request Processing (push) Has been cancelled
详情
PR Benchmark (SMG Components) / Benchmark Summary (push) Has been cancelled
详情
PR Test (SMG) / build-wheel (push) Has been cancelled
详情
Release SGLang Model Gateway to PyPI / build on windows (x86_64 - auto) (push) Has been cancelled
详情
Release SGLang Model Gateway to PyPI / build on macos (x86_64 - auto) (push) Has been cancelled
详情
PR Test (SMG) / python-unit-tests (push) Has been cancelled
详情
PR Test (SMG) / unit-tests (push) Has been cancelled
详情
PR Test (SMG) / benchmarks (push) Has been cancelled
详情
PR Test (SMG) / chat-completions (push) Has been cancelled
详情
PR Test (SMG) / chat-completions-4gpu (push) Has been cancelled
详情
PR Test (SMG) / e2e (push) Has been cancelled
详情
PR Test (SMG) / docker-build-test (push) Has been cancelled
详情
PR Test (SMG) / k8s-integration (push) Has been cancelled
详情
PR Test (SMG) / finish (push) Has been cancelled
详情
PR Test (SMG) / summarize-benchmarks (push) Has been cancelled
详情
Release SGLang Model Gateway Docker Image / publish (push) Has been cancelled
详情
Release SGLang Model Gateway to PyPI / build on macos (aarch64 - auto) (push) Has been cancelled
详情
Release SGLang Model Gateway to PyPI / build on linux (aarch64 - auto) (push) Has been cancelled
详情
Release SGLang Model Gateway to PyPI / build on linux (x86_64 - auto) (push) Has been cancelled
详情
Release SGLang Model Gateway to PyPI / build on linux (aarch64 - musllinux_1_1) (push) Has been cancelled
详情
Release SGLang Model Gateway to PyPI / build on linux (x86_64 - musllinux_1_1) (push) Has been cancelled
详情
Release SGLang Model Gateway to PyPI / Build SDist (push) Has been cancelled
详情
Release SGLang Model Gateway to PyPI / Upload to PyPI (push) Has been cancelled
详情
Release SGLang Kernels / build-cu129-matrix (aarch64, 12.9, 3.10, arm-kernel-build-node) (push) Has been cancelled
详情
Release SGLang Kernels / build-cu129-matrix (x86_64, 12.9, 3.10, x64-kernel-build-node) (push) Has been cancelled
详情
Release SGLang Kernels / release-cu129 (push) Has been cancelled
详情
Release SGLang Kernels / build-cu130-matrix (aarch64, 13.0, 3.10, arm-kernel-build-node) (push) Has been cancelled
详情
Release SGLang Kernels / build-cu130-matrix (x86_64, 13.0, 3.10, x64-kernel-build-node) (push) Has been cancelled
详情
Release SGLang Kernels / release-cu130 (push) Has been cancelled
详情
Release SGLang Kernels / build-rocm-matrix (3.10, 700) (push) Has been cancelled
详情
Release SGLang Kernels / build-rocm-matrix (3.10, 720) (push) Has been cancelled
详情
Release SGLang Kernels / release-rocm700 (push) Has been cancelled
详情
Release SGLang Kernels / release-rocm720 (push) Has been cancelled
详情
Release SGLang Kernels / build-musa43 (43, 3.10) (push) Has been cancelled
详情
Release SGLang Kernels / release-musa43 (push) Has been cancelled
详情
chore: import upstream snapshot with attribution
2026-07-13 12:38:16 +08:00

509 行
15 KiB
Python
原始文件 永久链接 Blame 文件历史

import os
import unittest
from types import SimpleNamespace
from sglang.srt.environ import envs
from sglang.test.ci.ci_register import register_cuda_ci
from sglang.test.run_eval import run_eval
from sglang.test.server_fixtures.disaggregation_fixture import (
PDDisaggregationServerBase,
)
from sglang.test.test_utils import (
DEFAULT_MODEL_NAME_FOR_TEST,
DEFAULT_MODEL_NAME_FOR_TEST_MLA,
DEFAULT_TIMEOUT_FOR_SERVER_LAUNCH,
popen_launch_pd_server,
try_cached_model,
)
register_cuda_ci(est_time=375, stage="base-c", runner_config="8-gpu-h20")
class TestDisaggregationMooncakePrefillLargerTP(PDDisaggregationServerBase):
@classmethod
def setUpClass(cls):
super().setUpClass()
# Temporarily disable JIT DeepGEMM
envs.SGLANG_ENABLE_JIT_DEEPGEMM.set(False)
cls.model = try_cached_model(DEFAULT_MODEL_NAME_FOR_TEST_MLA)
# Non blocking start servers
cls.start_prefill()
cls.start_decode()
# Block until both
cls.wait_server_ready(cls.prefill_url + "/health", process=cls.process_prefill)
cls.wait_server_ready(cls.decode_url + "/health", process=cls.process_decode)
cls.launch_lb()
@classmethod
def start_prefill(cls):
prefill_args = [
"--trust-remote-code",
"--disaggregation-mode",
"prefill",
"--disaggregation-bootstrap-port",
cls.bootstrap_port,
"--tp",
"4",
"--enable-metrics",
"--enable-request-time-stats-logging",
]
prefill_args += cls.transfer_backend + cls.rdma_devices
cls.process_prefill = popen_launch_pd_server(
cls.model,
cls.prefill_url,
timeout=DEFAULT_TIMEOUT_FOR_SERVER_LAUNCH,
other_args=prefill_args,
)
@classmethod
def start_decode(cls):
decode_args = [
"--trust-remote-code",
"--disaggregation-mode",
"decode",
"--disaggregation-bootstrap-port",
cls.bootstrap_port,
"--tp",
"2",
"--base-gpu-id",
"4",
"--enable-metrics",
"--enable-request-time-stats-logging",
]
decode_args += cls.transfer_backend + cls.rdma_devices
cls.process_decode = popen_launch_pd_server(
cls.model,
cls.decode_url,
timeout=DEFAULT_TIMEOUT_FOR_SERVER_LAUNCH,
other_args=decode_args,
)
def test_gsm8k(self):
args = SimpleNamespace(
base_url=self.base_url,
model=self.model,
eval_name="gsm8k",
api="completion",
max_tokens=512,
num_examples=200,
num_threads=128,
)
metrics = run_eval(args)
print(f"Evaluation metrics: {metrics}")
self.assertGreater(metrics["score"], 0.60)
class TestDisaggregationMooncakeDecodeLargerTP(PDDisaggregationServerBase):
@classmethod
def setUpClass(cls):
super().setUpClass()
# Temporarily disable JIT DeepGEMM
envs.SGLANG_ENABLE_JIT_DEEPGEMM.set(False)
cls.model = try_cached_model(DEFAULT_MODEL_NAME_FOR_TEST_MLA)
# Non blocking start servers
cls.start_prefill()
cls.start_decode()
# Block until both
cls.wait_server_ready(cls.prefill_url + "/health", process=cls.process_prefill)
cls.wait_server_ready(cls.decode_url + "/health", process=cls.process_decode)
cls.launch_lb()
@classmethod
def start_prefill(cls):
prefill_args = [
"--trust-remote-code",
"--disaggregation-mode",
"prefill",
"--disaggregation-bootstrap-port",
cls.bootstrap_port,
"--tp",
"2",
"--enable-metrics",
"--enable-request-time-stats-logging",
]
prefill_args += cls.transfer_backend + cls.rdma_devices
cls.process_prefill = popen_launch_pd_server(
cls.model,
cls.prefill_url,
timeout=DEFAULT_TIMEOUT_FOR_SERVER_LAUNCH,
other_args=prefill_args,
)
@classmethod
def start_decode(cls):
decode_args = [
"--trust-remote-code",
"--disaggregation-mode",
"decode",
"--disaggregation-bootstrap-port",
cls.bootstrap_port,
"--tp",
"4",
"--base-gpu-id",
"4",
"--enable-metrics",
"--enable-request-time-stats-logging",
]
decode_args += cls.transfer_backend + cls.rdma_devices
cls.process_decode = popen_launch_pd_server(
cls.model,
cls.decode_url,
timeout=DEFAULT_TIMEOUT_FOR_SERVER_LAUNCH,
other_args=decode_args,
)
def test_gsm8k(self):
args = SimpleNamespace(
base_url=self.base_url,
model=self.model,
eval_name="gsm8k",
api="completion",
max_tokens=512,
num_examples=200,
num_threads=128,
)
metrics = run_eval(args)
print(f"Evaluation metrics: {metrics}")
self.assertGreater(metrics["score"], 0.60)
class TestDisaggregationMooncakeMHAPrefillLargerTP(PDDisaggregationServerBase):
@classmethod
def setUpClass(cls):
super().setUpClass()
# Temporarily disable JIT DeepGEMM
envs.SGLANG_ENABLE_JIT_DEEPGEMM.set(False)
cls.model = try_cached_model(DEFAULT_MODEL_NAME_FOR_TEST)
# Non blocking start servers
cls.start_prefill()
cls.start_decode()
# Block until both
cls.wait_server_ready(cls.prefill_url + "/health", process=cls.process_prefill)
cls.wait_server_ready(cls.decode_url + "/health", process=cls.process_decode)
cls.launch_lb()
@classmethod
def start_prefill(cls):
prefill_args = [
"--trust-remote-code",
"--disaggregation-mode",
"prefill",
"--disaggregation-bootstrap-port",
cls.bootstrap_port,
"--tp",
"4",
"--enable-metrics",
"--enable-request-time-stats-logging",
]
prefill_args += cls.transfer_backend + cls.rdma_devices
cls.process_prefill = popen_launch_pd_server(
cls.model,
cls.prefill_url,
timeout=DEFAULT_TIMEOUT_FOR_SERVER_LAUNCH,
other_args=prefill_args,
)
@classmethod
def start_decode(cls):
decode_args = [
"--trust-remote-code",
"--disaggregation-mode",
"decode",
"--disaggregation-bootstrap-port",
cls.bootstrap_port,
"--tp",
"2",
"--base-gpu-id",
"4",
"--enable-metrics",
"--enable-request-time-stats-logging",
]
decode_args += cls.transfer_backend + cls.rdma_devices
cls.process_decode = popen_launch_pd_server(
cls.model,
cls.decode_url,
timeout=DEFAULT_TIMEOUT_FOR_SERVER_LAUNCH,
other_args=decode_args,
)
def test_gsm8k(self):
args = SimpleNamespace(
base_url=self.base_url,
model=self.model,
eval_name="gsm8k",
api="completion",
max_tokens=512,
num_examples=200,
num_threads=128,
)
metrics = run_eval(args)
print(f"Evaluation metrics: {metrics}")
self.assertGreater(metrics["score"], 0.60)
class TestDisaggregationMooncakeMHADecodeLargerTP(PDDisaggregationServerBase):
@classmethod
def setUpClass(cls):
super().setUpClass()
# Temporarily disable JIT DeepGEMM
envs.SGLANG_ENABLE_JIT_DEEPGEMM.set(False)
cls.model = try_cached_model(DEFAULT_MODEL_NAME_FOR_TEST)
# Non blocking start servers
cls.start_prefill()
cls.start_decode()
# Block until both
cls.wait_server_ready(cls.prefill_url + "/health", process=cls.process_prefill)
cls.wait_server_ready(cls.decode_url + "/health", process=cls.process_decode)
cls.launch_lb()
@classmethod
def start_prefill(cls):
prefill_args = [
"--trust-remote-code",
"--disaggregation-mode",
"prefill",
"--disaggregation-bootstrap-port",
cls.bootstrap_port,
"--tp",
"2",
"--enable-metrics",
"--enable-request-time-stats-logging",
]
prefill_args += cls.transfer_backend + cls.rdma_devices
cls.process_prefill = popen_launch_pd_server(
cls.model,
cls.prefill_url,
timeout=DEFAULT_TIMEOUT_FOR_SERVER_LAUNCH,
other_args=prefill_args,
)
@classmethod
def start_decode(cls):
decode_args = [
"--trust-remote-code",
"--disaggregation-mode",
"decode",
"--disaggregation-bootstrap-port",
cls.bootstrap_port,
"--tp",
"4",
"--base-gpu-id",
"4",
"--enable-metrics",
"--enable-request-time-stats-logging",
]
decode_args += cls.transfer_backend + cls.rdma_devices
cls.process_decode = popen_launch_pd_server(
cls.model,
cls.decode_url,
timeout=DEFAULT_TIMEOUT_FOR_SERVER_LAUNCH,
other_args=decode_args,
)
def test_gsm8k(self):
args = SimpleNamespace(
base_url=self.base_url,
model=self.model,
eval_name="gsm8k",
api="completion",
max_tokens=512,
num_examples=200,
num_threads=128,
)
metrics = run_eval(args)
print(f"Evaluation metrics: {metrics}")
self.assertGreater(metrics["score"], 0.60)
STAGING_ENV = {
"SGLANG_DISAGG_STAGING_BUFFER": "1",
"SGLANG_DISAGG_STAGING_BUFFER_SIZE_MB": "64",
"SGLANG_DISAGG_STAGING_POOL_SIZE_MB": "1024",
}
class TestDisaggregationStagingPrefillLargerTP(PDDisaggregationServerBase):
"""Prefill TP=4 -> Decode TP=2 with staging buffer enabled (MHA model)."""
@classmethod
def setUpClass(cls):
super().setUpClass()
envs.SGLANG_ENABLE_JIT_DEEPGEMM.set(False)
cls.model = try_cached_model(DEFAULT_MODEL_NAME_FOR_TEST)
cls.start_prefill()
cls.start_decode()
cls.wait_server_ready(cls.prefill_url + "/health", process=cls.process_prefill)
cls.wait_server_ready(cls.decode_url + "/health", process=cls.process_decode)
cls.launch_lb()
@classmethod
def start_prefill(cls):
prefill_args = [
"--trust-remote-code",
"--disaggregation-mode",
"prefill",
"--disaggregation-bootstrap-port",
cls.bootstrap_port,
"--tp",
"4",
"--enable-metrics",
"--enable-request-time-stats-logging",
]
prefill_args += cls.transfer_backend + cls.rdma_devices
env = {**os.environ, **STAGING_ENV}
cls.process_prefill = popen_launch_pd_server(
cls.model,
cls.prefill_url,
timeout=DEFAULT_TIMEOUT_FOR_SERVER_LAUNCH,
other_args=prefill_args,
env=env,
)
@classmethod
def start_decode(cls):
decode_args = [
"--trust-remote-code",
"--disaggregation-mode",
"decode",
"--disaggregation-bootstrap-port",
cls.bootstrap_port,
"--tp",
"2",
"--base-gpu-id",
"4",
"--enable-metrics",
"--enable-request-time-stats-logging",
]
decode_args += cls.transfer_backend + cls.rdma_devices
env = {**os.environ, **STAGING_ENV}
cls.process_decode = popen_launch_pd_server(
cls.model,
cls.decode_url,
timeout=DEFAULT_TIMEOUT_FOR_SERVER_LAUNCH,
other_args=decode_args,
env=env,
)
def test_gsm8k(self):
args = SimpleNamespace(
base_url=self.base_url,
model=self.model,
eval_name="gsm8k",
api="completion",
max_tokens=512,
num_examples=200,
num_threads=128,
)
metrics = run_eval(args)
print(f"[Staging PrefillLargerTP] Evaluation metrics: {metrics}")
self.assertGreater(metrics["score"], 0.60)
class TestDisaggregationStagingDecodeLargerTP(PDDisaggregationServerBase):
"""Prefill TP=2 -> Decode TP=4 with staging buffer enabled (MHA model)."""
@classmethod
def setUpClass(cls):
super().setUpClass()
envs.SGLANG_ENABLE_JIT_DEEPGEMM.set(False)
cls.model = try_cached_model(DEFAULT_MODEL_NAME_FOR_TEST)
cls.start_prefill()
cls.start_decode()
cls.wait_server_ready(cls.prefill_url + "/health", process=cls.process_prefill)
cls.wait_server_ready(cls.decode_url + "/health", process=cls.process_decode)
cls.launch_lb()
@classmethod
def start_prefill(cls):
prefill_args = [
"--trust-remote-code",
"--disaggregation-mode",
"prefill",
"--disaggregation-bootstrap-port",
cls.bootstrap_port,
"--tp",
"2",
"--enable-metrics",
"--enable-request-time-stats-logging",
]
prefill_args += cls.transfer_backend + cls.rdma_devices
env = {**os.environ, **STAGING_ENV}
cls.process_prefill = popen_launch_pd_server(
cls.model,
cls.prefill_url,
timeout=DEFAULT_TIMEOUT_FOR_SERVER_LAUNCH,
other_args=prefill_args,
env=env,
)
@classmethod
def start_decode(cls):
decode_args = [
"--trust-remote-code",
"--disaggregation-mode",
"decode",
"--disaggregation-bootstrap-port",
cls.bootstrap_port,
"--tp",
"4",
"--base-gpu-id",
"4",
"--enable-metrics",
"--enable-request-time-stats-logging",
]
decode_args += cls.transfer_backend + cls.rdma_devices
env = {**os.environ, **STAGING_ENV}
cls.process_decode = popen_launch_pd_server(
cls.model,
cls.decode_url,
timeout=DEFAULT_TIMEOUT_FOR_SERVER_LAUNCH,
other_args=decode_args,
env=env,
)
def test_gsm8k(self):
args = SimpleNamespace(
base_url=self.base_url,
model=self.model,
eval_name="gsm8k",
api="completion",
max_tokens=512,
num_examples=200,
num_threads=128,
)
metrics = run_eval(args)
print(f"[Staging DecodeLargerTP] Evaluation metrics: {metrics}")
self.assertGreater(metrics["score"], 0.60)
if __name__ == "__main__":
unittest.main()
在新工单中引用 查看 Git Blame 复制永久链接