mirrors / light-heart-labs--ods

项目文件夹

light-heart-labs--ods
文件
main
T
wehub-resource-sync 491752a783 chore: import upstream snapshot with attribution
2026-07-13 12:34:03 +08:00

9.5 KiB
原始文件 永久链接 Blame 文件历史

Intel Arc GPU Guide

Last updated: 2026-03-17

ODS supports Intel Arc discrete GPUs via the llama.cpp SYCL backend (docker-compose.arc.yml). This guide covers supported hardware, driver setup, known limitations, and performance expectations.

Supported Hardware

Tier: ARC (≥ 12 GB VRAM)

GPU VRAM Estimated tok/s Concurrent users Model
Arc A770 16 GB ~35 3–5 Qwen3.5 9B Q4_K_M
Arc B580 12 GB ~30 2–4 Qwen3.5 9B Q4_K_M

Tier: ARC_LITE (< 12 GB VRAM)

GPU VRAM Estimated tok/s Concurrent users Model
Arc A750 8 GB ~20 1–2 Qwen3.5 4B Q4_K_M
Arc A380 6 GB ~15 1 Qwen3.5 4B Q4_K_M
Arc A310 4 GB ~10 1 Qwen3.5 4B Q4_K_M (tight)

A310 note: 4 GB VRAM is borderline for Qwen3.5 4B Q4_K_M (~3.3 GB). The model will load but leaves little headroom for KV cache. Consider --ctx-size 4096 (set CTX_SIZE=4096 in .env) to reduce pressure.

Future / untested

Intel Arc B-series (Battlemage) cards ≥ 12 GB will automatically map to the ARC tier. Cards < 12 GB will map to ARC_LITE. Battlemage introduced 0x7d PCI device IDs; detect_gpu() in installers/lib/detection.sh may need an update when those cards become more widely available.

Host Driver Setup (Ubuntu / Debian)

1 — Add Intel GPU repository

# Import Intel GPG key and repo
wget -qO - https://repositories.intel.com/gpu/intel-graphics.key \
  | sudo gpg --dearmor -o /usr/share/keyrings/intel-graphics.gpg

echo "deb [arch=amd64 signed-by=/usr/share/keyrings/intel-graphics.gpg] \
  https://repositories.intel.com/gpu/ubuntu jammy unified" \
  | sudo tee /etc/apt/sources.list.d/intel-gpu-jammy.list

sudo apt update

2 — Install kernel and user-mode drivers

# Kernel module + firmware (i915 / xe)
sudo apt install -y linux-headers-$(uname -r) \
    intel-i915-dkms intel-fw-gpu

# Level Zero runtime (required for SYCL)
sudo apt install -y intel-level-zero-gpu level-zero

# OpenCL runtime (required for llama.cpp OpenCL fallback)
sudo apt install -y intel-opencl-icd

# Monitoring tools (optional but recommended)
sudo apt install -y intel-gpu-tools clinfo

3 — Add user to GPU groups

sudo usermod -aG video,render $USER
# Re-login (or newgrp render) for the change to take effect

4 — Verify

# Should list Intel Arc as an OpenCL device
clinfo | grep -A3 "Device Name"

# Should show Level Zero GPU
ze_info 2>/dev/null | grep -i "device name" || \
    ldconfig -p | grep libze_loader

# Should show render node
ls -la /dev/dri/renderD*

# Live GPU usage (Ctrl+C to exit)
sudo intel_gpu_top

Installation

The ODS installer auto-detects Intel Arc and selects the correct tier:

# Automatic (recommended)
./install.sh

# Force a specific tier manually
./install.sh --tier ARC
./install.sh --tier ARC_LITE

What the installer does for Intel Arc

  1. Phase 01 (preflight) — checks disk space (≥ 20 GB for model download)
  2. Phase 02 (detection) — confirms Arc via lspci + sysfs, validates Level Zero, /dev/dri, intel_gpu_top, and video/render group membership
  3. Phase 05 (docker) — validates docker-compose.arc.yml syntax
  4. Phase 06 (directories) — writes .env with GPU_BACKEND=sycl, N_GPU_LAYERS=99, VIDEO_GID, RENDER_GID, and Intel oneAPI env vars
  5. Phase 07 (devtools) — installs OpenCode and CLI tooling
  6. Phase 08 (launch) — runs docker compose -f docker-compose.base.yml -f docker-compose.arc.yml up -d

Docker Compose Overlay

ODS provides two Intel Arc overlays:

File Image When to use
docker-compose.arc.yml Built locally from intel/oneapi-basekit Default. Requires docker compose up --build on first run (~10–20 min).
docker-compose.intel.yml Pre-built ghcr.io/ggml-org/llama.cpp:server-intel-* Quick start — no build time. Set LLAMA_ARC_IMAGE=<tag> in .env.

Manual compose start

# Build and start (first time ~10–20 min build)
docker compose -f docker-compose.base.yml -f docker-compose.arc.yml up -d --build

# Subsequent starts (no rebuild)
docker compose -f docker-compose.base.yml -f docker-compose.arc.yml up -d

# Skip local build — use a pre-built image
LLAMA_ARC_IMAGE=ghcr.io/ggml-org/llama.cpp:server-intel-b8248 \
  docker compose -f docker-compose.base.yml -f docker-compose.arc.yml up -d

Key .env variables for Arc

GPU_BACKEND=sycl
N_GPU_LAYERS=99
VIDEO_GID=44          # auto-set by installer
RENDER_GID=992        # auto-set by installer
ONEAPI_DEVICE_SELECTOR=level_zero:gpu
SYCL_CACHE_PERSISTENT=1
ZES_ENABLE_SYSMAN=1
CTX_SIZE=32768        # ARC tier default

Known Limitations vs NVIDIA / AMD

Feature NVIDIA (CUDA) AMD (ROCm) Intel Arc (SYCL)
Installer maturity Tier B Tier A Tier C (experimental)
llama.cpp backend CUDA (native) HIP/ROCm (native) SYCL (via oneAPI)
SYCL kernel cache — — First-run JIT compile per container start (~30 s). Eliminated after first run with SYCL_CACHE_PERSISTENT=1.
Multi-GPU ✅ (native) ✅ (ROCm multi) ❌ Not supported. SYCL backend targets a single Arc GPU.
ComfyUI (image gen) ✅ CUDA overlay ✅ ROCm overlay ⚠️ No dedicated overlay. ComfyUI will use CPU fallback.
Whisper STT ✅ CUDA overlay ✅ ROCm overlay ⚠️ Runs on CPU (no Arc-accelerated Whisper image).
Flash attention ✅ ✅ ❌ llama.cpp SYCL does not yet implement Flash Attention.
FP16 compute ✅ Full ✅ Full ✅ Enabled (GGML_SYCL_F16=ON) — Arc FP16 throughput is competitive at this model size.
Docker image size ~6 GB ~8 GB ~15 GB (oneAPI Base Toolkit is large).
First-run build time Pull only Pull only ~10–20 min (compiles llama.cpp from source).
Windows support ✅ WSL2 ✅ WSL2 ⚠️ Experimental. Arc drivers for WSL2 are less mature than NVIDIA's.

Performance Expectations

Performance figures below are measured with Qwen3 models at Q4_K_M quantisation, --n-gpu-layers 99 (all layers on GPU), --ctx-size 16384.

GPU Model Prompt tok/s Generate tok/s Notes
Arc A770 (16 GB) Qwen3.5 9B Q4_K_M ~120 ~35 Comfortable fit; KV cache well within VRAM
Arc A750 (8 GB) Qwen3.5 4B Q4_K_M ~90 ~20 Model fits; limit CTX_SIZE to ≤ 16384
Arc A380 (6 GB) Qwen3.5 4B Q4_K_M ~70 ~15 Tight. Set CTX_SIZE=8192 for safety

Comparison to equivalent NVIDIA tiers

Intel Arc Comparable NVIDIA VRAM Generate tok/s delta
A770 (ARC) RTX 3060 12 GB (T1) 16 vs 12 GB Arc ~+5 tok/s on 8B (more VRAM headroom)
A750 (ARC_LITE) RTX 3060 12 GB (T1) 8 vs 12 GB Arc ~-10 tok/s (less VRAM, smaller model)

Intel Arc SYCL throughput is broadly similar to an equivalent NVIDIA card at the same VRAM tier. Arc's primary advantage is value (A770 16 GB retails at ~$250–300) rather than raw throughput.

Troubleshooting

llama-server exits immediately with SYCL error

SYCL error: code 6, ZE_RESULT_ERROR_DEVICE_LOST

Cause: Level Zero cannot enumerate a GPU device. Fix:

# Verify host driver
clinfo | grep "Device Name"
# If empty:
sudo apt install intel-level-zero-gpu level-zero
# Then restart the container
docker compose restart llama-server

Slow first inference after container start

Cause: SYCL kernel JIT compilation on first call (~20–60 s). Fix: Ensure SYCL_CACHE_PERSISTENT=1 is set in .env (the installer sets this automatically). Subsequent runs use the compiled kernel cache and start in < 5 s.

/dev/dri not found inside container

Error opening /dev/dri/renderD128: Permission denied

Cause: User not in render group, or Docker socket not passed through. Fix:

sudo usermod -aG render $USER
# Re-login, then:
docker compose -f docker-compose.base.yml -f docker-compose.arc.yml up -d

Container fails to start on WSL2

Intel Arc drivers on WSL2 are less mature than NVIDIA's. If the Arc GPU is not visible inside WSL2:

  1. Update Windows to the latest version (22H2+).
  2. Install the latest Intel Graphics driver from intel.com/arc-drivers.
  3. Verify the GPU is visible: wsl -- ls /dev/dri
  4. If still missing, fall back to CPU mode: ./install.sh --tier 1 (runs inference on CPU, no GPU passthrough).

intel_gpu_top shows 0% GPU engine utilisation during inference

This is a known display quirk when the compute engine is used heavily — intel_gpu_top sometimes under-reports Arc engine utilisation in older versions of intel-gpu-tools. Verify the model is actually running on GPU by checking VRAM:

# Should show non-zero VRAM used
sudo intel_gpu_top -l 1 | grep -i mem

Related Docs

在新工单中引用 查看 Git Blame 复制永久链接