/recipes/Baidu/ERNIE-Image.md

ERNIE-Image

Text-to-image online serving (ERNIE-Image 8B)

Summary

• Vendor: Baidu
• Model: baidu/ERNIE-Image / baidu/ERNIE-Image-Turbo
• Task: Text-to-image generation
• Mode: Online serving with the OpenAI-compatible API
• Maintainer: Community

When to use this recipe

Use this recipe when you want a known-good starting point for serving baidu/ERNIE-Image or baidu/ERNIE-Image-Turbo with vLLM-Omni for high-quality text-to-image generation.

ERNIE-Image is an 8B-parameter Diffusion Transformer (DiT) model that achieves state-of-the-art performance among open-weight text-to-image models. Key strengths include:

• Text rendering: Excellent for dense, long-form, and layout-sensitive text — ideal for posters, infographics, and UI-like images.
• Instruction following: Reliably handles complex prompts with multiple objects, detailed relationships, and knowledge-intensive descriptions.
• Structured generation: Effective for posters, comics, storyboards, and multi-panel compositions where layout matters.
• Style coverage: Supports realistic photography, design-oriented imagery, and stylized aesthetic outputs.
• Practical deployment: Can run on consumer GPUs with 24GB VRAM.

Two model variants are provided:

1. ERNIE-Image — The SFT model with stronger general-purpose capability and instruction fidelity, typically using 50 inference steps.
2. ERNIE-Image-Turbo — Optimized by DMD and RL for faster speed and higher aesthetics, requiring only 8 inference steps.

References

• Upstream model card: https://huggingface.co/baidu/ERNIE-Image
• Upstream model card (Turbo): https://huggingface.co/baidu/ERNIE-Image-Turbo
• GitHub: https://github.com/baidu/ERNIE-Image
• Blog: https://ernie-image.github.io/

Hardware Support

This recipe currently documents tested configurations for CUDA GPU serving. Add more sections for other hardware as community validation lands.

GPU

1 x RTX 4090 (Single GPU, 24GB VRAM, Minimum Recommended)

Command

ERNIE-Image (full model):

vllm serve baidu/ERNIE-Image --omni \
  --enable-layerwise-offload \
  --port 8091

ERNIE-Image-Turbo (distilled, faster):

vllm serve baidu/ERNIE-Image-Turbo --omni \
  --enable-layerwise-offload \
  --port 8091

2 x RTX 4090 (Multi-GPU, 24GB VRAM)

Command

ERNIE-Image (full model):

vllm serve baidu/ERNIE-Image --omni \
  --tensor-parallel-size 2 \
  --enable-cpu-offload \
  --port 8091

ERNIE-Image-Turbo (distilled, faster):

vllm serve baidu/ERNIE-Image-Turbo --omni \
  --tensor-parallel-size 2 \
  --enable-cpu-offload \
  --port 8091

Verification

After the server is ready, test with a simple request:

curl -X POST http://localhost:8091/v1/images/generations \
  -H "Content-Type: application/json" \
  -d '{
    "prompt": "A photo of a cat sitting on a laptop keyboard, digital art style.",
    "size": "1024x1024",
    "num_inference_steps": 50,
    "guidance_scale": 4.0,
    "seed": 42
  }' | jq -r '.data[0].b64_json' | base64 -d > output.png

For ERNIE-Image-Turbo, reduce the inference steps:

curl -X POST http://localhost:8091/v1/images/generations \
  -H "Content-Type: application/json" \
  -d '{
    "prompt": "A photo of a cat sitting on a laptop keyboard, digital art style.",
    "size": "1024x1024",
    "num_inference_steps": 8,
    "guidance_scale": 1.0,
    "seed": 42
  }' | jq -r '.data[0].b64_json' | base64 -d > output.png

Notes

• Memory usage: The 8B model requires significant VRAM. Use --enable-cpu-offload to reduce GPU memory footprint by offloading components to CPU when not in use.
• Key flags:
  • --omni — enables vLLM-Omni diffusion serving.
• Advanced features:
  • TP (Tensor Parallelism): --tensor-parallel-size <N> — distribute model weights across N GPUs.
  • SP (Sequence Parallelism): --usp <N> (Ulysses SP) and --ring <N> (Ring SP) for long-sequence workloads.
  • HSDP: --use-hsdp enables Hybrid Sharded Data Parallelism; use --hsdp-shard-size and --hsdp-replicate-size for fine-grained control.
  • Cache-DiT: --cache-backend cache_dit caches DiT intermediate outputs for faster generation; configure via --cache-config.
  • Layer offload: --enable-layerwise-offload offloads DiT layers to CPU for memory-constrained scenarios.
• Prompt Enhancer (PE): ERNIE-Image includes an optional 3B-parameter Prompt Enhancer model that expands brief user inputs into richer structured descriptions, improving output quality. Set use_pe=False in the request body to disable it if you prefer direct prompt processing or want to use larger LLMs (e.g., Gemini, ChatGPT) for prompt enhancement instead.
• Recommended settings:
  • ERNIE-Image: num_inference_steps=50, guidance_scale=4.0
  • ERNIE-Image-Turbo: num_inference_steps=8, guidance_scale=1.0