/llm-optimizer

大模型推理优化

综述

• LLM Inference Serving: Survey of Recent Advances and Opportunities

• Towards Efficient Generative Large Language Model Serving: A Survey from Algorithms to Systems

• A Survey on Efficient Inference for Large Language Models

• A Survey on Model Compression for Large Language Models

• BlendServe: Optimizing Offline Inference for Auto-regressive Large Models with Resource-aware Batching：https://arxiv.org/pdf/2411.16102

• FLUX: Fast Software-based Communication Overlap On GPUs Through Kernel Fusion: https://arxiv.org/abs/2406.06858

• BatchLLM: Optimizing Large Batched LLM Inference with Global Prefix Sharing and Throughput-oriented Token Batching：https://arxiv.org/abs/2412.03594