GPU Time-Slicing for Concurrent LLM Agents on Kubernetes
English summary
A systems-level deep dive that exposes the hidden microarchitectural costs of GPU time-slicing in Kubernetes when running concurrent LLM agents. It quantifies the actual overhead of co-locating agentic AI workloads and explains what it means for operational efficiency.
Chinese summary
这篇文章系统性地探讨了在 Kubernetes 上运行并发 LLM Agent 时，GPU 时间切片带来的隐藏微观架构成本。文章量化了共同调度 Agentic AI 工作负载的额外开销，并阐释了对运行效率的影响。
Key points
Reveals hidden microarchitectural overhead of GPU time-slicing.
揭示了 GPU 时间切片的隐藏微观架构开销。
Focuses specifically on concurrent LLM agent workloads.
聚焦于并发 LLM Agent 工作负载。
Quantifies co-location costs for agentic AI on Kubernetes.
量化了在 Kubernetes 上共同调度 Agentic AI 的成本。