← Journal · · Note
KubeCon 2024: six talks on GPU and what to take from them
Remember back in June when I promised: if we hit 1000 subscribers, I'll fly to KubeCon in Japan?
We hit it, I went, and now there's an article on Habr 🔥
By day two, I realized everyone had the same slides: quotas, tenant isolation, bin packing, and "GPUs are expensive - let's share them." So I analyzed six talks on GPU utilization and compared them with what we're doing.
Here's a quick summary from the article:
1️⃣ Inference became the main GPU consumer. Two-thirds of AI compute goes to inference, and the CNCF ecosystem is pivoting toward serving.
2️⃣ Everyone's sharing GPUs, and the slides are all the same. Multi-tenancy, quotas, and tight packing became the conference's main trend. Kueue, HAMi, vCluster, and HNC are different points on the same scale.
3️⃣ llm-d is becoming the LLM inference standard. KV-aware routing gives you a proportional win almost for free, while P/D without tuning easily makes things worse.
4️⃣ DRA became the foundation for new projects. GPU topology + NIC and composable hardware are already being built on it, so you should plan for migration now.
5️⃣ People come to KubeCon for open source. You can take almost everything from the talks home with you, plus talk to actual project maintainers.
And at the end - my impressions of how KubeCon differs from our conferences, plus some Japan: ramen, convenience stores, and Mount Fuji that never came out of the clouds 😅
Grab something strong (like tea) and dive in!