← Journal · · Note
Kontur podcast on LLM production inference: vllm, sglang and optimization
I recommend checking out the Kontur podcast on how they do production LLM inference. It's not often that people in the Russian market talk about this openly, but there's a lot to learn from their experience operating vllm/sglang, why they didn't go with Triton, and how to optimize inference for high traffic loads.
I actively use their summarizers at work - solid tool, happy to see a domestic product like this. I know Azik personally, he's doing great work :)