← Journal · · Talk sources
Following up on choosing infrastructure for LLMs
Hi! Thanks for listening to the talk ❤️
You can find the sources I used to prepare the material in this post.
I also open-sourced a whole lab for picking infrastructure for LLMs. You can pick the right GPU and inference configuration yourself after a couple of tests (for example, at my friends' place, in Selectel's cloud).
By the way, I recently stumbled on an interesting new project from k8s-sigs, inference-perf. It'll be interesting to compare it with GenAI Perf. If you've already tried it, share your thoughts in the comments :)
Also, as a bonus, I'm posting the pictures that didn't make it into the presentation but were part of the story too!
And now I'm off to drink some mead, skal to everyone!