AI is easy to use — but hard to scale.

In this episode of Domesticating AI, we’re joined by Daniel Dowler (Red Hat) to break down what actually happens when you move from calling APIs to running AI systems yourself.

Recorded on April 21st

Most developers interact with AI through APIs — fast, simple, and pay-per-token. But behind the scenes, those systems rely on GPU scheduling, batching, and infrastructure that doesn’t behave like traditional software.

We cover:

Why GPU scaling is fundamentally different from CPU scaling
Why tools like vLLM are becoming the default for high-performance inference
How Ray and Kubernetes fit into real-world AI systems
What parallelism (tensor, data, expert) actually means in practice
When self-hosting AI makes sense
When APIs are still the better choice

Claude Opus 4.7
https://www.anthropic.com/news/claude-opus-4-7
Qwen 3.6 (Alibaba)
https://qwen.ai/research
Kimi K2.6 (community discussion)
https://www.reddit.com/r/LocalLLaMA/s/kvRWb7uJgM

vLLM → https://github.com/vllm-project/vllm
Ray → https://github.com/ray-project/ray
Kubernetes → https://kubernetes.io
Kueue → https://kueue.sigs.k8s.io
LiteLLM → https://github.com/BerriAI/litellm
KServe → https://kserve.github.io

Daniel Dowler
Platform engineer at Red Hat focused on Kubernetes and AI infrastructure. Daniel works on how modern systems support real workloads, including GPU scheduling, distributed inference, and scaling AI in production environments. He recently spoke at Machine Learning Utah on AI infrastructure and clustering.

You don’t scale AI with replicas.

You scale it by managing scarce compute.

Subscribe on Spotify or Apple, and follow us on YouTube.

👉 Keep your AI on a leash.

🧠 News🔗 Tools & Tech Mentioned👤 Guest🎯 Key Takeaway