You ask an AI a question and it answers with total confidence. Most of the time, a confidently wrong answer is just an annoyance. But what if the question is medical, and there's a real patient on the other end? In that world, a hallucination isn't a bug, it's a patient-safety event. Sumit Gundawar is a London-based software engineer who builds the clinical platform for a UK longevity and aesthetic-medicine clinic, and his whole argument is that in high-stakes AI, the model is the easy part. Earning trust is the real engineering. We dig into grounding, refusal logic, human-in-the-loop design, and the messy frontier of longevity and biohacking, plus a live demo of an assistant that refuses to answer when it can't back up the claim. Let's get into it.

Episode sponsors

Six Feet Up
Talk Python Courses

Links from the show

Guest
Sumit Gundawar: linkedin.com

Course transcripts announcement: talkpython.fm/blog

Sumit Gundawar - JAX London Speaker: jaxlondon.com
Anthropic: anthropic.com
OpenAI Platform: platform.openai.com
Anthropic: anthropic.com
LangChain: langchain.com
OWASP: owasp.org
Pydantic: pydantic.dev
EU AI Act - Regulatory Framework: digital-strategy.ec.europa.eu
HIPAA - HHS: www.hhs.gov
NHS: www.nhs.uk
Llama: llama.com
Qwen - QwenLM on GitHub: github.com
OpenAI Platform: platform.openai.com
Hugging Face: huggingface.co
Llama: llama.com
Granola: www.granola.ai
HIPAA - HHS: www.hhs.gov
CodeRabbit: www.coderabbit.ai
Cursor Origin: cursor.com
GitHub Status: www.githubstatus.com
Midjourney Medical: www.midjourney.com
Neko Health: www.nekohealth.com
CERN: home.cern
ATLAS Experiment: atlas.cern

Watch this episode on YouTube: youtube.com
Episode #554 deep-dive: talkpython.fm/554
Episode transcripts: talkpython.fm

Theme Song: Developer Rap
🥁 Served in a Flask 🎸: talkpython.fm/flasksong

---== Don't be a stranger ==---
YouTube: youtube.com/@talkpython

Bluesky: @talkpython.fm
Mastodon: @talkpython@fosstodon.org
X.com: @talkpython

Michael on Bluesky: @mkennedy.codes
Michael on Mastodon: @mkennedy@fosstodon.org
Michael on X.com: @mkennedy