Every day billions of voice conversations happen across games, customer service calls, and financial transactions. Almost none of them are understood by machines. In this episode of Inference Time Tactics, Calvin Cooper and Yash Sharma sit down with Carter Huffman, CTO and co founder of Modulate, to explore the AI systems that can finally understand voice conversations in real time.

Modulate’s model Velma 2.0 powers voice intelligence across industries. From moderating voice chat in games like Call of Duty to detecting fraud in financial calls and analyzing customer support conversations, their system uses ensembles of specialized models to capture tone, intent, emotion, and conversational dynamics. Instead of relying on giant foundation models, Velma orchestrates over 100 specialized models to deliver higher accuracy at dramatically lower cost.

We talked about:

The challenge of processing a trillion hours of annual global voice traffic.
Scaling real-time moderation for massive platforms like Call of Duty.
Capturing nuance, tone, and sarcasm beyond basic text transcripts.
Ensemble architecture utilizing over 100 specialized models.
Orchestration layers that trim compute costs by identifying optimal model subsets.
Achieving order-of-magnitude cost savings compared to large foundational models.
Applying "exploration vs. exploitation" optimization to shifting conversation data.
Future development of "context graphs" to map participant intent and causality.

Resources Mentioned:

NeuroMetric Audio Leaderboard: https://leaderboard.neurometric.ai/?leaderboard=audio

Connect with Modulate:
Website: https://www.modulate.ai/

LinkedIn: https://www.linkedin.com/in/carter-huffman-a9aba05b

Velma: https://www.modulate.ai/velma

Connect with Neurometric:
Website: https://www.neurometric.ai/

Substack: https://neurometric.substack.com/

X: https://x.com/neurometricai/

Bluesky: https://bsky.app/profile/neurometric.bsky.social

Hosts:

Calvin Cooper

https://x.com/cooper_nyc_

https://www.linkedin.com/in/coopernyc

Yash Sharma

https://x.com/yash_j_sharma

https://www.linkedin.com/in/yashjsharma/