Building a Custom Claude Agent OS: Voice Agents, Orchestration, Memory Loops & Model Swapping

The speaker demos a custom “Claude agent operating system” built over two months to unify tools like Fable 5 and Hermes into a single, customizable mission-control UI with shared memory and organized outputs. They show how features evolved from simple chat into multiple model profiles, Hermes Mixture of Agents, a wake-word voice agent that can control the browser and local actions, and Hermes Oracle for pulling and linking breaking news and drafting social content. The system includes agent orchestration via group chat, pipelines, and a Paperclip-style org chart with departments, an inbox, dashboards, and progress history, plus Kanban workflows and integrations like NotebookLM video generation. It’s designed to swap models/CLIs quickly, run scheduled “loop” automations (outreach, news refresh), and maintain a growing memory/notes system. The OS and tutorials are offered through the AI Profit Boardroom.

00:00 Agent OS Overview
00:21 Why Mission Control
01:12 Profiles and Mixture
02:47 Keep It Simple
03:11 Voice Agent Demo
04:14 Oracle News and Content
04:29 Building With Claude
05:18 Agent Mastermind Chat
06:05 Paperclip Agent Org
08:07 Swap Models Fast
09:16 Kanban and Video Tools
10:12 Loop Engineering Automation
11:24 Memory Feedback System
12:12 Community and Access
13:14 Courses Coaching Wrap
14:00 Final Thanks