Evan Conrad, co-founder of SF Compute, joined us to talk about how they started as an AI lab that avoided bankruptcy by selling GPU clusters, why CoreWeave financials look like a real estate business, and how GPUs are turning into a commodities market.

Chapters:

00:00:05 - Introductions

00:00:12 - Introduction of guest Evan Conrad from SF Compute

00:00:12 - CoreWeave Business Model Discussion

00:05:37 - CoreWeave as a Real Estate Business

00:08:59 - Interest Rate Risk and GPU Market Strategy Framework

00:16:33 - Why Together and DigitalOcean will lose money on their clusters

00:20:37 - SF Compute's AI Lab Origins

00:25:49 - Utilization Rates and Benefits of SF Compute Market Model

00:30:00 - H100 GPU Glut, Supply Chain Issues, and Future Demand Forecast

00:34:00 - P2P GPU networks

00:36:50 - Customer stories

00:38:23 - VC-Provided GPU Clusters and Credit Risk Arbitrage

00:41:58 - Market Pricing Dynamics and Preemptible GPU Pricing Model

00:48:00 - Future Plans for Financialization?

00:52:59 - Cluster auditing and quality control

00:58:00 - Futures Contracts for GPUs

01:01:20 - Branding and Aesthetic Choices Behind SF Compute

01:06:30 - Lessons from Previous Startups

01:09:07 - Hiring at SF Compute

Chapters

00:00:00 Introduction and Background
00:00:58 Analysis of GPU Business Models
00:01:53 Challenges with GPU Pricing
00:02:48 Revenue and Scaling with GPUs
00:03:46 Customer Sensitivity to GPU Pricing
00:04:44 Core Weave's Business Strategy
00:05:41 Core Weave's Market Perception
00:06:40 Hyperscalers and GPU Market Dynamics
00:07:37 Financial Strategies for GPU Sales
00:08:35 Interest Rates and GPU Market Risks
00:09:30 Optimal GPU Contract Strategies
00:10:27 Risks in GPU Market Contracts
00:11:25 Price Sensitivity and Market Competition
00:12:21 Market Dynamics and GPU Contracts
00:13:18 Hyperscalers and GPU Market Strategies
00:14:15 Nvidia and Market Competition
00:15:12 Microsoft's Role in GPU Market
00:16:10 Challenges in GPU Market Dynamics
00:17:07 Economic Realities of the GPU Market
00:18:03 Real Estate Model for GPU Clouds
00:18:59 Price Sensitivity and Chip Design
00:19:55 SF Compute's Beginnings and Challenges
00:20:54 Navigating the GPU Market
00:21:54 Pivoting to a GPU Cloud Provider
00:22:53 Building a GPU Market
00:23:52 SF Compute as a GPU Marketplace
00:24:49 Market Liquidity and GPU Pricing
00:25:47 Utilization Rates in GPU Markets
00:26:44 Brokerage and Market Flexibility
00:27:42 H100 Glut and Market Cycles
00:28:40 Supply Chain Challenges and GPU Glut
00:29:35 Future Predictions for the GPU Market
00:30:33 Speculations on Test Time Inference
00:31:29 Market Demand and Test Time Inference
00:32:26 Open Source vs. Closed AI Demand
00:33:24 Future of Inference Demand
00:34:24 Peer-to-Peer GPU Markets
00:35:17 Decentralized GPU Market Skepticism
00:36:15 Redesigning Architectures for New Markets
00:37:14 Supporting Grad Students and Startups
00:38:11 Successful Startups Using SF Compute
00:39:11 VCs and GPU Infrastructure
00:40:09 VCs as GPU Credit Transformators
00:41:06 Market Timing and GPU Infrastructure
00:42:02 Understanding GPU Pricing Dynamics
00:43:01 Market Pricing and Preemptible Compute
00:43:55 Price Volatility and Market Optimization
00:44:52 Customizing Compute Contracts
00:45:50 Creating Flexible Compute Guarantees
00:46:45 Financialization of GPU Markets
00:47:44 Building a Spot Market for GPUs
00:48:40 Auditing and Standardizing Clusters
00:49:40 Ensuring Cluster Reliability
00:50:36 Active Monitoring and Refunds
00:51:33 Automating Customer Refunds
00:52:33 Challenges in Cluster Maintenance
00:53:29 Remote Cluster Management
00:54:29 Standardizing Compute Contracts
00:55:28 Unified Infrastructure for Clusters
00:56:24 Creating a Commodity Market for GPUs
00:57:22 Futures Market and Risk Management
00:58:18 Reducing Risk with GPU Futures
00:59:14 Stabilizing the GPU Market
01:00:10 SF Compute's Anti-Hype Approach
01:01:07 Calm Branding and Expectations
01:02:07 Promoting San Francisco's Beauty
01:03:03 Design Philosophy at SF Compute
01:04:02 Artistic Influence on Branding
01:05:00 Past Projects and Burnout
01:05:59 Challenges in Building an Email Client
01:06:57 Persistence and Iteration in Startups
01:07:57 Email Market Challenges
01:08:53 SF Compute Job Opportunities
01:09:53 Hiring for Systems Engineering
01:10:50 Financial Systems Engineering Role
01:11:50 Conclusion and Farewell