Latent Space Podcast: SF Compute – Commoditizing Compute to Solve the GPU Bubble Forever

Date: April 11, 2025
Host: Alessio (CTO at Decibel) & Zwicks (Founder of Small AI)
Guest: Evan Conrad (Founder of SF Compute)

Episode Overview

This episode delves into the shifting economics, business models, and technical realities of the GPU infrastructure market, focusing on how SF Compute offers a new paradigm for the acquisition and utilization of high-end compute. The discussion ranges from the state of "GPU bubble," the rise of compute as a commodity, challenges for traditional cloud providers, SF Compute’s path and pivots, liquidity and market design, as well as the future of financialization (futures, exchanges) of compute. Listeners get a transparent, sometimes technical account of how the market for AI hardware is evolving—and why the right business model matters for builders, startups, hyperscalers, and research.

Key Discussion Points & Insights

1. CoreWeave’s Model and the GPU Bubble

• Long-Term vs. Short-Term Contracts: CoreWeave’s success has been rooted in signing high-volume, long-term GPU contracts with very low credit risk customers (OpenAI, Microsoft), in contrast to the on-demand, software-layered approach of traditional CPU cloud providers.
  • Quote (Evan Conrad, 01:10): “Sell locked in long term contracts and don't really do much short term at all. A lot of people had this assumption that GPUs would work a lot like CPUs...That doesn't really work in GPUs.”
• Commodity vs. Value-Added Compute: The podcast explores why software differentiation doesn’t add much margin in GPU, due to extreme customer price sensitivity at massive scale.
• CoreWeave is Not Just a Cloud Provider: Their financials and risk structure are closer to a bank or a real estate company than AWS-like clouds.
  • Quote (Zwicks, 05:37): “It's a bank.”
• Why Not Hyperscalers? The episode unpacks why Nvidia (to avoid competing with its own customers) and Microsoft (to avoid customer concentration for Nvidia, among other reasons) haven’t mimicked CoreWeave’s approach.

Timestamps

• CoreWeave analysis: 00:41–08:30
• Business model deep dive and risk charts: 08:30–13:58
• Why Nvidia/Microsoft don’t do it directly: 13:58–16:14

2. GPU Market Risks, Failed Approaches & Economic Realities

• Failed Playbooks: Many attempted to replicate the high-margin, software-heavy “CPU cloud” model for GPU, misjudging high customer price sensitivity at large volumes and physical risk (inventory, depreciation).
• Margin Compression: Hyperscalers can always undercut independent providers, squeezing margins further and driving away those with high-risk, high-price models.
• Advice to Founders: "Don't buy hardware and try to layer on value-add software if your customers are hyperscale and price sensitive. If you combine real estate (hardware) and cloud software, you ‘get shot in the head’ financially." (Evan Conrad, paraphrased, 18:00–19:20)
• Real Winners: Decouple the two businesses—either be a hardware real estate company (e.g., CoreWeave), or a software orchestrator that does not own hardware (e.g., Modal).

Timestamps

• Market risk & business model failures: 10:45–19:20
• Advice to GPU cloud founders, “grim reaper” analogy: 18:31–19:20

3. The Founding and Evolution of SF Compute

• From AI Lab to Compute Broker to Spot Market: SF Compute’s origin story started as an AI lab (for music/audio models), forced to pivot when faced with prohibitive compute contract requirements. They then became brokers, matching buyers and sellers for partial contracts, before evolving into the most liquid spot market for GPU compute.
  • Quote (Evan, 20:47): “Originally we were not doing this at all. SF Compute started because we wanted to go train models for music and audio... We just ended up being forced into this [brokerage and then market model].”
• Enabling Granular Compute Leases: Their market innovation made it possible to sell and buy partial contract durations (e.g., an hour or a month on a 12-month contract), unlocking flexibility and utilization.

Timestamps

• SF Compute’s pivot and accidental GPU cloud: 20:35–25:49
• Birth of liquid GPU market, enabling new flexibility: 25:49–27:58

4. Market Dynamics, Utilization, and Liquidity

• Maximizing Utilization: With market-driven pricing, idle GPU capacity can be flexibly monetized, which improves vendor revenue compared to empty clusters.
• Real-Time and Secondary Markets: SF Compute effectively creates a “spot” market for compute, where the price falls until supply matches demand—enabling short-term (or preemptible style) usage previously unavailable.
• Economic Incentives for Flexibility: Vendors can now offer long-term, cancelable contracts—buyers pay a fee if they exit early, offering both flexibility and risk management lacking in traditional GPU cloud contracts.

Timestamps

• Market operation and vendor pitch, “cancelable” contracts: 25:49–27:58

5. The H100 Glut, Boom and (Project) Bust

• Origins of the Glut: Delays and then sudden influx of hardware after supply chain bottlenecks created oversupply; but demand itself did not decrease. As a result, GPU rental prices fell, but long-term, demand is likely to outstrip supply again, especially as test-time inference workloads rise.
  • Quote (Evan, 28:49): “It definitely seems like there's more demand for GPUs than there ever was. It's just that there was also more supply... My general prediction is that by the winter we will be back towards shortage.”

Timestamps

• H100 glut and bust explained: 27:58–31:18
• Demand-side factors for large clusters: 31:18–33:57

6. “Peer-to-Peer” and Decentralized Compute Skepticism

• Skepticism of DIY/crypto compute: Evan is highly skeptical of true peer-to-peer GPU “Uber for compute” models—they can't match the efficiency of datacenter clusters at scale, and hardware/networking realities (like Infiniband) make home clusters uneconomic for most use.
  • Quote (Evan, 34:20): “I'm like wildly skeptical of these... you just can't get around that physical limitation [speed-of-light and cluster interconnect].”

Timestamps

• Crypto/peer-to-peer models critique: 34:01–36:33

7. SF Compute Use Cases and Impact

• Accessibility for Researchers & Startups: By providing burst capacity and fine-grained contracts, SF Compute supports grad students, hackers, and early startups that traditional clouds spurn as “high risk” and unprofitable.
• Real-World Impact: Examples include startups like Standard Intelligence, Phind, and grantees from Schmidt Futures leveraging burst compute for research.

Timestamps

• Grad students/startup customer stories: 37:00–38:38

8. Venture Capital and Compute Arbitrage

• Role of VCs in Clusters: Temporarily, VCs offered compute as a service for portfolio companies, exploiting their lower cost of capital and willingness to take on credit risk not feasible for pre-revenue startups.
  • Quote (Evan, 39:06): “If you have a lot of money, it is way easier for you to get a loan than if you don't... equity for compute is always some arbitrage on the credit risk.”
• Short-Lived Opportunity: Evan thinks that once the market rationalized, this arbitrage closed and most value accrued to the first-movers (e.g. Andromeda).

Timestamps

• VC compute clusters and arbitrage: 38:38–41:13

9. Market Pricing Dynamics and Power User Features

• How Pricing Really Works: Burst/preemptible (“day-old milk”) compute is priced lowest right before it expires—customers can game the system by setting their own max prices and exploit off-peak or spot prices for large savings.
• Market Primitives Beat Hyperscaler Offerings: SF Compute offers flexibly customizable reservation periods (hourly to yearly) and limit orders, so users can optimize price vs. reliability in bespoke ways, beyond what Amazon/Azure/Google cloud spot or batch options provide.
  • Quote (Evan, 46:09): “SF Compute is like the power tool...”

Timestamps

• Market pricing, spot/preempt market, user tips: 41:58–46:13

10. Financialization: Toward a Spot & Futures Market in Compute

• Building a Spot and Eventually Futures Exchange: SFC aims to set an industry index price for compute, then introduce cash-settled futures, so data centers can hedge risk just as commodity producers do.
  • Quote (Evan, 47:59): “What we're trying to do is create an underlying spot market that gives you an index price... with that, you can create a cash-settled future.”
• De-Risking the Industry: Analogous to agriculture futures, compute futures allow actors to efficiently plan and price, reducing risk-fueled bubbles (and VC overinvestment).
• Not Derivatives Yet, But Soon: The team emphasizes they are not yet a derivatives exchange, but this is the logical future once spot liquidity and standardization are in place.
• Automatic SLA/Refunds and Reliability: SFC operates clusters, performs rigorous hardware and integration testing, and passes strict SLAs and automated refunds, going beyond the variable reliability norms of many hardware providers.
  • Quote (Evan, 52:53): “That's in our contract with all the underlying cloud providers.”

Timestamps

• Financialization, spot/futures market vision: 47:59–58:13
• Technical/custodial challenges, cluster auditing, refunds: 49:25–54:05

11. Commoditization & Standardization: Technical Details

• Contract Standardization: SFC supplies a “this or better” spec, persistent storage layers, and a UEFI shim to allow cluster imaging and provisioning at the BIOS/firmware level, enabling standardization needed for financial contracts and trader confidence.
• Trader Confidence/Institutional Volume: Hosts relate parallels to commodities and derivatives markets, and the necessity for trusted standards in contracts.
  • Quote (Zwicks, 56:43): “I need standard contracts, and so there basically needs to be the safe of a GPU.”

Timestamps

• Standardization and contract details: 55:04–57:38

12. Branding, Vibe, and the “Calm Company” Ethos

• Anti-Hype Design: The SFC website is intentionally minimalist and “calm,” signaling a mature, honest, and enduring brand versus the “black neon” hyperbolic norm in tech.
  • Quote (Evan, 61:19): “Everything we've been trying to do is go the other way... Our entire brand is just like, what if we were calm in nature?”
• Localization & Community: The company identifies strongly with the San Francisco community and landscape, often highlighting local vibes.

Timestamps

• Branding/vibes conversation: 61:19–63:47

13. Personal History, Email, and Lessons from Pivots

• Startup Vulnerability: Evan shares his journey from building mental health and email products, the challenge of pivoting, and the importance of not giving up (“don’t die advice”).
• Email Client Graveyard: Email startup efforts rarely succeed for systemic reasons—best executed as features by platforms with distribution (like Intercom), not as standalone clients.

Timestamps

• Evan's founder story and lessons from prior pivots: 65:57–69:06

14. Hiring & Company Pitch

• Hiring Philosophy and Open Roles: SFC is hiring for systems engineers (Linux, Rust, low-level) and fintech/financial systems roles. Emphasis on culture, impact, and working with “supercomputers, not soybeans.”
  • Quote (Evan, 69:09): “The things you do actually matter in a way that doesn’t necessarily always [in] all the companies. Functionally, we run supercomputers, not soybeans. It's a very cool place to work...”

Timestamps

• Hiring & pitch to candidates: 69:09–71:48

Memorable Quotes

• On traditional software margins in GPU:
  “There isn’t a billion dollars of software that you can realistically make... and if you do, you’re going to look like SAP.”
  – Evan, 16:31

• On risk and margin:
  “If you combine [hardware ownership and software services], and that's what the market does, you get shot in the head. But if you split them... you can make lots of money.”
  – Evan, 18:00

• On the goal of futures in compute:
  “The reason to create a derivative at all... is a risk reduction thing. That's what futures do... The whole point of SF Compute is to reduce the risk, reduce the technical risk, reduce the financial risk. Let's just chill out a little bit.”
  – Evan, 58:13

• On the anti-hype vibe:
  “By being anti-hype, you have created hype... the vibes are immaculate.”
  – Zwicks, 62:46

Notable Moments & Easter Eggs

• The calm, almost understated SF Compute website (“Why don’t you go black neon?”) – (61:25)
• Grad student and hacker-focused burst compute use cases – (37:00–38:38)
• Branding inspiration and the “serif font” moment for their event banner – (63:04)
• Mention of a hidden Easter egg on sfcompute.com/buy, courtesy of designer John Pham – (64:56)

For Listeners Who Missed the Episode

This episode is a deep dive into what it really takes—not just technically but economically and culturally—to turn AI compute into a liquid, accessible, and commoditized market. From the rise and fall of previous GPU cloud models to the emergence of spot and future markets and the war stories of pivots, SF Compute is at the center of a new paradigm. The episode is a must-listen for anyone building or investing in AI infra, or just interested in how financialization trends are about to transform the foundation of machine learning itself.

Further reading, show notes, and Easter egg hints at:
latent.space

Presented in the spirit and style of the Latent Space podcast—calm, precise, and deeply insightful.