Podcast Summary

Podcast: Generative Now | AI Builders on Creating the Future

Host: Lightspeed Venture Partners (Michael Mignano)
Guest: Rajarshi Gupta, Head of Machine Learning, Coinbase
Date: February 6, 2025
Episode Theme:
Artificial Intelligence and Crypto at Coinbase

Overview

This episode dives into the ways Coinbase integrates machine learning and generative AI across its platform, both for internal operations and customer-facing tools. Rajarshi Gupta, with a rich background in security and AI, shares a behind-the-scenes look at Coinbase’s AI transformation, discusses the practical challenges of deploying large language models (LLMs), and offers candid advice for AI founders on navigating enterprise adoption and industry gaps.

Key Discussion Points and Insights

Rajarshi Gupta’s Background and Journey (01:05–03:53)

• PhD at Berkeley, spent a decade at Qualcomm Research, where he built the first on-device malware detection ML engine for Android in 2015.
• Startup Experience in Security: At Balbix and Avast (merged into Gen).
• AWS Experience: GM on the SageMaker team.
• Coinbase: Leading AI for the last three years, overseeing a large, globally distributed team.

"We built the industry's first on-device machine learning engine... It shipped in all the high-end Huawei and Samsung and LG phones. So eventually it shipped over a billion chips."
– Rajarshi Gupta (02:44)

ML at Coinbase – Scope and Impact (05:31–06:41)

• Security: ML models guard every login and transaction.
• Personalization: ML decides what users see—personalized notifications, asset recommendations.
• Generative AI: Full team dedicated to gen AI, creating both internal and external tools.

Coinbase’s Adoption of Generative AI (06:53–11:19)

• Early Bet on GenAI: Despite crypto downturns, invested heavily following the release of GPT-3.5 (late 2022).
• Internally:
  • Coinbase Employee Assistant rolled out fall 2023, integrated with Glean for enterprise search, performance review writers, and custom tools for design, finance, etc.
  • Adoption: “Out of I think 3200 people, 2000 some people used it. So everybody across the company all the way to the C suite uses it.” (07:50)
  • Incident Response Bot and Text-to-SQL Bot built as APIs for team-driven extensions.
• Customer-side:
  • LLM Chatbot for support released Nov 2023, now handling “tens of millions of use requests.”
  • AI-powered Search: Gemini-like answers on help queries.
  • Research Assistance (upcoming): AI-driven crypto asset insights.

Model Selection and Technical Infrastructure (11:19–13:15)

• Coinbase GPT Platform: Multi-cloud, multi-LLM; leveraging Azure, GCP, AWS.
• Primary Models: Claude (Anthropic) and Gemini.
• Evaluation Challenges:
  • No true “confidence interval” for LLMs—relies on human eval, ‘judge’ LLMs, curated datasets.

"With LLMs, you don't know how good the answer is. You're having to do all these weird things as a different LLM as a judge..."
– Rajarshi Gupta (12:04)

Launch, Partnership, and Guardrails (13:34–16:05)

• Anthropic Partnership: Coinbase was an early major client. Jointly developed extra guardrail models for chatbot before launch.
• On the challenge: “We were really pushing the edge. And we were scared... fear was about the guardrails, right? What if you get your chatbot? What if somebody...?” (13:40)
• Anthropic’s support “kind of saved us so much time.”

Current and Future Challenges (16:05–18:40)

• Expectations vs. Reality:
  • AI adoption is “like Hogwarts... you have to go for seven years.” Getting results requires patient, deep work.
• Developer Productivity Tools:
  • Early, broad adoption of Copilot, Sourcegraph Cody, and Cursor.
  • Real productivity gains are nuanced:

"Developers don't code for eight hours a day... they’re trying to find data, do debugging. So, even if 25% of code is AI-written, that's 25% of two hours."
– Rajarshi Gupta (16:54)

• Bigger Futures: AI agents that truly “understand the problem,” not just autocomplete.

Infrastructure and Scaling – The GPU Bottleneck (18:40–22:27)

• Biggest challenge (late 2023–2024): GPU scarcity when scaling from internal usage to millions of customers.
• Solution: Load-balancing across AWS & GCP for availability.

"To my big surprise, getting the available GPUs was the biggest problem we faced. Like, by far. Like entire last year."
– Rajarshi Gupta (18:49)

• Complexity of Chatbot Flows: Each request can require 5–9 LLM calls for context, safety, and empathy.

Regulatory Challenge and AI’s Next Frontier (23:18–26:34)

• Bull run preparedness: Coinbase built enormous new infra to finally shed the meme: “crypto goes up, Coinbase goes down.”
• Feature Development: Legal, compliance, and regulatory uses for AI – e.g., instant translation and analysis of new laws in different jurisdictions.
• Enterprise AI Potential:
  • Optimization in HR, legal, finance, operations is largely untapped due to “the Hogwarts seven years problem.”

Advice for AI Startups & the Enterprise Gap (26:34–29:47)

• Biggest Gaps:
  • Honest, scientific LLM evaluation.
  • Most enterprise challenges are “plumbing” and integration, not the LLMs themselves.

"The AI is not the hard part of the problem. That part has already been solved. But how can you use the power of the gen AI models to solve this problem?"
– Rajarshi Gupta (28:37)

• Enterprise Search: Example of delayed solutions due to complex integration, not lack of ML innovation.

Open Q&A Highlights

Agentic AI & Crypto Convergence (32:05–35:33)

• Agentic AI x Blockchain:
  • "An AI agent cannot own a wallet with cash, but they can own a crypto wallet."
  • Micropayments via crypto unlock unique agentic AI behaviors: “.02 cents” transactions feasible only on blockchain.

"Blockchains and the ability to exchange crypto payments solves a huge problem for agentic AI, which we absolutely adore."
– Rajarshi Gupta (33:26)

Guardrails Framework for LLMs (35:34–39:53)

• Multi-tiered Guardrails:
  • Info-only → personalized info → action-enabled bots, each with stricter guardrails.
  • Extra benefit: Guardrails also protect human agents from abuse.

"We are doing a lot of guardrails and sometimes you get freebies."
– Rajarshi Gupta (39:30)

Evaluating LLMs: Restoring Rigor (39:53–42:50)

• Challenge: Lack of confidence metrics compared to classic ML.
• Academic input: LLMs mimic human speech, which evades old accuracy paradigms.

"We should as an industry be able to figure it out. And that's my ask, but I'm not qualified enough anymore."
– Rajarshi Gupta (41:29)

• Emerging trend: SLMs (small language models) as a path towards more determinism and tractable evaluations.

Notable Quotes & Memorable Moments

• On the transformative power of AI:

  "AI is like magic, but before you can do magic, you have to go to Hogwarts for seven years." — Rajarshi Gupta (15:10)

• On enterprise opportunity:

  "There is such an enormous amount of money in enterprise problems that can be solved with gen AI. It is unbelievable. It’s not easy though…" — Rajarshi Gupta (28:02)

• On integrating AI in regulated environments:

  "Coinbase is a very regulated company…we have a lot of people in the company, humans whose job is to make sure that we stay compliant…these are all use cases for AI."
  — Rajarshi Gupta (24:30)

• On crypto as a platform for AI agents:

  "Crypto wallets are big for us and the agents allow a wonderful mechanism…an AI agent cannot own a wallet with cash, but they can own a crypto wallet." — Rajarshi Gupta (33:07)

Important Timestamps

| Timestamp | Segment | |------------|---------| | 01:05–03:53 | Rajarshi’s background; story of the first Android ML engine | | 05:31–06:41 | ML at Coinbase: security, personalization, gen AI team | | 07:47–11:19 | Employee assistant, internal/external AI tool development | | 11:27–13:15 | Evaluation and management of LLM deployments | | 13:34–16:05 | Anthropic partnership, guardrails, launch anxiety | | 16:18–18:40 | Developer tools adoption, productivity nuance | | 18:40–22:27 | GPU availability struggles, platform scaling | | 23:18–26:34 | Future challenges: regulatory compliance, AI in operations| | 26:34–29:47 | Enterprise AI startup gap, integration is the challenge | | 32:05–35:33 | Agentic AI and blockchain synergies | | 35:34–39:53 | LLM guardrails: implementation, levels, human protection | | 39:53–42:50 | The challenge of LLM evaluation and measurement rigor |

In Closing

Rajarshi Gupta’s perspective underscores the real-world complexity of enterprise AI adoption—where model performance, productization effort, and organizational change outpace the “magic” of any new model. He spotlights the critical, lucrative gap for startups in turning AI’s raw capability into frictionless, compliant, integrated enterprise solutions.

“The AI is not the hard part of the problem. That part has already been solved. But how can you use the power of the gen AI models to solve this problem? That's where the big space lies.”
— Rajarshi Gupta (28:37)