Retrieval After RAG: Hybrid Search, Agents, and Database Design — Simon Hørup Eskildsen of Turbopuffer

Summary

---

Latent Space: The AI Engineer Podcast

Episode: Retrieval After RAG: Hybrid Search, Agents, and Database Design
Guest: Simon Hørup Eskildsen, Co-Founder of Turbopuffer
Date: March 12, 2026

---

Episode Overview

This episode explores the evolution of retrieval systems after the popularity of Retrieval-Augmented Generation (RAG). Simon Hørup Eskildsen, co-founder of Turbopuffer, offers a deep dive into the shifting landscape of hybrid search, AI agent workloads, and database design for modern, AI-driven applications. He shares the technical motivations behind Turbopuffer’s architecture, lessons from scaling at Shopify, the realities of startup building, and his principles for engineering excellence.

---

Key Discussion Points and Insights

1. Turbopuffer’s Mission and Architecture

[02:33 - 06:25]

• Turbopuffer is positioned as a search engine, focusing on full text and vector search for unstructured data.
• Eskildsen frames the goal:

  "We can compress into a few terabytes of weights how to reason with the world, but we have to somehow connect it to something external that actually holds that like in full fidelity and truth. And that's the thing that we intend to become."

• The platform leverages new cloud primitives—object storage, NVMe SSDs, and eventual/strong consistency to enable a fundamentally different architecture.
• Three foundational conditions to build a modern database company:
  • A new workload: AI-connected data search in every company.
  • Storage architecture leap: All-in on fast SSDs and object storage, no consensus layer.
  • Comprehensive query support: A big database company must eventually serve a wide variety of query plans for evolving customer needs.

---

2. Origin Story: From Shopify to Angel Engineering to Turbopuffer

[06:26 - 13:32]

• Draws on nearly a decade of experience at Shopify, especially the pain of scaling and maintaining Elasticsearch clusters.
• Describes his post-Shopify phase as "angel engineering," consulting on infra for startups like Readwise, Replicate, Causal.
• The "aha" for Turbopuffer: While helping Readwise, found vector search could improve recommendations, but the infrastructure cost was "insane" (30k/month just for vectors vs a 5k/month infra budget).

  "If the cost had been a tenth, we would have shipped it. And this was really the only data point that I had, right?... It haunted me."

• Spurred to build a fully object-storage-centric, cost-effective search database optimized for cloud primitives.

---

3. Technical Deep Dive: Cloud Storage, Consensus, and Database Design

[13:32 - 22:58]

• Details how object storage as primary storage and NVMe SSDs as cache open new possibilities.
• On S3’s evolution:

  "S3 only became consistent in December of 2020." ([17:19])

• Compare-and-swap support was a gating factor for metadata management across clusters; Google Cloud Storage had it first, AWS’s S3 added it in late 2024.
• Early customers’ (Notion, Cursor) need for low latency led to elaborate routing—buying dark fiber between AWS and GCP regions to avoid introducing extra consensus and state management layers.
• Technical stance: “I don't want state in two systems. The worst outages are the ones where you have state in multiple places that's not syncing up.” ([22:01])

---

4. Early Customer Stories: Cursor & Notion

[24:58 - 30:57]

• Cursor, second significant client (after Readwise), credits Turbopuffer with making their per-user economics work by reducing search-related infra costs by 95%.
• Cursor’s technical use case:

  “Cursor's workload is basically they will embed the entire codebase... They have their own embedding model... very good at semantic search... And so it's very good at: can you find me code that's similar to this or code that does this?”

• Cursor ensures security/privacy by encrypting codebase embeddings and obscuring file paths inside Turbopuffer.
• Notion’s case: Technical credibility and cost/vs. egress/latency tradeoffs were more important than Turbopuffer’s architecture per se.

---

5. Evolution of Retrieval Workloads: From RAG to Agent-Led Hybrid Search

[31:55 - 34:22]

• Modern workloads: Shift from single RAG-style context window search to agentic patterns where LLM agents drive many concurrent searches, and retrieval is dynamic and fine-grained.
  • “Now you have agentic search, where the model is both writing and changing the code and it's searching it again later.”
• Hybrid search (semantic + keyword/regex/SQL):

  “All workloads are hybrid. You want the semantic, you want the text, you want the regex... It's silly to be all in on one particular query pattern.” ([31:17])

• Engineering response: Dropping query pricing 5x to support emerging, high-concurrency agentic use.

---

6. Startup Journey: Pricing, Funding, and Picking Investors

[34:22 - 41:22]

• Early pricing was “Vibe pricing”—rough estimates based on infra costs and a bit of margin, later optimized as real usage revealed cost structure.
• Bootstrapped beginnings—ran on founders’ credit cards until after Cursor joined as a customer.
• On choosing first investor Lockey (despite others being “database experts”):

  "I just called Locky... if this doesn't have PMF by the end of the year, we'll just return all the money to you... I just play with open cards. Locky was the only person that didn't freak out. He was like, I've never heard anyone say that before.” ([39:13])

---

7. Team Building, P99 Engineers, and Company Culture

[41:48 - 48:26]

• “Joining vs. starting”: Simon did not set out to found a company, Turbopuffer evolved from a blog post and proof-of-concept.
• “P99 engineer” concept: Internal framework for ultra-high-talent density, emphasizing high agency, track record of "bending the computer to their will," and a natural obsession with systems/things (e.g., maps, trains).
  • Traits: High agency, trade-off thinking, proven ability to deliver breakthroughs.
  • Hiring bar: “The default should be, we're definitely not hiring this person. And, you know, if everyone was like, I'll maybe throw a punch, then this is not the right team.”
• Example:

  “ANN v3 can search 100 billion vectors with a p50 of around 40 milliseconds and a p99 of 200 milliseconds... the Chief Architect of TurboPuffer... just bent the software... in a six to eight week period.” ([46:34])

---

8. Future Vision: Roadmap and Beyond

[51:12 - 57:06]

• Act One: Vector search only.
• Act Two: Adding full-text search, optimizing for new, LLM-driven workloads.
• Current Focus: More full text features and improved scale—serving “common crawl-level” datasets, improving feature coverage and dashboard UX, and supporting organizations moving from legacy search platforms.
• Looking Ahead:
  • Will expand to additional workloads (OLAP, logging, time series) only as customer demand becomes clear—fierce focus over feature sprawl.
  • Possible support for graph query patterns as evolving customer need.

  "If you want to build a big database company, the database over time has to implement more or less every query plan... But when you're a startup, your only moat is really just focus." ([54:41])

---

9. Personal Touches: Tea, Maps, and Pokémon-Grade Nerdiness

[57:06 - end]

• Simon’s obsession with tea (keeps an Airtable of 200+ teas), draws parallel to P99 engineering mentality—total, lifelong curiosity and obsession.
• Maps as a metaphor for good developer relations/devrel—mapping journeys, showing boundaries and paths.

---

Notable Quotes (with Timestamps)

• On honesty with funders:

  "If this doesn't have PMF by the end of the year, we'll just return all the money to you... When I don't know how to play a game, I just play with open cards."
  — Simon ([39:13])

• On architecture and the cloud:

  "You could turn off all the servers that TurboPuffer has and we would not lose any data because we are completely all in on object storage. And this means that our architecture is just so simple."
  — Simon ([03:14])

• On team standards:

  "The default should be, we're definitely not hiring this person. And, you know, if everyone was like, I'll maybe throw a punch, then this is not the right team."
  — Simon ([45:12])

• On first principles and technology timing:

  "S3 only became consistent in December of 2020... NVMe SSDs were also not in the cloud until around 2017... Compare and swap was not in S3 until late 2024."
  — Simon ([17:19-21:04])

• On focus and sequencing:

  "When you're a startup, your only moat is really just focus. So you have to lay out the facts and you have to not get overeager."
  — Simon ([54:44])

---

Key Timestamps

• [02:33] – Turbopuffer’s product definition and philosophy
• [06:26] – Simon’s background, Shopify experience, and angel engineering
• [13:32] – Technical shift: object storage & cloud architecture
• [22:01] – State management philosophy (avoiding Zookeeper, consensus pain)
• [24:58] – Notion and Cursor case studies, cost savings, practical impacts
• [31:17] – Discussion of hybrid/agentic workloads, evolution beyond RAG
• [34:44] – Pricing philosophy, infra cost realities, and bootstrapping
• [39:13] – Choosing Lockey as investor; open cards approach
• [45:12] – P99 engineer philosophy, team culture, hiring standards
• [51:12] – Roadmap vision: acts, feature growth, and caution against overreaching
• [57:06] – Tea stories, quirks, and obsessive nerd curiosities

---

Memorable Moments

• Simon’s "Vibe pricing" and running a cloud infra bill off his credit card.
• The story of physically buying dark fiber for a startup database to support multi-cloud low-latency workloads.
• The “P99 engineer” standard and the cultural bar for hiring.
• Parallels between obsession with tea, maps, and the kind of talent that shapes foundational infrastructure.

---

Conclusion

This episode offers an in-depth, candid, and highly technical portrait of the next generation of retrieval systems—where “search after RAG” blends hybrid search methods, high concurrency, and data architectures native to modern cloud primitives. Simon Hørup Eskildsen pulls back the curtain on the real tradeoffs, the founding struggles, and the uncompromising standards behind Turbopuffer’s journey—setting a new bar for AI-driven infrastructure.

---

Transcript

[A]

I don't think I've said this publicly before, but I just called Lockheed and was like, local Aki, like, if this doesn't have PMF by the end of the year, like, we'll just like return all the money to you. But it's just like I don't really. Justine and I don't want to work on this unless it's really working. So we want to give it the best shot this year and like, we're really going to go for it. We're going to hire a bunch of people and we're just going to be honest with everyone. Like, when I don't know how to play a game, I just play with open cards. Locky was the only person that didn't, that didn't freak out. He was like, I've never heard anyone say that before.

[B]

Hey everyone. Welcome to the lit in Space podcast. This is Alessio, founder of kernelads and I'm joined by Swix, editor of Lydian Space.

[C]

Hello. Hello. We're so recording in the kernel studio for the first time. Very excited. And today we're joined by Simon Eskildsen of turboparfur.

[B]

Welcome.

[A]

Thank you so much for having me.

[C]

Turboparfur has really gone on a huge tear and I do have to mention that you're not my newest member. The Danish Akus mafia, where there's a lot of legendary program have come out of it like Jan Rasmus Lehrdov, Anders Hejlsberg and the V8 team and Google Maps team. You're mostly Canadian now, but isn't that interesting? There's so many. So much strong Danish presence.

[A]

Yeah. I was writing a post not that long ago about sort of the influences. So I grew up in Denmark. Right. I left when I was 18 to go to Canada to work at Shopify. And so I would still say that I feel more Danish than Canadian. This is also the weird accent. I can't say th. Because this is like I don't. You know, my wife is also Canadian and I think, I think like one of the things in Denmark is just like there's just such a ruthless pragmatism and there's also a big focus on just aesthetics. Like they're like very. People really care about like where what things look like. And Canada has a lot of attributes. US has a lot of attributes. But I think there's been lots of great things to carry. I don't know what's in the water in Aarhus though. And I don't know that I could be considered part of the Always mafia quite yet compared to the phenomenal individuals you just mentioned. Barasa Znov is also Danish Canadian. Yeah, I don't know where he lives now, but. And he's the php.

[C]

Yeah. And obviously Toby German will move to Canada as well. Like this is like import that. Interesting talent move.

[B]

I think I would love to get from you the definition of turbopuffer, because I think you could be a vector db, which is maybe a bad word now in some circles you could be a search engine. It's like, let's just start there and then we'll maybe run through the history of how you got to this point.

[A]

For sure. Yeah. So turbopuffer is at this point in time a search engine. Right. We do full text search and we do vector search and that's really what we're specialized in. If you're trying to do much more than that, like, then this might not be the right place yet. But TurboPuffer is all about search. The other way that I think about it is that we can take all of the world's knowledge, all of the exabytes and exabytes of data that there is, and we can use those tokens to train a model, but we can't compress all of that into a few terabytes of weights, Right? We can compress into a few terabytes of weights how to reason with the world, how to make sense of the knowledge, but we have to somehow connect it to something external that actually holds that like in full fidelity and truth. And that's the thing that we intend to become. Right? That's like a very holier than thou kind of phrasing. Right. But being the search engine for unstructured data is the focus of TurboPuffer at this point in time.

[B]

And let's break down. So people may say, well, didn't elasticsearch already do this? And then some other people might say, is this search on my data? Is this closer to rag than to like a xr, like a public search thing? How do you segment the different types of search?

[A]

The way that I generally think about this is like there's a lot of database companies and I think if you want to build a really big database company, sort of, you need a couple of ingredients to be in the air, which only happens roughly every 15 years. You need a new workload. You basically need the ambition that every single company on earth is going to have data in your database multiple times. You look at a company like Oracle, right? I don't think you can find a company on Earth with a Digital presence that doesn't somehow have some data in an Oracle database, right? And I think at this point that's also true for Snowflake and Databricks, right? Fifteen years later, or even more than that, there's not a company on earth that doesn't indirectly or directly is consuming Snowflake or Databricks or any of the big analytics databases. And I think we're in that kind of moment now, right? I don't think you're going to find a company over the next few years that doesn't directly or indirectly have all their data available for search and connected to AI. So you need that new workload. Like you need something to be happening where there's a new workload that causes that to happen. And that new workload is connecting very large amounts of data to AI. The second thing you need, the second condition to build a big database company is that you need some new underlying change in the storage architecture that is not possible from the databases that have come before you. If you look at Snowflake and databricks, right, commoditized like massive fleet of HDDs, like that was not possible. It just wasn't in the air in the 90s, right? So we just didn't build these systems. S3 and so on was not around. And I think the architecture that is now possible, that wasn't possible 15 years ago is to go all in on NVME SSDS. It requires a particular type of architecture for the database that is difficult to retrofit onto the databases that are already there, including the ones you just mentioned. The second thing is to go all in on object storage, more so than we could have done 15 years ago. Like we don't have a consensus layer. We don't really have anything. In fact, you could turn off all the servers that Turbo Buffer has and we would not lose any data because we are completely all in on object storage. And this means that our architecture is just so simple. So that's the second condition, right? First being a new workload. That means that every company on earth, either indirectly or directly, is using your database. Second being there's some new storage architecture that means that the companies that have come before you can't do what you're doing. I think the third thing you need to do to build a big database company is that over time you have to implement more or less every query plan on the data. What that means is that you can't just get stuck in like this is the one thing that a database does. It has to be ever evolving because when someone has data in the database, they over time expect to be able to ask it more or less every question. So you have to do that to get the storage architecture to the limit of what it's capable of. Those are the three conditions.

[C]

I just wanted to get a little bit of the motivation. Right, so you left Shopify, you're like principal engineer, infra guy. You also hit a kernel labs inside of Shopify and then you consulted for Readwise and that kind of gave you that idea. I just wanted you to tell that story. Maybe you've told it before, but just introduce the people to the new workload. The sort of aha moment for turbopuffer for sure.

[A]

So yeah, I spent almost a decade at Shopify. I was on the infrastructure team from the fairly, fairly early days around 2013. At the time it felt like it was growing so quickly and everything. All the metrics were, you know, doubling year on year compared to the what companies are contending with today. It's very cute growth. I feel like some companies are seeing that month over month. Of course, Shopify compound has been compounding for a very long time now, but I spent a decade doing that and the majority of that was just make sure the site is up today and make sure it's up a year from now. And a lot of that was really just the, you know, the Kardashians would drive very, very large amounts of data to, to, to Shopify as they were rotating through all the merch and building out their businesses and we just needed to make sure we could handle that. Right. And sometimes these were events with a million requests per second. And so, you know, we, we had our own data centers back in the day and we were moving to the cloud and there's so much sharding work and all of that that we were doing. So it's been a decade just scaling databases because that's fundamentally what's the most difficult thing to scale about these sites. The database that was the most difficult for me to scale during that time and that was the most aggravating to be on call for was elasticsearch. It was very, very difficult to deal with and I saw a lot of projects that were just being held back in their ambition by using it. And I mean self hosted. Self hosted because. Yeah, and there's almost like 2015. Right. So it's like a very particular vintage. Right. It's probably better at a lot of these things now. It was difficult to contend with and I'm just like, I just think about it. It's an inverted index. It should be good at these kinds of queries and do all of this. And we often couldn't get it to do exactly what we needed to do or basically get Lucene to do like expose Lucene raw to what we needed to do. So that was like just something that we did on the side and just panic scaled when we needed to but not a particular focus of mine. So I left and when I left I wasn't sure exactly what I wanted to do. I mean I'd spent like a decade inside of the same company. I'd like grown up there. I started working there when I was 18.

[C]

You only do Rails?

[A]

Ye. I mean yeah, Rails and he's a Rails guy. I love Rails. So good.

[B]

We all wish we could still work in Rails.

[A]

I know, I know, I know.

[C]

But some I tried learning Ruby. It's just too much like too many options to do the same thing. That's my. I know there's a way to do it.

[A]

I love it. I don't know that I would use it now like given cloud code and cursor and everything but still if I'm just sitting down and writing a tsonal code, that's how I think. But anyway I left and I wasn't. I talked to a couple companies. I was like I don't. I need to see a little bit more of the world here to know what I'm going to like focus on next. And so what I decided is like I was going to. I called it like Angel Engineering where I just hopped around in my friends companies in three months increments and just helped them out with something. Right. And just vested a bit of equity and solved some interesting infrastructure problem. So I worked with a bunch of companies at the time. Readwise was one of them. Replicate was one of them. Causal. I don't know if you've tried this. It's like a. It's a spreadsheet engine. Yeah. Where you can do distribution. They sold recently. We used that in FP and a turbopuffer. So a bunch of companies like this and it was super fun and so when the ChatGPT moment happened I was with Readwise for a stint. We were preparing for the reader launch. Right. Which is where you queue articles and read them later. And I was just getting their postgres up to snuff like which basically boils down to tuning autovacuum. So I was doing that and then this happened and we were like oh maybe we should build a little recommendation engine and some features to try to hook in the LLMs. They were not that good yet, but it was clear there was something there. And so I built a small recommendation engine, just, okay, let's take the articles that you recently read, right? Like embed all the articles and then do recommendations. It was good enough that when I ran it on one of the co founders of Readwise, like I found out that I got articles about having a child. I'm like, oh my God. I didn't know that they were having a child. I wasn't sure what to do with that information. But the recommendation engine was good enough that it was suggested articles about that. And so there was, there was recommendations and it actually worked really well. But this was a company that was spending maybe five grand a month in total on all their infrastructure. And when I did the napkin math on running the embeddings of all the articles, putting them into a vector index, putting it in prod, it's going to be like 30 grand a month. That just wasn't tenable, right? Like Readwise is a proudly bootstrapped company and paying 30 grand for infrastructure for one feature versus five, it just wasn't tenable. So sort of in the bucket of this is useful. It's pretty good. But let us, let's return to it when the costs come down.

[C]

Did you say it grows by feature? So 5 to 30 is by the number of like what's the, what's the scaling factor? It scales by the number of articles that you embed.

[A]

It does, but what I meant by that is like five grand for like all of the other, like the Heroku Dinos postgres, like all the other.

[C]

And this, then the storage is 30.

[A]

Yeah. And then like 30 grand for one feature, right? Which is like what other articles are related to this one? So it was just too much, right, to power everything. Their budget would have been maybe a few thousand dollars, which still would have been a lot. And so we put it in the bucket of okay, we're going to do that later. We'll wait for the cost to come down. And that haunted me. I couldn't stop thinking about it. I was like, okay, there's clearly some latent demand here. If the cost had been a tenth, we would have shipped it. And this was really the only data point that I had, right? I didn't, I didn't, I didn't go out and talk to anyone else. It was just. So I started reading, right? I couldn't, I couldn't help myself. Like, I didn't know what like a Vector index is. I generally barely knew about how to generate the vectors. There is a lot of hype about. This is early 2023, there was a lot of hype about vector databases. They're raising a lot of money. As I really didn't know anything about it, it's like, you know, trying these little models, fine tuning them. Like I was just trying to get sort of a lay of the land. So I just sat down, I have this GitHub repository called Napkin Math. And on Napkin Math there's just rows of like, oh, this is how much bandwidth. Like this is how many. You know, you can do 25 gigabytes per second on average to DRAM. You can do, you know, 5 gigabytes per second of writes to an SSD, blah, blah, all of these numbers, right? And S3, how many you could do. How much bandwidth can you drive per connection? I was just sitting down, I was like, why hasn't anyone built a database where you just put everything on object storage and then you puff it into NVME when you use the data and you puff it into dram if you're querying it a lot? This seems fairly obvious. And the only real downside to that is that if you go all in on OPIX storage, every write will take a couple hundred milliseconds of latency. But from there it's really all upside, right? You do the first query, it takes half a second. And it sort of occurred to me, it's like, well, the architecture is really good for that. It's really good for object storage. That's really good for NVMe SSD.

[B]

It's.

[A]

Well, you just couldn't have done that 10 years ago. Back to what we were talking about before, you really have to build a database where you have as few round trips as possible, right? This is how CPUs work today. It's how NVMe SSDs work. It's how S3 works that you want to have a very large amount of outstanding requests, right? Like basically go to S3, do like that, thousand requests to ask for data in one round trip. Wait for that, get that, like make a new decision, do it again and try to do that maybe a maximum of three times. But. But no databases were designed that way. With NVMe SSDs you can drive within a very low multiple of DRAM bandwidth if you use it that way. And same with S3, you can fully max out the network card, which generally is not maxed out. You get very, very good bandwidth, but no One had built a database like that. So it's like, okay, well can't you just take all the vectors and plot them in the proverbial coordinate system? Get the clusters, put a file on S3 called clusters JSON and then put another file for every cluster. You know, cluster one, JSON, cluster two, JSON. You know that like it's two round trips, right? So you get the clusters, you find the closest clusters and then you download the cluster files like the closest end. And you could do this in two round trips.

[C]

You're in nearest neighbors locally.

[A]

Yes, yes. And you would build this file, right, Just like ultra simplistic, but it's not a far shot from what the first version of Turbo Buffer was. Why hasn't anyone done that in that moment?

[B]

From a workload perspective, you're thinking this is going to be like a read heavy thing because you're doing recommended. Like is the fact that writes are so expensive now. Oh, with AI you're actually not writing that much.

[A]

At that point I hadn't really thought too much about. Well, no, actually it was always clear to me that there was going to be a lot of writes because at Shopify the search clusters were doing, I don't know, tens or hundreds of QPs, right? Because you just have to have a human sit and type in. But we did. I don't know how many updates there were per second. I'm sure it was in the millions right into the cluster. So I always knew there was like a 10 to 100 ratio on the read write. In the readwise use case, it's even, even in the readwise use case, there'd probably be a lot fewer reads than writes. Right. There's just a lot of churn on the amount of stuff that was going through versus the amount of queries. I wasn't thinking too much about that. I was mostly just thinking about what's the fundamentally cheapest way to build a database in the cloud today using the primitives that you have available. And this is it, right? You just. Now you have one machine and let's say you have a terabyte of data in S3. You pay the $200 a month for that and then maybe 5 to 10% of that data needs to be in NVMe SSDs and less than that in DRAM. Well, you're paying very, very little to inflate the data.

[C]

By the way, when you say no one else has done that, would you consider NEON to be on a similar path in terms of being sort of S3 first and separating the compute and storage.

[A]

Yeah, I think what I meant with that is just build a completely new database. I don't know if we were the first. I mean, I hadn't. I just looked at the napkin math and I was like, this seems really obvious. So I'm sure, like 100 people came up with it at the same time. Like the light bulb and every invention ever, right? It was just in the air. I think NEON was first to it and they're trying. They're retrofitted it onto postgres. Right? And then they built this whole architecture where you have it in memory and then you sort of like, you know, mmap back to S3. And I think that was very novel at the time to do it for oltp. But I hadn't seen a database that was truly all in. Right. Not retrofitting it. The database built purely for this. No consensus layer. Even using compare and swap on OPIC stories to do consensus. I hadn't seen anyone go that all in. And I mean, I'm sure there was someone that did that before us. I don't know. I was just looking at the napkin math.

[C]

When you say consensus layer, are you strongly relying on S3 strong consistency? You are. Okay, so that is your consensus layer.

[A]

It is the consistency layer. And I think also, like, this is something that most people don't realize, but S3 only became consistent in December of 2020.

[C]

I remember this coming out during COVID and like, people were like, oh, it was like. It was just like a free upgrade.

[A]

Yeah, they were just.

[C]

They just announced it. We saw consistency, guys. And like, okay, cool.

[A]

And I'm sure that they just. They probably had it in prod for a while and they're just like, it's done. Right? And people are like, okay, cool. But that's a big moment, right? Like, NVMe SSDs were also not in the cloud until around 2017, right? So you just sort of had like, like 2017 NVMe SSDs. And people are like, okay, cool. There's like, one skew that does this. Whatever, right? Takes a few years. And then the second thing is, like, S3 becomes consistent in 2020. So now it means you don't have to have this, like, big foundation DB or like, Zookeeper or whatever sitting there contending with the keys, which is how, you know, that's what Snowflake and others that have gone. Exactly. Just gone. Right? And so just push to the, you know, whatever. How many hundreds of people they have working on S3 solved and then compare and swap was not in S3 at this point in time, by the way.

[C]

I don't know what that is, so maybe you want to explain.

[A]

Yes, yes. So what compare and swap is, is basically you can imagine that if you have a database, it might be really nice to have a file called Metadata JSON. And metadata JSON could say things like, hey, these keys are here and this file means that, and there's lots of metadata that you have to operate in the database. Right? But that's the simplest way to do it. So now you might have a lot of servers that want to change the metadata. They might have written a file and want the metadata to contain that file. But you have a hundred nodes that are trying to contend with this metadata JSON. Well, what compare and swap allows you to do is basically just you download the file, you make the modifications and then you write it. Only if it hasn't changed while you did the modification. And if not, you retry. Right? You should just have this retry loops. Now you can imagine if you have 100 nodes doing that, it's going to be really slow, but it will converge over time. That primitive was not available in S3. It wasn't available in S3 until late 2024, but it was available in GCP. The real story of this is certainly not that. I sat down and like big brained it. I was like, okay, we're going to start on GCS. S3 is going to get it later. Like, it was really not that we started. We got really lucky. Like we started on GCP and we started on gcp because Shopify ran on gcp. And so that was the platform I was most available with. Right. And I knew the Canadian team there because I'd worked with them at Shopify. And so it was natural for us to start there. And so when we started building the database, we're like, oh yeah, we have to build. We really thought we had to build a consensus layer, like have a zookeeper or something to do this. But then we discovered the compare and swap was like, oh, we could kick the can. We'll just do metadata JSON and just. It's fine, it's probably fine. And we just kept kicking the can until we had very, very strong conviction in the idea. And then we kind of just hinged the company on the fact that S3 probably was gonna get this. It started getting really painful in like mid 2024 because we were closing deals with Notion, actually that was running in aws. And we're like, trust us, you really Want us to run this in gcp. And they were like, no, I don't know about that. Like we're running everything in aws. And the latency across the cloud was so big and we had so much conviction that we bought like, you know, dark fiber between the AWS regions in Oregon, like in the Inter Exchange. And GCP is like, we've never seen a startup like what's going on here? And we're just like, no, we don't want to do this. We were tuning like tcp, windows, everything to get the latency down because we had so high conviction in not doing a metadata layer on S3. So those were the three conditions, right? Compare and swap to do metadata, which wasn't in S3 until late 2024, S3 being consistent, which didn't happen until December 2020. And then NVMe SSDs, which didn't land in the cloud until 2017.

[C]

I mean, in some ways like a very big cloud success story that you were able to put this all together. But also doing things like buying dark fiber, that actually is something I've never heard.

[A]

I mean it's very common when you're a big company, right. You're like connecting your own data center or whatever. But it's like it was uniquely just a pain with notion because

[C]

if you're

[A]

buying in Ashburn, Virginia, right, like US east, the Google, like the GCP and AWS data centers are like within a millisecond on each other on the public exchanges. But in Oregon, uniquely the GCP data center sits like a couple hundred kilometers east of Portland and the AWS region sits in Portland. But the network exchange they go through is through Seattle. So it's like a full like 14 milliseconds or something like that. And so anyway, yeah, so we were like, okay, we have to go through an exchange in Portland. Yeah.

[C]

And you'd rather do this than like run your Zookeeper and like yes, way

[A]

rather it doesn't have state. I don't want state in two systems. And I think all of that is just informed by Justine, my co founder. And I had just been on call for so long. And the worst outages are the ones where you have state in multiple places that's not syncing up. So it really came from just a very pure source of pain of just imagining what we would be okay being woken up at 3am about and having something in zookeeper was not one of them.

[C]

We were talking to like a notion or something. Do they care or do they just,

[A]

they just Care about latency, latency cost.

[C]

That's it.

[A]

They just cared about latency. Right. And we just absorb the cost. We're just like, we have high connection in this. At some point we can move them to aws. Right. And so we just, we'll buy the fiber. It doesn't matter. Right? And it's like 5,000. And we usually when you buy fiber, you buy like multiple lines and we're like, we can only afford one, but we will just test it. That when it goes over the public Internet, it's like super smooth. And so we did a lot of. Anyway, it's. Yeah, it was. That's cool.

[B]

Can imagine talking to GCP rep and it's like, no, we're going to buy because we know we're going to churn. We're going to turn from you guys and go to AWS in like six months. But in the meantime we'll do this.

[A]

I mean like they, you know, this workload still runs on gcp for what it's worth. Right? Because it's so. It was just, it was so reliable. So it was never about moving off gcp. It was just about honesty. It was just about giving notion the latency that they deserved. Right. And we didn't want them to have to care about any of this. We also, they were like, oh, egress is going to be bad. It was like, okay, screw it. Like we're just going to like VPC peer with you and aws. We'll eat the cost. Yeah, whatever needs to be done.

[B]

And what were the actual workloads? Because I think when you think about AI, it's like 14 milliseconds. It doesn't really matter in the scheme of like a model generation.

[A]

Yeah. We were told the latency. Right. That we had to beat.

[B]

Right.

[A]

So we're just looking at the traces. Right. And then sort of like hand like, you know, kind of like looking at the trace and then thinking, what are the other extensions of the trace? Right. And there's a lot more to it because it's also when you have. If you have 14 versus 7 milliseconds, right, you can fit another round trip. So we had to tune TCP to try to send as much data in every round trip, pre warm all the connections. And there was. There's a lot of things that compound from having these kinds of round trips. But in the grand scheme it was just like, well, we have to beat the latency of whatever we're up against.

[C]

Which is like they, I mean notion is a database company they could have done this themselves. They do lots of database engineering themselves. How do you even get in the door? Just talk through that kind of.

[A]

Last time I was in San Francisco, I was talking to one of the engineers actually, who was one of our champions at Notion, and they were just trying to make sure that the per user cost matched the economics that they needed. It's like the way I think about is I have to earn a return on whatever the clouds charge me and then my customers have to earn a return on that. And it's like very simple. Right. And so there has to be gross margin all the way up and that's how you build the product. And so then our customers have to make the right set of trade offs that Turbo Buffer makes. And if they're happy with that, that's great.

[C]

Do you feel like you're competing with build internally versus Buy or buy versus buy?

[A]

Yeah. So sorry, this was all to Build up to your question. So one of the Notion engineers told me that they'd sat and probably on a napkin, like, drawn out like, why hasn't anyone built this? And then they saw Turbo Buffer, like, well, literally that. So. And I think AI has also changed the Buy versus build equation in terms of it's not really about can we build it, it's about do we have time to build it? And I think they, like, I think they felt like, okay, if this is a team that can do that and they feel enough of like an extension of our team, well, then we can go a lot faster, which would be very, very good for them. And I mean, they put us through the, through the test. Right. Like, we had some very, very long nights to, to do that poc and they were really our second big customer after Cursor, which also was a lot of late nights. Right?

[B]

Yeah.

[C]

I mean, should we go into that story, the sort of Cursor story? They credit you a lot for working very closely with them. So I just want to hear. I've heard this story from Swale's point of view, but I'm curious what it looks like from your side.

[A]

I actually haven't heard it from Swaley's point of view, so maybe you can now cross reference it. The way that I remember it was that the day after we launched, which was just, you know, I'd worked the whole summer on the first version. Justine wasn't part of it yet because I just, I didn't tell anyone that summer that I was working on this. I was just locked in on building it because it's Very easy otherwise to confuse talking about something to actually doing it. And so I was just like, I'm not going to do that. I'm just going to do the thing. I launched it and at this point, TurboPuffer is like a rust binary running on a single eight core machine in a TMUX instance. And me deploying it was like looking at the request log and then like command seeing it or like control seeing it to just like, okay, there's no request, let's upgrade the binary. Like, it was like literally the scrappiest thing you could imagine it was on purpose. Because just like at Shopify we did that all the time. Like we like move. Like we ran things in TMUX all the time to begin with before something had at least the inkling of pmf. So it's like, okay, is anyone going to hear about this? And one of the Cursor co founders, Arvid, reached out and he just, you know, the, the Cursor team are like all I am. Oh, like contenders. Right. So they just speak in bullet points and facts. There's like this amazing email exchange just of this is how many QPS we have. This is what we're paying, this is where we're going, blah, blah, blah. And so we're just conversing in bullet points. And I tried to get a call with them a few times, but they were so. They were like really riding the PMF bull here. Just like late 2023. And one time Swaley emails me at like at 5. No, what was it? Like 4am Pacific Time? Saying like, hey, are you open for a call now? And I'm on the east coast. And it was like 7:00am like, yeah, great, sure, whatever. And we just started talking and something then I didn't know anything about sales. It was something that just compelled me. I have to go see this team. Like, there's something here. So I, I went to San Francisco and I went to their office. And the way that I remember it is that Postgres was down when I showed up at the office. Did Swaley tell you this? No. Okay. Postgres was down. And so it's like they were distracting with that. And I was trying my best to see if I could help in any way. Like I knew a little bit about databases. Back to tuning Autovacuum, it's like, I think you have to tune out of vacuums. And so we talked about that and then that evening just talked about what would it look like? What would it look like to work with us? And I just said, look, we're all in. We will just do what we'll do whatever, whatever you tell us. Right. They migrated everything over the next like week or two and we reduced their cost by 95% which I think like kind of fixed their per user economics. And it solved a lot of other things. And we were just. Justine. This is also when I asked Justine to come on as my co founder, she was the best engineer that I ever worked with at Shopify. She lived two blocks away and we were just, okay, we're just going to get this done. And we did. And so we helped them migrate and we just worked like hell over the next month or two to make sure that we were never an issue. And that was the Cursor story. Yeah.

[C]

And is code a different workload than normal text? I don't know. Is it just text? Is it the same thing?

[A]

Yeah. So Cursor's workload is basically they will embed the entire code base. Right. So they will chunk it up in whatever they do. They have their own embedding model which they've been public about and they find that on their evals. It's, there's one other evals where it's like a 25% improvement on a very particular workload. They have a bunch of blog posts about it. I think it works best on larger code bases. But they've trained their own embedding model to do this. And so you'll see it if you use the Cursor agent, it will do searches. And they've also been public around how they've, I think they post trained their model to be very good at semantic search as well. And that's, that's how they use it. And so it's very good at like can you find me on the code that's similar to this or code that does this? And just in disc queries. They also use GREP to supplement it.

[C]

Yeah, of course it's been a big topic of discussion is raggedead because grep,

[A]

you know, and I mean like I just, we, we see lots of demand from the coding company.

[C]

Search every part.

[A]

Yes, we, we, we see demand. And so I mean I'm, I like case studies. I don't like, like just doing like thought pieces on this is where it's going. And like trying to be all macroeconomic about AI that has turned out to be a giant waste of time because no one can really predict any of this. So I just collect case studies and I mean Cursor has done a great job talking about what they're doing, and I hope some of the other coding labs that use TurboPuffer will do the same. But it does seem to make a difference for particular queries. I mean, we can also do tags, we can also do regex. But I should also say that cursors like security posture into TurboPuffer is exceptional. Right. They have their own embedding model which makes it very difficult to reverse engineer. They obfuscate the file paths. It's very difficult to learn anything about a code base by looking at it. And the other thing they do too is that for their customers, they encrypt it with their encryption keys in TurboPuffer's bucket. So it's really, really well designed.

[C]

And so this is like extra stuff they did to work with you because you are not part of Cursor.

[A]

Exactly.

[C]

And this is just best practice when working on any database, not just you guys. Okay, yeah, that makes sense. Yeah. I think for me the learning is kind of like all workloads are hybrid. You want the semantic, you want the text, you want the regex, you want SQL. I don't know, but it's silly to be all in on one particularly query pattern.

[A]

I think I really like the way that Swaley at Cursor talks about it, which is I'm going to butcher it here. And I'm a database scalability person. I don't know anything about training models other than what the Internet tells me. And the way he describes is that this is just like cache compute. Right. It's like you have a point in time where you're looking at some particular context and focused on some chunk and you say this is the layer of the neural net at this point in time that seems fundamentally really useful to do cache compute like that. And how the value of that will change over time, I'm not sure, but there seems to be a lot of value in that.

[B]

Maybe talk a bit about the evolution of the workload, because even like search, like maybe two years ago it was like one search at the start of like an LLM query to build the context. Now you have a gentic search, however you want to call it, where the model is both writing and changing the code and it's searching it again later. Yeah. What are maybe some of the new types of workloads or changes you've had to make to your architecture for it?

[A]

I think you're right. When I think of rag, I think of, hey, there's an 8,000 token context window and you better make it count. And search was A way to do that now? Yeah. Everything is moving towards just let the agent do its thing. Right. And so back to the thing before. Right. The LLM is very good at reasoning with the data and so we're just the tool call. Right. And that's increasingly what we see our customers doing. What we're seeing more demand from from our customers now is to do a lot of concurrency. Right. Like Notion does a ridiculous amount of queries in every round trip just because they can't. And I'm also now when I use the cursor agent, I also see them doing more concurrency than I've ever seen before. So a bit similar to how we designed a database to drive as much concurrency in every round trip as possible. That's also what the agents are doing. So that's new. It means there's an enormous amount of queries all at once to the data set while it's warm in as few turns as possible.

[C]

Can I clarify one thing on that?

[A]

Yes.

[C]

Are they batching multiple users or one user is driving multiple.

[A]

One user driving multiple.

[C]

One agent driving parallel searching a bunch of things.

[A]

Exactly, yeah.

[C]

So the connation also did this for the fast context thing, like eight parallel at once.

[A]

Yes.

[C]

An interesting problem is, well, how do you make sure you have enough diversity so you're not making the same request eight times?

[A]

And I think that's probably also where the hybrid comes in, where that's another way to diversify. It's a completely different way to do the search. That's a big change. Right. So before it was really just like one call and then the LLM took however many seconds to return. But now we just see an enormous amount of queries, so we just see more queries. So we've like tried to reduce query, we've reduced query pricing. This is probably the first time actually, I'm saying that. But the query pricing is being reduced like 5x and we'll probably try to reduce it even more to accommodate some of these workloads of just doing very large amounts of queries. That's one thing that's changed. I think the write ratio is still very high. Right. Like there's still an enormous amount of writes per read, but we're starting probably to see that change if people really lean into this pattern.

[B]

Can we talk a little bit about the pricing? I'm curious because traditionally a database would charge on storage, but now you have the token generation that is so expensive where like the actual value of like a good search query is like much higher because they're like saving inference time down the line. How do you structure that as like, what are people receptive to on the other side too?

[A]

Yeah. The turbopuffer pricing in the beginning was just very simple. The pricing on these for search engines before TurboPuffer was very serverful. Right. It was like, here's the VM, here's the PER hour cost. Right? Great. And I just sat down with like a piece of paper and said like, if turbopuffer is like really good, this is probably what it would cost with a little bit of margin. And that was the first pricing of TurboPuffer. And I just like sat down. I was like, okay, like this is like probably the storage amp or whatever on a piece of paper.

[C]

It was Vibe pricing.

[A]

It was very Vibe price. And I got it wrong. Oh, well, I didn't get it wrong, but like Turbo Puffer wasn't at the first principle pricing. Right. So when cursor came on TurboPuffer, it was like, like I didn't know any VCs. I didn't know like, I just, like, I don't know. I didn't know anything about raising money or anything like that. I just saw that my GCP bill was, was hot. Was a lot higher than the cursor bill. So Justine and I were just like, well, we have to optimize it. And I mean to the chagrin now of, of it, of, of the VCs, it now means that we're profitable because we had so much pricing pressure in the beginning because it was running on my credit card and Justine and I had spent like, like tens of thousands of dollars on like compute bills and like spinning off the company and like very like, like bad Canadian lawyers and like things like to like get all of this done because we just like, we didn't know. Right. If you're like steeped in San Cisco, you're just like, you just know. Okay, like you go out, raise the pre seed round. I, I never heard the word pre seed at this point in time.

[C]

When you had Crystal, you had notion you had no funding.

[A]

With Cursor, we had no funding. Yeah. By the time we had notion, Lockheed was here. Yeah. So it was really just. We Vibe priced it 100% from first principles, but it wasn't, it was not performing at first principles. So we just did everything we could to optimize it in the beginning for that so that at least we could have like a 5% margin or something. So I wasn't freaking out because Cursor's bill was also going like this as they were growing. And so my liability and my credit limit was like actively calling my bank. It's like I need a bigger credit. Like it was. Yeah. Anyway, that was the beginning. Yeah. But the pricing was. Yeah. Like storage rights and quarry. Right. And the, the pricing we have today is basically just that pricing with duct tape and spit to try to approach like you know, like a margin on the physical underlying hardware. And we're doing this year, you're going to see more and more pricing changes from us. Yeah.

[C]

And like is how much does stuff like VPC peering matter? Because you're working in aws, LAN where egress is charged and all that.

[A]

You know, we probably don't like, we have like an enterprise plan that just has like a base fee because we haven't had time to figure out SKU pricing for all of this. But I mean, yeah, you can run TurboPuffer either in SaaS. Right. That's what Cursor does. You can run it in a single tenant cluster. So it's just you. That's what notion does. And then you can run it in BYOC where everything is inside the customer's vpc. That's what for example, Anthropic does.

[C]

What I'm hearing is that this is probably the best CRO job for somebody who can come in and help you with this.

[A]

Like Turbo Puffer hired like I don't know what, what number this was, but we had a full time CFO as like the 12th hire or something at Turbo Puffer. I think I hear a lot of comp. I don't know how they do it. Like they have a hundred employees and not a cfo. It's like having a CFO is like you run out of business, man. Like, you know, it's so good. Yeah, like money Mike, like he just, you know, just handles the money and a lot of the business stuff. And so he came in and just helped with a lot of the operational side of the business. So like coo, cfo, like somewhere in between.

[C]

Just a quick mention of Lockheed just because I'm curious. I've met Laki and like he's obviously a very good investor now on physical intelligence, I call it a generalist super angel.

[B]

Right.

[C]

He invests in everything. And I always wonder like, you know, is there something appealing about focusing on developer tooling, focusing on databases, going like I've invested for 20 years in databases versus being like a Laki where he can maybe like connect you to all the customers that you need.

[A]

This is an excellent question. No one's asked Me this. Why Locky? Because there was a couple of people that we were talking to at the time and when we were raising, we were almost a little. We were like a bit distressed because one of our peers had just launched something that was very similar to turbopuffer. And someone just gave me the advice at the time of just choose the person where you just feel like you can just pick up the phone and not prepare anything and just be completely honest. And I don't think I. I've said this publicly before, but I just called Locky. I was like, local. Ay. Like, if this doesn't have PMF by the end of the year, like, we'll just like return all the money to you. But it's just like, I don't really. Justine and I don't want to work on this unless it's really working. So we want to give it the best shot this year and like, we're really going to go for it. We're going to hire a bunch of people and we're just going to be honest with everyone. Like, when I don't know how to play a game, I just play with open cards. And Locky was the only person that didn't, that didn't freak out.

[C]

Out.

[A]

He was like, I've never heard anyone say that before. As I said, I didn't even know what a seed or precede round was like before, probably even at this time. So I was just like very honest with him. And I asked him like, locky, have you ever, have you ever invested in database company? He was just like, no. And at the time I was like, am I dumb? Like, but I think there was something that just like really drew me to Lockheed. He is so authentic, so honest. Like, and there's something just like, I just felt like I could just play, like, just say everything openly. And that was, that was. I think that that was like a perfect match at the time. And, and, and honestly still is. He was just like, okay, that's great. This is like the most honest, ridiculous thing I've ever heard anyone say to me. But like that, like that. Why is it ridiculous to say competitor launch, this may not work out? It was more just like, if this doesn't work out, I'm going to close up shop by the end of the month, the year. Right, right. Like it was. I don't know, maybe it's common, I don't know. He told me it was uncommon. I don't know. That's why we chose him. And he'd been phenomenal the other People were talking at the time were database experts, like they knew a lot about databases and Lockheed didn't. This turned out to be a phenomenal asset. Right. Like Justine and I know a lot about databases. The people that we hire know a lot about databases. What we needed was just someone who didn't know a lot about databases, didn't pretend to know a lot about databases, and just wanted to help us with candidates and customers. And he did. And I have a list, right, of the investors that I have a relationship with. And Locky has just performed excellent in the number of sub bullets of what we can attribute back to him. Just absolutely incredible. And when people talk about like no ego and just the best thing for the founder, I like, I don't think that anyone, like even my lawyer is like, yeah, Locky is like the most friendly person you will find.

[C]

Okay. This is my most glowing recommendation I've ever heard.

[A]

He deserves it. He's very special.

[B]

Yeah, yeah.

[C]

Okay. Amazing.

[B]

Since you mentioned candidates, maybe we can talk about team building. You know, like, especially in sf, it feels like it's just easier to start a company than to join a company. I'm curious your experience, especially not being in SF full time and doing something that is maybe, you know, a very low level of detail and technical detail.

[A]

Yeah. So joining versus starting. I never thought that I would be a founder. I would start with it. Like Turbo Puffer started as a blog post and then it became a project and then sort of almost accidentally became a company. And now it feels like it's, it's like becoming a bigger company. That was never the intention. The intentions were very pure. It's just like, why hasn't anyone done this? And it's like I want to be the, like, I want to be the first person to do it. I think some founders have this. Like, I could never work for anyone else. I, I really don't feel that way. Like, it's just like I want to see this happen and I want to see it happen with some people that I really enjoy working with and I want to have fun doing it. And this, this has all felt very natural on that, on that sense. So it was never like join versus versus versus found. It was just this found me at the right moment.

[B]

Well, I think there's an argument for you should have joined cursor. Right. So I'm curious, like how you evaluate it.

[C]

Okay.

[B]

I should actually go raise money, make this a company versus like, this is like a company that is like growing like crazy. It's like an interesting technical Problem, I should just build it within cursor. And then they don't have to encrypt all this stuff. They don't have to obfuscate things like, was that on your mind at all?

[A]

Or before taking the small check from Locky, I did have like a hard like, look at myself in the mirror of like, okay, do I really want to do this? And because if I take the money, I really have to do it right. And so the way I almost think about it is like, you kind of need to like, you kind of need to be like fucked up enough to want to go all the way. And that was the conversation where I was like, okay, this is going to be part of my life's journey to build this company and do it in the best way that I possibly can. Because if I ask people to join me, ask people to get on the cap table, then I have an ultimate responsibility to give it everything. And I think some people, it doesn't occur to me that everyone takes it that seriously and maybe I take it too seriously, I don't know. But that was like a very intentional moment. And so then it was very clear like, okay, I'm gonna do this and I'm gonna give it everything.

[B]

A lot of people don't take it this seriously.

[C]

Let's talk about. You have this concept of the P99 engineer. People are 10xing. Everyone's saying maybe engineers are out of a job, I don't know. But you definitely see a P99 engineer and I just want you to talk about it.

[A]

Yeah. So the P99 engineer was just a term that we started using internally to talk about candidates and talk about how we wanted to build the company. And you know, like, everyone else is like, we want a talent dense company. And I think that's almost become trite at this point. What I credit the cursor founders a lot with is that they just arrived there from first principles of like, we just need a talent dense, talent dense team. And I think I've seen some teams that weren't talent dense and like seen the counterfactual run, which if you've run in been in a large company, you will just see that like it's just logically will happen at a large company. And so that was super important to me and Justine and it's very difficult to maintain. And so we just needed, we needed wording for it. And so I have a document called traits of the P99 engineer and it's a bullet point list. And I look at that list after every single interview that I do and in every single recap that we do and every recap we end with, I end with some version of, I'm going to reject this candidate completely irregardless of what the discourse was. Because I want to see people fight for this person. Because the default should not be, we're going to hire this person. The default should be, we're definitely not hiring this person. And, you know, if everyone was like, I'll maybe throw a punch, then this is not the rights do you operate?

[C]

Like, if there's one chance, they must have at least one champion who's like, yes, I will put my career on. On the line for this.

[A]

You know, I think career on the line, yeah. You know, like, I would say someone needs to, like, have both fists up and be like, I'd fight, right? And if one person said, then okay, let's do it right, and it doesn't have to be absolutely everyone. Right. And like, the interviews are always the sign that you're checking for different attributes. And if someone is like, like knocking it out of the park in every single attribute, that's fairly rare, but that's really important. And so the traits of the P99 engineer, there's lots of them. There's also the traits of the P, like triple nine engineer and the quadruple nine engineer. This is like, it's a long list. I'll give you some samples. Right. Of what we look for. I think that the P99 engineer has some history of having bent, like, their trajectory or something to their will. Right. Some moment where it was just. They just, you know, made the computer do what it needed to do. There's something like that. And it will occur to them at some point in their career and hopefully multiple times. Right.

[C]

Give me an example of one of your engineers that, like, I'll give an engine.

[A]

So we launched this thing called ANN v3. We're also. We're working on v4 and v5 right now. But ANN v3 can search 100 billion vectors with a p50 of around 40 milliseconds and a p99 of 200 milliseconds. Maybe other people have done this. I'm sure Google and others have done this, but we haven't seen anyone, at least not in a publicly consumable SaaS that can do this. And that was an engineer, the Chief Architect of TurboPuffer Nathan, who more or less just bent the software, was not capable of this, and he just made it capable for a very particular workload in A six to eight week period with the help of a lot of the team. There's numerous examples of that at Turbo buffer, but that's really bending the software and x86 to your web. Well, it was incredible to watch. You want to see some moments like that?

[C]

Isn't that triple N?

[B]

I think that was only 29.

[C]

I feel like this is too high for Nathan.

[A]

Nathan is. Nathan is like, yeah, there's a lot of nines after that P. So I think that's one trait. I think another trait is that the P99 spends a lot of time looking at maps. Generally it's their preferred ux. They just love looking at maps. You ever seen someone who just like sits on their phone and just like scrolls around on a map or did you not look at maps a lot? You guys don't look at maps?

[C]

I guess I'm not feeling there. I don't know, maybe you just.

[A]

Disqualified. What about trains? Do you like trains?

[C]

I mean, not enough. Okay, this is just like weaponized autism is what I call it.

[B]

Like

[A]

I love looking at maps. Like, it's like my preferred UX and just like I, you know, I like

[B]

lots of places instead of like random places.

[C]

So like, you know.

[A]

Yes.

[C]

Okay, there you go.

[B]

So instead of like random places, this like how do you explore the maps?

[A]

No, it's.

[C]

It's just a joke, a lab. It's like you are just obsessed by something and you like studying a thing.

[A]

The origin of this was that at some point I read an interview with some I.O.I. gold medalist and it's like, what do you do in your spare time? It's just like I like looking at maps and I was like, I feel so seen. Like I just like love like scrolling out like, oh, Canada is so big. Where's Baffin Island? I don't know. I love it. Anyway, okay, so the traits of P99. P99 is obsessive, right? Like there's just like you'll find traits of that we do an interview at TurboPuffer or like multiple interviews that just try to screen for some of these things. So there's lots of others. But these are the kinds of traits that we look for.

[C]

I'll tell you, some people listen for like some of my DevRel stuff. I do think about DevRel as maps. You draw a map for people maps show you the what is commonly agreed to be the geographical features of what a boundary is and it also shows you what is not doing. And I think a lot of developer tools companies try to tell you they can do everything. But let's be real, your three landmarks are here. Everyone comes here, then here, then here. And you draw a map and then you draw a journey through the map. And to me that's what developer relations looks like. So I do think about things that way.

[A]

I think the P99 thinks in trade offs, right? The P99 is very clear about, you know, hey, turbopuffer, you can't run a high transaction workload on TurboPuffer, right? It's like the right latency is 100 milliseconds. That's a clear trade off. I think the P99 is very good at articulating the trade offs in every decision, which is exactly what the map is in your case, right?

[C]

Yeah, yeah. My world, my world.

[B]

How do you reconcile some of these things? When you're saying you bend the will of the computer versus the trade offs, I think sometimes it's like, well, these are the trade offs. But the three nines is like actually it's not a real trade off because we can make something that nobody has ever made before and actually make it work.

[A]

The way I think about the bending trajectory to your will is if you sit down and do the napkin math, right? Where you're just like, okay, if I have 100 machines, they have this many terabytes of disk, they have this bandwidth, whatever, right? And you sit down and you just do the high school napkin math on this is how many QPS we should be able to drive to it. Similar to how I did the Vibe pricing, right? If you can sit down and do that and then you observe the real system and you see, oh, we're off by like 10x. Bending trajectory to your will is like just making the software get closer and closer to that first principle line. The P99 might even be able to cross the line, right? By finding even more optimizations than from first principle. So bending the software to your will is about that. That right? Like 100 millisecond P99 to, to S3. I mean now you're talking like someone really high agency that like goes to Seattle, finds the S3 team and it's like, how are we going to make this 10? You know, like it's. That's not quite what we talk about, right? But yeah.

[C]

What's the future? TurboPuffer?

[A]

Turbo Puffer started out. Act one of Turbo Puffer was Vector search. That was all we did to begin with. Act 2 of Turbo Puffer is, is and was full text search. Turbopuffer today has a fairly state of the art full text search engine. We beat Lucene on some queries in particular very long queries that we've optimized for because those are the text search queries we see today. They're generated by LLMs or augmented by LLMs and we see them on web scale data sets, right? Like someone searching for a very long text string on all of Common Core all we beat Lucene on some of those benchmarks and we expect to continue to beat Lucene on more and more queries. That's the performance and scale Turbo Buffer does phenomenally now at full text search performance and scale. What we work on now is more and more features for full text search. People expect a lot of features with full text search and full text search is still very valuable. Right? If you go in and you press command K and you search for si, an embedding based search might be like oh, this is something agreeable because that's C. That's yes in Spanish, right.

[B]

In Italian too.

[A]

But in full night search that's the prefix of maybe a document of like, you know, these are all the reasons I hate Simon. Right? Like this is like, that's a completely different. So that augmentation to like how the human brain works and mapping like data to user is very important. But it's a lot of features that feature grind is what we're firmly on and you will see us just adding to the changelog every month, just more and more full text search features. So we're like fully compatible and we're seeing people move from some of the traditional search engine onto TurboPuffer for that. That's a big focus of TurboPuffer this year. The other focus of TurboPuffer this year is just on scale. We're seeing more and more companies that want to search basically common crawl level types of data sets both internally accompanied and externally at a time like Corey, like 100 billion vectors or 100 billion documents at once. This is trick and we want to make it cheaper and we want to make it faster. That's a big focus for Turbo Buffer this year. That's, you know, we just released ANN V3, which we talked about before. We're working on ANN V4 and we also have planned what we're going to do with Ann V5. Right. And then on FullTech Search we're working on a lot of these features will be like FTS v3, but it will all roll out incrementally. Those are some of the really big features. And then the other thing is our dashboard. Have any of you ever logged into the TurboHoffer dashboard? Dashboard, there's not very much there. It almost looks like if a founder 2 years ago just sat down and wrote enough dashboard that there was at least something there and then other people just sort of added stuff on for the next two like the, the following two years and then at some point SSO and other things to just catch up and it may or may not be what happened, but adding like I want PHP my admin back. Like do you, do you guys remember, like it was, it was so good. Right. And I think that like software hardware integration between the dashboard of the console of the database and the database itself. I'm really excited for that. There's lots of other things that are going to come out in the next. Like we talked a bit about some pricing and things like that, but those would be some of the big hitters right now.

[C]

You talk about eras of turbofar. I just, I have to ask like yes, there's the stuff that you're working on this year but like I'm sure in your mind you already have the next phase that you're already thinking about.

[A]

Act three.

[C]

Yes.

[A]

Act four.

[C]

Yeah.

[A]

Act five, what I was saying about

[C]

that none of the candidates, you don't have to decide.

[A]

Yeah. But you know, I'll just say that if you want to build a big database company, the database over time has to implement more or less every query plan because when you have your data in a database, you expect it to over time not just search, but also, hey, I want to aggregate this column, I want to join this data, all of that, that. But when you're a startup, your only moat is really just focus. So you have to lay out the facts and you have to not get overeager. And I think we've seen some of our peers get very overeager and overextend themselves. And what I keep telling the team, I was just having breakfast this morning with our CTO and chief architect and we were talking about like what we're most likely to regret at the end of the year is having tried to do too much and so Act 3 candidates could be, you know, a bunch of simpler OLAP queries. Right. It could be lending ourselves a little bit more into. We see some people who want to do traces and logging and things like that. Some very simple use cases could be that. Right. It could be maybe some time series. Some people are trying to do that. Right. Like there's lots of different things that you can do with TurboPuffer but for now, like if you're trying to do Not Search on TurboPuffer is the primary use case. You probably shouldn't, but we see some customers that are like oh, like at some point Cursor moved like 20 terabytes of Postgres data into TurboPuffer because it's there, it works. And these particular query plans we know work well and so they just moved it all to defer sharding. So we look for patterns like that in what future acts of turbopuffer are going to be before firmly doubling down on them. But we wouldn't today if you're using turbopuffer it should be because search is very important to you and then we might do a lot of auxiliary queries to that. But that should not be the main reason to go to turbopuffer at this point in time.

[C]

Yeah, you didn't mention. One thing I was looking for was graph type queries like graph database graph queries. Can you basically trivially replicate this with what you already have?

[A]

We see some people doing that because

[C]

you have parallel queries and it's the same thing.

[A]

Exactly. So we see some people doing that. Right. Like at the under. Like turbo buffer is just a kv. Right. And then we expose things on top of it. So we are seeing people do that. And I think our roadmap is very much just the database that connects AI to a very large amount of data is what the path is to do that in the right order, which is what a good startup is around. What is the order to do things in. Our customers are P99 and they will tell us what they care most about next. And so some of them are doing graphs now and if they need more graph database features, they'll be banking our door and we'll prioritize accordingly.

[C]

Okay, give us the tea you kindly gifted us your favorite tea. This is Yabukita Kamaruicha from the Green Tea Shop.

[A]

That's right.

[C]

Tell me about your love of tea.

[A]

Yeah, we were just talking beforehand about caffeine, I think and especially when I'm on a trip like this to San Francisco. I consume a lot of caffeine, but this is my preferred, preferred caffeine. It's this green tea. I have an air table with 200 teas that I've tried over time over the past like 15 years and this one is my favorite. Now when you drink a tea there's different. There's like six different types of tea. I like green tea in particular, I generally prefer for Chinese green tea and I don't really like Japanese Green tea. But this little prefecture somewhere in Japan has specialized in like, they're like Japanese, but doing it the Chinese way. And it's just phenomenal. But then the interesting thing about the tea world is that all of the different, like you can find this particular tea, there's probably, you know, hundreds of places that sell it, but they all go to a different family. Right. On whatever mountain that they have these. These like Camellia sinensis bushes on. And this woman, Japanese woman in Toronto from the green tea shop. I don't know, she just like. I found a really good family because that's the best one. The best time of year to get this is in a few months when they do the spring harvest. Now it's like kind of old. It's just like, I love the spring for the fresh tea, so I hope you enjoy it, but it's not the right time of year.

[C]

It's out of season. Yeah, I actually didn't even know tea has seasons. This is how unsophiscated I am. But I think, like, it ties in with like, you know, loving maps and being obsessed and being P99 everything that you do. Yeah, but that's great.

[B]

Awesome. Well, as we were saying, we have instant hot water at Kernel, so met lover can come by any.

[A]

I have a little T kit where I bring a. Where I bring like a little thermometer to like a little thermal works thermometer. Last Friday, when we do demos, I have this thing where if there's not enough demos, then I fill the remaining time talking about something completely ridiculous as an incentive for people to actually demo. And last time I spent 20 minutes walking through my air table and going through my entire T travel kit, including the temperature monitor. Because, like, yeah, you show up, there's only a boiler, you can't get it to the right. You know, you need this at 80 degrees. But anyway, yeah, sorry, we have an

[B]

electric kettle with the temperature thing at home.

[C]

Now, I would watch this. You should start a company, YouTube, but it doesn't have anything about search. It just has tea and like other rants.

[A]

I don't think I could talk, but something that I started doing. Do. Do you two know Sam Lambert of Planet Scale?

[C]

Of course. Very outspoken guy.

[A]

I love the guy. And we just. We just last, like last week we just went on X Live and just sat and like shut the. For like an hour. And I think we'll probably do that again. Yes, we'll probably come up there. Well, I don't know what we'll call maybe P99 live. Or the P99 pod or something like that.

[C]

That peod.

[A]

Peod.

[C]

Cool. Well, thank you so much for your time. I know you have to go, but this is a blast, and you're clearly very passionate and charismatic, so I. I bet you'll get some P99 engineers out of this podcast.

[B]

Yeah.

[A]

Thank you so much for having me. It was a pleasure.