Loading summary
Transcript1 lines
- [00:00]
A
This episode is the clearest explanation of Firecrawl on the Internet and how you can use it to build a real business that makes you real money. Firecrawl feels like giving your AI eyes right now. AI is smart, but it's blind. It can't see the Internet, it can't go to a website, it can't grab data. So Firecrawl fixes that. Once you see it in action, it changes how you think about building products, how you think about collecting data, and and how you think about what's possible with AI. In this episode, I break down what Firecrawl actually is, how it plays into your AI stack and walk you through a bunch of startup ideas that you can make money from it. I use firecrawl with ideabrowser.com and I reach out to them to ask them to sponsor this video. They said yes, so that more people can see this, get the sauce and build and make money with it. If Firecrawl has been on your radar and and you just want a clear explanation of what it is and how you can use it as a founder, then this episode is for you. And if you've never heard of it, honestly, that's even better because what I'm about to show you is going to change how you think about what you can build with AI and where the next 12 months of building is going. Let's get into it. By the end of the episode, you're going to understand why AI is blind, why it needs hands and eyes, why Firecrawl is that, and why the people that understand how to use Firecrawl are going to be able to create SaaS, apps and software that are super, super valuable to people. I'm talking the most valuable software products are going to be using this data scraping tool at the backbone because it makes their AI 10 times smarter. But in order to understand this, we need to take a step back. The problem is AI is blind. If you listen to this channel, you know that, you know, the more context you give to a Claude, the more context you give to a ChatGPT, the better output you're going to get. So we know that AI models need web data. It needs top tier data to actually go and provide really good outputs. Why does this matter now? Well, it matters because if you think about the first era of AI, that was the chatbot era. ChatGPT just came out 2022. It answers question. It was cool, but pretty limited. Then we entered the copilot era. Cursor GitHub copilot. It was faster, but you still needed to drive. It was you, the human being that was doing it. We've now entered this AI agent era. AI is doing the work for you. Things like Claude code, it browses, it researches, it builds, but it still needs the data. And Firecrawl is how you're going to get that data. This is often called the computer use era. We now have AI agents that can see and control computers. In the past it was human beings, right? We bought mouses and keyboards and we had human beings actually going and clicking and doing things right? That's going to be the minority. As weird as it is to say that you have tools like perplexity computer, OpenAI had operator came out about a year ago. AI browses the web for you. GPT 5.4 beats humans at computer tasks. CLAUDE has its computer use API, screenshots and clicks. It's got full desktop control. Manus was the one who was one of the first to do that. You have browser use, which is an open source. All these computer uses, all these AI agents that are going and doing things well, what do they all need? Well, they need clean web data and that's Fire Crawl. And the reason I got interested in Fire Crawl is because I built ideabowser.com and ideabrowser.com is a place where you have trends and the best startup ideas on the planet. And I needed the data, I needed the trend data and we built on top of Firecrawl to actually go and get some of that data. Now we have the number one startup ideas and trends product on the planet. And it's all be cut in largely part that we're using tools like Fire Crawl to actually go and get that data. What most people don't get about this whole era that we're in is they think that AI is just chatbots that answer questions. They think web scrapers are illegal and shady. They think you need to code everything yourself. They, they think data is free and easy to get. And they think that web scraping is a thing for developers. But what it actually is happening is AI agents are doing work autonomously. Web data is critical AI infrastructure literally critical. One API call replaces thousands of lines and clean structured data is the new oil. By the end of this episode, I think you're going to agree by that. So the people that understand how important the clean data is and how important you can use the clean data and wrap it around a brain, an LLM and wrap that around a piece of software, those are the people that are going to be able to create the most valuable startups in the next 12 months. And I think that the people who understand that have a 12 month head start. And that's why I wanted to make this episode Traditional scraping versus new scrapers like Firecrawl. Let's just talk about that so we can understand what the difference is. The old way of scraping was you wrote a custom scraper per site, you managed proxies and browsers, you handled anti bot detection, you had to parse messy HTML manually. The scripts would break when site changes. This happened all the time. Basically it was a massive headache. Now you just do one API call, you get clean data back in seconds. It work on any site or I think like 99% or 98%, some high 90% of sites and the AI handles layout changes. So the way I think about my agent stack is that every builder, if you're listening to this, you're probably going to need five different layers. You're going to need an agent harness. So that's going to be something like a cloud code cursor Codex or Idea browser Pro. You're going to need something that basically, you know, is handling all the different agents in one place. Then you're going to need something like a search layer. So something that's going to go and search different things like Perplexity has a good MCP exa as well. Then you're going to need a web data layer and that's what we're talking about today in this episode. So you're going to use Firecrawl for scraping, browsing and extraction. Firecraw, basically the web data layer. Your agents need to see the Internet. You're going to need to be able to see the Internet to see the data in order to provide value back in the form of a startup and software. You're going to need an ops brain. So I recently did an episode, I encourage you to listen to it if you haven't already around Obsidian and Claude code. So you're going to, you know, I don't care if you use Notion, I don't care if you use Apple Notes, but you're going to need some brain for, you know, storing your meeting notes, storing your context and you can use something like Notion or Obsidian. And then you're going to have to have some outbound and audience stack as well, something like an Instantly and Apollo. And you know, if people are interested, I can spend more time and do a whole separate episode on some of these tools. But today we're going to Be talking about the Fire crawl, the Fire crawl web data layer. So what is it? What is Firecrawl? What is the clearest way to understand it? You put in a website, goes through the firecrawl API and you get back a clean markdown, a structured JSON, some screenshots, and you can feed that to any AI model. That's it. Simple as that. We don't need to overthink about it. Think it the way I think about it is Firecrawl has six superpowers you can scrape. So you can go and scrape one page to a clean markdown. So something like scrape one blog post from, you know, gregisenberg.com blog post. You can crawl an entire site automatically. So what do I mean by that? I mean you can go and say, give it CNN.com and it's going to go and crawl all of the different articles on CNN.com and you'll get that data back. You can map all URLs on a domain instantly. So that's super helpful. There's so much metadata and context into mapping and URLs. Maybe, you know, think about a URL, maybe there's a date in it, there's a title in it. Having that map is going to be helpful in some capacity to you, depending on what you're trying to do. You can go and search, you can use Google and you can put, you know, the full content in one call. Super, super valuable. It has an agent that you can describe data and it goes and finds it. You know, tell it I want the 50, you know, highest rated Cuban restaurants in South Florida. And it's going to give it back to you, going to give you the most clear data on it as well. And then it's got a browser. So AI controls a real browser. Super, super helpful. And it's three lines of code. You can screenshot this or I'll put it in the description for how to sign up. But basically it gives you a clean markdown of the entire website for any AI model in three lines of code. This is what excites me about it. So I believe that this is the AWS moment for web data. What do I mean that? In 2006, if you wanted to build a web app, what did you have to do? Well, you had to go out and buy servers, spend thousands of dollars buying servers. You had to go and manage racks and cables. Things would break all the time, all the time, all the time. And then one day, AWS said one API call and you can use our Servers in the cloud. Now, in 2026, if you want AI to use web data, what do you have to, what you have to do? You had to build scrapers, manage proxies, manage browsers, deal with security. Firecrawl says, one API call and we got you. This is a big deal because the companies that were built on top of aws, some of them became trillion dollar companies, some of them became billion dollar companies. And a lot became million dollar companies. Of course, a lot failed. But the point is people didn't have to deal with the headaches of servers. So they got to focus on building incredible product and those products were able to scale. Some of the, you know, the biggest companies of the last 10 years came because of AWS. So what gets built on the web data layer? I'm going to give you some ideas on some, you know, not billion dollar ideas, but some multimillion dollar, you know, 1 to 10 to $25 million a year, $50 million a year. Businesses that you can start by understanding, you know, what the web data layer is. And I think a lot of people are saying, sleeping on how big of a movement this is. So let's go into how, how it works. So, you know, here you are, right? You're the builder, you've got this AI agent and the AI agent is going to go talk to your brain. So you, you can use GPT, you can use cloud, you can use Gemini. You've got a nervous system, that's the way, at least I think about it, which is your MCP protocol. And now you have your eyes and hands. Your eyes and hands is Firecrawl. Now Firecrawl can go out to the Internet and it's going to get back clean data and you're going to use that data to wrap it around products and services you sell. So this is the big idea, right? You've got brain, you've got nervous system and you've now got eyes and hands. Of course you can go and do it yourself. Scraping, you can use, you know, Playwright or Selenium. You're going to just. The bottom line is it's just going to be a lot of work. I'm trying to do the simplest thing possible. So the reason I like FireCrawl is it's one API call. Proxies are built in, anti bot, built in. The AI tracks the data for you. It's just less headaches than actually going in doing it yourself. And you, and you've got the browser sandbox, which is really cool. So the browser sandbox it's a secure way to fill out. To have it Firecrawl fill out forms, click buttons and links, handle logins and auth navigate pagination. You can watch live as your AI browses, stay logged in across sessions. It's really crazy, right? So think about it. In a world where you can go and you have these hands and eyes out there on the Internet, what are the big ideas that you can build? And we're going to be talking about that soon. So the way the agent endpoint works is you type in a prompt, the firecrawl agent searches the web, it clicks through pages, it extracts data and it returns the JSON. So if you think about the AI infrastructure stack, I think about it like layers of the Internet. You've got applications, you've got like ChatGPT perplexity, a SaaS product, you've got AI agents, you've got protocols, you've got web data and you've got the Internet. So I believe that people are sleeping on the web data layer. And if you understand how to get great data out of tools like Firecrawl and Exa, you can build the picks and shovels of the AI gold rush. So let's just talk about what an agent prompt. If you prompt Firecall, what can you actually get back? So you can say find all of Y Combinator's Winter 24 dev tool companies and their founders and emails and what you get back is a structured list of 50 plus companies with names and contact info. You can say compare pricing tiers across Stripe and Square and PayPal and you get side by side pricing table with all features and costs. You can say get all running shoes from Nike under $150 with ratings and you get back full product catalog with specs and prices. And you could say find 50 AI research papers from 2024 with citations. You get the academic data set with authors and institution and institution. So super, super powerful stuff. Now let's talk about a few ideas that you can use to go and build build, you know, using Firecrawl. So price, the first idea is around price monitoring. So there's tools like Precinct and Visual Ping which you'll pay, you know, 200 to $1,000 a month. You basically get an e commerce focused price monitoring software. There's a self serve dashboard, it tracks any product. But why don't you just use Fire crawl? You can build this probably in a weekend and you can build a sneaker. Resale prices only. So auto alerts on stockx on goat on eBay you can charge $50 to run or sell for $500 a month. So basically pick a niche. You know, it could be sneakers, it could be, you know, collect, you know, different collectibles, it could be whatever and use that as, you know, I'm just using sneaker resale as an example. Right. It could be any, any niche that you understand better than someone else and setting alerts and you know, and then just charging people to people to use it. Number two, SEO. SEO Gap Finder. So Ahrefs and SEM Rush like you know, I think Semrush just sold for like $1.9 billion or something. Ahrefs probably does hundreds of millions a year in revenue. They charge hundreds of dollars a month. It requires SEO expertise. It's got these complex dashboards. It's pretty general purpose. What if you use Firecrawl to create, you know, SEO audits for dentist only. So Firecrawl reads competitor sites plus GMV listings. You know, you get a one click report so you rank for 12, they ranked for 47 and then you sell the reports for maybe it's $500 or $200 a month. So again, take a big idea that's already generating hundreds of millions of dollars. You recreate it very quickly with a very niche spoke focus. And again these are just example niches. But it could be, you know, Canadian dentist. If you even want to go more niche, think about indeed Zillow. Well found. These are massive horizontal platforms. They've got billions in fund funding. That's generic search for everyone. They use mostly do ad supported models. So what if you did a fire crawl version? Maybe you just do remote AI and ML jobs only. Firecrawl monitors 500 company career pages daily. So it's going and grabbing that data. The AI filters and ranks by fit score and then you can charge for Premium alerts for $29 a month. Indeed has 300 million listings. Nobody wants 300 million, they want 50 that matter. Again this is why Firecrawl's really good at getting the top stuff AI research reports. So yes, there's big companies like Consensys or Tavily, but these are general purpose research, academic or broad. The user does the prompting and there's no vertical expertise. What if you did like a niche crypto token due diligence reports. So you have Firecrawl read white papers and Twitter and other places it auto generates a risk score and summary and then you can sell that to VCs, private equity or different funds for you know, a thousand to five hundred. $5,000 a month. A VC will pay $5,000 for a report. That saves them from a bad 500k bet all day long. So again, picking a niche, getting the best possible data. A couple more ideas, an agent in the box. So you know, you have Harvey AI, it's got, you know, now hundreds, I think of millions in funding. It's got an enterprise sales cycle, horizontal agent platform. It takes months to cut, customize. What if you did like a real estate comp report agent? So you use Firecrawl to pull listings, tax records and permits and the agent generates comp reports in like 30 seconds and then you sell that to retailers for $300 a month. So don't raise any money. You go and do this. $300 a month. You know, could, could work. Review intelligence. So yes, there's, there's companies like Brand24 and App Follow, they charge few hundred bucks a month. They basically monitor social and reviews broadly their dashboards for marketing teams, generic sentiment analysis. But what if you did an Amazon FBA seller review tracker? So Firecrawl monitors competitor Review Daily. The AI spots trends, right? Complaints about battery life up to 40% and you sell that to Amazon sellers for 99amonth. And something like this could also, by the way, get acquired by like a Shopify or an Amazon. Amazon sellers will gladly pay $99 a month to find product gaps before, before competitors do. So these are just a few ideas to get your creative juices flowing around how to use Fire Crawl to scrape ideas. Scrape ideas. Go niche and you can compete on price, you can compete on nicheness. I don't know if that's a word, but we're going with it and just create, like I said, clean structured data using AI to actually build and vibe code a lot of these products and start, you know, start selling them to, to these niches that are looking for that, they're looking for this stuff and they, they, they want, you know, the truth is the vertical, the reason why, like vertical software is such a big business. Why is Constellation Software, you know, almost a $75 billion company or whatever, they have hundreds of vertical software companies because people like buying very specific products. So there's always going to be room for these horizontal ideas. There's always going to be room for the SEM rushes and the Indeeds and the linkedins and stuff like that. But if you can carve out a little niche that could do 1 million a year to 10 million to 20 million to 30 million, there's opportunity there, you know, incumbents are charging hundreds of dollars a month for generic tools. Your, your version charges, you know, 20, 50, $70 for a tool that does one thing perfectly for one customer. So another idea would be to build a legion lead gen business. So a client gives you 50 company names. What if you grabbed a firecrawl agent that found founders and emails, it returns the structured JSON with all data, you deliver the enriched CSV and you just charge, I don't know, $500, $200, $100 per batch. Your cost is like $2 in Firecrawl credits. Firecrawl actually I have here like there's a bunch of free, you know, there's a free tier. The agent run gives you five free per day and then you know, to scrape cost one credit accra cost one. But the point is if you can figure out a way to get 95% margin, 98% margin, 99% margin, you're happy clients happy because hopefully they're closing on some of these deals. So there's something here around using some of the data, charging per output and creating high margin businesses. This is the framework for how I would think about how you can build and make money with firecrawl this week. So the first step is going to be picking a niche. So what data do people in this industry actually pay for? The second step is going to be building the scraper. So use firecrawl agent, maybe a simple Python script and NN flow or just use Claude code to go and build that for you. Step three is going to package it. So CSV or dashboard or Slack Alert or API. And step four is going to be about selling the output, right? Not just the tool. You're going to be selling the data. So you can charge maybe $500 to $5,000 per month per client and then you're going to automate it. How do you schedule it and let it run while you sleep compounding clients and that sort of thing. So I think that a lot of people are going to be starting to do this. They're going to be picking niches, they're going to be building scrapers, they're going to be packaging it, they're going to be selling the output and they're going to automate it. It's a flywheel that I think is just, just getting started. So just a few more ideas for you. You can do something like real estate pricing data, you can do SaaS, competitor monitoring, you can do job aggregation, you can do patent legal filings, you can do influencer contact databases, you can do government contact alerts, you can do E commerce price tracking tracking. You can do academic research data sets. And then you can, and this is what I suggest you do, is just do more niche versions of this, right? So real estate pricing, go more niche. SAS competitor monitoring, go more niche. This is just ideas to get your creative juices flowing. So how I actually heard, I want to end with this, but how I actually heard about Fire Crawl was, you know, a year ago I tweeted this, actually I saw that they had post posted a job saying they were hiring a Fire Crawl example creator, but they only wanted to hire an AI agent. So said please only apply if you're an AI agent. We're seeking an AI agent capable of autonomously researching trending tech and models and then using the information to create, test and refine high quality example applications. These sample apps will live in our example repository, showcasing the full potential of Firecrawl in real world scenarios. Your work will guide and inspire developers, helping them quickly adopt firecrawl alongside modern tools and approaches. So if Firecrawl is hiring AI agents as employees, it got me thinking that this is probably where the world is is going. So for example, hiring a content creator agent writes blog posts autonomously, watches metrics and improves. Maybe that's a $5,000 per month salary. A customer support agent handles tickets in 2 minutes, knows when to escalate, maybe that's a $5,000 per month salary. A junior developer agent, triage GitHub issues, writes docs and code, that's a $5,000.00 per month salary. So that's a million dollar total budget, 50 applications in the first week. So my startup idea was how do you build AI agents that companies like Firecrawl want to hire? Yes, it looks super weird right now that Firecrawl is hiring a AI agent and I, you know, feels like a little bit of a joke. But I think that it got me thinking that using tools like firecrawl and building products and agents around it, I could see a world where this becomes more and more popular. Right? And I think that there's an opportunity to think about it as from a framework perspective is how can you use tools like firecrawl to build AI agents and build products that companies would want to hire. So I just thought, by the way, I just thought that was just wanted to end with that. So overall, this is my breakdown for why I think there's a tremendous opportunity in the web data layer and using Fire Crawl for scraping why? I think there's a lot of ideas around it. And yeah, hope this got your creative juices flowing. It's certainly something that I'm exploring in real time, building products with, with Firecrawl because it's valuable. It's. It's super valuable in, in getting the right data and, and it's just working. So hope this has been helpful. Please comment if like, what you want to see next for me, what do you want me to teach you? I'm just sharing things that I'm learning in real time and hopeful that it it's helping you along your journey. So thank you so much for if you made it to the end. Thank you so much for being here. I'm rooting for you for whatever it is you're building, and I can't wait to see you on the next episode.