5 Tips and Misconceptions about Finetuning GPT-3

Summary

---

Artificial Intelligence Masterclass

Episode: 5 Tips and Misconceptions about Finetuning GPT-3

Host: David Shapiro
Date: December 31, 2024

---

Overview

In this episode, David Shapiro demystifies the process of finetuning GPT-3, sharing five core tips and addressing common misconceptions. He emphasizes the power and versatility of GPT-3’s base models and advocates for pragmatic, thoughtful adoption of finetuning—only when absolutely necessary. Drawing from hands-on experience and community insights, Shapiro’s tone is conversational, occasionally humorous, and always focused on practical utility. He advocates for multidisciplinary teamwork and highlights the importance of language and creativity in leveraging large language models.

---

Key Discussion Points and Insights

1. Start with “Plain Vanilla” GPT-3 (Prompt Engineering First)

• Timestamp: [01:31 – 05:00]
• Shapiro advises against immediately jumping into finetuning. Instead, he urges listeners to first master basic prompt engineering.
  • “GPT3 is not like any other tool you’ve ever used, NLP or otherwise. It’s not like an SVM, it’s not like a regression model. It’s not even like other neural networks.” [01:54]
  • Many users underestimate GPT-3’s out-of-the-box capabilities and overestimate the need for specialized data or finetuning.
  • Example: Instead of writing code to scrape dates from text, just ask GPT-3 directly for the dates.
  • Encourages adopting a mindset shift from traditional machine learning approaches: “Some people just, they have their, their old school data science mindset, like, oh, I need…data set. I need to, you know, come up with rules. And I’m like, just ask it.” [02:13]
• Highlight:
  • Subtitle: Team Composition Matters
  • Shapiro recommends building teams that include people with strong language backgrounds (e.g., English majors, philosophers), not just computer scientists.
    • “If you want to have a dynamite team using large language models, make sure you’ve got someone who understands language on your team. Maybe hire a librarian, an English major, a philosopher.” [04:21]
  • Observes a notable split: those with humanities backgrounds “get it” faster because they think in language and narrative.

---

2. Building Finetuning Datasets Is 100x More Effort Than Prompt Engineering

• Timestamp: [05:00 – 06:30]
• Creating datasets for finetuning is time-consuming and labor-intensive relative to prompt engineering.
  • “Building fine tuning data sets is 100 times more effort than prompt engineering. For that reason alone, start with plain vanilla GPT3. It’ll carry you way farther than you think it will.” [05:10]
• Shapiro recommends: Only consider finetuning after extensive experience with prompt engineering—and even then, question if it’s truly necessary.

---

3. Use Natural Language Separators for Demarcation

• Timestamp: [06:30 – 08:56]
• When building finetuning datasets, use clear, meaningful natural language to indicate where the prompt ends and the response begins.
  • “In the OpenAI documentation, they just use like hashtag, hashtag, hashtag. And while that can work, it’s semantically meaningless. So what I usually do is I will add like just a couple words ... that helps teach GPT3 1 what its task actually is.” [06:50]
• Benefits:
  • Explicit, human-readable instructions help GPT-3 more easily understand and adapt to multiple tasks within the same dataset.
  • Allows for dynamic task switching at inference time without needing multiple finetuned models: “By using a natural language separator… you can actually switch tasks without having to switch between different fine tuned models.” [08:28]

---

4. Leverage GPT-3 Itself to Generate Synthetic Data

• Timestamp: [08:56 – 13:00]
• Utilize GPT-3 to synthesize example data for finetuning (e.g., simulate conversations).
  • “It takes me an hour flat to make a new fine tuning data set because it’s just a couple of scripts to take in some raw material and massage those into prompts ... and then you’re ready to go.” [11:37]
• Clarifies common misconception: massive datasets are unnecessary.
  • “There was one person on the forum who thought that he needed 200,000 samples to fine tune GPT3 for a good chatbot. I said no, you need 200, not 200,000.” [12:14]
  • In real use, “It’s like 18 cents to fine tune with 200 samples... if you use DaVinci it’s $1.80.” [13:53]
  • Most tasks—chatbots, Q&A—work with small (e.g., 200 samples) datasets if well designed and cleaned.
• Publicly shares his datasets for community testing and improvement.

---

5. Finetuning Increases Consistency but Reduces Creativity

• Timestamp: [13:53 – End (~15:00)]
• Finetuning models for a particular style or perspective increases output consistency, but at the cost of creative flexibility.
  • “Plain vanilla GPT3 is really creative. It’s more creative than any 10 humans. But fine tuning… it’ll kind of lose those other perspectives.” [14:32]
  • For creative and open-ended tasks, prompt engineering is preferred.
  • For tasks requiring rigid consistency or fixed formats, finetuning is the best approach.

---

Notable Quotes and Memorable Moments

• On team composition:
  “So, hire humanities.” [04:37]

• On prompt engineering vs. traditional ML:
  “You don’t have to worry about the math…honestly, [CS algorithmic thinking] won’t help you with using GPT3.” [03:48]

• On data requirements:
  “You need 200, not 200,000.” [12:20]

• On model behavior:
  “Fine tuning tends to increase consistency at the cost of creativity…so just keep that in mind.” [14:41]

---

Timestamps for Important Segments

• [01:31] — Episode intro and core premise
• [02:00] — Why prompt engineering comes first
• [04:20] — Humanities vs. CS mindsets in GPT-3 usage
• [05:10] — The true cost and effort of dataset creation
• [06:50] — Semantic demarcation in dataset design
• [08:30] — Task separation during inference
• [11:37] — Using GPT-3 to generate datasets and share results
• [12:14] — Real dataset size and cost breakdown
• [14:32] — Trade-offs: consistency vs. creativity

---

Summary Takeaways

• Start with the default GPT-3 and leverage prompt engineering to its fullest—only consider finetuning after reaching its limits.
• Prioritize team members who understand language; humanities and philosophy backgrounds add unique strengths.
• Use meaningful language separators in datasets to help models distinguish tasks and instructions.
• Exploit GPT-3’s generative powers to synthesize datasets. Small, well-crafted datasets (circa 200 samples) often suffice.
• Recognize the trade-off: finetuning enhances reliability and consistency but narrows creative response diversity.

---

For practical examples and public datasets referenced by David Shapiro, check the episode description for resources and repository links.

Transcript

[A]

This episode is brought to you by State Farm. Listening to this podcast Smart Move Being financially savvy Smart Move Another smart move Having State Farm help you create a competitive price when you choose to bundle home and auto bundling. Just another way to save with a personal price plan. Like a good neighbor, State Farm is there. Prices are based on rating plans that vary by state. Coverage options are selected by the customer. Availability, amount of discounts and savings and eligibility vary by state.

[B]

Running a business comes with a lot of what ifs, but luckily there's a simple answer to Shopify. It's the commerce platform behind millions of businesses including Thrive Cosmetics and Momofuku, and it'll help you with everything you need. From website design and marketing to boosting sales and expanding operations. Shopify can get the job done and make your dream a reality. Turn those what ifs into Sign up for your $1 per month [e-mail] specialoffer with savings over $390 this shopping season, VRBO helps you swap gift wrap time for quality time with those you love most. From snow on the roof to sand between your toes, we have all the vacation rental options covered. Go to VRBO now and book a last minute week long stay. Save over $390 this holiday season and book your next vacation rental home on VRBO. Average savings $396 select homes only.

[C]

Good morning everyone. David Shapiro here. I wanted to make a video about fine tuning with GPT3. At present, my most popular video is about fine tuning GPT3 for a specific task. But I wrote a post on the OpenAI community about just some tips and observations that I had about fine tuning. Both from my own experiments but from helping other people. So for instance, I've been approached by lots of people, startups and students wanting to get help with fine tuning and you know, so hey, I can just help everyone. So let me give you five tips and misconceptions about fine tuning GPT3. My first tip is start with normal GPT3 and prompt engineering. Get good with GPT3 before you jump into fine tuning. GPT3 is not like any other tool you've ever used, NLP or otherwise. It's not like an svm, it's not like a regression model. It's not even like other neural networks. For instance, a lot of people jump in thinking, assuming that they need fine tuning when they haven't even used GPT3 before. And I say jump in, give it a test drive. It's way more powerful than you think it is. And so some people just, they have their, their old school data science mindset, like, oh, I need, I need, you know, data set. I need to, you know, come up with rules. And I'm like, just ask it. There was one case where a recent case where someone was trying to scrape dates from unstructured text and I'm like, just ask for the dates. You don't need to fine tune anything. Pardon me, I'm a little bit parched, so be drinking my tea while I talk. It's still too hot. Ow. Yeah, so jump into GPT3. Plain vanilla. GPT3 is way more powerful than you think it is. It knows a lot more. It's read, you know, many gigabytes or terabytes of text. I don't even know how much data it was trained on. I've seen different numbers. It depends on who you ask. So it's not like anything else. When you, when you fine tune GPT3, that's actually transfer learning, which means you're getting the benefit of most of the learning that it's already got. So it doesn't work the way that you think it does unless you've read the papers and played with it. Let's see. Oh, another thing this is one of, this is a really big misconception is prompt engineering means you have to be really good at language. And it's really interesting. My partner and I, my partner is getting her master's degree in information science. So she bridges the gap between humanities and computer science. So if you work with philosophers, with writers, with digital humanities folks, they get it really fast. Why? Because they understand language, they understand rhetoric, they understand using the written word to communicate ideas. However, you put GPT3 in front of a die hard math first computer science major, and they often can't see the language for the math. Another way of saying that is they see the algorithm first without seeing the implication of the language. And it's really interesting to see that some people put the language first and some people put the math first. So with GPT3, you don't have to worry about the math. So if you came up from, you know, hardcore, you know, computer science, algorithmic thinking, that honestly won't help you with using GPT3. And so some people ask me like, okay, well what, what, what, what? Sorry, stuttering. If you've watched some of my other videos, you know that I stutter sometimes. I don't even identify as a stutterer. It just happens sometimes everyone stutters. Anyways, sorry, but I lost my train of thought. Oh, yeah, team composition. If you want to have a dynamite team using large language models, make sure that you've got someone who understands language on your team. Maybe hire a librarian, an English major, a philosopher. I was working with one startup for a while where they were, they all had, they all had humanities training, they understood philosophy, they understood all sorts of other psychology. Psychologists get it really well too. And they all, they understood it. I showed them what GPT3 could do and they're like, this is amazing. And I see what you mean, like, this is capable of philosophical reasoning. And of course, you know, you show a philosopher that a machine is capable of philosophical reasoning and they're blown away. And you show a computer scientist and they're like, yeah, whatever. It's just this really weird dichotomy. So hire humanities. So that's, that's all, that's all within. Tip number one, start with GPT3, plain vanilla. Number two. Tip number two, building fine tuning data sets is 100 times more effort than prompt engineering. For that reason alone, start with plain vanilla GPT3. It'll carry you way farther than you think it will. If you, if you take GPT3 to its limits. If you say, okay, I've worked with this tool for months and I can't get it to do what I need it to do, then maybe it's time for fine tuning. But even then, maybe not. I'll get into that in a second. But yeah, like building fine tuning data sets, it's super, super hard. Let's see now. Okay, let's assume that you've done your homework. You've decided, yes, I do need fine tuning. My first tip is use natural language separators or demarcators to identify where the prompt begins and the completion prompt ends. And the completion begins. Sorry, in the OpenAI documentation, they just use like hashtag, hashtag, hashtag. And while that can work, it's semantically meaningless. So what I usually do is I will add like just a couple words, like if in my, in my question generating fine tuning data set, I have like, here's, you know, a block of text and then I say like ask questions and then with a, with a colon. And GPT3 really learns. Okay, that's where the task begins. And so just like, you know, one to one to five words giving instructions right at the end of the prompt, that helps teach GPT3 1 what its task actually is without having to infer what the task is, because then you can be very explicit about what its task is, but also that it's more semantically meaningful. And the reason that semantic meaning is important is because if you, if you fine tune a data set to do multiple tasks, it needs to differentiate between those tasks because like let's say, let's say you're trained, you're fine tuning chatbot, right? And you want to, you want to train this chatbot to ask questions or provide facts or, you know, answer questions. So ask or answer questions, provide facts or just whatever you need to be able to differentiate what you want your fine tuned model to do. And by, by using a natural language separator, that means that at inference time, when it's in production, you can actually switch tasks without having to switch between different fine tuned models. And that will save you a lot of time because let's say you make, you know, one fine tuned data set that's got 200 samples of asking questions and 200 samples of answering questions. Now you've got one fine tuned model to use and you're ready to go. You're off to the races. So natural language separators, that's a really big one. That's number three. Number four is use GPT3 itself to make synthetic data sets. I use this quite extensively. GPT3 is able to simulate any kind of conversation. And so you can, you can either, you know, scrape web data, which is legal. There was another, what was it, 9th District US Circuit Court just said, like yes, using scrapes web data is perfectly legal as long as it's publicly accessible. It's not hacking, it's not theft. If someone puts public data on the Internet, you're free to use it. So I scrape Reddit, public Reddit experience.

[B]

A membership that backs your business journey with American Express Business Platinum. When you pay with membership rewards points for all or part of an eligible flight booked with a qualifying airline. Through Amex Travel, you can get 35% of those points back up to 1 million points back per calendar year. American Express Business Platinum there's nothing like it. Terms apply. Learn [e-mail] Business Platinum.

[D]

Shopping is hard, right? But I found a better way. Stitch Fix Online Personal styling makes it easy. I just give my stylist my size, style and budget preferences. I order boxes when I want and how I want. No subscription required. And he sends just for me pieces plus outfit recommendations and styling tips. I keep what works and send back the rest. It's so easy. Make style easy. Get started [e-mail] Spotify that's stitchfix.com Spotify.

[C]

Your planet is now marked for death.

[D]

Model Studios the Fantastic Four First Steps is now streaming on Disney.

[A]

We will protect you as a family.

[D]

Light them up, Johnny Marvel's first family is certified fresh on Rotten Tomatoes.

[C]

That is fantastic.

[D]

And critics say it's one of the best superhero movies of all time. Marvel Studios the Fantastic Four first steps now streaming on Disney. Rated PG 13.

[C]

What time is it, Ben? It's clover time as a way of getting kind of some raw material. But then what you can do is is. And oh, check the description. I've got a public repo with all of my publicly available fine tuning data sets so you can go see what I'm talking about. But by synthesizing a data set one, it's way easier. You write a few really good prompts to say to generate the kind of output that you want, the kind of input and output that you want, and then you're off to the races. It takes me an hour flat to make a new fine tuning data set because it's just a couple of scripts to take in some raw material and massage those into prompts which the latest instruct models instruct series of GPT3 models are really great at generating synthetic output or synthetic data sets and then you're ready to go. So one thing that I need to say there though is going Back to point 1, fine tuning GPT3 is not like conventional ML. There was one person on the forum who thought that he needed 200,000 samples to fine tune GPT3 for a good chatbot. I said no, you need 200, not 200,000. 200. And I did the numbers and I was like, this is like what a dollar? Like.

[A]

When the holidays start to feel a bit repetitive, reach for a Sprite Winter Spiced Cranberry and put your twist on tradition. A bold cranberry and winter spice flavor. Fue Sprite Winter Spice Cranberry is a refreshing way to shake things up. This sip in season and only for a limited time. Sprite obey your thirst.

[B]

Ford BlueCruise Hands Free highway driving takes the work out of being behind the wheel, allowing you to relax and reconnect while also staying in control. Enjoy the drive in BlueCruise enabled vehicles like the F150 Explorer and Mustang Mach E available feature on equipped vehicles terms apply. Does not replace safe driving. See Ford.com BlueCruise for more details.

[C]

Depending on the model you use, it's like 18 cents to fine tune with 200 samples. I think if you use DaVinci it's $1.80, right? So Curie can do most tasks. So you fine tune a Curie model. It'll be faster and cheaper. Cost 18 cents to fine tune with 200 samples and that was like a high water mark. That's if you use a thousand tokens per training sample, which most of them are going to be a tenth of that because because of how aligned the models are now. So you know, it takes way less data than you think to get started. And I proved my point. I gave that's that that post is actually why I publicly posted my fine tuning data set. I said, go grab my Chatbot fine tuning data and run it yourself if you don't believe me. And he went and he did. He said, this is impressive. He said, you know, there's still some problems, but it performs way better than I thought it would on just 200 samples. So yes, and I used GPT3 to make the fine tuning set and then I went and manually cleaned it up by hand. That's way faster than doing a whole data set by hand. Number five. Fine tuning tends to increase consistency at the cost of creativity. Sometimes that's what you need because GPT3 on its own can be really creative, right? GPT3 is able to adopt any mental framework. You can tell it like pretend to be Yosemite Sam. That was Looney Tunes back in the day. That's dating myself. It can pretend to be Bugs Bunny, Spider Man. You can talk to it about philosophy, you can talk to it about racism. It can take the perspective of different religions, right? So plain vanilla GPT3 is really creative. It's more creative than any 10 humans. But fine tuning, because you're going to show it a bunch of data set samples from a particular perspective, it'll kind of lose those other perspectives. So just keep that in mind. If you're trying to do a creative task, prompt engineering is going to be better than fine tuning. But if you, if you really do need consistency, and a lot of tasks do, I'm not saying that this is a bad thing, just something to keep in mind is if you want, if you want your GPT3 app to be able to provide really creative answers or solutions, fine tuning is going to reduce that ability. Whereas if you need something to be very consistent, like if there's a particular format to follow, that's where fine tuning really shines. So yeah, five tips and misconceptions about GPT3 and fine tuning. I hope you found this helpful. Thanks for watching. Oh, also like and subscribe.