Artificial Intelligence Masterclass

Episode: 5 Tips and Misconceptions about Finetuning GPT-3

Host: David Shapiro
Date: December 31, 2024

Overview

In this episode, David Shapiro demystifies the process of finetuning GPT-3, sharing five core tips and addressing common misconceptions. He emphasizes the power and versatility of GPT-3’s base models and advocates for pragmatic, thoughtful adoption of finetuning—only when absolutely necessary. Drawing from hands-on experience and community insights, Shapiro’s tone is conversational, occasionally humorous, and always focused on practical utility. He advocates for multidisciplinary teamwork and highlights the importance of language and creativity in leveraging large language models.

Key Discussion Points and Insights

1. Start with “Plain Vanilla” GPT-3 (Prompt Engineering First)

Timestamp: [01:31 – 05:00]
Shapiro advises against immediately jumping into finetuning. Instead, he urges listeners to first master basic prompt engineering.
- “GPT3 is not like any other tool you’ve ever used, NLP or otherwise. It’s not like an SVM, it’s not like a regression model. It’s not even like other neural networks.” [01:54]
- Many users underestimate GPT-3’s out-of-the-box capabilities and overestimate the need for specialized data or finetuning.
- Example: Instead of writing code to scrape dates from text, just ask GPT-3 directly for the dates.
- Encourages adopting a mindset shift from traditional machine learning approaches: “Some people just, they have their, their old school data science mindset, like, oh, I need…data set. I need to, you know, come up with rules. And I’m like, just ask it.” [02:13]
Highlight:
- Subtitle: Team Composition Matters
- Shapiro recommends building teams that include people with strong language backgrounds (e.g., English majors, philosophers), not just computer scientists.
  - “If you want to have a dynamite team using large language models, make sure you’ve got someone who understands language on your team. Maybe hire a librarian, an English major, a philosopher.” [04:21]
- Observes a notable split: those with humanities backgrounds “get it” faster because they think in language and narrative.

2. Building Finetuning Datasets Is 100x More Effort Than Prompt Engineering

Timestamp: [05:00 – 06:30]
Creating datasets for finetuning is time-consuming and labor-intensive relative to prompt engineering.
- “Building fine tuning data sets is 100 times more effort than prompt engineering. For that reason alone, start with plain vanilla GPT3. It’ll carry you way farther than you think it will.” [05:10]
Shapiro recommends: Only consider finetuning after extensive experience with prompt engineering—and even then, question if it’s truly necessary.

3. Use Natural Language Separators for Demarcation

Timestamp: [06:30 – 08:56]
When building finetuning datasets, use clear, meaningful natural language to indicate where the prompt ends and the response begins.
- “In the OpenAI documentation, they just use like hashtag, hashtag, hashtag. And while that can work, it’s semantically meaningless. So what I usually do is I will add like just a couple words ... that helps teach GPT3 1 what its task actually is.” [06:50]
Benefits:
- Explicit, human-readable instructions help GPT-3 more easily understand and adapt to multiple tasks within the same dataset.
- Allows for dynamic task switching at inference time without needing multiple finetuned models: “By using a natural language separator… you can actually switch tasks without having to switch between different fine tuned models.” [08:28]

4. Leverage GPT-3 Itself to Generate Synthetic Data

Timestamp: [08:56 – 13:00]
Utilize GPT-3 to synthesize example data for finetuning (e.g., simulate conversations).
- “It takes me an hour flat to make a new fine tuning data set because it’s just a couple of scripts to take in some raw material and massage those into prompts ... and then you’re ready to go.” [11:37]
Clarifies common misconception: massive datasets are unnecessary.
- “There was one person on the forum who thought that he needed 200,000 samples to fine tune GPT3 for a good chatbot. I said no, you need 200, not 200,000.” [12:14]
- In real use, “It’s like 18 cents to fine tune with 200 samples... if you use DaVinci it’s $1.80.” [13:53]
- Most tasks—chatbots, Q&A—work with small (e.g., 200 samples) datasets if well designed and cleaned.
Publicly shares his datasets for community testing and improvement.

5. Finetuning Increases Consistency but Reduces Creativity

Timestamp: [13:53 – End (~15:00)]
Finetuning models for a particular style or perspective increases output consistency, but at the cost of creative flexibility.
- “Plain vanilla GPT3 is really creative. It’s more creative than any 10 humans. But fine tuning… it’ll kind of lose those other perspectives.” [14:32]
- For creative and open-ended tasks, prompt engineering is preferred.
- For tasks requiring rigid consistency or fixed formats, finetuning is the best approach.

Notable Quotes and Memorable Moments

On team composition:
“So, hire humanities.” [04:37]
On prompt engineering vs. traditional ML:
“You don’t have to worry about the math…honestly, [CS algorithmic thinking] won’t help you with using GPT3.” [03:48]
On data requirements:
“You need 200, not 200,000.” [12:20]
On model behavior:
“Fine tuning tends to increase consistency at the cost of creativity…so just keep that in mind.” [14:41]

Timestamps for Important Segments

[01:31] — Episode intro and core premise
[02:00] — Why prompt engineering comes first
[04:20] — Humanities vs. CS mindsets in GPT-3 usage
[05:10] — The true cost and effort of dataset creation
[06:50] — Semantic demarcation in dataset design
[08:30] — Task separation during inference
[11:37] — Using GPT-3 to generate datasets and share results
[12:14] — Real dataset size and cost breakdown
[14:32] — Trade-offs: consistency vs. creativity

Summary Takeaways

Start with the default GPT-3 and leverage prompt engineering to its fullest—only consider finetuning after reaching its limits.
Prioritize team members who understand language; humanities and philosophy backgrounds add unique strengths.
Use meaningful language separators in datasets to help models distinguish tasks and instructions.
Exploit GPT-3’s generative powers to synthesize datasets. Small, well-crafted datasets (circa 200 samples) often suffice.
Recognize the trade-off: finetuning enhances reliability and consistency but narrows creative response diversity.

For practical examples and public datasets referenced by David Shapiro, check the episode description for resources and repository links.

Transcript

A (0:00)

This episode is brought to you by State Farm. Listening to this podcast Smart Move Being financially savvy Smart Move Another smart move Having State Farm help you create a competitive price when you choose to bundle home and auto bundling. Just another way to save with a personal price plan. Like a good neighbor, State Farm is there. Prices are based on rating plans that vary by state. Coverage options are selected by the customer. Availability, amount of discounts and savings and eligibility vary by state.

B (0:30)

Running a business comes with a lot of what ifs, but luckily there's a simple answer to Shopify. It's the commerce platform behind millions of businesses including Thrive Cosmetics and Momofuku, and it'll help you with everything you need. From website design and marketing to boosting sales and expanding operations. Shopify can get the job done and make your dream a reality. Turn those what ifs into Sign up for your $1 per month trial@shopify.com specialoffer with savings over $390 this shopping season, VRBO helps you swap gift wrap time for quality time with those you love most. From snow on the roof to sand between your toes, we have all the vacation rental options covered. Go to VRBO now and book a last minute week long stay. Save over $390 this holiday season and book your next vacation rental home on VRBO. Average savings $396 select homes only.

C (1:31)

Good morning everyone. David Shapiro here. I wanted to make a video about fine tuning with GPT3. At present, my most popular video is about fine tuning GPT3 for a specific task. But I wrote a post on the OpenAI community about just some tips and observations that I had about fine tuning. Both from my own experiments but from helping other people. So for instance, I've been approached by lots of people, startups and students wanting to get help with fine tuning and you know, so hey, I can just help everyone. So let me give you five tips and misconceptions about fine tuning GPT3. My first tip is start with normal GPT3 and prompt engineering. Get good with GPT3 before you jump into fine tuning. GPT3 is not like any other tool you've ever used, NLP or otherwise. It's not like an svm, it's not like a regression model. It's not even like other neural networks. For instance, a lot of people jump in thinking, assuming that they need fine tuning when they haven't even used GPT3 before. And I say jump in, give it a test drive. It's way more powerful than you think it is. And so some people just, they have their, their old school data science mindset, like, oh, I need, I need, you know, data set. I need to, you know, come up with rules. And I'm like, just ask it. There was one case where a recent case where someone was trying to scrape dates from unstructured text and I'm like, just ask for the dates. You don't need to fine tune anything. Pardon me, I'm a little bit parched, so be drinking my tea while I talk. It's still too hot. Ow. Yeah, so jump into GPT3. Plain vanilla. GPT3 is way more powerful than you think it is. It knows a lot more. It's read, you know, many gigabytes or terabytes of text. I don't even know how much data it was trained on. I've seen different numbers. It depends on who you ask. So it's not like anything else. When you, when you fine tune GPT3, that's actually transfer learning, which means you're getting the benefit of most of the learning that it's already got. So it doesn't work the way that you think it does unless you've read the papers and played with it. Let's see. Oh, another thing this is one of, this is a really big misconception is prompt engineering means you have to be really good at language. And it's really interesting. My partner and I, my partner is getting her master's degree in information science. So she bridges the gap between humanities and computer science. So if you work with philosophers, with writers, with digital humanities folks, they get it really fast. Why? Because they understand language, they understand rhetoric, they understand using the written word to communicate ideas. However, you put GPT3 in front of a die hard math first computer science major, and they often can't see the language for the math. Another way of saying that is they see the algorithm first without seeing the implication of the language. And it's really interesting to see that some people put the language first and some people put the math first. So with GPT3, you don't have to worry about the math. So if you came up from, you know, hardcore, you know, computer science, algorithmic thinking, that honestly won't help you with using GPT3. And so some people ask me like, okay, well what, what, what, what? Sorry, stuttering. If you've watched some of my other videos, you know that I stutter sometimes. I don't even identify as a stutterer. It just happens sometimes everyone stutters. Anyways, sorry, but I lost my train of thought. Oh, yeah, team composition. If you want to have a dynamite team using large language models, make sure that you've got someone who understands language on your team. Maybe hire a librarian, an English major, a philosopher. I was working with one startup for a while where they were, they all had, they all had humanities training, they understood philosophy, they understood all sorts of other psychology. Psychologists get it really well too. And they all, they understood it. I showed them what GPT3 could do and they're like, this is amazing. And I see what you mean, like, this is capable of philosophical reasoning. And of course, you know, you show a philosopher that a machine is capable of philosophical reasoning and they're blown away. And you show a computer scientist and they're like, yeah, whatever. It's just this really weird dichotomy. So hire humanities. So that's, that's all, that's all within. Tip number one, start with GPT3, plain vanilla. Number two. Tip number two, building fine tuning data sets is 100 times more effort than prompt engineering. For that reason alone, start with plain vanilla GPT3. It'll carry you way farther than you think it will. If you, if you take GPT3 to its limits. If you say, okay, I've worked with this tool for months and I can't get it to do what I need it to do, then maybe it's time for fine tuning. But even then, maybe not. I'll get into that in a second. But yeah, like building fine tuning data sets, it's super, super hard. Let's see now. Okay, let's assume that you've done your homework. You've decided, yes, I do need fine tuning. My first tip is use natural language separators or demarcators to identify where the prompt begins and the completion prompt ends. And the completion begins. Sorry, in the OpenAI documentation, they just use like hashtag, hashtag, hashtag. And while that can work, it's semantically meaningless. So what I usually do is I will add like just a couple words, like if in my, in my question generating fine tuning data set, I have like, here's, you know, a block of text and then I say like ask questions and then with a, with a colon. And GPT3 really learns. Okay, that's where the task begins. And so just like, you know, one to one to five words giving instructions right at the end of the prompt, that helps teach GPT3 1 what its task actually is without having to infer what the task is, because then you can be very explicit about what its task is, but also that it's more semantically meaningful. And the reason that semantic meaning is important is because if you, if you fine tune a data set to do multiple tasks, it needs to differentiate between those tasks because like let's say, let's say you're trained, you're fine tuning chatbot, right? And you want to, you want to train this chatbot to ask questions or provide facts or, you know, answer questions. So ask or answer questions, provide facts or just whatever you need to be able to differentiate what you want your fine tuned model to do. And by, by using a natural language separator, that means that at inference time, when it's in production, you can actually switch tasks without having to switch between different fine tuned models. And that will save you a lot of time because let's say you make, you know, one fine tuned data set that's got 200 samples of asking questions and 200 samples of answering questions. Now you've got one fine tuned model to use and you're ready to go. You're off to the races. So natural language separators, that's a really big one. That's number three. Number four is use GPT3 itself to make synthetic data sets. I use this quite extensively. GPT3 is able to simulate any kind of conversation. And so you can, you can either, you know, scrape web data, which is legal. There was another, what was it, 9th District US Circuit Court just said, like yes, using scrapes web data is perfectly legal as long as it's publicly accessible. It's not hacking, it's not theft. If someone puts public data on the Internet, you're free to use it. So I scrape Reddit, public Reddit experience.