← Reshaping Workflows with Dell Pro Precision and NVIDIA RTX PRO GPUs
Reshaping Workflows with Dell Pro Precision and NVIDIA RTX PRO GPUs
The AI News Revolution with The Neuron
00:46:39
The Neuron’s founders join Logan Lawler to reveal how AI news, agents, and local GPUs are changing workflows in business, education, and daily life.
Loading summary
Transcript91 lines
- [00:05]
A
Welcome to Reshaping Workflows with dell Pro Max PCs and Nvidia, where innovation meets real world impact in high performance computing.
- [00:20]
B
Welcome back to another exciting episode of Reshaping Workflows with Dell Pro Max and Nvidia RTX GPUs. I'm Logan Lawler, your host and today got a very special episode on Deck. If you watch, you know, Reshaping Workflows, we've interviewed, you know, kind of industry leaders from Nvidia, we've interviewed a few folks from my team partners. But, you know, variety is the spice of life, as they say. I like to kind of spice it up. And my friends at the Neuron, Grant and Corey are here with me and it's going to be a little bit different. But before we get into that, let's start with this. Grant, Corey, you know Grant. Let's go, Grant. First take, you know, 30 seconds a minute, introduce yourself, tell everyone where you work, what you do, and then we'll go to Corey.
- [01:06]
A
Sure, yeah. So I'm Grant. I write for the AI newsletter, the Neuron. I've been writing it pretty much for a year and some change now a couple months and we cover, you know, the top news in AI every day. The that's impacting the future, but written in a way that's very accessible for people. We find that in the AI industry there's two camps, there's quick AI summary snippets of what happened, and then there's super technical deep dives and there's not enough content that's in the middle. Just trying to explain to people what's going on, what's important and why you should care. And so that's what we focus on at our company.
- [01:48]
C
And I'm Corey Knowles, editor of the Neuron Newsletter, co host of the Neuron podcast with Grant here and managing editor of experimental content in AI and technology Advice. Most of my career I've been working as a journalist, but spent particularly the last decade in disruptive tech and looking at different things, which is what landed me in the AI space and learning every day for a number of years now and really trying to, like Grant said, explain some complex stuff to people who have no technical background necessarily, or at least limited.
- [02:25]
A
We try to keep up with the technical folks as well.
- [02:27]
C
We do, we do.
- [02:29]
B
And I appreciate the intro. So we, I told you for a couple reasons, if you're listening right now, why this is different is one is we've never had kind of a thought leader from the AI or kind of newsletter news space So I, I mean, there is no AI news space. So as far as I'm concerned, you're the New York Times the news, because you're the only one that I read.
- [02:49]
C
Can we put that on a business card somewhere?
- [02:51]
B
I mean, I don't know if that'll get through legal or compliance. Unfortunately, they're pretty locked down in that. But hey, my personal endorsement. Hey, you guys got it. But I've been, I've been reading the Neuro and I think it was described, you know, perfectly well. It is kind of that I would say middle ground where it is one, it's fun and there, I'm not going to say about any other newsletters. There's a bunch of AI newsletters out of there. But the way Grant and the team kind of write, I mean, they use cats a little bit too much. I'm more of a dog person. But hey, I mean, I'm not here to judge, but it's written a funny, hilarious way where it's very easy to consume for really anyone, whether it be, you know, you're an IT professional, you're a data scientist, or you're just, you know, a knowledge worker that's like, hey, how can I use AI to make my life a little bit easier, you know, when I do my, my nine to five in corporate life? So I thank you. Nailed that. But the reason I wanted to have y' all on is we have not. And I, and I think this is a, this will morph into a question here and we'll start off with this question. But AI is evolving so rapidly. I, even though in my best intent, you know, the neuron comes out kind of every day, I read it most days, maybe not all the way through. Some days I just file it away and I get back to it. But with the, the reason I want to have the neuron on was to talk about how quickly kind of AI is evolving, tools, things like that. So we'll start with a question. I'll start with. Grant is, I mean, we all know AI is evolving, but let's go back to the start where you were like a year ago when you first started this job. Would you say AI is accelerating faster at this point or was it accelerating faster a year ago when you first started in this job?
- [04:26]
A
That's a good question. I think that it's accelerating faster now, but perhaps not for the reasons that you would think. Like you would think it would be accelerating faster now because the models are just so much better and like they're self improving or, you know, the crazy kind of sci fi reason I think that is happening, but I think people are just better at using them a year from now. So like all of the engineers have like a year's worth of time that they've spent learning how to use these different codes, let alone the fact that you have coding agents like to help write, like what is the SAT? Like 90% of code is now written by AI at a lot of these companies, especially the AI labs. So I think that there's a lot of human reasons why it's accelerating in addition to the AI models getting better. And Dylan Patel, who's a really good analyst of the industry, he had a really great interview in Invest like the best recently and he broke down all of the different facets of what's going on. And there's a lot of technical stuff going on. But the number one thing that people are like, okay, is we just have to make sure that every time the models are getting better. And it seems like to this point not only are they getting better, they're able to work for longer and longer on tasks for you. That's where the agent side of things comes into play and that is its own kind of accelerating force. So I think there's the human element where we're just better at using these tools so they can do more for us. The models are getting better. They have consistently gotten better with maybe one blip, you could say, like GPT 4.5. It was technically better, but it maybe wasn't consumer friendly. They couldn't run it at cost from what we understand in the media. And the agents are here and they're getting better.
- [06:11]
C
The other thing is more people are hearing about it and more people are learning about it. More people are getting hands on with it than ever before. Like when this started, it was, it moved fast, immediately. But it was a small group of people. Not every company in the world was on board. Not every person you knew was at the only thing they were talking about. And as more and more people come in, as more and more businesses are adopting use cases, as we're seeing just an increasing number of product rollouts that aren't even necessarily just models, but products that use these models, even from the AI companies to do different things and make it into more of a, I would say kind of an ecosystem of things that can perform those tasks. It's really fast. The number of days that a story drops and then Grant and I have to blow up the whole newsletter and just as we get it done, another story drops that we absolutely Cannot not have. And we have to blow it up again. I mean, it happens often.
- [07:14]
A
You know what I also want to add to what I said. It's also impossible for me to answer this question accurately because I am so in it every day. When you look back, one of the newest model to come out from Anthropic is Claude Sonnet 4.5. Right. And that is an improvement on Sonnet 4, which maybe came out like a couple months ago. And then that was on top of Sonnet 3.7, which was maybe in February. So the increasing deployment of new models is also accelerating.
- [07:50]
C
And every company.
- [07:52]
A
Yes, across all the companies. So even though it feels like it's like Sonnet 4, that's old news at this point. It's like, no, it was actually like only a couple months ago at best.
- [08:03]
B
I mean, it's a great point. And I agree with the point around is that I think the news and the developments are all happening, but I think people are adopting, so it's more kind of commonplace. And I'll tell you kind of two stories. Well, I'll tell you one story is that, I mean, even this weekend I was at my daughter's softball tournament and we were looking for a place and one of the moms was like, hey, like, where do you want to go eat? And another mom pulled out of the phone and I was like, what are you Googling? She goes, nah, I went to chat GPT and asked and I was like, hmm, okay. But to your point is I. I also think that I was in a room and I won't tell you where, who with, but it was businesses, it was a corporate audience. And I asked, you know, how many of you are doing really anything with AI? And I don't mean necessarily developing, but I'm saying even using like within your business, and it was maybe 15%. So I think the consumer ramp has been very, very quick. I think the traditional Fortune 500 ramp has been slower. Now, granted, that varies between industries and stuff like that, but you kind of set an interesting point about, you know, how quickly things are getting better. Right. Minus a few blips. But here's a question I want to ask because you guys do a lot more of the benchmarking and testing and performance than I do because you're hands on with everything. Right? Is. And I'll ask Corey and we'll go to Grant is on that development curve. You all see it because you're in it every day. Would the normal person who's going, like you said from Claude and I don't even know, 4.0 to 4.5. Like are they actually seeing those differences or is that a difference more in, you know, speed, performance, power consumption, cost? Or is it something that, I mean, for example, in terms of performance is bad analogy. I always use a lot of softball analogies, but you take a really crappy Walmart bat versus an Easton ghost bat and it is night and day different. Like there's no comparison. Like you use them, you're like, whoa, like I know why I'm paying $500 for this now versus 50 bucks. But within, you know, let's use Claude Sonne as an example. To the normal person that's using it, you know that person, that engineer you referenced that's using it to write code, are they seeing a difference or is it just like this is just part of the deal?
- [10:14]
C
I really feel like we have reached a point where the average person, like, I don't mean like the average worker or the imagination, but the average human, some of these differences will seem small despite the fact that they still have a massive impact to maybe science, maybe research, maybe mathematics, to coding, to all kinds of fields. We're seeing that in the really technical stuff now, which I think is super cool. But for the average person who's going on here and trying to find out how long to cook a baked potato, the fact is it's been able to do that well for a while. If you're not asking it the things that really leverage that ability, you may very well not notice a huge difference, even though it is definitely present. Like, I mean when we look back, you know, the first research model is just a year old, just barely. And to think of how far we've come since then and what an amazing impact that was. Like I remember the first time I used 01 and I remember sitting there and throwing these, these hardcore like symbolic logic and algebra problems at it and stuff, trying to see what it could do and being pretty blown away. And now we are so far ahead of the. Every company is ahead of that. That's the thing. It's not like just those models from that one company, like that happened and now every company has great research models and as these leverage up day to day, like average person around the house type use cases or normal life are going to get limited probably with the exception of, I would say multimodal use. And maybe the idea of, I keep calling it AI of things, the idea that you're going to have little small models in your refrigerator, in your microwave, in your doorbell cam, stuff like That I think there's a day coming in the near future where that will be there in a maybe less noticed way.
- [12:17]
A
Okay, I'm going to caveat what Corey said a little bit. So OpenAI released this statistic when they launched GPT5, which As people might remember, GPT5 was the first one that came with this router. And the router is essentially an AI that basically decides what type of question you're asking. And they said that at the time that they launched it. Something like, I'm going to butcher this statistic so I'll send you the actual Link. Something like 2% to 7% of people, like all ChatGPT users, all 700 million weekly users have ever used reasoning models. And reasoning models are the.01 ones. So GPT5 was the first time that people had like the general audience has actually used the smartest AI possible. I think that really did a number on people's perception because like you have doctors who are still using 4.0, which is kind of like the consumer friendly version that's very simple to diagnose patients. And they were like, wait, no, we have to like force people to use thinking more often because it's like only 7% of people ever used it. And so I think that that was like a huge increase and it's still growing. Like I don't think everyone gets access to the and will continue to for.
- [13:37]
C
Quite some time, I think.
- [13:39]
B
I agree. Well you made a good point and I'm already just going to be all over with the questions. And I love just crazy wild questions is that and, and I'll, and I'll reveal my age here. I look a beautiful 42 as of about a month ago. And I remember in high school this will date me, an elder millennial is going to, you know, doing a book reporter research report is going to and getting an encyclopedia. I mean the Internet was a thing but it wasn't like it is now, right? Like I had to go to encyclopedia, had to do the research and I have noticed a difference and I'm not trying to stereotype, you know, different generations, but I look at my daughter and it's very much like I'll just Google it and get the answer versus I'm going to like kind of research. And I think a little bit of that being able to figure it out and a little bit of that critical thinking is being lost because we serve it up so quickly. So Grant, we'll start with you and then go to Corey is that let's use the reasoning model Example right, where you're, you're feeding it, it's basically doing all of what you could do in your brain in an AI model and you know, adoption, 7%. But one day it will be 100. Regardless if it's OpenAI or whoever, it'll be 100%. Do you think that that is taking away from humanity's ability to think critically and reason critically from what AI is offering as a, not as a service, but as an option to reason and think for you?
- [15:05]
A
Yes, unequivocally, yes.
- [15:07]
B
I agree too. I agree too, 100%.
- [15:10]
A
No, like, you know, Logan, you're probably familiar with this, but we covered a study that MIT put out that was about how like essentially when you write a paper with AI, you don't remember anything because he didn't actually write it. And you know, you kind of have to essentially still be an active participant in the creation process to get any sort of benefits cognitively. So yeah, I think that's a huge issue. Also another statistic OpenAI shared was I think it's like 73% of people use it for non work tasks and they use it for practical guidance. So by that that could be asking it for help, for questions, for support on things literally to the point where you're outsourcing your decision making. Like the example you said where it's like, yeah, let me just Google. We do that with Google. So we're already used to it and it's not like, you know, maybe that's hurt our ability to.
- [16:02]
B
The difference though is that when I Google, hey, I want to go to eat an Italian restaurant, it'll give me options but I still have to go in and look and see which one has the best reviews, which one looks good, which one has the menu versus hey, I want an Italian restaurant with ability to take 25 people there that has a kids menu that's under 20 bucks. All of that research and decision is eliminated with AI, not necessarily with Google. But anyways, didn't mean to catch up.
- [16:28]
C
On something my vegan nephew can eat.
- [16:29]
B
Yeah, yeah, exactly. And that one kid can eat. Yeah, exactly.
- [16:33]
A
In some ways I see that as a net positive. In other ways I think like, because like we make too many decisions a day. Like I think we're, we all get burned out by how many decisions we have to make a day, but we don't want to outsource all of our decisions. So I think there's some things that they could do in terms of the design of these products to make you maybe don't ever give you only one answer? Maybe they have to give you like three. They have to give you every option that fits your criteria and then you can still make the decision yourself. There's a design way to solve for that. But honestly, that sounds nice, the fact that you could just say these are my parameters, you tell me where to go. Go ahead, Corey, what's your take?
- [17:12]
C
My take is that if you're using AI in a way that is taking away your ability to think, you are doing it wrong. I really believe that this is a tool with the ability to help an individual learn beyond honestly any level we've ever had. Ask it to teach you, ask it to explain things to you. If you have a question about a concept, don't ask for an answer, ask for the answer and the why and then you're reading something you wouldn't like. I grew up with those same encyclopedias. The only difference was my set was like 8 years old, which meant by then the information in it was largely useless, probably for the most part. And I do love that. I do still love physical books and I do miss doing research in a library versus a database. But I really believe that for a person who wants to learn, this is a tool that has the ability to take them farther than they ever could have gone in less time.
- [18:11]
B
I love that and I agree. I can't write Python code, you know, just natively, but I can now like and enough where it'll get me through and I can limp through a few things but it works like it absolutely works. Here's kind of a follow up question to that is I think it was recently and I think I read this in the neuron because that's where I get all my info. But OpenAI added kind of a school, I don't know if you call it like parameter or button. What are your thoughts? We'll start with Corey and Grant as we kind of talked about the critical reasoning thing and it's kind of a two edged sword, right? Because with using AI it is such a skill. It's much like I had a computer in my house when I was like 6 years old and I work for, I work for Dell and I wrestle this with my daughter specifically where I want her to have skills that are relevant, that will teach her. But then also I don't want to lose it on that core education because I believe that that's important. So I guess what are your thoughts on, you know, AI in education specifically from any sort of generative model or reasoning model, like what are your Thoughts on it? We'll start with Corey and we'll go.
- [19:19]
C
To Grant, I think initially to first off, speak with, you know, study along. I think that's what it's called. Is that right, Grant?
- [19:27]
A
The it's either learn with me or study with me. Study with me and OpenAI both have this, I think one.
- [19:31]
C
Yeah, they both do. And I think that's a great tool when it comes to education. One problem we have is that education is sometimes lagging when new technology comes around. So, like, if you're a student graduating from high school this past May or this coming May, you're not ready for the world you're walking into right now. Frankly, we need to reach a point where we think less about AI as a means to cheat and more of it as a way to customize personal education for each student. Like, you know, we have IEP type programs for, for students that struggle in one way or another, or students with learning disabilities, which is wonderful that we have those. But imagine if every student could have one. I mean, I feel like it's really important that people in the education space, and a lot of them are, we hear from them regularly in our email, are looking for new and creative ways to both teach them how to interact and use LLMs to their benefit as a learning, like as a personalized tutor who can help you through individual problems. Like, when you're in a classroom setting, there's 28, 35, however many people there, and they can only go so slow, they can only go so fast. And as a result, some kids get left behind. There's problems you don't fully understand. This is a tool that could stop that in its tracks if used properly. And I look forward to seeing a future, I'm going to guess, two or three years in advance, where education is really embracing this tool as a means to help their smartest kids excel and their kids that are struggling the most excel even farther.
- [21:13]
A
Yeah, I'll second what Corey said. I'm forgetting his name right now. But there's a researcher that Dwarkesh Patel, another really great AI YouTuber podcast host, talks about a lot, who did a study maybe back in the early 2000s or the 90s, that basically showed that personalized tutoring dramatically increases the rate at which you learn. It has much, much, much better benefits, as you can imagine. That's just a better way to learn. It's just we don't have the human capacity to do it. We don't have enough labor for that. So in a way, that is the perfect way to solve a lot of education problems. There's a school that's a little out there that's doing this called Alpha School. They've been featured a lot in the press. And basically they have a two hour learning process where basically they only do school for two hours a day. But they use a really sophisticated system that essentially uses personalized tutoring to even make the lesson plans for what you're learning as a kid personalize and individualize to your interests. So basically it's like if you like superheroes, they'll write superheroes into the lesson plan to help you understand it, which is a little silly because it's taking note of your process and where you fall short and basically creating these personalized lesson plans. And they do it in only two hours a day. And their kids are accelerating, if you believe their statistics. And some kids do it based on mastery learning, which means that you don't move on to another subject until the system and the teachers figure out that they say, okay, you're good, you got it. And that means kids who accelerate are three or four grade levels ahead of their classmates in some cases because they're just learning that much faster. So, yeah, I think that system, that's an expensive private school. It's like $40,000 a year or something like that. But if we have software that does that and we can roll that out to all of the schools everywhere. Come on, let's go, we're off to the races.
- [23:11]
C
Imagine how smart the next generation could be or the generation after that. When you, when you think of going through your entire 12, 13 years of education that way, and maybe you don't need 12, 13 years by then. Maybe you're studying at the university level at 15 years old. I mean, you know, I don't know.
- [23:29]
A
We just have to design it in a way that like, it doesn't offload the thinking and onloads the thinking, it augments the thinking.
- [23:35]
B
Yeah, exactly. I agree. So what I love about talking with y' all and one of the other reasons I have is that I don't talk about it a whole lot because I'm more in kind of the, the data science, like developer side and some of the people we have on. But the biggest thing for, if you're listening to this and you're kind of an AI novice is really, you know, from a prompting standpoint. And we're not going to get into it, but I do want to cover this question. We'll start with Corey and then we'll go to Grant is I am not by any means a great prompter and. But what I have kind of learned, and this is my process, I want you to kind of tell me, grade my process and then tell me kind of your. And we'll limit it to the scope of not necessarily coding, let's just say kind of a reasoning model, right? Let's just use reasoning models example, doesn't matter which one. What my process is, I don't necessarily flood the context window. Right. I am more kind of straight to the point and I'll use a very pointed example that I do is that I use Nvidia's Parakeet model to take audio from this, Transcribe it about 38 minutes audio in about 15 seconds. It, you know, provides punctuation, timestamps, capitalization, the whole thing. And I do that because I'm usually write some sort of content or blog on each one of these and I don't have to go back and listen to myself talk for 30 or 40 minutes. It stinks. But what I'll do in that, and I'll usually use like, you know, LM Studio and you know, usually chat GPT OS and like use kind of the reasoning part of it. And I'll say, hey, my first kind of prompt is, hey, this is the content, you know, from this kind of blog or from this podcast. Okay. Hey, what do you think is a good format for a blog post? Like, what does a successful blog post look like? Boom. Okay, now that you have this, you know, I loaded in as, you know, kind of a rag, and I'll say, hey, now you've seen this. Take this content and plug it into that perfect blog format. And then boom. And then we're kind of off to the races, right? And I kind of keep it very simple. I'm not the best, but I would love to hear kind of your tips and tricks on how to get the most out of a reasoning model. Because I think everyone is very different. And no matter what, you'll get an answer. But what is good? What is good look like, I guess, is the question.
- [25:50]
C
I'd say it depends on the problem. For, say, a blog post, I would absolutely do something not that dissimilar to what you're talking about. It's, you know, I'll go in and I will have an idea and I'll just be like, hey, I want to write a blog post. I tell it what I want up front. I think that's important. Like, here's the goal that you're going to be chasing and then I'll say, here's some information I have. It's going to be about this. It's going to be for this person. It's going to be on this topic. You know, I want roughly this amount of length. I'd like, you know, a fair mix of, you know, bullets and paragraphs or something like that. And then I would tell it probably to craft me a prompt that will do that for me anytime I want. Then I take that prompt and I would throw it into a project where it's there for me all the time. And I know that anytime I want an article like this, I can go there where then I can go in and just simply write a little synopsis about what I want. You know, a couple of sentences like, you know, this is going to be about mixture of experts, and it's going to explain it in simple terms so folks can understand more clearly and in a. In a way that makes sense to them. Use analogies or whatever. And then I would say, here are my notes. Please, you know, find some impactful quotes from there and pull those out. Put this together, and then let me look at what you've got and I'll work through it and just go from there.
- [27:12]
B
I never save. Like, I can't believe I never thought to do that. But I always rewrite it, dude. I rewrite it every time. And it's always a little different, you.
- [27:20]
C
Know, like nobody uses projects. And I'm here to tell you, Grant and I live by projects. Like, anytime we do something that's great, it gets sucked into a project immediately.
- [27:31]
A
Yeah. So the way you can think of projects is like custom GPTs. Like, if you remember when custom GPTs were big and everyone was trying to implement those projects is just a much simpler version of that where it saves your prompt and any related context. So it kind of has built in rag. And then basically the prompt is essentially system instruction. So you could copy in, let's say your parakeet transcript, copy it in, and then you have all the things that Corey described, and then you have maybe a style guide. You can just paste the style guide in there. Everything that I write with the Neuron, I have a Neuron general purpose project folder that has our style guide that has previous examples of our writing style to emulate and just notes that I've picked up and gathered over time. I'm like, hey, if I'm writing a main story for our article, I need to make sure that these three or four things are hit. So I've added that to the context over time and Then that way, anytime I want to write anything, I can send it into that folder and I can give it a little bit of context, what you're saying where you just say minimal, hey, write a story on this. And then, boom, it writes in your style. It's pretty wild.
- [28:39]
B
It's wild. I was going to ask one of the questions, I was going to ask was, do you use AI to write the neuron? And I knew the answer was probably yes. But, like, when you read it, it's not. It's not. Doesn't feel like it. Let me just be honest, it doesn't feel like it at all. If you would have said no, I would have also believed you.
- [28:54]
A
Well, because it's a. It's a painstaking process of me fighting.
- [28:58]
C
The machine and it goes through a lot of human reviews still. Like, Grant sits down with these and.
- [29:02]
A
Works through extensively, sometimes until three in the morning. It's not a, like, simple automated process like some of the other people doing this probably are much better at it than I am. But the way that I approach it is basically I say, okay, I want to try and have the AI write it, and then I hate what the AI writes and then I give it a bunch of notes and then I write half of it myself. Or the opposite. I'll write half of it myself. Then I'll say, please fix this, it's trash. And then I'll make a hybrid Frankenstein version.
- [29:29]
C
Yeah, it's very Frankenstein by the time it's done, usually.
- [29:32]
A
Yeah. So it's definitely a messy process. But for every section that you read in the newsletter, I have a specific project to write exactly in that format. Yeah. So it's really easy for me to like, if I want to add like a news brief, I like, just give it the context and boom, it writes it in that way. If I want to add a tool to try, I give it the context and boom, it writes it like exactly the way I like it.
- [29:51]
B
Coming to that and piggybacking off of that is. And this is just, you know, my shameless plug about Nvidia and local AI. Right. There's a lot of different tools out there, Right. Some run in the cloud, some run locally, etc. So I know that I sent you both out two systems and I'm going to probably get this wrong. I know, Grant, I sent you out the mobile, the new Dell Pro Max. I think it was the premium mobile and then, yeah, it was the premium and then core. I sent you the new Dell Pro Max micro with the 20 gig ADA just because the new Blackwells haven't been released yet. So one, I would love to hear your thoughts and if they stink, you can absolutely tell me. And then I want to talk about each one of you. Kind of pick your, you know, you provided several and we're going to show the videos while you kind of walk through them. But kind of pick your favorite use case and talk about, you know, don't have to name brands, don't do any of that. But like what value did you see by kind of running it local? Was it faster, was it quicker, was it more efficient? You know, were you able to do stuff that you haven't been able to do on your old computer? What's some of the value of that for someone watching this that you could give, you know what I mean, give the value of local AI and why you should be thinking about it. And we'll start with Corey and then we'll go to Grant.
- [31:03]
C
Yeah, I'm running the micro with an i7 Ultra 64 gig of RAM and that ADA architecture and I'll be honest, I'm in love. The truth is it's really small, like surprisingly small, but it's dense like a brick and I've yet to push anything to it and it crash and tell me no. And I love trying to make it crash. That is like I actually crashed mine, Grant crashed his.
- [31:30]
A
We'll get to that.
- [31:30]
C
Grant killed one, we'll get to that.
- [31:32]
A
It's a skill issue. It's my fault.
- [31:35]
C
You know, for me, where I'm really, really seeing a lot of value with these is running benchmarks on open, open source, open weights AI models and trying to see like how do they run, how fast do they run. Like previously todoist, you know, I was running a 4060 on a laptop and it'll do a surprising amount of stuff frankly, but anything bigger than like 9 billion parameters and I'm needing to use hugging face spaces, things like that to try and have access to models that my current system just couldn't run as a fact of the matter. And with this what I'm able to do is I'm able to run much larger models. Like I can run GPT, OSS20B and I've, you know, there's a demo of me running it in low reasoning, medium reasoning, high reasoning and you know, the GPU stays cooler running a bigger model than my laptop GPU does running way smaller models. Like it keeps super consistent temperature. The system as a whole honestly is, I really thought being a small form factor like this and remembering, like the early, I can't remember what they were right now, but little bitty computers like that, that always had heat dissipation issues. And this really just does not. You know, I keep my, I keep my monitors up where I can see what my temps are looking like and they run very reasonably, even with image generation, because I do a lot of image generation and I dabble in video, but it's not a thing I'm great at. But with image generation, it doesn't really grunt, you know, it gets up around, I'd say, I think 74 degrees Celsius is about the hottest I've seen it. And you know, they say 75 to 85 is like prime running area. It cools back down quickly, it runs hard. But the key is, what this gives me is the ability to test these all on one machine because I know how the machine runs. And when I look at this, when you're doing this online, you can't compare tokens per second from one model to the next because you don't know what the computer is where. With this I'm able to run anything up to 35 billion parameters quite easily and, and look at those tokens per second number and really see where that speed's happening, what I'm getting out of each. And it just results in better data, frankly a lot better data in what we're seeing in models. And the ability to compare them head to head, that's been a really big thing as well, as I mentioned, local imagegen. But one thing to know about generating AI images on your computer is you're essentially trading your patience for quality. You decide, how much quality can I push and not lose my mind waiting for this thing to generate. And one of the things I like to do is I like pretty high resolution images, so I don't mind waiting a little bit. So the difference is this makes that bar higher. You know, I can pretty quickly generate thousand pixel square images, you know, in about a minute, which is pretty fast, and really crank up the steps in them so I get a higher quality out of what I'm seeing. And it's just a minor trade off there between patience and quality is what it's the thing I love to say about it because it's, you know, you go as fast as you want, but you get what you wait for.
- [34:54]
B
Exactly. So tell me, Grant, how did you crash it? I'm very curious.
- [34:59]
A
Okay, so you mentioned not to mention brands, but may I? Because I think it's relevant.
- [35:03]
B
Yeah, it's fine.
- [35:04]
A
And I don't know if you've talked about this before, but the way that you make the easiest way to make image models or images on your computer is with Comfy ui.
- [35:12]
B
Oh, I use that all the time. It's my good view.
- [35:14]
A
Yeah, yeah, yeah. It's awesome. It's great. So let me paint the picture for you. I'm running Comfy UI. I'm running LM Studio. I also have Docker open, I have GitHub open, I have Google Chrome open with like at least two windows full of like 10 tabs each. And I'm streaming a video. And on top of all that, I have Steam open and I'm downloading three games. I fire up Quen Image. I believe it is the new Quen Image editor, and I'm generating an image with that and it's working. I'm also downloading not only games, I'm downloading folders off the computer because I'm trying to push this thing to the absolute limit. And at some point I realized my downloads have slowed to a stop. And I have no idea how to see if my Comfy is actually loading because there's no loading bar. So I don't know what's happening at this point. And then it just shuts down. So I think it's fair to say that that is an extreme example that you really have to push it to get there.
- [36:22]
B
That's. Did it just shut down or did it blue screen or did it just literally go and say, hey, we've reached an issue and we're restarting? And then it restarted.
- [36:30]
A
It just blacked.
- [36:32]
B
It just went done.
- [36:33]
A
Yeah, it was just like, you know what? I'm done.
- [36:36]
B
I don't think I have done that. Like, I've done it where I've like ran out of video. Like, I run out of vram and stuff and it's like, we're done, bro. Like, we're just. We need to go over. Like, I've done that and I've crashed a few things. But me, it's nothing atypical, but like, I mean, I love that story because I think that is the kind of. The cool thing about local AI is that it does have a limit. And finding that limit is kind of cool. And if you want more of a limit, you have a bigger GPU or a little bit bigger system, or you have a tower versus mobile. There's a lot of things that you can, you can do. But if you were trying to test the limit in the cloud, there is no limit. The limit is Your wallet. The limit's your wallet. Like how much do you want to push your wallet? And I don't know, would it have been worth it? Probably not. Yeah.
- [37:20]
A
And that didn't cost me anything.
- [37:21]
B
It cost you nothing?
- [37:22]
C
No, nothing.
- [37:24]
A
I just restarted my computer and it was all good.
- [37:26]
B
Yeah. And we're all great. Exactly.
- [37:29]
C
Were you concerned when you restarted that it might not be all great?
- [37:32]
A
So I'm comparing this. So just for context, I have a. I think it's an MB16, 250, 24 gigabyte GPU, 64 gigabyte RAM. I'm comparing this to an M2 that I have for my personal use like Apple with like 16 gigabytes unified memory.
- [37:49]
C
That's a big difference.
- [37:50]
A
Yeah. So it's huge, huge upgrade. And it's a Mac. Right. So I'm comparing PC to Mac as well. I've had PCs all my life, but it's just a different load up screen. And I'm like, did I kill it? Like did I kill it? Bitlocker is, you know, asking me for some code from Microsoft. I'm like, what the hell am I going to do? Like what is this? And yes, once I got through that screen of confusion, then I was like, okay, we're good, we're good.
- [38:17]
B
I love that. Well, we're getting up against it and I. This has been a fantastic episode, but I want to end it kind of with this and you know, I guess let's look forward future a year from now. So we kind of talked about a year ago. We talked about present day. We'll start with Grant and then we'll go to Corey is where do you think if we did this episode? Exactly, you're in the future, what would we be talking about?
- [38:41]
A
So I'm going to give a little context. OpenAI released this roadmap that essentially covered what they were planning to release or focus on. I don't know if it's like year one, year two, year three, year four, year five. Obviously if they succeed or fail that might push it longer or shorter. But first they did obviously chatgpt, then they did reasoning models with 01 and that came out like Corey said last of last year. This year was all about agents and they famously broadcasted this and that's why everyone has been focused on agents this whole year. The next thing that they're now starting to seed into the atmosphere is that they're working on AI for science. So if they're successful in agents really taking off and getting adopted, which people are still kind of Iffy on like, you know, what's the use case besides deep research and coding agents? We haven't had too many good general purpose agents yet, though there are some cool demos. Can they then take it to the next level where they're actually using AI to automate science? And there's some really cool projects that have been announced in this regard. There's a company called Periodic Labs that essentially is combining AI models with an actual automated lab. So they're going to be running experiments where they're taking streaming tokens like you and I do when we're chatting, but then they're streaming the tokens using tools that is basically like, okay, go run this experiment now in an actual lab environment and then come back, give us the data and training on real world data. That funnel is sick. So I don't know if we're going to have updates on that project in a year. It might take longer before they're ready to share. But that's a huge area that I think in a year from now, if there is progress there that's exciting and meaningful, we'll definitely be talking about that. The flip side is maybe, you know, there's a bubble pop a lot of people talk about. This could happen anytime between the next two weeks and two years from now, if ever. There's a wide spectrum of possibilities there given how transformative this is. So we could be talking about a bit of an AI winter.
- [40:45]
C
We'll see, I'd say, I think number one, we'll definitely still be talking about GPUs every day. I don't think that's going anywhere. I think from a consumer perspective, specifically world models are what you're going to see really explode over the next six to 12 months. I'm seeing some amazing things people are doing right now, even just with video and creating these like cinematic shorts that they're. I say shorts, maybe, you know, five minutes, 12 minutes. These short films they're making using entirely AI based. And I think you're going to see that get bigger. I think you're going to see feature films, possibly even in a theater within the next. By the end of 26, I would say you'll see a feature film in a theater. These world models will be doing some amazing stuff in gaming really soon. Like we've come a long way from just text based games created in a chat interface to using tools like Grant Black Forest Labs. What's their tool called?
- [41:48]
A
Flux Plux.
- [41:49]
C
Thank you. Well, no, the world model. Oh no, is that.
- [41:53]
B
But you're talking Like Nvidia Cosmos kind of world model. Yes.
- [41:58]
C
Okay, yeah, these things are right now, you know, they're working on Unreal Engine. That means game developers are actively working with these and creating things. And what you're going to start to see is patch updates and new content drops come faster I think is one of the first impacts of that. They'll be able to do that kind of stuff faster. There'll be more new content, which is.
- [42:18]
A
Great piggybacking off of that. The reason they're focusing on coding agents so hard now is if you automate software and make the software iteration, loop so much faster, everything else gets faster. Like you can make games so much faster, you can make the automations in manufacturing so much faster. Everything gets faster if you can fix that.
- [42:36]
C
And I think we'll see the first system kind of improving itself within the next year. There's glimmers of that right now that are things we've seen and even Suspicions that maybe OpenAI is working with something like that already and some improvements that came over late summer. I'm not going to say we'll be at full self recursion, but I do think we're reaching a point where these models are going to be improving themselves as they go as well.
- [43:05]
B
And I'll say this and then we'll wrap it up is that I have a partner that I work with and for people that don't know is like, yes, short films, all that, not to take away from creative people, but there's more AI in what in TV and shows and stuff than you would ever think. And the best example on my part, I can't name any names, but he works on a show on a channel and we might. That channel might start with an H and they have done this show and they're doing a redux of the show, but to create the scene in between the talking would require them to literally go out to the desert like wherever and like film this and all this. And he was like, nah, man. Like he's using his Del Pro Max, his T2 to generate these scenes because there's not really a consistent character throughout him. It's more of like a storytelling thing. And he's like, yeah, like we saved millions of dollars on this production because no one cares what the sand looks like as long as it looks somewhat real and there's an oasis in the background. Like no one cares. And he's like, why would we even mess with it?
- [44:11]
C
Never even notice.
- [44:12]
B
They'll never even notice. Yeah, they'll never even notice. But no, this was great. I knew this would be a great episode. So we'll, we'll end it. Here is Grant. We'll start with Grant and then go to Corey. Tell everyone you know how they can connect with you. You know where to follow. For example, you guys podcast on the Neuron. We'll include the link below but tell them where they can find you and connect with you and then we'll wrap it up.
- [44:33]
A
Yeah. So we have a podcast on YouTube, Spotify and Apple Podcasts called the Neuron. AI explained we about seven, almost 10,000 subscribers just on YouTube. Almost double that on Spotify and growing. And then we have our newsletter you can find@the neurondaily.com or theneuron.AI. we have two websites. It's just a thing we're working on that we're working on it. Work in progress. And then I'm personally on x dot com. I don't even remember my handle. Don't follow me there. Just follow the Neuron on there.
- [45:07]
B
Just follow her.
- [45:08]
A
I just follow AI news on X on boring.
- [45:10]
C
I love that you can follow me at Corey Knowles. C O R E Y N O L E S and find me on Sora. I'm having a lot of fun on Sora right now at See no Evil. Come make some goofy videos of me.
- [45:23]
A
Oh yeah. Have you tried Sora too, Logan?
- [45:25]
B
I have not. I mean I've been traveling. I mean you are in the two days in which I'm at home pretty much this entire quarter. So no, I haven't. I've done, I've done nothing. Like I've got hardware stacking up at home. We've got products launching next week. Well, as of the time of this recording, we have products launching next week. So no, I mean I plan on getting back into it. That's going to be my Q4 for sure. Well, cool. Well, I appreciate it. If you haven't, please check out the Neuron newsletter. It is well worth the sign up to check out the podcast. I'll be on you know your guys podcast hopefully in short order. And with that, you know, granted, Corey really appreciate the time, you know, have, have a great day. And this is Logan with reshaping workflows with Nvidia RTX Pro and we're.
- [46:28]
A
Do what you want. Do what you want.
- [46:31]
B
This podcast was produced in partnership with Amaze Media Labs.