← Reshaping Workflows with Dell Pro Precision and NVIDIA RTX PRO GPUs
Reshaping Workflows with Dell Pro Precision and NVIDIA RTX PRO GPUs
Mastering Comfy UI: Next-Level AI Video & Image Creation
00:42:22
Discover Comfy UI secrets, pro GPU workflows, and game-changing tips with Julien Durand and Logan Lawler. Elevate your AI creativity in this episode.
Loading summary
Transcript85 lines
- [00:05]
A
Welcome to Reshaping Workflows with dell Pro Max PCs and Nvidia, where innovation meets real world impact in high performance computing.
- [00:20]
B
Welcome back to Reshaping Workflows with the Dell Pro Max and Nvidia RTX Pro GPUs. I'm your host, Logan Lawler and I cannot tell you how excited I am to have this episode finally is I got my start, you know, when I moved into this role and I got my start kind of on the AI team leading AI strategy for Delpro Max. You know, one of the first, you know, quote unquote AI workflows use case I did was for an animation studio using ComfyUI. And I learned enough to be dangerous enough to accomplish, you know, kind of the objective I was looking for. But we have the man, the myth, Elian. You might also know him as the mid journey man if you follow him on social media. He has quite the social media presence. So Julian, thank you for joining me today. If you would take just a few minutes, give everyone kind of background who you are, you know, kind of how you got involved with Comfy and then we'll just dive right in.
- [01:17]
A
First of all, thanks for having me on Logan. I'm very excited and yeah, I've been waiting for this for, for quite some time since we met at the upscale conference and we talked after. I was like really, really looking forward to this moment. But yeah, so how I got started on Comfy. So first of all, I guess I'm going to talk a little bit about myself. I'm a mechanical engineer by trade and I'm also a photographer. So I think I have like the tinkerer mind and I also have the, you know, like an eye for beauty, what looks good. So this is why, you know, like I think I'm wired for, for using Comfy. But how did I discover Comfy? So I first started with using Midjourney, which was obviously very far from what, you know, the kind of things I do in Comfy. Now that was fun, you know, that was the creative side of things. But I, I mean like I said, I still use Midjourney. It's, it's a great tool but I was not getting my, the kick out of like, you know, it was not exciting. It was just too, too simple in a sense, you know. And then I started to see these weird like videos and that's when I started to use, I heard about disco diffusion and I started to use that one a little bit.
- [02:28]
Which was like, you know, like a way to generate like videos at the time. And finally right after that I saw another type of video which was Warp Fusion. I don't know if you remember those animations which were style transfer. I got into that and that was really awesome. And that's when I learned a lot about control nets and all these concepts. And finally a little after this.
- [02:52]
I got exposed to SDXL in Comfy, but I didn't really fall in love then. When I did fall in love in Comfy is when I started to see and hear about an animatediff that motion model. And that's when I started to dive in and. And I literally fell in love with the UI and the. Because at first I was using automatic 1111 for deform as well. I should have said that. And The UI of automatic 11.11 was quite simple but also quite limiting in a sense. And when I first used Comfy I felt overwhelmed the first day or so, you know, as everyone does by the way, let's be real, you know, everybody's kind of overwhelmed. At first at least was because things are getting so much easier now. But I was like, oh my goodness, this is a lot to take in. And then I was like, wait a minute. It actually taught me how workflow, how diffusion works because the complexity of config actually is its strength because you see you go from conditioning and then this and then this and you end up in the ksampler and then you decode. And so I fully embraced it, I fully fell in love with it and, and that's how I, you know, I just, you know, kept on using it more and more and more and, and it's just, yeah, the, the amount of things you can do with it is just non stop.
- [04:10]
B
So anyways, that I agree, I mean Comfy for me, very similar and I, and I am not technical at all. I had to become technical for this role and I've never had more hot sweats in my life. Trying to get it installed kind of the first time and getting it to work and installing Cuda and all this kind of stuff. But the thing about Comfy I love, and then we'll kind of dive right into questions is that it is as complex or as simple as you want to make, can do insane things or it can be very straightforward. And the other thing I love about Comfy, it's, you know, free. We'll put the link in the description where you can download it. But it is got a really great community, you know, with support, it's very well managed. You know, the GitHub repository is managed very, very, very well. So you can always find support. But first question I think that I want to ask is that, you know, with Comfy, right you've got a lot of different ways to run it, right? The first one being you can kind of get the portable package, you know, you get the latest commits available on Windows or you can do the manual install. Do you have a preference or recommendation kind of on which one you would pick?
- [05:19]
A
So I personally do the Git clone like you know, like the old fashioned manual way. It's just because I like to have the control and it's funny so I don't have a really good reason why but it's just like I'm a creature of habit I would say. And I just started you know, like with a manual like Git, you know, like install should I say?
- [05:40]
B
Yep.
- [05:41]
A
It was kind of a way for me actually the reason why I liked that way at first is there was a set of custom node I believe that I had no choice but to be on the manual install at the time. Yeah, I don't remember what it was but it made me switch from portable to that. And since I actually fell in love with the idea of learning how to branch or to be able to branch, you know, like do test like a different branch or for a new feature that's coming out and use the nightly version of Comfy, a new front end. I like that but it's kind of like more advanced but somewhat recently part of testing something for Comfy I did the desktop install and I was actually on my Mac actually I saw one time friends Mac sorry but my wife, I did just for fun install Comfy and it worked surprisingly well. They did a great job with the installer.
- [06:34]
B
Yeah, it's great. I mean I've used both. I like the ability with the portable because it's kind of maintained for you and you're not having to necessarily go in and update dependencies and all that kind of stuff. But. But the one key thing I think people need to understand about Comfy is you can really install it on any os, right? Whether it's Windows, Linux, you know, Mac OS doesn't really matter. But where it comes down is you can run it off a CPU and I have a personal stat that I have tried but I would be curious to hear, have you ever played around with Comfy and generated something with a gpu, then took off the GPU support and then did it just on CPU to how long it takes?
- [07:12]
A
It's a great question because my answer will be Reversed than what you just said. But before when I was on the Mac like a couple years ago, I did run you know like diffusion. It was not comfy but same thing, it was automatic 11.11 on CPU only or you know like on the Mac chip. And it was insanely slow. And the reason why I ended up building a PC was purely only to be able to run comfy, you know. And it, this is why I and I went from 30 seconds per. Let's just say I think it was something like that. 30, 35 seconds per frame. Sorry to you know, a second or two per frame. And it was like wait a minute. So that was game changer. So I know you can, but I don't do it. I just stick to Cuda, you know, that's just.
- [08:04]
B
I do too. But I did it because I knew we were going to record this and did it out of morbid curiosity. And I personally like. I mean I know there's so many diffusion models, video models out there. I personally like Flux. It's just me, I like it. And it was just using the traditional workflow that you get. Nothing fancy, you know, just a 10 by 24 to create the image. I want exact same prompt seed, you know, everything. Nothing changed with you know, a 6000 not Blackwell, but it was an ADA 48 gig card running in my tower was about 15 seconds. That same exact image. CPU only which was an i9 I believe if I remember correctly was 14th gen. I think I can't remember the amount of cores and all that, but it was the highest in 99. Was an hour. It was an hour.
- [08:54]
A
That's crazy. Yeah.
- [08:55]
B
And GPU is absolutely required, you know, for this. It's absolutely required.
- [09:01]
A
I agree with you. It's a like the, the Nvidia like you know, like infrastructure is just a must have, you know or.
- [09:07]
B
Agreed. Exactly. So let me ask you, you made a really interesting point. You talked about how, you know, you kind of started with X computer and then you're like wow, I really need kind of a GPU and you can run, I mean comfortable run on anything. It's just how much time do you want to take? But what have you found some limitations of other than time? What are some of the limitations within Comfy of not having you know, a dedicated GPU or a large enough gpu?
- [09:35]
A
Yeah. So I think like my main like limitations are for sure that what you just listed just. I'll give you an example. How about this? A real life example. So what I'm going to show you today is because 1.2.2, you know, came out last week. And so I've been messing around with it and playing with this and it's pretty incredible. But it took me, I was struggling for an hour on. Currently I'm running a 4090, right, with 24 gigs of vram, which is, you know, it's not. So it's pretty good. And it usually runs for overall all these mainstream workflows and models, and it would not run. I was, you know, running out of a memory like a vram and I was like, what is going on? And I was asking around and everybody was like, no, you're good, you're good, you're good. And you know what it was, I figured it out is I was using Light X2V to go from 20 steps to just 4 steps to render quicker. And the list of Loras listed on the website, there was like 20 of them. I was like, I don't know, I'm just going to pick the biggest one. I mean, you know, like out of like three. And then so I took. There was like, rank 64, rank 256. I picked the 256. And that was the reason why I barely went over my 24 gigs and I was getting out of memory every time. And that was it. So these limitations are still, to this day frustrating because that's one or another great example is when I do, let's say I do a style transfer project or video where I. Let's say I have a video of you dancing, Logan, because I heard you're a great dancer. But let's say I have a video of you dancing, right, for whatever reason and the background is interesting or not. And I need to ideally load multiple control nets, right? And to get your pose, get, you know, like the depth, get the line art, you name it, all these things, plus animative model, plus this, plus this, plus that, whatever, right? I am limited right now with 24 gigs, you know, for example, or so if I had less than 24 gigs, it would be even more limited. And so logging all these models on the vram, I have to basically be strategic about things and be careful and conscious of how I run my runs.
- [11:49]
B
You know, it makes total sense. And I mean, and I know we're getting you set up with the, the Dell Pro Max, with Blackwell, with the 96 gigs. That's a limitation even I personally run into because I'm running, you know, a 48 gig ADA, desktop or desk or fixed, you know, a 6000 ADA with 48 gigs and I was using Nvidia's kind of Cosmos model, Cosmos world model because I was just like, hey, you know, it's really more physical, like physical AI versus like video whatever generation. And that's one very big model. But two, like if I tried to generate more than 10 seconds vram error out of vram, completely done. And I think that is a thing where you. I've seen so many people that are used, you know, GeForce cards or 4090s, 50 90s, 50 70s, 4070s and it'll do the trick. But once you start getting into models that are bigger, specifically in video, you are going to run out of your or you're going to have to get extremely creative on how you quantitize the model and like bitrate and stuff and how much you can actually produce. But let's not keep them waiting. I know you created this workflow, I would love to see it. If you want to go ahead and share your screen and kind of walk us through kind of what the workflow is, what you used, I would love to see it.
- [13:00]
A
So what's funny is the workflow that I'm going to show you and actually I'm going to give you a little, I'm going to give everybody a little trick which you love is. And let me show you, let me, you know, spin, spin the workflow. So here we go.
- [13:13]
B
Well, Julian's getting this set up is that when what's great about Comfy is a lot of the stuff is pre installed where you can go into the command center or the command and basically download a bunch of stuff but also go check out Cven AI, go check out Hugging Face. You can go get a lot of these models, whether it's Flux or other ones and bring them straight into Comfy, which is fantastic. But now you, you've got it up, let's go.
- [13:35]
A
What I do love about Comfy is, you know, like people, a lot of people don't know this is. And it's kind of funny because you just hinted about it. If you go to Workflow and browse templates you have a bunch of you know, like built in workflows you can play with. Actually I use the pictures. So Flux for example, you said you like Flux. You have now the new crea and you. And these workflows are all basically.
- [14:01]
Not factory. Yeah like factory or like certified or made by Comfy, the company directly and are really and using only nodes that are coming with Comfy. So let's just say I wanted to Use crea dev. And what's amazing, it runs right away, as you can see. But there is this pop up and this popup is amazing because it tells you, hey, you're missing these models. So you can download them. And if you don't know where to put them, well, guess what? It always tells you here where these models need to go. And so then you're ready to go. And so what I did is for fun today is because it's really exciting and it's a great test for the Nvidia GPU is I of course wanted to play with the 1.2.2 and Comfy. The latest Comfy is coming with four workflows. I mean the fourth one is not listed here because I need to update. But anyways, the 14B text to video, the 14B image to video and the 5B video generation where you can do text or image. And the fifth one is kind of like a clever workaround where it's an image to the video but with a start and end frame. And that's my favorite. It's amazing. It's super fun and it creates amazing like incredible renders. So I'm going to pull that up. So this is the workflow that you would get from Comfy. But I made some changes.
- [15:27]
Thanks to the great community like Purrs, for example, I'm sure you've heard about Pers.
- [15:33]
Was talking to him and we optimized this a little bit and right there, using this Lora right here, the Lite X2V. Let me show you this Lora here. Oh, by the way, this is the story I was telling you. See the rank 256 versus rank 64. This lower lets you basically increase the speed or by lowering the number of steps. And actually it's very interesting and very good to know for especially in the case of Nvidia. And great GPUs is they went from one model to two models. A high noise at 14 billion parameters and a low noise at 14 billion parameters. But the way it goes is you have your latent that goes through the high nose to set the stage. The overall motion of the whole scene that you want and then that latent get passed on after in this case two steps from zero to two, it gets passed on to the low noise. The low noise is going to be like the upscaler. It's going to refine the textures, it's going to add the tweaking. So think about it, it's genius. You have a 28B model that runs at 14B so, so that's pretty great. And anyways, what I did is like I said, I implemented LinkedIn the Light X to go four steps because it just makes it that much faster. And also I love to interpolate. So I render at 16 frames and then I interpolate twice to get 32 frames and to get smoother motions. So what I think is fun and what I wanted to do, which by the way, I loaded the wrong one, but same difference here. It is very similar. I loaded the text video here where you put your prompt, but this one is the one I wanted to show. So what I figured would be fun is to just. Let's just have some fun with the Dell logo, for example, right? So let's just make the Dell logo morphing into. Let's see here, so many options like this, kind of like a gradient sunset. I'm just going to queue it, you know, with like the Dell Promax Tower, this would take a lot less time, but right now it is what it is. It's not terrible. Currently it's like a minute and a half, give or take. But like I said, I have to be careful. You know, I had to choose the right. I am basically riding the line, right, with my GPU at this point. But like with the, you know, the Nvidia Pro card, it would just be. I wouldn't have to worry about it. It would just run like. And as you can see right now, like, look, I don't know if you're seeing, but like my background is stuff though stop being blurry because like the GPU is, is getting pushed. So who knows, I may not even be.
- [18:17]
B
Yeah, it's getting pushed. I think it's at 100%. It looks like it's maxed.
- [18:21]
A
And that is right there. Exactly what. And that's why I'm going to stop just to be safe, because I don't need to. To run this to show you, because I made some before, but this is. I'm glad that this happened because this is exactly why I can't wait for the Pro Tower because of this right there. Because you see how careful I have to be. The good news is I thought about this and I just preemptively, literally made a couple renders and I will show you the before and after.
- [18:49]
B
And this is why you're an engineer, because you thought about this. I would have never thought about this.
- [18:53]
A
Right. I always like to prepare, but this one, it's exact same workflow, precisely the exact same, with the same images. And this is what I Got. And it's so fun.
- [19:04]
B
That's incredible.
- [19:07]
A
And I can extend it. This is 81 frames, by the way.
- [19:11]
B
Okay.
- [19:12]
A
And look, you know, like if someone from Dell may see this, like, oh, can we use this for an advertising agency? You know, a campaign coming up. It's. It's fun. I love it. I was very surprised. So that's one that I made.
- [19:25]
B
It's amazing.
- [19:26]
A
Then I kept on going and I had a little more fun. I said, let's just turn the logo. Let's melt it. Melt it down a little bit and set it on fire.
- [19:33]
And here is that. I love it.
- [19:35]
B
I love how the E shrunk.
- [19:37]
A
Yeah, me too. It's funny. It's like it's melting a little bit. Exactly. The last one that I made was this one which was like more like a. You see, it's very subtle, but you see how good the motions are and it's really fun. But again, the only downside is I have to be careful right now. I cannot just do this is it. I'm doing this and nothing else. Literally nothing else. The other workflow that I wanted to show you, if it's okay to. Unless you want to see more WAN stuff. But if you want to see the next workflow, which is again like a very, very powerful workflow that it was a combined effort where it's a. This is the beauty of Comfy because it's open source. This workflow I'm going to show right now is the exact definition of why Comfy is awesome. Is someone has an idea, they make a little workflow, they share it and then someone else takes it, improves it, reshares it, and so on and so forth. And the version I'm going to show you now is the one that I worked on last. It's the last iteration and it's a video upscaler. It still works like, I believe one has like a very powerful video upscalers too now, which I haven't tried, but I've heard great things about them. But this one I'm going to show you right now is still like very irrelevant. This workflow, what it does is you load a video. And I did what I did. The video that I did load was actually. But the video I upscaled is this one. Because the one thing I didn't specify is I did generate the one video that. Because I don't have a Dell Promax yet. So I had to render at 640p, which is quite low res. The quality, it's a very high quality, 640p though I'm sure you can see from your side. But it is still unfortunately at 640p. So this workflow I put in a video and what I did is I implemented some math to just look at the shortest side of my video, which in this case it's a square. So it's the same 640 by 640. And then I can just say, first of all, because it's using a GAN mixed with tile diffusion, and I basically give the number of my GAN upscaler model, which is a 4x. That's what I put here. And I also put my target resolution, let's say I want to get to 1080p, right? And then I set my target here and then it goes through like, you know, like an animatedif and a very low denoise. As you can see, the Denoise strength is. Where is the. It's right. I'm not seeing it here. Yeah, right there. 0.2. It's a 0.2 denoise. So it's very, very slight. And it does it in tile. That's the only way this works because I see, I do 2x2, otherwise it would. And that's funny because currently I have to be careful, right, with the 4090 with 24 gigs of vram. But guess what? On the T2 with the.
- [22:30]
B
You're good.
- [22:30]
A
I don't need to. I can literally replace this node by just a full on regular case sampler and just upscale like full pop right away. But anyways, it's still a clever way to break down an image that's a size into little quadrants and you just diffuse the quadrants and there is a little bit overlap so that it's seamless. Otherwise you see lines.
- [22:51]
B
Okay, this is way beyond what I can do in Comfy.
- [22:55]
A
But that's the beauty of this though.
- [22:57]
B
Is you could share that workflow with me and I could just drag it on and then boom, there it goes. That's the best part about Comfy.
- [23:03]
A
You use it and then you start thinking, man, I don't like the fact that it's always doing. I'm going to keep it insanely simple. You're like, I don't like the fact that it's always upscaling by the shorter side. And you're like, I always want to upscale by the longest side. And you're like. And you see this, you're like, wait a minute. And then you just start, you know, dissecting the workflow and you change things, you tweak things and you make Things your own. So the result, which I will show you because I already rendered it just in order to save some time, obviously. Right. Okay. So I'm actually going to pull the. Because I have it right here. This way you'll see the difference. So this one was the straight out of comfy with 1.2.2 at 640p here. Okay, here we go. So that's the 640p that we saw. I'm going to play it again. And literally, after running the workflow that I just shown. Boom. Oh, that was on the fire one. Sorry, my bad. I'm gonna just. I forgot I did the fire because I figured, like, fire would be more interesting. So that's the little fire one that we generated earlier. Right. We've won 2.2 at 640p. And this is the one after Upscale.
- [24:10]
B
That's insane.
- [24:11]
A
And look how more it's smoother. It's just. And it's also 1080p, you know?
- [24:18]
B
That's correct.
- [24:18]
A
1080X1080.
- [24:20]
B
That's amazing.
- [24:26]
A
That's why I love Comfy. Because you want to hear something crazy. Yes, I do. You don't need to. But guess what? And that's the beauty of Comfy, you know, like, you can literally take this, right? And you know, like something that's coming in. Comfy. I'm pretty sure I can talk about it because it's. I think they already announced it. But you have subgraphs. Subgraphs is. I don't know if you're familiar with TouchDesigner.
- [24:47]
B
I've heard of it.
- [24:49]
A
So you could basically make a node, and within that node you have a bunch of nodes, right? And when you zoom out of that node, you have an input and output, for example. So the same is coming to Comfy, where I know this is my input, always the video, and my output is the upscale video, right? So I could literally make this upscaler, a subgraph and call it the upscaler. Right. I could then take this and put it on this workflow, which was the 1.2.2. And always.
- [25:22]
Basically replace these two here by the upscaler right here. So you would only see that one node, which would be called upscaler, where I would take this video, upscale it, and at the end of that one workflow, it's automation. Think about it.
- [25:36]
B
That's insane.
- [25:37]
A
And you know why I say this is? Let's just say I have my T2 there on the ground right there, ready to run. And I know I'm going to do grocery shopping. And I know I have to upscale 20 videos or generate 20 videos. I can just queue right there, you see at the bottom right you can queue.
- [25:56]
B
Oh, I love the queue feature.
- [25:58]
A
Yeah, it's the best. It's awesome. And I can just queue 20 videos, run, walk away. And guess what? I come back Later, I have 20 full on upscale videos with different seeds and so different flavors and I can pick the best one and they're all 10 ADP right away.
- [26:11]
B
It's amazing. I mean I love that feature. Another feature that I love and we're going to definitely do a follow up. The other thing I love is the wild card feature. Have you ever used that in Comfy? Oh you have? I've actually, I've used a different feature that you've had. There's no way like I don't think.
- [26:27]
A
So describe it to me because it doesn't ring a bell.
- [26:29]
B
So basically you just. It's a, it's an image generation thing, right. But what you do is like if you're looking for. And when I was doing it was the animation like the animation studio where they. We kind of had the character, right. We have the control net for the character reference design, all that. And then basically we wanted to put the character in a bunch of different scenes. So we kind of created a lore around it where it was like I can't say the name of the character but character X, you know, in a dark dungeon, whatever. And if you, if you, yes, you can create 20 different images of that but you can't create 20 different images of 20 different things. So basically what it does, it's a workflow and I'll have to share it with you at Adobe Max, but it's at the top where basically you have a text file and you write down kind of like, you know, whatever it would be, right? Like wearing a, you know, in the background at a beach, at a pool, at your house and all the different ones you. And then when you queue it it basically once it generates it will then drop to the next line and replace that wildcard with what you have in the text document. So that way when you do it you can have. Because for example, if you were going to create that video and you were to say, you know, create the Dell logo merging into fire water, space, whatever, each one of those would require a new run versus you could just do that. Come up with what I like. Yeah, super cool.
- [27:55]
A
That's cool. And so this what you just described, which sounds like amazing and again this is like the automation side of Comfy which is Great is the fact that you can. I don't know if you've played with a Flux Redux. Have you played with that one?
- [28:10]
B
I mean, I've played mostly with what. I always get the models wrong. Not Flux now, but like Flux Dev, but not necessarily the Redux.
- [28:19]
A
So let me show you real quick something. I'm.
- [28:21]
B
Nah, I mean, you can show whatever you want, I guess. You know, why you pull that up? You know, maybe talk about, you know, we. I mean, we have image, we have video, but maybe talk a little bit about how you can control characters. Because that's the thing where I think people get really hung up and it's by no means perfect, but let's say you. You're using Comfy for like, you know, creating a storyboard. You want the character to kind of look the same. Maybe talk a little bit about like, you know, pose control nets, like kind of depth, you know, to be able to control what a character looks like in different poses but in different scenes.
- [28:57]
A
If that makes sense to answer your question. So, I mean, for example, if it's like in the video world, right, One tool that is very powerful and that's kind of like the talk in town, I should say lately, is one vase. One vase is kind of like a Swiss army knife where you just fit it a video, a driving video and control at a depth map and a style frame. And then what it does, it just cooks all these things together. And you will have the character replaced by. Sorry, like the person in the video replaced by the character that you want from that image. And. And it's going to stick to that person correctly because it's using the different normals, open, posed, depth map and all that good stuff. So that's one way to do it. And I know another thing, like you said, it's really good to not be afraid with Comfy. I mean, with diffusion in general. That's another thing too, is to train multiple loras, because loras can be stacked, right? So let's just say you tell me, hey, Julian, I'd like you to make a scene in the restaurant where there is a big crowd in the back. And I want the people in the front to look like a specific person. And I want them to wear a very unique, average dress code that is very specific to a very specific year, you know, and it has to be a very specific color. So what you just told me, I'm like, okay. And the restaurant has to have a very unique. Sorry. The image in general has to have a very unique style. What you just told me is like, okay, I need four Loras at least crowd Lora a color like a lighting Lora, let's say, which can be done in post production. But of course. Right. But you might as well just try to do it in Comfy. I would do a Lora to train the face of that character and what he looks like and Lora on that dress code. And then I would just stack them all and then use like you talked about your wild card, you know, something like that for images. And another thing that's very powerful is a flux context.
- [31:07]
B
Right.
- [31:08]
A
Where you can actually, for example, just. That's such a crazy model where you just tell it what to change and then it does the thing. So check this out. Let's just say I generated one image that you're like, okay, I like this, but why is my guy standing up? He needs to be sitting down. I could try to reprom that, but I could also try to feed that image that is good right into context saying change nothing at all except make the guy sitting down on a chair.
- [31:41]
B
Really? I've never played with that one.
- [31:43]
A
Oh yeah, check it out. It's really, really, really good. And then what's crazy, you can train Loras and the Lora is for flux context are tasks.
- [31:56]
It's not a style or anything. Is you train a Lora on a task to do. So for example, you could trade a Lorac on sit people down, which you would feed an image and then you would end up without doing anything with people on that first image sat down on a chair. It's pretty incredible. Yeah. The one I have on my screen now, it's all red because it's the. Oh, that's another thing too right now that is a pain is because I. I'll tell you something really interesting and why I cannot wait to have like a T2 tower, like a. A powerful GPU, you know, like. Like the Pro, the Nvidia Pro. It's the fact that right now I. I have a regular Comfy that is for anything that is not one and all these more, you know, like hungry models and a union one and you name it. And so I had to try to install Sage Attention and that was such a pain on Windows. And I eventually worked with someone that gave me a script that installed Comfy with the right wheel for Triton and Sage Attention. And now I basically have two Comfy one for video like the one I was showing and the one without video. Guess what? When you have 96 gigs of VRAM and a powerful GPU like this, who cares about, you know, I'm just being silly here, but who cares about like sage attention? Just run the whole. You just run the model. Yeah, so I just would have one comfy installed because it would just run. But the exciting thing too is I'm getting a little bit sidetracked here, but it's actually a really great point I wanted to talk to you about is having 96 gig of vram is sometimes even too much. And it's like you have untapped potential. So what you can do is why not take 24 gigs and do simpler image generation renders and another 24 gigs for you name it, and the remaining 48 to do one generation. So you can split the cuda in like let's say 4, 3, 2, whatever and you can spin multiple comfies or. And to just basically do different things at the same time. And that is a very, very exciting, you know, like a mindset and possibility, I should say.
- [34:13]
B
I agree. And even if you're using an outside tool where you're generating and then running something through Topaz or whatever, it may be like, it's fascinating. I mean, you're right, it is untapped. It is untapped. I guess, you know, we've been chatting a while. It's been about 37 minutes. A couple of questions for you and we'll probably start winding it down. We'll definitely have you back on to do like a more in depth, like, you know, workflow and stuff. But I guess you, you know a lot more than I do. I'm surprised that I actually knew something you didn't know, which is shocking. But I mean, I'm shocked. But for someone the very first time, what are the, what are the three workflows that you think someone should start with? You know, assuming they don't have a super high GPU and stuff like that, like, what are the three that will like get them hooked up into Comfy?
- [35:01]
A
I mean to me like the. Any image generation workflows, I mean, obviously I wouldn't even try anything except going to browse templates and just start with basics right there. And image generation is the one you start with this one. It is literally if you have nothing and you start Comfy and you delete everything, this is the one that's going to be loaded. This right here is a must. Not only you must start with this, but you must delete the nodes and recreate the nodes yourself because that teaches you how it works. You load your checkpoint and from the checkpoint you're going to have your model, the clip for the prompt and then these conditions go in the ksampler. The ksampler takes the model, the VA to be able to decode encode and you need to set latent the concept of diffusion, that is what it's teaching you here. And once you're done diffusing with the seed, the different steps and so on and so forth, you decode and you get your image 100%. I would start with this and I would use it. I would struggle to download the molds like I said, you know, like everybody else at first. Then it would run. I would be like, okay, cool, I just made my first image. And then I would delete everything and do it again until that's just me. I would just learn it like this. After that for sure is I would put a lot of time on the various flux models because they're really good in my opinion. And definitely I would do a schnell or dev like these models to just generate images. And then I would have so the redux which we'll talk about next time, we'll leave this one for round two. I will keep it. Another thing that's really good too is because once you make images you're like, okay, cool, I think I nailed images. And then if you do flux context, you can just say, okay, I know how to modify images. Then people may want to do some video and you can either do. That's why I did it on purpose today because I knew it was going to be a round one is to show how anyone can start using video by using the built in workflows. Right. But you can also do that. But another thing that's really fun to learn is and that's what I use in the upscaler workflow is animatedif.
- [37:12]
You can literally have some fun and just build something that will generate a bunch of images back to back. You know, a batch, like a batch of images generate 200 images of a sunset and you're going to get 200 different sets because the seed is going to keep on changing. It's going to make no sense. So how do you keep it consistent? That's when you load a model like animatediff with a sliding window where it's going to generate like 16, then 16, then 16 with an overlap. So it's just going to be ever so slightly changing sunset. And if you do this, you understand, you know. Yeah. And that is just talking about like stuff that is now kind of older, but I think it's still really good to learn this because it's always like you always see ksumper, you know, in all these workflows, even in audio workflows you see ksamper.
- [38:02]
B
Have you had any success with audio in Comfy? I have not personally, but I was.
- [38:07]
A
Lucky enough to be at a conference where you know like Comfy himself was there and he was playing a lot with these. I forgot the name. I think it was Vasefun or forgot the name of Ace. Fun. I forgot the name. Anyways, there is an audio model and he found a way to modify the latent for the audio to make the output much higher quality and anyways there were tricks that he shared by the way, which I mean like I could share with you later on where the output was much better than if you would just use the workflow like this. Which by the way I'm pretty sure, pretty sure you can probably like. Yeah, see right there, Stable Audio or astep. That's the one I was talking about. And that one as is, you know, if you change, nothing was too not great like you said. But then you can literally fine tune little things and you get like a totally usable output.
- [38:56]
B
Yeah, dude, we could keep going but I don't want to. I don't want to blow it all, man. This is fantastic. Like here's what I want you to do is, you know, at the end of the episode what I like to do is have you take like 30 seconds to a minute, imagine that someone hasn't watched any of the episode up until this part and give them the 30 second, one minute kind of high level what they should be taking away about Comfy in this episode, what they should walk away with.
- [39:26]
A
We've Comfy. Just don't be afraid to, to challenge yourself and push through the difficulty at first. It's totally worth it. The more you use it, the easier it gets and the more fun it gets because you realize that you can do things that you could not do with simpler system. It's endless possibilities.
- [39:46]
The reward is greater than the effort you put in the suffering. It's totally awesome. And then before you know it, you'll be able to generate things like this.
- [39:57]
And then you get the emotion and the excitement and you're like, oh man, I made this. I created a workflow that let me do this and it's just awesome. It's worth it. And the community is here to help.
- [40:14]
B
I love it. So you're no longer the mid journeyman. You're the bird woman. I'm going to start calling you the bird Woman.
- [40:21]
A
I actually, yeah, changed now I just, I just go by my name. I just, I just go by Julian AI Art. So Julian, it's just I just go as myself now. I'm not. I've never been affiliated with Internet Journey, by the way, so.
- [40:32]
B
I know. Well, you, you, you weren't affiliated officially, but you were known for a man that could pull a lot out of it. So with that, hey, I really appreciate, love having you on, Julian. Tell everyone where they can find you on social media, whether it's LinkedIn or Instagram, and then we'll go ahead and close it out and we'll definitely get something on the books to go. And the two dot over or the, the 201 version of comfy.
- [40:53]
A
Sounds good. Yeah. So I would say like the, the easiest place to look for me because I'm the most active on Instagram. Julianai Art. So J U L I E N the French way. Julianai Art. Or it's easier to find all my, all my links on my website, which is Julianai Art. So again, J U L I E N A I Art. That's it.
- [41:13]
B
Awesome. Well, thank you, Julian, for coming on. Really appreciate it. And I think to summarize this episode, this is one of my favorite episodes because it's kind of where I got my start with, you know, different AI kind of toolkits and processes. And I think a lot of people start here. Comfy UI is so powerful. What I tell people, much like Julian, every day is before you can get started. Now you have to be able to take that first leap. And Comfy is kind of a great place to do it. And it's one of those tools that can be as simple or as complex as you, as you really want it to be, but you have to take that first step. And with that, this is reshaping workflows with Dell Pro Max and Nvidia RTX GPUs. Until next time, keep your Comfy workflows running locally and we'll see you on the next one.
- [42:11]
A
Do what YOU want. Do what you want.
- [42:13]
B
This podcast was produced in partnership with Amaze Media Labs.