← Reshaping Workflows with Dell Pro Precision and NVIDIA RTX PRO GPUs
Reshaping Workflows with Dell Pro Precision and NVIDIA RTX PRO GPUs
How Orbital Studios Transforms Production with AI Tools
00:31:40
Take a behind-the-scenes look at how AI and Dell Precision are unlocking new creative powers in filmmaking and virtual production.
Loading summary
Transcript66 lines
- [00:00]
A
Foreign.
- [00:05]
B
Welcome to Reshaping Workflows with Dell Pro Precision and Nvidia, where innovation meets real world impact in high performance computing.
- [00:20]
C
Welcome back to another riveting episode of Reshaping Workflows with Dell Pro Precision and Nvidia RTX GM GPUs. I'm your host, Logan Lawler. I'm going to apologize up front. We have a really good friend on this, this episode. We have Wilder from Orbital Studios. More about him in a second. But we've known each other for a while, so it's going to be a great episode. We're talking about AI, virtual production, and probably everything in between. Somehow I dragged the drag the cat in with Cindy here on my episode. But, Cindy, go ahead. For everyone that doesn't know you, which I'm sure everyone does, I'm kind of
- [00:55]
B
getting tired of carrying these episodes for you. Like, I understand you're having to call in like, the big dogs to, like, just be the talent on these podcasts, but. Hi, Cindy Olivo again, marketing for media and entertainment.
- [01:10]
C
I know you're fantastic, Cindy. You're legend in your own mind. So while we're actually here, Wilder, man, what's up? How are you? Tell everyone who you are, what you do, and we'll jump right into it.
- [01:20]
A
I'm Wilder Herms. I'm the head of AI and tech for Orbital Studios. Used to be the virtual production supervisor, and, you know, now I get to play with more toys. We got a big collection of toys over there, but now it's even more.
- [01:39]
B
So. Wilder, how has your role evolved? Like, you just said it yourself, you started off as like, head of vp, and now you're in this new AI role. And what I really love about Orbital is that you guys fully embraced AI, like out of the gate, where I think a lot of studios and people in the industry were a little nervous about it. So why was Orbital different?
- [02:01]
A
We're a little bit of a smaller team. And to AJ's credit, he really, like, runs the company just as a startup. Like, no matter how big Orbital gets, it's still going to act like a startup. And so we have all the advantages of that. Like, we can pivot quickly. And, you know, we as AI was evolving, he and I were both watching it and dabbling separately. And then when we realized, okay, we both have Claude accounts and we both have mid Journey accounts, and we both have OpenAI accounts and, like, paying for a lot of separate accounts here, we should probably talk about this. And then it just kind of evolved into, yeah, how does this affect the, the film world and how would we want it to affect the film world? And that kind of led into, okay, maybe if we get involved, if we actively participate, we can kind of shape that path. So that's what dragged us in in terms of my path coming from virtual production to AI stuff. So much of virtual production is already this weird blend of whatever works on the wall. You know, sometimes it's images, sometimes it's video, sometimes it's 3D. So we really have the full, just gamma of everything. And you know, it's, it's a continuation of the filmmakers like the Art of Illusion and just constantly, just getting over the hurdle of disbelief and making people believe that things are real. And you know, for a long time AI wasn't there. And you know, people made fun of all the images and the extra fingers and you know, they made fun of the video too. And it's like, oh, this is, look at Will Smith eating spaghetti and everything. And then it evolves into, wow, this is really good. I can't tell if this image is real or AI. And now getting there with video, like, is this real video? Is this AI video? You can't tell anymore. You know, cats with bazookas are amazing.
- [04:04]
C
You mentioned, you know, kind of diffusion models, right? Specifically video. Like, you're right. Like stable diffusion, 1.0, lots of cats, not real great, lots of extra fingers, like 27 digits. Can you name, since you've been so involved kind of in this, in this and testing different models. Like what was the first one? The first, first, let's say image diffusion model. And the first video diffusion model where you kind of were playing around with it and you're like, wow, like which one was the one where you're like, okay, this could be something I can roll with.
- [04:36]
A
Image wise. Mid Journey has always had the kind of advantage of like they built their audience. And so regardless of whatever they trained on, they were constantly getting the feedback of people liking and selecting their favorites. And so that really got, you know, they got the snowball effect going or the network effect or whatever going for Mid Journey. And to date they still kind of have a very unique, very creative, capable model. Probably more so than any of the other ones. Like there's a lot of other image models that are really good and can just knock realism out of the park or that specialize in certain things. But Mid Journey has this great creative range that a lot of them don't. And then video wise, I don't remember the first one, but I remember kind of the turning Point for me was like when I started to see controls built into it, you know, because as filmmakers that's like, that's what we want, right? We want to be able to direct the camera and all that. So as soon as you started being able to tell it what shot type, and I don't even remember the first one that did that, if it was,
- [05:50]
C
I think cling or flux were one of the, one of the first ones to let you do like camera angles and positioning and Runway as well.
- [05:58]
A
And now it's like, you know, a lot of the recent ones are starting to have the little 3D ball where like you can move it up, vertically or down and then you can move it left and right and like you're starting to get that level of fine tune, which is amazing. And I remember kind of like the, I guess the test or the bar from Mike, my co worker who was like at Epic and worked in Unreal for so long. His thing was like, I'm not going to use it until it can take a director's note. Like when you can tell it move the camera three feet to the left, then it's usable. And that was his bar. And I think we're starting to really actually be there.
- [06:43]
B
So Alder, just for people tuning in for the first time, explain to them, give them just a quick overview of virtual production and why it's the future of filmmaking.
- [06:54]
A
Virtual production is an evolution of obviously green screen and visual effects and a lot of techniques that have been used for a while. But it really good virtual production is using LED walls and using camera tracking and using a lot of sophisticated techniques to get your shot in camera, to basically get your visual effects on set on the day. So you can act, your actors can see what environment they're in. You can really line up your shot and there's a, a freedom of, you know, it's like getting to paint with the actual paint colors, getting to actually use the see what the final is going to be like. That's what virtual production really excels at.
- [07:43]
B
Really quickly though, like for just so people could put it into context. Like what show or movie did people really see? Like the capabilities of virtual production?
- [07:55]
A
Mandalorian Star Wars TV series was like the first one that really kind of took off where they used it heavily and you know, there were some like behind the scenes reasons for that, like directors and talent wanting to stay in LA and not go all over the world and stuff. But like it, it showed people like, oh, you can actually get amazing imagery through this method and there were a lot of hiccups and they, you know, they had a lot of hurdles to fight through for that first round because any new technology is really rough when you first try it. But now it's been really refined and you know, people come into our stage having heard a lot of life nightmare scenarios like, oh, virtual production is so hard, or oh, you can't do this, you can't do rain, you can't do direct sunlight. Like is all these different things people tell you you can't do, can't do foliage, whatever. And then we're like, no, here's foliage, foliage, there's rain, here's sunlight. Like you can, you can do it all. You know, it's much more straightforward now than it used to be. When I started, there was like very few guides and very few educational resources on it. And now it's pretty actual main line of study for people in film school.
- [09:11]
B
So you're saying that they didn't fly a crew to Mandalore? Got it.
- [09:15]
A
No, no passenger ship to Mandalore.
- [09:17]
C
For those less educated on virtual production, then I'm going to ask a follow up question about AI is the way to think about green screen, right? But in Orbital Studio, right, there's a big LED wall where you as an actor, me can go out. And it seems like I'm actually in the place, right? Like I'm not going to the beach, I'm not, whatever. And it's all projected via a whole host of software, cameras and all of that. But let's tie it into the AI piece, right? As head of AI of Orbital Studios, like what have you been doing with AI that fits into that, that pipeline, in that virtual production pipeline?
- [09:54]
A
So far, a lot of what we've actually been using on shoots and things is what a lot of people like to say. AI is a tool, right? So we've used it for upscaling, we've used it for inpainting, we've used it for replacing things. You know, sometimes we'll have like a video environment or, you know, selection of stills that we put together and make it look kind of 3D. And sometimes we just don't have enough. We need to extend or continue the environment over to the side. And you know, if you were to go through traditional VFX pipelines, like a lot of that just takes way too long. And that's one of the main advantages of AI is like you just have this crazy toolbox that's super fast. You can, you know, oh, we don't have another mountain over there, okay? Let's generate another mountain in the background. I mean, the, the traditional. Like when we do tours and demos, one of the things that people love to see from regular virtual production is just how malleable it is. We can move the sun in real time. You know, we could. We can pick up a mountain and put it on the other side of stage, which, you know, good luck doing that on any other set in the world. Right? But with AI now you can say, you know, I would like a building on top of that mountain. Oh, we don't have a 3D model of that building. Okay, let's just generate the building. It's an expanded toolbox. And just like anything, like if you don't, if you've never used the tools before, then you're going to need somebody who has, who can hold your hand and say, well, this is how you do it. Here's the buttons, here's your options. What do you want to do?
- [11:39]
C
Okay, so let's do an example, right? A lot of what you guys do is through Unreal in terms of the projection, right? Whether you have a still or an image or a video behind to replicate the behind the background or the foreground of the scene, right? So basically, I guess the question I'm trying to get at is how before kind of AI, how. How did that image or whatever was done in Unreal, how was that made and what was the time that it took versus now with AI being able to whip out, you know, a video and, or, you know, a background still to be able to show a director in real time. Like, hey, instead of having the forest, stupid analogy, forest with red oak trees. You've got it with evergreen trees.
- [12:20]
A
All the traditional pipelines, you would go through either a 2D pipeline, like compositing images or compositing different videos together, or you go through a 3D pipeline where you actually bring in 3D models. Models and swap out the trees, or you swap out whatever you want to change out. And both of those are time consuming. And both of those often require, like, multiple stages. You have to hunt down the assets that you want and then you have to bring them in and integrate them. And with Unreal, like, there's a massive amount of 3D assets available. You know, they have a great fab library. There's lots of libraries that you can pull from, but they don't always match perfect. And so there's art to like building out environments. And, you know, people get paid plenty of money to just do that stuff. And I actually rather like doing that. But again, the speed option with AI, now you can have an image instantly, right? You can have a video pretty quickly. One of the other things that we. One of the use cases that we've done a lot of is where someone will have a still of what they want. It's like, here's a still of a wheat field and it's perfect. But we've only found, you know, video clips that are not perfect. It's like, well, how about we just animate that still, throw it in the AI. Okay, now the weed's moving.
- [13:52]
C
Okay, so it makes sense, right? That makes total sense.
- [13:55]
B
Wilder, talk to us a little bit about the Orbital infrastructure. Like, what are you guys using? What about our new products are you most excited about? How has that increased your capabilities in the virtual production environment, but with the work that you're doing in AI? I want to talk about the hardware.
- [14:15]
A
It's funny, we put a lot of the resources from you guys towards running the walls initially, and I have like, over time peeled off more and more of them into the AI division to be the AI stack. What benefits both of those is the speed of doing things, and especially for the stage, the reliability of being able to run a bunch of computers all linked together at the same time. You know, because that you. You have the issue of it all being part of a chain. So you don't want one link in the chain going down. But the chain also has to move really fast for virtual production. Like, virtual production is dealing with milliseconds. So if something is off by too many milliseconds, then you get a desync on the wall and nobody wants that, you know, director will be like, well, how come this slice is different than that slice? And, you know, it's. It's not fun. So the systems that you guys make are just rock solid. And, you know, we've. We've put them through so many paces and oftentimes take them with us to, like, remote jobs. Because Orbital does a lot of remote work. We're one of the few stages that will go and actually just build a new wall somewhere at someone else's stage and bring our own set of computers and run them and pack them up and bring them right back afterwards. So the dependability of the hardware and the speed, I mean, another way of like saying how important the speed of the graphics card and the CPU is just that when I started virtual production, the systems that we were using, we were barely able to hit kind of the expected frame rates, like the minimum frame rate that you need to run is 24. And ideally, you want to be able to run at least 48 or 60, so you have some room for slow mo potentially. And that just wasn't happening on a lot of maps, a lot of environments. We're barely getting 24, and now it's a lot easier to be way above that, you know, thanks to you guys constantly shoving beastly GPUs into boxes that it's kind of surprising that they fit inside of.
- [16:44]
C
Yeah, it's kind of like Cindy's ego inside this episode. Shocking. All right, we're going to continue. So I know a lot of what you've done, Wilder. You know, you had a T2 kind of set up, and you kind of coded. And I mean, by trade, right, you're an artistic guy, but by trade, you're not, like, a software dev.
- [17:01]
A
Right.
- [17:02]
C
I would love to show if you can, and if not, we can just kind of talk through it, is that you kind of use, you know, different. You kind of vibe coded your way into a standalone application that you guys use at Orbital as part of the production pipeline for AI. So I would love for you, if you can, show it, if not, but, like, be able to talk through kind of the process.
- [17:21]
A
Right.
- [17:21]
C
And, like, how you were able to do that.
- [17:23]
A
I was able to not just, like, kind of vibe code an app, but we've made, like, six of them now, and we're. We're weaving them together into, like, a singular kind of architecture, mainly because from the, like, discussions that AJ and I were having about, like, the direction of filmmaking and where it's going, there's always kind of, like, this siloing of, like, the. The money people use certain tools, and the creative people use certain tools, and then the scheduling and organizing people use certain tools. And, you know, physical production uses one set, and digital production uses a different set. And, you know, with. With AI, we can start linking all of those things. And the. The things that web development and. And, like, software development is really good at is, like, databases and connecting things. And as soon as you have that backbone, then you can start linking in all of the other AI superpowers, like image generation and video generation, all the other tools. But to build that stuff, that's where a beastly system comes in. I'll show you the kind of link of a bunch of the apps that we have. They're all running on separate systems now, but I built all of them inside of the T2 tower, and that was like. I don't know. I don't know if it was like, A super luxury. But it felt like it because I felt like I couldn't do it on any other system to be able to like load in different models all at the same time and actually test them against each other, live in the same app. Like, how's this image model better than this one? And can I connect these to a language model at the same time? So there's a lot of that kind of testing that I just wouldn't have been able to do if I had to constantly switch systems.
- [19:25]
C
Right. Well, I think that's where the Pro Max and the GPU shines. Right? I mean, 96 gigs is a lot. You know, before Nvidia moved from the ADA to black hole generation, not saying it wouldn't have been possible, right. But the amount. I mean, you could go to the cloud to do that. I mean, Orbital isn't going to go put in a $3 million data center, let's be honest. Or you go to the cloud and then once AJ or your finance person, whoever that is at Orbital, sees the bill, they're going to be like, wilder, you're shut down. Because all that tinkering is going to cost money. Right? And I think that that's the lesson to be learned here, is that, you know, having the right device not only allows you to do different deployments and do different things, but it allows you to like, optimize and actually do new and cool stuff that can change your workflows, change your company. But unless you're willing to upfront that in the cloud, then it's better just to invest in a device. At the end of the day, I
- [20:14]
A
kind of feel like for a stage or any company that's doing a lot of this, you almost eventually want both. You want several systems that you can kind of distribute things to later or down the line when you're doing inference type stuff or generation type stuff. But while you're building, it's. It's really ideal to have a singular, very powerful system that you can just throw whatever you want at and, and test everything. That's been a great luxury working on it, and it let me do some cool stuff that I just wouldn't have been able to try. A few of those things I can't show today, but I'll show some of the stuff that we, we can.
- [20:59]
B
So, Wilder, just so everyone knows, like, how big of a deal Orbital is, like, what are you guys working on? What can you talk about? Or what have you worked that you can talk about?
- [21:08]
A
I'm actually the worst person to ask that Because I constantly forget all this stuff that we did. We were just having lunch the other day, and we put on a playlist of, like, random music videos that have shot at Orbital. And they were all fun and interesting, and we both, everybody was like, I forgot about that. I forgot about that one. I forgot about that one. But we've, we've done several TV series, too. We worked on Snowfall. We worked on miniseries with Danny Trejo and another one with the History Channel and Tom Hanks blinking on the name of both of those. But we did History's Greatest Heist with Pierce Brosnan.
- [21:49]
B
You guys do a lot of commercials, too, don't you?
- [21:52]
A
We do a lot of commercials. We've done several things with features as well, but a lot of these have longer timelines where we'll, we'll shoot something with someone and it doesn't come out for a year or year and a half or something and can't talk about it until it's out. And Orbital is one of the most versatile stages because we're open to all of those. Like, we don't really limit it, and we're open to kind of not remote work, but, like, mobile wall setup. So, like, we went to Canada, you know, a couple months ago and did a setup at a stage out there. And we're going to Chicago to Cinespace in a few weeks.
- [22:38]
B
And you're coming to Nab, going wherever
- [22:41]
A
Dell tells us to go. But yeah, we, we have some cool stuff planned for that one.
- [22:46]
B
You're also expanding, too. I mean, I, I, I saw something about Europe. I mean, you're going to be in France for Cannes, so it sounds like there's just a lot going on.
- [22:56]
A
Yeah, it never stops at Orbital for such a small team. We go a lot of places and
- [23:02]
C
kind of, because we have the video version of this and the YouTube version or in the podcast version of this is audio only. So kind of narrate just a little bit as you're talking of what you're seeing and doing.
- [23:12]
A
What we're seeing is a fun little kind of space environment with what I kind of refer to as like the Infinity Stones. But each of these is a different little app that helps in some part of our process for making films, running virtual production, and is the beginning of our AI tool bag. So, like, a couple of these, like our Codex 1 is just a internal wiki that I built where we just have all of our information about the different stages and LED panels and all the different tools that we use. And then I'll come back To Orion. That's the kind of fun 1. Space board. This is just a doodling kind of mood board thing that all artists use, something like this. And we wanted to build our own so that we could tie these to the database that all of our projects are going into. You know, people use a lot of separate tools. Like there's Miro for mood boards and a bunch of separate ones. But being able to connect one directly to a film that you're working on is one of the biggest pluses there. And I think a lot of people are going to start building their own tools. And we just decided, let's build everything. Let's build all the tools that we want to use. So we also built inventory tracker, where, you know, I can throw in a item, we can put a QR code to it and scan it when someone checks it out and takes it to a shoot. Yeah. So then that comes to this one Nebula, which is like, this is the beginning step of our visual AI tool, where you can do all of the stuff that you would expect to be able to do, but streamlined into a filmmaker workflow. So it's image generation. We have image upscaling and image editing in here, but all based around things, tasks and workflows that creative filmmaker would do, like designing a location or designing a character, composing a scene or defining a style. And all of these then lead into what you can do with it later for putting them together into storyboards. And if we click on one of these, you go to start anew. You can give it a name, call it Logan, and we'll label you as male. And I don't know if you want to be young adult or middle age.
- [26:21]
C
Yeah, definitely middle age. Aging. Millennial.
- [26:24]
A
Okay. Okay. Give you European, unless there's a. Some extra in there I don't know about. Average, athletic, stocky.
- [26:34]
C
I don't know.
- [26:35]
A
Where you at?
- [26:35]
C
A little stocky, little husky. You know, I like to say a thick, chubby, you know, whatever.
- [26:42]
A
We'll call you a detective, and we'll put you. If I can spell. And we'll put you in the 1950s.
- [26:53]
C
Let's upload an image. Let me. Well, we didn't really plan ahead for this.
- [26:57]
A
We can do a video version of it later. Okay. Send it to you. Okay.
- [27:02]
C
Yeah, we can plug that in later.
- [27:04]
A
So I. I totally get it offline right now, actually. It usually goes faster than this if it's online. Yeah, well, that's not quite stocky, but okay.
- [27:17]
C
I mean, honestly. Yeah, right there. Beefcake? Probably not. Yeah, I think that's good. Kind of a detective. Yeah.
- [27:27]
A
But yeah, then we can regenerate images or edit them or say, okay, you know, I want a moderately different variation of this and I would like to see this from let's say three quarter left we have a button. We could generate all the angles, but these, these controls will expand and stuff. And it's just all the general like image generation tools that you would expect where you can do variations.
- [28:05]
B
Nothing like Logan. It kind of looks like Jason Bateman kind of.
- [28:09]
C
I was going to say like Jason Bateman or like maybe kind of like a Ewan McGregor a little bit.
- [28:15]
B
Yeah, maybe like a Jason Bateman Ewan McGregor mashup. This is a really cool Wilder, but I think we're, we're getting close to time and I think we can have fun with this next time we're in the studio and you've got like actual pictures of Logan. I've got some ideas.
- [28:28]
C
Imagine someone just started watching this episode and you want to give them the 30 second elevator pitch on L or on AI. Orbital virtual production. What is what you'd want them to walk away remembering from this episode?
- [28:43]
A
Virtual Production basically changed filmmaking when it hit the scene, it was a new avenue and there a whole new way of thinking about what you shoot and how you can shoot it. And there were just things that weren't possible before and timelines that weren't possible before. And now with AI, it expands that even further and we're not fully there yet even, but just with the tools that are out today and some of the ones that Orbital's building on these Dell workstations, you're able to open up your creative options for filmmaking and say things that would have been really expensive to do before are not anymore. Some things that just wouldn't be possible in a certain time frame. Now they are. And I think the more that people like dip their toes into the AI world and all of the tools that are available, the more that it moves away from being some scary thing to just, oh, these are tools, Cool, let's use them. And Orbitals pretty deep in that path. We're using everything and testing everything. And because of this T2 workstation, I'm able to literally test dozens of models in the same day and throw a bunch of them head to head and say, okay, this one's definitely the best. And then we can incorporate it into our workflow.
- [30:27]
B
Let everyone know you'll be at NAB with us and what you're planning for that. Give us a little teaser. I'm really excited about that. So make sure everyone comes by the Dell booth. If you're heading into NAB this year,
- [30:40]
A
I think everyone has to come by the Dell booth. We're going to have something that's a little bit interactive and a little bit intergalactic.
- [30:50]
C
Interactive and intergalactic. And fantastic.
- [30:54]
A
And fantastic.
- [30:55]
C
Yep. So, Wilder, really appreciate the time. Cindy. Always appreciate you coming on and really up leveling the episode. So with that, this is reshaping workflows with Dell Pro Precision and Nvidia RTX Pro GPUs signing off. And we'll see you on the next one.
- [31:19]
A
Do what you want. Do what you want. Do it. This podcast was produced in partnership with Amaze Media Labs.