Loading summary
Transcript50 lines
- [00:03]
Samantha Murphy
Lets go back some years, say 2016 or 2017. Almost everyone is on their phones. In one scenario, teenagers are playing around with Snapchat filters, sending funny photos to friends and recording goofy videos. They're picking a puppy filter, which adds a pair of dog ears and a wagging tongue to their faces. In another scenario, a family is running around with their mobile phones, chasing directions to random places like a bush or a park statue. They're trying to catch a small, bright orange Neo dinosaur called Charmander. For points. Come on, Charmander. Welcome to the group. Char. Char. Or they're trying to catch a butterfly called Butterfree.
- [00:48]
Rafael Pajes
You got a Pokemon and a new friend.
- [00:51]
Samantha Murphy
Yeah. None of these characters were in the real physical world, but they moved and interacted with us so realistically, it was like we could almost touch them. If you recall, this was the global phenomenon of the Pokemon Go mobile game.
- [01:07]
Dr. Randall Hill
We're back now with something that may explain any odd behavior you've seen on the streets recently. The smash hit mobile game app Pokemon Go has only been out for a few days, but it's already got millions following their smartphones to the most random places.
- [01:25]
Samantha Murphy
With both of these apps, the digital world seemed to coexist with us. They supplemented our understanding of our physical space. They created experiences, some memorable and others playful. So what did these fun concepts tell us about the future of our relationship with technology? Or about how they can create stories about ourselves or culture and our society? As more of these applications become increasingly popular, we're seeing how some software applications can create realistic digital experiences, like a historical battle reenactment in a park, or having a museum's art collection come to life through a phone. All of this is spatial storytelling, and it's part of a bigger trend that's shaping how we interact with technology. I'm senior tech correspondent Samantha Murphy. Kelly, this is the next innovation.
- [02:23]
Dr. Randall Hill
Today we're going to take a journey through new and exciting worlds, the worlds of virtual reality and augmented reality.
- [02:34]
Ben Lang
So essentially, augmented reality is using computers.
- [02:36]
Dr. Randall Hill
To digitally enhance the real world. And we were talking to one software developer who said that as we look.
- [02:41]
Ben Lang
Back, looking down at our phones as we're walking down the street will seem like the Dark Ages immersive technologies.
- [02:48]
Rafael Pajes
So AR VR are past the hype. Now they are productively being used in many industries outside of just gaming and entertainment.
- [02:56]
Samantha Murphy
In a previous episode, we talked about how new forms of technologies like LED screens and giant animatronic robots were making use of our physical space to tell meaningful stories. That episode was an introduction into the Evolving world of mixed and extended reality. In this episode, I want to explore how a similar form of technology, augmented and virtual reality, lives within the umbrella of mixed reality and how its unique capabilities bring in the user in a much different way.
- [03:29]
Michael Guerin
I give a lot of talks and lectures and different things and I often start by asking how many people have used augmented reality before? And almost everybody says no. And then I how many people have used a lens or a filter on Snapchat or Instagram? And almost everybody has done that. So that's all augmented reality technology. They just don't call it augmented reality, they call it a lens or a filter. So people are using this type of thing. Just it's when it's really simple and seamless and the jargon of augmented reality isn't often used.
- [04:00]
Samantha Murphy
A few years ago, Michael Guerin was leading a group of city planners and urban designers in Dublin. It was a project that focused on how tourists and citizens could better interact with the city. They worked with tech companies, universities, research centres and city councils.
- [04:16]
Michael Guerin
The goal was to bring more technology to Dublin, trial and test it in a confined space, and then, if it made sense, roll it out to the rest of the city or the country. So I ended up working with over 50 different startups, everything from 5G technology, AR VR, e scooters, driverless cars, all this type of thing. And I felt like you could create really good experiences with this technology. And then when I looked at what was in the market, I was surprised that nobody was really doing that to the level that I thought was possible. So I decided to leave my job and do it myself.
- [04:47]
Samantha Murphy
In 2021, he launched Impazar, a company dedicated to creating location based immersive experiences.
- [04:54]
Michael Guerin
So immersive can mean a lot of things. It tends to mean things that are all around you, using visuals and sound and. But for us, we're focused on one particular niche, which is location based augmented reality experiences on your mobile phone. So we can build experiences on headsets and that type of thing, but all of our experiences are accessed through your own mobile phone. You go to a location, you hold up your phone and then the scene appears around you. And it's a bit like being in a. If you've ever been to one of those plays in a theater where you're like part of the show and it's kind of like that, so the characters are all around you, the props are all around you, you hit play and the story plays out around you. So the goal is that you can walk around the experience and the real World is there too.
- [05:34]
Samantha Murphy
Infazar takes specific locations and develops stories based on the precise mapping of a place. So instead of walking into a world with a headset, you lean into your physical space with added layers of information. Kind of like the way we interacted with our favorite Pokemon Go characters on our phone. As CEO, Michael's main objective with Impazar is to create meaningful and accessible experiences so visitors can really connect with the area. The stories are usually authentic and true to its location. Last year, the global accounting firm Grant Thornton decided to transition from single use to reusable coffee cups. So for World Earth Week, they worked with Impazar to promote their sustainability mission. They created an experience to visualize over 100,000 coffee cups saved from landfill each year.
- [06:27]
Michael Guerin
We created a 90 second experience which is only activated within the lobby of Grant Thornton. You hold up your phone and it brings you on a journey of the impact that you and the company has made in terms of waste, basically. So if you have one coffee per week, a lot of people have one per day, but if you have just one per week, that is probably 50 coffee cups per year. There is 2,000 staff in Grant Thornton in Dublin, so that's 100,000 coffee cups saved from landfill every year. That's if everybody has one coffee per week. So it's a big number. But when you see 100,000 coffee cups, you really understand the difference. So the experience starts with 50 coffee cups falling from the ceiling, landing on the bin and around it. Then a seagull flies in and it shows you the impact of marine life with the waste and littering. And then the experience moves on and the whole lobby is flooded with 100,000 coffee cups. So you're like standing like waist deep in coffee cups. So it's a really simple experience, but the feedback at the end of it was phenomenal.
- [07:26]
Samantha Murphy
Infazar has now delivered over 130 experiences in areas such as employee and visitor engagement and sustainability for companies such as Salesforce, Google and others. If you could create something special for virtually anyone, what kind of stories would you tell and how would they be different?
- [07:46]
Dr. Randall Hill
In many ways, we feel like we're pushing the research forward. And the way this links back to XR and the different flavors of XR, AR, VR, MXR, is that a lot of it that we focus on is first of all how to create the content for that environment, because I think that's what's sorely needed, but also what those technologies are useful for and how they can be used in a way that is most appealing and effective.
- [08:18]
Samantha Murphy
Dr. Randall Hill is the Executive Director of the Institute for Creative Technologies at the University of Southern California. For nearly 20 years, he's been supervising several teams of researchers, artists, engineers and other experts on different mixed reality technologies.
- [08:36]
Dr. Randall Hill
What we've really tried to do at the Institute is focus on really positive uses of the technology and from training applications like how to operate a watercraft in a littoral zone, or how to team with AI agents in a scenario, or with or share a space where you have a distributed command and control type system and you want to be able to share a map or share a situational awareness with other people over that distributed space.
- [09:08]
Samantha Murphy
Dr. Hill first got into mixed reality when he worked at NASA in the early 90s. He used early forms of the tech for things like steering a Mars rover. But when he joined the Institute in the early 2000s, he noticed the potential of this advancing technology. Earlier in his career, Dr. Hill worked as a military officer. Some of his first projects at the Institute focused on helping people work through post traumatic stress.
- [09:35]
Dr. Randall Hill
In the early 2000s, we started to explore the use of VR for things like behavioral health. In particular, we had a clinical psychologist here created an application called bravemind for treating post traumatic stress in VR. One of the very first projects we did@ the ICT wasn't specifically a VR project, but it certainly had all the components that would be useful for telling stories in VR. We call it the mission rehearsal exercise because we're funded by the DoD. We had an army type scenario and it was based on some of the things they were doing. And in this scenario, the participant plays the role of a lieutenant, a young lieutenant who's basically put in the pressure cooker. He goes into a room. We had a 8 1/2 foot tall screen, wraparound screen that's 30ft wide, and a 10.2 audio system. And when he walks in, he or she walks in. They're faced with a dilemma. The dilemma is that one of their soldiers who's driving a vehicle, has hit a child. And the child you see lying in the street, the mother's weeping, there's a crowd that's forming, the media is showing up, there's a helicopter flying overhead. And basically we created these characters, virtual humans that you could have a conversation with. And the participant had to kind of reason through what to do next, how to deal with all these pressures at once. So it's very interactive. It involved early stage AI, virtual humans, and all of the elements of immersion that you would expect in a VR system. That's the type of storytelling. I think that I would love to see more of as we use VR XR types of applications.
- [11:28]
Samantha Murphy
As a leader of a Creative Studio, Dr. Hill's main mission is to build creative forms of mixed reality, including augmented reality, in productive ways. He advises his team of researchers to produce content that can have a positive impact on users.
- [11:44]
Ben Lang
It's kind of been a continuous state of experimentation with different developers trying different things for all these different use cases and really having to figure out what sticks and what doesn't. And what we often find is the more specific the use case and the vision behind it, like let's say a piece of AR software that solves a very specific problem. Those tend to be the things that stick and find value and continue to iterate and build.
- [12:14]
Samantha Murphy
That's Ben Lang, an analyst in the AR VR industry for almost 15 years. He's also the executive editor and co founder of a leading mixed reality industry news site called road to VR. And in the late 80s, mixed reality had generated some media attention. But by the mid-90s it had fizzled out. So in 2011, as a budding journalist, Ben found himself writing about how companies like Apple were exploring different forms of virtual reality.
- [12:43]
Ben Lang
It was just me and my curiosity and my journalism skills at the time wanting to figure out what's going on, where things are headed and what could VR ultimately become in the future. I was sort of just, you know, looking at the various scraps and things of the prior era. You had people who were hobbyists just sticking, you know, accelerometers onto some like off the shelf, like 3D personal 3D glasses you could buy and hacking together like some immersive experiences by, you know, combining a head worn display with head tracking to make it feel like when you look around in virtual reality, you're actually in there. So really early on it was just like hackers and there was this, you know, nascent community. It wasn't even, you wouldn't even call it an industry for many years yet at that point it was just a community of sort of hobbyist people seeing what they could do with what was out there and having sort of an idea in their head of like, hey, in the future this could be so cool.
- [13:42]
Samantha Murphy
Ben was part of a group of AR VR enthusiasts, hackers, hobbyists, entrepreneurs. But over the past decade or so, his reporting has tracked the fast evolution of virtual and augmented reality. He's followed how lighter hardware, lightning fast processors and ultra precise sensors have turned XR from a 1 off experiment into something you can hold in your Hand. What used to take a whole lab now fits into a device you can slip into your pocket. This year, Michael and the Impazaar team pushed this one step further as they launch l', Raio, a tool that enables anyone to create their own spatial storytelling experience in minutes. It's been described as the Canva for ar. So if your phone contains a new portal for spatial storytelling, could anyone become a spatial storyteller?
- [14:34]
Rafael Pajes
So we ended up in a little bit of everything, providing the tech for creators and developers that wanted to integrate it into their own creations and their own projects, but also building a couple of products that would allow us to evangelize a little bit more the market. Also get the technology closer to the everyday user. So my name is Rafael Pajes. As you can probably guess by my accent and my name, I'm not Irish, I'm from Spain. But I am the co founder and CEO of an Irish company called Volograms. So what we do is that we work with this very cool technology called volumetric video volumetric capture, which is a way of recording people in 3D.
- [15:09]
Samantha Murphy
Think Pokemon Go characters, like the kind I described earlier. Their rendering on your phone screen appear so realistic, almost as if you could pick one up and carry it. They're powered by AI models and facial tracking capabilities. They analyze your physical features, like your face or your entire body, and animate it by using photogrammetry, a form of extracting measurements from a scene. This form of volumetric video capture is popular among companies like Meta or Apple. They take your main physical attributes and convert them into fun animated avatars. They can speak and move around. Volograms avatars are different. They're you, your body.
- [15:52]
Rafael Pajes
We've trained a deep learning model. It's basically with all the photo, all the 3D models that we generated in the studio, we, we trained a model that is now able to guess how the shape of a person is by looking at that person from one single photo. So take a photo of you, put it through the pipeline, and the pipeline is going to give me your back, how you look from behind and stuff like that. We always wanted to be able to provide a technology that anyone could use. So we wanted to have a solution that anyone with a phone could do. So just anyone, whatever they are in the world, can access the technology without the need of having to travel to Dublin, London, LA or whatever the next volumetric capture studio is.
- [16:32]
Samantha Murphy
So imagine creating a volumetric rendering of yourself dancing and then sending it to one of your grandparents on their birthday, or sending one of your friends hugging each other and then using it in the corner of a video. Volograms wants users to embrace how different forms of augmented reality can go beyond animated avatars or silly holograms. When Volograms launched seven years ago, the 3D space was largely unexplored. Aside from mainstream 3D experiences in movies, there wasn't much of a demand or even an interest for cool applications like Volograms. But now it's a bit different.
- [17:08]
Rafael Pajes
If I talk about the words being used the most, I'm pretty optimistic about the sports space. I think that sometimes we're not really aware of all the technology that is behind the broadcasting and the content that is being showed. So you probably have seen if you follow any sports like this type of 360 replays or stop motion things that they seem to be doing sometimes. So this is actually powered by technology similar to ours. There are plenty of stadiums right now that have installed many cameras, so we can take that footage and now start creating 3D content with it.
- [17:43]
Samantha Murphy
FollowGrams has worked with Ireland's Fox Sports division for 3D content of sports games. It was an expansion of something that really kicked off a few years ago.
- [17:54]
Dr. Randall Hill
Dallas Mavericks continue to be a on the cutting edge in terms of bringing the best television experience in the NBA partnered with three dimensional video to give the viewer a three dimensional look at the play on the court. Specially placed cameras around the arena show unprecedented looks never seen before in an NBA game.
- [18:15]
Samantha Murphy
In 2022, the NBA launched its first broadcast rendered with volumetric video. It was a spring game between the Brooklyn Nets and the Dallas Mavericks. It marked a big transition in the world of sports broadcasting, not just for the technology, but for the fans as well. It seemingly changed the way viewers interacted and watched sports. You could now follow a player from virtually any angle or experience a shot from the three point line. Imagine yourself as a fly in the middle of all the action. The launch was part of an ESPN effort called NBA Courtview. More than 100 cameras were placed around the court, capturing and analyzing player movement. The data collected would then help generate multiple dimensions of the same image. Sportscasters had previously used volumetric video capture to explain player movement using 360 replays and stop motion. But ESPN's broadcast hinted at a future in which fans could experience the spirit and energy of a live game without actually being there. Now sports leagues like the NBA and Major League Baseball are exploring virtual stadiums and fan zones. But even though the future of spatial storytelling is promising, there are still challenges.
- [19:32]
Dr. Randall Hill
Ahead, the uses for this technology are really so many of them are unexplored at this point. And the barrier to doing this, one barrier is our own imagination, but also the ability to create the content at scale to achieve those goals. And so one of the challenges, the cost of creating content. If you go to any big movie, you know, if you sit through the credits at the end, you see the hundreds or thousands of people who had to contribute to the special effects for that. And so for us, one of the challenges is how can we help bring the cost down for creating content.
- [20:09]
Samantha Murphy
When I spoke to Dr. Hill about the potential for AR and VR capabilities, he emphasize that the industry is still quite young. Despite much of the media attention. Spatial storytelling isn't getting enough funding, not just because of how expensive some of the projects can be, but also because it requires an established market of consumers. Some AR VR applications require between $100,000 and $300,000 or more. But if companies don't have a solid base of users, there's no guarantee it could get picked up. When Rapha launched Volograms, they struggled to get funding because they hadn't yet found a market of consumers. They wanted to pursue growing their brand, but found that sharing their technology with other agencies also helped.
- [20:54]
Rafael Pajes
So we wanted to have something that was as simple as I take a picture and here you go. Similar to the, let's say, the creative process that you would have on social media and stuff like that, that basically has completely disrupted how we consume the media. So we thought if we simplify the process for content creation for vrar, we can change the whole industry. But of course, VRAR is a very tough market. I don't even want to mention the word Metaverse. That seems to be doomed now. So when we started, it was still very early days of the 3D space. So this meant that we had to work on the whole pipeline, from capturing to doing the reconstruction, to the distribution, to the integration. So for a small company, it was very complicated to handle the whole thing. So we tried to focus on one thing or the other. But as the field itself was quite nascent, we found it very complicated to sell the technology without selling, like, the final product. So we were always like, challenged, like, what should we do?
- [21:52]
Samantha Murphy
Infazar had a similar kind of experience.
- [21:55]
Michael Guerin
I mean, in some ways it was a little bit mad because most people hadn't seen or heard of the technology. So it was definitely trying to convince early adopters to work with us. So the city of Aveiro in Portugal was one of them. Spike island was another. They were both forward thinking individuals that wanted to try something new for their visitor experience. So it was a much harder sell at that time, probably for a couple of reasons. The technology was more in its infancy. I didn't have any credibility as such in the market, you know, and we hadn't delivered anything. Now we've delivered obviously over 100 experiences. It makes it a whole lot easier whenever we're having conversations with prospective clients. But in the early days, it was very much trying to convince people to trust me that I could deliver an experience which is better than what they had.
- [22:42]
Samantha Murphy
The unique successes of Volograms and Infazar have proven that the excitement behind spatial storytelling is generating not just interest, but also activity. In 2024, the hologram market, which includes products like Vologram's 3D image capture and Infosar's interactive experience, was valued at nearly $4.5 billion. Growing interest from the gaming and sports community have encouraged innovators to continue creating new applications. Still, experts believe it's not just about the kinds of projects you create, but how you apply them.
- [23:19]
Dr. Randall Hill
I think it takes a will to invest in something that has an educational use and not just purely entertainment. But at the same time, I think the education and training sectors should be looking at some of the tools that the entertainment industry or the game industry has constructed. And so there's got to be a will to create that kind of content.
- [23:43]
Samantha Murphy
We're now noticing how different forms of augmented reality and virtual reality are helping doctors, teachers and manufacturers better understand how to make their jobs more effective. But with projects like Infazar and Volograms, the excitement may lie in seeing how fun applications can encourage creativity, memories and meaningful stories.
- [24:04]
Ben Lang
There's this very vast new world of possibilities that's starting to open up as we move toward improved augmented reality functionality. So we're just starting to get a taste of this augmented reality as sort of a way to enhance everyday life.
- [24:20]
Samantha Murphy
The world of spatial storytelling is quickly becoming a way for users to step inside stories, literally and figuratively. The work happening now is setting the stage for the next era of spatial storytelling, showing how the industry is moving from experimentation to meaningful everyday impact.
- [24:40]
Michael Guerin
And then the last thing about Invisao. So we have been creating all of these experiences all around the world. I think we've a good grasp of how to create a high quality experience that goes beyond kind of gimmick and is more impactful and meaningful.
- [24:54]
Rafael Pajes
I think we're going to see more and more ways to create content just by typing or just by providing, like, a photo as a reference, as we do right now. And this is one of the, let's say, biggest strengths from the technology point of view.
- [25:07]
Samantha Murphy
Thanks for listening to the next innovation. This series was produced by Situation Room Studios and powered by Enterprise Ireland. Investing in the next wave of innovation. Our executive producer is Christine Barata and our senior producer is Sharon Barrero. Emily Beaman and Leila Sharrawi are the associate producers. Additional production assistance by Global Situation Room. A special thanks to Michael Guerin, Dr. Randall Hill, Ben Lang and Rafa Pages. I'm your host, Samantha Murphy. Kelly, until next time.