← AI to ROI
AI to ROI
The Power of Eye Tracking for the Enterprise - with Adam Gross, Co-Founder & CEO of HarmonEyes
00:33:30
Loading summary
Transcript35 lines
- [00:00]
A
Foreign.
- [00:08]
B
Welcome to today's episode of the AI to ROI podcast. Today I am joined by Adam Gross, co founder and CEO of Harmonize. Today, Adam and I will be discussing four different topics. First, what is eye tracking and why is it valuable to the enterprise? Second, how AI is applied to the science of eye tracking. Third, a couple different enterprise use cases. And fourth, the tangible benefits and ROI of corporate eye tracking with the use of AI. So, Adam, can you please take a moment to give a brief overview of your journey to becoming a guest here on the AI to ROI podcast?
- [00:54]
A
Yeah, first of all, nice to see you, Ray, and thanks for having me on the podcast. I look forward to our conversation. So, to give you a little bit of background, I started the company in 2012 with my co founder, Dr. Melissa Huntfelve, who is a renowned eye tracking scientist. And really, the story begins when you think about eye tracking is the technology behind recording and capturing eye movement behavior. So a little different than maybe an image of your retina or scanning your iris. This is all about how your eyes move, the patterns that they move in, the signatures, even things like pupillometry, etc. And so there are certain signatures of your eye movement behavior that correlate to different aspects of your health, your vision, your performance, your cognition. And really, when we started the company, there had been kind of decades of research that drew these correlations, yet the technology hadn't really caught up. And many of the use cases were kind of utilitarian at the time. And so our goal was to take eye tracking, the idea of capturing and understanding eye movement behavior, to draw those correlations in software and science and turn those into really useful and practical and actionable insights and applications. We started that 12 years ago, and we've developed dozens of eye tracking software products over the years, and we've procured the largest validated eye tracking database in the world at this point, over 15 million unique records. And so now we're kind of, now we're entering into the world of AI and using that data to train models.
- [02:56]
B
Okay, well, this is exciting to me because as we mentioned when I was first introduced to you and we discussed the episode, my daughter about 15 years ago had the opportunity to benefit from using eye tracking. So many. Maybe we start, Adam, with more traditional individual consumer use cases and why it's valuable, and then we'll get into more. Some enterprise use cases.
- [03:21]
A
Yeah. So one of the really wonderful aspects of eye movement behavior, and therefore eye tracking, is that the characteristics that make it such a powerful sensor is the fact that it's objective. Right. So when your daughter was doing some of these eye movement testing 15 years ago, it's hard to fake that. Right? It's hard to fake a micro eye movement. And so the data that you get is objective, it is passive and it's quantifiable. And so some of the early use cases around eye tracking have been for more control purposes. So people who have conditions that don't allow them to communicate, you can communicate through your eyes. Right. If we understand the position of your eyes through valuable software. And that use case still exists today. And that kind of migrated to other use cases around market research. And where we took this with our parent company, Right Eye, was delivering valuable medical and performance testing around eye movements. So understanding vision skills if you're an athlete, but also understanding a visual impairment if you have a traumatic brain injury or a concussion or kind of detecting previously unidentified vision issues that are difficult to identify with the naked eye. And so those are some of the kind of use cases that have existed. We've been a leader in the medical device space over the last decade. And so while we've really penetrated the vision care and the neurology community on the clinical side, we've also had a really big impact on the performance side through our relationships with elite military, through Special Operations Command of United States Military and also dozens of professional sports teams and partnerships with Major League Baseball and the NBA, et cetera, over the years. And so through that exercise or through that business, we've been able to procure this valuable data set and now we're able to deliver when we combine it with AI. And I don't know we'll get into that, but we're able to leverage that and train models that aren't necessarily assessment based but that are real time. And that's really where the AI helps us understand a person in real time and even predict how they're going to change in, in the future.
- [06:01]
B
Well, before we get into, because I love the NBA and MLB and the military use cases, but you know, as a society you can't go more than like five minutes watching TV or talking to someone without the words AI or the letters AI coming up. And there's many different types of AI. You know, there's a generative AI that everyone's using today with ChatGPT. But can you talk a little bit about maybe that you mentioned the evolution of the technology, when and how you started using AI and the type of it and how that has evolved over the last couple years?
- [06:36]
A
Yeah. So time timing wise About a couple years ago, we started to recognize a few things changing in our market. So to understand eye tracking, I'm going to back up a little bit real quick. Eye tracking, the technology of capturing eye movement behavior is mostly camera based. And so for, you know, for years and decades, frankly, those cameras are, you know, were very big and bulky and expensive. And even with our own medical device, it was, we had to outsource that hardware manufacturing to another company. We had to charge a lot of money. So it becomes a limiting factor when you're trying to really scale markets, when you've got to sell an expensive piece of equipment in order to leverage the software that we had. But a couple years ago we started to see a shift where camera based technology started to become better, faster, cheaper. And so now the same signal that we used to get from an expensive, you know, expensive medical device, we can extract that signal from the webcam that we're using right now on the computer. Computer. We can extract that signal from the camera on your phone, from the camera, on the eye tracking cameras, on smart glasses and in the cabins of vehicles and cockpits. And so we started to see that, okay, there's this kind of address, this addressable market that's from a device perspective that we could deploy. And we're kind of in a unique position because I go back to that proprietary database that we have, we now have the, the data to train models on. And so what we were missing was we've always been pretty great at machine learning, but what we were missing is the AI part. And so as we all know now a couple years back, the technology has changed, the foundation has changed and AI capabilities became more accessible. And so we started to hire AI capable engineers and AI researchers. And so what we did was we started to train these models on important user states, things like cognitive load and mental reserve and drowsiness. And we'll get into that in a little bit, I suspect. And in order to train those models and tune those models and retune them, you not only have to follow what the ground truth is and go through your, you know, your data exploration, you've got to be able to leverage AI and build neural networks in order to understand these kinds of user statuses in real time. And then what we did was we used AI to take it to a whole nother level, which is okay, in the past, most performance related user states were delivered after the fact. And therefore your intervention in an enterprise environment would need to be, you know, post facto and you, you deal with the consequences. What we Thought was if it was possible, was that if we could deliver a predictive, a predictive output, then the idea is that if you know ahead of time that someone's going to reach an overloaded mental state, right. Or have a low cognitive reserve, right. Or increase or about to become at a reach a high level of drowsiness, that if you know ahead of time, then you can intervene and you can prevent bad things from happening. And so we leverage AI not only to build and train our models, but also to deliver the time to transition to the next level, which is quite unique. Being able to predict, not only kind of understand the change in status, but predict when that change is going to occur.
- [10:19]
B
You know, this is interesting because you, you mentioned the pilot application. I'm thinking truck drivers, where based upon tracking the eyes, you could tell me in the moment you're getting tired, your cognitive load is too high, but you're talking about the predictive case. So can you give me a couple examples of how companies could use that more predictive capability?
- [10:42]
A
Yeah. So sticking with the driver use case, being able to understand, say the current level of say, cognitive load. Cognitive load is the current level of mental processing that your brain is demanding while you're performing a task. A low level of cognitive load is not where you want to be. It means you're unattentive, you're probably bored, and you're not focused. Similarly problematic is an overload situation where you have really no capability to handle anything that may come up and surprise you. In driving, we can imagine all those situations where you really want to be where that, where the performance level is optimal, whether you're driving or you're performing surgery, is at a moderate level. And so what we like to, where we want performers to have that consistent level of moderate is where you're going to see the best outcome, performance outcomes. And so going back to how our models are delivered, if you're driving a vehicle and you are either in a low load situation or an overload situation, and we know ahead of time that you're going to reach that problematic state, say 15 minutes ahead of time, then there can be pre programmed interventions in the vehicle. It could be haptic feedback, it could be delivering cold air, it could be audio feedback, it could be a number of different interventions. And we work with our customers to come up with the appropriate interventions for the appropriate use case. And that way we can prevent that person, prevent that person from reaching an overload state and hopefully prevent a poor outcome, whether it be an accident or not being able to anticipate something happening on the road.
- [12:31]
B
So, Adam, how specific is that intelligence to me, Ray Reich and how my eyes behave versus the general population is 90, 95% kind of general, where you can take those 15 million and learn it from everyone, or do you need to know mine specifically?
- [12:49]
A
That's a great question, and we don't get that a lot, but it's such an important question and we've spent a lot of time working on this and we use AI techniques to solve this issue. And so you're exactly right. If we deliver a model for cognitive load or drowsiness or attention, and it works for 90, 95% of the people, I wouldn't say that's scalable to the masses. Right. Because you can't have a calculator working 95% of the time. A calculator's going to work all the time or else you're not going to use the calculator. And so what we do in our model development process, not to get too much in the weeds here, but we obviously use a ground truth to develop the model. Right? So let's just take cognitive load, for example. The ground truth, or the gold standard, is the NASA TLX survey. It's what most organizations will use to develop a cognitive load model, whatever the sensor is or the data that you're using. So we'll use that as our ground truth for a base model. Cognitive load will tune and retune and we'll get it to a high level of accuracy. But once we deploy it, like you just mentioned, if I'm deploying it to Ray, your ground truth and your, your ground truth is different than my ground truth, we may be different age, different sex, different experience levels. Right. And so it is critically important that we understand the entire population so that I can real time understand your individual differences as compared to mine. And so what we use is advanced machine learning and AI in order to understand Ray as an individual. And that model very quickly adapts and adapts to you as an individual and it trains on your individual differences so that it's more accurate for you, just like it is more accurate for me. And that's a really important distinguishing factor of how we work and wouldn't be possible without these kinds of advanced machine learning and AI techniques. And it also, you know, what it does is it makes these models that we're deploying generalizable across the entire population. And it's not possible without that massive database that I told you about, because inside of that database is a normative Data set of 12 different age groups segmented by sex and many other different aspects of a human being. And so we can honestly say that from an eye movement perspective, we understand, harmonize, understands the entire population. And therefore, with just a few seconds of data, we can put you in the appropriate bucket, understand who Ray is versus Adam, and understand those individual differences and therefore have that model adapt to you.
- [15:48]
B
So we've talked about some non traditional white collar jobs. We talked about the pilot, the truck driver, But a lot of the people who listen to the podcast are managing white collar administrative roles, call centers, et cetera. So maybe if you could bring. I have two questions. The first job is if you're measuring things like cognitive load, drowsiness, right? That's going to impact my output. How do you correlate that reading of my cognitive load or drowsiness to my output capability? Maybe it's number of calls I can process, how quickly I can close a case. How do you integrate those two things, Adam?
- [16:29]
A
So the first step in that is understanding, identifying and understanding employee drowsiness, employee overload, and these models over a period of time, just take an eight hour shift, for example, right? Understanding that employee over the course of that day. There are a number of ways that you can utilize that output. Number one, like I mentioned before, in the pilot scenario, you can utilize it in real time and you can deliver specific interventions to that employee, right? If you think that they're going to reach a problematic state while they're performing a critical task. And that way you can prevent, you know, you can mitigate risks, you can mitigate poor decision making and you can improve performance outcomes. Another way to do it is if you think about like an after action review, taking a day in the life of that employee and looking at their levels over time, and even tying the problematic states, like maybe a high level of drowsiness, tying it to specific events, to specific outcomes and decisions that they've made. Say we're talking about a remote operator whose job it is to sit in front of, you know, three or four different monitors and, and monitor. It could be a drone operator, it could be a vehicle operator, it could be a lot of different things, right? That's a tedious, sometimes mundane, sometimes overloading experience when there's a lot of things happening. And in many of those cases, these operators have to make quick, really important decisions, whether it's to solve a problem or whether it's to transfer to say, a tier one level. Right? And so that remote operator, being able to understand the state of that operator in real Time can give a lot of information to say, the team leader. Right. And so if the team leader has access to that information, then that team leader can take a proactive approach to make sure that that operator over time or that operator in real time is at their best. Another way of looking at it is that if you take that employee over a longer period of time and you're starting to identify trends of a lot of high levels of load, high levels of drowsiness, low mental reserve in the tank over long periods of time, those are direct correlations to burnout. And so having that objective insight, you can then intervene and you can reduce attrition. Right. You can improve performance outcomes. If you understand that your employees, you know, kind of reaching a, they're for long periods of time in a burnout situation, then there's a lot of positives
- [19:23]
B
that, you know, Adam, this has popped in my head. This is a couple years ago, I had the founder and CEO of a company called Gong on and he was the really founder of this concept of conversational intelligence where they would record conversations that sales reps or call center reps were having. And it could actually, with all, you know, billions and millions of these recordings, tell by the tone, the semantic analysis of how the relationship was being established between the potential customer or the customer and the individual. This is like visual intelligence. Based upon the eye tracking. You can actually put intelligence into what the visual acuity is the wrong word, how it's going to impact job performance. I love this concept.
- [20:10]
A
Yeah. And it's, I would say it's, it's less about emotional intelligence and it's more about cognitive intelligence and even attentive attention intelligence. Right. So being able to understand the cognitive state of an employee over long periods of time, being able to stand their levels of attention, are they tending, are they paying attention to the right, the right things at the right time, are they distracted, et cetera, over long periods of time. It's quantifiable evidence for team leaders and management to inform large employee bases and positioning everything from like I mentioned before, real time interventions to after action reviews and preventing people from, you know, improving attrition. And even on the training side, what we're starting to see is a faster track to training proficiency because the trainers now understand what the trainees are going through. They can, they can, they can identify, you know, very early on the people who are going to make it and maybe the people who aren't and they can maybe cut their losses on those folks that they don't think can hack it. In a high stress environment. And furthermore, they can shorten the time to training proficiency, which has a, you know, in the namesake of your podcast, a real impact on roi.
- [21:35]
B
I love that. It's a great. Because you never know when you're doing training, whether it's instructor led or even self service kind of digital training, when you're reaching that point of overload, saturation, you know, kind of cognitive load is too high. Are you actually seeing corporations doing this today? Adam?
- [21:56]
A
We are. We're in the middle of several programs with enterprise customers, with military customers, and with high performance customers we work with. You know, if you think about a high performance customer with a high risk situation as NASA. Right. So astronauts being able to in real time understand these different user states, they don't want to take a subjective test. Right? They don't. They're looking for passive, non invasive, continuous signals that identify problems that can do all the things that we've talked about on this podcast. And so we're working with NASA and preparing to deliver these kinds of models in space. And like I mentioned before about military use cases and enterprise use cases with remote operators, with training and simulation and many other areas. Yes.
- [23:00]
B
Yeah, one other thing, I'm just thinking about the potential here. So having a camera in front of you is important, but in some roles, let's say it's a warehouse worker, right? Or a forklift operator, etc. Maybe there's not a capability to have a fixed camera. Are we actually now getting to the point where we can use like smart glasses, etc. To be the eye tracking mechanism?
- [23:25]
A
Yeah, in those use cases that's most likely what we're seeing is most likely the case is deploying smart glasses where we can extract that information when someone is performing a mobile task, active, active task, and they're not kind of bound to their desk. So one of the things that we pride ourselves on is that we're device agnostic and so anywhere that we can extract an eye tracking signal, so any camera based device, if we've got that signal, we can deliver these models. And it kind of goes back to what I talked about in the introduction, which is thankfully these cameras have become commoditized and they're virtually everywhere we are, not in every single case, but in many, many cases. Or they can be with the advent of smart glasses now. And so we can be almost everywhere an employee is. And so we can deliver these models across platforms and in many cases without the need to purchase new hardware. You know, if you're a desk environment,
- [24:38]
B
so Two final questions. Our time's already getting close to coming to an end. If I heard this and I'm an executive In a Fortune 500 company, I'm like, we could use it here, we could use it there. I could see the use case here. Do you have a standard practice of or best practice of where do I start? Do you help me identify where I should do a more constrained proof of concept or test case?
- [25:02]
A
Yeah, it. Most of the time it's where the pain point is. It's where the problems are. Right. It's. It's where maybe performance, their performance decrements on a specific team. Where we tend to see a lot of high stress environments. You can imagine that's where a lot of problems incur occur. Because when you've got employees who are dealing with high stress situations for hours on end and day after day and week after week and month after month, you tend to see that burnout situation. You tend to see problems with performance outcomes and then ultimately you tend to see really high stress levels and then you ultimately see burnout. And it's then that leads to attrition issues. And so we tend to kind of go where the problems are and there's no shortage of those. So we will typically just evaluate where those are and then we'll enter in that particular area.
- [26:08]
B
Well, you opened up the door for me where there's no shortage of those opportunities where the problems are. Can you give me a specific use case of and how the return on investment was measured by that company?
- [26:23]
A
Yeah, I would say one organization is that was a good use case was for pilot training. So being able to understand, and this was in a flight simulator. So being able to understand the pilot over time in real time. And what we started to see and what we started to quantify as evidence is the time to the training proficiency and the time to training proficiency. Having these kinds of metrics really inform not only the trainer and management, but also the pilots. They had a better understanding and therefore they could train to their weaknesses. Say another use case is in a remote operator situation. We talked about that call center type of environment. Being able to intervene in real time for things like employee fatigue and overload before safety incidents occur, shorting that time to training proficiency and then ultimately improving attrition rates. These have all been kind of proof points for ROI for enterprises that we've been involved in.
- [27:36]
B
But incident reduction has been one of those. Yes, very interesting. Okay, well, Adam, one last question. Cautions, is there any. Be careful of this. If you're thinking about Eye tracking and helping that improve productivity. Are there any mistakes or cautions you can share?
- [27:56]
A
Well, one of the things that, one of the things that we're really passionate about is privacy. And so we know that's a, that's a big word these days, especially as a lot of our, you know, bio signals are being shared. And the path is one direction is that we're all going to be sharing more information as a result of this AI tsunami. And typically people think, well, if I'm going to share more information, I'm just going to have to give up more privacy. And there's this kind of friction that happens until people get out of the technology something that they can't live without. That's typically where we'll be, okay, I'll, I'll invest in giving up my privacy for that. But we don't see it that way. So what we do is we actually don't. We refuse to collect, store or record any eye tracking information or any personally identifiable information. So every second that we're delivering our outputs of these models into our SDK to the customer, we're actually destroying the prior second's data. And so from an employer perspective, it's really up to them how they want to use the data. But from our perspective, we want to reduce that risk and that caution. And so we feel like that's really important. Some people might say, well, don't you want the data? I mean, we've got so much data, we can always use more data. But we can accomplish that in other ways in our own research efforts. But working in enterprise, we really have zero tolerance for any sort of privacy and data leakage.
- [29:41]
B
That's very interesting. Okay, let's give the audience a chance to know you through two rapid fire questions. First is who owns measuring and validating the return on investment for AI software in a company you're providing AI enabled eye tracking. Who do you think owns the. Make sure it's really providing a tangible return that a CFO can love.
- [30:05]
A
If I understand your question correctly, I would say that the employee owns their data and I would say that the output of that data, the performance output of that data maybe is a co ownership between the employee and the employer. Right. The employer has been collecting performance data on employees forever. It's their right to do so. And so I would argue that that COO owns that performance output data and is just another tool in their toolkit in order to evaluate those employees. And so for evaluation purposes, but also for improvement purposes, because that's what they're
- [30:51]
B
after and last question. We do have early career professionals here and they're like, they're worried about AI, so they could take their job. It could make employment harder. It might change my job. So any advice you have for those recent college graduates about understanding and getting
- [31:10]
A
comfortable with AI, it's interesting you asked that question because I've got a recent college grad student and students that are going to graduate in the next year. And so we have this, we have this conversation quite often. And in our house, I would say, you know, my recommendation would be not to look at it as something that can take my job. Right. I would look at it as it may change my job. And if something's going to change my job, the best possible way in order to prevent my job from being eliminated is to become the best at it. Right? So what I mean by that is if, if I, if I'm a little bit insecure that AI is going to change my job and therefore ultimately put my job at risk, I've got to, I've got to understand how it's going to change my job and I've got to become an expert really darn fast. And so immerse yourself in the tool, kind of connect it to the use case, to your job, understand how it can improve, how it can make, make things better, better, and be a leader in that regard. So that when the time comes, an employer is looking at, you know, maybe a RIF or something like that due to increased proficiencies or efficiencies from AI, who are the people that I'm going to keep on board as an employer? Right. I'm an employer. I want the people who have adopted this technology who are actually trying to take it to the next level because we know there's always going to be a human in the loop, right? And so be that human in the loop.
- [32:45]
B
Adam Gross, Co Founder CEO of Harmonize, thank you so much for being a guest here on the AI to RLI podcast.
- [32:52]
A
I enjoyed it. Thanks, Ray, for having me.
- [32:54]
B
And to our listening audience, if you're enjoying the conversation we're having about how AI as a technology can lead to ROI as a financial performance measurement. It mean the world to us.
- [33:04]
A
Good.
- [33:04]
B
And follow AI and RI in your favorite podcast podcasting app. Go and give us that five star rating and reach out to me at Ray Reich on LinkedIn. Let me know if there's a guest you'd like us to have. Goodbye, everyone. Thanks, Adam.
- [33:14]
A
Thank you.