Podcast Summary: "How Latent AI Brings Intelligence to the Edge"

Podcast: Reshaping Workflows with Dell Pro Max and NVIDIA RTX GPUs
Host: Logan Lawler (Dell Technologies)
Guest: Jags Kandasamy (Co-Founder & CEO, Latent AI)
Date: November 6, 2025

Episode Overview

This episode dives into how Latent AI is transforming the deployment and efficiency of artificial intelligence at the edge—delivering intelligence closer to where the data is created. Host Logan Lawler interviews Jags Kandasamy, who explains why shrinking AI models and optimizing them for diverse hardware—from powerful servers to remote sensors—is crucial in modern workflows. The conversation covers Latent AI’s innovative approach to model compression, hardware-software co-design, real-world use cases (like oil & gas surveillance via satellite connections), and the recent launch of Latent Agent, a tool that empowers software—not necessarily AI—engineers to harness AI in their applications. The episode also offers a behind-the-scenes look at data labeling and future workflow innovations.

Key Topics & Insights

1. Why Smaller, Edge-Optimized AI Models Matter

Timestamps: 00:48–03:40

• Data Explosion & Bandwidth Limitations:
  AI’s rise is fueled by massive, exponentially growing data from IoT, sensors, & mobile. Yet, sending all this to the cloud is impractical:

  “If you were to collect all this data and start to send it up to the cloud and process it there, you are not going to have enough bandwidth... The more the bandwidth you open up, the more devices come online and more the data that they pump up and they start to clog the pipes up.”
  — Jags Kandasamy (02:08)

• Real-Time Decisions at the Edge:
  Critical decisions, especially in industrial, military, and healthcare, need to happen where data is generated, not after cloud round-trips.

2. Real-World Use Case: Remote Oil & Gas Surveillance

Timestamps: 04:28–05:56

• Challenge: Unmanned sites need real-time surveillance, but streaming raw video over SATCOM is prohibitively expensive.
• Latent AI Solution:
  • Deploy quantized (compressed) models on edge devices to monitor site intrusions.
  • Send only pertinent event data (not bandwidth-heavy video), enabling fast alerts and local audible warnings.
  • Ran on Dell Native Edge hardware.

“So we are helping them run quantized models and smaller models at the edge, at the distributed remote locations that are doing surveillance monitoring… Sending a blurb of data through Satcom is faster than uploading a video.” – Jags (04:28)

3. Model Quantization—Accuracy vs. Compression

Timestamps: 05:56–08:50

• Definition: Shrinking model parameters from 32-bit floating point to as low as Int8, making them lighter and faster for edge devices.
• Pitfall: Naive quantization often reduces accuracy.
• Latent AI’s Value: Advanced IP (originating from SRI for DARPA) maintains model accuracy after quantization.
• Comprehensive Hardware Lab: Measures ~20 metrics per run, across different hardware, for optimal deployment choices.

“The logic that we have brought in… is how do you maintain the accuracy of the model while shrinking it?... That is the core competency, core IP that we bring to the table.” – Jags (06:50)

4. The Hardware–Model Co-Design Approach

Timestamps: 09:35–10:54

• Customizable to Use Case: Users set priorities (speed, power, accuracy); the system recommends the best combination.
• Scalability: Edge deployments can be cost-optimized by matching hardware to use case needs—not overprovisioning.

“You have to consider that as well… what is the best value prop that I can get with my hardware? … All of that can be compressed into that model selection.” — Jags (09:35)

5. Supporting New & Foundational Model Architectures

Timestamps: 12:35–13:51

• Flexible Support:
  Latent AI readily supports major DNN architectures (e.g., YOLO, EfficientDet) and transformer-based LLMs.
• Driven by Customer Requirements:

  “There is only so many buttons that I can push as a startup… we are working on a customer requirement, but from a foundation perspective we support the transformer architecture easily.” – Jags (13:38)

6. Introducing "Latent Agent" — The AI Agent for Software Engineers

Timestamps: 14:23–20:53

• Natural Language, No-Nerd-Required:
  Built as a natural language agent, Latent Agent enables software (not AI/ML) engineers to design, train, and deploy AI models—with minimal expertise.
• VS Code Integration:
  Available as a VS Code extension; software devs register, interact with the agentic workflow, and deploy object files for their apps.

“We’ve taken the 747 cockpit and changed it into a car cockpit—an automatic car. ... Any software developer ... can design their model, ... train the model and extract the object file, ... and deploy.” – Jags (15:05)

• Remote Hardware Access:
  Users can compile for and test AI models on Latent AI’s lab hardware—no local Jetson/RTX required.

7. Linear, Well-Defined Agent Workflows

Timestamps: 22:24–23:48

• Not a Generalist Agent:
  Latent Agent is specialized and narrowly focused on building and deploying models—not trying to “do everything.”

“[W]e tried to bake cookies with this and we failed. So we stuck to our lane. ... That North Star ... how do we enable a software engineer to build ML models? That was very clear.” – Jags (22:24)

• Automatic Bug Detection:
  E.g., automatically locates and patches memory leaks during C object file compilation.

8. Next Steps: Assistive Data Labeling and Edge Retraining

Timestamps: 25:22–27:01

• Data Labeling:
  New tools label vision data (images, video) up to 80x faster—“human in the loop” approach—originated from US Navy needs.
  • Visual clustering and mass-labeling.
  • Search for objects across datasets.

“We can ingest all those 10,000 images, identify objects in each one … and visually … cluster similar looking objects together.” – Jags (27:59)

• Descriptive Labeling:
  Uses dual VLMs (vision language models) to generate and cross-validate scene descriptions for richer labels—coming soon.

9. Edge Retraining for Rapid Adaption

Timestamps: 26:26–27:01

• Deploy in Tactical Environments:
  Enable rapid, local retraining of edge-deployed models (e.g., on a Dell Toughbook) to respond to new data/drift—no need to send data to the cloud for retraining.

Memorable Quotes

• On Edge AI:
  “You need every software developer to be thinking about incorporating AI into their workflow.” – Jags (17:40)

• On Quantization:
  “When you do that [quantize], automatically, the accuracy of the model will drop. The logic that we have brought in ... is how do you maintain the accuracy of the model while shrinking it?” – Jags (06:50)

• On Agent Focus:
  “We tried to bake cookies with this and we failed. So we stuck to our lane.” – Jags (22:24)

• On Democratizing AI:
  “Software engineers, bunch of them—30 million is like one of the rough counts ... But software developers are not AI experts ... This is going to be kind of the new wave.” – Logan (18:23 & 18:33)

• On Platform Flexibility:
  “You’re not necessarily over buying—you’re buying kind of, from Latent AI all the way down to the hardware, you’re buying stuff that makes sense for that use case, which I absolutely love.” – Logan (10:54)

Important Timestamps

• Main introduction & Latent AI’s approach — 00:48–03:40
• Remote surveillance use case — 04:28–05:56
• Model compression & unique value — 06:50–08:50
• Hardware-software co-optimization — 09:35–10:54
• Support for new AI models — 12:35–13:51
• Latent Agent launch & philosophy — 14:23–20:53
• Agent workflow and focus — 22:24–23:48
• Data labeling & upcoming features — 25:22–29:44
• Rapid edge retraining — 26:26–27:01
• Where to learn more about Latent AI / Final summary — 30:45–31:37

Where to Learn More

• Latent AI Website: latentai.com
• Contact: jags@latentai.com

Episode Takeaways

• The trend is not always “bigger is better” for AI models—edge applications need right-sizing for real impact.
• Latent AI’s technology makes AI accessible to every software developer, not just ML/AI specialists.
• Real-world edge AI deployments demand practical, cost-conscious, and scalable solutions—ensuring optimal model accuracy, speed, and value for each workflow.
• The future of AI onboarding is plug-and-play, not build-from-zero.