The Jim Rutt Show: EP 221 George Hotz on Open-Source Driving Assistance

Release Date: February 6, 2024

Introduction

In Episode 221 of The Jim Rutt Show, host Jim Rutt engages in an in-depth conversation with George Hotz, a renowned hacker and entrepreneur known for his pioneering work in open-source self-driving car technology. This episode delves into Hotz's journey from his early days in hacker circles to founding Comma AI, an open-source self-driving car company. The discussion covers various facets of autonomous driving, contrasting approaches with industry giants like Tesla and Waymo, and touches upon legal considerations and Hotz's other ventures, including Tiny Grad.

George Hotz: From Hacker to Autonomous Driving Innovator

Background and Early Achievements

Jim Rutt introduces George Hotz, highlighting his early recognition as a participant in the prestigious Johns Hopkins Center for Talented Youth (CYT) program—a testament to his intellectual prowess from a young age. Rutt humorously notes that Lady Gaga was also part of the CYT, albeit without any known interactions between her and Hotz.

At [01:15], Hotz responds succinctly, indicating no personal interactions with Lady Gaga during their time at CYT.

Hacking Milestones

Hotz first gained notoriety by being the first to jailbreak the iPhone's carrier lock at age 17, a move that resonated with advocates of open systems against Apple's closed ecosystem. His hacking ventures continued with Sony over the PlayStation 3, leading to legal settlements but cementing his reputation in the tech community.

Transition to Autonomous Driving

Joining Google's Project Zero

Rutt mentions Hotz's stint at Facebook and his recruitment into Google's Project Zero—a team of elite white hat hackers tasked with uncovering vulnerabilities in critical systems. However, Hotz notes that his focus shifted away from security to artificial intelligence (AI):

[02:52] George Hotz: "Project Zero kind of led me to AI. I'm thinking like why am I looking for these vulnerabilities myself? How do I write software that looks for vulnerabilities?"

Founding Comma AI

The conversation transitions to Hotz's motivation to start Comma AI, an open-source self-driving car system. Initially aiming to build software for Tesla to replace the Mobileye chip, the contract fell through, prompting Hotz to pursue the project independently. Despite rapid progress in developing an autopilot clone, selling it to car manufacturers proved challenging.

Open-Source Self-Driving Systems vs. Industry Giants

Philosophical Differences

Hotz emphasizes a fundamental shift from traditional autonomous driving approaches, which rely heavily on specialized hardware and high-resolution mapping. Comma AI focuses on leveraging existing car camera systems and software to create a more flexible and scalable solution.

[06:20] George Hotz: "There's one system that can drive cars and it's human beings self-driving cars. Most of the ones you see like Cruise and Waymo, are really fancy remote control cars."

Critique of Current Systems

Hotz critiques the reliance on lidar and high-precision maps used by companies like Waymo and Tesla. He argues that these systems are economically unsustainable and technologically fragile, relying too much on centralized infrastructure that can fail, unlike Comma AI's decentralized approach.

[12:52] George Hotz: "That's absurd. There's a lot of criticisms of self-driving cars, but it's definitely not that one... How does that explain the human?"

Comparison with Tesla and Waymo

Hotz contrasts Comma AI's approach with Tesla’s and Waymo’s:

• Tesla: Uses powerful onboard computing with extensive data collection but still faces issues like phantom braking and lane misalignment.

• Waymo/Cruise: Operate under "level four" autonomy within tightly controlled environments, relying on remote operators to handle exceptions, making them economically unsustainable in Hotz's view.

[33:26] George Hotz: "Tesla has positive unit economics and Waymo has hilariously negative unit economics."

Technical Insights into Comma AI's OpenPilot

Behavioral Cloning and Simulation Training

Hotz explains the challenges of behavioral cloning—training a model to mimic human driving by learning from data. Without corrective mechanisms, small errors accumulate, leading to significant deviations from intended behavior. Comma AI addresses this by:

1. Adding Corrective Measures: Introduces algorithms to adjust steering based on lane detection to maintain stability.

2. Training in Simulation: Utilizes a proprietary simulator that reprojects real-world driving data with slight perturbations, allowing the model to learn corrective actions without relying on human intervention during training.

[15:33] George Hotz: "We refer to it as behavioral cloning... If you don't train in simulation, behaviorally cloned problem, you're going to have no corrective pressure."

Data Acquisition and Diversity

Comma AI boasts the second-largest driving dataset globally, amassed from over 10,000 weekly active users across diverse geographies. This extensive and varied data enhances the model's robustness compared to competitors limited to specific regions.

[19:10] George Hotz: "We have a massively diverse set. Waymo has all the same streets in Scottsdale... Now we have everywhere in the world."

Installation and Usability

Implementing Comma AI's OpenPilot is user-friendly, requiring minimal hardware modifications. Users simply connect a device via a Y-splitter to the car's existing camera system, enabling advanced driver assistance features without invasive changes.

[21:29] George Hotz: "Most new cars... have one plug that connects to it. All you have to do is install and connect."

Legal and Regulatory Considerations

Liability and Safety Standards

Hotz addresses concerns regarding legal liability and safety standards. Comma AI adheres to ISO 26262 standards, ensuring that their system cannot render the car uncontrollable. Responsibility for driving decisions remains with the human operator, who must maintain attention and control.

[43:41] George Hotz: "We limit the maximum amount of torque the system is capable of applying to the wheel... It’s on you."

Handling Malfunctions and User Compliance

The system is designed with multiple redundancies to prevent mechanical failures from causing accidents. Additionally, driver monitoring ensures users remain attentive, alerting them if they become distracted.

[44:42] George Hotz: "We have the best driver monitoring in the world... we force respect through effective monitoring, not coercion."

Tiny Grad: Simplifying Machine Learning Frameworks

Introduction to Tiny Grad

Towards the end of the episode, Hotz introduces Tiny Grad, a machine learning framework developed to compete with established platforms like TensorFlow and PyTorch. The key differentiator is its minimalistic design, comprising only 5,200 lines of code, which enables easier adaptability and deployment across various hardware environments.

[55:54] George Hotz: "Tiny Grad is a machine learning framework... it's 100x simpler. The code base is only 5,200 lines."

Applications and Future Goals

Tiny Grad is already in use within OpenPilot for running models on devices, and its simplicity allows for easy porting to new hardware accelerators. The long-term vision includes developing machine learning ASICs (Application-Specific Integrated Circuits) to optimize performance further.

[57:27] George Hotz: "It's used in OpenPilot to run the model on the device... the long-term goal of Tiny Grad is to build machine learning ASICs."

Vision for the Future

Beyond Self-Driving Cars

Hotz articulates a broader vision where solving self-driving cars serves as a stepping stone toward general-purpose robotics. The ultimate ambition is to create artificial life forms capable of performing complex tasks autonomously, such as cooking and cleaning.

[46:08] George Hotz: "Our goal is to solve self-driving cars as a jumping off point to general-purpose robotics... a $25,000 robot companion that comes home and cooks for you."

Commitment to Open-Source and Ethical Development

Emphasizing transparency and user control, Comma AI maintains an open-source ethos. Hotz is committed to resisting external pressures, such as patent trolls, ensuring that the company's innovations remain accessible and ethically developed.

[52:25] George Hotz: "I'm legit willing to do it... I don't want to oversell anything. Buy it or don't buy it, that's up to you."

Conclusion

This episode offers a comprehensive look into George Hotz's approach to autonomous driving through Comma AI's open-source framework. By challenging the conventional methods employed by industry leaders and advocating for a more decentralized and user-friendly system, Hotz presents a compelling case for the future of self-driving technology. Additionally, his work on Tiny Grad underscores his commitment to simplifying and democratizing machine learning, paving the way for broader innovations in AI and robotics.

Notable Quotes:

• George Hotz at [02:52]: "Project Zero kind of led me to AI. I'm thinking like why am I looking for these vulnerabilities myself?"
• George Hotz at [06:20]: "There's one system that can drive cars and it's human beings self-driving cars."
• George Hotz at [12:52]: "That's absurd... How does that explain the human?"
• George Hotz at [19:10]: "We have a massively diverse set... Now we have everywhere in the world."
• George Hotz at [34:06]: "Tesla and Comma both have businesses where we sell things to consumers at a profit."
• George Hotz at [43:41]: "We limit the maximum amount of torque the system is capable of applying to the wheel... It’s on you."
• George Hotz at [55:54]: "Tiny Grad is a machine learning framework... it's 100x simpler. The code base is only 5,200 lines."

For more insights and detailed discussions, listeners are encouraged to tune into The Jim Rutt Show and explore Comma AI and Tiny Grad through their respective websites.