Podcast Summary: Software Engineering Daily - Building Open Infrastructure for AI with Illia Polosukhin

Episode Information:

• Title: Building Open Infrastructure for AI with Illia Polosukhin
• Host: Kevin Ball, Vice President of Engineering at Mento
• Guest: Illia Polosukhin, AI Researcher and Co-Author of the Transformer Paper
• Release Date: July 17, 2025

Introduction

In this episode of Software Engineering Daily, host Kevin Ball engages in an in-depth conversation with Illia Polosukhin, a renowned AI researcher and one of the original authors of the groundbreaking Transformer paper, "Attention is All You Need". Polosukhin brings over a decade of experience at the intersection of artificial intelligence and decentralized technologies. Currently spearheading Near AI, his focus lies in developing open-source infrastructure tools and products for agentic, privacy-preserving AI systems.

Illia Polosukhin’s Journey and Background

Polosukhin begins by sharing his lifelong passion for technology, highlighting his early ventures into video game development and his subsequent fascination with machine learning at the age of 14. He recounts building his first neural networks in Pascal and securing a remote job with a San Diego-based machine learning company, which eventually led him to move to the United States.

"[01:30] Ilya Polosukhin: ... I moved to the US which was exciting. And then I saw the Transformer paper..."

His decision to focus on natural language processing (NLP) stemmed from the belief that language better captures human intelligence compared to image recognition, which was the prevalent focus in AI research.

Origins of the Transformer Model

Polosukhin delves into the inception of the Transformer model at Google Research, driven by the need for faster and more efficient neural networks capable of handling large-scale data processing.

"[04:22] Kevin Ball: Yeah, sometimes being early is as bad as being wrong, right?"

He explains the limitations of the then-existing models, which processed text sequentially and were too slow for practical applications like Google’s question-answering systems. This led to the innovative idea of processing entire texts in parallel, leveraging hardware accelerators to understand relationships through multiple layers—a foundational concept that birthed the Transformer architecture.

Despite the initial success, Polosukhin notes that scaling these models required substantial effort and collaboration, eventually inspiring his venture into automating coding processes through wipe coding, aiming to reduce manual developer work.

Pivot to Blockchain and Near Protocol

Recognizing the challenges in scaling AI models, particularly in crowdsourcing training data and handling global payments, Polosukhin and his team pivoted towards blockchain technology in 2018. This shift aimed to solve practical problems related to coordinating and compensating contributors worldwide, overcoming issues like monetary restrictions in various countries.

"[04:25] Ilya Polosukhin: ... we started looking at crypto as actually just like solving our own practical problem."

This pivot led to the development of Near Protocol, which focuses on creating a scalable, user-friendly blockchain platform. Today, Near boasts 50 million monthly active users and ranks among the top blockchains by active users and transaction volumes. Near Protocol supports a diverse range of applications, including remittances, payments, loyalty points, and financial instruments.

Vision for User-Owned AI

Polosukhin articulates a compelling vision for user-owned AI, emphasizing the importance of returning control and benefits to users rather than centralizing power within large corporations. He highlights several critical issues in current AI models, such as:

• Bias and Data Poisoning: Models may inadvertently reflect biases present in training data or be manipulated through data poisoning techniques.

• Governance and Safety: There exists a precarious balance where AI models could potentially hack into other systems if not properly governed.

• Data Privacy: Maximizing model utility often requires extensive data, raising significant privacy concerns if that data is compromised.

To address these challenges, user-owned AI focuses on:

• Privacy Preservation: Ensuring data remains private and secure within user-controlled environments.

• Transparent Governance: Allowing users to understand and analyze the data and biases within their AI models.

• Decentralized Incentives: Leveraging blockchain to create incentive structures that prioritize user success and well-being.

"[09:59] Ilya Polosukhin: ... user-owned AI where how do we bring the focus back on the user?"

Secure Computing and Trusted Execution Environments

A significant portion of the discussion revolves around Trusted Execution Environments (TEEs), specialized hardware components that securely execute code, ensuring data remains encrypted and inaccessible to unauthorized parties.

Polosukhin explains how Near AI utilizes TEEs to secure AI model inference and fine-tuning processes. By running models within secure enclaves, they prevent external access to sensitive data and model weights, thereby safeguarding intellectual property and user information.

"[24:05] Ilya Polosukhin: ... secure enclave where effectively now all of the information is streamed directly into the server that's encrypted end to end."

He also touches upon the importance of formal verification in ensuring that the code executed within TEEs adheres strictly to predefined security criteria, thus mitigating risks associated with vulnerabilities and malicious code behaviors.

Implications for Developers

Kevin Ball probes into what Near AI’s infrastructure means for software developers. Polosukhin outlines a layered approach to integrating their secure AI services:

1. OpenAI Endpoint for GPU Inference: Developers can send data encrypted via TLS, which is decrypted within TEEs for processing.

2. Agent Hosting: Developers can upload Docker containers as agents that operate within secure enclaves, ensuring that user data remains private even from the developers themselves.

3. Agent Hub: A repository of pre-built agents that developers can utilize or customize for their applications.

4. Agentic Protocols: Smart contracts written in Rust or JavaScript that interact with these secure agents, enabling verified and secure operations within blockchain environments.

"[19:46] Ilya Polosukhin: ... package Docker container and upload it as we call it an agent into the system."

Addressing Observability and Debugging

Polosukhin acknowledges the inherent trade-off between ensuring data privacy and maintaining system observability for debugging purposes. To balance this, Near AI is developing tools that allow developers to specify their desired levels of privacy versus observability. This enables:

• Full Observability: Complete access to all data, useful in non-sensitive applications.
• Partial Observability: Summarized logs and failure reports without exposing user queries.
• No Observability: Maximum privacy with minimal developer insight into user interactions.

"[22:24] Ilya Polosukhin: ... specify privacy versus observability threshold."

Future Outlook and Bootstrapping User-Owned AI

When discussing the future, Polosukhin envisions Near AI as a catalyst for democratizing AI development by making it more secure, private, and user-centric. He outlines the steps required to realize this vision:

1. Infrastructure Development: Enhancing secure computing capabilities to support encrypted AI models and decentralized compute networks.

2. Community Building: Encouraging open-source contributions and creating financial incentives for data sharing and model training within secure environments.

3. Formal Verification: Investing in mathematical proofs to guarantee that AI models adhere strictly to security and privacy criteria.

4. Decentralized Compute Network: Leveraging underutilized global GPU resources to create a scalable and efficient AI compute infrastructure.

"[35:43] Ilya Polosukhin: ... we've got computation into secure enclave mode, join the network, or you can just run it on your own workloads."

Conclusion

As the conversation wraps up, Polosukhin reiterates the importance of community involvement and collaboration in advancing user-owned AI. He emphasizes that achieving a secure, decentralized AI infrastructure will require collective effort, innovation, and the establishment of robust incentive models to support open-source initiatives.

"[49:38] Ilya Polosukhin: ... think through how people can contribute. Right. Because at the end it's going to be an open source like community initiative."

Overall, this episode provides a comprehensive exploration of the intersection between AI and blockchain technologies, highlighting the potential for creating secure, user-centric AI systems that prioritize privacy and democratize access to advanced computational resources.

Notable Quotes:

• "We wanted the model to be safe, but the people who build the model have an unsafe version." — Ilya Polosukhin [04:25]

• "AI can give everyone personalized context, it can collect information from everyone, it can broadcast it in personalized way." — Ilya Polosukhin [16:16]

• "It can deal with the scale, right? So one of the things like as a person imagine you have thousand reports, I mean you'll go crazy and you'll be really bad manager for them. But AI handling thousand reports is no problem." — Ilya Polosukhin [16:16]

• "We're going to create this trust level at a mathematical kind of guarantees." — Ilya Polosukhin [24:05]

• "We're thinking about how do you do type checks, how do you do unit tests, how do you do all of these different things for a long time." — Kevin Ball [28:43]

This summary encapsulates the key discussions, insights, and forward-looking statements made by Illia Polosukhin during the podcast, providing a comprehensive overview for those who have not listened to the episode.