The AI Breakthrough That Could Transform the World in 2025 - WSJ Tech News Briefing

Summary

WSJ Tech News Briefing: The AI Breakthrough That Could Transform the World in 2025

Release Date: December 27, 2024
Host: James Rundle
Guest: Christopher Mims, Tech Columnist

Introduction

In the December 27, 2024 episode of WSJ Tech News Briefing, host James Rundle delves into the pervasive influence of artificial intelligence (AI) and its potential to revolutionize various industries by 2025. The episode features an insightful conversation with tech columnist Christopher Mims, who explores the advancements, applications, and challenges of generative AI beyond the well-known ChatGPT models.

Understanding the Transformer Architecture

Christopher Mims begins by elucidating the foundational technology behind modern AI: the transformer architecture. He references the seminal 2017 paper by Google DeepMind titled "Attention Is All You Need," which introduced a suite of algorithms that have since ignited a surge in AI development.

"In 2017, some researchers at Google DeepMind published a paper called Attention Is All You Need. And that started this supernova explosion of AI that we've seen since."
— Christopher Mims [01:21]

Mims explains that transformers enable computers to act as universal learners, capable of extracting and understanding structured data across various domains. This versatility is what powers models like ChatGPT and extends their applicability beyond conversational agents.

Applications of Transformer Models in 2025

Drug Discovery and Synthetic Biology

Mims highlights how companies are leveraging transformer models to innovate in fields such as drug discovery and synthetic biology. By training models on comprehensive datasets of organic molecules and their properties, AI can propose new molecules for specific applications, including potential cancer treatments or enzymes to degrade environmental pollutants like plastics.

"They've fed them every organic molecule which has ever been characterized in a scientific paper... and then you prompt it with, well, I want a molecule that treats this particular cancer."
— Christopher Mims [02:39]

Robotics and Autonomous Systems

In robotics, transformer models are being used to empower robots to learn and perform complex tasks autonomously. Mims cites the example of Physical Intelligence, a company demonstrating a robot capable of folding laundry by learning the task independently, marking a significant milestone in robotics.

"The end result is a company called Physical Intelligence has been showing off a robot that can fold your laundry... it basically learned on its own how to fold laundry."
— Christopher Mims [02:35]

Limitations and Challenges of Transformative AI

Data Availability

One of the primary challenges in developing specialized transformer models is data scarcity. While language models benefit from the vast amounts of text available on the internet, other applications lack similarly extensive datasets. This limitation creates a barrier for startups and smaller companies attempting to innovate in specialized AI domains.

"The biggest limitation when you are trying to create a new transformer model is always data... there isn't a similar gigantic free library of actions that a robot could take."
— Christopher Mims [04:51]

Distinguishing Hype from Reality

Mims emphasizes the importance of separating hype from reality in AI advancements. He cautions that while transformer models can perform impressive tasks, they lack true intelligence and understanding, functioning instead as sophisticated pattern recognizers without conscious comprehension.

"These models absolutely aren't intelligent in any real sense of that word... it's proof that you can have really facile, fluid communication with zero intelligence behind it."
— Christopher Mims [06:06]

Considerations and Concerns with AI

Overreliance on AI

Mims identifies overreliance on AI as a significant concern. Delegating critical decision-making to AI systems without fully understanding their processes can lead to errors and biased outcomes, posing risks across various sectors.

"If we hand over decision making to these AIs and we don't have sufficient knowledge about how they actually make decisions, we can get ourselves into a world of trouble."
— Christopher Mims [08:33]

Environmental Impact

The environmental footprint of AI is another pressing issue. Training and operating large AI models consume substantial energy, contributing to environmental degradation. While optimization efforts may improve efficiency, the growing demand for AI capabilities could exacerbate energy consumption.

"It's incredibly energy hungry now... the more we ask of AI, the more energy it's going to take to answer our questions."
— Christopher Mims [08:33]

Investment and Market Stability

Mims warns of the potential for malinvestment within the AI sector. The current surge in AI investments may lead to a bubble, with numerous startups failing if their technologies do not deliver the anticipated productivity gains. This scenario could result in another "AI winter," dampening innovation and market confidence.

"There's a lot of big companies that are going to make big investments in certain types of AI and ultimately find that they don't yield the productivity boosts they were hoping for."
— Christopher Mims [08:33]

Future Outlook on AI Development

When discussing the trajectory of AI development, Mims suggests that the rapid pace of generative AI advancements is likely to slow. Instead, the focus will shift to integrating AI into everyday applications and optimizing its use for practical benefits.

"We're definitely hitting a wall in terms of improvements in their performance... we're going to have decades of people figuring out how to make it a part of their lives and their businesses."
— Christopher Mims [10:42]

Mims likens this phase to the installation phase of a technology, similar to how personal computers and mobile phones transitioned from early adoption to widespread integration into society.

Conclusion

The episode of WSJ Tech News Briefing offers a comprehensive exploration of the transformative potential of AI, particularly through transformer models. Christopher Mims provides a balanced perspective, highlighting both the groundbreaking applications and the inherent challenges facing AI as it approaches mainstream adoption by 2025. As AI continues to evolve, it remains crucial to navigate its development thoughtfully, addressing ethical, environmental, and economic considerations to harness its benefits responsibly.

Notable Quotes

Christopher Mims [01:21]: "In 2017, some researchers at Google DeepMind published a paper called Attention Is All You Need. And that started this supernova explosion of AI that we've seen since."
Christopher Mims [02:39]: "They've fed them every organic molecule which has ever been characterized in a scientific paper... and then you prompt it with, well, I want a molecule that treats this particular cancer."
Christopher Mims [06:06]: "These models absolutely aren't intelligent in any real sense of that word... it's proof that you can have really facile, fluid communication with zero intelligence behind it."
Christopher Mims [08:33]: "If we hand over decision making to these AIs and we don't have sufficient knowledge about how they actually make decisions, we can get ourselves into a world of trouble."
Christopher Mims [10:42]: "We're definitely hitting a wall in terms of improvements in their performance... we're going to have decades of people figuring out how to make it a part of their lives and their businesses."

Produced by Julie Chang | Host: James Rundle | Theme Music: Jessica Fenton & Michael Lavalle
Special thanks to Katherine Milsop, Ayesha Al Muslim, Scott Soloway, Chris Sinsley, and Falana Patterson for their contributions.

Transcript

Amazon Representative (0:00)

Amazon Q Business is the Generative AI Assistant from aws. Because business can be slow, like wading through mud. But Amazon Q helps streamline work, so tasks like summarizing monthly results can be done in no time. Learn what Amazon Q Business can do for you@aws.com LearnMore welcome to Tech News Briefing.

James Rundle (0:22)

It's Friday, December 27th. I'm James Rundle for the Wall Street Journal. We're hearing from our reporters and columnists about some of the biggest companies, trends, people and tech and what could be in store for 2025. Coming up on today's show, artificial intelligence is everywhere, propelled by the runaway success of OpenAI's ChatGPT and other models. But the tech behind generative AI is far more than just the engine for a fancy chatbot. Researchers are exploring how the technology might be used to create bacteria that eats plastic, self driving cars or potential cures for cancer. Our tech columnist Christopher Mims joins us to talk about how the bleeding edge of AI research may go mainstre next year. Christopher Many people have become familiar with AI as essentially a conversational search tool in recent months. Thanks to ChatGPT and other platforms, however, the underlying technology has greater applications. Tell us about the transformer and why it's so important.

Christopher Mims (1:21)

So in 2017, some researchers at Google DeepMind, which is their AI outfit, published a paper called attention is all you need. And that started this supernova explosion of AI that we've seen since. And what was key about that paper was introduced a suite of algorithms which give us a new model for how to create in a computer a universal learner, something which can extract from any large body of data that has inherent structure in it, like language, the sort of underlying order of that data. And it's the reason that we have ChatGPT, for example. But what's interesting is the world is full of structured data which we can apply the transformer architecture or algorithms to. And the result is kind of a GPT for all kinds of things, right? For drug discovery, for, for synthetic biology, for self driving cars, for robots, et cetera.

James Rundle (2:35)

So what are the ways in which companies are hoping this can be used in 2025?

Christopher Mims (2:39)

Companies are using it to, for example, create new molecules. And the analogy here is when you're using ChatGPT, you're not really having a conversation with an AI. It's like you're in the same Google Doc and you are collaborating, you're writing a collaborative story, but the narrative is it's a chat, so you write some than the robot write some etc. And in biology, what people have done is instead of feeding these transformer models, all of the text on the Internet, which is what it took to get ChatGPT, they've fed them every organic molecule which has ever been characterized in a scientific paper. And everything we know about that molecule, what it does in the real world, its function. And so then you go to that kind of bio GPT and you prompt it with, well, I want a molecule that does this, you know, it treats this particular cancer. And just like ChatGPT, it continues the dialogue, it auto completes what might come next, which is instead of a sentence, it's a proposed sequence of molecules which would make up this new drug potentially, or it could be a new enzyme that would go into bacteria to digest all the plastic in the great Pacific garbage patch. So this is to me one of the most interesting and compelling examples. But in this same vein, people are taking these transformers and feeding them tons of actions that a robot could take. And then they can help power robots which can teach themselves how to perform certain tasks. And that's been kind of a holy grail of robotics. So the end result is a company called Physical Intelligence has been showing off a robot that can fold your laundry. Turns out this is probably the hardest problem in robotics right now. It's even harder than the Boston Dynamics robot doing parkour. And what's key about that is it basically learned on its own how to fold laundry. It wasn't a series of scripted actions.