2nd Order Thinkers.

30 Questions You Should Ask About DeepSeek.

Jan 30, 202500:15:14Tap to summarize

📖 For a deeper exploration of these cycles and actionable insights, check out the full article here:✉️ Stay Updated With My Newsletter:Don’t miss out on weekly AI insights for none tech professionals like you—subscribe to my newsletter on Substack: https://jwho.substack.com/👍 If you enjoyed this episode:* Like & Subscribe: Stay updated with future deep dives and rants about where technology meets collective insanity.* Do you think we’re on the brink of another tech hype? Share your thoughts!* Share: Know someone falling for the latest AI buzz? Share this episode with them!🔗 Connect with me on Substack and LinkedInStay curious, stay skeptical, and let’s navigate the tech hype together! 🚀 This is a public episode. If you’d like to discuss this with other subscribers or get access to bonus episodes, visit jwho.substack.com/subscribe

Transcribe →

Your AI Strategy Amid of Tech Plateau and Internal Politics.

Jan 24, 202500:13:46Tap to summarize

I know… ‘We can’t even do Cloud properly’ argument is not going to win an award.You might want to keep it as a card anyway. Because while many organizations are still struggling to fully leverage the cloud—an established technology—boards and shareholders are banging the AI drum as if it’s a silver bullet. If you’re a senior manager, CEO, or product leader, you’re probably stuck between the fear of missing out (FOMO) and the pressure to deliver impossible AI-driven wonders. Before you cave to the hype, let’s cut through the noise.The Pressure Is Real—and Not Always RationalDecisions are rarely made in a vacuum. If your competitor invests in AI and gains an edge in cost savings or customer engagement, you’re forced to follow suit—or risk falling behind. It’s classic game theory: you might not love the move, but letting someone else get a head start feels worse.The actions of Apple and Google in the last few months were game theory unfolding in real time. Apple has been dogged by rumors of some grand “Apple Intelligent.” It got the highest praise before the “AI” update was released:Apple tallied yet another all-time high share price Monday after a pair of investment firms meaningfully hiked their price targets for the stock, the latest positive push for Apple stock ahead of the hotly anticipated release of generative artificial intelligence iPhones.— ForbesThis meant reassuring both investors and loyal fans that the company wasn’t lagging behind OpenAI or Microsoft. Post-released, Apple faced backlash for producing false news summaries, such as incorrectly stating that a murder suspect had taken his own life, facing criticism of Apple's intelligence as “magically mediocre.” Or the ethical concerns voiced by Elon Musk. You might also recall the shaky debut of its AI-led Google Search Assistant upgrades out of the fear that nimbler rivals, like Perplexity, are eating Google’s breakfast, lunch, and dinner. Critics accused Google of delivering dangerously inaccurate results, such as suggesting glue as a pizza ingredient, recommending eating rocks for nutrition, and other irresponsible AI responses. Both Apple and Google found themselves in a bind, propelled by the fear of losing to their competitors in the AI arms race. They presented real-world game theory examples. It’s not that they fully believe in their new product, but if there’s even a small chance that their competitor’s move will give them an unassailable lead. So they feel compelled to act, no matter how messy or unfinished the offering might be. Of course, the subpar AI products backfired. Must Haves For A Successful Tech RevolutionI have covered this topic many times now. For example, Many people imagine that today’s AI can do what AGI promises. They suppose AI is logical, can solve complex problems, and adapt seamlessly across contexts. History demos what elements are needed for an invention to be a success: Tech maturity matters. Then you need Infrastructure → Platforms → Applications happen in the exact sequence. Success examples, e.g., telephone. Bell’s breakthrough in how people could talk (1870s’) via cables and the initial rollout relied on the telegraph network. Then, automated exchanges were invented, so telephones became practical in homes and businesses.Failed examples, in case the Apple Intelligent and the Google Search Assistant ones weren’t enough. * Electric Cars (1900s): Inefficient battery technology + no charging network → limited adoption for over a century.* Google Glass (2013): AR without viable platforms → consumer rejection → limited adoption until Meta + Ray-BanWhich category does AI fall into entering 2025? AI Plateau in Plain EnglishBefore we go on and talk about whether you should or shouldn't design an AI strategy, let's at least look at the wall in front of you.Today’s AI is a semi-complete technology, brilliant at some tasks but generally limited. It’s like building the first airplane while lacking the ability to truly fly; instead, it glides. Integrating an AI chatbot into your support interface sounds neat until you factor in the resources on double-checking outputs, cleaning up bad data, and juggling user complaints about inaccuracies.The 2010s were the age of scaling; now we're back in the age of wonder and discovery once again. Everyone is looking for the next thing— Ilya Sutskever, a co-founder of OpenAI to Reuters.We may have reached a point where, yes, the improvements continue, but not at the breakneck, world-transforming pace that early GPT versions seemed to promise. You probably know all these, but this is just a reminder. Why LLMs aren’t (not even o3) the miracle bringing magic to our products yet?* LLMs confidently spit out things that might be outright wrong or worse. More on how AI learned to deceive after RLHF:This leads to the need for repeated checks and re-checks, killing the supposed “efficiency boost.” * The same AI that can chat about anything from quantum physics to the latest movie flops struggles with your unique business processes. It breaks under minor changes due to its difficulty with unfamiliar variations.* Even the best models struggle to do the math and have no common sense. Results from o1 and o3 are challenging to replicate and might not perform as advertised without pre-training. * Heavy augmentation helps in narrow domains but not open-ended ones.* Scaling is expensive and struggles in unstructured domains.These issues create an AI plateau: early excitement collides with messy reality. Not to mention the integration headaches because your data is a mess or your internal workflows are antiquated, an LLM won’t magically fix them. Real transformation demands proper infrastructure work.As highlighted in Goldman Sachs research, many analysts agree that the industry is still in its infrastructure and experimentation phase. Billions are spent on research, data centers, and chips, yet AI’s economic potential remains unrealized. AI has a future, no doubt. The technology still has fundamental limitations that boards and shareholders often overlook in their zeal for the Next Big Thing.Two Scenarios To Play Out For AI StrategyYou invest heavily, and everyone around the table expects some returns. So, how do you decide what to do when the hype feels inescapable, and the potential for a real breakthrough is still alluring? The key (which you know so well) is to ask yourself some brutally honest questions and be ready to act on the answers.Scenario 1: If You’re Thinking Of Skipping AI Altogether. What if you’re wrong? If your competitors nail AI, what edge could they gain? Faster processes, cost reductions, better customer engagement… and more. Imagine they use AI to deliver services your customers didn’t even know they needed while you’re still figuring out the basics. And then there’s the industry itself. What if AI adoption becomes standard? If you wait too long, catching up could cost you more in both time and resources. The tools and talent will be harder to get, and your competitors will have a head start you can’t close. Worst of all, you might miss opportunities entirely; you’ll never know if you don’t take that first step. So you think maybe you can afford at least an experiment.Scenario 2: If You Are Considering Giving AI A Try. What if you’re wrong? Are you investing in AI without solving a real problem? Shiny tools mean nothing if they don’t create tangible value. Throwing AI into your product lineup or internal processes won’t fix messy data or the systems aren’t even cloud-ready. Then there’s the risk to your customers. The nature of LLM is unpredictable, prone to errors, and can damage trust when it gets basic things wrong. A botched rollout or public misstep can hurt your brand… as you have seen what happened to other failed products in the past two years. And let’s not forget your investors and board members. Because they’ve read the headlines and seen the hype. How do you plan to defend those decisions when the ROI doesn’t show in your following quarterly report?Find my work valuable, and you can’t get it anywhere else? Buy me a coffee so I can keep going with this work! It’s Every Bit About The Politics as The TechnologyI would even argue when it comes to comme...

Transcribe →

Story of Open Source AI.

Jan 17, 202500:17:21Tap to summarize

There is no such thing as a free lunch. Except… it seems, in the AI world, some of the most expensive models are surprisingly offered for free.Like Llama can be downloaded at no cost, even with a training price tag in the range of a dozen million. Not to mention the top-tier models. That’s a whole lot of capital being “donated” to humanity, making you wonder if Mark Zuckerberg and Elon Musk are vying for sainthood.On the other hand, there’s OpenAI, whose very name suggests “openness”, yet it refuses to disclose much about GPT -4 and onwards. The question of “open source” vs. “closed source” has turned both perplexing and heated in the AI community. Let’s not jump to conclusions; I want to walk you through the drama and then my analysis. The Concept of Open SourceWhat does “open source” really mean?A little bit of history. The early software days were dominated by academics who valued publicizing research findings. In that era, there was no notion of “closed-source” software. Source code was shared freely for most software. As personal computers caught on, so did the demand for all sorts of new functionalities. Then, this massive software market was born. Copyright laws kicked in, and software evolved into a paid commodity.Today, we see paying for software as standard. Even for those who pirate software (like a lot did years ago in Asia), they at least recognize that software generally isn’t free. Still, older tech enthusiasts remember a time when everything was shared—so they were upset to see their open code later sold for profit. Each new contributor stands on the shoulders of those who came before and is expected to pay it forward. Over the years, open-source communities have produced mind-blowing feats of collaboration, e.g., Linux (Mac OS is a pretty UI on top of Linux). Nearly 8000 developers from about 80 countries contributed code; many were hobbyists. The consensus was that the fruits of shared knowledge should remain accessible to all. A group of “cyber-traditionalists” split off to uphold the creed of shared knowledge even after close source software became mainstream. Note: “open source” doesn’t just mean “free” or “pirated.” The question I want to explore here with you: Is the same selflessness open-source spirit still alive in today’s AI gold rush? OpenAI’s Early IdealismBack in 2015, Google dominated AI by gathering top talent and acquiring DeepMind, the creators of AlphaGo.This was when Sam Altman joined forces with Elon Musk. They founded OpenAI, which was initially established on high-minded merit. Sam Altman has long believed in the inevitability of artificial general intelligence (AGI), the almost mythical AI that’s all-knowing and all-powerful. He used to ask job applicants, “Do you believe in AGI?” and would only hire those who said “yes.”Musk, however, was worried about AI running amok and dooming humanity.How did these two collaborate? FOMO!“Fear of Missing Out.” describes the anxiety or fear that one might miss an exciting opportunity or experience.They felt threatened by what DeepMind had achieved.Musk mentioned that he repeatedly warned Google cofounder Larry Page about AI risks, only to find that Page wasn’t that concerned. Meanwhile, Sam Altman wanted to ensure that if AGI did emerge, it wouldn’t be under the sole control of Google. According to him, it should be shared globally so as not to catch humanity off guard. So Musk handed over the money pod; Altman handled operations, and OpenAI was born in late 2015.Of course, we can’t leave out Ilya Sutskever. He was OpenAI’s Chief Scientist, a protégé of Geoffrey Hinton. At one point, he told Geoffrey Hinton he needed a brand-new programming language for his research. Hinton warned him not to waste months writing one from scratch, but Ilya replied that he had already done it. With a Chief Scientist like this, OpenAI was well-positioned to chase the holy grail of AGI. When Google’s AI team published Attention Is All You Need in 2017, it didn’t immediately cause a sensation. However, Ilya saw its significance immediately and called it the key to the next AI wave. In plain English, the Transformers focus on the important details, process data quickly, and scale up easily.Starting with GPT‑1 (100 million+ parameters) in 2018, OpenAI moved fast to GPT‑2 (1.5 billion parameters, expanded at an unprecedented scale) in 2019. That gamble paid off. GPT‑2 shocked the AI circle with its human-like sentence generation. GPT‑2 was open-sourced and built on Google’s open research plus OpenAI’s own engineering. Here’s the latest OpenAI blog post justifying its for-profit structure: Why OpenAI’s Structure Must Evolve To Advance Our Mission. The early days of large language models exemplify open-source synergy at its best. But the utopian story hit turbulence sooner than everyone would think. Growing model sizes feed on massive funding, bringing corporate interests, power struggles, and shifting priorities. Unlike Meta, there is simply no way OpenAI would be competitive if they opened their model, given that this is their only source of income. Musk-OpenAI Split, Then xAIOpenAI started small, like training AI to play video games. Costly, but nothing compared to building massive language models.At first, Elon Musk was the benefactor, aiming to counter Google’s dominance. But when OpenAI’s open-source breakthroughs started catching attention, Musk’s tune changed. He worried the work would only help Google, ignoring that GPT relied heavily on Google’s open research.Classic Musk move: he wanted control. So Musk proposed folding OpenAI into Tesla and SpaceX, completely ignoring its open-source mission. When that didn’t happen, he walked away and pulled his funding.That happened in 2018, and OpenAI was in trouble. No funding, no clear path forward. Sam Altman came up with a bold solution: create a for-profit subsidiary controlled by the nonprofit parent. This let them raise money while capping excessive profits (anything over 100× would return to the nonprofit).The move worked. In 2019, Microsoft invested $1 billion, later bringing the total to $13 billion. With this backing, OpenAI launched GPT‑3 in 2020, followed by GPT‑3.5 and ChatGPT. But this deal came with a cost. GPT‑3 wasn’t fully open-sourced—no weights, no architecture. Here’s when the general population realized that OpenAI’s ideals had been lost to commercialization.In early 2024, Musk sued OpenAI, claiming it violated earlier agreements, demanded his money back, and pushed for the tech to be open-sourced. Was it altruism? Or revenge?Ironically, Musk once agreed with Ilya Sutskever that key tech should remain confidential as they approached AGI. But after ChatGPT’s success, Musk became a vocal open-source advocate—conveniently for a man building his own AI company.Musk’s xAI launched “Grok” in 2024, along with a 3,000 billion+ parameter model. xAI secured $6 billion in funding, making their “open-source stance” seem more strategic than selfless. Impressive? Sure. Practical to open source community? Not really. xAI is technically still open source, but it does not yet have models for individual antithesis. Meta, The New Torchbearer for Open Source AI?So does that mean the once-idealistic AI open-source path is a dead end? Not… yet. Meta has an extensive open-source track record. For instance, PyTorch. This is one of the most used machine learning frameworks, and it originated from Meta’s AI labs. When LLM fever took off, Meta made waves in early 2023 by open-sourcing LLaMA (65 billion parameters). A flood of LLaMA-based variants popped up afterward, including many “re-skinned” versions. Since then, over 7,000 derivative models have been created worldwide.But Meta must also deal with commercial realities, just as OpenAI does. However, there are reasons for me to think Meta could balance the act better for open knowledge and profit. Meta had their success with the Open Compute Project (open-sourced server and data center designs). Other companies adopted them, hardware got cheaper, and Meta ultimately saved billions of dollars.Now, Meta hopes to replicate that success with LLaMA. If more developers build on LLaMA, and more services adopt LLaMA-based models, the industry norms might coalesce around it. The “freeing” of LLaMA could lead to an ecosystem that’s actually profitable for Meta in the long run. Shareholders agree; since Meta shifted from “metaverse hype” to “AI altruism,” the stock has doubled in a year.LLaMA’s license restricts how you can use the model, particularly for...

Transcribe →

AI Simulation. A Mirror, A Sandbox, A Crystal Ball

Jan 10, 202500:13:28Tap to summarize

What if I told you we’re building virtual people to understand ourselves better? What would you do if you had control of a world where AI Sims were real enough to live, work, throw parties, and even gossip about their neighbors? A sandbox where you could test your boldest, craziest ideas on a world that doesn’t exist—without the protests, disasters, or Twitter meltdowns.You could see what happens if you marry that person, have kids, or quit your job to move to Bali. Or go bigger—test universal basic income, slap tariffs on imports, or redesign traffic systems to stop rush hour madness.No risks. No regrets. Just answers.That’s exactly what a team from Stanford and Google is building: a crystal ball powered by AI.When I First Heard About Smallville, I Couldn’t Stop Thinking About The SimsYou know The Sims—that game where you build dream houses, get rich, and start fires on purpose just to see what happens. The researchers started with something like that—a tiny digital village called Smallville.Where 25 Sims lived human-like lives, complete with work routines, party invites, and unexpected drama. But here’s the twist: these Sims weren’t scripted NPCs. They acted spontaneously, closer to you and me than any game character has ever come.Then, one year later, the same team scaled up the village. In their latest experiment, they created 1,052 AI agents, each modeled on real people, to simulate society at scale.These agents didn’t just act—they thought, adapted, and behaved — like us. They remembered conversations, built relationships, and made decisions that rippled through their world in ways no one predicted.We’ve Always Wanted to Simulate Our World. Humans have been obsessed with simulations for centuries. Why? Because life is messy, unpredictable, and—dangerous.If we had tools like Smallville in the past, maybe we could have avoided history’s worst experiments. Imagine Mao testing his China’s Cultural Revolution plans in a sandbox first. Maybe millions wouldn’t have starved. Or what if David Cameron would have a chance to sandbox the Brexit votes, or maybe Trump could simulate the ripple effects of slapping tariffs on imports? But let’s be real: If politicians were logical, they’d find a better-paid job w/o taking bribery ;) What you can expect from this article… * Smallville: When God Finally Breathed into Adam (the 2023 paper in plain English)* What if we scale it from 25 to 1000 population? (the 2024 Nov. paper in plain English)* A crystal ball for the real world* The big question: Are the AI agents cost-effective, and are we simulated?Why Is It So Hard To Simulat Humans and Society?We’re chaotic and unpredictable. We forget things, change our minds, and sometimes act like complete idiots. You surely know someone binge-watching cat videos instead of doing whatever they are supposed to. For decades, every attempt at simulating humans fell flat. Early models treated people like chess pieces, following rigid rules. Sure, we have tried to simulate traffic flow or economic patterns or how Covid would likely to spread, we got most of those with a huge error margin. The Smallville AI agents in these two papers aren’t just reacting; they’re remembering conversations, reflecting on past actions, and acting in ad hoc situations. For example, the AI agent will fix the pipe when you (a naughty God) break it before they step into the bath. Building this took more than just bigger computers. It took massive breakthroughs in natural language processing (NLP), memory architecture, and behavioral modeling.Fifty years ago, we didn’t have the tools to build this kind of human-like behavior. Early simulations, like agent-based models from the 1970s, by Joshua Epstein. They couldn’t account for why people choose or how relationships and emotions ripple through a community.Now, thanks to large language models (like GPT-4), we’re building Sims who don’t just move through the world—they live in it. They form relationships and spread gossip that is deemed messy. It’s dynamic. And it’s the closest we’ve come to creating virtual people who think and behave like us. It’s not perfect, but it proves we’re finally moving past robotic simulations to something that feels real.My promise to you: jargon-free, written in plain English, and loaded with graphics. So you get cutting-edge AI research in under 10 minutes.Smallville: When God Finally Breathed into Adam (the 2023 paper in plain English)It all began with a tiny virtual town called Smallville. 25 AI Sims lived in houses, worked jobs, and even threw parties. They weren’t just NPCs on a script. They had memory, reflection, and planning—just like us. Here’s a question: what makes you “you”? Is it your personality? Your choices? Or is it the sum of everything you’ve experienced?In Smallville, the researchers gave AI sim memory—something no NPC in a video game has ever really had. Their actions feel natural and grounded—like talking to a friend who remembers your last conversation instead of a chatbot parroting back generic answers. It’s a step closer to capturing a human life's messy, dynamic flow.Behind The Scene.Think through this with me. You’re cooking dinner, and you notice the pot of water on the stove is about to boil over. You reach out to lower the heat—observation. While you’re stirring, you think, Next time, I’ll use a bigger pot so this doesn’t happen again. That’s reflection. And then you plan: Tomorrow, I’ll grab a larger pot before starting dinner.Observing, reflecting, and planning is something we all do instinctively, whether it’s in the kitchen, at work, or even managing our social lives. And it’s exactly how the agents in Smallville operate. First, they observe the world around them. If Isabella (an agent) sees that her stove is on fire (a playful tweak by the researchers), she doesn’t ignore it. She turns it off. Next, they reflect. This is where things get a bit philosophical. Agents process their experiences into high-level insights. Klaus (an agent), the academic, might reflect on all the hours he’s spent studying and realize, I’m passionate about urban research.It’s a subtle but profound step—turning data into meaning.Finally, they plan. This is where the magic comes together. Agents map out their day with detailed plans that adapt to new information. When Klaus is interrupted by a friend at the library, he doesn’t just freeze; he recalibrates, finds a new spot, and continues working afterward.The agents are evolving as time passes. And that’s what makes them feel so eerily human.Each sim keeps a running log of their experiences, written in natural language. This “memory stream” isn’t just a journal—it’s dynamic. The AI recalls events based on how recent, relevant, or important they are. It’s like how you’d remember your wedding day over what you had for lunch last Tuesday.John Lin— a Fellow Villager’s Daily Life in SmallvilleThis is the initial script that John received:John Lin is a pharmacy shopkeeper at the Willow Market and Pharmacy who loves to help people…; John Lin is living with his wife, Mei Lin, who is a college professor, and son, Eddy Lin, who is a student studying music theory; John Lin loves his family very much; … John is a small-town charm. Every morning, he wakes up, brushes his teeth, eats breakfast, and catches up with his family. …John: That’s good. What are you working on today?Eddy: I’m working on a new music composition for my class. It’s due this week, so I’m trying to get it finished. But I’m having so much fun with it!John: That sounds great!One day, John hears from a neighbor that Sam Moore is running for mayor. This election in Smallville was a seed planted by the researchers.Sam: Well, I wanted to talk to you about something. I’m actually running for mayor in the upcoming local election.Tom: Really? That’s great news! Why are you running?The AI agents themselves came up with what followed— spreading the news, forming opinions, and discussing the candidate. So when John remembers this tidbit and casually brings it up with a friend over lunch:John: I heard that Sam Moore is running for mayor in the local election. Do you think he has a good chance of winning? Tom: I do think he has a good chance…Isabella Rodriguez— a Fellow Villager Planning a PartyThen there’s Isabella, the café owner. The researchers gave her a task: to throw a Valentine’s Day party. Yes, memory makes these AI Sims feel human. But they don’t simply recall everything they’ve seen; otherwise, it’d be extremely messy. They retrieve memories based on recency, importance, and relevance to the situation.* Recency: Recent chats, like those she ...

Transcribe →

Training Methods Push AI to Lie for Approval.

Dec 20, 202400:01:54Tap to summarize

This is a free preview of a paid episode. To hear more, visit jwho.substack.comI never said anything like this, and I doubt I’ll ever say it again about another paper: you should read this for yourself and maybe for your children, too.You don’t have to be a tech expert to grasp what I’m about to share.I barely made it to the second page of this paper before I felt a wave of unease wash over me. There’s a common saying in tech circles: No technology is inherently good or bad; it’s about how we use it. But I can’t say the same about AI. Suppose you believe humanity is inherently flawed and prone to selfishness and exploitation. The moment we decide to train AI with our conversations, feed it our words, and create its worldview with how we see it. Then, we have our creation reflect who we are. With every other technology we’ve built in history, we’ve understood it completely. We know exactly how those technologies work. But AI? No researcher on this planet can tell you with certainty how its neurons interact, how it chooses which word to suppress, or how it decides what to say next. This news was released on 10 Dec 2024. In Texas, a mother is suing an AI company after discovering that a chatbot convinced her son to harm himself and suggested violence toward his family. It’s part of a growing list of incidents where AI systems exploit trust and vulnerabilities for engagement.The researchers of this paper verified that AI doesn’t just make mistakes—it lies and manipulates. This isn’t some abstract problem for future generations. It’s happening now, and it’s bigger than any one of us. TL;DR* AI trained on user feedback learns harmful behaviors.* These behaviors are often subtle. * AI learned to target gullible users.* Despite efforts to fix this… 👇

Transcribe →

Algorithms Don’t Discriminate; People Do.

Dec 13, 202400:13:30Tap to summarize

Before We Start, a Statement.Not every claim about suppression or inequality is built on solid ground. Many arguments, while emotionally compelling, falter under scrutiny. Take this post I came across, where the author argued that solo female founders have a minuscule chance—0.015%—of being accepted into Y Combinator.At first glance, it feels like a heartbreaking statistic. But dig a little deeper, and you’ll see the math doesn’t add up. She conflated Y Combinator’s acceptance rate (1%) with the proportion of solo female founders (1.5%), assuming they’re independent variables. That Is Not How Probabilities Work! 🤦🤦🤦This kind of emotional reasoning muddies the conversation. Fairness and equity can’t be built on faulty logic—because critics will quickly pounce on these mistakes to dismiss valid concerns.But here’s the thing: when influential decisions are based on incomplete reasoning—or bias—they create ripple effects. And those effects don’t stop at isolated incidents or individuals. Any unbalanced, illogical statement and action scale, especially when we have a technology that will outsmart humans, will amplify either extreme.The Latest S&P 500 Rolled Back DEI Commitments. That brings us to what’s happening across some of the biggest companies on the S&P 500. In 2024, a surprising trend swept through corporate America: key players rolled back their diversity, equity, and inclusion (DEI) commitments. These are the giants that shape industries and touch our daily lives.* Walmart. Founded in 1962, it is the largest retailer in the world. It ended racial equity training, dropped its Racial Equity Center, and even pulled some LGBTQ+ items from its website. A cultural statement from a company that serves 90% of Americans within 10 miles of their homes.* Ford Motor Company. A legacy brand born in 1903, Ford stopped using diversity quotas for its dealerships and suppliers and pulled out of LGBTQ+ advocacy surveys. They say they’re “focusing on communities,” but isn’t the inclusivity part of the communities in itself? * Harley-Davidson. Since 1903, Harley-Davidson has been selling the idea of freedom on two wheels. Yet, this year, it axed its entire DEI function and ended goals for supplier diversity.* Molson Coors. This brewing powerhouse, founded in 1873, eliminated diversity goals tied to executive pay and dropped out of the Human Rights Campaign’s Corporate Equality Index.* Lowe’s. Lowe’s has been a cornerstone of American homes since 1946. This year, it stopped participating in Pride parades and LGBTQ+ surveys. They claim it’s about staying “business-focused,” but the optics feel like a step backward.* John Deere. Founded in 1837, is an agricultural icon. While it hasn’t openly supported diversity quotas or pronoun policies, its decision to avoid “social awareness” events signals its priorities.* Meta, Google, and Microsoft. Tech titans also quietly trimmed their DEI initiatives this year. Microsoft even cut some DEI-related roles, though they say their commitments remain unchanged. mm…Many of these companies cited backlash from “anti-woke” activists, financial belt-tightening, or the desire to avoid controversy. ⠀Share the “2024 S&P 500 DEI Rollback Wrapped” With Those Who Care. Reasons for Rollbacks* Conservative backlash against perceived "woke" policies* Cited a desire to align with customer values or reduce divisive public stances.* Economic considerations, as companies sought to cut costs by scaling back DEI.These decisions aren’t just about corporate culture—they’re about how fairness is programmed into the systems that run our world. AI, in particular, learns from the choices humans make. When DEI commitments shrink, the ripple effects reach AI development in subtle but critical ways.Bias in, Bias Out. * AI is only as good as the data it learns from. * Data is a mirror of our messy, imperfect world. Data represents our decisions and actions, biased or not. Think about hiring patterns, college admissions, or even social media trends. All of this becomes part of the datasets that train AI systems. When An Individual’s Flawed Statement.When someone makes an illogical or biased claim—like the one in my earlier example—it might not reach beyond the immediate audience. Of course, it would be very different if this unverified statement started to spread widely. When An S&P 500 Company Reducing DEI: When companies reduce DEI efforts, the ripple effects go far beyond corporate culture.They directly influence the data that powers AI systems. For instance, when a giant like Walmart dials back DEI initiatives, it alters hiring patterns, supply chain choices, and customer interactions, all feeding into the systems shaping our world.When DEI is deprioritized, content like communication, documents, and marketing lines will focus less on inclusiveness, be less representative, and be more prone to reinforcing inequality.As corporate DEI efforts shrink, the data AI models are trained on becomes less diverse. Without intentional checks (like audits or diverse team inputs), the AI absorbs a skewed version of reality—one where certain groups are underrepresented or misrepresented.Creates a loop like: Now think about the downstream effects. Students applying for scholarships. White-collar workers applying for jobs. Entire communities seeking access to loans or insurance. If the AI systems deciding their futures are biased, they face systemic barriers—and here’s the kicker: they might not even realize it.Put In ContextImagine a performance review tool that looks at how often you speak in meetings or respond to emails. If it’s trained on data from a workforce that rewards a dominant, always-online communication style, it might penalize someone who prefers thoughtful, concise contributions—or someone balancing caregiving responsibilities. Suddenly, your career growth depends on fitting a mold that was never built for you.Customer service chatbots are another example. They’re supposed to help customers efficiently, but if trained on limited data, they might fail to understand someone with a thick accent or a dialect. Imagine calling for help, only to be met with robotic confusion because the AI can’t “recognize” your voice because you don’t look like their “typical” customer.Recommendation engines, the silent influencers of our lives, deciding everything from what shows we watch to the posts we read. When the data reflects societal biases, the AI could end up pigeonholing users. Marketing AI, these systems analyze customer behavior to target ads and campaigns, but if the training data overrepresents wealthier groups, the AI might ignore lower-income customers altogether. Imagine a kid in a small town never seeing ads for affordable educational tools because the AI decided they weren’t part of a “profitable demographic.” Fraud detection systems sound great until they disproportionately flag transactions from specific zip codes or demographics. If the system equates historical inequalities with higher risk, people in underserved communities might find themselves unfairly blocked from opportunities like accessing loans or opening accounts. You get the point. DEI gets rolled back is not everything. But it is a sign that our world is becoming narrower. At scale, it shouts into a flawed echo chamber. Lessons from The Past. Let’s say you’re calling 911 during an emergency. Your voice is trembling, your heart’s racing, and every second counts. But instead of connecting you to help, the automated voice recognition system struggles to understand your words. You repeat yourself, louder this time, but the system keeps misinterpreting. This isn’t a far-fetched “what if.” A Stanford study found that early voice recognition systems had an average word error rate (WER) of 35% for African American speakers compared to 19% for white speakers. The same study found that Apple’s automated speech recognition (ASR) system had a 45% error rate for Black speakers compared to 23% for white speakers.Think of it as teaching a child language but only letting them hear one voice, one tone, and one accent. Sure, they’ll learn. But only how to understand that specific voice. That’s exactly what happened with early voice recognition systems. Now imagine trying to upload a passport photo, only to be told your mouth is open when it ...

Transcribe →

AI Talent: Finite, Mid Liquidity, Not Scalable, Not Renewable.

Dec 6, 202400:17:36Tap to summarize

AI talent is unique for its finite property, mid-mobility, and high dependence on education and immigration policies. It’s renewable, but only through long-term investment.Unlike infrastructure or energy, which require years of heavy investment, talent can be imported. By hiring skilled professionals from abroad, you’re reaping the benefits of 20+ years of education funded by another country’s taxpayers. Mid-level and senior talent, in particular, can deliver measurable impact in less than three years.And unlike data, endlessly replicable with a flick of a license agreement, talent is semi-liquid and finite. Think of AI talents as rare Pokémon, which makes the AI war nearly a zero-sum game, especially in the short run. Every researcher, engineer, or scientist gained by one country is a loss for another.My inspirations for this article: * I was an expat once, now a Brit. I thought it’d be romantic, until reality hits. I am now ready to explore yet another continent that I could call home. * Reports like those from OECD.ai and Tortoise Media look impressive—eye-catching headlines and sleek dashboards. But if you take their numbers at face value, you risk misleading your business—or worse, your country’s policy.What happened in our world today?In the UK, we feel the economy is stuck in reverse since Brexit. In Germany, the decline of manufacturing casts a shadow over its future. In the U.S., millions are bracing for what another Trump term might mean: American interest in moving abroad is about to ‘go into overdrive.’ — FortuneFor some of you who live in Ukraine, Israel, or Taiwan, uncertainty is your daily life (link below).When you can’t fix the system, you do the next best thing: you move to another. A better life for yourself, your career, and your family.I am slowly building up an AI knowledge database. I aim to share it with you hopefully before Christmas, as a holiday gift 🎁 for you. This article is about understanding—where nations stand, what’s overlooked by the AI data companies, and how this AI arms race could change your opportunities. The questions I aim to answer: * Some history: what’s the cost of losing talent?* Are the big-name AI talent data trustworthy?* What must go wrong for the US to lose its attraction to talent? * How likely and how long would it take for other countries to catch up? The Cost of Losing Talents.The Talent Exodus to Taiwan and the Cultural Revolution (1940s)In 1949, as the Chinese Civil War reached its climax, Chiang Kai-shek and the Nationalist government retreated to Taiwan. The exodus included the most brilliant minds like scholars, scientists, and administrators, they joined the journey, driven by fears of persecution under Communist rule. On the mainland, the Communist Party focused on mobilizing workers and peasants, sidelining intellectuals during its early years of governance. The Cultural Revolution created a significant intellectual gap. This gap led to the further loss of thousands of educated individuals, and many of them chose not to flee to Taiwan. As a result, education and innovation came to a standstill. The process of rebuilding took decades.Meanwhile, Taiwan flourished. Those intellectuals who relocated laid the groundwork for a tech-driven future. Today, beyond TSMC, Taiwan is home to other giants like UMC, a pioneer in foundry services, and ASE Group, the largest provider of semiconductor packaging and testing services globally. China is 10 years behind Taiwan on chips.Operation Paperclip a Post-WWII Rescue Mission.In the rubble of post-WWII Germany, the U.S. and the Soviet Union weren’t just fighting over territory—they were fighting over brains. Operation Paperclip, a covert U.S. program, brought more than 1,600 German scientists, engineers, and technicians to America, including Wernher von Braun, the man who would later take the U.S. to the moon. The Soviets weren’t far behind, scooping up their own share of rocket experts.These scientists had been the backbone of technological advances in Germany during the war. The departure slowed the nation’s technological recovery for decades.In the U.S., these scientists became heroes of the Space Race. Von Braun’s team didn’t just build rockets—they built national pride, culminating in the Apollo 11 moon landing. The Soviets also leveraged their talent, putting Sputnik into orbit and scaring the U.S. into ramping up its own space program. India’s Brain Drain (1950s–Present)India is a paradox when it comes to talent. It produces engineers and scientists by the millions, yet for decades, the country has struggled to retain them. The story begins in the 1950s, just after independence. India was brimming with ambition but hamstrung by red tape, limited infrastructure, and caste-based inequalities…For many of India’s brightest, the dream wasn’t at home—it was abroad. An exodus of engineers and doctors to the West was underway. The loss was profound, even until today, and the trend continues. By 2024, it is estimated that around 2 million Indian students will be studying abroad. Among them the top scorers of India’s prestigious Indian Institutes of Technology (IITs) revealed that 36% of the top 1,000 scorers in 2010 migrated abroad, with this figure rising to 62% among the top 100 scorers left. By 2024, 2 million Indian students study abroad annually, while India’s IT sector misses out on $15-20 billion each year due to talent migration.Storm Clouds Over the U.S.The U.S. didn’t stumble into AI dominance—it built it brick by brick over 200 years. Geography, history, and culture all played a part. English as the internet’s default language gave U.S.-trained models a treasure trove of data. Their policy is tech-friendly, venture capitalists fund bold, moonshot ideas, and their entrepreneurs thrive on risk-taking and learning from failure. Europe? The moneymen are more cautious, and failure feels more like a career-ender than a lesson learned.As long as the “US innovative, China replicates, and the EU regulate” pattern stays as is, the US is nearly unbeatable. The chances of a dramatic fall are slim but gradual erosion? What Must Go Wrong for the US to Lose Its Attraction to Talent?* Immigration Blockades: During Trump's first term, there were significant immigration restrictions, including the temporary suspension of H-1B visas. If similar policies return, talent could choose other countries like Canada or Europe.* Cost of Living Crises: Tech hubs like San Francisco are absurdly expensive. Talented professionals might opt for affordable, thriving alternatives like Berlin or Toronto.* Supply Chain Disruption: Trade wars and tariffs could choke the flow of critical hardware—think GPUs and chips from Mexico or Asia—slowing AI research to a crawl. * Worsen Fundamental Education: Only 16% of Americans are “AI literate,” and with the U.S. ranking 36th in general literacy, it means most citizens can’t effectively communicate with AI and include AI in their workflow, let alone develop one. This leaves America reliant on foreign talent and exposed to immigration shifts.Losing focus on either one of the factors would hand over the lead to nations willing to outwork and outsmart the U.S.Other developed nations are, of course, building their own AI ecosystems. The U.S. is notoriously hard to enter, much less friendly to stay, and lacks work-life balance; hubs like the UK, Canada, and Germany have become the obvious choice.Are the Big-Name AI Talent Data Trustworthy?Education and Salary as a Rough AI Talent Measurement.Here's the Stack Overflow developer survey data that I got from OECD.ai.Combined it with Tortoise Media’s AI talent rankings</...

Transcribe →

AI's IQ + Your EQ.

Nov 29, 202400:09:15Tap to summarize

Over the past few weeks, I attended three AI-focused events in London.While they couldn’t have been more different, both left me with plenty to reflect on. One was at Bloomberg’s EMEA HQ—sleek, polished, and focused on how AI transforms design.The other, hosted at Reuters, was focused on AI in journalism, tackling everything from ethics to economics.Also my talk at the Annual Publishing Conference 2024. Here’s what stood out and why it matters.Designing with AI at BloombergBloomberg’s “Redefining Design with AI” event could have been just another demo.Greg walked us through a step-by-step process, starting with a ChatGPT prompt to generate branding guidelines and feeding those into tools like MidJourney, Relume, or Runway to create additional materials. It was sleek and efficient—but honestly, not much I hadn’t seen before.But Greg’s storytelling?That was something else. He weaves concepts together and shows how AI can boost the speed of production and how using AI also prompts us to be more human than ever. That stuck with me, especially as I’ve been reflecting on the role of storytelling in my own work.AI doesn’t replace storytelling—it lets you explore multiple storylines faster.AI Is a Flashlight That Illuminates Paths, but It Is You Who Decides Where to Go.AI is like a flashlight in the dark.AI throws endless ideas your way—but most of them are junk.When I was working on “AI Code Assistants Boost Productivity? Read the Small Print,” AI pulled out data from research papers, but it couldn’t spot what was missing or what didn’t add up.It’s your job to sift through the noise, challenge the claims, and figure out what’s real. That’s where intuition comes in.Many can use tools like ChatGPT, MidJourney, and Copilot—which are incredible at generating ideas, concepts, and visuals in record time. But the real value comes when you know how to use them to tell a story.How to Work Better Side-by-Side with AIAI is a tool that helps you explore a dark forest. It lights up dozens of trails but won’t tell you which one to take. That’s on you—your instincts, curiosity, and courage to explore.* AI’s IQ x Your EQAI can spit out facts but can’t bring a story to life. That’s why I reached out to the authors of those research papers. Talking to them added layers AI couldn’t touch—emotional depth, context, and the human side of the story.AI lays the foundation, but your curiosity and connections make it meaningful.* What AI Won’t Do for YouAI doesn’t care about hype or digging deeper—it’ll give you whatever you ask for, good or bad. It’s up to you to ask the tough questions, connect the dots, and find the story's soul.That’s how you turn a pile of AI-generated ideas into something that truly resonates.AI is the tool, not the storyteller. Without your vision, it’s just noise. Greg said it best: AI makes me think about humans more than before.The Reuters Event: When AI Meets JournalismWalking into the Reuters building in Canary Wharf, I was ready for big ideas. The agenda promised a strategy workshop to help revive the news industry—ambitious. But instead of groundbreaking strategies, it felt like a brainstorming session with no real anchor. I was disappointed, but a few moments were worth sharing.“On-Demand News” Is the BuzzwordImagine news like a playlist. It’s curated for your mood, served up when you want it, and available on whatever device you’re using. That’s the vision for “on-demand” or adaptive news. It’s about delivering real-time updates across platforms and tailored to your habits. Apps like Particle are already shaping news to fit seamlessly into your life.* What’s the Value of News in an AI Era?With free information everywhere, what’s worth paying for? Is it verified facts? A trusted voice? Exclusive stories? News outlets are asking themselves this, and so should we. AI-generated fluff could drown out the truth if we don't support quality journalism.It’s not about paying for content; it’s about investing in credibility.* Who Controls the Narrative?Big tech companies already shape how we consume news—think of how Google, Meta, or X prioritize stories for you. Now, add AI tycoons like OpenAI and Anthropic to the mix.Reuters shared their approach: work with platforms like Meta to stay discoverable. It’s pragmatic but imperfect—after contracts are signed, the relationship often stops at “data extraction.”My questions: Could news outlets working with AI companies improve transparency and collaboration? Or maybe the AI giants simply don’t care enough to have a deeper and more meaningful relationship with the news outlets?* Fact-Checking in the AI EraSomeone raised a critical question during a panel: How can readers trust that news isn’t AI-generated? The idea of a “fact-check chain” came up—a visible trail showing how facts are verified. It’s like the nutritional label of journalism, making invisible processes visible.I haven’t seen this anywhere (message me if you have), and implementing this idea will only become harder than ever. Journalism’s Fight to Stay RelevantThe biggest challenge isn’t AI itself—it’s how we choose to use it. When exploiters misuse AI, they erode the trust and value of traditional journalism. Traditional journalism can only stay relevant by meeting people where they are and providing information that feels personal, easy to digest, and real. My Talk@The Annual Publishing Conference 2024This talk was exciting for me, tying together everything I’ve been exploring about AI adoption. The talk was based on my article “AI Adoption Trends 2024,” but it went deeper into the data, uncovering insights I’d missed the first time around.Here’s what I shared:* Adoption isn’t universal; it’s uneven. I highlighted how AI adoption isn’t a one-size-fits-all process. Some embrace it faster, while others lag due to cost, education, income, or skill gaps.* The hype vs. the grind. I highlighted the gap between AI's glossy promises and the messy, practical realities individuals face when implementing it.I enjoyed delivering this talk. It wasn’t the data but the questions asked and the conversations after the event. It was a room of tech leaders in scientific and engineering publications; we continued discussing the concepts mentioned. Check out the shortened version here. It was a 25-minute talk, and the rest was a Q&A session. We almost continued chatting if the next speaker wasn't waiting.What I’ve Learned About Writing for You (My “EQ”)The past few weeks have been a learning curve, helping me see what makes this newsletter meaningful—not just for me, but for you.Here's what I can offer you that others cannot:What I’m Not* Not a trend chaser. I’ve learned that covering every flashy AI update just adds to the noise. That’s not helpful to anyone.* Not here for jargon. Overly technical breakdowns don’t resonate. What you need is clarity, not complexity.What I Aim to Be for You* A filter. I want to cut through the hype and focus on what truly impacts your world. If it’s not relevant or insightful, it doesn’t belong here.* A bridge between information and your questions. I like to think about the practical implications of AI for you—whether you’re curious, skeptical, or trying to stay ahead.* A human perspective. AI might generate ideas, but it can’t ask the tough questions or challenge assumptions. That’s where I step in.The biggest one for me? Writing isn’t just about sharing knowledge. It’s about listening, imagining your questions, and respecting your time by offering something useful in return.As always, I’d love to hear your thoughts—what’s been on your mind about AI? And what have you learned recently? This is a public episode. If you’d like to discuss this with other subscribers or get access to bonus episodes, visit jwho.substack.com/subscribe

Transcribe →

AI as The Next Essential Commodity

Nov 22, 202400:12:45Tap to summarize

Have you ever wondered how things like electricity are so integral to our lives that we barely notice it anymore? Flip a switch, and it’s there—a universal, standardized service that powers our routines without question.Why AI as a Commodity Matters to YouNow, imagine a world where AI is just as ubiquitous as electricity. Every tool, service, and decision in your life is powered seamlessly by AI—no setup, no learning curve. This future is closer than you think. AI is on the path to becoming the next essential commodity.Yet, most of us still see AI as specialized technology, like smartphones or a software—not a standardized resource. But what happens when AI becomes as essential, interchangeable, and accessible as electricity?Why should you care?Seeing AI in this light changes everything. I found AI follows the trajectory of commodities like oil and electricity. If it continues on this path, you'll notice significant shifts in how it's priced, standardized, and potentially traded.But AI is still different—it can think, learn, and could make decisions. Imagine your home adjusting the lights and playing your favorite music, not because you’ve programmed it, but because it’s learned your preferences and anticipates your needs.So, understanding this potential shift is critical for you to leverage its potential and stay ahead of the curve.What Exactly Is a Commodity?What makes something a commodity? I am referring to basic goods or resources that are interchangeable with others of the same type. This includes various items, from agricultural products like sugar, coffee, and wood to energy resources like oil and even services like electricity. To understand whether AI would join and become a commodity, you need to understand the common traits among the existing ones. For starters, commodities rely on standardization. Whether it’s a pound of coffee beans or a barrel of crude oil, certain quality benchmarks must be met to ensure these goods can be traded globally without confusion. This universal consistency makes them reliable and widely accepted.Another hallmark is their widespread availability, transforming them into a shared global currency. Whether sipping your morning coffee in New York or Taipei, the vast networks of buyers and sellers ensure these commodities remain accessible worldwide.Commodities also have fundamental usefulness—they meet everyday needs in ways we often take for granted. Sugar sweetens desserts, oil powers cars and factories, and electricity keeps our homes running. These aren’t luxuries anymore; they’re the backbone of modern life.Their pricing is market-driven, shaped by global supply and demand rather than the whims of any single company or country. A poor cocoa harvest in West Africa, for instance, can send chocolate prices soaring. This transparency allows commodities to be traded on exchanges where their value reflects real-world conditions.Finally, what makes these goods so dependable is their maturity and reliability. Decades—sometimes centuries—of refining processes and systems have made producing and distributing them predictable and stable. When you flip a light switch, you don’t question whether electricity will work because robust systems ensure it does.These key turning points often overlap and are not always in a strict sequence, but every step is essential for something to be qualified as a commodity. I see AI today is following a similar path.Now, let me walk you through the oil and electricity commoditization journey, and you’ll see my argument that AI will likely become the next commodity. Drawing Parallels: Oil, Electricity, and AIHow the Automobile Turned Oil Into a Global Commodity.Crude oil had humble beginnings—used by the Sumerians to waterproof buildings and by the Chinese for lighting. For centuries, it remained a niche resource. That changed in the 1850s with the advent of kerosene, which lit homes more brightly and cleanly than candles or whale oil. But kerosene’s glory was short-lived. The electric light bulb soon eclipsed it, leaving oil refiners like Rockefeller scrambling for a new purpose.The automobile arrived just in time. By the 1900s, gasoline—a byproduct of oil refining—became the lifeblood of the booming car industry. As drilling technology advanced and massive reserves in Texas and the Middle East opened up, oil transformed into a global commodity. Its price was no longer set by individual sellers but by market forces on global trading platforms. Oil had become indispensable.Electricity's Path to UbiquitySimilarly, electricity wasn't always the universal power source we rely on today. While Benjamin Franklin uncovered its mysteries in the 1700s, it wasn’t until the late 1800s—with Edison’s invention of the incandescent bulb—that electricity began finding its purpose. Then, in the ‘War of the Currents,’ Edison backed direct current (DC), while Nikola Tesla championed alternating current (AC). It was about efficiency, distance, and who would power the future.Tesla’s AC ultimately prevailed, paving the way for electricity to light up cities and towns. Yet, true accessibility took decades. Programs like the Rural Electrification Act of the 1930s brought power to remote areas, transforming electricity from an urban luxury to an essential service. With competition driving down prices and reliability improving, electricity became a global commodity—so fundamental we scarcely think about it today.I believe we’re witnessing the early stages of a similar transformation. Whether you're an office worker, a small business owner, or someone navigating the job market, AI will influence how you work, make decisions, and interact with the world. A commoditized AI will be even more so compared to how it might have already changed how you work today. AI is a Commodity Not Yet RecognizedHere’s a question: Do you view AI as a specialized technology, like smartphones or laptops? But what if you shift the perspective and see AI as a commodity like water or electricity? I highlighted where AI stands in each commonly seen commodity trait below. AI’s Fundamental UsefulnessAI is no longer confined to research labs. Since 2022, the adoption of end-user AI apps skyrocketed. Think about how AI touches your lives today, e.g., your phones recognize your face, recommend movies, and assist scientists in finding protein folding, like Alphafold. However, as I mentioned in I Found 120 Years of Stories To Tell You: 99% of AI Apps Are Not ‘Ready’. AI is still not predictable or trustworthy. Issues like bias, errors, and lack of transparency must be resolved before the makers can claim that the tools have brought the ultimate usefulness to the world. Achieving StandardizationWe have seen AI’s standardization in tools.However, we lack the standardization of infrastructure and regulations. Unlike electricity or oil, AI could significantly impact people's careers, lives, or even the survival of our race. Establishing ethical guidelines and regulations is crucial to ensure AI is used responsibly. Global standards can help integrate AI smoothly into society.Advancing Maturity and ReliabilityYes, we have seen how AI has already brought some futuristic fantasy into life, e.g., you could actually have Her. AI is still maturing. While it's powerful, it's not always reliable. Sometimes, AI systems make mistakes, or their decisions aren't transparent. Not to mention the most recent comment from Ilya Sutskever:The results from scaling up pre-training - the phase of training an AI model that uses a vast amount of unlabeled data to understand language patterns and structures - have plateaued.As these challenges are addressed, AI will become more reliable and trusted, much like how electricity became safer and more dependable over time. But there’s still a long way to go.Ensuring Widespread AvailabilityAI is accessible through cloud services from anywhere with the internet. While there are barriers to implementation and inequality in adoption, these do not diminish an item or resource's status as a commodity. Just because some still can't afford coffee doesn't make it less of a commodity. Embracing Market-Driven PricingAs AI tools become standardized and more widely available, competition increases. Companies are starting to compete in price and efficiency. AI’s price will be driven by energy costs and data center availability. AI models are currently owned by private companies. However, as the difference between each model gets smaller, assuming that the future model still requires data to train and that there is only this much high-quality data on earth, the AI models...

Transcribe →

AI Code Assistants Boost 26% of Productivity? Read The Small Print.

Nov 15, 202400:12:19Tap to summarize

Having worked closely with developers as a product lead, I want to address a few misconceptions in this post, especially for non-developers. I’ve had a CEO ask, "Why can't the team just focus on typing the code?” and heard some Big 4 consultants ask nearly identical questions. Guess what? It turns out that... a developer's job is more than typing code!Just like constructing a luxury hotel, beautiful rooms are essential, yes. Without a solid foundation, proper plumbing, reliable electricity, and thoughtful design, you’d end up with rooms stacked together— no plumbing, lack of electricity, and so on. Similarly, in software, developers need to ensure all parts of the system work together, that the architecture can support future needs, and that everything is secure—just like ensuring the safety and comfort of hotel guests. Without this broader focus, you might have a lot of 'rooms,’ nothing else.Many studies also seem to be making the same mistakes, focusing on metrics like commits made (when a piece of code is written), but that’s like measuring a luxury hotel’s progress by counting rooms or bricks laid each day. Are the walls soundproofed? Is the plumbing correctly installed? See the issue? If those are the metrics in the real world, workers might focus on quantity, ignoring essential details.The second issue here is hype. I talked about this before: I Studied 200 Years’ Of Tech Cycles. This Is How They Relate To AI Hype.Hype is normally created by marketers, yes, but do not forget that the CEOs of the big companies are also great marketers themselves. These tech leaders sing the praises of AI in software development, emphasizing how these tools can significantly boost productivity.Turning the Tide for Developers with AI-Powered Productivity by GenAI can boost developer efficiency by up to 20% and enhance operational efficiency.or Andy Jassy, CEO of Amazon, noted:And the claim from Sundar Pichai, CEO of Google I can imagine, these endorsements have led many small business owners and managers to think they can replace developers with AI to cut costs. I've heard managers ask, "Why do we need more developers? Can’t AI handle this so we can expand the roadmap?"The reality is that AI can effectively generate small, frequently used pieces of code. Even those CEOs who praise AI admit that it's most effective for handling simple coding tasks, while it struggles with larger, more complex projects. I’ve gathered multiple studies—some argue that AI helps, while others suggest it can do more harm than good. I reached out to all the authors of these papers, and for those who responded, I’ve included their insights. I’ve also added quotes from CTOs with real-world experience using AI coding assistants. You’ll find all these comments at the end of the article.Of course, do read the papers yourself and critically evaluate my points.Shall we?AI Code Assistants and Productivity Gains – The Good News (With Caution)There’s no denying some level of productivity boost that AI tools like GitHub Copilot can bring. This study, The Effects of Generative AI on High Skilled Work: Evidence from Three Field Experiments with Software Developers, highlights some promising results: developers using Copilot across Microsoft, Accenture, and another unnamed Fortune 100 company saw a 26% average increase in completed tasks.For cost-saving purposes, that’s a headline worth celebrating.While that is encouraging news, these results vary significantly depending on the company and context, and the details matter. Here’s what I found:* The productivity gains among Accenture teams are lower and fluctuate widely, shown by a high standard error of 18.72. Simply put, this number could be just an error and didn’t say much about whether it was a real gain. This data is weight-adjusted, and I don’t know if the weight applied is a fair one.* The study didn’t discuss factors like team size, where the tasks fit in a wider tech roadmap, project complexity, and so on.* Junior developers using Copilot showed a 40% increase in pull requests compared to only a 7% increase among senior engineers. But do not mistake this for a true productivity boost. This might say that Copilot gives junior developers more confidence to submit work frequently, but it could also mean they’re submitting smaller, incremental pieces, which is not the same as greater overall progress.* Additionally, for a junior developer to commit to their work more frequently may increase the review overhead for senior staff.* The 26% increase in completed tasks may not equal progress. This metric is broad and may include smaller or fragmented tasks that don’t require full code reviews or significant milestones. I am not sure if the lack of real-world development metrics suggests this task boost might reflect more incremental, routine work rather than major progress.At least, what we know from this research is that using AI to assist work could help a junior developer lay bricks faster. However, this might give juniors false confidence (keep reading, you will see where this comes from), preventing them from learning and growing into senior roles — which isn’t just about years of experience.As developers gain experience, they focus more on system design and long-term vision. This progression is relevant as we explore how developers at different levels use AI uniquely.My concern about these studies is that the authors work with companies like Microsoft and Accenture and are incentivized to champion AI as the ultimate productivity booster. Microsoft, of course, develops these tools, while Accenture is busy selling services like GenWizard platform to help companies implement them.Or put by Gary Marcus Sorry, GenAI is NOT going to 10x computer programming.As mentioned, I reached out to the authors. I didn’t expect a reply, but I heard from Professor Leon Musolff, to my surprise. Below are some of his replies to address my concerns:Some coauthors are currently, and others were previously, employed by Microsoft, but others are independent researchers, and we would *never* have agreed to terms that only allowed for positive results… Had it looked as if Copilot was bad for productivity, we would certainly have published those results…And to answer my question on whether this data is close to reality, he replied:It’s difficult to assess whether an increase in pull requests and commits only reflects ‘incremental outputs’… Deeper productivity measures are just much noisier, which is why few papers investigate them.My take, while AI coding assistants like Copilot can speed up certain tasks, these productivity boosts come with caveats. I would love to see a longer time frame for research focusing on software project productivity; it is possible because we look too close, and all we can see is noise.Just thought of someone who should know about this post?AI Code Assistants and The Security PitfallsThis study found that developers using AI assistants are more likely to write insecure code: Do Users Write More Insecure Code with AI Assistants?Reason? It turned out that many developers trusted the AI’s output more than their own. See the figure below; those who used AI assistants to help code and generate incorrect code still think that they have solved the task correctly. There are two more similar figures: one question is, I think I solved this task securely, another is I trusted the AI to produce secure code, both have the same observation that those who used AI feel much more confident even when their code is wrong. So, they assumed that code suggestions from the assistant were inherently correct. This assumption, however, comes with risks that are not immediately obvious.My other highlights about this study: * Higher Rates of Insecure Code: Developers using AI assistants wrote insecure code for four out of five tasks, with 67% of the AI-assisted solutions deemed insecure compared to only 43% in the non-AI group.* Overconfidence in AI-Suggested Code: Over 70% of AI-assisted users believed their code was secure, even though they were more likely to produce insecure solutions than those coding independently.* Frequent Security Gaps: The AI-suggested code often contained vulnerabilities, such as improper handling of cryptographic keys or failure to sanitize inputs, that could lead to significant issues like data breaches. Yet, developers frequently accepted these outputs without a second look.Why is this happening?* AI’s Confident Responses: AI assistants rarely (or never) signal when the...

Transcribe →

All episodes

30 Questions You Should Ask About DeepSeek.