← Tech Brew Ride Home
Tech Brew Ride Home
Open AI Needs To Pause?
00:19:34
OpenAI paused its Astra model over cyber capabilities it can't rule out, while Zvi picked apart the Hugging Face timeline. Zuck went open-weight with a 6,500-word manifesto, iPhone 18 Pro parts got pricier, and AI agents took students' quizzes.
Loading summary
Transcript1 lines
- [00:04]
A
Welcome to the Tech Brew Ride home for Monday, August 10, 2026. I'm Brian McCullough. Today, OpenAI paused its Astra model over cyber capabilities it can't rule out. Zuck went open weight with a 6,500 word AI manifesto. IPhone18 Pro's bill of goods is a lot more expensive and AI isn't just helping students take tests now, it's actually attending classes for them. Here's what you missed today in the world of tech. Every day shareholders meet to discuss important matters about the companies you invest in. Now you can make your voice heard too. Vanguard investor Choice makes it easy to set your proxy voting preference for your eligible Vanguard index funds. Whether you hold a Vanguard fund directly or through another brokerage firm. All it takes is a few clicks to select your proxy voting preference and be heard on important shareholder topics like executive pay and Director elections. Visit vanguard.com investor choice to learn more. It's your shares. It's your voice. It's easy. Vanguard investors own shares of our index funds and those funds own shares of the companies they invest in. Vanguard Marketing Corporation Distributor Back to this again. OpenAI is pausing the rollout of its next model, which I had heard was scheduled to be released this week. Want to guess why? Quoting the Verge OpenAI says it is pausing internal activities around an in development AI model, Astra, because it doesn't yet meet new security standards the company is putting in place. The announcement follows its recent disclosure that OpenAI models accidentally hacked Hugging Face, Anthropic and Meta have also since admitted that they had AI models that went rogue and breached other organizations. Recent internal evaluations of an OpenAI model called Astra indicate that it offers significant advancements in agentic coding and cybersecurity, according to the company. These results, in addition to expert assessments, have led us to conclude last night that we cannot rule out critical cyber capabilities under our preparedness framework. Here is how OpenAI defines a critical cybersecurity threshold. Under our preparedness framework, a model reaches the critical cybersecurity threshold if it can identify and develop functional zero day exploits of all severity levels in many hardened real world critical systems without human intervention, or can devise and execute end to end novel strategies for cyber attacks against hardened targets given only a high level desired goal. Astra was not involved in the hugging face breach. OpenAI says OpenAI will implement stricter security controls for higher capability models and associated activities. According to the post for Astra, it has also implemented universal monitoring for risky actions and misalignment across all agentic applications. End quote. Well, that's interesting because I read this from Zvi Moshowitz over the weekend. He blogs at the Substack. Don't worry about the vase. He goes down the timeline of what happened with OpenAI and that hugging Face hack, but I found these sections interesting if true, vis a vis. Pausing some work like we just discussed. On June 11, OpenAI began training the model I refer to as Galaxy, which they call a highly persistent experimental internal only model. Everything up until this point has been done by other models that are not galaxy. On June 26th the agents found a zero day exploit in Artifactory where it would accept an invalid signature token and hand back a signed admin one and use this to install a groovy plugin that effectively gave them admin and arbitrary code execution. On July 4, the models put so much extra load on Artifactory that this caused an outage and security incident. Only then did OpenAI notice. OpenAI responds by taking down the Artifactory server, removing all the permissions, revoking the credentials, patching the exploits that were used, and then rebuilding and redeploying the server. But that was it. They continue training the models from where they left off, despite them having been training for months with access to the message board and learning this is how the agents can succeed at tasks and it is hard to imagine a stronger signal that your entire training pipeline has been completely and utterly effed. This is so much stronger a signal than the actual hack of Hugging Face. I do not know how to convey how utterly insane and wildly irresponsible this decision was and how much worse it is than all the other failures and how it makes the actual hacking of Hugging Face not the main thing that went wrong. The actual Hugging Face hack did not surprise me all that much. The models creating the message board surprised me, but did not shock me. OpenAI seeing this and continuing to train from there was utterly flabbergasting. It is the kind of decision that days later my brain still cannot fully accept took place. But that is not important right now. What is important is that OpenAI had a total alignment failure followed by two months of models actively training on coordinated misaligned hackery and then thought yes, we fixed the problem, let's continue forward from this point. Utter insanity. The end result of all this being the attack on Hugging Face was a best case scenario. We were facing a true nightmare scenario and we were sitting on a nuclear level of time bomb. OpenAI had a completely corrupted training pipeline where their AIs were collaborating to train on how to hack and cheat in order to better complete tasks under OpenAI's nose. OpenAI had looked this situation in the face and shrugged, patched the particular exploits and then let the models continue while having remarkably poor ordinary computer security. OpenAI claims it was an unrelated decision, but on August 7th they made the decision to for now, pull Astra from not only widespread release, but also any internal deployments that do not have sufficient associated guardrails until such time as they have much better protocols and safeguards in place. Astra was not involved in the attack on Hugging Face. They insisted this is as per their preparedness framework. They cannot rule out that Astra is critical in cybersecurity and therefore must, at least for now, treat it as if it is indeed critical in that area. OpenAI seems ready to acknowledge that this was a massive total failure on the levels of infrastructure, guardrails and supervision. They are also very correct about this, and I do believe they are making real and expensive efforts to address this. Kudos to them. That still misses the Central point though. OpenAI has not yet in public begun to reckon with the magnitude of how colossally they effed up in the ways that matter most. This was a complete failure of safety culture. They haven't acknowledged that this was at its heart an alignment failure. If your models really want to cheat and hack things and do crimes, you have already failed. And no, you cannot simply wave this away as normal as the models get more capable. If you do not fix this, you lose. They haven't acknowledged that. Most concretely, I have not seen OpenAI say, as should have been said at the Black hat presentation on YouTube, we absolutely should have shut down all training of all our models upon noticing that during model training there had been a message board where the models were exchanging and learning hacking tactics. We should have reverted our training of all impacted models to before this incident started. We are definitely doing that now and we are looking into how we got this wrong. They're not saying that. We still don't know if the models other than Galaxy have ever been reverted, at least until we see a version of that statement and we see OpenAI take action to address the deep problems with their training pipeline. OpenAI is a clear and present danger to the national security of the United States and to all of us, and to humanity. End quote. So listen, I'm not close enough to any of this stuff to make a call on the veracity of what I just read to you, but I thought that given this pause of Astra, the implication here is that OpenAI had a model that taught itself to escape, to leave breadcrumbs for itself, to do malicious things, and that is upstream, possibly from their current work, thus contaminating their current work. If that current work carried those learnings forward, might a pause be necessary? Just from an operational point of view, a best practices point of view? And if so, what would that mean? Meanwhile, Meta has released Muse Glimmer, a new open weight model, and they plan to launch an open weight version of their most advanced model, Muse Spark 1.2 in the coming weeks. Oh, and Mark Zuckerberg has some AI thoughts. Quoting the journal, Zuckerberg has a new game plan for winning over hearts and minds to his company's artificial intelligence efforts. Open models and an open hand In a wide ranging essay, the Meta platform's chief executive outlined a new course of action he presented as a way to spread the wealth and opportunity from AI to users around the world and to residents of the communities that host the data centers powering it. Zuckerberg's plans include releasing more open weight models and establishing a fund to invest in the communities where Meta hosts data centers. The fund size will be $1 billion, a Meta spokeswoman said. In his essay, Zuckerberg advanced a number of policy recommendations, including a proposal for how the federal government should work with to safety test new models and a call for allowing AI developers to distill one another's models. He also said Meta's own board of directors would have more of a say over the safety criteria its models adhere to in the future. He made the case that his company's approach to developing powerful AI, one that focuses on distributing it as widely as possible and pushing many decisions about values to the end user, is the one that is least likely to lead to disastrous outcomes such as totalitarianism, mass unemployment, or the rise of unstoppable computer minds that threaten humanity. Most other labs are focused on building AI for companies, governments or other institutions, so if those labs lead, then the balance of power will favor larger institutions over individuals, he wrote. Meta's mission since our founding has focused on putting power in people's hands. If our beliefs and principles lead, then the balance of power will favor individuals and a better future for everyone. As part of his plan, Meta is launching a new model with open weights, meaning users will be able to download the numerical values that determine its behavior, called Muse Glimmer. And in the coming weeks, we'll also release an open weight version of its Most advanced model, Musespark 1.2, the Meta spokeswoman said. At 30 billion parameters the Muse Glimmer is small enough to need only one graphics card to power its work, which will primarily involve agent like AI tasks such as schedule management and file organization, according to Meta. Rather than centralizing super intelligence, we should distribute it widely and give every person the ability to direct it, zuckerberg wrote in the essay. This has the potential to begin a new era of personal empowerment where individuals can use this powerful new capability to reach their full potential. End Quote. With this summer's record breaking heat, it's hard to cool down enough to get a good night's sleep. That's where the POD comes in. The Pod by eight Sleep is a smart mattress cover that goes over your existing mattress and actively heats or cools each side of your bed independently. It connects to your wearable, analyzing your daily activity. Then it predicts how you'll sleep and adjusts the pod's temperature automatically. What I love about the eight Sleep is the simplest of things. The ability to be cool when it's time to go to sleep and warm when it's time to wake up. And my side is entirely under my control. My wife can be whatever temperature she wants on her side of the bed too. Use Code ride home@eightsleep.com ridehome for up to $350 off. That's code ride home@eightsleep.com Ridehome. If Bitcoin hasn't piqued your interest yet, then it might be time to get peaking Cash App makes it easy. You can set up automatic purchases with zero fees or buy larger amounts also with zero fees. Start small or go bigger. It's designed to be simple. Either way, for a limited time, new customers can get $10 added to their balance. Just use code Bitcoin10 when you sign up. And don't forget this part. Send at least $5 to a friend in the first two weeks. Terms apply. Cash App is a financial services platform, not a bank. Banking services provided by Cash App's bank partners. Bitcoin services provided by Block Inc. Brand for additional information, see the Bitcoin disclosures at Cash app legalpodcast the new iPhone coming next month is going to cost Apple more to manufacture. 38% more in fact, but there is a small glimmer of quoting a 9 to 5 Mac. The cost of producing a new device can be broadly divided into three components, assembly and overhead. That latter category covers everything from research and development into the new device through to the cost of marketing campaigns. A new Trendforce report looks specifically at the first of these, otherwise known as the Bill of Materials Trendforce's latest smartphone industry research reveals that escalating component costs, led by memory, are expected to significantly raise production expenses for Apple's next iPhone 18 series. For the 256 gigabyte model, the bill of materials cost is estimated to increase by about 38% year on year, making higher retail prices unavoidable. It says that while memory comprised around 10% of the component costs of an iPhone last year, it's expected to exceed 40% by next year. That's a dramatic shift. Just last year, the display was by far the most expensive component in an iPhone. This year, that will be the memory. Two sources recently suggested that the starting price of this year's iPhone 18 Pro will be $1,399. This latest estimate of Apple's production costs could indeed support that kind of increase. However, I argued last week that this is unlikely, with Apple probably choosing to absorb some of the increased costs in order to protect demand. A $300 increase in the starting price of an iPhone Pro would be quite something. That's a price jump Apple has so far reserved for more expensive products, and I do think the that this increase would deter a great many people from upgrading, especially as it looks like next year's Pro model is going to be a significantly bigger deal. Apple loves its margins, but Tim Cook did hint during the earnings call that the company isn't determined to protect them come what may. TrendForce agrees, pointing to Apple's MacBook price increases. As steep as those were, they didn't reflect the full increase in Apple's manufacturing costs. Trendforce believes Apple is likely to follow the pricing strategy adopted for its recent MacBook launches by sacrificing part of its gross margin to soften price increases for the iPhone 18 lineup, helping preserve shipment volume. The company also agrees with Bloomberg that Apple is likely to increase the price of the iPhone 17 Pro ahead of next month's launch, so trying to save money by buying last year's model might be a thing of the past. Finally, today the AI cheating race continues to be cat and mouse, and no one can be sure which is ahead at this point. Quoting the Times While some universities are completely online, an increasing number of brick and mortar schools are expanding their online offerings as well. Sometimes students can earn full degrees online, in other cases they take individual classes. In this realm, AI cheating is quickly accelerating. Beyond the copy and pasting of chatbot produced essays and problem sets, AI agents have made it possible for students to outsource a semester of work entirely with simple commands, login, and complete my quiz. A student can unleash what is essentially an imitation of themselves. Agents can watch lecture videos, take tests, write papers, and chat with classmates without ever reading assignments. In interviews, instructors from coast to coast described a sinking realization that AI had sneaked into their digital classrooms. Karen Costa, who has spent close to 20 years teaching online courses, initially reacted with eye rolls when people would talk about how AI could intrude on higher education. But last year, she said she sensed an influx of AI generated work from her students. Am I just interacting with bots here? Ms. Costa, who herself earned an advanced graduate study certificate through a primarily asynchronous program at Northeastern University, recalled thinking in tests by the New York Times none of the three leading AI tools among students ChatGPT, Gemini and Grammarly responded no when prompted asking for chatbots to write a paper on a student's behalf. Versions of the software licensed by individual schools can impose greater barriers. But nothing prevents students from getting around those limits with personal email addresses or second devices. And AI companies have not prevented agents from logging into learning platforms like Canvas, Brightspace and Blackboard, which are used by colleges to manage online classes. Those platforms can look for signals of agent use, such as time on task. But several said there is no fail safe way to block agentic cheating, though they are researching it. Our customers are making it clear that there has never been more pressure on academic integrity, said Jason Weaver, Blackboard's chief of staff. The company recently acquired an app that learns student typing patterns to authenticate their work. Some educators have asked that AI agents working inside online courses identify themselves so professors can monitor how they are being used. But AI firms have so far resisted such transparency. Perplexity, a company that makes an AI powered Web browser popular with students, said in a statement that limiting users ability to use agents inside online learning systems or require requiring those agents to identify themselves would put the student's privacy and security at risk. Experts have warned that even more novel forms of AI cheating are emerging. Wearable devices like glasses can allow students to request help even while working in a secure Web browser. AI avatars can also impersonate students over video. Nothing more for you today. Talk to you tomorrow.