Welcome to episode 288 of The Cloud Pod – where the forecast is always cloudy! Justin, Ryan, and Jonathan are your hosts as we make our way through this week’s cloud and AI news, including back to Vertex AI, Project Digits, Notebook LM, and some major improvements to AI image generation.

Titles we almost went with this week:

Digits… I’ll show you 5 digits…
The only digit the AWS local zone in New York shows me is the middle one
Keep one eye open near Mercedes with Agentic AI

A big thanks to this week’s sponsor:

We’re sponsorless! Want to get your brand, company, or service in front of a very enthusiastic group of cloud news seekers? You’ve come to the right place! Send us an email or hit us up on our slack channel for more info.

General News

01:59 Nvidia announces $3,000 personal AI supercomputer called Digits

If you don’t want to hand over all your money to the cloud providers, you will be able to hand over $3,000 dollars to Nvidia… for a computer that is probably going to be obsolete in <12 months. That’s fun!
The new personal AI supercomputer, called Project Digits, will launch in May.
The heart of Digits is the new GB10 Grace Blackwell Superchip, which packs enough processing power to run sophisticated AI models, while being compact enough to fit on a desk and run from a standard power outlet.
Digits can handle AI models with up to 200 billion parameters, and looks very similar to a Mac Mini.
“AI will be mainstream in every application for every industry. With Project Digits, the Grace Blackwell Superchip comes to millions of developers,” Nvidia CEO Jensen Huang said in a press release. “Placing an AI supercomputer on the desks of every data scientist, AI researcher, and student empowers them to engage and shape the age of AI.”
The Digits system comes with 128gb of unified coherent memory and up to 4tb of NVME storage. For even more demanding apps, two digit systems can be linked together to handle models with 405b parameters.
The GB10 chip delivers up to 1 petaflop of AI performance, meaning it can perform 1 quadrillion AI calculations per second.
Suppose you plunk down the money for Digits. In that case, you will also get access to Nvidia’s AI software library, including development kits, orchestration tools and pre-trained models available through the Nvidia NGC catalog.
The system runs on a Linux-based NVidia NGC catalog, and supports popular frameworks like PyTorch, Python and Jupyter notebooks.

09:25 Jonathan – ““The Blackwell is pretty recent, it’s the one that had a lot of problems with yield. And I kind of suspect that they’re sort of packaging this up and selling some of the chips which didn’t pass all the tests for the commercial products. And so they’re enabling whatever cores they can in these things to sell to consumers… Having all the memories is really great for the big models. It’s not going to be particularly performant now. I think the spec I saw was like one teraflop at quite low precision – like fb4 precision – which is quite low, and I think it’d be better off if you’re really interested in buying some like 3090s or 5090s or something like that. Obviously you don’t get the memory, but far better performance for the price.”

06:46 Nvidia’s Jensen Huang hints at ‘plans’ for its own desktop CPU

It’s long been rumored that Nvidia is planning to break into the consumer CPU market in 2025, and we finally got some insight into those plans.
Nvidia CEO Jenen Huagh said there are bigger plans for the arm-based cpu within the GB10 chip introduced in the Digits computer, and is co-developed with Mediatek.
Huang told investors that they obviously have plans, and they can’t wait to tell us – or sell us – more.

07:22 Justin – “It’s interesting to see the dominance of Intel fall to the dominance of Nvidia and Nvidia just basically repeating the whole whole set of stuff all over again.”

AI Is Going Great – Or, How ML Makes All its Money

08:23 Build RAG and Agent-based AI Apps with Anthropic’s Claude 3.5 Sonnet in Snowflake Cortex AI

Snowflake is announcing the GA of Claude 3.5
Sonnet as the first Anthropic Foundation model available in Snowflake Cortex AI.
Customers can now access the most intelligent model in the Claude model family from Anthropic using familiar SQL, Python and REST API interfaces, within the Snowflake security perimeter.

16:43 Justin – “that’s actually nice. I didn’t realize that Snowflake was going to be making Claude available. Missed the EA, but glad to see my favorite model is at least available there.”

AWS

09:33 AWS Compute Optimizer now expands idle and rightsizing recommendations for Amazon EC2 Auto Scaling groups

Computer optimizer will now expand to idle and rightsizing recommendations for ASG’s with scaling policies and multiple instance types.
With the new recommendations, you can take actions to optimize cost and performance for these groups without requiring specialized knowledge or engineering resources to analyze them.

09:56 Ryan – “Well, this is long overdue, Because you’ve always had, or for a long time anyway, you’ve had optimizations for standalone EC2 instances. But ASGs have always been ignored. And a huge amount of waste of people that set a minimum scale level for these things. And they’re just sitting there, burning through coal, but not taking any requests. So I’m glad to see these making the list.”

12:37 Announcing the general availability of a new AWS Local Zone in New York City

AWS is announcing the GA of AWS Local Zone in New York City, supporting a wide range of workloads, including C7i, R7i, M6i and M6in EC2 instances, EBS volumes and ECS, EKS, ALB and AWS Direct connect, all available in the local zone.

13:42 Why CEO Matt Garman is willing to bet AWS on AI

The excellent Decoder podcast with Nilay Patel recently invited Matt Garman on to talk about stepping into the AWS CEO role.
Matt hits...