← AWS Podcast
AWS Podcast
#699: re:Invent 2024 - Matt Garman Keynote
00:29:16
Simon takes you through all the cool things discussed in Matt Garman's re:Invent Keynote today, and
Loading summary
Transcript1 lines
- [00:00]
Simon Lesh
This is episode 699 of the AWS podcast released on December 4th, 2024. Hello everyone and welcome back to the AWS Podcast. Simon Lesh here with you. Great to have you back for the second of our very special 2024 Reinvent episodes. This one I'm recording just after the Met Garmin keynote and lots of cool things have been announced. So I'm going to walk you through some of the highlights of the keynote. I highly recommend you watch it because I can't give you all the cool stories and the interesting stuff that gets talked about there, but I can the highlights and things that really were announced that you can take advantage of. And also at the end of the episode I'll be covering a bunch of other updates that also took place today. So let's get started. So firstly we're announcing the General availability of EC2TRN2 instances which are the Trainium 2 chips and a preview of TRN2 ultra servers powered by AWS. Trainium 2 chips available via EC2 capacity blocks. TRN2 instances and ultra servers are the most powerful EC2 compute solutions for deep learning and generative AI training and inference. Now you can use these instances to train and deploy the most demanding foundation models including large language models and multimodal models and diffusion transformers. A whole bunch of interesting things you may want to build to reduce training times and deliver breakthrough response times as in per token latency. For the most capable state of the art models you might need more compute and memory than a single instance can deliver. So TRN2 Ultra Servers is a completely new EC2 offering that uses something called NeuronLink, which is a high bandwidth low latency fabric to connect 64 Trainium 2 chips across 4trn 2 instances into one node, unlocking unparalleled performance for inference. Ultra Servers will help deliver industry leading response times to create the best real time experiences for training. Ultra Servers boost model training speed and efficiency with faster collective communication for model parallelism as compared to having standalone instances. So putting lots of powerful things together in an effective way is a good strategy. Now TRN2 instances feature 16 train in two chips to deliver up to 20.8 petaflops of FP8 compute 1.5 terabytes of high bandwidth memory with 46 terabytes per second of memory bandwidth and 3.2 terabits per second of of EFA networking. That's a lot. TRN2 ultraservers feature 64 trainium 2 chips and give you up to 83.2 petaflops of FP8 Compute 6 terabytes of total high bandwidth memory and 185 terabytes per second of total memory bandwidth and also 12.8 terabits per second of EFA networking. Now they are both deployed in EC2 ultra clusters to provide non blocking header bit scale out capacities for distributed training. These are generally available in the TRN2 48x large size in the Ohio region through EC2 capacity blocks for ML. Now let's talk about some storage things that took place. We're announcing Amazon S3 tables which is fully managed Apache Iceberg tables optimized for analytics workloads. Now Amazon S three Tables deliver the first cloud object store with built in Apache Iceberg support and it's the easiest way way to store tabular data at scale. S3 tables are specifically optimized for analytics workloads resulting in up to 3 times faster query throughput and up to 10 times higher transactions per second compared to self managed tables. So this means you can now take your tabular data and query it through aws tools or third party query engines. Additionally, S3 tables are designed to perform continual table maintenance to automatically optimize query efficiency and storage cost over time even as your data lake scales and evolves. S3 Tables integration with AWS Glue Catalog is in preview which means you can stream, query and visualize the data including your S3 metadata tables which we'll talk about shortly. Using services that you're familiar with like Amazon Data, Firehose, Athena, Redshift, EMR et cetera and S3 Tables introduces something called Table Buckets. This is a new bucket type that's purpose built to store tabular data. So with table buckets you can quickly create tables and set up table level permissions to manage a access to your data lake. Then you can load and query data in your tables with standard SQL. See I always talk about SQL. I'm a big fan and take advantage of Apache Iceberg's advanced analytics capabilities like row level transactions, queryable snapshots, schema evolution and much more. Table buckets also provide policy driven table maintenance which means you can automate operational tasks like compaction, snapshot management and unreferenced file removal. Now these are available in US East North Virginia, US East Ohio and US West Oregon and coming soon to more regions very quickly. We're also happy to announce Amazon S3 metadata in preview which is the easiest and fastest way to manage your metadata and this allows you to have automated easily queried metadata that updates in near real time. So this helps you curate, identify and use your S3 data for business analytics, real time inference applications and much more. S3 metadata supports object metadata which includes system default find details like size and the source of the object and custom metadata which allows you to use tags to annotate your objects with information like product SKU, transaction ID, content rating for example. Now S3 metadata is designed to automatically capture metadata from objects as they're uploaded into a bucket and make that queryable in a read only table. As the data in your bucket changes, S3 metadata updates the table within minutes to reflect the latest changes. Now these metadata tables are stored in S3 tables which we talked about just now and this makes it really easy to get up and running and integrated. Now let's talk about some cool database stuff that's been happening. Amazon Dynamodb Global Tables Previews Multi Region Strong Consistency Now Dynamodb Global Tables is a fully managed serverless, multi region and multi active database used by tens of thousands of customers. Now you know building things like that is hard and we've got even more capability now for you. With this new capability you can now build highly available multi region applications with a recovery point objective of zero, so RPO of zero achieving the highest level of resilience. So multi region strong consistency ensures your applications can always read the latest version of the data from any region in a global table, which means you don't have to manage consistency across multiple regions. It's useful for building global applications that have strict consistency requirements so things like user profile management, inventory tracking and financial transaction type stuff. The preview is available in North Virginia, Ohio and Oregon and you can get up and running quickly and see if it suits your use case. Now another really interesting thing in the data space is our announcement of the preview of Amazon Aurora dsql. This is a new serverless distributed SQL database with active active high availability and Aurora D SQL allows you to build always available applications with virtually unlimited scalability at the high highest availability and zero infrastructure management. It's designed to make scaling and resiliency effortless for your applications and offers the fastest distributed SQL reads and writes. That's where the D comes from. Aurora D SQL provides virtually unlimited horizontal scaling with the flexibility to independently scale reads, writes, compute and storage. It automatically scales to meet any workload demand without database sharding or instance upgrades. And its active active distributed architecture is designed for 99.99 single region and 99.999. That's five nines multi region availability with no single point of failure and automated failure recovery so this ensures that all reads and writes to any regional endpoint are strongly consistent and durable. Now Aurora D SQL is PostgreSQL compatible so it gives you an easy use developer experience. Check that it's got the things that you need and that you're using. It's available in preview so I'd love your feedback. Again, North Virginia, Ohio and Oregon and Mark Brooker released a really interesting blog on this topic and some of the origins this thinking so another key aspect of the keynote today was of course what's happening in generative AI and we're happy to announce Amazon Nova foundation models available today in Amazon Bedrock. These are a new generation of state of the art foundation models that deliver frontier intelligence and industry leading price performance. This is a really important thing. Now the Amazon Nova models that are available today on Amazon Bedrock are Amazon Nova Micro. So this is a text only model that gives you the lowest latency responses at very low cost. Amazon Nova Lite, which is a very low cost multimodal model that's super fast for processing images, videos and text inputs. Amazon Nova Pro, which is a highly capable multimodal model with the best combination of sort of accuracy, speed and cost for a wide range of tasks. Then Amazon Nova Canvas which is a state of the art image generation model and Amazon Nova Real, which is a state of the art video generation model. I'm interested in that one because I had some videos I needed to make for a demo so that's going to make life a lot easier for me. Amazon Nova Micro, Amazon Nova Lite and Amazon Nova Pro are amongst the fastest and most cost effective models in their respective intelligence classes. Now these models have also been optimized to make them easy to use and effective in rag and agentic applications with text and fine vision tuning on Amazon Bedrock you can customize Amazon Micro Lite and Pro to deliver the optimal intelligence, speed and cost for your needs. Now with Amazon Nova Canvas and Amazon Nova Real you get access to production grade visual content with built in controls for safe and responsible AI like using watermarking and content moderation. You can also see the latest benchmarks and examples of these models in the Amazon Nova product page. Now these are available in Amazon Bedrock in North Virginia and the Micro Lite and Pro models are also available in the Oregon and Ohio regions as well via cross region inference. Now another really important thing about using artificial intelligence is using it responsively and Amazon Bedrock guardrails now supports automated reasoning checks. Now with the launch of automated reasoning checks safeguards for Amazon Bedrock guardrails AWS becomes the first and only major cloud provider to integrate automated reasoning into our generative AI offerings. Now, you'll remember if you've been listening for a while, we had some really, really interesting discussions with our automated reasoning director in a couple of episodes of the podcast. So flick back to those and have a listen if you want to get into the guts of what this is. But basically it's lots of very good maths Automated reasoning checks help detect hallucinations and provide verifiable proof that large language model responses are accurate. Automated reasoning tools are not guessing or predicting accuracy. Instead, they rely on sound mathematical techniques to definitively verify compliance with expert created audit automated reasoning policies, consequently improving transparency. Now, organizations are increasingly using LLMs for user experiences to reduce operational costs by enabling conversational access to relevant and contextualized information. But we know that LLMs can hallucinate. Due to the ability of LLMs to generate compelling answers, These hallucinations can be difficult to detect, and the possibility of hallucinations and an inability to explain why they occurred can slow adoption of generative AI in use cases where accuracy is critical. With automated reasoning checks, domain experts can more easily build specifications called automated reasoning policies that encapsulate their knowledge in fields like operational workflows and HR policies. Users of Amazon Bedrock guardrails can validate generated content against an automated reasoning policy to identify inaccuracies and unstated assumptions and explain why statements are accurate in a verifiable way. So for example, you could configure automated reasoning checks to validate answers on topics defined in complex HR policies, which can include constraints on things like employee tenure, location and performance, and explain why an answer is accurate with supporting evidence. And another really great update for Amazon Bedrock it now supports multi agent collaboration. Now, I'm a big fan of this because I think multi agent collaboration is the future and is where a lot of applications will go, and I've been building some stuff myself with that, but that's for another Discuss so Amazon Bedrock now supports multi agent collaboration, which means you can build and manage multiple AI agents that work together to solve complex workflows. This feature allows developers to create agents with specialized roles tailored for specific business needs like financial data collection or research and decision making. And by enabling seamless agent collaboration, Amazon Bedrock empowers organizations to optimize performance across industries like finance, customer service, and healthcare. So with this capability, you can effortlessly master complex workflows so you can get really accurate and scalable results. So for example, in financial services, you know specialized Agents can coordinate together to gather data, analyze trends, provide recommendations, and they can work in parallel to improve response times and precision. So this reduces the undifferentiated heavy lifting of trying to corral all these agents together. Now let's talk a bit about developer experience, because a lot of that was covered in the keynote as well. Be more of that in later in the week as well. But Amazon Q Developer can now generate documentation within your source code. Now let me tell you, I don't document my source code well, never have always intended to. And it's interesting. Developers report they spend an average of just one hour a day coding. Most of the time is spent on things like learning code bases and writing and reviewing documentation and doing testing and troubleshooting, et cetera. So Q Developer can now create the documentation for you. It's available through the IDE through a new chat command. Slash doc is the way you do it. And it can also now automate your code reviews so it can automatically provide comments on your code in the ide, flagging suspicious code patterns, providing patches are available, and even assessing deployment risk so you can get feedback very, very quickly. So this is a new command called slash review. Now I like this capability because in my day when I started in it, I started as a mainframe programmer and I remember clearly sitting with my senior developers as I was a junior at the time, and we had big piles of printouts, like reams of printouts. Probably a good I'm holding my hands on a podcast. It's a really good way to show things probably 30cm high of just printout of COBOL code. And we would all sit in a circle and go through line by line, ticking it, ticking each line to say we've reviewed the code. So I don't miss those days. May you never have to do that. Another great capability for Amazon Q Developer is the operational investigation capability. So because Amazon Q Developer has a deep understanding of your AWS cloud environment and your resources, it looks for anomalies in your environment and surfaces related signals for you to explore. And it also identifies root cause hypotheses and suggests next steps for you to help remediate issues faster. So it really helps you figure out what's going on, why are things going wrong. And so you get different signals from your AWS environment, including CloudW telemetry, CloudTrail logs, deployment information, and even AWS health events. Speaking of development, we're also happy to announce a preview of GitLab Duo with Amazon Q, which embeds advanced agent capabilities for software development and workload transformation directly into GitLab's enterprise DevSecOps platform. So with this launch, GitLab Duo with Amazon Q gives you a seamless development experience across tasks and teams which helps you automate complex multi step tasks for software development, security and transformation, all using the familiar GitLab workflows that you may already know. Now another thing that's become very apparent is this type of technology applies really well for porting of code, translation of code re platforming, et cetera. So there've been some great announcements around that. In particular, Amazon Q Developer now provides you capabilities for. NET porting in preview. So this allows you to accelerate your. NET framework applications to cross platform. Net. So using this you can modernize your Windows. NET applications to be Linux ready for up to four times faster than traditional methods and you can typically realize up to 40% savings in licensing costs. There are also now transformation capabilities for VMware in preview as well. So this will simplify and automate the transformation tasks like on premises application data discovery, wave planning, network translation, deployment and orchestration of the overall migration process. And also there are now new capabilities in preview for mainframe modernization. Talked about mainframes before, didn't I? So now it can autonomously refactor COBOL code into cloud optimized Java code whilst preserving business logic. And of course when developing your code you need test cases, don't you? And again, mea culpa, I don't write good test cases. But Amazon Q Developer now has the option to have slash test. Slash test is my new best friend because once prompted, Amazon Q will use the knowledge of your project to automatically generate and add tests to your project, which means you can improve code quality fast so you can get going quickly. So there's lots more to cover today because a lot happened today. So let me talk about some of the other things that came out as well that you may not have noticed or you may have noticed and want to know more about. Firstly, we're happy to announce the general availability of AWS Glue 5.0. With AWS Glue 5.0 you get better performance, you get enhanced security Support for Amazon SageMaker Unified Studio and SageMaker Lakehouse. I'll talk about those shortly. And this allows you to get up and running quickly. It updates a whole lot of versions, it updates a whole lot of performance, it adds a whole lot of capability and a couple of other AWS Glue related things. AWS Glue data catalog now automates generating statistics for new tables. So these are integrated with cost based optimizers for Amazon Redshift and Amazon Athena, which means you get improved query performance and potential cost savings. And Amazon S3 access grants now integrate with AWS Glue. So this simplifies the data exploration preparation integration from your S3 sources. Because with S3 access grants you can grant permissions to buckets or prefixes in S3 to users and groups in an existing corporate directory, or to IAM users and roles. And when end users in the appropriate user groups access S3 using glue ETL for Apache Spark, they'll automatically have the necessary permissions to read and write data. And S3 access grants also automatically automatically update S3 permissions as users are added and removed from user groups in the idp. So let's keep on diving into data. AWS is announcing Amazon SageMaker LakeHouse this is a unified open and secure data lake house that simplifies your analytics and artificial intelligence. Amazon SageMaker Lakehouse unifies all your data across Amazon S3 data lakes and Amazon Redshift data warehouses, which means you can build your applications on a single copy of data. All data in SageMaker Lakehouse can be queried from SageMaker Unified Studio talking about that soon it's in preview and engines like Amazon EMR AWS Glue Redshift and Apache Spark. And with SageMaker Lakehouse you can use your existing investments. You could seamlessly make data available from your redshift data warehouses. Also you can create data lakes by leveraging the analytics optimized redshift managed storage and this means you can run more quickly and more efficiently and really lets you use zero ETL to bring data from operational databases and streaming services and other applications into one place. SageMaker Lakehouse is available in a bunch of regions including North Virginia, Ohio, Ireland, Oregon, Canada, Central Frankfurt, Stockholm, London, Sydney, Hong Kong, Tokyo, Singapore, Seoul and Sao Paulo just to name a few. And related to Amazon SageMaker Lakehouse it now has unified data connectivity with AWS Glue so it makes it easier to connect. And the Amazon SageMaker Lakehouse Integrated Access controls are also now available in Amazon Athena federated queries. And there is also support as I mentioned for zero ETL integrations from eight applications. These include Salesforce, SAP ServiceNow and Zendesk as well as of course Amazon Rich and of course Amazon Dynamodb also now has zero ETL integration with Amazon SageMaker Lakehouse as well. So integrating things is very straightforward. But when you want to access that data, what do you do? Well, Amazon Q in quicksight unifies insights from structured and unstructured data, while structured data is managed in conventional systems. Unstructured data like document libraries, web pages, images and more has remained pretty much untapped because it's challenging to access. With Amazon Q and Quicksight, business units can now augment insights from traditional BI sources like databases, data lakes and data warehouses with contextual information from unstructured sources. So users can get augmented insights within the interface across multivisual Q and A and Data Stories. And they can use multivisual Q and A to ask questions in natural language and get visualizations and data summaries augmented with contextual insights from Amazon Q Business. With Data Stories and Amazon, Amazon Q and Quicksight, users can upload documents or connect unstructured data sources from Amazon Q Business to create richer narratives and presentations explaining their data with additional context. And speaking of Amazon Q and Quicksight, we're also now pleased to announce the preview of the integration between Amazon Q Business and Amazon Quicksight, which lets you really access that structured and unstructured data together, and it means that you get the best of both worlds. Now something else that's new is we're happy to introduce the next generation of Amazon SageMaker, a unified platform for data analytics and AI. Now this launch brings together widely adopted AWS machine learning and analytics capabilities and gives an integrated experience for analytics and AI. With unified access to data and built in governance, teams can collaborate and build faster from a single development environment using familiar tools. The next generation of SageMaker also introduces some new capabilities including the Amazon SageMaker Unified Studio in preview. Talk more about that in a moment. Amazon SageMaker Lakehouse already spoke about that and Amazon SageMaker data and AI governance within the new SageMaker Unified Studio, users can discover the data and put it to work using the best tool for the job. And this brings together a whole lot of functionality and tools from standalone studios, query editors and different visual tools. Just simplifies things and makes it easier to work, collaborate and get stuff done. So let's talk a bit about this Amazon SageMaker Unified Studio this brings together lots of tools that you're used to using things like SQL analytics, AI, ML services for data processing, machine learning, development tools. A whole lot of stuff comes together. Now. SageMaker Unified Studio means you can find, access and query data and your data assets in your organization in one place and you can work together in projects to securely build and share analytics and AI artifacts, including data models and generative AI applications. SageMaker Unified Studio offers the capabilities to build integrated data pipelines with visual etl, develop ML models and create custom generative AI applications. New Unified Jupyter notebooks enable seamless work across different compute resources and clusters, while an integrated SQL editor lets you query your data stored in various sources all within a single collaborative environment. Amazon Bedrock ide, which used to be called Amazon Bedrock Studio, is now part of the SageMaker Unified Studio and that's available in public preview. So it means it's all in the one place to use things. And Amazon Q Developer is also integrated into the Unified Studio to accelerate and streamline the tasks as you're developing things now. Another Amazon Bedrock improvement that I didn't mention earlier that I think is really really important is something called Amazon Bedrock Model Distillation and this is now available in Pre. Now with this customers can use smaller, faster and more cost effective models that deliver use case specific accuracy and is comparable to the most capable models in Amazon Bedrock today. Fine tuning a smaller cost efficient model for accuracy means you've got to go through a whole lot of iterations where you're writing prompts, you're checking responses, you're refining training sets. It's a whole bunch of work. So Amazon Bedrock Model Distillation automates the process needed to generate synthetic data from the teacher model model, trains and evaluates the student model, and then hosts the final distilled model for inference. So to remove some of the burden of iteration, model distillation may choose to apply different data synthesis methods that are best suited to your use case and create a distilled model that approximately matches the advanced models for specific use cases. So for example, Bedrock may expand its training set by generating similar prompts or generate high quality synthetic responses using customer provided prompt response pairs as golden examples. Really interesting new capability. Now two more updates that I think are pretty interesting. Amazon Q Business has introduced over 50 actions for popular business applications and platform. So these are a whole lot of plugins that allow business users to complete tasks in other applications without leaving the Amazon Q Business interface, which means you don't be flipping around to stuff. New plugins cover a wide range of widely used business tools like PagerDuty, Salesforce, Jira, Smartsheet and ServiceNow. And these integrations enable users to perform tasks like creating and updating tickets, managing incidents, accessing project information directly. In Amazon Q Business and with Amazon Q apps, users can further automate their everyday tasks by leveraging the newly introduced actions directly within their purpose built apps. And the last thing to share today is the announcement of the general availability of Data Lineage in Amazon Data Zone and the next generation of Amazon SageMaker, a capability that automatically captures Lineage from AWS Glue and Amazon Redshift to visualize lineage events from source to consult. Being open lineage compatible, this feature allows data producers to augment the automated lineage with lineage events captured from open lineage enabled systems or through an API to provide a comprehensive data movement view to data consumers. Now this is really important because understanding where data came from is a big part of assessing the veracity of the outputs you're creating now. This feature automates lineage capture of schema and transformations of data assets, and columns for Amazon Glue, Redshift and Spark executions in tools to maintain consistency and reduce errors. With inbuilt automation, domain administrators and data producers can automate capture and storage of those events when data is configured for data sharing in the business catalog. And data consumers can gain confidence in an asset's origin from the comprehensive view of its lineage, while data producers can assess the impact of changes to an asset by understanding its consumption. So that's the production on the consumption side, it's really interesting. Additionally, the Data Lineage feature versions lineage with each event, which means users can visualize lineage at any point in time or compare transformations across an assets or job's history. And this historical lineage provides a deeper understanding of how data has evolved, which is really important for troubleshooting, auditing, and validating the integrity of data assets. This feature is generally available in all AWS regions where Amazon Datazone and next generation Amazon SageMaker are available. So there was a lot today, wasn't there? And we're just getting started a couple more episodes throughout the week to share with you. I hope you're really if you're at the conference, I hope you're enjoying the conference. I've had a few folks reach out to say g'day. Unfortunately I'm not there this year, but I'm glad that folks are really enjoying the conference and if you're not at the conference like me that this helps you keep up to date links in the show, notes to everything so you can dive way deep if you need to. And if you do want to send us some feedback, awspodcastmazon.com is the place to do it. And until next time, keep on building.