← AWS Podcast
AWS Podcast
#698: re:Invent 2024 - Monday Night Live with Peter DeSantis
00:25:32
Simon takes you through the first updates from a BIG week of re:Invent 2024! Shownotes: https://d29
Loading summary
Transcript1 lines
- [00:00]
A
This is episode 698 of the AWS podcast released on December 3rd, 2024. Hello everyone and welcome back to the AWS Podcast. Simon Lejibwe with it. Great to have you back for the first of our very special re invent 2024 episodes. This is being recorded just after Peter DeSantis Sunday Night Live event where he talked a lot about some really interesting innovations around chips, around networking and a whole bunch of other cluster stuff. So if you want to geek out on technology, check the replay of that keynote. Definitely very very cool. Now we have lots of updates to share. This is the first sort of tranche of updates so we'll get them across to you so you can keep up to date. Let's start with the topic of the AWS Marketplace and now offers EC2 image builder components from independent software vendors so you can find and subscribe to them and you can then incorporate them into into your own golden images through the Image builder. Makes it nice and easy. There's also a new streamlined deployment experience for data bricks on AWS so you can jump into the AWS Marketplace and get up and running very very quickly. Let's talk About Analytics. Amazon OpenSearch Service 0ETL integration with Amazon Security Lake is now available. This integration allows you to efficiently explore voluminous data sources that were previously cost prohibitive to analyze, helping you streamline your security investigations and get comprehensive vis of your security landscape. So you can now get up to date information very very quickly and you can selectively ingest data and eliminate the need to manage complex data pipelines and Amazon CloudWatch and Amazon OpenSearch services also launched an integrated analytics experience and zero ETL integration between the two services. So you can now take advantage of OpenSearch's piped processing language PPL and OpenSearch SQL. Additionally, CloudWatch customers can accelerate troubleshooting with out of the box curated dashboards for vendor logs like VP CloudTrail and WAF. And it means that OpenSearch customers can now analyze CloudWatch logs without having to duplicate data. And AWS Clean Rooms has announced support for collaboration with datasets from multiple clouds and data sources, so this allows companies and their partners to easily collaborate with data stored on Snowflake and Amazon Athena without having to move or share their underlying data among collaborators. Quick update for Application integration Amazon EventBridge and AWS STEP functions now have integration with private APIs and this is powered by AWS PrivateLink and Amazon VPC Lattice, which makes it easier for customers to accelerate the innovation and modernize their applications with fully managed connectivity to private HTTPs based APIs. Customers can now securely integrate their legacy systems with cloud native applications using event driven architectures and workflow orchestration, which means you can innovate faster and you can have higher security and regulator and you can have higher security and better regulatory compliance. Now let's talk about artificial intelligence and machine learning. We are happy to introduce latency optimized inference for foundation models in Amazon Bedrock. This is available in public preview and it gives you faster response times and improved responsiveness. Currently these new inference Options support anthropic Claude 3.55 haiku Meta's Llama 31, 405 bill and 70 billion and Meta's Llama 3.145 b and 70 b models offering reduced latency compared to standard models without compromising accuracy as verified by Anthropic. With latency optimized inference In Amazon Bedrock, Claude 3.5 haiku runs faster on AWS than anywhere else. Additionally, with latency optimized inference in Bedrock, those LLAMA models also run faster than any major cloud provider. Amazon Bedrock Knowledge Bases now supports rag evaluation in Preview. This capability allows you to evaluate your retrieval, augmented generation or rag applications built on Amazon Bedrock knowledge bases. You can evaluate either information retrieval or the retrieval plus the content generation and these evaluations are powered by LLM as judge technology. With customers having choice of several judge models to use. For retrieval evaluation you can select from metrics like context, relevance and coverage. For retrieve plus generational evaluation you can select from quality metrics like correctness, completeness and faithfulness or hallucination detection as well as responsible AI metrics such as harmfulness, answer refusal and stereotyping. You can also compare across evaluation jobs in order to compare knowledge bases with different settings like chunking strategy or vector length and different content generating models. And Amazon Bedrock now supports rerank API to improve the accuracy of those rag applications that we were just talking about. The reranker models rank a set of retrieved documents based on their relevance to users queries, helping to prioritize the most relevant content to be passed to the foundation model for response generation. Now, semantic search in RAG systems can improve document retrieval relevance but can struggle with complex or ambiguous queries. For example, a customer service chatbot asked about returning an online purchase might retrieve documents on both return policies and shipping guidelines. Without proper ranking. The generator response could focus on shipping instead of returns missing the user's intent. Now Amazon Bedrop provides access to RE ranking models which will address this by reordering retrieved documents based on their relevance to the user query. This ensures the most useful information is sent to the foundation model for response generation, optimizing the context window usage and also potentially reducing costs. Now Amazon Bedrick model of Now Amazon Bedrock Model evaluation now includes LLM as a judge in preview. We talked about this just briefly in relation to another update. This lets you choose an LLM as your judge to ensure you have the right combination of evaluator models and models being evaluated. You can choose from several available judge LLMs on Amazon Bedrock. You can also select curated quality metrics like Correctness, completeness and Professional style and tone, as well as responsible AI metrics like harmfulness and answer refusal. You could also now bring your own prompt dataset to ensure the evaluation is customised for your data and you can compare results across evaluation jobs to make decisions faster. And Amazon Bedrock Knowledge bases now provide auto generated query filters for improved retrieval. So this allows you to improve retrieval accuracy by ensuring the documents retrieved are relevant to the query. And this extends the existing capability of manual metadata filtering, which means you can do it without having to do it manually. Amazon Bedrock Knowledge bases also now support custom connectors and the ingesting of streaming data. This is a cool one. So this enhancement enables customers to ingest specific documents from any custom data source and reduce latency and operational costs for intermediary storage. While ingesting streaming data. For instance, you can now keep a knowledge base continuously updated with the latest market data, meaning that you're getting the most up to date information. You don't have to sort of do a periodic resync or full sync. You can do a streaming sync. And speaking of streaming, Amazon Bedrock Knowledge bases now support streaming responses as well. So this is the Retrieve and Generate Stream API for knowledge bases and this allows knowledge base customers to receive the response as it's being generated by the LLM rather than waiting for the complete response. And if you enjoy building gen AI apps, Party Rock has improved app discovery and announces upcoming free daily use. So it is now easier to find your applications using search and Explore and Build. And also there'll be a new and improved free daily usage model which will replace the current free trial grant in 2025. Amazon Q Business now adds support to extract insights from visual elements within documents, so now it can answer questions and get information about things it sees in your doc documents. So with this launch you can now uncover valuable insights within visual content. Things like PDFs, Microsoft PowerPoint and Word, Google Docs and Google Slides. And Amazon Q Business provides transparency by surfacing the specific images utilized to generate the responses, which means you can contextualize the extracted information. Now we have some fantastic compute update, the first being the general availability of the EC2 P5EN instances. Powered by the latest Nvidia H200 tensor core GPUs, these instances deliver the highest performance in Amazon EC2 for deep learning and high performance computing applications. These have up to 8 of the H200 GPUs which have 1.7 times GPU memory size and 1.5 times GPU memory bandwidth than the H100 GPUs and these instances pair those GPUs with high performance 4th generation Xeon scalable processors which means you can get up to 4 times the bandwidth between the CPU and GPU which boosts performance even further. There is also now access to up to 3200 gig per second of third generation EFA network access using Nitro V5 so you're getting up to 35% improvement in latency as well and a whole bunch of other capabilities. These are now available in U.S. east Ohio, U.S. west Oregon and Asia Pacific Tokyo as well as U.S. east Atlanta. We're also happy to announce the general availability of EC2 storage optimized i8G instances. These are the best performance for EC2 for storage intensive workloads. They're powered by AWS Graviton 4 processors that deliver up to 60% better compute performance than the previous generation i4G instances. The i8G instances use the latest third generation AWS Nitro SSD's local NVMe storage that give you up to 65% better real time storage performance per terabyte whilst giving you 50% lower storage I O latency and up to 60% lower IO latency variation. It's all kinds of good. You can get instance sizes up to 24x large which gives you 768 gigabytes of memory and 22.5 terabytes of instant storage. What a world we live in so you can get up and running with those. We're also happy to announce the next generation high density storage optimized i7 IE instances. Again large data intensive workloads. These run the fifth generation Intel Xeon scalable processors with an all core turbo frequency of 3.2 GHz which gives you up to 40% better compute and 20% better price performance than the i3en instances. As a reminder, always check to see if you're up to date with your instance types. You can get lots of benefits, lots of other improvements in terms of speed, the SSDs, et cetera. Again, it's all goodness and Amazon EC2 has also introduced the allowed AMIs to enhance your AMI governance. This is a new account wide setting that allows you to limit the discovery and use of Amazon machine images within your AWS accounts. You can now simply specify the Aware accounts or the AMI owner aliases permitted within your account and only amis from those owners will be visible and available to you to Launch in your EC2 instances. Amazon CloudWatch Container Insights has launched enhanced observability for Amazon ECS so you can see more and we're also happy to announce Amazon EKS Hybrid Nodes. With EKS Hybrid nodes you can use your on premises and Edge infrastructure as nodes in Amazon EKS Clusters and Amazon EKS Hybrid nodes unifies Kubernetes management across environments and offloads. The kubernetes control plan management to AWS for your on premises and edge applications. There's also a new capability called Amazon EKS Auto Mode and this is a new feature that fully automates Compute storage and networking management for Kubernetes clusters. Amazon EKS Auto Mode simplifies running Kubernetes by offloading cluster operations to aws, which improves the performance and security of your applications and helps optimize compute costs. Another new update related to AWS Outposts customers can now attach block storage volumes backed by NetApp on premises Enterprise Storage Arrays and Pure Storage Flash array to Amazon EC2 instances on AWS Outposts directly from the AWS Management Console. So this makes it easier to leverage third party storage with Outposts when you use it in that configuration. And the final topic for Compute we're happy to announce the Amazon Elastic VMware service in preview. It's a new native AWS Service to run VMware Cloud Found or VCF within your VPC. The goal here is to automate and simplify deployments and gives you a ready to use VCF environment on aws, which means you can get up and running quickly and use the same software and tools that you use on your on premises environment. Now let's talk about customer engagement and the Amazon Connect team have been very very busy. A bunch of updates that will run through here. The Connect Content lens now automatically categorizes your contacts using Generative AI. Amazon Connect has also launched AI guardrails for Amazon Q in Connect so you can limit what happens and how things operate. There are also new Intraday Forecast dashboards so you can see what's going on every 15 minutes with predictions for the rest of day, call volumes and answer queue times, et cetera. There's also a new AI assistant for customer segments and trigger based campaigns and you also have the ability to record audio during IVR and other automated interactions. And Amazon Connect Contact Lens now supports external voice so you can use your existing voice system to help improve your customer experience. You can also now do external voice transfers as well to an existing voice system. And there are now built in dashboards for Amazon Connect Contact Lens to analyze your conversational AI bot performance. And there's also simplified conversational AI bot creation. And you can also automate your agent performance evaluations using generative AI as well. But they're not done because Amazon connect now supports WhatsApp business messaging as well, so you can now integrate that into your environment to deliver personalized experiences to your customers who use WhatsApp. And there is also now generative AI powered self service with Amazon Q in Connect so this allows you to directly converse with end customers and reason over undefined intents for those more ambiguous scenarios and give customers accurate responses. And there's also a new Salesforce Contact center with Amazon Connect in Preview. This is a groundbreaking offer that integrates native digital and voice capabilities into Salesforce Service Cloud, delivering a unified and streamlined experience for agents. And finally, Amazon Connect now makes it easier to collect sensitive customer data within chats. You can now support inline chat interactions like processing payments, updating customer information or collecting customer data without requiring the customer to switch channels or navigate to another page on your website. Let's talk databases. We're happy to announce the general availability of Amazon MemoryDB multi region which is a fully managed active active multi region database that lets you build multi region applications with up to five nines of availability and microsecond read and single digisec and single digit millisecond write latencies. MemoryDB is a fully managed VAL key and Redis OSS compatible database service providing multi az durability, microsecond read and single digit millisecond write latency and high throughput. And valkey is an open source high performance key value data store stewarded by the Linux foundation and it's a drop in replacement of Redis oss. So this lets you build really cool applications across the globe. If you ever tried to build anything multi region when you've had to coordinate all the management of the data across those regions it can be hard. Well our old friend undifferentiated heavy lifting goes away with Amazon MemoryDB multi region and Amazon and Oracle database at AWS is now in a limited preview. This lets customers access Oracle Database services in Oracle Cloud Infrastructure managed exadata infrastructures within AWS data centers and customers can easily and quickly migrate their Oracle database workloads, including real application cluster workloads to the service within AWS with minimal to no changes. All right, let's talk management and governance. Amazon CloudWatch has added network performance monitoring for AWS workloads using flow monitors and AWS has announced Amazon CloudWatch database insights so you can consolidate logs and metrics from your applications, your databases and the operating system they run on, and a unified view in the console with pre built dashboards. We're also happy to announce the general availability of Invoice Configuration which lets you customize your invoicing experience to receive separate AWS invoices based on your organizational structure and AWS Control Tower has launched managed controls using Declarative Policies. These policies are a set of new optional controls that help you consistently enforce the desired configuration of a service. So for example, customers can deploy declarative policy based preventative control that disallows public sharing of AMIs. Declarative policies help you ensure the controls configured are always enforced regardless of the introduction of new APIs or when new principles or accounts are added. Now some really interesting updates in the realms of migration and transfer. We are happy to announce the launch of the AWS Data Transfer Terminal. This is a secure physical location where you can bring your storage devices, connect directly to the AWS network and upload data to aws, including Amazon Simple Storage Service, EFS and others using a high throughput connection. Currently data transfer terminals are located in Los Angeles and New York and you can reserve a time slot to visit your nearest data transfer terminal facility to upload data. So let's talk a bit more about it. So these terminals are ideal for customer scenarios that create or collect large amounts of data that have to be transferred to the AWS cloud quickly and securely on an as needed basis. These use cases span various industries and applications, so video production data for media and entertainment training data for advanced driver assistance systems in the automotive industry and migrating legacy data in the financial industry or even uploading equipment sensor data in the industrial and agricultural industries. By using a data transfer terminal, you can significantly reduce the time it takes to upload that data in just minutes as opposed to weeks or days. Once the data is uploaded to aws, you can efficiently analyze your large data sets with with Amazon Athena Train and run machine learning on SageMaker or build applications using EC2 pretty much anything you want. We're also happy to announce AWS Transfer Family Web Apps. These are a new resource that you could use to create a simple interface for accessing your data in Amazon S3 through a web browser. With Transfer Family Web Apps, you can provide your workforce with a fully managed, branded and secure portal for your end users to browse, upload and Download data in S3 and the AWS data and the AWS Database Migration Service Schema Conversion with Generative AI is now available. This feature is currently available for database schema conversion for commercial engines like Microsoft SQL Server to Amazon Aurora, Postgres SQL Compatible Edition and RDS for Postgres SQL. Using Generative AI recommendations, you can simplify and accelerate your database migration projects, particularly when converting complex code objects which typically require manual conversion such as stored procedures, functions or triggers. AWS DMS Schema conversion with generative AI converts up to 90% of your schema so it can save you time. Now let's talk about networking and content delivery. VPC Lattice now includes TCP support with VPC resources, so you can now access all your application dependencies through a VPC Lattice service network. You're able to connect to your application dependencies hosted in different VPCS accounts and on premises using additional protocols including TLS HTTP, HTTP, yes and our tcp. This new feature expands upon the existing HTTP based services support, which means you can share a wider range of services across your organization and we're also happy to announce access to VPC resources over AWS PrivateLink. These resources, such as databases or clusters can be in your own VPC or on premises network need not be load balanced and can be shared with other teams in your organization or with external independent software vendor partners. Let's talk security identity and compliance and we're happy to announce the general availability of AWS Security Incident Response. This is a new service that helps you prepare for and respond to and recover from security events. This service offers automated monitoring and investigation of security findings to free up your resources from routine tasks. It provides communication and collaboration features to streamline response coordination and direct 24. 7 access to the AWS Customer Incident Response Team or CIRT. Now security incident Response integrates with existing detection services like GuardDuty and third party tools through the AWS Security Hub to rapidly review your alerts, escalate high priority findings and with your permission, implement containment actions. You can pre configure Incident Response team members, you can set up automatic notifications, you can manage case permissions, and you can use communication tools like video conferencing and in console messaging during security events. By accessing the service through a single centralized dashboard in the AWS Management Console, you can manage active cases, review your resolved cases, track metrics, and do it all in real time. And if you require specialized expertise, you can connect 24. 7 to the AWS cert in only one step. Amazon GuardDuty has introduced GuardDuty Extended Threat Detection. This new capability lets you identify sophisticated multistage attacks targeting your accounts, workloads and data. You can now use attack sequence findings that cover multiple resources and data sources over an extensive time period. This lets you spend less time on that first level analysis and more time responding to critical severity threats to minimize business impact. Now GuardDuty extended threat detection uses artificial intelligence and machine learning algorithms trained at AWS scale and automatically correlates security signals from across AWS services to detect critical threats. The capability allows for the identification of attack sequences like a credential compromise followed by data exfiltration, and represents them as a single critical severity finding. The finding includes an incident summary, a detailed events timeline, mapping to mitre, ATT and CK tactics and techniques, and remediation recommendations. AWS Verified Access now supports secure access to resources over non HTTPs protocols in preview this includes protocols like tcp, SSH and rdp. With this launch, Verified Access enables you to provide secure VPN less access to your corporate applications and resources using AWS Zero Trust Princip. This feature eliminates the need to manage separate access and connectivity solutions to your non HTTPs resources on AWS and it simplifies your security operations. And to wrap up, we're going to talk storage and Amazon S3 has added new default data integrity protections and this builds upon S3's existing durability posture. The latest AWS SDKs now automatically calculates CRC based check sums for uploads as data is transmitted over the network. S3 independently verifies these checksums and accepts objects after confirming the data integrity that was maintained in transit over the public Internet. Additionally, S3 now stores a CRC based whole object checksum in object metadata even for multi part uploads, which helps you verify the integrity of an Object stored in S3 at any time. And this new capability is something I've been waiting for for a long time. Storage Browser for Amazon S3 is now generally available. Hooray. This is an open source component you can add to your web applications to provide your end users with a SIMP interface for data stored in S3. With storage browser for S3 you can provide authorized end users such as customers, partners and employees with access to easily browse, download and upload data in S3 directly from your own applications. Storage browser for S3 is available in the AWS Amplify, React and JavaScript client libraries. Now this is a new project so we really invite you to provide feedback on the roadmap, which gives you outlines of the plan for adding new capabilities for storage browser for S3. And it is backed by AWS Support, which means customers with AWS business and Enterprise support plans get 24. 7 Access support to cloud support engineers as well Amazon S3 has launched Amazon S3 has launched storage Classes for AWS Dedicated Local Zones so Dedicated Local Zones are a type of AWS infrastructure that's fully managed by AWS but built for exclusive use by you or your community and placed in a location or data center specified by you to help you comply with regulatory requirements in dedicated Local zones. These storage classes are purpose built to store data in specific data perimeters, helping you support data isolation and data residency use cases. And finally, we're happy to announce Amazon FSX Intelligent Tiering, a new storage class for fsx. Now this is really useful because it costs up to 85% less than the FSX SSD storage class and up to 20% less than a traditional HTTP Nest storage on premises. And it brings full elasticity and intelligent tiering to nest. This is for Amazon FSX for OpenZFS. So a bunch of really cool updates. We're just getting started. Tune in each and every day where post the keynotes. I'll be giving you all the updates and all the goodies and hopefully there'll be something interesting there for you. I'm sure there is. We do love to get your feedback. AWS PodcastAmazon.com is the way to do it. And until next time, keep on building.