AWSRe:Invent 2020 AnnouncementsECSWell-Architected FrameworkWell-Architected ToolFargateBatchHoneycodeAppFlowNo-CodeLow-CodeCloudTrail
After three exhausting weeks we’ve reached the end of virtual re:Invent 2020. Or not, since 678 sessions have been added for January 12-15. In any case, it’s a good time to review the third week’s launches and announcements, and close the re:Invent 2020 blog series.
This week featured Werner Vogels’ keynote, and leadership sessions for IoT, Compliance, and Serverless. Between Monday December 14th and Thursday 17th we’ve seen another 53 releases, including five completely new services. Let’s look at those first.
New services
As always, Werner Vogels’ keynote was fascinating. If you haven’t seen it, it’s well worth looking it up when they release it on YouTube. This year, Werner spoke from an old sugar factory in Amsterdam. In a two hour talk he used this backdrop and context to lift the covers on Amazon’s thinking about dependability, reliability and observability. During his keynote four new services were released or announced, largely relating to these topics: Amazon Managed Service for Grafana (AMG), Amazon Managed Service for Prometheus (AMP), AWS Fault Injection Simulator (FIS) and CloudShell. A day later, Amazon announced the Amazon Location Service.
Amazon Managed Service for Grafana (AMG)
AWS News - AWS Blog - Product Page
Grafana is a very popular open-source observability platform with a wide variety of use cases. It focuses on interactive visualization, where the underlying data is provided through an extensible plug-in system. Grafana offers an optional enterprise license with additional plugins and support.
Although Grafana dashboards are widely used, deploying a secure and performant Grafana server can be a lot of work. With Managed Service for Grafana, Amazon performs the undifferentiated heavy lifting for you. This allows you to start building your Grafana dashboards in minutes. Additionally, Amazon Managed Service for Grafana provides native integration for multiple AWS Services, including Amazon Managed Service for Prometheus, Amazon CloudWatch, Amazon Elasticsearch Service, AWS IoT SiteWise, Amazon Timestream, and AWS X-Ray.
Amazon Managed Service for Grafana also integrates with AWS Organizations and CloudFormation StackSets, allowing you to read data from many AWS accounts and regions.
Pricing for AMG is based on active users: an Editor user with write permissions costs $9 per month, while a Viewer user will cost $5 per month. You will only be billed for users who log in to a Grafana workspace. A user is considered unique per workspace, so if a single user logs in to three workspaces in a month the license for this user will be invoiced three times as well.
Amazon Managed Service for Prometheus (AMP)
AWS News - AWS Blog - Product Page
Prometheus is another popular open-source application in the observability space. It consists of multiple micro-services (distributor, ingester, query-frontend, querier and a few others), which can pull metrics from a large range of data sources. A common use case for Prometheus is to collect container metrics in Kubernetes environments. Prometheus stores the collected metrics in a time-series database, which can be queried through the PromQL query language.
Grafana is often used to visualize the data collected by Prometheus. As such it comes as no surprise Managed Service for Prometheus and Managed Service for Grafana were released together. You can, however, use one of the services without the other. Pricing for AMP is based on three components: the number of samples collected ($0.002 / 10,000 samples for the first 2 billion samples), metric storage ($0.03/GB) and Query Processing Minute or QPM ($0.142/QPM). Amazon will automatically scale all underlying components based on usage.
AWS Fault Injection Simulator (FIS)
Amazon can now break your environment as-a-service. Amazon Fault Injection Simulator integrates with a number of core AWS services, like EC2, ECS, EKS and RDS, and randomly generates real world faults in those systems. These include throttling, high CPU loads and high memory loads. The product has only been announced, so no documentation, blog or news article is available yet. We can expect more failure modes and more supported services at launch and in the future.
The guiding principle behind Amazon FIS is that human imagination cannot conceive of every failure mode (in the time we have available). And even if we could, actually testing all those failure modes would cost a lot of time. A well architected, highly available environment should be able to withstand any type of failure, and Amazon FIS allows you to generate these failures for you.
For example, an EC2 instance with an overloaded CPU should fail its health checks, and be removed from its load balancer and the autoscaling group. Amazon FIS allows you do define Experiment Templates that specify which components you want to test. The metrics of the environment are monitored through CloudWatch and EventBridge. When a failure is detected the experiment is aborted, allowing engineers to fix the issue. These tests can be run in an acceptance or test environment, but also in production (when you’re feeling especially confident). The experiments can also be configured as part of a deployment pipeline, which allows an application update to be stress tested before before being rolled out to a wider audience.
With Amazon FIS executing game days has become easier, which helps customers improve the quality and resilience of their applications. This service is focused on very mature workloads with reliability as a key metric. Garden-variety customers running common workloads that are ‘good enough’ will probably not benefit from FIS as much, because they know and accept that their key metric is cost effectiveness - not resilience.
AWS CloudShell
AWS News - AWS Blog - Product Page
CloudShell is an interesting release aimed at quite a specific audience.
There are a number of ways to interact with AWS resources: the Web Console, SDKs, CloudFormation, SAM, and the APIs. There are some wrappers around these, such as Terraform, CDK, Pulumi and more. Technically, everything is a wrapper around the APIs, but let’s not go into that rabbit hole right now.
For many developers, the CLI is the preferred way of interacting with AWS. It lets you quickly list your buckets and IAM users, copy an object, take a snapshot, and literally everything else you can do in AWS. It even does a number of things you can’t do in the web console, like listing EC2 Instance Profiles.
The downside for the CLI is that you need to authenticate with AWS, preferably through temporary credentials. You will also need to keep your CLI up-to-date, and you might require additional tools like the ECS CLI, SAM CLI or Elastic Beanstalk CLI.
AWS CloudShell removes these requirements by providing a CLI in the AWS Web Console. The CLI is available at the click of a button, and will use the credentials you are logged in with.
There are no credentials to manage, no tools to install, and nothing to maintain. You get 1GB of free persistent storage, and you can even install additional packages with yum. CloudShell is a big convenience for the many developers and engineers who are already familiar with the CLI.
Amazon Location Service
AWS News - AWS Blog - Product Page
The last newly announced service is the Amazon Location Service. This service allows developers to build applications with map and geo features, for example displaying the location of an address or device, retrieving the latitude and longitude for an address, defining geofences, or even recommending routes from one location to another.
In the announcement blog post Jeff Barr writes “Priced at a fraction of common alternatives, Amazon Location Service gives you access to maps and location-based services from multiple providers on an economical, pay-as-you-go basis”. However, the pricing - like for many AWS services - might bite you when you start using it at scale. The request-based pricing reads as follows:
| Capabilities | Requests | Price per 1,000 |
|---|---|---|
| Maps | Map tiles retrieved | $0.04 |
| Places | Addresses geocoded | $0.50 |
| Addresses geocoded (stored result) | $4.00 | |
| Positions reverse-geocoded | $0.50 | |
| Positions reverse-geocoded (stored result) | $4.00 | |
| Tracking | Position writes | $0.05 |
| Batch position reads | $0.05 | |
| Geofencing | Positions evaluated | $0.20 |
| Geofences created, deleted or described | $0.05 | |
| Geofence list | $0.05 | |
| Geofence-months | $0.20 | |
| Service Resources | Resource create, read, update, delete or list | $0.01 |
In other words, don’t start building your Google Maps competitor based on Amazon Location Services just yet. But if you have an existing app or website that benefits from location data, Amazon Location Service is definitely worth comparing to existing solutions.
IoT releases
This week also saw a large amount of IoT releases, affirming Amazon’s commitment to this technology segment. The full list of IoT releases can be found below:
| Name | AWS News | AWS Blog |
|---|---|---|
| AWS IoT SDK for Embedded C version 202012.00 includes over-the-air update (OTA) library and PKCS #11 implementation | link | |
| FreeRTOS adds cellular LTE-M interface library to support cellular IoT based applications | link | |
| Announcing Amazon Sidewalk Integration for AWS IoT Core | link | |
| AWS IoT Device Defender adds support for custom metrics | link | |
| Announcing support for Alarms (Preview) in AWS IoT Events and AWS IoT SiteWise | link | |
| Introducing AWS IoT Core for LoRaWAN: connect and manage LoRaWAN devices at scale easily and cost-effectively | link | link |
| Announcing AWS IoT SiteWise Edge (Preview), a new capability of AWS IoT SiteWise to collect, process, and monitor industrial equipment data on-premises | link | |
| Announcing FreeRTOS Long Term Support | link | link |
| AWS IoT Analytics can now store processed IoT data in data stores using Apache Parquet format | link | |
| Enhanced error handling capabilities in AWS IoT Analytics data processing pipelines | link | |
| AWS IoT Core Device Advisor now available in preview | link | |
| Introducing AWS IoT EduKit | link | |
| Introducing AWS IoT SiteWise plugin for Grafana | link | |
| AWS IoT Greengrass 2.0 provides an open source edge runtime and new capabilities for building and operating IoT device software | link | link |
| AWS IoT Core adds the ability to deliver data to Apache Kafka clusters | link | |
| AWS IoT SiteWise launches support for Modbus TCP and EtherNet/IP protocols with enhancements to OPC-UA data ingestion | link | |
| AWS IoT Device Management introduces Fleet Hub, a new, easy way to monitor and interact with IoT device fleets | link | |
| Announcing AWS IoT Device Defender ML Detect public preview | link | link |
Systems Manager releases
The most awkwardly named service also got a lot of love this week, with three new capabilities: Fleet Manager, Change Manager, and Application Manager. And yes, that means their full names are Systems Manager Fleet Manager, Systems Manager Change Manager, and Systems Manager Application Manager, to complement the existing features Systems Manager Patch Manager, Systems Manager State Manager, and Systems Manager Session Manager. Sigh. If only they could have hired a Systems Manager Naming Manager.
Systems Manager provides a single pane of glass for all server resources, their patch levels and configurations. The new releases focus on managing instances in and out of AWS, virtual or physical, running Linux, Windows or MacOS. They are largely aimed at making a systems operator’s life easier.
Systems Manager Fleet Manager
AWS News - AWS Blog - Product Page
I’m just going to quote directly from the AWS Blog here, because there’s no way I could describe it better:
Available today, Fleet Manager is a new console based experience in Systems Manager that enables systems administrators to view and administer their fleets of managed instances from a single location, in an operating-system-agnostic manner, without needing to resort to remote connections with SSH or RDP. As described in the documentation, managed instances includes those running Windows, Linux, and macOS operating systems, in both the AWS Cloud and on-premises. Fleet Manager gives you an aggregated view of your compute instances regardless of where they exist.
This feature mainly comprises of functionality that was already available in Systems Manager. Previously you would have to click through multiple screens and collect the information you need from multiple sub-services. Fleet Manager aggregates this information for easy access.
Systems Manager Change Manager
AWS News - AWS Blog - Product Page
The AWS Blog for the Change Manager release states “… not making changes is stasis, followed by irrelevance, followed by death”. That’s… quite an existential view on patch management, but I guess Sébastien Stormacq has a point. Systems Manager Change Manager allows systems operators to control which changes are being released onto their servers. By allowing only approved changes, the chance of disruptions is lowered, making sysops life easier. With Change Manager historical changes can be reported and audited, allowing for an additional level of control.
Systems Manager Application Manager
AWS News - AWS Blog - Product Page
Application Manager extends System Manager’s functionality into the application landscape. This is primarily a visibility feature, providing operators a single place to look for application and infrastructure issues. A useful feature in Application Manager is the option to mark detected operational issues as Open, In Progress or Resolved. This allows multiple people in a team to effectively prioritize and coordinate issues, which is extra useful when everyone is working from home.
Other announcements
Because re:Invent wouldn’t be re:Invent without dozens of small releases, please find a list below.
| Name | AWS News | AWS Blog |
|---|---|---|
| Now Secure Your SageMaker Studio Access Using AWS PrivateLink and AWS IAM SourceIP Restrictions | link | |
| AWS Single Sign-On now supports Microsoft Active Directory (AD) synchronization | link | |
| AWS SDK for JavaScript version 3 is now generally available | link | link |
| Announcing the first AWS Wavelength Zone in Tokyo, Japan | link | |
| Deploy Microsoft Active Directory Infrastructure on Amazon EC2 using AWS Launch Wizard | link | |
| AWS Cost Anomaly Detection is now generally available | link | link |
| AWS Launch Wizard now supports SAP application software installation | link | |
| Three new digital courses for AWS Partners | link | |
| APIs now available for the AWS Well-Architected Tool | link | link |
| Amazon FSx is now available in the AWS GovCloud (US) Regions | link | |
| Introducing Amazon SageMaker ml.P4d instances for highest performance ML training in the cloud | link | |
| AWS Nitro Enclaves is now available in 6 additional regions | link | |
| Amazon Data Lifecycle Manager now automates copying EBS snapshots across accounts | link | link |
| AWS Cloud Map is now available in the AWS Africa (Cape Town) and Europe (Milan) AWS Regions | link | |
| Attribute-Based Access Control (ABAC) for the AWS Key Management Service | link | |
| Amazon FSx now supports on-premises access from additional IP address ranges | link | |
| Amazon Connect supports Amazon Lex chatbots with Latin American Spanish and German | link | |
| Announcing Unified Search in the AWS Management Console | link | |
| AWS Personal Health Dashboard now supports organization-wide event aggregation | link | |
| Cost & Usage Report Now Available to Member (Linked) Accounts | link | |
| Announcing Amazon Route 53 support for DNSSEC | link | |
| AWS Well-Architected Guidance Engine now available in AWS Control Tower | link | |
| Amazon SQS Now Supports a High Throughput Mode for FIFO Queues (Preview) | link | |
| EC2 Image Builder now supports container images | link |
I’m running out of time and space for this blog, so I can’t dive into each of these releases. However, I put the most significant announcements at the bottom of the list. I would highly recommend reading the linked articles for additional context.
One more thing
Twitter announced that they have signed a multi-year agreement with AWS to serve their timelines. This marks the first time Twitter is using public cloud services for their real-time timelines. A big win for both AWS and Twitter!
Interestingly, they’re naming Graviton2 and having their compute workloads closer to the end user as deciding factors. These have been big topics at re:Invent, and I wrote a blog post about each: AWS is coming to a data center (or pizza parlor) near you! and The Future of Cloud Runs on ARM.
Conclusion
This marks the end of three weeks of virtual re:Invent 2020. It was intense. So much info, so many keynotes, all those new services, releases and announcements. I felt a longer re:Invent might be easier to keep up with, but instead it’s been going just as fast as re:Invent in Vegas. Just as fast, but longer.
I missed meeting people in person, but it was well compensated by the interaction on Twitter on Slack. Big shoutout to my fellow APN EMEA Ambassadors Ben Bridts and Rolf Koski for all the lively discussions! And thanks to AWS engineers / evangelists / architects / enthusiasts Matthew S. Wilson, Aidan W Steele, Danilo Poccia, Chris Munns, Steven Bryen and Ricardo Sueiras for their content and engagement!
As every year, I was impressed by AWS’ continued growth. They continue expanding into new areas, as well as broadening the features and functionality of existing services. I’m looking forward to using many of these services next year. But first I’m off for two weeks to catch up on sleep. See you next year!
This article was the last in a series published around re:Invent 2020. If you would like to read more about re:Invent 2020, check out my other posts:
- AWS re:Invent 2020 Day 1: Top 5 Announcements
- AWS re:Invent 2020 Day 1: S3 Announcements
- AWS re:Invent 2020 Day 2: AWS is coming to a data center (or pizza parlor) near you!
- AWS re:Invent 2020 Day 3: Optimizing Lambda Cost with Multi-Threading
- AWS re:Invent 2020 Day 4: Top 5 Announcements for Wednesday - Friday
- AWS re:Invent 2020 Day 5: AWS is Adding a Region Near You!
- AWS re:Invent 2020 Day 8: Building a Multiplayer Game Architecture with Global Accelerator Custom Routing
- AWS re:Invent 2020 Day 9: Roundup Week 2
- AWS re:Invent 2020 Day 10: The Future of Cloud Runs on ARM
I share posts like these and smaller news articles on Twitter, follow me there for regular updates! If you have questions or remarks, or would just like to get in touch, you can also find me on LinkedIn.
