AWS re:Invent 2022
Another reinvent has come and gone and what a week it has been!
With 6 hotels, 65,000+ attendees and over 3000 sessions, it can be hard to navigate the amount of content generated throughout the week.
Over the span of the 5 days, I attended a number of different talks, events and mixers! In this blog I want to take you through some of the key takeaways I have taken from the event, that you might have missed.
Event driven architecture
Event driven architecture is something we have been actively working on within our software development projects in Hamilton Robson, so when it came to registering for sessions, it was something I definitely tried to align my schedule with.
I started off the week attending Eric Johnson’s — “Building next-gen applications with event-driven architectures” session. This was a great start to the week, Eric took us on a journey through a number of common architecture patterns and some of the benefits/ drawbacks to using them in certain scenarios:
- Synchronous request-response model — simple, has low latency and fails fast — however, some of the main disadvantages of this model is the affects receiver failure can have and also throttling.
- Asynchronous point-to-point queue model — Decreases temporal coupling, is resilient to receiver failure and the receiver controls the consumption rate, which helps to protect the downstream system. However, this model still has its drawbacks which include response correlation, backlog recovery and failures.
- Asynchronous point-to-point router model — which still does come with its disadvantages including location coupling as the sender maintains routing logic and overall the sender complexity increases!
- Asynchronous message router event bus model — which overalls reduces location coupling, allows us to have efficient senders and receivers as it takes a lot of the logic out of the sender and allows you to move it into configuration. For this implementation Eventbridge can be used and you can have pattern matching to determine targets, which overall allows your logic to be more centralised.
At Hamilton Robson we utilise the queue model in a number of our projects and work specifically with AWS SQS. This talk was really good for understanding possible areas in which we can extend and improve our system in the future. One thing that really stood out for me was when Eric said “Lambda should be used to transform and not only to transport.” I think something we have all been guilty of is creating Lambdas solely for moving data to the right place and triggering downstream functionality.
To overcome this, a service mentioned in this talk and a number of other talks at re:Invent was AWS Step functions, which really seem like they are the key for the future of serverless event orchestrations. Eric had explained if you have to write a lot of if/then logic then you should probably be using step functions. They allow you to create workflows, state machines and perform parallel processing.
Whats new with Serverless
Sticking to the theme of event driven architectures, there are a bunch services we use at Hamilton Robson to implement this pattern. Another one of my favourite talks was “Whats new with serverless” presented by David Boyne. In this session David dove into a lot of the recent serverless announcements and what they mean for future development, there was a lot covered in this talk but some of the key announcements that stood out to me, that I feel could directly impact they way I work build with AWS are:
- Lambda URLs —these were announced earlier this year, but it was great to hear from David on some of the main use cases of when these should be used. When working with triggering Lambdas from a client over HTTPS, we have typically used API Gateway on our projects and so I was very interested to see why or when it would be appropriate to use the Lambda URLs instead. Lambda URLs are free and you only pay for the Lambda invocation, whereas with API Gateway you are also paying for the request. Something that really stood out to me as a use case is when running longer workloads. Something I’m sure a lot of us have came up against when working with API gateway is the 29 second request timeout — using Lambda URLs this timeout can be a long as the full 15 minutes! Although this is super cool, there still is a lot of specific use-cases that will still require us to use API Gateway for, one major one being the authentication type that Lambda URLs use is IAM Auth, but with API Gateway we can also use API keys, Congnito or use Lambda for custom authorisers — we can also have used AWS WAF when working with API Gateway. I do think the introduction of Lambda URLs will definitely help us when it comes to making quick proof of concepts, as it allows us to get off the ground quicker because we would require less services and overhead.
- Lambda Snap Start — This was announced to significantly reduce Lambda cold starts (for Java-based Lambdas). When a Lambda is invoked, its runtime needs to be created, code downloaded and environment set up — what SnapStart now does is creates a snapshot of this initialised runtime after the first request to the service and then when future requests come in it resumes from the snapshot. I found this really cool, even though at Hamilton Robson we don’t work specifically with any Java Lambdas, I think this concept of reducing cold starts is great.
- Introduction of more Intrinsic functions to perform data transformations with Step Functions. These can help reduce the need to add your own custom Lambdas into your workflow for data transformation as they can handle a lot of basic data transformations out of the box for example: array handling, JSON data manipulation, encoding/decoding and mathematical operations. Step Functions are something we are starting to integrate into our projects and most of our workloads involve some form of data processing so knowing that opting for this route can handle some of this general data processing for us and allow us to focus on other business specific logic is a huge benefit.
- Event Bride Scheduler — This was announced slightly before re:Invent, but is something I was very excited about and was great to hear David talk about it in a bit more detail. At Hamilton Robson we have a number of scheduled events running with EventBridge — however, we had some pain points with it especially when it came to things like one-time schedules and setting schedules in different time zones — both of which are now addressed with this latest release. Schedules can now support time zones and allows you to have one time schedules that scale up to millions of different schedules . Really cool to hear these announcements as they are definitely something we will be able to extend our system to use.
Another great Serverless session was “Get started building your first serverless, event driven application” presented by Emily Shea, Head of Application integration at AWS. This was a great session and took a lot of the knowledge from other serverless sessions one step further by doing a walkthrough of what AWS serverless services to use and best practices when developing with them. Emily showed a number of code samples and architecture diagrams throughout the session and shared tips on developer tooling, application design and how to evolve your application over time. Loads of learnings from this session that will definitely help when building out more serverless projects.
Metaverse/ 3D image generation
When at a conference I always like to challenge myself to attend a talk that isn’t in my area of expertise to learn something totally different.
Metaverse is something I have been hearing more and more about, so when I saw the lineup for this panel, it piqued my interest and I decided to attend — which I’m very glad I did, the conversation was very interesting!
The panel started with each member describing what Metaverse means to them which included definitions of bringing the 3rd dimension into the 2D digital world and merging the physical world with the digital world. One of the panellists had described it as: “Metaverse now is like explaining the internet in 97 — we don’t know what we don’t know”.
However, the overall consensus with the panel is that we still have a lot of challenges emerging, especially when it comes to compute, network and storage. Which leads to questions of if the infrastructure required is ready yet?
There is a gap in the compute power and latency issues when running compute in the cloud! Another bottleneck to the Metaverse is the automation of creating 3d objects — at the minute this is too slow and costly of a process and is something that will need to be improved upon.
CTO Keynote
One of the most anticipated sessions at re:invent is always the CTO Keynote by Werner Vogels and it did not disappoint.
This year there was a key opening theme of Synchrony Vs Asynchrony — if you did not catch his matrix style talk opening, I’d recommend you check it out here, as it’s a great explanation of how the world is is asynchronous and what the world would look like if everything was synchronous!
Overall we need to build evolvable architectures, we don’t build everything at once as we typically start small and over time our systems grow in complexity- with this in mind we need to build in a decoupled way that allows us to extend and evolve easier in the future! This is where building asynchronous event driven architectures comes back into play.
This fits into so many of the other sessions I attended throughout the week and it was great to hear Werner double down on the importance of being event driven and how it is essential for operating at global scale.
In the later half on Werner’s keynote he discussed the importance of 3D images and simulations — which kinda tied into some of the bottlenecks discussed on the Metaverse panel around the generation of 3D images/models.
Werner said “A 3D model is worth 1000 pictures”. Overall, people want to be able to visualise everything and for this to work we need a fusion of models, sensors and data in order to have spatial intelligence!
At re:invent AWS Sim Space Weaver has been announced to make spatial simulation accessible for everyone! It aims to take away the heavy lifting by handing networking and storage management by allowing you to run real-time spatial simulations in the cloud and at scale. With SimSpace Weaver, simulation developers are no longer limited by the compute and memory of their hardware.
This is a pretty cool announcement, as Werner said “simulations have a crucial role in innovation” — which is true, as it allows you to trial things that could be dangerous or very expensive to test in the real world.
Summary
Overall listening to these keynotes and breakout architecture sessions has really allowed me to think with a fresh mindset about how our services are communicating and if there is anything new we could implement to try and future proof ourselves.
There was a lot happening this week and these are only some of my highlights, there are still some really awesome talks that I wanted to see, but couldn’t quite get fitted into my schedule this week! But the good thing is all breakout sessions are recorded and so are now available to watch back — if you were unable to attend re:Invent this year or like me weren’t able to fit absolutely everything you wanted to see, you can check them out here.
