My Serverless Handbook - My serverless journey on AWS

The time has come for me to introduce my new project, My Serverless Handbook

My AWS Serverless journey began back in 2015. I transitioned into cloud development from an embedded world. I got the opportunity together with a few colleagues to make this move, I can say it turned out great. Working with AWS and serverless for almost a decade now is one of the best things I have ever done. This is my story.

The first solution we built was based on S3 and Lambda functions as primary components. This was the start, this was when I fell in love with serverless, and I have never looked back.

In this post I will talk about my journey, I will talk about my highlights and some of the architectures I have created. Hopefully you will get to know me.

But First!!!

Introducing My Serverless Handbook

What is My Serverless Handbook? It's my collection of solutions, tutorials, code examples, architectures, and lesson learned.

You will find all of that content on https://serverless-handbook.com this will be my home base, this is where you will find all my serverless solutions. I will not abandon this blog, here I will write about everything, and new solutions to the My Serverless Handbook will most of the time be accompanied with a deep dive blog post here.

Make sure to follow me on LinkedIn and X to receive the latest updates on the project.

I really hope you will enjoy my collection of serverless solutions!!

How it started

My cloud journey started 2015 at Sony Mobile, with a opportunity to create a new cloud based solution to collect sign up information and feedback, from end consumers. I had not been working with cloud technologies at all before and this was a challenge that was impossible to turn down. It was almost like going back to school, learning something completely new. Sure, I had learned new things before in my career, but not something that was this different. I had been working with embedded solutions for many years prior to this move.

I had a great mentor that helped me a lot the first time, made sure we kept the solution on the right track, gave us feedback and made sure we learned all we could. We got advice to visit not only the AWS Summit in Stockholm in the spring of 2016, but also AWS re:Invent in November. This was my first re:Invent and wow! I learned so much, and I have not missed a single re:Invent after that.

Not only did I have a great mentor. I also had very supportive managers. They supported my transition and my journey to 100%, if it wasn't for these persons I would probably not be where I am today.

I early on decided to get certified, it gave me the incentive to study hard and mix theory with practice in my day to day work. I used courses from both A Cloud Guru and LinuxAcademy, and sure enough, at re:Invent 2016 I sat my first ever AWS exam, and I passed.

This was the start of my journey toward fully certified and the Golden Jacket.

My first architecture

The very first version of this new cloud solution, should handle registrations from users. Registrations needed to be searchable based on several conditions, such as country. We quickly opted in for a serverless and managed services solution. Users signed up from a mobile application. From the application we called Amazon Cognito to assume a IAM role for temporary credentials. The application collected needed information and uploaded this to an file in an S3 bucket. From here a AWS Lambda function was invoked that processed and stored the information in CloudSearch, of all services. CloudSearch would probably not be the service I would go for today, but it got the job done.

So not only was my first solution serverless, it was also event-driven and used my favorite architecture pattern, storage first. Was it coincident? I think not, it was probably my future self calling.

My first architecture take two

Now, this was not the end of the project or this solution. It was a great success and the project got more signed up users than expected. This is actually when the actual work and project started.

The solution was previous just an anonymous sign up, we didn't really know the user behind the screen. We only had the identifier for the users mobile device. This needed to evolve, there was a need for proper user registration but at the same time keeping the anonymous sign up. If a user registered more features should be enabled, like providing feedback directly to the development team and for the development team to ask followup questions on the feedback. We also built in a survey functionality, allowing for project managers to send anonymous surveys to all users, registered or not.

At this phase the solution evolved and we started working on a more complex solution. We introduced API Gateway, user management via Cognito User Pool, data was stored in DynamoDB, and some more places. After building the solution I realized that it would not scale as we hoped. This was still early in my career, I had been working with cloud for less than one year. After re:Invent 2016 I came back full of ideas on how to improve the solution and evolve the architecture. This is why re:Invent is such a great conference, it's all about learning.

We decided that we like to use GraphQL for our API, to better support filters and not having to do that many REST calls. Now, this was way before AWS AppSync, we created our own GraphQL service by running Apollo in a Lambda function integrated with API Gateway. Each service then also run a GraphQL endpoint that the function used to resolve data.

We kept the storage first approach for device registration, but moved to SQS directly integrated with API Gateway. We built our own poller in StepFunctions, way before SQS could invoke Lambda functions. You can read more on this solution in Serverless projects over the years. Data was now stored in DynamoDB and we used DDB streams to index the data in the managed version of ElasticSearch, that today is called OpenSearch.

However, this was not all. There was a need for service to service communication, the decision was to talk GraphQL between our services as well. Then we could reuse what was built for the public API. The need for an service discovery solution was on our radar. We based our solution on how Netflix had implemented one of their solutions. Each of our services registered with their responsibility and the endpoint to call in a central service. Every service then called the service discovery to get the endpoint for a specific task or data set. Some caching was their as well to reduce the number of calls.

The next step

Due to some circumstances it became time for me to try my wings somewhere else. I joined Infor and the M3 team as Cloud Architect. I got to work with some of the smartest and skilled people I know. I learned so much during this time. I got to do several internal presentations around cloud, serverless, and microservices. This sparked a new interest in me, speaking and presenting.

Due to personal reasons my tenure at Infor became rather short, I really enjoyed working there. It's a great company and the people are amazing. I had really good managers that supported and inspired me.

In my next move I joined Knowit as consultant and Cloud Architect, I got to work on some really interesting projects. I spent most of my time as DevOps specialist at IKEA creating CI/CD strategies and methodologies for deploying to cloud. It was during this period that I was invited to join the AWS Ambassador program for the first time. I had to leave the program for a period, but I did a boomerang and came back to it later.

IoT and Serverless

IoT and serverless are two technologies that go very well hand in hand. I had a growing interest for IoT and had started to experimenting building different IoT gadgets and connecting that to AWS using IoT Core. I got the chance to join a growing team at AssaAbloy building a major IoT platform. The solution had been running as a PoC for a few years and was now ready to take the next step in the evolution.

I joined the team as Lead Cloud Architect with the focus on increasing the scalability of the solution, not only from a perspective of number of connected devices, but also development speed. Data was already now in the center and there was a clear vision to be able to do more with all the collected data. Therefor it was important that the solution could support the data analytics team.

After joining the team we slowly started to transform the solution into a true serverless solution. Event-driven architecture, with EventBridge as the key component, became center of everything.

Data was flowing from connected devices into IoT Core as events. This was routed to a Kinesis DataStream where a Kinesis FireHose picked it up and sent to an OpenSearch cluster and S3 for easy data access. This part kept evolving and data could be transformed and served by Athena to different consumers later on.

From Kinesis data was transformed and sent to EventBridge as cleaned up events, and here the publish flow was done. We could now hook up all different services we had as consumers to EventBridge for them to process and store the data as they saw fit.

From an API point of view, data was served to a webb-application over API Gateway that was integrated with containers running Fargate. Containers was chosen for several reasons, one being that they were all written in Java. The system had two clear parts, the data serving that was all done with containers and the ingestion part that was built using Lambda functions and StepFunctions. The ingestion part was all event-driven and asynchronous. While the data serving to the webb-application was a classic request/response model. Containers was all Java based while Lambda functions was all in Golang.

Joining Sigma Technology Cloud

In September 2021 I started my current adventure as Head of AWS at Sigma Technology Cloud. In this role I get to advocate for serverless in my work as advisor for different clients. I can use my passion for serverless to help everyone fall in love as well. Serverless is a great tool to have when building scalable and cost-effective solutions.

I quickly boomeranged back into AWS Ambassador program, in my role I experienced one of my proudest moments as AWS Ambassadors. Keep reading to find out what....

My role also allow me to engage even more with the community, in different forms, hosting and arranging events, speaking, and blogging. Leading the internal competence group I can use all my community learnings and channel that into the organization, helping all my colleagues discover and engage with the community. Helping them all grow to the next level, in the same way I have had mentors help me grow.

A very proud moment when one of my colleagues also was accepted into the AWS Community Builders program.

My favorite serverless AWS services

When I look at the solutions I have been creating lately there is a clear pattern, what services I tend to use over and over again.

With all of the available SDK integrations StepFunctions has quickly become one of my favorite services. It enables me to write powerful workflows without writing any code, just calling the AWS services directly from the workflow it self. With the latest addition of HTTP endpoint integration for 3rd party APIs it became even more powerful. I only create Lambda functions in cases where I need something more advanced, for everything else StepFunctions it is!

Event-driven architectures are very powerful, and for that I need a good event broker. EventBridge gives me exactly that, there are other services that can act as broker but EventBridge is my favorite. You can check out Event-Routers and Brokers in AWS. Navigating the Event-Driven Landscape where I write about different options in AWS.

My community journey

In the end of October of 2018 I created my very first blog post Building a CloudFormation macro. Creating my first post was scary, writing was nothing I had done before, and you are vulnerable when putting your content out in the public. However, the support I got when doing internal presentations at Infor motivated me to do this. When looking back this was for sure the right thing to do. I kept writing blog posts and after writing six posts I took the next step.

I had two interests that drove me at this point in time. I got deep into voice based UI and Alexa skills, and GitOps. I took those interests and all the learnings I had from Sony and Infor and submitted my very first call for paper. I was accepted and gave my very first public talk, GitOps in action at Øredev conference in November of 2019. Now, I can tell you this was scary and I had never been so nervous before. 5 minutes before my talk was going to start my smart watch alerted me of high pulse and suggested deep breathing exercise. However, I had so much fun so I evolved the talk and submitted a CFP to AWS Community Day in Stockholm, I got accepted again. And in end of January 2020, right before Covid hit hard, I gave my talk GitOps in action, powered by Alexa. In this talk I deployed to production using voice commands with Alexa. I used Unicorns for my demo solution already back then.

This was just the start and todays date I have delivered more than 15 public talks and posted more than 40 blog posts.

Today you can find all my public talks and blog posts in a single place at my primary site https://jimmydqv.com

Speaking at re:Invent

So far, because I'm sure there will be more highlights, but as of now the top highlight of my public speaking career is when I got the chance to speak at re:Invent 2022. When I got accepted I couldn't really believe it. When I told my manager that I was accepted, it took her a some time to realize what I was saying. After that moment, she was all in and supported me all the way. I showed her the first draft of the talk and she totally bashed it, which was a good thing, for the 2nd version turned out great and was the version I presented.

For me re:Invent is the holy graal of tech conferences, it's focused on learning and it's focused on developers. My talk was "IoT Enabled smoker for great BBQ" and I talked about three things I really love. Serverless, IoT, and BBQ, could it be better? I don't think so. Some day I hope to return to the re:Invent stage and give another talk.

If I was nervous before my very first public talk back in 2019, it's nothing compared with how nervous I was the hour before I took the stage at re:Invent. Not joking, I was sitting on the floor shaking for a few minutes. The minute I started talking it all went away, as it always does, I truly enjoyed the time on stage. Having almost 100 people in the audience and the getting a perfect 5.0 CSAT score, well wow!

AWS Certification SME and Lead SME

In 2019 I was invited to my first item writing workshop as an AWS Certification SME for the, now deprecated, Alexa Skill Builder certification. Shortly after the first workshop I did a second, but this time for Solutions Architect Professional. I have been in the SME program since then, and I still am, primarily working on SA-Pro. After contributing for a few years I was nominated to become Lead SME for SA-Pro, in this role I get to review questions, mentor, and coach new SMEs. A fantastic opportunity and I really enjoy this.

AWS Ambassador

The AWS Ambassador program is a community of highly qualified and influential employees at AWS Partners. It's an invite only community that consists of so many amazing and smart people. It gives me the opportunity to interact with some of the best and learn from all of them. Being invited to this program is a great acknowledgement of all the hard work I put into the community in a whole.

I was invited to join the program at my time at Knowit in 2019, but as the program require that you work for an AWS Partner I had to leave the program when I joined AssaAbloy.

When I then joined Sigma Technology Cloud I quickly boomeranged back into the program. I have been an active member in the community since then which also led to a very special award.

Awarded #1 Top AWS Ambassador in EMEA

Every year AWS Ambassadors get together for a Global Summit. We get information about upcoming changes in the program, have technical deep dive sessions with AWS service teams and key leaders within AWS, we have business focused sessions and exchange ideas. Often Ambassadors give lightning talks on different topics, which is always very appreciated. Of course, we also have a lot of fun talking, networking, having some great laughs. This is the time of the year, with exception for re:Invent, that I get to meet my fellow Ambassadors and just have a great time.

We also celebrate success and the Ambassadors that has been contributing the most, being the most active, are appointed. The top Ambassadors receive an award based on this.

In 2023 I was awarded #1 Top Ambassador in EMEA, I can't start to describe what a proud moment this was for me, it was huge. I'm not active and contribute to the community to receive any awards, I do it because I love helping other people and see them grow. But, receiving this award feels great and is the icing on the amazing community cake!

I do have my eyes set on the 2024 award as well... We just have to wait and see.

I'm very prod to be part of this fantastic community

AWS Community Builder

Since 2021 I have been in the AWS Community Builders program. This is an program that offers technical resources and networking opportunities to AWS enthusiasts and thought leaders passionate about sharing content with the community. The program has an application process running yearly, where you can apply and motivate why you should be part of the program.

Of course, I'm part of the Serverless cohort in the program. This is a very active community with great people, I have met people that I today call very good friends and talk to on a daily basis. This program together with the Ambassador Program has really helped me grow and motivated me to keep sharing content with the world.

I'm very prod to be part of this fantastic community!

AWS User Group Leader

Giving back to the community, that has helped me so much, is all natural. One way I do this is in my role as user group leader.

I run the AWS User Group Scania (Skåne) together with Marcus Bladh. We arrange, host, and speak in all different form of Meetups. I also try and help out as much as I can, hosting and speaking, in the AWS User group in Stockholm. Not being located in Stockholm, or an official leader, I still try to lend a hand as often as I can.

I gave my second public talk in 2019 at the AWS Community Day Nordics in Stockholm. Now I not only speak, I help out arranging as much as I can as well.

All of these events are great learning opportunities for everyone, experienced or new to AWS. Helping out arranging I can really give back to the community.

What's next?

What the future holds I don't know. But I know it will be serverless, it will be event-driven, and it will be on AWS.

I also know that the AWS Community will keep being a large part of me and my work. There is nothing more rewarding then to be able to give back to this amazing group of people. I hope to be able to inspire new emerging cloud talents in the same way so many community members has inspired me.

My Serverless Handbook is the first step on this new community journey that I don't know the ending to.

To everyone in the community! Thank you for being awesome every single day!

Final Words

This was my cloud journey summarized in a blog post, it was impossible to cover all details from a decade long cloud journey, but I picked my favorite moments and challenges.

I would like to give a shout out to one person that has been there every step of the way, been supportive all the way, and have supported me to do all of this crazy trips. That is my wonderful and loving wife, Helena! Without her I would not have come this far! She is the best support I have.

Don't forget to follow me on LinkedIn and X for more content, and read rest of my Blogs

As Werner says! Now Go Build!