Vending Machine for AWS accounts.

To separate different workloads, environments, playgrounds, lab areas, from each other the preferred way is to use separate AWS accounts. AWS Organizations offers a nice way to handle all of your AWS accounts from a single locations. Creating and managing the accounts in a manual way can be a time consuming work. I needed a way to be able to manage the accounts and to setup resources in the accounts at the time of creation in an automatic way.

A blog post from AWS gave the ideá to build my own account vending machine. I used the basics from the post and twisted and refined it to fit my use case much better.

The solution described in this post is available on GitHub

The background

Why did I need to create so many accounts and why did I need to create resources in the created accounts?

The need for this come from my work. From time to time I run workshops, often with around 20-30 participants. In the beginning I created one new AWS account for each workshop and then created a separate IAM user for each attendee. After the workshop I just deleted the account and with that all resources the attendees had created. What was the problem with this then? First of all I often run into problems where one attendee had created something that interfered with a resources belonging to a different attendee. Also AWS service limits kept biting me and I had tro rise all the limits I thought needed to be raised. Most times I was able to rise them all but from time to time I forgot about some limit. This created a problem during the workshop. Also I got requests after the workshop that some attendees wanted to keep playing around and asked if they could keep access to the account.

Second we are a small team of Organization admins that create and handle the lab accounts people in the company request. For lab accounts we like to deploy things like Budgets and alerts in the lab account. We also like to create a cross account role so we can access the lab account from our admin account. And on purpose we have separated our admin account from the Organization master account. The reason for this is that the master account also has access to accounts that the admins should not have access to.
But why does admins need access to the lab accounts? The reason for that is that we help out if the owner of the account need help debugging or setting up resources.

Solution Overview

The solution interacts with AWS Organizations to create accounts and Organizational Units (OU). It uses Lambda and Step Functions to coordinate the different steps in the process. Amazon SNS and SES is used for notifications and Amazon S3 is used both for triggering the creation and store information about created accounts.

{:class="img-responsive"}

Creating a new account

The process of creating an account is started by uploading a "Account description" file to the S3 bucket. That will trigger a Lambda function that will fetch data for the account and starting the Step Function, that will coordinate the creation process.
The entire process is fully automatic and both the Organization Admin team and the account owner (IAM user) will be notified when the account is created.

Why Step Function?

The Vending Machine performs several tasks that should be done in sequence, it creates an organization unit, create an account, move the account to the OU, deploys CloudFormation, and more. This is a typical use case for Step Functions as I see it. There is also a need to wait for the account creation to finish before moving on, doing wait and sleep in Lambda is not a good use case I think.

{:class="img-responsive"}

As you might see I use a parallel state to run states in sequence, it might look a bit weird, right? What I want to do is catch errors in any task and move to an error task. Doing that from each and every task gets messy. So I adopted this setup from Serverless Hero Yan Cui.
In the last state I will store the account creation event in S3 for use later, updating the account or similar.

Account description

As mentioned, to start create a new account a json file describing the account should be uploaded to the S3 bucket. In the description file you specify information such the account name, account e-mail, Organization Unit name, account owner (IAM User) that should be an e-mail address. You also specify an array with CloudFormation templates that should be deployed in the child account. When specify the parameters for the CloudFormation templates you can substitute to values available in fields the json file. This is done by setting value to { {json field name} }.
Below is an example of an account description file.

{
    "accountName": "Test Account",
    "accountEmail": "awsaccount@example.com",
    "ouName": "TestAccounts",
    "iamUser": "iamuser@example.com",
    "iamPassword": "InitialPassword",
    "accountRole": "organization-account-role",
    "adminAccount": "123408753636",
    "adminAccountRole": "cross-account-admin-role",
    "cfnTemplates": [
        {
            "templateName": "account-iam-user.yaml",
            "stackName": "avm-owning-iam-user",
            "parameters": [
                {
                    "key": "Username",
                    "value": "{ {iamUser} }"
                },
                {
                    "key": "Password",
                    "value": "{ {iamPassword} }"
                }
            ]
        },
        {
            "templateName": "cross-account-admin.yaml",
            "stackName": "avm-cross-account-admin",
            "parameters": [
                {
                    "key": "RoleName",
                    "value": "{ {adminAccountRole} }"
                },
                {
                    "key": "RequireMFA",
                    "value": "true"
                },
                {
                    "key": "OtherAccountNumber",
                    "value": "{ {adminAccount} }"
                }
            ]
        }
    ]
}

During the account creation process additional fields will actually be added to the description file. That is:

{
    "rootOuId": "r-xxxx",
    "ouId": "ou-xxxx-yyyyyyyy",
    "accountRequestId": "car-hgdjagdgjdgsdh",
    "createAccountStatus": "STATUS",
    "accountId": "4963463846832"
}

Account storage

After the account has been created the full account description file will be stored in a S3 bucket with a folder structure reflecting organization unit structure.
The files can then be index by Glue and Athena for search and query. It would also be possible (in the future) to feed the files back into the vending machine to update CloudFormation templates and similar.

Conclusion

This Account Vending machine help me create new accounts in a fast and dynamic way. There are still some outstanding problems. There is not a way to update already existing accounts with new CloudFormation templates. This is something that is still being developed. The main functionality is however in place.

Feel free to try it and if you do any improvements please contribute them back to me.

You find the code on GitHub