Introduction

In this article you will be exposed to creating your own AWS CDK app that retrieves secrets from HashiCorp Vault instance. Please note that these instructions concern the CloudFormation version of CDK, not CDK-TF or CDK-k8s.

This solution uses Python language for constructing a CDK app. This app will generate a CloudFormation template for us and allow us to deploy it.

As part of this solution, we will deploy an EC2 instance with data stored from a key-value retrieved from Vault. Just note that storing unencrypted secrets on a filesystem is a recipe for disaster. More about this later.

Solution

Vault setup

You can install Vault by downloading its single binary in a compressed zip. Those zip files can be found on HashiCorp's website for Vault.

You can download a Linux release with below curl command:

$ curl https://releases.hashicorp.com/vault/1.5.4/vault_1.5.4_linux_amd64.zip -O 

Start dev instance of Vault server. Remember not to use the dev instance in production workloads. HashiCorp has a reference architecture for production type workloads.

$ vault server -dev

In another terminal, store a secret in your vault instance:

$ export VAULT_ADDR="http://127.0.0.1"
$ export VAULT_TOKEN="${root_token_from_vault}"
$ vault kv put /secret/foo "name"="super_secret_data_that_shouldnt_be_stored_decrypted_on_disk"

Cheeky secret, huh?

You can grab the secret using:

$ vualt kv get /secret/foo
====== Metadata ======
Key              Value
---              -----
created_time     2020-10-17T18:01:24.474567948Z
deletion_time    n/a
destroyed        false
version          4

==== Data ====
Key     Value
---     -----
name    super_secret_data_that_shouldnt_be_stored_decrypted_on_disk

CDK setup

Depending on what Linux distribution or macOS setup you are running from, you'll need Python 3.x and NodeJS to run CDK. Figure it out on your own buttercup.

macOS with Homebrew:

$ brew install nodejs10

CentOS Linux:

$ sudo yum install centos-release-scl
$ sudo yum-config-manager --enable rhel-server-rhscl-7-rpms
$ sudo yum install rh-python36 rh-nodejs10

Enable Python 3.6 and NodeJS:

$ scl enable rh-python36 rh-nodejs10 bash

Install CDK using NPM:

$ npm install aws-cdk

Install Python CDK and Vault modules. You'll use PIP wrapped in a Python virtual environment.

$ mkdir -p cdk-app; cd cdk-app
$ pythom -mvenv .
$ source bin/activate
$ pip install aws-cdk.core aws-cdk.aws-ec2 aws-cdk.aws-iam hvac

Initiate a basic CDK app:

$ cdk init $TEMPLATE_NAME --language python

Basic app

Let's start your basic CDK app in python. All we are going to do is just stand up an EC2 instance with an IAM Instance Profile.

In a nutshell CDK apps are Stacks and Apps that are built using basic building blocks known as Constructs.

Constructs are basic cloud components that you typically would represent using CloudFormation resources. Construct library is your CDK reference that you typically would find in CloudFormation. Constructs can come in 3 different flavors: * AWS CloudFormation-only or L1: Lowest components of CloudFormation resource types * Curated or L2: Specific use cases * Patterns or L3: entire AWS architectures

At its highest point, a CDK contains a set of constructs that define a CloudFormation template. At its basic point, a CDK app contains the core module that defines constructs used by CDK itself.

We start our CDK app by importing the CDK modules and creating a construct object. Our construct will create an IAM role, instance profile, and an EC2 instance with some UserData injected.

In addition, we will also pass Vault secret into our UserData to show you how you can store secrets in plaintext on a filesystem. Please note that this is a superbly terribad idea that you should feel horrible about. We're just showcasing a bad example to steer you away from it and to ensure that you really understand what you're copy-pasta-ing from the Internet. Yes, that includes the bad examples of curl | bash all over the Internet.

Anyway...

#!/usr/bin/env python

from aws_cdk import (
    aws_ec2 as ec2,
    aws_iam as iam,
    core
)

class Ec2InstanceStack(core.Stack):
    def __init__(self, app: core.App, id: str, **kwargs) -> None:
        super().__init__(app, id, **kwargs)

In below example, we will create a basic IAM role with a trust policy. Remember to stay in your enabled Python 3.x virtual environment.

# AssumeRole trust relationship must be a JSON object.
assume_role_policy = {
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "Service": [
                    "ec2.amazonaws.com"
                ]
            },
            "Action": [
                "sts:AssumeRole"
            ]
        }
    ]
}
iam_role = iam.CfnRole(
    self, "IamRole",
    role_name='ec2-instance-role',
    description='Created with CDK',
    assume_role_policy_document=assume_role_policy,
    path='/'
)

Let's add an instance profile and an EC2 instance so this is actually useful. Note how each resource in CDK app can be referenced using .ref, like !Ref intrinsic function in CloudFormation.

# IAM instance profile
iam_instance_profile = iam.CfnInstanceProfile(
    self, "IamInstanceProfile",
    roles=[iam_role.ref],
    instance_profile_name='ec2-instance-profile',
    path='/'
)

# EC2 instance
instance_type = ec2.InstanceType("t2.small")
ec2_instance = ec2.CfnInstance(
    self, "Instance",
    availability_zone='us-west-2a',
    image_id='ami-0123456789',
    instance_type='t2.small',
    iam_instance_profile=iam_instance_profile.ref,
    key_name='some_ssh_keypair_name',
    monitoring=False,
    network_interfaces = [
        {
            'deviceIndex': '0',
            'associate_public_ip_address': False,
            'subnet_id': 'subnet-0123456'
        }
    ],
    tags=[
        {
            'key': 'Name',
            'value': "app-server"
        }
    ],
    user_data=(
        "#!/usr/bin/env bash\n"
        "echo \"{0}\" > /tmp/file\n".format(secret_data)
    )
)

Getting secrets from Vault API using Python

For this function, we will use hvac. It's an open source library for interacting with Vault API.

To inject secrets into your basic app, grab them by their key in Vault.

import hvac

hvac_client = hvac.Client(
    url=os.environ['VAULT_ADDR'],
    token=os.environ['VAULT_TOKEN']
)

data = hvac_client.client.secrets.kv.read_secret_version(path=key)

Final app code

In the final example, I stuff my hvac functions into an object so it's easier for me to reference.

#!/usr/bin/env python

from aws_cdk import (
    aws_ec2 as ec2,
    aws_iam as iam,
    core
)
import json
import hvac
import os

class VaultAccess(object):
    def __init__(self):
        self.client = hvac.Client(
            url='http://127.0.0.1:8200',
            token=os.environ['VAULT_TOKEN']
        )

    def get_value(self, key):
        return self.client.secrets.kv.read_secret_version(path=key)

class Ec2InstanceStack(core.Stack):
    def __init__(self, app: core.App, id: str, **kwargs) -> None:
        super().__init__(app, id, **kwargs)

        # Enable Vault
        v = VaultAccess()

        # AssumeRole trust relationship must be a JSON object.
        assume_role_policy = {
            "Version": "2012-10-17",
            "Statement": [
                {
                    "Effect": "Allow",
                    "Principal": {
                        "Service": [
                            "ec2.amazonaws.com"
                        ]
                    },
                    "Action": [
                        "sts:AssumeRole"
                    ]
                }
            ]
        }
        iam_role = iam.CfnRole(
            self, "IamRole",
            role_name='ec2-instance-role',
            description='Created with CDK',
            assume_role_policy_document=assume_role_policy,
            path='/'
        )


        secret_data = v.get_value('/foo')['data']['data']['name']

        instance_type = ec2.InstanceType("t2.small")
        ec2_inst = ec2.CfnInstance(
            self, "Instance",
            availability_zone='us-west-2a',
            image_id='ami-0123456789',
            instance_type='t2.small',
            key_name='some_ssh_keypair_name',
            monitoring=False,
            network_interfaces = [
                {
                    'deviceIndex': '0',
                    'associate_public_ip_address': False,
                    'subnet_id': 'subnet-0123456'
                }
            ],
            tags=[
                {
                    'key': 'Name',
                    'value': "app-server"
                }
            ],
            user_data=(
                "#!/usr/bin/env bash\n"
                "echo \"{0}\" > /tmp/file\n".format(secret_data)
            )
        )

app = core.App()
Ec2InstanceStack(app, "Instance")
app.synth()

Synthesize the CDK app

You can synthesize the CDK app using cdk command:

$ cdk synth > template.yml; less template.yml

References


Comments

comments powered by Disqus