Getting Started with AWS CodePipeline, AWS CDK, and GitHub Repositories

A quick guide to AWS CDK and AWS CodePipeline for beginners

Photo by Florian Olivo on Unsplash

Prerequisites and AWS CDK Basics

The AWS Cloud Development Kit (CDK) is an infrastructure-as-code (IaC) framework that allows developers to define the cloud resources needed to deploy, run, and monitor their applications using their preferred programming language. Building out application infrastructure using AWS CDK offers several benefits over configuring resources manually in the console. Namely, code that defines cloud resources is no different than any other code. It can be checked into a source control repository, shared amongst developers, tested, reused, etc.

For those completely new to AWS CDK and want to learn the basics, I recommend cdkworkshop.com. Going through this workshop will teach you the fundamentals and workflow of AWS CDK. You will define resources and deploy a simple API and learn what are and how to use CDK Constructs and Stacks. After completing the base content of the workshop, definitely go through the CDK Pipelines section in the Advanced Topics. We will be building off of key concepts introduced here for this article.

The workshop also details some prerequisites such as setting up an AWS account that are needed to use AWS CDK. These prerequisites are also needed to follow along in the pipeline we will be building in this article. If you have all that’s needed, it’s time to build your own pipeline!

Create A New Project and GitHub Repository

To get started, let’s create our CDK project and a GitHub repository we will push our code to.

First, create a directory for our project:

mkdir cdk-pipeline && cd cdk-pipeline

Then, initialize the CDK project (we’ll be using Typescript):

cdk init app --language typescript

Project directory after CDK initialization

After initializing our CDK project, the project directory will now contain everything needed to get started. Be sure to remove the autogenerated test/ directory and any test files within it. We don’t need those for this simple project.

Let’s jump into GitHub and create a new remote repository. After choosing a name/description and creating the repository, you’ll see instructions for pushing your code:

Instructions for pushing existing repository to GitHub

Follow these instructions in you CDK project directory to do an initial push to GitHub.

Create and Add GitHub Access Token to AWS Secrets Manager

In order to access the GitHub repository we created in the previous section, AWS needs an access token to authenticate to the GitHub API. First, we need to create an GitHub access token. Go to your Account Settings (not repository settings), Developer Settings, then Personal access tokens.

At this menu, select Generate new token to be taken to the new access token creation menu. Enter a descriptive note and select the repo and admin:repo_hook scopes. This will give AWS the required permissions to create a webhook for detecting new commits and reading the repository in CodePipeline. Generate the token and you’ll see the generated access token string, be sure to save this because this is what we’ll be storing in AWS Secrets Manager.

Adding GitHub Access Token to AWS Secrets Manager

Why do we need something like AWS Secrets Manager to store our access token? Hardcoding the access token string into our CDK code and checking that in to our GitHub repository is a huge security risk. Keeping the access token in Secrets Manager avoids this risk while also allowing us to access the access token value via a simple API call using the AWS SDK. Furthermore, Secrets Manager centralizes where secrets are stored so multiple applications can access them and provides other features such as credential rotation which is essential for production use-cases.

To add our access token to Secrets Manager, navigate to Secrets Manager in the AWS console and select Store a new secret. You’ll be prompted to select a secret type, select Other type of secrets. Under “Specify the key/value pairs to be stored in this secret”, select Plaintext and paste in the access token. It should look something like this:

AWS Secrets Manager Console

Go to the next page and enter a name for the secret. The name is what we’ll be using to query the secret. After this, we do not need to configure anything else, so continue going to Configure Rotation and Review pages and select Store to store your new secret.

Now, we are ready to start developing the pipeline in our CDK project!

Defining Our Pipeline Stack

The rest of this article will be focused on building out our CDK pipeline and making sure it can pull changes from our GitHub repository. First we need to install the AWS CDK dependencies we need to define the resources we want.

Install these dependencies in your CDK project:

npm install @aws-cdk/aws-codebuild @aws-cdk/aws-codepipeline @aws-cdk/aws-codepipeline-actions @aws-cdk/pipelines @aws-cdk/core

We will be using these packages to build out our pipeline. In the future, when a different AWS resource/service is needed, like S3, we’d have to install the aws-cdk packages related to that resource in order to use it within a CDK project.

Open up your CDK project directory and change the cdk.json file to this.

This configuration will allow the creation of CDK Pipelines. Now we can add the actual pipeline stack and stages to pipelinelib/cdk-pipeline-stack.ts.

Let’s break down what’s going on here piece-by-piece. The first few steps involve initializing some variables we need in the pipeline:

1. We define a CdkPipelineStack class that extends cdk.Stack. This is the pipeline stack that will house any source and build stages we define within in.
2. The GitHub access token is retrieved from AWS Secrets Manager and stored in a variable.
3. Source and cloud assembly artifacts are defined. They will store the repository source code and the generated the cloud formation template, respectively

Once these are set up, we define the CdkPipeline construct itself:

1. The cloud assembly artifact from earlier is passed in as a prop.
2. A sourceAction object is passed in. This is how we get the source code from GitHub. We create a new GitHubSourceAction and pass in the source artifact, the access token, name of the repo owner, the repo name, branch and trigger.
3. Finally, a synthAction prop is defined. The synth action builds the source code and synthesizes the output into the cloud assembly object to be used to mutate the pipeline.

And that’s it.

Initial CDK Deploy

Let’s try deploying our pipeline stack to AWS. Try:

cdk synth && cdk deploy

You may need to run cdk bootstrap, CLI tool will tell you if need to before running the above commands.

Navigate to the CloudFormation console in AWS to see the CloudFormation template being created.

CloudFormation console

The deploy will take a few minutes to set all the AWS resources up. Once deployed, go to the CodePipeline console to view the newly created pipeline!

CodePipeline console

Looks like the Source stage failed, if we check the details it looks like AWS could not any branch “main” in the repository. That’s because we haven’t pushed our code to our repository yet.

Pulling Source Code from GitHub

Go to your repository for this project. Then go to the Webhooks section in the repository Settings. You should see a webhook that was created by AWS:

GitHub repository settings

This verifies that AWS did indeed find our repository. Now we just need to push our project code to it. Simply follow the given instructions to push an existing repository. After that’s done, go back to the CodePipeline console in AWS:

AWS CodePipeline console

Success! We are able to push changes to GitHub and have AWS CodePipeline retrieve those changes. Follow pipeline as the Build and UpdatePipeline steps are executed. If all goes well, your console should look something like this:

CodePipeline console

You can try making changes in your CDK code and pushing that to the repository and see the process start over with the new changes. Note that you do not have to use the cdk synth or cdk deploy command again, CDK Pipeline handles that for you.

Tearing Down AWS Resources

Remove Your GitHub Access Token From AWS Secrets Manager

At the time of writing, AWS Secrets Manager charges $0.40 per secret per month. We shouldn’t leave our access token sitting in AWS indefinitely (that’s almost $5.00 per year!), so go to the Secrets Manager console and delete the access token. Simple enough.

Removing CodePipeline and Other CDK Resources

We also don’t want to leave around the other AWS resources we created with CDK. The CDK CLI makes it simple to delete these resources with the command:

cdk destroy

After running this command, you’ll be able to see the CloudFormation deletion in progress. The CodePipeline will also be deleted.

Next Steps

Okay, so you’ve set up a CDK pipeline that you can mutate to fit your application needs. You’ve learned how to hook up a GitHub repository to push code changes to AWS CodePipeline via Webhooks. This is just the beginning of CI/CD with AWS. Here are some next steps you can try on your own to further increase your skills:

1. Add actual application code so that the pipeline picks up changes whenever a new commit is made. This would require adding source and build stages to your pipeline.
2. Add testing and deployments to your pipeline for you application. For this, definitely read up on AWS CodeDeploy.

Resources + Further Reading

1. AWS CDK: https://docs.aws.amazon.com/cdk/latest/guide/home.html
2. AWS Secrets Manger: https://docs.aws.amazon.com/secretsmanager/latest/userguide/intro.html
3. AWS CodePipeline: https://docs.aws.amazon.com/codepipeline/latest/userguide/welcome.html
4. AWS CodeBuild: https://docs.aws.amazon.com/codebuild/latest/userguide/welcome.html