From Github to AWS CI/CD and ECS

on

AWS_CICD
AWS CICD

We are going to deploy our application in Github to ECS (Fargate), using a CICD pipeline in AWS.

Architecture

Overall design
Overall design

Github(Source stage)

In this pipeline, the first stage is source code management. We used GitHub since it has more features compared to AWS CodeCommit and we are more familiar with Github.

Connecting Github to build stage: When creating the pipeline, you can select Github as the source.

connect_github_to_pipeline
Connect GitHub to pipeline


AWS CodeBuild

In this stage, we added all the relevant configurations. I will mention some of those here.
  • Environment image: aws/codebuild/amazonlinux2-x86_64-standard:3.0.
  • Enable "New service role".
  • Select the VPC:  This is optional. In our case, we selected the VPC, subnets, and an appropriate security group because our build needs to access to the RDS during the build.
  • Enable "Use a buildspec file".
  • Enable CloudWatch logs.

During the build stage, we build the docker image and push it to ECR. builspec.yml contains all the commands and hooks.

version: 0.2
phases:
  install:
    runtime-versions:
      java: corretto11
  pre_build:
    commands:
      - echo Logging in to Amazon ECR...
      - aws --version
      - $(aws ecr get-login --region $AWS_DEFAULT_REGION --no-include-email)
      - REPOSITORY_URI=123123123123.dkr.ecr.ap-southeast-1.amazonaws.com/company/api
      - COMMIT_HASH=$(echo $CODEBUILD_RESOLVED_SOURCE_VERSION | cut -c 1-7)
      - IMAGE_TAG=build-$(echo $CODEBUILD_BUILD_ID | awk -F":" '{print $2}')
  build:
    commands:
      - echo Build started on `date`
      - echo Running maven tasks on `date`
      - mvn install -DargLine="-DDBURI=company-database.cluster-qweqweqweqwe.ap-southeast-1.rds.amazonaws.com -DDBSECRET=/scrt/api-dev-rds -DORIGINS=http://localhost:3000,https://www.company-admin.com" -Pprod
      - echo Building the Docker image...
      - docker build -t $REPOSITORY_URI:latest .
      - docker tag $REPOSITORY_URI:latest $REPOSITORY_URI:$IMAGE_TAG
  post_build:
    commands:
      - echo Build completed on `date`
      - echo Pushing the Docker images...
      - docker push $REPOSITORY_URI:latest
      - docker push $REPOSITORY_URI:$IMAGE_TAG
artifacts:
  files:
    - appspec.yaml
    - taskdef.json
    - imageDetail.json

Create ECS

Next step to create the ECS to create a Blue/Green Deployment.

Once we have completed the build process, a docker image will be uploaded to the ECR. Next, we need a way to deploy these docker images to the servers. For this, we need ECS.

Amazon Elastic Container Service (Amazon ECS) is a highly scalable, fast, container management service that makes it easy to run, stop, and manage Docker containers on a cluster. 

Our plan is to deploy API to a private subnet behind an application load balancer. The application will be exposed to the world through the load balancer, secured by AWS Cognito.

Let's break down this into sections, so we can understand this better

Task-definition

Task definition is a text file, in JSON format, that describes one or more containers. Task definition is the blueprint of the application. In order to run your Docker containers in ECS, you must create a task definition. Check the below taskdef.json for our project.


At first, this might seem too much, but it is much easier to use the console to create a task definition.
Check the below image, there are few definitions and the one we are using for this deployment is highlighted.

Task_definitions
Task definitions

Check the next image, it shows an overview of the ECS and you will see that there are one service and one running task which was created using the task definition.

ECS overview
ECS overview

Fargate or EC2? Fargate removes the need to provision and manage servers. When using fargate, you don't manage your own ec2 instances. It removes one of your concerns and gives more time to focus on the application.

How does a task definition connect to services? The next section contains more details about this.

Services

Service enables you to run and maintain a specified number of instances of a task definition simultaneously in an Amazon ECS cluster. Task definition and services combined can be used to run docker images pulled from ECR.

Once you have created a task definition in ECS, you can specify the number of tasks that you want to run.  To run and maintain the specified number of tasks simultaneously you have to use services.

ECS services and tasks (Fargate)

ECS blue/green - CodeDeploy

How to connect ECS blue/green to CodeDeploy? First, we finished creating the ECS cluster, service, and task definition to support blue/green deployment. We used AWS Fargate instead of ec2. 

This will create a deployment project in AWS CodeDeploy.

Next, you have to connect this CodeDeploy project to your pipeline. In order to successfully deploy your application, you need to have a taskdef.json file and appspec.yaml file in your code base.

To create the taskdef file, we copied the JSON content from the task definition that we created from the console.
Taskdef JSON
Taskdef JSON

Check the content of the appspec.yaml below

version: 0.0
Resources:
  - TargetService:
      Type: AWS::ECS::Service
      Properties:
        TaskDefinition: "arn:aws:ecs:ap-southeast-1:123123123123:task-definition/tom-java-taskdef-fargate:10"
        LoadBalancerInfo:
          ContainerName: "tomcat-java-container"
          ContainerPort: 80

Finishing the pipeline

Once all these stages are integrated, we were able to finish the CICD pipeline.
CICD pipeline
CICD pipeline

Exposing API to the internet.

We need to expose our API to the internet in order for our mobile app and web UI to communicate with it. When implementing the ECS, we used Fargate in private subnets. We used an application load balancer with the target groups in the blue/green deployment. You need to have the application load balancer ready when implementing the ECS.

Securing the API

To secure our API, we used AWS Cognito.

Cognito_integration
AWS Cognito integration

Our API handles authenticated requests after the application has obtained an access token. Our API (or in this case, the resource server) is able to verify the access token obtained by the client from AWS Cognito.

How to update the Task definition?

Example: If your API uses values obtained from the  Environment variables and you want to add more Environment variables to the containers, you will have to update your task definition.

To do that, first, you have to create a new revision of the task definition. Once done, update the service to use the new task definition. This will execute a new deployment.

Once the deployment is done, copy the content from the JSON tab in the new revision of the task definition and paste it into the taskdef.json. Update the appspec.yaml with the new task definition arn.

Push the changes to GitHub to trigger a build.

Conclusion

  1. Connect GitHub to the source stage.
  2. Add the build stage.
    1. maven install.
    2. build a docker image.
    3. push to ECR.
  3. Create an application load balancer.
  4. Create ECS with blue/green deployment support.
  5. Add ECS blue/green deployment to the deploy stage.
  6. Push changes to GitHub and test.

Reference:


NOTE: If you have different opinions or see any issues in this article, please comment and let us know.

Comments

Post a Comment