Most CI implementations (e.g. Jenkins, GitHub Actions, etc.) provide a compute environment where we can run our CI pipeline steps.

The CI implementation present a few challenges:

• Lack of tooling to build and deploy requiring custom setup beforehand
• Limited scripting capability
• Company decides to change providers

The following approach attempts to address thse challenges by requiring only a CI environment with docker and docker-compose(optional) installed.

Example

We will look at an NodeJS API (repository).

It has two steps with the following requirements:

1. Build
   • yarn
   • node
2. Deploy
   • yarn
   • node
   • Environment variables
     • AWS_ACCESS_KEY_ID
     • AWS_SECRET_ACCESS_KEY

Define steps

1. Build

   • build.Dockerfile

   • build script

2. Deploy

   • deploy.Dockerfile

   • deploy script

Rather than writing docker run commands for each step, we can use docker-compose to manage them:

• Each docker-compose service (e.g. build and deploy) represents a step in our pipeline
• Each step is configured to run the script in its corresponding container
• The entire project directory is mounted as a volume on the /app path
  • This enables and output from build to be reflected outside the container and thus shared between steps
• Any required environment variables are passed from the CI environment into the container environment
• If the step requires Docker in Docker, e.g. to build images, then it just needs a Docker client pointing to Docker server on the CI environment
  • Mount the Docker socket as a volume

    volumes:
        - /var/run/docker.sock:/var/run/docker.sock

Usage

Depending on how the CI environment requires the steps to be defined. We simply need a place to call:

docker-compose build build
docker-compose run build
docker-compose down

Some CI environments do not isolate steps, so each step could share some resources. This is an issue for docker-compose as our projects will likely be run from similar directories and have similar service names (e.g. build and deploy). To avoid conflicts, we simply need to set the COMPOSE_PROJECT_NAME variable in our CI environment, which will be prepended to all resources created by docker-compose.

GitHub Actions