Host Your Own Docker Registry for CI/CD

Hey folks! Recently, we published a guide on deploying your Reflex app to DigitalOcean. During that process, we encountered the need for a Docker registry to host our Docker images built during GitHub Actions workflows, our continuous deployment pipeline. Today, we'll show you how to quickly set up your own registry on your infrastructure, whether it's a VPS, EC2 instance, DigitalOcean Droplet, or any similar server with a public DNS name.

Background

When working with containerized applications and CI/CD pipelines, having your own Docker registry can offer several benefits:

1. Control: You have full authority over your image storage and distribution.
2. Privacy: Keep your images private and secure within your own infrastructure.
3. Performance: Faster image pulls and pushes when your registry is close to your deployment environment.
4. Cost-effectiveness: For high-volume operations, self-hosting can be more economical than using paid services.

What You Need to Know First

Before we dive in, make sure you have:

1. A server with a public IP address and DNS name (could be on DigitalOcean, AWS, your own hardware, etc.)
2. SSH access to this server
3. Basic knowledge of Docker and Linux command line
4. Docker installed on your server
5. (Optional) A domain name for setting up HTTPS

Setting Up Your Own Docker Registry

Let's walk through the process of setting up your own Docker registry.

1. Prepare Your Server

First, ensure your server is up-to-date:

sudo apt update && sudo apt upgrade -y

If you haven't already, install Docker:

curl -fsSL https://get.docker.com -o get-docker.sh
sudo sh get-docker.sh

2. Create a Storage Location for Your Registry

Create a directory to store your Docker images:

sudo mkdir -p /var/lib/registry

3. Run the Registry Container

Now, let's run the official Docker registry image:

sudo docker run -d \
  -p 5000:5000 \
  --restart=always \
  --name registry \
  -v /var/lib/registry:/var/lib/registry \
  registry:2

This command:

• Runs the registry in detached mode (-d)
• Maps port 5000 on the host to port 5000 in the container
• Sets the container to always restart
• Names the container "registry"
• Mounts the host directory we created to the container

4. Configure Security (Optional but Recommended)

For production use, you should secure your registry with HTTPS and basic authentication.

Set Up HTTPS

1. Obtain an SSL certificate (you can use Let's Encrypt for free certificates)
2. Place your certificate and key in /etc/docker/certs/
3. Run the registry with HTTPS support:

sudo docker run -d \
  -p 5000:5000 \
  --restart=always \
  --name registry \
  -v /var/lib/registry:/var/lib/registry \
  -v /etc/docker/certs:/certs \
  -e REGISTRY_HTTP_TLS_CERTIFICATE=/certs/domain.crt \
  -e REGISTRY_HTTP_TLS_KEY=/certs/domain.key \
  registry:2

Add Basic Authentication

1. Create a password file:

docker run --entrypoint htpasswd httpd:2 -Bbn username password > auth/htpasswd

2. Run the registry with authentication:

sudo docker run -d \
  -p 5000:5000 \
  --restart=always \
  --name registry \
  -v /var/lib/registry:/var/lib/registry \
  -v /etc/docker/certs:/certs \
  -v /auth:/auth \
  -e REGISTRY_HTTP_TLS_CERTIFICATE=/certs/domain.crt \
  -e REGISTRY_HTTP_TLS_KEY=/certs/domain.key \
  -e REGISTRY_AUTH=htpasswd \
  -e REGISTRY_AUTH_HTPASSWD_PATH=/auth/htpasswd \
  -e REGISTRY_AUTH_HTPASSWD_REALM="Registry Realm" \
  registry:2

5. Test Your Registry

You can test your registry by pushing an image to it:

docker pull ubuntu:latest
docker tag ubuntu:latest your-server.com:5000/my-ubuntu
docker push your-server.com:5000/my-ubuntu

Replace your-server.com with your server's domain or IP address.

Using Your Registry in CI/CD

Now that your registry is set up, you can use it in your CI/CD pipelines. Here's an example of how to modify your GitHub Actions workflow to use your private registry:

name: CI/CD Pipeline

on:
  push:
    branches: [ main ]

jobs:
  build-and-push:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v2

    - name: Build and push Docker image
      env:
        REGISTRY: your-server.com:5000
        IMAGE_NAME: my-app
        TAG: ${{ github.sha }}
      run: |
        echo ${{ secrets.REGISTRY_PASSWORD }} | docker login $REGISTRY -u ${{ secrets.REGISTRY_USERNAME }} --password-stdin
        docker build -t $REGISTRY/$IMAGE_NAME:$TAG .
        docker push $REGISTRY/$IMAGE_NAME:$TAG

Make sure to set up the REGISTRY_USERNAME and REGISTRY_PASSWORD secrets in your GitHub repository settings.

Conclusion

Setting up your own Docker registry gives you full control over your container images and can be an essential part of your CI/CD pipeline. While it requires some initial setup and ongoing maintenance, it offers flexibility and potentially cost savings for teams with specific needs or high-volume operations.

Remember to keep your registry and the server it's hosted on up-to-date and secure. Regularly backup your image data and monitor the health of your registry to ensure smooth operations.

Happy containerizing!