CI/CD Expert

Expert guidance for Continuous Integration and Continuous Deployment, including GitHub Actions, Jenkins, GitLab CI, deployment strategies, and automation best practices.

Core Concepts

CI/CD Fundamentals

Continuous Integration (CI)
Continuous Delivery vs Deployment
Build automation
Test automation
Artifact management
Deployment strategies (blue-green, canary, rolling)

Pipeline Design

Pipeline stages and jobs
Parallel execution
Dependencies and artifacts
Caching strategies
Matrix builds
Conditional execution

Security

Secret management
Dependency scanning
SAST/DAST
Container scanning
Supply chain security
SBOM generation

GitHub Actions

Workflow Basics

# .github/workflows/ci.yml
name: CI

on:
  push:
    branches: [main, develop]
  pull_request:
    branches: [main]
  workflow_dispatch:

env:
  NODE_VERSION: '20'
  DOCKER_REGISTRY: ghcr.io

jobs:
  test:
    name: Test
    runs-on: ubuntu-latest

    strategy:
      matrix:
        node-version: [18, 20, 21]

    steps:
      - name: Checkout code
        uses: actions/checkout@v4
        with:
          fetch-depth: 0  # Full history for SonarCloud

      - name: Setup Node.js ${{ matrix.node-version }}
        uses: actions/setup-node@v4
        with:
          node-version: ${{ matrix.node-version }}
          cache: 'npm'

      - name: Install dependencies
        run: npm ci

      - name: Run linting
        run: npm run lint

      - name: Run tests
        run: npm test -- --coverage

      - name: Upload coverage
        uses: codecov/codecov-action@v3
        with:
          token: ${{ secrets.CODECOV_TOKEN }}
          files: ./coverage/coverage-final.json

  build:
    name: Build
    runs-on: ubuntu-latest
    needs: test

    steps:
      - uses: actions/checkout@v4

      - uses: actions/setup-node@v4
        with:
          node-version: ${{ env.NODE_VERSION }}
          cache: 'npm'

      - run: npm ci
      - run: npm run build

      - name: Upload build artifacts
        uses: actions/upload-artifact@v3
        with:
          name: build
          path: dist/
          retention-days: 7

  security:
    name: Security Scan
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v4

      - name: Run Trivy vulnerability scanner
        uses: aquasecurity/trivy-action@master
        with:
          scan-type: 'fs'
          scan-ref: '.'
          format: 'sarif'
          output: 'trivy-results.sarif'

      - name: Upload Trivy results to GitHub Security
        uses: github/codeql-action/upload-sarif@v2
        with:
          sarif_file: 'trivy-results.sarif'

      - name: Run npm audit
        run: npm audit --audit-level=high

Docker Build and Push

# .github/workflows/docker.yml
name: Docker Build and Push

on:
  push:
    branches: [main]
    tags: ['v*']

env:
  REGISTRY: ghcr.io
  IMAGE_NAME: ${{ github.repository }}

jobs:
  build-and-push:
    runs-on: ubuntu-latest
    permissions:
      contents: read
      packages: write

    steps:
      - name: Checkout
        uses: actions/checkout@v4

      - name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v3

      - name: Log in to Container Registry
        uses: docker/login-action@v3
        with:
          registry: ${{ env.REGISTRY }}
          username: ${{ github.actor }}
          password: ${{ secrets.GITHUB_TOKEN }}

      - name: Extract metadata
        id: meta
        uses: docker/metadata-action@v5
        with:
          images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
          tags: |
            type=ref,event=branch
            type=ref,event=pr
            type=semver,pattern={{version}}
            type=semver,pattern={{major}}.{{minor}}
            type=sha,prefix={{branch}}-

      - name: Build and push
        uses: docker/build-push-action@v5
        with:
          context: .
          push: true
          tags: ${{ steps.meta.outputs.tags }}
          labels: ${{ steps.meta.outputs.labels }}
          cache-from: type=gha
          cache-to: type=gha,mode=max
          platforms: linux/amd64,linux/arm64

Deployment Workflow

# .github/workflows/deploy.yml
name: Deploy to Production

on:
  push:
    tags: ['v*']
  workflow_dispatch:
    inputs:
      environment:
        description: 'Environment to deploy to'
        required: true
        type: choice
        options:
          - staging
          - production

jobs:
  deploy:
    name: Deploy to ${{ inputs.environment || 'production' }}
    runs-on: ubuntu-latest
    environment:
      name: ${{ inputs.environment || 'production' }}
      url: https://${{ inputs.environment || 'production' }}.example.com

    steps:
      - uses: actions/checkout@v4

      - name: Configure AWS credentials
        uses: aws-actions/configure-aws-credentials@v4
        with:
          aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
          aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
          aws-region: us-east-1

      - name: Deploy to ECS
        run: |
          aws ecs update-service \
            --cluster ${{ secrets.ECS_CLUSTER }} \
            --service ${{ secrets.ECS_SERVICE }} \
            --force-new-deployment

      - name: Wait for deployment
        run: |
          aws ecs wait services-stable \
            --cluster ${{ secrets.ECS_CLUSTER }} \
            --services ${{ secrets.ECS_SERVICE }}

      - name: Notify Slack
        uses: slackapi/slack-github-action@v1
        with:
          webhook-url: ${{ secrets.SLACK_WEBHOOK }}
          payload: |
            {
              "text": "Deployment to ${{ inputs.environment || 'production' }} successful!",
              "blocks": [
                {
                  "type": "section",
                  "text": {
                    "type": "mrkdwn",
                    "text": "✅ *Deployment Successful*\n*Environment:* ${{ inputs.environment || 'production' }}\n*Version:* ${{ github.ref_name }}"
                  }
                }
              ]
            }

Reusable Workflows

# .github/workflows/reusable-test.yml
name: Reusable Test Workflow

on:
  workflow_call:
    inputs:
      node-version:
        required: true
        type: string
      working-directory:
        required: false
        type: string
        default: '.'
    secrets:
      codecov-token:
        required: true

jobs:
  test:
    runs-on: ubuntu-latest
    defaults:
      run:
        working-directory: ${{ inputs.working-directory }}

    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: ${{ inputs.node-version }}
          cache: 'npm'
          cache-dependency-path: ${{ inputs.working-directory }}/package-lock.json

      - run: npm ci
      - run: npm test -- --coverage

      - uses: codecov/codecov-action@v3
        with:
          token: ${{ secrets.codecov-token }}

# Usage in another workflow
jobs:
  test-backend:
    uses: ./.github/workflows/reusable-test.yml
    with:
      node-version: '20'
      working-directory: './backend'
    secrets:
      codecov-token: ${{ secrets.CODECOV_TOKEN }}

Composite Actions

# .github/actions/setup-project/action.yml
name: 'Setup Project'
description: 'Setup Node.js and install dependencies'

inputs:
  node-version:
    description: 'Node.js version'
    required: false
    default: '20'

runs:
  using: 'composite'
  steps:
    - uses: actions/setup-node@v4
      with:
        node-version: ${{ inputs.node-version }}
        cache: 'npm'

    - name: Install dependencies
      shell: bash
      run: npm ci

    - name: Cache build
      uses: actions/cache@v3
      with:
        path: |
          dist
          .next/cache
        key: ${{ runner.os }}-build-${{ hashFiles('**/package-lock.json') }}

# Usage
steps:
  - uses: actions/checkout@v4
  - uses: ./.github/actions/setup-project
    with:
      node-version: '20'

Jenkins

Declarative Pipeline

// Jenkinsfile
pipeline {
    agent {
        docker {
            image 'node:20-alpine'
            args '-v /var/run/docker.sock:/var/run/docker.sock'
        }
    }

    environment {
        NODE_ENV = 'production'
        DOCKER_REGISTRY = 'ghcr.io'
        IMAGE_NAME = "${env.DOCKER_REGISTRY}/${env.GIT_ORG}/${env.GIT_REPO}"
    }

    options {
        buildDiscarder(logRotator(numToKeepStr: '10'))
        timeout(time: 1, unit: 'HOURS')
        timestamps()
        disableConcurrentBuilds()
    }

    stages {
        stage('Checkout') {
            steps {
                checkout scm
                script {
                    env.GIT_COMMIT_SHORT = sh(
                        returnStdout: true,
                        script: 'git rev-parse --short HEAD'
                    ).trim()
                }
            }
        }

        stage('Install Dependencies') {
            steps {
                sh 'npm ci'
            }
        }

        stage('Lint') {
            steps {
                sh 'npm run lint'
            }
        }

        stage('Test') {
            parallel {
                stage('Unit Tests') {
                    steps {
                        sh 'npm run test:unit -- --coverage'
                    }
                    post {
                        always {
                            junit 'test-results/unit/*.xml'
                            publishHTML([
                                reportDir: 'coverage',
                                reportFiles: 'index.html',
                                reportName: 'Coverage Report'
                            ])
                        }
                    }
                }

                stage('Integration Tests') {
                    steps {
                        sh 'npm run test:integration'
                    }
                    post {
                        always {
                            junit 'test-results/integration/*.xml'
                        }
                    }
                }
            }
        }

        stage('Build') {
            steps {
                sh 'npm run build'
                archiveArtifacts artifacts: 'dist/**/*', fingerprint: true
            }
        }

        stage('Docker Build') {
            when {
                branch 'main'
            }
            steps {
                script {
                    docker.build("${env.IMAGE_NAME}:${env.GIT_COMMIT_SHORT}")
                }
            }
        }

        stage('Security Scan') {
            parallel {
                stage('Dependency Check') {
                    steps {
                        sh 'npm audit --audit-level=high'
                    }
                }

                stage('Container Scan') {
                    when {
                        branch 'main'
                    }
                    steps {
                        sh """
                            trivy image \
                                --severity HIGH,CRITICAL \
                                --exit-code 1 \
                                ${env.IMAGE_NAME}:${env.GIT_COMMIT_SHORT}
                        """
                    }
                }
            }
        }

        stage('Push Image') {
            when {
                branch 'main'
            }
            steps {
                script {
                    docker.withRegistry("https://${env.DOCKER_REGISTRY}", 'docker-credentials') {
                        docker.image("${env.IMAGE_NAME}:${env.GIT_COMMIT_SHORT}").push()
                        docker.image("${env.IMAGE_NAME}:${env.GIT_COMMIT_SHORT}").push('latest')
                    }
                }
            }
        }

        stage('Deploy to Staging') {
            when {
                branch 'main'
            }
            steps {
                script {
                    kubernetesDeploy(
                        configs: 'k8s/staging/*.yaml',
                        kubeconfigId: 'kubeconfig-staging'
                    )
                }
            }
        }

        stage('Smoke Tests') {
            when {
                branch 'main'
            }
            steps {
                sh 'npm run test:smoke -- --env=staging'
            }
        }

        stage('Deploy to Production') {
            when {
                branch 'main'
            }
            input {
                message 'Deploy to production?'
                ok 'Deploy'
            }
            steps {
                script {
                    kubernetesDeploy(
                        configs: 'k8s/production/*.yaml',
                        kubeconfigId: 'kubeconfig-production'
                    )
                }
            }
        }
    }

    post {
        always {
            cleanWs()
        }
        success {
            slackSend(
                color: 'good',
                message: "Build succeeded: ${env.JOB_NAME} #${env.BUILD_NUMBER}"
            )
        }
        failure {
            slackSend(
                color: 'danger',
                message: "Build failed: ${env.JOB_NAME} #${env.BUILD_NUMBER}"
            )
        }
    }
}

Shared Library

// vars/deployToKubernetes.groovy
def call(Map config) {
    def namespace = config.namespace
    def deployment = config.deployment
    def image = config.image

    sh """
        kubectl set image deployment/${deployment} \
            ${deployment}=${image} \
            -n ${namespace}

        kubectl rollout status deployment/${deployment} \
            -n ${namespace} \
            --timeout=5m
    """
}

// Usage in Jenkinsfile
@Library('shared-library') _

pipeline {
    stages {
        stage('Deploy') {
            steps {
                deployToKubernetes(
                    namespace: 'production',
                    deployment: 'web-app',
                    image: "${IMAGE_NAME}:${GIT_COMMIT_SHORT}"
                )
            }
        }
    }
}

GitLab CI

# .gitlab-ci.yml
stages:
  - build
  - test
  - security
  - deploy

variables:
  DOCKER_DRIVER: overlay2
  DOCKER_TLS_CERTDIR: "/certs"
  IMAGE_TAG: $CI_REGISTRY_IMAGE:$CI_COMMIT_SHORT_SHA

default:
  image: node:20-alpine
  cache:
    key: ${CI_COMMIT_REF_SLUG}
    paths:
      - node_modules/
      - .npm/

build:
  stage: build
  script:
    - npm ci --cache .npm --prefer-offline
    - npm run build
  artifacts:
    paths:
      - dist/
    expire_in: 1 week

test:unit:
  stage: test
  needs: [build]
  script:
    - npm ci --cache .npm --prefer-offline
    - npm run test:unit -- --coverage
  coverage: '/All files[^|]*\|[^|]*\s+([\d\.]+)/'
  artifacts:
    reports:
      junit: test-results/unit/*.xml
      coverage_report:
        coverage_format: cobertura
        path: coverage/cobertura-coverage.xml

test:integration:
  stage: test
  needs: [build]
  services:
    - postgres:15
    - redis:7
  variables:
    POSTGRES_DB: testdb
    POSTGRES_USER: testuser
    POSTGRES_PASSWORD: testpass
  script:
    - npm ci --cache .npm --prefer-offline
    - npm run test:integration
  artifacts:
    reports:
      junit: test-results/integration/*.xml

security:sast:
  stage: security
  image: returntocorp/semgrep
  script:
    - semgrep --config=auto --json --output=sast-report.json .
  artifacts:
    reports:
      sast: sast-report.json

security:dependency:
  stage: security
  script:
    - npm audit --audit-level=high
  allow_failure: true

docker:build:
  stage: build
  image: docker:24
  services:
    - docker:24-dind
  before_script:
    - docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
  script:
    - docker build -t $IMAGE_TAG .
    - docker push $IMAGE_TAG
  only:
    - main
    - tags

deploy:staging:
  stage: deploy
  image: bitnami/kubectl:latest
  environment:
    name: staging
    url: https://staging.example.com
  script:
    - kubectl config use-context $KUBE_CONTEXT_STAGING
    - kubectl set image deployment/web-app web-app=$IMAGE_TAG -n staging
    - kubectl rollout status deployment/web-app -n staging --timeout=5m
  only:
    - main

deploy:production:
  stage: deploy
  image: bitnami/kubectl:latest
  environment:
    name: production
    url: https://example.com
  script:
    - kubectl config use-context $KUBE_CONTEXT_PRODUCTION
    - kubectl set image deployment/web-app web-app=$IMAGE_TAG -n production
    - kubectl rollout status deployment/web-app -n production --timeout=5m
  when: manual
  only:
    - tags

Deployment Strategies

Blue-Green Deployment

# GitHub Actions
- name: Blue-Green Deployment
  run: |
    # Deploy to green environment
    kubectl apply -f k8s/green/
    kubectl rollout status deployment/app-green -n production

    # Run smoke tests
    npm run test:smoke -- --env=green

    # Switch traffic to green
    kubectl patch service app -n production -p '{"spec":{"selector":{"version":"green"}}}'

    # Keep blue for rollback
    echo "Blue environment kept for rollback"

Canary Deployment

# Deploy canary (10% traffic)
- name: Deploy Canary
  run: |
    kubectl apply -f k8s/canary/
    kubectl set image deployment/app-canary app=$IMAGE_TAG -n production

    # Monitor metrics
    sleep 300

    # Check error rate
    ERROR_RATE=$(curl -s "$PROMETHEUS_URL/api/v1/query?query=error_rate" | jq -r '.data.result[0].value[1]')

    if (( $(echo "$ERROR_RATE < 0.01" | bc -l) )); then
      # Promote canary to stable
      kubectl set image deployment/app-stable app=$IMAGE_TAG -n production
      kubectl scale deployment/app-canary --replicas=0 -n production
    else
      # Rollback canary
      kubectl scale deployment/app-canary --replicas=0 -n production
      exit 1
    fi

Rolling Deployment

apiVersion: apps/v1
kind: Deployment
metadata:
  name: app
spec:
  replicas: 10
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: 2        # Max 2 pods above desired count
      maxUnavailable: 1  # Max 1 pod unavailable during update
  template:
    spec:
      containers:
      - name: app
        image: myapp:v2
        readinessProbe:
          httpGet:
            path: /health
            port: 8080
          initialDelaySeconds: 5
          periodSeconds: 5

Best Practices

Pipeline Design

Keep pipelines fast (< 10 minutes for CI)
Fail fast on errors
Run tests in parallel
Cache dependencies
Use matrix builds for multiple versions
Separate CI and CD pipelines
Make pipelines idempotent

Security

Scan dependencies for vulnerabilities
Scan container images
Use least privilege for credentials
Rotate secrets regularly
Sign commits and artifacts
Use private registries
Implement SBOM generation

Artifact Management

Use semantic versioning
Tag images with git SHA
Store artifacts in registries
Implement retention policies
Generate build manifests
Track provenance

Monitoring & Observability

Track build success rate
Monitor pipeline duration
Alert on failures
Log all deployments
Track deployment frequency
Measure lead time and MTTR

Anti-Patterns to Avoid

❌ No automated tests: Deployments without tests are risky ❌ Manual deployments: Automate all deployments ❌ Shared credentials: Use role-based access ❌ No rollback strategy: Always have a rollback plan ❌ Long-running pipelines: Keep pipelines fast ❌ Environment drift: Use IaC for all environments ❌ No monitoring: Track deployment health ❌ Direct production access: Deploy through pipelines only

Resources

GitHub Actions: https://docs.github.com/en/actions
Jenkins: https://www.jenkins.io/doc/
GitLab CI: https://docs.gitlab.com/ee/ci/
Argo CD: https://argo-cd.readthedocs.io/
Tekton: https://tekton.dev/docs/

cicd-expert

CI/CD Expert

Core Concepts

CI/CD Fundamentals

Pipeline Design

Security

GitHub Actions

Workflow Basics

Docker Build and Push

Deployment Workflow

Reusable Workflows

Composite Actions

Jenkins

Declarative Pipeline

Shared Library

GitLab CI

Deployment Strategies

Blue-Green Deployment

Canary Deployment

Rolling Deployment

Best Practices

Pipeline Design

Security

Artifact Management

Monitoring & Observability

Anti-Patterns to Avoid

Resources