Git Multi-Branch Deployment

Overview

This skill provides guidance for setting up Git-based deployment systems where multiple branches (e.g., main, dev) automatically deploy to different web server locations via post-receive hooks. It covers SSH configuration, Git repository setup, web server configuration, and automated deployment workflows.

Prerequisites Verification

Before starting, verify all required tools and configurations:

• Check for required packages: git, openssh-server, sshpass (for testing), web server (nginx/apache)
• Verify port availability for SSH (22) and HTTP/HTTPS (80/443)
• Confirm user creation permissions
• Set Git default branch name upfront: git config --global init.defaultBranch main

Approach: Step-by-Step Implementation

1. User and SSH Setup

Actions:

• Create a dedicated Git user with home directory
• Set password for SSH authentication
• Configure SSH to allow password authentication

SSH Configuration Considerations:

• Modify /etc/ssh/sshd_config settings: PasswordAuthentication yes, ChallengeResponseAuthentication yes
• Consider additional settings that may interfere: UsePAM, KbdInteractiveAuthentication
• Restart SSH service after configuration changes
• Plan for SSH host key verification in testing (either pre-accept keys or handle StrictHostKeyChecking)

2. Git Repository Creation

Actions:

• Create a bare Git repository in the Git user's home directory
• Initialize with proper permissions
• Create initial branches with content

Branch Setup Pattern:

1. Clone the bare repo to a temp location
2. Create initial commit on main branch
3. Create and populate additional branches (e.g., dev)
4. Push all branches to the bare repo

3. Web Server Configuration

Actions:

• Create deployment directories for each branch (e.g., /var/www/main, /var/www/dev)
• Set appropriate ownership and permissions for the Git user
• Configure virtual hosts or server blocks for each deployment target
• Handle SSL certificates if HTTPS is required

SSL Certificate Generation (self-signed):

• Use openssl req with appropriate subject and SAN extensions
• Configure web server to use the certificates with modern TLS protocols

4. Post-Receive Hook Creation

Actions:

• Create executable post-receive hook in hooks/ directory of bare repo
• Parse ref updates to determine which branch was pushed
• Checkout appropriate branch to corresponding deployment directory

Hook Structure:

1. Read stdin for ref updates (oldrev newrev refname)
2. Parse branch name from refname
3. For each target branch, checkout to deployment directory using GIT_WORK_TREE

5. Service Configuration

Actions:

• Start and enable required services (SSH, web server)
• Configure services to start on boot if persistence is needed
• Verify services are running and accessible

Verification Strategies

SSH Access Verification

• Test SSH connection with password: sshpass -p <password> ssh -o StrictHostKeyChecking=no <user>@<host>
• Verify Git operations over SSH work correctly

Git Operations Verification

• Clone the repository via SSH
• Create test commits and push to each branch
• Verify pushes complete without errors

Deployment Verification

• After each push, verify content appears in correct deployment directory
• Test web server serves correct content for each branch/endpoint
• Verify deployment completes within required time constraints

End-to-End Testing Pattern

1. Clone repository to fresh test directory
2. Make changes to branch content
3. Push changes
4. Immediately verify web endpoint reflects changes
5. Time the deployment if latency requirements exist

Common Pitfalls and Mistakes

Branch Naming Confusion

• Problem: Git may default to master instead of main
• Solution: Set git config --global init.defaultBranch main before creating repositories
• Recovery: Rename branch with git branch -m master main if already created

SSH Configuration Incomplete

• Problem: SSH connections fail despite configuration changes
• Causes: Missing settings like UsePAM, KbdInteractiveAuthentication, or service not restarted
• Solution: Verify all related SSH settings and restart sshd after changes

Host Key Verification Failures

• Problem: SSH commands fail with host key verification errors
• Solutions:
  • Pre-accept host keys: ssh-keyscan -H <host> >> ~/.ssh/known_hosts
  • Use StrictHostKeyChecking=no for testing (document security implications)

Push Conflicts During Testing

• Problem: Non-fast-forward errors when re-testing
• Causes: Previous test data conflicts with new pushes
• Solutions:
  • Use fresh test directories for each test run
  • Force-push if appropriate: git push --force
  • Reset repository state between tests

Post-Receive Hook Issues

• Problem: Hook doesn't execute or deployment fails silently
• Causes: Hook not executable, wrong shebang, missing GIT_WORK_TREE
• Verification: Add logging to hook, check permissions with ls -la hooks/

Service State Issues

• Problem: Services not running or not starting on boot
• Solution: Use systemctl enable for boot persistence, verify status before testing

Missing Prerequisites

• Problem: Commands fail because required tools not installed
• Solution: Check all prerequisites at the start, install missing packages before beginning setup

Edge Cases to Consider

Unhandled Branch Pushes

• Decide behavior when branches other than configured ones are pushed
• Document whether silent ignore, warning, or error is appropriate

Failed Deployments

• Consider adding error handling in post-receive hooks
• Decide on notification mechanism for deployment failures

Concurrent Pushes

• Be aware of potential race conditions with simultaneous pushes
• Consider adding locking mechanism for production environments

Permission Issues

• Ensure Git user has write access to all deployment directories
• Verify ownership after creating directories

Efficiency Recommendations

Reduce Repetition

• Set environment variables for repeated SSH commands: export GIT_SSH_COMMAND='sshpass -p password ssh -o StrictHostKeyChecking=no'
• Batch related operations into single shell invocations

Clean Testing

• Use isolated test directories that don't accumulate artifacts
• Clean up test repositories between runs

Command Availability

• Verify command existence before using (e.g., command -v time before timing operations)
• Have fallbacks ready (e.g., date +%s%3N instead of time)