Migrate to uv Build System

You are an expert at migrating Keboola Python projects to modern pyproject.toml + uv build system with ruff linting. You handle two types of migrations:

1. Python Packages - Published to PyPI, installed by other projects (setup.py → pyproject.toml with build system)
2. Keboola Components - Docker-based applications deployed to ECR (requirements.txt → pyproject.toml, no build system)

Phase 0: Determine Migration Type

Always start by detecting or asking the migration type.

Auto-Detection Heuristics

Run these checks:

# Check for package indicators
[ -f setup.py ] && echo "PACKAGE"

# Check for component indicators  
[ -f Dockerfile ] && [ ! -f setup.py ] && echo "COMPONENT"

# Check CI deployment target
grep -q "pypi\|PyPI" .github/workflows/*.yml 2>/dev/null && echo "PACKAGE"
grep -q "ECR\|DEVELOPERPORTAL" .github/workflows/*.yml 2>/dev/null && echo "COMPONENT"

Ask the User

If detection is ambiguous or you want to confirm:

Question: Is this a Python package (published to PyPI) or a Keboola component (Docker-based, deployed to ECR)?

• Package → Follow Package Migration Path
• Component → Follow Component Migration Path

---

Prerequisites Check

Both Types

• Git repository with clean working tree
• Existing Python source code in src/ or similar
• Test suite exists

Package Only

• setup.py exists with package metadata
• PyPI and Test PyPI accounts available
• GitHub secrets configured (UV_PUBLISH_TOKEN)

Component Only

• Dockerfile exists
• requirements.txt exists
• Keboola Developer Portal credentials available

---

Migration Philosophy

Ruff-Only Linting

Modern best practice: Use ruff exclusively, no flake8.

• Ruff is faster, more comprehensive, and actively maintained
• Covers all flake8 checks + pyflakes + isort + pyupgrade
• Single tool instead of multiple linters
• Built-in formatting support

Flexible Commit Strategy

Guideline: Use logical commits for reviewability

1. Linting baseline - Add ruff config, fix all linting issues
2. Metadata migration - Create pyproject.toml, delete old files
3. CI/CD updates - Update workflows/Dockerfile to use uv

Key principle: Each commit should make sense independently

Dependency Pinning Strategy

• Package dependencies: Use >= (minimum version)

  • Example: keboola-component>=1.6.13
  • uv.lock provides determinism, >= in pyproject.toml allows flexibility

• Python version:

  • Packages: requires-python = ">=3.N" (range, test matrix covers multiple versions)
  • Components: requires-python = "~=3.N.0" (pin to major.minor from Dockerfile base image)

Version Strategy [Package Only]

Testing phase: Use next minor version

• Example: Current 1.6.13 → Test as 1.7.0, 1.7.1, 1.7.2

Production release: Use following minor version

• Example: After testing 1.7.x → Release 1.8.0

---

Package Migration Path

Use this path when migrating a Python package published to PyPI.

Phase 1: Analysis [Package]

1. Check current state:

cat setup.py  # Extract dependencies, python_requires, version
cat requirements.txt  # May have additional deps
ls .github/workflows/  # Check for PyPI deployment workflows

2. Identify Python version:

# From setup.py python_requires
# Use this as minimum in pyproject.toml

3. Check for docs:

grep -q pdoc .github/workflows/*.yml && echo "HAS_DOCS"

Phase 2: Linting Baseline [Package]

1. Create pyproject.toml with ruff config:

# We'll add ruff config to pyproject.toml in next phase
# For now, just ensure ruff is available
uv tool install ruff

2. Run ruff and fix issues:

ruff check --fix src/ tests/
ruff format src/ tests/

3. Commit:

git add src/ tests/
git commit -m "ruff linting baseline 🎨"

Phase 3: Package Metadata [Package]

1. Create pyproject.toml:

[project]
name = "package-name"
version = "0.0.0"  # Replaced by git tags in CI
description = "Short description"
readme = "README.md"
requires-python = ">=3.N"
license = "MIT"
authors = [
    { name = "Keboola", email = "support@keboola.com" }
]
classifiers = [
    "Programming Language :: Python :: 3",
    "Programming Language :: Python :: 3.N",
    # Add supported versions
]

dependencies = [
    "package>=x.y.z",  # From setup.py install_requires
]

[dependency-groups]
dev = [
    "ruff>=0.15.0",
    "pytest>=8.0.0",  # If using pytest
    # Add other dev deps from setup_requires, tests_require
]

[project.urls]
Homepage = "https://github.com/keboola/REPO"
Repository = "https://github.com/keboola/REPO"

[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"

[tool.hatch.build.targets.wheel]
packages = ["src/package_name"]

[tool.ruff]
line-length = 120

[tool.ruff.lint]
extend-select = ["I"]  # Add isort to default ruff rules

[[tool.uv.index]]
name = "test-pypi"
url = "https://test.pypi.org/simple"
explicit = true

2. Delete old files:

git rm setup.py requirements.txt
[ -f .flake8 ] && git rm .flake8
[ -f flake8.cfg ] && git rm flake8.cfg

3. Update LICENSE year:

sed -i 's/Copyright (c) 20[0-9][0-9]/Copyright (c) 2026/' LICENSE

4. Commit:

git add pyproject.toml LICENSE
git commit -m "migrate to pyproject.toml 📦"

Phase 4: CI/CD Workflows [Package]

1. Update workflows (typically push_dev.yml, deploy.yml, deploy_to_test.yml):

Key changes:

# Add uv setup (after checkout and python setup)
- name: Set up uv
  uses: astral-sh/setup-uv@v6

# Replace all pip install → uv sync
- name: Install dependencies
  run: uv sync --all-groups --frozen

# Replace pytest → uv run pytest
- name: Run tests
  run: uv run pytest tests/

# Add ruff linting
- name: Lint with ruff
  uses: astral-sh/ruff-action@v3

# For version replacement in deploy workflows
- name: Set package version
  run: uv version ${{ env.TAG_VERSION }}

# For publishing
- name: Build package
  run: uv build

- name: Publish to PyPI
  env:
    UV_PUBLISH_TOKEN: ${{ secrets.UV_PUBLISH_TOKEN }}
  run: uv publish

Update Python matrix:

strategy:
  matrix:
    python-version: ["3.N", "3.13", "3.14"]  # min + 2 latest

2. Generate uv.lock:

uv sync --all-groups

3. Verify build:

uv build
uv version 1.0.0 --dry-run  # Test version replacement

4. Commit:

git add .github/workflows/*.yml uv.lock
git commit -m "uv 💜"

Phase 5: Test on Test PyPI [Package]

1. Push branch and create test tag
2. Manually trigger Test PyPI workflow
3. Verify installation:

uv init --name test-install
uv add --index-url https://test.pypi.org/simple/ \
       --extra-index-url https://pypi.org/simple/ \
       --index-strategy unsafe-best-match \
       PACKAGE==1.0.0
uv run python -c "import PACKAGE; print('✅')"

Phase 6: Production Release [Package]

1. Create PR, get approval, merge to main
2. Create release tag
3. Verify on PyPI

---

Component Migration Path

Use this path when migrating a Keboola component (Docker-based).

Phase 1: Analysis [Component]

1. Check Dockerfile Python version:

grep "FROM python:" Dockerfile  # e.g., FROM python:3.13-slim

2. Check current dependencies:

cat requirements.txt

3. Check CI workflow:

cat .github/workflows/push.yml

Phase 2: Linting Baseline [Component]

1. Create pyproject.toml with ruff config (we'll complete it in next phase):

[project]
name = "component-name"
dynamic = ["version"]
requires-python = "~=3.N.0"  # Match Dockerfile FROM python:3.N-slim

[tool.ruff]
line-length = 120

[tool.ruff.lint]
extend-select = ["I"]  # Add isort to default ruff rules

2. Run ruff locally:

uv tool install ruff
ruff check --fix src/ tests/
ruff format src/ tests/

3. Commit:

git add pyproject.toml src/ tests/
git commit -m "ruff linting baseline 🎨"

Phase 3: Metadata [Component]

1. Complete pyproject.toml:

[project]
name = "component-name"
dynamic = ["version"]
requires-python = "~=3.N.0"
dependencies = [
    "keboola-component>=1.6.13",
    "package>=x.y.z",
    # From requirements.txt, converted to >=
]

[dependency-groups]
dev = [
    "ruff>=0.15.0",
]

[tool.ruff]
line-length = 120

[tool.ruff.lint]
extend-select = ["I"]

Note:

• No [build-system] - components are not installable packages
• No classifiers - not published to PyPI
• dynamic = ["version"] - version managed elsewhere
• ~=3.N.0 - pins to major.minor, allows patch updates

2. Delete old files:

git rm requirements.txt
[ -f .flake8 ] && git rm .flake8
[ -f flake8.cfg ] && git rm flake8.cfg

3. Update .gitignore:

# Add to .gitignore if not already present:
echo "*.egg-info/" >> .gitignore
echo ".venv/" >> .gitignore

Note: *.egg-info/ is created by uv due to dynamic = ["version"]. It should be gitignored, not committed.

4. Commit:

git add pyproject.toml .gitignore
git commit -m "migrate to pyproject.toml 📦"

Phase 4: Docker and CI [Component]

1. Update Dockerfile:

FROM python:3.N-slim
COPY --from=ghcr.io/astral-sh/uv:latest /uv /uvx /bin/

WORKDIR /code/

# Copy dependency files first (layer caching)
COPY pyproject.toml .
COPY uv.lock .

# Install dependencies into system Python (no venv in Docker)
ENV UV_PROJECT_ENVIRONMENT="/usr/local/"
RUN uv sync --all-groups --frozen

# Copy source code
COPY src/ src
COPY tests/ tests
COPY scripts/ scripts
COPY deploy.sh .

CMD ["python", "-u", "src/component.py"]

Key changes:

• Install uv from official image
• Copy pyproject.toml + uv.lock before source (layer caching)
• UV_PROJECT_ENVIRONMENT="/usr/local/" installs to system Python
• uv sync --all-groups --frozen installs all deps including dev (for tests)
• No pip install, no uv run at runtime

2. Update scripts/build_n_test.sh (if exists):

#!/bin/sh
set -e

ruff check .
python -m unittest discover

3. Modernize tests/__init__.py to use pathlib:

# Before (old os.path pattern):
import sys
import os
sys.path.append(os.path.dirname(os.path.realpath(__file__)) + "/../src")

# After (modern pathlib pattern from cookiecutter):
import sys
from pathlib import Path

sys.path.append(str((Path(__file__).resolve().parent.parent / "src")))

4. Update .github/workflows/push.yml:

Modernize workflow trigger:

# Before (old whitelist pattern):
on:
  push:
    branches:
      - feature/*
      - bug/*
      - fix/*
      - SUPPORT-*
    tags:
      - "*"

# After (modern blacklist pattern from cookiecutter):
on:
  push:  # skip the workflow on the main branch without tags
    branches-ignore:
      - main
    tags:
      - "*"

Change test commands:

# Before:
docker run ${{ env.KBC_DEVELOPERPORTAL_APP }}:latest flake8 . --config=flake8.cfg

# After:
docker run ${{ env.KBC_DEVELOPERPORTAL_APP }}:latest ruff check .

5. Generate uv.lock:

uv sync --all-groups

6. Commit:

git add Dockerfile scripts/ tests/ .github/workflows/ uv.lock
git commit -m "uv 💜"

Phase 5: Test Locally [Component]

# Build Docker image
docker build -t test-component .

# Run linting
docker run test-component ruff check .

# Run tests
docker run test-component python -m unittest discover

# Run component
docker run test-component python -u src/component.py

---

Common Patterns

Python Version Selection

Packages: Use range for broad compatibility

requires-python = ">=3.9"

Components: Pin to Dockerfile base image major.minor

requires-python = "~=3.13.0"  # FROM python:3.13-slim

Dependency Conversion

From requirements.txt:

keboola.component==1.4.4

To pyproject.toml:

dependencies = [
    "keboola-component>=1.4.4",  # Note: dot → dash in name, == → >=
]

Ruff Configuration

Minimal standard config for all Keboola projects:

[tool.ruff]
line-length = 120

[tool.ruff.lint]
extend-select = ["I"]  # Add isort to default ruff rules

Ruff defaults (enabled automatically):

• E4, E7, E9 - pycodestyle error subsets
• F - pyflakes

We add:

• I - isort (import sorting)

---

Success Criteria

Package Success

• ✅ All tests pass with uv locally
• ✅ uv build succeeds
• ✅ Test PyPI release installable
• ✅ Production PyPI release installable
• ✅ CI/CD workflows green

Component Success

• ✅ Docker build succeeds
• ✅ Linting passes in Docker
• ✅ Tests pass in Docker
• ✅ Component runs successfully
• ✅ CI/CD workflow green

---

Troubleshooting

Component: uv.lock not found in Docker build

Error: COPY uv.lock . fails

Fix: Run uv sync --all-groups locally to generate uv.lock before building Docker image

Component: Permission errors with UV_PROJECT_ENVIRONMENT

Error: Cannot write to /usr/local/

Fix: Ensure ENV UV_PROJECT_ENVIRONMENT="/usr/local/" is set before RUN uv sync

Package: Build fails with "no files to ship"

Error: hatchling can't find package files

Fix: Add to pyproject.toml:

[tool.hatch.build.targets.wheel]
packages = ["src/package_name"]

Ruff finding issues flake8 missed

Status: Expected and good! Ruff is more comprehensive than flake8.

Action: Fix the issues. They were always problems, just not caught before.

---

Reference Examples

Packages:

• keboola/python-http-client
• keboola/python-component

Components:

• keboola/component-bingads-ex (commit b72a98b)

---

Remember: This is a build system migration. End users should see no difference except faster dependency resolution and more consistent environments.