Automate Your Python Development Workflow with Makefiles and Shell Scripts

Repetitive tasks slow you down. Running the same sequence of commands to lint, test, and deploy gets old fast and it introduces human error. Two tools that have been around for decades still solve this problem better than most modern alternatives: Makefiles and shell scripts.

This post walks through practical ways to use both in a Python project, with examples you can adapt immediately.

Why Makefiles for Python Projects?

Makefiles were designed for C build systems, but they work well as a general-purpose task runner for any project. The key advantages:

• Zero dependencies - make is available on virtually every Unix-based system.
• Self-documenting - your teammates can run make help to see available commands.
• Consistent interface - regardless of what tools you use under the hood, contributors run the same commands.

A Makefile at the root of your project replaces scattered notes like "remember to run black before committing" with a single, enforced command.

Setting Up a Basic Python Makefile

Create a file named Makefile at the root of your project:

.PHONY: install lint test format clean help

PYTHON := python3
PIP := pip

help: ## Show this help message
    @grep -E '^[a-zA-Z_-]+:.*?## .*$$' $(MAKEFILE_LIST) | \
        awk 'BEGIN {FS = ":.*?## "}; {printf "\033[36m%-15s\033[0m %s\n", $$1, $$2}'

install: ## Install dependencies
    $(PIP) install -r requirements.txt

format: ## Format code with Black
    black .

lint: ## Run flake8 linter
    flake8 . --max-line-length=88 --exclude=.venv,migrations

test: ## Run tests with pytest
    pytest tests/ -v

check: format lint test ## Run format, lint, and tests together

clean: ## Remove Python cache files
    find . -type f -name '*.pyc' -delete
    find . -type d -name '__pycache__' -exec rm -rf {} +

Now instead of remembering multiple commands, your workflow becomes:

make check   # formats, lints, and tests in sequence
make clean   # removes cache artifacts

The .PHONY declaration tells make these are not files, they are always executable regardless of whether a file with that name exists.

Chaining Tasks and Handling Errors

By default, make stops execution if any command returns a non-zero exit code. This is the behavior you want. A failing lint step should block your test run.

If you ever want to intentionally ignore errors for a specific command, prefix it with a dash:

clean:
    -rm -rf .pytest_cache
    -find . -name '*.pyc' -delete

For sequential dependencies, list them as prerequisites:

deploy: check ## Deploy after all checks pass
    echo "Deploying to production..."
    ./scripts/deploy.sh

Running make deploy automatically runs check first.

Using Shell Scripts for More Complex Logic

Makefiles handle task orchestration well, but when you need conditional logic, loops, or user input, a dedicated shell script is cleaner.

Here is an example setup script that creates a virtual environment, installs dependencies, and copies an example environment file:

#!/bin/bash
# scripts/setup.sh

set -e  # Exit immediately on any error

PYTHON=python3
VENV_DIR=".venv"

echo "Creating virtual environment..."
$PYTHON -m venv $VENV_DIR

echo "Activating virtual environment..."
source $VENV_DIR/bin/activate

echo "Installing dependencies..."
pip install --upgrade pip
pip install -r requirements.txt

if [ ! -f .env ]; then
  echo "Copying .env.example to .env..."
  cp .env.example .env
  echo "Update .env with your credentials before running the app."
fi

echo "Setup complete."

Make it executable:

chmod +x scripts/setup.sh

Then reference it in your Makefile:

setup: ## Run project setup script
    ./scripts/setup.sh

New contributors can clone the repo and run `make setup.

Automating Database Migrations in Django

If you work with Django, you probably run makemigrations and migrate constantly. Add them to your Makefile:

migrate: ## Apply database migrations
    $(PYTHON) manage.py migrate

makemigrations: ## Generate new migrations
    $(PYTHON) manage.py makemigrations

shell: ## Open Django shell
    $(PYTHON) manage.py shell

createsuperuser: ## Create a Django superuser
    $(PYTHON) manage.py createsuperuser

These targets are small conveniences individually, but across a week of development they save meaningful time and prevent typos in long commands.

Environment-Specific Variables

You can pass variables into a Makefile from the command line, which makes environment-specific workflows straightforward:

ENV ?= development

run: ## Run the development server
    ENVIRONMENT=$(ENV) $(PYTHON) manage.py runserver

Then call it like this:

make run ENV=staging

This pattern works well when you have different configuration files or Docker Compose setups per environment.

Pre-commit Hooks as a Complement

Makefiles are great for manual and CI runs, but pre-commit hooks enforce standards automatically before each commit. The two approaches complement each other well.

Install pre-commit:

pip install pre-commit

Create .pre-commit-config.yaml:

repos:
  - repo: https://github.com/psf/black
    rev: 24.3.0
    hooks:
      - id: black
  - repo: https://github.com/PyCQA/flake8
    rev: 7.0.0
    hooks:
      - id: flake8

Install the hooks:

pre-commit install

Add a Makefile target for it:

pre-commit: ## Run pre-commit hooks on all files
    pre-commit run --all-files

Integrating with CI/CD

One underrated benefit of a Makefile is that your CI pipeline and local environment run the exact same commands. In GitHub Actions:

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: '3.12'
      - name: Install dependencies
        run: make install
      - name: Run checks
        run: make check

No duplication of logic between your local workflow and CI. If make check passes locally, it will pass in CI, barring any environment differences.

Actionable Takeaways

• Start with five targets: install, lint, test, format, and clean. Expand from there.
• Add a help target using the grep/awk pattern shown above, it makes your Makefile self-documenting.
• Keep shell scripts focused: one script per concern. A setup script should not also deploy your app.
• Use set -e in every shell script to catch errors early instead of silently continuing.
• Mirror your CI commands locally by calling make targets in both places rather than duplicating logic.
• Commit your Makefile to version control as it is part of the project's developer experience, not a personal preference file.

A well-structured Makefile and a handful of focused shell scripts eliminate the cognitive overhead of remembering how to run your project. That consistency compounds over time, especially when onboarding new contributors or switching between multiple projects.