Dockerize a Python AI app

Beginner · 15 min · Docker Desktop or Docker Engine

"Works on my machine" stops being a problem when your machine ships with the app. A Docker image bundles your code, the exact Python version and every dependency, and runs the same on your laptop, a teammate's Mac and any cloud. It's also what most hosting platforms (Railway, Render, Fly.io, Cloud Run, ECS) deploy.

Install Docker Desktop (Windows/macOS) or Docker Engine (Linux), start it, and check: docker --version.

---

1. The app

A minimal FastAPI app that calls an LLM. Any app works the same way:

chat-api/
├── app/
│   └── main.py
├── requirements.txt
├── .env              # your keys: never goes into the image
├── .dockerignore
└── Dockerfile

app/main.py

import os
from fastapi import FastAPI
from openai import OpenAI

app = FastAPI()
client = OpenAI()                                    # reads OPENAI_API_KEY from the environment
MODEL = os.getenv("MODEL", "gpt-4o-mini")

@app.get("/health")
def health():
    return {"ok": True}

@app.post("/ask")
def ask(question: str):
    reply = client.chat.completions.create(model=MODEL, messages=[{"role": "user", "content": question}])
    return {"answer": reply.choices[0].message.content}

requirements.txt

fastapi==0.115.6
uvicorn[standard]==0.34.0
openai==1.59.7

Pin exact versions (copy them from pip freeze): an image built next month should contain the same packages as one built today.

---

2. The Dockerfile

Dockerfile

# 1. Small official Python base image
FROM python:3.12-slim

# 2. Sensible Python defaults for containers
ENV PYTHONDONTWRITEBYTECODE=1 \
    PYTHONUNBUFFERED=1 \
    PIP_NO_CACHE_DIR=1

WORKDIR /app

# 3. Dependencies first: this layer is cached until requirements.txt changes
COPY requirements.txt .
RUN pip install -r requirements.txt

# 4. Then your code (changes often, so it comes last)
COPY app/ ./app/

# 5. Don't run as root
RUN useradd --create-home appuser
USER appuser

EXPOSE 8000
CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8000"]

Why each part matters:

• -slim images are a few hundred MB smaller than the full ones and have everything most AI apps need.
• Copy requirements.txt before the code. Docker caches each step; editing main.py then only re-runs the last steps, so rebuilds take seconds instead of reinstalling every package.
• PYTHONUNBUFFERED=1 makes print and logs appear immediately in docker logs.
• --host 0.0.0.0 is required: 127.0.0.1 inside a container is unreachable from outside it.
• Non-root user: if the app is ever compromised, the attacker isn't root inside the container.

Using uv instead of pipStreamlit app

FROM python:3.12-slim
COPY --from=ghcr.io/astral-sh/uv:latest /uv /usr/local/bin/uv
WORKDIR /app
COPY pyproject.toml uv.lock ./
RUN uv sync --frozen --no-dev --no-install-project
COPY app/ ./app/
ENV PATH="/app/.venv/bin:$PATH"
CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8000"]

Pin the uv image tag (for example uv:0.8) in real projects. See Manage GenAI projects with uv.

Replace the last two lines:

EXPOSE 8501
CMD ["streamlit", "run", "app.py", "--server.address=0.0.0.0", "--server.port=8501"]

---

3. Keep secrets and junk out: .dockerignore

COPY copies everything it's pointed at. Without a .dockerignore, your .env (with API keys), your 2 GB virtual environment and your Git history can end up inside the image, where anyone who pulls it can read them:

.dockerignore

.env
.env.*
.venv/
venv/
__pycache__/
*.pyc
.git/
.pytest_cache/
.ipynb_checkpoints/
data/
models/
chroma_db/

Never put API keys in the Dockerfile (ENV OPENAI_API_KEY=...) either: they're stored in the image layers.

---

4. Build and run

docker build -t chat-api .
docker run --rm -p 8000:8000 --env-file .env chat-api

• -t chat-api names the image.
• -p 8000:8000 maps port 8000 on your machine to port 8000 in the container.
• --env-file .env passes your keys at run time, so they never touch the image.

Then open http://localhost:8000/docs or:

curl http://localhost:8000/health
curl -X POST "http://localhost:8000/ask?question=What+is+RAG"

Useful commands:

Command	Does
docker ps	Running containers
docker logs -f <id>	Follow a container's output
docker exec -it <id> bash	A shell inside a running container (debugging)
docker images	Images and their sizes
docker system prune	Free disk space from old images and containers

---

5. Add a vector database with Docker Compose

Real RAG apps need more than one container. Docker Compose starts them together from one file:

compose.yaml

services:
  api:
    build: .
    ports: ["8000:8000"]
    env_file: .env
    environment:
      DATABASE_URL: postgresql://rag:rag@db:5432/rag     # "db" is the service name below
    depends_on:
      db:
        condition: service_healthy

  db:
    image: pgvector/pgvector:pg16
    environment:
      POSTGRES_USER: rag
      POSTGRES_PASSWORD: rag
      POSTGRES_DB: rag
    volumes: ["pgdata:/var/lib/postgresql/data"]          # data survives restarts
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U rag"]
      interval: 5s
      retries: 10

volumes:
  pgdata:

docker compose up --build          # build and start both; Ctrl+C to stop
docker compose up -d               # in the background
docker compose logs -f api
docker compose down                # stop (add -v to also delete the database volume)

Containers reach each other by service name (db:5432), not localhost. For Chroma instead, use the chromadb/chroma image and connect with chromadb.HttpClient(host="chroma", port=8000).

---

6. Common errors

Error	Cause	Fix
Browser can't reach the app	Server bound to 127.0.0.1, or no -p	--host 0.0.0.0 and -p 8000:8000
ModuleNotFoundError in the container	Package missing from requirements.txt	Add it (pinned), rebuild
AuthenticationError / key is None	Keys not passed in	--env-file .env (or set variables on the platform)
Every build reinstalls everything	Code copied before requirements	Copy and install requirements first
Image is 3+ GB	torch with CUDA, or .venv copied	.dockerignore; CPU-only torch wheels if you don't need a GPU
Can't reach Ollama on the host	localhost means the container itself	Use http://host.docker.internal:11434
exec format error on a server	Built on Apple Silicon (ARM), running on x86	docker build --platform linux/amd64 ...

---

Checklist

• .dockerignore excludes .env, venvs, data and Git history.
• Requirements are pinned and installed before the code is copied.
• Keys come from the environment at run time, never from the image.
• The app listens on 0.0.0.0 and has a /health endpoint.
• The container runs as a non-root user.

More in Docker and deployment.

Next: Deploy your AI app for free →