What the FastAPI Docs Don't Tell You About Production

The service had been live for weeks. Locally, staging, early production — fast, clean, no issues. Then a new client onboarded, traffic doubled overnight, and P99 latency climbed from under 100ms to over 4 seconds.

No 5xx spike. No memory pressure. No CPU ceiling. Just slow.

Two things were wrong. Both came directly from patterns copied out of the FastAPI docs.

---

Problem 1: A New DB Connection on Every Request

The FastAPI quickstart doesn't show you how to manage shared resources. So most services end up doing this:

# ❌ What most people write first
@app.get("/orders/{order_id}")
async def get_order(order_id: str):
    conn = await asyncpg.connect(DATABASE_URL)  # new connection every request
    order = await conn.fetchrow("SELECT * FROM orders WHERE id = $1", order_id)
    await conn.close()
    return order

Fine locally. Under 50 concurrent requests on Cloud Run hitting Cloud SQL, this means 50 simultaneous TCP handshakes + authentication attempts. PostgreSQL has a finite connection limit. Requests queue waiting for a slot. Latency compounds.

The fix is a connection pool initialised once at startup via FastAPI's lifespan hook — not per-request, not as a module-level global:

# ✅ The right way
from contextlib import asynccontextmanager
from fastapi import FastAPI
import asyncpg

@asynccontextmanager
async def lifespan(app: FastAPI):
    app.state.db = await asyncpg.create_pool(DATABASE_URL, min_size=2, max_size=10)
    yield
    await app.state.db.close()

app = FastAPI(lifespan=lifespan)

@app.get("/orders/{order_id}")
async def get_order(order_id: str, request: Request):
    async with request.app.state.db.acquire() as conn:
        return await conn.fetchrow("SELECT * FROM orders WHERE id = $1", order_id)

Same pattern applies to Redis clients, HTTP sessions, and any SDK that's expensive to initialise. If it's shared, it belongs in lifespan.

---

Problem 2: Stack Traces Leaking to Clients

While diagnosing the latency issue, we found something else in Cloud Logging. Clients were receiving this:

{
  "detail": "500: Internal Server Error\nTraceback (most recent call last):\n  File \"/app/routers/orders.py\", line 34...\nasyncpg.exceptions.TooManyConnectionsError: ..."
}

Internal file paths. Dependency names. Exception types. Leaking silently for weeks.

FastAPI's default behaviour on an unhandled exception exposes more than you want in production. The fix is a single catch-all exception handler:

# ✅ Global exception handler
import logging
from fastapi import Request
from fastapi.responses import JSONResponse

logger = logging.getLogger(__name__)

@app.exception_handler(Exception)
async def unhandled_exception_handler(request: Request, exc: Exception):
    logger.exception(
        "Unhandled exception",
        extra={"path": request.url.path, "method": request.method}
    )
    return JSONResponse(
        status_code=500,
        content={"error": "internal_server_error", "message": "Something went wrong."}
    )

Full traceback goes to Cloud Logging — where only your team sees it. Clients get a safe, consistent error shape. One file, set it once, never think about it again.

---

Three More Things the Docs Skip

1. Structured logging

Default uvicorn logs are human-readable text. Cloud Logging expects JSON. Replace the default logger before your first deploy:

import logging, json, sys

class JSONFormatter(logging.Formatter):
    def format(self, record):
        return json.dumps({
            "severity": record.levelname,
            "message": record.getMessage(),
            "logger": record.name,
        })

handler = logging.StreamHandler(sys.stdout)
handler.setFormatter(JSONFormatter())
logging.root.handlers = [handler]
logging.root.setLevel(logging.INFO)

2. Uvicorn config for Cloud Run

Don't copy --reload from the quickstart into your Dockerfile. For Cloud Run:

CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8080", "--workers", "1", "--loop", "uvloop", "--timeout-keep-alive", "30"]

Single worker per instance (Cloud Run scales horizontally, not vertically), uvloop for async performance, no --reload.

3. A health check that actually checks something

@app.get("/health")
async def health(request: Request):
    try:
        async with request.app.state.db.acquire() as conn:
            await conn.fetchval("SELECT 1")
        return {"status": "ok"}
    except Exception:
        return JSONResponse(status_code=503, content={"status": "degraded"})

A health endpoint that just returns {"status": "ok"} is a lie. Cloud Run routes traffic based on this endpoint — a lying health check sends requests to a broken instance.

The framework didn't fail. The configuration did.

Next: S01E02 — FastAPI + Pydantic in Production: Contracts, Validation, and Versioning.