The service had been live for weeks. Locally, staging, early production — fast, clean, no issues. Then a new client onboarded, traffic doubled overnight, and P99 latency climbed from under 100ms to over 4 seconds.
No 5xx spike. No memory pressure. No CPU ceiling. Just slow.
Two things were wrong. Both came directly from patterns copied out of the FastAPI docs.
Problem 1: A New DB Connection on Every Request
The FastAPI quickstart doesn't show you how to manage shared resources. So most services end up doing this:
# ❌ What most people write first
@app.get("/orders/{order_id}")
async def get_order(order_id: str):
conn = await asyncpg.connect(DATABASE_URL) # new connection every request
order = await conn.fetchrow("SELECT * FROM orders WHERE id = $1", order_id)
await conn.close()
return order
Fine locally. Under 50 concurrent requests on Cloud Run hitting Cloud SQL, this means 50 simultaneous TCP handshakes + authentication attempts. PostgreSQL has a finite connection limit. Requests queue waiting for a slot. Latency compounds.
The fix is a connection pool initialised once at startup via FastAPI's lifespan hook — not per-request, not as a module-level global:
# ✅ The right way
from contextlib import asynccontextmanager
from fastapi import FastAPI
import asyncpg
@asynccontextmanager
async def lifespan(app: FastAPI):
app.state.db = await asyncpg.create_pool(DATABASE_URL, min_size=2, max_size=10)
yield
await app.state.db.close()
app = FastAPI(lifespan=lifespan)
@app.get("/orders/{order_id}")
async def get_order(order_id: str, request: Request):
async with request.app.state.db.acquire() as conn:
return await conn.fetchrow("SELECT * FROM orders WHERE id = $1", order_id)
Same pattern applies to Redis clients, HTTP sessions, and any SDK that's expensive to initialise. If it's shared, it belongs in lifespan.
Problem 2: Stack Traces Leaking to Clients
While diagnosing the latency issue, we found something else in Cloud Logging. Clients were receiving this:
{
"detail": "500: Internal Server Error\nTraceback (most recent call last):\n File \"/app/routers/orders.py\", line 34...\nasyncpg.exceptions.TooManyConnectionsError: ..."
}
Internal file paths. Dependency names. Exception types. Leaking silently for weeks.
FastAPI's default behaviour on an unhandled exception exposes more than you want in production. The fix is a single catch-all exception handler:
# ✅ Global exception handler
import logging
from fastapi import Request
from fastapi.responses import JSONResponse
logger = logging.getLogger(__name__)
@app.exception_handler(Exception)
async def unhandled_exception_handler(request: Request, exc: Exception):
logger.exception(
"Unhandled exception",
extra={"path": request.url.path, "method": request.method}
)
return JSONResponse(
status_code=500,
content={"error": "internal_server_error", "message": "Something went wrong."}
)
Full traceback goes to Cloud Logging — where only your team sees it. Clients get a safe, consistent error shape. One file, set it once, never think about it again.
Three More Things the Docs Skip
1. Structured logging
Default uvicorn logs are human-readable text. Cloud Logging expects JSON. Replace the default logger before your first deploy:
import logging, json, sys
class JSONFormatter(logging.Formatter):
def format(self, record):
return json.dumps({
"severity": record.levelname,
"message": record.getMessage(),
"logger": record.name,
})
handler = logging.StreamHandler(sys.stdout)
handler.setFormatter(JSONFormatter())
logging.root.handlers = [handler]
logging.root.setLevel(logging.INFO)
2. Uvicorn config for Cloud Run
Don't copy --reload from the quickstart into your Dockerfile. For Cloud Run:
CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8080", "--workers", "1", "--loop", "uvloop", "--timeout-keep-alive", "30"]
Single worker per instance (Cloud Run scales horizontally, not vertically), uvloop for async performance, no --reload.
3. A health check that actually checks something
@app.get("/health")
async def health(request: Request):
try:
async with request.app.state.db.acquire() as conn:
await conn.fetchval("SELECT 1")
return {"status": "ok"}
except Exception:
return JSONResponse(status_code=503, content={"status": "degraded"})
A health endpoint that just returns {"status": "ok"} is a lie. Cloud Run routes traffic based on this endpoint — a lying health check sends requests to a broken instance.
The framework didn't fail. The configuration did.
Next: S01E02 — FastAPI + Pydantic in Production: Contracts, Validation, and Versioning.
