Docker for Backend Engineers Who Aren't DevOps
Most backend engineers learn Docker by copying a Dockerfile from the internet, running it locally, and moving on. That works until it doesn't — images that are 2GB when they should be 200MB, builds th
#fastapi
Posts tagged #fastapi
Deploying AI Workflows on Cloud Run: Latency, Cold Starts, and Scaling
The team set min_instances = 3 to eliminate cold starts on the AI workflow service. Reasonable thinking — LLM calls already add 5–15 seconds of latency, a 4-second cold start on the first request afte
Testing AI-Dependent Code: Mocking LLMs, Evaluating Outputs, and Avoiding Flaky Tests
The test checked whether the extracted invoice total matched the expected value. It passed nine times out of ten. On the tenth run, the model formatted the number differently — "1,234.56" instead of "
Prompt Versioning and Cost Control: Running LLMs Responsibly in Production
The monthly bill was ten times the estimate. Investigation took two hours. Three things were wrong. A retry loop was calling the model on every validation error — including schema mismatches that woul
Async AI Pipelines in FastAPI: Streaming, Queuing, and Long-Running Requests
The document extraction endpoint was synchronous. A request came in, the service sent the document to OpenAI, waited for the full response, parsed it, wrote to the database, and returned. Each request
Structuring OpenAI and Gemini Calls in FastAPI: Prompts, Fallbacks, and Retries
The fallback was supposed to be temporary. When the OpenAI API was unavailable, the service would return the last cached result for that user rather than failing the request. Clean degraded experience
