The team set min_instances = 3 to eliminate cold starts on the AI workflow service. Reasonable thinking — LLM calls already add 5–15 seconds of latency, a 4-second cold start on the first request afte
The test checked whether the extracted invoice total matched the expected value. It passed nine times out of ten. On the tenth run, the model formatted the number differently — "1,234.56" instead of "
The monthly bill was ten times the estimate. Investigation took two hours. Three things were wrong. A retry loop was calling the model on every validation error — including schema mismatches that woul
The document extraction endpoint was synchronous. A request came in, the service sent the document to OpenAI, waited for the full response, parsed it, wrote to the database, and returned. Each request
The fallback was supposed to be temporary. When the OpenAI API was unavailable, the service would return the last cached result for that user rather than failing the request. Clean degraded experience
The integration looked solid. In testing, the AI-powered document extraction service worked exactly as designed — pull a PDF, send it to the model, parse the structured output, write to the database.
