A low cost-per-request claim is meaningless until the architecture, traffic mix and excluded work are visible. We built a citation-first retrieval service for repetitive industrial maintenance questions, exercised it through a public GCP edge, and retained every failed stage beside the final one-million-request result.
What did the final run achieve?
The final mixed run completed 1,000,000 successful HTTP responses with zero errors in 17.1 minutes. It sustained 976.8 requests per second, with 13.45 ms p95, 18.07 ms p99 and 39.70 ms p99.9 end-to-end latency from a same-region Cloud Run load job through a global external Application Load Balancer.
What exactly was in the request path?
A Go API ran on Cloud Run with 2 vCPU, 1 GiB, concurrency 80 and a one-to-ten instance range. Stable fault-code questions first used an exact-match lane. Broader retrieval fused PostgreSQL full-text and 64-dimensional pgvector HNSW candidates in Cloud SQL. Each response returned a grounded extract and three source identifiers.
Is this LLM inference latency?
No. The benchmark intentionally excludes per-request generative-model work. It measures hybrid retrieval, citation assembly and response transfer. Adding an LLM would add model latency, token charges and a separate answer-quality gate. Calling this an uncached LLM cost would be inaccurate.
How did we prevent a cache-only result?
Mixed traffic was 95% from a 100-question hot set and 5% forced cache bypass. The final run recorded 949,697 cache hits and 50,303 misses. A separate forced-miss test sent all 10,000 requests to the database path and completed at 24.96 ms p95 with zero errors.
What failed before the million-request stage?
The first hybrid ranker was fast but recovered the expected source in the top three for only 33.8% of 500 synthetic queries. We rejected it and added an exact fault-code lane, after which the same deterministic construction reached 100% recall@1 and recall@3. A first 1,000-RPS burst also hit cold per-instance caches and reached 221.51 ms p95. Another calibration used a client timeout shorter than the service timeout and exceeded its error budget by nine requests. All three runs remain in the raw evidence.
What did one million requests cost?
Using measured Cloud Run allocation, the exact request count, response-body bytes and public U.S. list rates, the marginal upper bound was $0.85 per million retrieval requests. That includes Cloud Run CPU, memory and requests, load-balancer response processing and a conservative North America internet-egress line. It excludes free tier and discounts.
What would the platform cost each month?
The modeled production baseline is $238.61 per month: regional-HA Cloud SQL at the tested shape, duplicated SSD, one global forwarding rule and one warm Cloud Run instance. Adding measured marginal traffic produces about $242.86 per month at five million retrieval requests, before model inference, ingestion, retained observability, labour and taxes.
What does this add for a field-service business?
The useful wedge is repetitive, source-sensitive troubleshooting: let common fault codes return fast cited procedures, keep an auditable retrieval trail, and reserve slower generation or human escalation for ambiguous cases. The value is not just speed; it is controlling where expensive reasoning is actually used.
What does the quality result not prove?
The 20,000-chunk corpus and its gold queries are deterministic and synthetic. Perfect source recovery proves the test harness and routing rule, not the correctness or safety of real maintenance advice. A customer release still needs approved manuals, real user phrasing, access-control tests, semantic answer grading and a documented escalation path.