Recipes
Agent-safe pipeline with secret masking
When agents handle user data, mask before any logging or storage - secrets should never reach an LLM or a kv store unredacted.
Batch LLM with rate limiting
Prevents failure at job 847 of 10,000 - throttle paces the pipeline, tok gates each doc before the API call wastes a request.
Cache LLM response
Avoids duplicate API calls for identical prompts - the hash keys the cache so reruns are free.
Chunked RAG pipeline
Keeps every chunk within the embedding model's token limit - no silent truncation during indexing.
Fetch and summarise a page
Catches oversized pages before the API call - no wasted request on a doc that won't fit in context.
JWT-based key lookup
Ties storage to identity without custom parsing - the JWT carries the key, so the lookup stays stateless and auditable.
Retry flaky API call
Transient 500s don't kill the pipeline - coax retries with backoff so one bad request doesn't stop the run.
Scrub secrets from LLM output
Catches secrets the model echoes back before they reach storage - one leaked API key in kv is a breach.
SSE stream to text
Raw SSE is unusable without parsing - sse extracts the content field so downstream tools get clean text, not protocol framing.
Token-checked LLM call
Prevents silent truncation - the model never sees a prompt it can only half-fit.
Validate LLM output before it propagates
Bad structured output exits 1 before reaching downstream systems - you catch schema drift at the source, not in production.