Recipes
Agent-safe pipeline with secret masking
If an agent handles user data, mask it first, so secrets never reach the LLM or the kv store.
Batch LLM with rate limiting
throttle releases filenames at a fixed rate, so each API call stays inside the rate limit.
Cache LLM response
The content hash is the cache key, so a rerun with the same input makes no API call.
Chunked RAG pipeline
Each chunk stays within the token limit of the embedding model, so indexing does not truncate text silently.
Daily token budget
The counter refuses the call once the daily limit is reached, so a runaway loop cannot use up your API budget.
Diff review budget
A diff that is too big fails fast and cheap, instead of being cut off halfway through the review.
Fetch and summarise a page
If the page is too long, the pipeline stops before the API call, so you do not pay for a request that cannot fit.
JWT-based key lookup
The JWT carries the key, so the lookup needs no custom parsing and no extra state.
Page change detector
You find out when a page changes, and you need no scraper, database, or diff tool.
Preview LLM request
You see the exact request, after masking, before any money or secrets leave your machine.
Repo token map
You see which files are too big for your context window before you paste them into a prompt.
Reproducible eval sample
Two eval runs see the same prompts, so a score change comes from your change and not from the sample.
Retry flaky API call
A transient 500 does not stop the run, because coax retries the call with backoff.
Scrub secrets from LLM output
A model can echo a secret back, and mask removes it before it reaches storage.
SSE stream to text
Raw SSE has protocol framing that other tools cannot read, so sse extracts the text field and prints clean text.
Structured secret-free logs
Plain script output becomes searchable JSON logs, and secrets never reach the log file.
Token-checked LLM call
If the prompt is over the budget, the pipeline stops and the model never sees a prompt that only half fits.
Validate LLM output before it propagates
A bad structured response exits 1 before it reaches downstream systems, so you catch schema drift at the source.