Chunked RAG pipeline
Keeps every chunk within the embedding model's token limit - no silent truncation during indexing.
Split a document into chunks and store each for retrieval.
Pipeline
DOC_ID=$(vrk uuid)
cat doc.txt | vrk chunk --size 512 --overlap 64 | while read -r chunk; do idx=$(echo "$chunk" | jq -r '.index'); text=$(echo "$chunk" | jq -r '.text'); vrk kv set "doc:${DOC_ID}:chunk:${idx}" "$text"; done