Grounded answers
How a document becomes something an agent can answer from — chunking, the context header, query rewriting, and the relevance floor.
"Answers from your documents" is easy to claim and hard to do well. This page describes what the platform actually does between a file upload and a grounded sentence, including the parts that are usually left unsaid.
From file to chunk
Ingest: once per document
The file is parsed to text. Format-specific extraction happens here; everything downstream sees plain text.
Split into chunks sized in tokens — but text is split by characters, so the ratio between them is estimated from the document's own script. A Latin ratio applied to Chinese would produce chunks several times larger than intended.
Each chunk is prefixed with its document title (and optionally a generated note) for embedding only. The stored text is unchanged; the header exists so the vector reflects what the passage is part of.
The chunk plus its header is embedded into a vector.
Stored encrypted and scoped to the space. From here it is retrievable — and only within that space.
Runs when a document is uploaded. The query path below runs on every question.
Query: once per question
The customer asks something — often a follow-up that only makes sense in context: "and what's the rate on that one?"
If the question depends on earlier turns, it is rewritten into a self-contained one for the retrieval vector only. The model still receives the customer's original wording, and any failure falls back to it.
The (possibly rewritten) question is embedded with the same model used for the chunks.
Cosine nearest neighbours, ranked by distance, scoped to that end user's space at the database level rather than by an application filter.
Matches further than the maximum distance are dropped. Returning fewer passages than asked for is the correct outcome — padding to a fixed count is how retrieval systems produce fluent, wrong answers.
Whatever survived is decrypted and goes to the model. If nothing survived, the agent says it doesn't have the answer.
The floor is the step that decides whether the agent answers at all.
An uploaded document is parsed to text, split into chunks, embedded, and stored encrypted. Two details in the splitting matter more than they look.
Chunk size is measured in tokens, but text is split by characters — so the platform has to convert between them, and the conversion is language-dependent. A rough Latin ratio of ~4 characters per token is badly wrong for Chinese, Japanese and Korean, where a single character is closer to a whole token. Using one global constant means CJK documents get chunks several times larger than intended: chunks overflow the context they were sized for, and retrieval returns walls of text.
The splitter therefore estimates the ratio from the content of the document itself, interpolating between the CJK and Latin ratios by how much of the text is CJK. A mixed Chinese/English contract lands between the two rather than at either extreme.
Paragraphs are kept whole where possible. Chunks accumulate paragraph by paragraph up to the target size; only a paragraph that is itself oversized gets hard-split. Consecutive chunks overlap by a configurable amount, carrying the tail of the previous chunk forward so a fact that straddles a boundary is still retrievable from both sides.
Defaults are chunk_target_tokens and chunk_overlap_tokens in the platform configuration. They are
tenant-tunable rather than fixed guarantees — the right size depends on your documents.
The context header
A chunk embedded on its own loses the fact that it came from a particular document. "Clause 7.2 applies where the borrower is in default" retrieves poorly when the query is "what happens if I miss a payment on the bridge product" — the chunk never says which product it belongs to.
Before embedding, each chunk is prefixed with a context header — the document title, and optionally an LLM-generated situational note — so the vector reflects both the passage and what it is part of. The stored text is unchanged; only the embedding input carries the header.
Query rewriting: the part most systems skip
Retrieval quality is usually discussed as a ranking problem. Often it is not — often the query simply contains no information.
A customer asks "and what's the rate on that one?" The pronoun refers to something said two turns ago. Embedding that sentence produces a vector that is close to nothing in particular, and the retrieval for that turn is near-random. No amount of reranking fixes it, because the signal was never there.
So before embedding, a follow-up that depends on prior context is rewritten into a self-contained question. Three deliberate choices:
- It biases toward over-triggering. A regex of referring expressions decides whether a rewrite is attempted. Triggering unnecessarily costs one cheap call and a self-contained question comes back unchanged; failing to trigger costs that turn's retrieval entirely.
- It rewrites into a fuller sentence, not keywords. Embedding models are trained on sentences. Reducing to keywords loses relationships ("the borrower's liability to the lender" and its reverse collapse to the same vector) and negation ("which cases do not require a guarantee").
- It only affects the retrieval vector. The message sent to the model is always the customer's original wording. The rewrite never enters the prompt, the conversation history, or storage — and any failure falls back to the original sentence, so the worst case is exactly the old behaviour.
Search, and the relevance floor
The query vector is compared by cosine distance against chunks scoped to that end user's space. The scoping is not an application-level filter that could be forgotten — it is enforced at the database level.
The top matches are then filtered by a maximum distance. This is the part that separates a useful agent from a confident one:
Reception is staffed Monday to Friday, 9:00 to 17:00.
Appointments are booked at least one working day ahead.
Cancellations inside 24 hours are charged at half the fee.
Founded in 1998 by two partners, now a team of nine.
Illustrative. Sample distances for a made-up clinic. Real values depend on your documents and the embedding model — nothing here is measured, and the floor is tenant-configurable.
Returning fewer than the requested number of chunks is the correct outcome, not a shortfall. If nothing in your knowledge base is relevant, the agent gets nothing and says so, rather than being handed the least-irrelevant passages and inventing a bridge between them. Padding results to a fixed count is how retrieval systems produce fluent, sourced-looking answers that are wrong.
The filter is applied after the ranked fetch rather than in the SQL predicate — the results are already distance-ordered so the outcome is identical, but it keeps a clean index scan instead of degrading into a full-table post-filter.
Decryption happens last
Chunk content is stored encrypted per space. Only the chunks that actually matched are decrypted, with a cipher loaded for that tenant and user — retrieval operates on vectors, and plaintext appears only for the passages that are about to be used.
What this means in practice
- Multilingual document sets do not silently degrade — the chunker adapts to the script.
- Follow-up questions retrieve as well as opening questions.
- The agent's honest "I don't have that" is a designed outcome of the relevance floor.
Retrieval quality depends far more on what you upload than on any parameter here. See Spaces & knowledge for how to organize a knowledge base, and Partners if you would rather have someone do it with you.