Grounded answers
How a document becomes something an agent can answer from — chunking, the context header, query rewriting, and the relevance floor.
"Answers from your documents" is easy to claim and hard to do well. This page describes what the platform actually does between a file upload and a grounded sentence, including the parts that are usually left unsaid.
From file to chunk
Ingest: once per document
The file is parsed to text — and, for documents that have figures, the figures are read too (see below). Format-specific extraction happens here; everything downstream sees plain text.
Split into chunks sized in tokens — but text is split by characters, so the ratio between them is estimated from the document's own script. A Latin ratio applied to Chinese would produce chunks several times larger than intended.
Each chunk is prefixed with its document title (and optionally a generated note) for embedding only. The stored text is unchanged; the header exists so the vector reflects what the passage is part of.
The chunk plus its header is embedded into a vector.
Stored encrypted and scoped to the space. From here it is retrievable — and only within that space.
Runs when a document is uploaded. The query path below runs on every question.
Query: once per question
The customer asks something — often a follow-up that only makes sense in context: "and what's the rate on that one?"
If the question depends on earlier turns, it is rewritten into a self-contained one for the retrieval vector only. The model still receives the customer's original wording, and any failure falls back to it.
The (possibly rewritten) question is embedded with the same model used for the chunks.
Cosine nearest neighbours, ranked by distance, scoped to that end user's space at the database level rather than by an application filter.
Matches further than the maximum distance are dropped. Returning fewer passages than asked for is the correct outcome — padding to a fixed count is how retrieval systems produce fluent, wrong answers.
Whatever survived is decrypted and goes to the model. If nothing survived, the agent says it doesn't have the answer.
The floor is the step that decides whether the agent answers at all.
An uploaded document is parsed to text, split into chunks, embedded, and stored encrypted. Two details in the splitting matter more than they look.
Chunk size is measured in tokens, but text is split by characters — so the platform has to convert between them, and the conversion is language-dependent. A rough Latin ratio of ~4 characters per token is badly wrong for Chinese, Japanese and Korean, where a single character is closer to a whole token. Using one global constant means CJK documents get chunks several times larger than intended: chunks overflow the context they were sized for, and retrieval returns walls of text.
The splitter therefore estimates the ratio from the content of the document itself, interpolating between the CJK and Latin ratios by how much of the text is CJK. A mixed Chinese/English contract lands between the two rather than at either extreme.
Paragraphs are kept whole where possible. Chunks accumulate paragraph by paragraph up to the target size; only a paragraph that is itself oversized gets hard-split. Consecutive chunks overlap by a configurable amount, carrying the tail of the previous chunk forward so a fact that straddles a boundary is still retrievable from both sides.
Defaults are chunk_target_tokens and chunk_overlap_tokens in the platform configuration. They are
tenant-tunable rather than fixed guarantees — the right size depends on your documents.
Figures, not just the text layer
Most document extraction stops at the text layer. Upload a datasheet, an installation manual, or a report, and a typical pipeline pulls the prose and silently drops everything that was drawn rather than typed — the wiring diagram, the board layout, the chart in the middle of the page. The file was "processed"; half of what it says was never read. Ask "which header is the CPU fan on" and the answer isn't in the index, because it was only ever in a picture.
This platform reads those too.
- Scanned or image-only PDFs — no text layer at all — are rendered page by page and transcribed by a vision model, so a photographed contract or a scanned form becomes searchable text.
- Mixed documents — a text layer plus figures — are the case others miss. Pages that carry a real figure are rendered and described by the vision model, and the description is folded into the document's text so it chunks, embeds, and becomes retrievable alongside the prose. A board-layout page comes back as the connector names and where they sit; a chart comes back as the trend and the numbers.
- Embedded images in Word files are pulled from the document and described the same way.
The hard part is not calling a vision model — it is not calling it forty times on the same decorative sidebar. A manual repeats a logo and a page border on every page; describing those would be pure cost and pure noise. So figure detection distinguishes a genuine diagram (many vector paths, a large photo, a montage of images) from template decoration (the same image, by content, recurring across pages) and only sends the pages that carry information.
Describing figures uses the vision model, so it costs more than text-only extraction. Two guardrails:
a figure-heavy document asks for confirmation before it runs up the bill, and image-description spend
shows as its own line — vision_image — in usage, so you always see where it went.
The context header
A chunk embedded on its own loses the fact that it came from a particular document. "Clause 7.2 applies where the borrower is in default" retrieves poorly when the query is "what happens if I miss a payment on the bridge product" — the chunk never says which product it belongs to.
Before embedding, each chunk is prefixed with a context header — the document title, and optionally an LLM-generated situational note — so the vector reflects both the passage and what it is part of. The stored text is unchanged; only the embedding input carries the header.
Query rewriting: the part most systems skip
Retrieval quality is usually discussed as a ranking problem. Often it is not — often the query simply contains no information.
A customer asks "and what's the rate on that one?" The pronoun refers to something said two turns ago. Embedding that sentence produces a vector that is close to nothing in particular, and the retrieval for that turn is near-random. No amount of reranking fixes it, because the signal was never there.
So before embedding, a follow-up that depends on prior context is rewritten into a self-contained question. Three deliberate choices:
- It biases toward over-triggering. A regex of referring expressions decides whether a rewrite is attempted. Triggering unnecessarily costs one cheap call and a self-contained question comes back unchanged; failing to trigger costs that turn's retrieval entirely.
- It rewrites into a fuller sentence, not keywords. Embedding models are trained on sentences. Reducing to keywords loses relationships ("the borrower's liability to the lender" and its reverse collapse to the same vector) and negation ("which cases do not require a guarantee").
- It only affects the retrieval vector. The message sent to the model is always the customer's original wording. The rewrite never enters the prompt, the conversation history, or storage — and any failure falls back to the original sentence, so the worst case is exactly the old behaviour.
Search, and the relevance floor
The query vector is compared by cosine distance against chunks scoped to that end user's space. The scoping is not an application-level filter that could be forgotten — it is enforced at the database level.
The top matches are then filtered by a maximum distance. This is the part that separates a useful agent from a confident one:
Reception is staffed Monday to Friday, 9:00 to 17:00.
Appointments are booked at least one working day ahead.
Cancellations inside 24 hours are charged at half the fee.
Founded in 1998 by two partners, now a team of nine.
Illustrative. Sample distances for a made-up clinic. Real values depend on your documents and the embedding model — nothing here is measured, and the floor is tenant-configurable.
Returning fewer than the requested number of chunks is the correct outcome, not a shortfall. If nothing in your knowledge base is relevant, the agent gets nothing and says so, rather than being handed the least-irrelevant passages and inventing a bridge between them. Padding results to a fixed count is how retrieval systems produce fluent, sourced-looking answers that are wrong.
The filter is applied after the ranked fetch rather than in the SQL predicate — the results are already distance-ordered so the outcome is identical, but it keeps a clean index scan instead of degrading into a full-table post-filter.
When retrieval succeeds and still fails
The relevance floor catches the case where nothing is close enough. There is a harder case it cannot see: every passage clears the floor, the topic is right, and the one fact the question was about is not in any of them.
Measured on this site's own pre-sales agent. Asked "if I'm on Starter and my usage grows 30% a month, what will I pay in six months", retrieval returned three passages: the Starter plan page, a guide to token usage, and the terms of service. All relevant. None of them states a price — the plan page is a page about who the plan is for. Given a document titled "Starter" with no number in it, the model supplied one. Over five runs it produced $5, $250, $99, $0 and $29, each with a citation marker attached. The real figure is $49.
Nothing failed in a way anything could detect. The distances were fine, so there was no retrieval miss to report. The citation pointed at a passage that really was retrieved. Only the answer was wrong.
A second pass, decided in code
So after the first search, two questions are asked of what came back — both answerable with string comparisons, neither delegated to a model:
Is the thing being asked about even mentioned? Proper nouns in the question, plus terms drawn from the titles of the documents in your knowledge base — a vocabulary someone curated rather than one we guessed. If a term appears in the question and in none of the retrieved passages, that term gets searched for on its own.
Is the side of it being asked about there? This is the half the first check misses, and the half
the measurement above landed on: Starter was in the retrieved text; the price was not. So each
question is also matched against what it asks for. Some of those have hard formats — prices carry a
currency mark, durations carry a time unit, contact details look like an email or a URL — and material
that genuinely covers that side almost never omits the format. Others (refunds, compliance, trials,
integrations) have no format, so the signal is the topic word itself, written out per language rather
than left to vector similarity to bridge.
When either check comes up short, the derived terms are searched once more and the results merged in. Once, never a loop. A false alarm costs one vector query — no extra model call, no extra round trip — while a miss costs an invented figure with a citation attached to it.
- Starter — who it's forA small, defined book of business — a few dozen customers.no price
- Token usage — what countsYour dashboard shows usage as two colours, prompt and completion.no price
- Terms of serviceYou may purchase additional tokens in advance at the published rate.no price
Two things worth being plain about. This is a floor under fabrication, not a proof of correctness — it notices that a side of the question is unrepresented, not that an answer is right. And it does not remove the need for the knowledge base to actually contain the fact: the fix that made the example above work in production was also giving each plan a short document that states its price, because a long question about growth and overage will out-compete a single combined pricing table every time.
When the question names an article number
Semantic similarity is the wrong instrument for a citation. "Article 18", "Part 9", "ISO 9408" — these do not mean anything; they identify something. Two rows of a standards table that differ by one digit are, to an embedding, the same sentence.
So identifiers in the question are extracted and searched for as text, and a passage that carries the exact one asked for is pinned ahead of the semantic ranking and cannot be moved by it. That last clause is the whole point, and it is worth saying plainly because it was once not true here: the protection was described in the code and never implemented, so a reranker sorted the pinned passage back down the list. Measured on a standards table where the neighbouring rows differ by a single digit, the one row that named the part asked about ranked first with reranking off and outside the top 20 with it on — and the answer came from the neighbouring row.
Two consequences follow from treating an identifier as identity rather than meaning:
When the identifier appears in more places than the context can hold, the citation is not discarded. It used to be: if "Article 18" matched more chunks than would fit, the whole group was dropped on the reasoning that a sample is not an answer. That is true of taking the first few in file order. It is not true of taking the ones nearest the question. Sorted that way in one measured case, the two passages that answered it sat at cosine 0.27 and 0.31 and the noise began at 0.51 — so the gap itself says where to stop, and no threshold has to be guessed. When the distances form a smooth slope instead of a step, nothing is rescued: an even spread means the passages are not distinguishable, and that really is a sample.
The nearest passage by vector distance is always carried into the answer's context. Reranking is allowed to reorder; it is not allowed to discard the single closest match. In a measured run the reranker pushed that passage out of the context in one retrieval in seven, and in a hand-checked sample of those, the discarded passage was sometimes the only one that answered the question at all. It is added rather than substituted — nothing else is displaced to make room.
When the model asks for more material
Sometimes the passages in front of the model do not settle the question, and it searches the knowledge base itself rather than answering. That action is the signal. It has read the material and reached past it, so what it needs is the next batch — not the same one again, which is what a naive implementation returns and which tells it nothing.
The next batch is already in hand. Retrieval fetches a wider candidate pool than it shows, ranks it once, and shows the top of it; the rest is held for exactly this moment. Serving it costs no second search, no second ranking, and no second relevance pass — only the one model call that was already happening. A first implementation did re-run the search, and on one question that turned 22 seconds into 71, which is how the cost of a "second look" gets measured rather than assumed.
It is served once per turn. After that a repeat search behaves normally again, because "you would get the same passages back" is a signal too, and removing it makes a model dig indefinitely.
Decryption happens last
Chunk content is stored encrypted per space. Only the chunks that actually matched are decrypted, with a cipher loaded for that tenant and user — retrieval operates on vectors, and plaintext appears only for the passages that are about to be used.
A number is not a meaning
Everything above is machinery. This section is the part you control, and it moves results more than any setting on this page.
Retrieval matches meaning. A value on its own carries very little of it, so a fact can sit in your text and still be unreachable — silently, because search keeps returning something.
This line appeared in 1,480 of 1,500 chunks in a real customer catalogue:
Reading level: 1 · Age: 5 · Words: 1951And "which books suit a child just starting to read on their own" came back with a book about musical
prodigies. 1 bears no resemblance to beginner reader the way one word resembles another.
Adding one sentence next to the value fixed it, measured on that record:
Reading level: 1 · Age: 5
Suitable for around age 5, for children just starting to read on their own.| query | before | after |
|---|---|---|
| which books suit a child just starting to read on their own | 0.625 | 0.540 |
| 什么书适合刚开始自己读的孩子 | 0.626 | 0.552 |
| beginner reader age 5 | 0.598 | 0.498 |
Lower is closer, and — see below — two unrelated passages sit around 0.61 on this model. So the last row moves from "barely better than chance" to a real match, and the gain holds across languages.
The general rule: if a question would be answered by a property rather than by the prose — price, date, level, availability, stock — put it in words as well as in a field. Structured values are for filtering; retrieval finds what is written.
One thing that is not worth the effort, because we measured it: exports often repeat themselves (a
summary and a description carrying identical prose). Removing the duplication changed retrieval
distance by +0.002 across 60 records — nothing. An embedding is a direction, not a budget that
repeated text uses up.
Distance is not a score, and it is not comparable across models
One more thing worth knowing before you read any distance figure. Embedding models compress text into
a narrow cone, and how narrow differs per model. On the model this platform runs today, two
completely unrelated passages sit about 0.61 apart — so a hit at 0.52 is a strong match, not a
weak one, and 0.5 is not a meaningful cut-off anywhere.
Two practical consequences:
- Judge a distance against the unrelated baseline, never against a number you remember from somewhere else. The knowledge starmap in the console marks that baseline on its scale for exactly this reason.
- Longer questions retrieve better than keywords. A single word against a 900-character passage is a weak signal even when the passage is literally about that word; a full question is a much closer match. Real users write sentences, which is the good case — but if you test with one-word queries, expect the numbers to look worse than the system actually performs.
What retrieval cannot do, and what answers those questions instead
Everything above is about finding the passage that answers a question. Three kinds of question are not about a passage at all, and no amount of tuning makes similarity search answer them:
- Counting — "how many do you have", "how many are non-fiction"
- Numeric filtering — "which ones are under 200 words", "between 1000 and 2000"
- Grouping — "how many per category", "which author appears most"
These fail in a way that is easy to miss. Asked "are there any around 500 words", an agent with only vector search will report on whichever record happened to be in its top few — a true statement about four documents, presented as a statement about the collection.
The structured index
A knowledge base can therefore carry a second, small table alongside its vectors: one row per document, built from the fields the documents already contain. Values are copied character for character — nothing is generated, nothing is rewritten.
With it, the agent picks the tool that fits the question:
| Question | Answered by |
|---|---|
| "anything about dinosaurs?" | vector search |
| "how many do you have?" | the table |
| "about dinosaurs, suitable for a five-year-old" | the table filters, vector search ranks |
| "what is the link?" | the table, quoted exactly |
The fields are proposed by reading a few of your documents and then verified against the whole collection: a field found in only a fifth of them is dropped and reported, because filtering on it would silently omit the rest. Building or changing the index does not re-embed anything — the values are read out of text that is already stored — so it does not disturb retrieval that is already working.
Documents that share no machine-readable header get no index, and the platform says so rather than building a table whose values are all different. That is not a limitation to work around: a table like that cannot be counted or grouped meaningfully, and having it would only invite questions it answers badly.
Why a link in an answer used to belong to the wrong item
There is a second, quieter failure that the same table fixes.
Retrieval returns chunks, but a title, a product code, a link and an image live in the document header — that is, in chunk 1, while the passage that answered the question may be chunk 7. So the model receives four or five fragments from three or four documents and has to work out which link belongs to which title.
When it gets that wrong, nothing looks wrong. The link is real, it opens, the image renders — it simply belongs to a different item. On a 4,500-document catalogue, 11 of 20 test answers paired a title with another document's link.
Fields you mark as exact — the link, the image, the identifier — are now never shown to the model at all. It receives a placeholder, writes the placeholder, and the platform substitutes the stored value on the way out. A model cannot mistype an address it never saw. The same 20 questions came back 20 of 20 correct.
Which link is the link to send a customer is a business fact your data does not state, so you name those fields yourself; everything else is proposed for you. More in Structured index.
When your material genuinely has no answer
Sometimes the honest answer is "that isn't in your material", and sometimes that is the end of a useful conversation. Public-source fallback is an opt-in, per-agent alternative: when retrieval comes up empty, the agent may search the web, read one of the pages it finds, and answer from that.
It is off unless you turn it on, and three things are true whenever it is on.
Reading the page is part of it, not an extra. A search result is a title and a two-line summary, and the figures that matter — a validity period, a fee, a deadline — are usually only in the body. One page in our evaluation set states "valid for five years" in its snippet and "renewal before: 9 months" only in the page itself. Given the summary alone a model has the hint without the fact, and fills the gap from memory. So one switch enables both searching and reading; enabling only search is measurably the worst configuration, not a more conservative one.
Searching without reading is not a permitted outcome. If the agent searches and then answers without opening a page, the platform makes it go back and read one before the answer is delivered. The difference is not academic: asked who currently runs Japan's PMDA, the summaries alone produced a confident wrong name; the same question with the page read produced the right one.
Anything from a public page is labelled, and you can see what it searched. The citation marker is a different colour, the panel shows the domain and states plainly that this is a public source rather than your material, and the distinction is applied by the platform — not asked of the model, which would forget on exactly the occasion it mattered.
Pages the agent read and results it only found are both listed, drawn differently (solid versus dashed) and labelled differently, because they are different grades of evidence. An earlier version showed only the pages it read, and the effect was the opposite of the intent: on a turn where the fetch failed, the answer came from summaries and carried no source markers at all — visually identical to an answer from your own material. Hiding the weaker evidence hid the fact that the evidence was weak.
Every such answer lands in a review queue, under Knowledge review in the console. Question, answer, source URL, a snapshot of the page as it read it, and any figure the outbound fact check could not trace to a source.
What gets written when you approve is a fresh entry, rewritten from the question and the material — not the answer the visitor received. The answer was addressed to one person and shaped by that conversation: it opens with their name, refers to "your product lines", and ends with a call to action. Filed verbatim, one visitor's private context becomes everyone's answer. So the entry is regenerated without any of it, with the title written from the material rather than from the question — a question phrased "…about my product" is a title that quietly carries someone's context into a place where titles weigh heavily in retrieval. Email addresses and phone numbers are masked before anything is stored. You see the rewritten entry, edit it if you want, and approve that.
Rejecting keeps the record, so the same page is not re-reviewed next month.
The number we would rather you knew
Measured on 24 questions the knowledge base genuinely could not answer, run three times: answers traceable to a real source went from 8% to 33%, and figures we could not trace to any source fell from 13 to 6. That second number is not zero. About one answer in twelve still carries a figure we cannot account for, which is why the fact check ships with the feature rather than after it, and why the review queue exists at all.
What this means in practice
- Multilingual document sets do not silently degrade — the chunker adapts to the script.
- A diagram or chart inside a document is answerable, not silently dropped with the rest of the picture.
- Follow-up questions retrieve as well as opening questions.
- The agent's honest "I don't have that" is a designed outcome of the relevance floor.
- A question that names an article or standard number gets the passage carrying that number, not the one that reads most like the question.
- What the answer then claims about the material is checked against the material — see Checks on the answer.
Retrieval quality depends far more on what you upload than on any parameter here. See Spaces & knowledge for how to organize a knowledge base, and Partners if you would rather have someone do it with you.