Concepts

Getting your knowledge in, in an afternoon

Most of what an agent needs to know is already written down — on your website, and in the PDFs your team already sends customers. Importing from a URL covers the public half; uploading covers the rest.

Also calledimport websitecrawl websitesite importupload documentsbulk uploadzip uploadseed knowledge base

The most common reason a pilot stalls is not the model. It is that someone has to sit down and "prepare the knowledge base" — and that job never has an owner, so it never starts.

It is also unnecessary. The material almost always exists already. It is on your website, and it is in the handful of PDFs your team emails to customers every week. The work is not writing it. The work is getting it in.

Two ways in, for two kinds of material

Your website — import from a URL. Point the platform at your site and it reads the public pages, turning each into a document. This covers the material you have already written for customers: service descriptions, pricing pages, FAQs, policy explainers, blog posts that answer the questions people actually ask.

Everything else — upload the files. Rate sheets, product manuals, the internal one-pager that explains what your team can and cannot promise. Anything behind a login, anything that only exists as a PDF, anything a page never says out loud.

You do not have to upload these one at a time. Zip the folder and upload that: the importer walks the subdirectories and brings in every .md, .txt, .pdf, .html and .docx it finds, skipping images, .DS_Store and the other debris your compression tool adds. Each file becomes its own document, named by its path inside the archive — so policies/2026/rates.pdf and policies/2025/rates.pdf stay distinguishable instead of both showing up as "rates.pdf".

A file that fails to parse — an encrypted PDF, a corrupt document — is skipped and reported. It does not sink the other thirty-nine.

The split is not arbitrary. Website import is fast and broad but shallow — it can only see what a visitor sees. Uploads are where the material lives that makes your agent different from a search engine pointed at your site.

What the import cannot do

Worth knowing before you plan around it.

Pages that build their content with JavaScript come back empty — the importer fetches HTML, it does not run a browser. Anything behind a login is out of reach. PDFs linked from the site are skipped; they go through the upload path (individually, or zipped in bulk). It respects robots.txt. And it is a one-time import, not a subscription: when your site changes, the knowledge base does not change with it. Run the import again to refresh.

Reading the pages costs nothing. You see the exact number of pages and characters, and an estimated token cost, before anything is added — and nothing is added until you say so.

If your site is a normal server-rendered site — WordPress, Webflow, Squarespace, most CMS-built marketing sites — the import will read it. Single-page apps that render everything client-side are the case that fails, and it fails loudly rather than silently importing blank pages.

A worked example

A mortgage brokerage with a bilingual site and a shared drive full of rate sheets.

Import the site, English only. They point the importer at their home page and set the path prefix to /en/. Without that, the page budget goes to the same eight articles in three languages — the same content, three times the cost, and near-duplicate hits at retrieval time. The import finds eight pages through the sitemap: four long guides, the resources index, the team and about pages, and contact. Around 45,000 characters. They confirm; a minute later the knowledge base has eight documents.

At this point the agent can already answer "do you do physician loans?" and "how does a virtual closing work?" — because those articles were written years ago and have been sitting on the blog.

Upload the four things the site never says. The current rate sheet. The document checklist for self-employed applicants. The one-pager on what loan officers may not promise over chat. A scanned copy of the state disclosure. None of these are on the website; two of them should not be. They were already sitting in one shared folder, so they went up as a single zip rather than four uploads.

Write the boundary, not the facts. In the knowledge base instructions: this is our current policy and takes precedence over general industry practice; rates must always be quoted with their effective date; anything about a specific application must be handed to a human. Facts live in the documents. Instructions say how to treat them.

Total elapsed time: an afternoon, most of it spent deciding which internal documents were safe to include — which is the part that actually deserves a person's judgement.

Where the work really is

Notice what the example did not involve: writing new content, formatting anything, tagging, or building a taxonomy. The retrieval layer does not need those.

What it did involve was two decisions no tool can make for you. Which internal material is safe for customers to reach through an agent. And what the agent is not allowed to promise. Budget your time for those two, not for the import.

Set up your first knowledge base →

Want the mechanics?How retrieval decides what to answer from