A language model is frozen in time. About your company, it knows only what you give it.
A language model is frozen in time. It knows the world only up to the date its training data was collected, and nothing about what happened five minutes ago. About your contracts, price list and internal procedures, it knows nothing at all. If AI is to answer like someone from your company, you have to give it that knowledge. The question is how.
Two ways: AI reads everything, or looks up only what it needs
The first way is simple. You put whole documents straight into the prompt and the model finds the answer in them itself. Today's models can do it, hundreds of pages fit into one prompt.
The second way is search. The technical name is RAG (what it is) (retrieval augmented generation): documents are split into smaller parts in advance, stored in a special database, and for each question only what relates to it is pulled out. The model gets five relevant paragraphs instead of everything you have.
With a person
A new colleague has two options. Read the whole binder before answering a client. Or call the colleague who knows the files, and she hands over exactly the one page he needs.
With AI
Putting in everything is like reading the whole binder. Search is the colleague who knows the files. Both ways work, but each for a different kind of work.
To compare, AI needs to see whole documents
Search finds what is in the text. It can't find what is missing from the text. If you ask which requirements from the brief didn't make it into the final version, search returns fragments of both documents. But what's missing doesn't show up in fragments.
When the model gets both documents in full, it can compare them. That's why, for a smaller, closed set of documents and questions that need a view of the whole, we recommend the simpler way. No database, and nothing in it that can break.
“A law firm compares a draft contract from the other side with its own template. Search would find similar clauses. Only when the model sees both texts in full can it say which protective clause is missing from the draft.”
When there are many documents or they change, search helps
Putting a whole manual into every question costs something. The model has to process it again every time, and you pay for it every time. Search processes documents once, when storing them in the database, and sends the model only a few paragraphs per question.
The second reason is accuracy. Research shows that a model loses attention in very long text. It overlooks a paragraph hidden in the middle, or fills it in with made-up details from around it. When it gets only five right paragraphs, it focuses on what matters.
The third reason is volume. Years of company data won't fit into any prompt. Without a layer that picks out what matters, it doesn't work.
With a person
A clinic receptionist doesn't look through the whole price list and every policy on each call. She has a card at hand with what people ask most often.
With AI
Search prepares such a card for AI for each question separately. Cheaper, and with less risk of getting lost in the details.
Your documents are the biggest weakness
Company documents were written by different people in different years. A 2019 policy says something different from the 2024 one. If both are in the database, AI answers once by one and once by the other. People then say it makes things up. In reality, it is faithfully quoting your contradictions.
Three rules apply to every knowledge base:
- Only the valid version belongs in the database. Replaced policies and old price lists go out. It doesn't work without document management.
- AI should be able to ask a follow-up question. When a question is unclear, such as “what's the discount?”, it's better to ask which client than to guess.
- AI must not be smarter than your data. If the documents allow three answers, AI should give all three. If it's an opinion, it must not present it as fact.
“An accounting firm uses AI to answer client questions about deadlines and documents. Old guides from before a change in the law stayed in the database. AI answers confidently and incorrectly. The fix wasn't a better model, but one afternoon of cleaning up documents and a rule that every guide has a validity date.”
Ask yourself
Do you know which version of your policy or price list is valid today? And could a new colleague tell without asking?
How we choose the right way
| SITUATION | WHAT WE RECOMMEND |
|---|---|
| A few dozen documents, questions about comparisons and connections | Put documents in whole, no database |
| Hundreds of documents, frequent questions about specific facts | Search (RAG) |
| Data changes every day | Search with continuous updates |
| Documents contradict each other | Clean up first, then build |
The two ways can be combined. Search finds the right documents, which are then put in whole so AI sees the connections.
A knowledge base is more work with content than with technology
When we build a knowledge base, we don't write code in the first days. We look at your documents: what's valid, what contradicts what, what people actually ask. Only then do we choose whether the simple way is enough or search is needed.
The technology can be built in a few days. People's trust in the answers comes from AI working with data that is in order.
In practice, we work in four steps:
Inventory. We map where knowledge lives: shared drives, emails, the CRM and people's heads. We find out what's valid, what's duplicated and what contradicts what.
Owner and validity. Every document gets a person responsible for it and a date until which it's valid. Without that, the base clogs up again within six months.
Test questions. Together with your people, we prepare a set of real questions with the correct answers. We use them to check whether AI answers correctly, and check again after every change.
Feedback. When AI answers wrong, people can report it with one click. Most often it's not a model error, but a missing or outdated document.
The conclusion is simple: data first, then technology. A company that cleans up its documents builds a good knowledge base with almost any technology. A company with chaos in its documents won't build one even with the best model.
