# AI doesn't make things up out of malice. It fills the gaps you leave it. Jaro Rais · 6 min read · 2026-10-14 · AI and language models Source: https://itashu.co/en/know-how/why-ai-makes-things-up You know the stories of drivers who trusted their navigation so blindly that they drove down a boat ramp straight into a lake. We laugh at them. **We do the same with AI, it just isn't visible right away.** AI summarizes a contract or proposes a procedure so confidently that we believe it. Most of the time it's right. But sometimes it sends us into the lake. ![A map: one road leads down a ramp into a lake, the other, with a current map, to the destination](https://itashu.co/img/art-11-hero-1280.webp) When AI confidently gives wrong information, it's called a hallucination ([the myth that AI always makes things up](https://itashu.co/en/know-how/five-ai-myths#kapitola-01)). We put it simply: AI makes things up. ## AI doesn't look for the truth. It predicts how the truth would sound. A language model on its own doesn't look anything up. **It predicts what the right answer would look like.** Most of the time prediction and truth match. When they don't, the result sounds just as convincing. It happens for three reasons: - **The model learned to sound certain in training.** Hesitation gets penalized in training, a fluent answer gets rewarded. So it sounds certain even where it shouldn't. - **The model fills in gaps.** When data is missing or unclear, it often doesn't ask and improvises instead. - **A stronger model isn't automatically more truthful.** Sometimes it's wrong all the more convincingly. - **With a person:** A new colleague who is afraid to admit they don't know something would rather answer with a guess. They deliver the guess in the same tone as a fact. - **With AI:** The model behaves the same way. Not because it lies, but because that's how it was trained. The company's job is to create an environment where it doesn't have to guess. ## With agents, making things up is rarer, but more dangerous The good news: **an agent that can check things makes things up less.** When it has access to your data, search or systems, it doesn't have to guess. It can look. The bad news: an agent doesn't just answer anymore. It creates tasks, changes data, sends invitations. When it makes a mistake in one step of the chain, the error doesn't stop in the chat window. **It goes into your systems, and with full confidence.** > **In practice:** “An agent helps the purchasing team check supplier contract deadlines. One contract is missing a date. The agent doesn't flag it, but fills in a likely date. The team then plans around a deadline that never existed. It's not a software bug. The model did exactly what it was built to do: complete the pattern.” ![A missing piece in a wall filled with a made-up part](https://itashu.co/img/art-11-gap-1280.webp) ## Four decisions that limit making things up Making things up isn't fate. It's a consequence of how the system is designed. We use four rules in every deployment. ### 1. Give AI an up-to-date map A navigation system with a five-year-old map doesn't know about the new road or the closed bridge. **AI without access to your data is exactly that kind of navigation.** ([How to give it company knowledge.](https://itashu.co/en/know-how/language-model-frozen-in-time)) We connect it to the places where your valid data lives: the CRM, contract storage, accounting. Then it doesn't estimate, it reads. ### 2. Let it calculate with a tool, not from memory When you ask a colleague for a quarterly cost overview, you want them to open the spreadsheet, not estimate from memory. **AI without tools calculates from memory.** With tools, it checks the figure in the system and does the math with a calculator. ### 3. Mark the boundaries AI must not cross Even a capable person sometimes answers questions outside their field with confidence. So does an agent. That's why we define in writing what it may do, what it may not, where it gets data and which steps someone must approve. **The narrower the boundaries, the less room for confident mistakes.** ### 4. For important decisions, let a person have the final say AI prepares a fast, thorough first draft. For contracts, money and client communication, a person reviews and approves it. That's not a failure of the technology, but good system design. ![Four rules in a row: a map, a calculator, a boundary and a person who approves](https://itashu.co/img/art-11-rules-1280.webp) - **With a person:** Cruise control handles long stretches of motorway. But at the on-ramp, changing lanes or in an unexpected situation, you want a driver at the wheel. - **With AI:** Let AI handle routine steps on its own. Where a mistake is expensive, let it propose and a person decide. > **In practice:** “A law firm uses AI for the first summary of a case file. AI has access only to that file's documents, cites the page for every claim, and when it can't find something, writes ‘not stated in the file’. A lawyer signs off the summary. AI saves time, responsibility stays with a person.” **Ask yourself:** When AI tells you something today, can you check where it got it? And does the system allow it to say “I don't know”? ## It's not about a better model, but a better design The difference between AI that gets you to your destination and AI that sends you into the lake isn't the model. **It's the decisions about how the system is built.** What data it sees, what tools it has, where its boundaries are and who checks the result. That's why, in every solution, we also test how AI behaves when it doesn't know the answer. A system that can say “this isn't in the documents” is more valuable than one that always answers something. --- Tell us where AI answers you too confidently today. We'll show you how to limit it. https://itashu.co/en/consultation