The misconception, and the distinction that replaces it
Ask most operators whether their business is "ready" for AI and the answer comes back the same way: not yet, the data's a mess, someone needs to clean it up first. That instinct is understandable — it comes from years of software that needed rigid, structured input to work at all, where a missing field or an inconsistent date format could break an import. It is also, for most practical builds, wrong.
A knowledge system built well doesn't need your documents standardized before it can read them. It needs to find the right passage and answer from it, which is a fundamentally different bar than "every record follows the same schema." What actually determines whether a build goes smoothly has almost nothing to do with tidiness and almost everything to do with two properties of the material itself: does it agree with itself, and can anyone actually get to it. Messy but consistent and reachable data is a fine foundation. Pristine-looking but contradictory or locked-away data is not. That single distinction — messiness versus contradiction and inaccessibility — is the filter to run every source through before you assume a cleanup project has to happen first.
What genuinely doesn't matter
A short list of things that feel like blockers and, in practice, rarely are:
Inconsistent formatting. One document uses headers and bullet points, another is a wall of prose, a third is a spreadsheet with merged cells and no clear structure. None of that stops a system from finding and using the right passage. It reads for meaning, not layout.
Mixed file types. PDFs, Word documents, emails, chat exports, spreadsheets, presentation slides, wiki pages — a build can index across all of them in the same project. You do not need to convert everything into one format before anything can start.
Scanned documents, within reason. A scanned contract or a photographed form is not automatically disqualifying. It adds a processing step — covered in detail below — but a folder of scanned PDFs is not, on its own, a reason to delay.
Prose instead of structured fields. A policy written as three paragraphs of plain English is just as usable as a table with rows and columns, sometimes more so, because the system reads for meaning rather than parsing fixed fields. You do not need to convert your handbook into a database before it's useful.
If your hesitation is mainly about how the material looks, that hesitation is usually solving the wrong problem.
What genuinely does matter
The things that actually stop a build are quieter and easier to miss on a first look through your files, because they don't look messy — they look like normal business reality.
Contradiction with no way to resolve it. Two documents state different things and nothing tells you which is current: a wiki page says one return window, a PDF from two years ago says another, and neither is marked as superseding the other. A system built on top of that material will find both passages and either flag the conflict honestly or hand back an answer that's right half the time — and no amount of engineering resolves a contradiction a human hasn't resolved first.
No record of what supersedes what. Even without an outright contradiction, version history matters. If five drafts of a policy exist in a shared drive and nothing marks which one is live, a system indexing all five has no way to know which to trust, and neither would a new hire reading the same folder.
Data locked in a system nobody can export from. Some platforms make it genuinely difficult to get material out in any usable form — no export function, no API, no admin access left inside the business. If nobody can pull the underlying records out, no amount of source-file tidiness helps, because the system can't reach the material at all.
Permissions nobody can untangle. If access rules across your systems are already a mess — former employees with lingering access, folders shared to "everyone" years ago and never revisited, nobody sure who's supposed to see what — that has to get sorted before a retrieval layer can safely inherit it. A system that mirrors broken permissions just automates the mess faster.
Knowledge that exists only in someone's head. If the real answer to a common question lives in one person's memory and was never written down anywhere, no indexing project finds it, because there's nothing to index. That's not a data preparation problem — it's a documentation gap, and it has to be closed by writing the answer down before any system can retrieve it.
How to audit your own sources in an afternoon
You don't need a formal assessment to get a rough read on where you stand. Pick the three to five sources that would matter most to a build — the wiki, the shared drive, the CRM, the support ticket history — and run each one through the same four questions:
- Do any two documents here disagree, and would I know which one to trust? Pull up two similar documents on the same topic and see if they say the same thing. If they don't, that's the contradiction problem, not a formatting one.
- Can I actually export this, or does it only live inside one tool? Check whether there's a real export path or an API, not just a screen you can look at. If the answer is "only by copying and pasting one record at a time," that's the inaccessibility problem.
- Do I know who can currently see this, and does that match who should? A rough sense is enough at this stage — the goal is spotting an obviously broken permission model, not auditing every folder.
- If I asked five people the same question about this material, would I get five different answers? If the honest answer is yes, some of what you need may only exist as tacit knowledge, and that has to be written down before it can be retrieved.
Running this against a handful of sources, rather than trying to audit everything at once, is usually enough to tell you whether you're looking at a normal build or a documentation project in disguise.
Scanned and handwritten documents: what OCR actually gets you
Scanned paperwork deserves its own answer because it behaves differently from every other source type. Optical character recognition — OCR — turns the pixels in a scanned image into searchable text, and modern OCR is genuinely good at typed or printed material: contracts, forms, printed letters, most invoices. Run it against a clean scan of a typed page and you'll usually get text accurate enough to index and retrieve from directly.
Handwriting is a different case. Recognition accuracy on handwritten notes varies enormously depending on how legible the writing is, and inconsistent handwriting produces inconsistent results — some pages transcribe cleanly, others don't. The honest approach is to treat handwritten material as a smaller, separate category: worth extracting where it matters, but worth spot-checking the output rather than assuming it's reliable at the same level as a typed document. Poor scan quality — skewed pages, low resolution, faded ink — degrades OCR accuracy on typed text too, so a quick look at scan quality before committing a large batch to processing saves rework later. Our guide to automating data extraction from PDFs goes into the mechanics of this in more depth, and the same extraction approach applies to a broader set of document types under our AI document review work.
Deciding what's in scope versus what should be archived
Not everything you have needs to go into an index, and treating "we have it" as the same question as "it should be searchable" is a common mistake. Material worth indexing is current, gets referenced or asked about with some regularity, and reflects how things actually work today. Material worth archiving instead — kept, but not fed into the retrieval layer — is superseded, rarely relevant anymore, or interesting only as a historical record.
A retired pricing sheet from three years ago is a clear archive case. A support macro nobody has touched in years but that still gets copied into new tickets by habit is a less obvious one, and worth a real look before you decide. When in doubt, the question to ask is simple: if this document surfaced in an answer today, would that answer be right? If not, it belongs in storage, not in the index.
Who needs to be in the room
Preparation is rarely a task for one person, because no single person usually has full visibility into where the contradictions and the tacit knowledge actually live. The people worth pulling in early are the ones who touch the source systems daily — the support lead who knows which macros are actually current, the ops person who knows which spreadsheet is the real one versus the copy nobody updates, whoever administers access on the platforms in question. A short session with each of them, walking through the audit questions above against their part of the business, surfaces more in an hour than a person working alone would find in a week going through files cold.
It's also worth including whoever owns the decision on what gets archived versus indexed. That call carries some judgment, and it's better made by someone accountable for the material than inferred by whoever happens to be doing the technical work.
What this costs, and why it's scoped separately
When source material has real contradiction, inaccessible systems, or permission problems that need sorting first, that work is its own project — with its own scope and its own price, agreed before it starts, not folded quietly into the build. That's not a way of padding an estimate. It's the only honest way to price it: a build that assumes clean, reachable, permission-sane source material is a genuinely different piece of work from one that has to sort out conflicting policy documents first, and a buyer deserves to see which one they're paying for. Our pricing page explains how we structure that distinction, and if you want a sense of what "ready enough" looks like before you get on a call, the AI readiness checklist is a faster way to check than reading through every file yourself. Retrieval systems built on top of well-scoped source material are sometimes called RAG — our RAG glossary entry explains the mechanism if you want the underlying detail.