Grounding Copilot Studio on your docs: three places vector indexing silently fails
Pointing an agent at a vector index looks like configuration work. It is design work - and the defaults are wrong for documentation. Three failure modes that produce a finished-looking index and an agent that finds nothing.
Retrieval-augmented generation has a demo problem: it looks done long before it works. You stand up an Azure AI Search index, point Copilot Studio at it, ask a question from the documentation, and get a plausible answer. Then a real user asks a real question and the agent shrugs. The index is green. The connector is green. Nothing is wrong - except retrieval.
Grounding an agent on documentation failed for me in three specific places, and each one failed silently.
The wizard chunks by size, not meaning
The portal's "Import and vectorize" wizard is the natural first move, and it is the first mistake. It splits documents at a fixed size - roughly two thousand characters - with no regard for where one concept ends and the next begins. For a measure glossary, that welded a dozen metric definitions into single blobs. A question about one metric retrieved a chunk containing twelve, and the model paraphrased whichever it liked.
Documentation has structure; the chunking has to follow it. A markdown-aware split - one chunk per heading at the right depth - keeps one concept per chunk. The retriever can only be as precise as the boundaries you give it.
A chunk that doesn't know its own name
Splitting on headings introduces the second trap: the heading is not part of the body. Embed only the body text and the chunk describing Average Engagement Score may never contain the words "Average Engagement Score" - so the embedding doesn't match name-based questions, which is most of them.
The fix is mechanical once you see it: embed the heading path - document title, section, subsection - together with the body. Every chunk carries its own name and its place in the hierarchy into vector space.
The vectorizer that wasn't there
The quietest failure: an index can hold perfectly embedded documents and still have no way to embed the question. In Azure AI Search that runtime piece - the vectorizer - is separate configuration, and nothing warns you it's missing. Documents went in through your ingestion pipeline; queries arrive with no pipeline at all. Vector retrieval returns nothing, the agent falls back to whatever else it has, and the whole thing degrades into confident vagueness.
An index that looks finished and a retriever that finds nothing are the same artifact - seen before and after the first real question.
If retrieval seems weirdly useless, check the vectorizer before you re-chunk anything.
Facts in the index, behavior in the prompt
The last lesson is architectural. It is tempting to push everything into the index - definitions, query-writing guides, formatting rules. But retrieval is for facts the agent looks up; instructions are for behavior the agent always has. Procedural knowledge retrieved sometimes is behavior the agent exhibits sometimes. Keep how-to-act in the system prompt and what-is-true in the index, and both get stronger.
The short version
Chunk on meaning, not size. Embed each chunk's name with its body. Confirm the vectorizer exists before judging retrieval quality. And split facts from behavior - index the first, prompt the second. None of these failures announce themselves; all of them are five-minute fixes once named.