Repository Indexing for AI Coding Assistants

Q: Why does my AI coding assistant hallucinate functions that do not exist?

Almost always because the retrieval step did not put the real function in the context window, so the model generated something plausible instead. This is a retrieval failure that presents as a model failure, which is why it is so consistently misdiagnosed. Three mechanisms produce it. First, the function exists but was never retrieved — the embedding of the question did not land near the embedding of that code, often because the user described behaviour while the code uses different vocabulary. The model, lacking the real API, invents a reasonable-sounding one. Second, the function was retrieved but arrived truncated. Fixed-size chunking splits a definition across a boundary, so the model sees a signature with no body or a body with no signature, and fills in the rest. Third, the index is stale: the function was renamed or its signature changed this morning, and the index still describes last night state, so the model confidently uses an API that no longer exists. The diagnostic is straightforward and almost nobody runs it. Log the exact chunks retrieved for the failing question and read them as if you were a new engineer. Could you have written the correct answer from only that context? If not, the problem is retrieval and no model upgrade will fix it. If yes, it is a prompting or model issue and worth investigating as such. Teams without this instrumentation spend real money upgrading models to fix indexing defects, and then conclude that AI coding tools do not work well on their codebase. The underlying cause is usually one of a small set of fixable things: fixed-size chunking, pure vector search with no lexical component, no symbol graph, or a nightly reindex.

Q: Why is fixed-size chunking bad for source code?

Because source code has hard syntactic boundaries that carry meaning, and a fixed-size window ignores all of them. Split a 500-token window across a function and you get one chunk containing a signature and the first few lines, and another containing the remainder of the body with no indication of what function it belongs to, what it returns, or what its parameters are. Retrieve either and the model is working from a fragment. Prose degrades gracefully under this treatment — half a paragraph is still readable and partially useful. Code does not: half a function is frequently worse than nothing, because it looks complete enough to act on while omitting the contract. Four further problems compound it. Class context disappears, so a method body arrives with no indication of which class it belongs to or what state it can access. Imports are stripped, so type names in the chunk are unresolvable and the model guesses at what they refer to. Related code is separated arbitrarily, since a chunk boundary can land between a function and the helper directly beneath it that it calls. And decorators, annotations and doc comments get severed from the definitions they describe, which is where the intent usually lives. The fix is AST-aware chunking: parse with a real grammar and cut on function, method and class boundaries so every chunk is a complete, meaningful unit. Tree-sitter is the standard choice because it parses incrementally, is fast enough to run on save, and has grammars for effectively every language. Then enrich each chunk with the file path, the enclosing class or module, and the resolved imports — so that reading the chunk alone is sufficient to understand the code, which is exactly the standard you would apply to a snippet shown to a new engineer.

Q: Do I need a symbol graph, or is vector search enough?

You need both, and the symbol graph is the component that most reliably distinguishes a genuinely useful assistant from a mediocre one. Vector search answers similarity questions: find code that does something like this, find the module that handles authentication, show me examples of this pattern. That is real value and it is what most setups implement. A symbol graph answers structural questions, and these are the ones that matter most during actual engineering work. Where is this function defined? Every place it is called. What breaks if I change this signature. What implements this interface. What does this class inherit from and what overrides what. Embeddings cannot answer these at any useful quality, because the answer is not a similarity relation — it is an exact set determined by the language semantics, and being approximately right is being wrong. Consider a refactor. The user asks to rename a method and update all callers. Vector search returns chunks that look similar to the method, which will include unrelated methods with similar names and miss call sites whose surrounding code reads nothing like the definition. A symbol graph returns exactly the call sites, because it recorded them at parse time. The difference between a refactor that works and one that silently breaks four call sites is entirely in this component. The tooling exists and does not need inventing. The Language Server Protocol already defines how editors query definitions and references, LSIF defines a portable format for precomputing that data, and Sourcegraph has demonstrated the approach across very large codebases. The reason most assistant setups skip it is that embeddings are dramatically easier to build — chunk, embed, store — while a symbol graph needs per-language parsing and resolution. The effort is real, and it is where the differentiated capability lives.

Q: How often should the index be rebuilt?

Continuously — on file save or on commit — because a stale code index is actively worse than a smaller fresh one. This differs from document retrieval, where yesterday index is usually fine. Code changes constantly and every change potentially invalidates what the index says. If a function was renamed at 09:30 and the index rebuilt at 02:00, then for the rest of the day the assistant will confidently reference a function that no longer exists, and produce code that does not compile. The failure is worse than a miss because it is confident and specific. Nightly rebuilds made sense when reindexing meant reparsing and re-embedding an entire repository, which was slow and expensive. Incremental parsing removed that constraint. Tree-sitter reparses a changed file in the order of tens of milliseconds by reusing the unchanged portion of the syntax tree, so per-file reindexing on save is entirely practical, and only chunks that actually changed need re-embedding. The practical architecture: watch the filesystem or hook into version control events, reparse only changed files, diff the resulting chunks against what is indexed, re-embed only genuine changes, and update the symbol graph edges that the change touched. On a branch switch, reindex the delta between branches rather than the whole tree. Two details matter in practice. Handle deletions properly — removed files must be evicted, or the index continues serving code that no longer exists, which is the same confident-wrong failure. And version the index against a commit SHA so you can detect staleness explicitly and degrade gracefully, rather than silently serving results from an unknown point in history.

Q: Should I index test files, and how should they be treated?

Yes, index them — but linked to the code they exercise rather than as ordinary standalone files, because tests are the single clearest statement of intended behaviour available in most repositories. Their value is specific. A test shows how an API is meant to be called, including the argument shapes and orderings that documentation frequently omits. It enumerates the edge cases someone thought were important. It states expected behaviour explicitly, which is often the only place intent is written down at all. And when a model is generating code that calls an existing function, a test for that function is frequently more useful context than the implementation itself. The failure mode when tests are treated as ordinary files is that they compete with production code in retrieval. A search for a function returns its tests, its mocks, its fixtures and its helpers, crowding out the implementation and the call sites. Worse, the model may pattern-match on test-specific idioms and generate production code full of mocks and assertions. The fix is to link them: record the association between a test and the code under test — derivable from imports, naming conventions, or coverage data — and use that link at assembly time rather than at retrieval time. Retrieve the implementation, then attach its tests deliberately as supporting context. Two related judgments. Fixtures and factories are valuable for the same reason and should be indexed with the same linking treatment, since they show what valid data looks like. And snapshot files, recorded HTTP cassettes and other large generated test artefacts should generally be excluded — they are bulk with almost no signal, and they dilute retrieval exactly as vendored code does.

Q: What should be excluded from a code index?

Anything your team did not write and does not maintain, plus anything generated — which in a typical repository is the large majority of files by volume and almost none of the signal. Exclude by default: dependency directories such as node_modules, vendor, and virtual environments; build output and compiled artefacts; lockfiles; minified or bundled JavaScript; generated API clients and protobuf output; database migrations beyond the most recent, which are historical rather than current truth; large data fixtures; and binary assets. The cost of not excluding is higher than it first appears. Indexing and embedding cost money proportional to volume. Retrieval quality degrades because a query for a utility function returns four plausible matches from three different vendored libraries and one from your own code, and the model has no way to know which is authoritative. And the model may generate code following the conventions of a dependency rather than your codebase, which produces code that works but looks foreign to every reviewer. Three judgment calls come up repeatedly. Documentation is usually worth indexing, since READMEs and architecture notes carry intent that the code does not. Configuration files are worth indexing but must be scanned for secrets first, because embedding a credential into a vector store is a data-leak you will find hard to reverse. And a monorepo containing genuinely first-party shared libraries should index them — the test is authorship and maintenance, not directory location. Start with an aggressive default exclusion list and add back deliberately. The instinct to index everything so nothing is missed produces measurably worse results than a smaller, curated index, for exactly the same reason that a focused knowledge base beats a document dump in any other retrieval system.

Q: How do I evaluate whether my code retrieval is any good?

Build a small evaluation set from real questions your team actually asks, and measure whether the retrieved context is sufficient — separately from whether the final answer was right. That separation is the entire point, because without it you cannot tell a retrieval defect from a model defect and will spend money on the wrong one. The practical method: collect 50 to 100 real questions from your team, ranging from lookups to impact analysis to feature requests. For each, record what a knowledgeable engineer says the answer should be and which files or symbols are genuinely needed to produce it. Then run retrieval and measure two things. Context sufficiency: could a competent engineer answer correctly reading only the retrieved chunks? This is the metric that matters most and it needs human judgment on a sample rather than an automated proxy. And recall against the known-necessary set: what fraction of the files a human said were required were actually retrieved. Track precision too — how much of the retrieved context was irrelevant — because noise consumes budget that relevant context needed, and top-k concatenation often looks fine on recall while wasting most of the window. Break results down by question type, because the failure modes differ sharply. Lookups by exact name test lexical retrieval. Behaviour descriptions test semantic retrieval. Impact questions test the symbol graph, and these are where setups without one fail completely. Cross-module questions test whether chunk metadata carries enough context. Two operational habits make this durable rather than a one-off exercise. Log retrieved chunks for real usage, sampled, so you can inspect failures as they occur in the wild. And re-run the evaluation set when you change chunking, the embedding model, or the reranker — these changes routinely improve one question type while regressing another, and without the set you will not notice.

Q: Is this different from ordinary RAG, or just RAG with extra steps?

It is RAG, but the retrieval strategy differs enough that applying document-RAG defaults produces a system that underperforms badly — so the distinction is worth taking seriously rather than treating as a detail. Five properties of code drive the differences. Code has exact syntactic structure, so chunking should follow the grammar rather than a character count; prose has no equivalent hard boundary. Code has precise cross-references — a call site refers to exactly one definition, and a symbol graph can record that exactly, whereas prose references are fuzzy and embeddings are an appropriate tool. Identifiers demand exact matching: when a user names getUserById they mean that function, not the semantically similar fetchAccountByIdentifier that an embedding model places nearby, which makes lexical search mandatory rather than a nice-to-have. Code changes constantly, so continuous incremental reindexing matters far more than for a document corpus. And code carries strong structural signal in paths and module organisation that has no real analogue in a flat document set and should be used in ranking. What carries over unchanged: hybrid retrieval with reranking is the right default in both worlds, context assembly under a token budget matters in both, chunk metadata improves both, and the evaluation discipline — measure whether retrieval was sufficient before blaming generation — is identical. The honest summary is that code retrieval is document retrieval plus a symbol graph, plus AST-aware chunking, plus mandatory lexical search, plus continuous indexing. Each addition is individually modest and collectively they are the difference between an assistant that feels like it understands your codebase and one that feels like it is guessing. Teams that ship a document-RAG pipeline pointed at a repository and conclude the technology is not ready have usually never implemented any of the four.

Satyam Kumar

Back to Blog

ai-architecture

Your Coding Assistant Is Only As Good As Its Index. Nobody Audits the Index

By Satyam KumarJuly 31, 202610 min read

repository indexing code search ai coding assistants rag for code tree-sitter symbol graph hybrid search ast chunking code context lsif language server protocol developer tools retrieval architecture embeddings software architecture 2026

Your Coding Assistant Is Only As Good As Its Index. Nobody Audits the Index

Frequently Asked Questions

Share this article

Twitter LinkedIn WhatsApp

Satyam Kumar

Founder & AI Architect, AppScale LLP

AI & Cloud Architect. Helping teams build systems that scale to millions.

LinkedIn GitHub

Your Coding Assistant Is Only As Good As Its Index. Nobody Audits the Index

Frequently Asked Questions

Share this article

Comments

Leave a comment