Essay 08 · Knowledge graphs & retrieval
Pure vector search is cheap at index time but fails when business logic spans hundreds of interlinked filings. Knowledge Graph RAG costs multiples more upfront in extraction tokens, but pays for itself when single-chunk retrieval failures carry multi-million dollar compliance or investment risks. Here is the architectural decision matrix for budget holders.
The standard vector database stack (chunks, embeddings, cosine similarity) works for isolated document lookups like vacation policy details or contract clauses. However, nearest-neighbor vector similarity measures vocabulary and semantic proximity, not relational truth. When analyzing complex portfolio risk, 500-token chunking severs entity relationships and cross-document guarantees across document boundaries.
Vector search retrieves fragments that look relevant while remaining blind to hidden relational dependencies like cross-default clauses, debt covenants, and executive overlap in footnotes. Two chunks can share key relationships while sharing no vocabulary, causing vector engines to drop critical facts below the threshold.
When you chunk a 100-page document into 500-token blocks, you sever entity relationships across boundaries. If company ownership, supplier dependencies, or revenue commitments span multiple sections or separate filings, those facts get scattered into isolated vectors.
When a query arrives, top-k retrieval pulls the nearest vector neighbors in embedding space. But semantic similarity is not relational logic. Two chunks might share identical language about supply chain risk while referring to completely different entities, years, or business units. The result is a context window filled with surface-level matches that lack the connective tissue required for genuine synthesis.
Knowledge Graph RAG fixes this failure mode by replacing raw text proximity with explicit entity-relationship structures and hierarchical community summaries:
The lab's FundHouse engine makes the failure concrete. It runs an evidence pipeline over 250 Nifty-listed companies, and the hardest class of question is exactly the one vector chunking handles worst: a covenant or restriction sitting dozens of pages from the main credit agreement it modifies. An evidence-grounded architecture that preserves entity relationships and lineage across multi-page filings is what turns those audit checks into verifiable, source-linked verification runs. That is the same property GraphRAG formalizes with explicit entity-relationship structure, and it is the reason graph-style retrieval belongs in regulated document workloads.
A similar pattern runs in Alfred, the governed SDR agent layer I run, with a constitution, budgets in code, and a human veto on every send. Instead of treating news feeds and market notes as isolated text streams, Alfred maps executive changes, institutional holdings, and strategic partnerships directly onto an entity model. When an account signal triggers, the agent walks the graph to determine which ongoing engagements and historical conversations are impacted, producing grounded morning briefs with clear evidence trails.
Before picking your retrieval architecture, evaluate your workload across query complexity, ingestion cost, context efficiency, and risk tolerance:
In practice, the most robust enterprise architectures use a hybrid approach: vector indexes for rapid semantic entry points, and knowledge graphs for structured traversal and global reasoning.
References
Go deeper: FundHouse case study
Work with me
I work with technology leaders and AI architecture teams to design grounded retrieval systems, entity ontologies, and evidence-backed agent layers. Let's review your retrieval stack and build a system that shows its work.