---
title: "RAG, CAG and KAG: choosing how AI accesses knowledge"
source: "https://www.matteovincenti.com/en/blog/rag-cag-kag-ai-knowledge"
language: en
author: "Matteo Vincenti"
published: 2026-10-24
category: "AI"
tags: ["RAG", "CAG", "Knowledge Graph"]
---

# RAG, CAG and KAG: choosing how AI accesses knowledge

Compare RAG, CAG and KAG through documents, caching and relationships. Practical decision criteria, update costs and open-source projects.

![An open binder and a green index-card drawer, with cards linked by brass rings to an older manual.](https://cdn.sanity.io/images/xrqkwry7/production/229e86a479e2532611a615771444f30b95bf1485-1536x1024.png?w=1400&fit=max&auto=format)

Category: AI · Tags: RAG, CAG, Knowledge Graph

A team wants an AI assistant to help people use its software documentation. The first questions are about enabling a feature. Then someone asks: “Can we retire API 2 without breaking integrations that are still in use?”

The latest manual may explain the new feature perfectly while saying nothing about an old connector that remains active. An assistant retrieving only that manual has relevant material, but it does not yet have the evidence needed to retire an API.

That is where I would start when choosing between **RAG, CAG and KAG**. Retrieving passages, reusing a corpus in context and querying relationships serve different purposes. The choice depends on what evidence a response needs and how we will keep it current.

## A small inventory and two questions

Consider a software service with data exports and integrations. We have these records:

_A small inventory and two questions_

| Item | Information | Source |
| --- | --- | --- |
| Export 5 | Uses API 3 | Export 5 manual |
| API 3 | Requires Auth 2 for authentication | API 3 specification |
| Legacy connector, still active | Supports only API 2 | Integration inventory |

For “What do we need to enable Export 5?”, we can follow Export 5 → API 3 → Auth 2 and look up the corresponding instructions.

For “Can we switch off API 2?”, the legacy connector is the decisive record. While it remains active, the answer is no. If we do not know how complete the inventory is, the absence of other connectors from the results does not establish that none exist.

Finding and connecting relevant information is one task. Establishing how much we can trust a dependency inventory is another. Putting incomplete records into a graph leaves that problem intact.

## RAG: choosing what to read for each question

**Retrieval-Augmented Generation** retrieves external information and supplies it to the model when it prepares a response. In a document pipeline, texts are split into passages, often called *chunks*, and made searchable. When a question arrives, the system selects useful passages and puts them in the context: the material the model can use for that request. This is the basic mechanism introduced by the [RAG paper](https://arxiv.org/html/2005.11401v4).

Retrieval can look for words and identifiers, compare numerical representations of meaning called *embeddings*, or combine both methods. For an exact identifier such as API 2, lexical search is a sensible baseline. Semantic search can help when the question uses different language from the manual.

In our example, a search about Export 5 should find the correct version of its manual and the API 3 specification. A system can make further searches to complete the chain. The user’s permissions and software version should guide document selection.

I would use RAG as the baseline when there are many documents, they change frequently, or different people can access different subsets. Selection reduces the material sent to the model, but it also becomes a failure point: missing the legacy connector can produce the wrong retirement recommendation.

The separate [RAG retrieval lab](https://www.matteovincenti.com/en/blog/enterprise-rag-wrong-document) examines how the wrong document gets retrieved. Here, an earlier question matters: do we need an explanation from the documents, a complete list, or a dependency check?

## CAG: preparing the corpus before the questions

In the [Cache-Augmented Generation paper](https://arxiv.org/html/2412.15605v2), CAG loads a bounded collection of documents into the model’s context in advance and reuses the resulting computation state. Each question is appended to that prepared context, without document retrieval for every request.

The **KV cache** stores the *key* and *value* vectors computed as tokens are processed. Reusing them avoids repeating part of the work on an existing prefix; new tokens still use the preceding context. Memory use grows with sequence length. This is inference state, separate from the model’s learned weights: caching a manual does not train the model. [Hugging Face’s cache explanation](https://huggingface.co/docs/transformers/main/en/cache_explanation) describes the mechanism.

CAG is worth testing when many questions use the same corpus, it fits within the available context and it stays stable long enough to reuse. Documentation for one software version, used by a group with the same access rights, is one possible case.

Caching does not enlarge the context window or make a long corpus free to process. The paper’s tables show that the advantage varies with the dataset and baseline: on HotpotQA-large, the RAG pipeline that retrieves five passages using lexical BM25 search achieves a higher BERTScore than CAG. BERTScore measures semantic similarity to a reference answer. These comparisons use selected corpora and Llama 3.1 8B. They can inform an experiment, rather than predict the outcome for a company’s knowledge base.

### CAG and prompt caching are different choices

A provider’s *prompt caching* can reuse a repeated prefix within a RAG pipeline too: stable instructions first, retrieved documents afterwards. Reuse conditions depend on the implementation. For example, [Anthropic’s documentation](https://platform.claude.com/docs/en/build-with-claude/prompt-caching) describes prefix matching and cache lifetime.

CAG chooses to preload the corpus. Prefix caching optimizes reuse of part of a prompt. For a cost comparison, I would include RAG with any available caching; otherwise, an optimization shared by both approaches may look like an architectural advantage.

## KAG: making relationships explicit

A **knowledge graph** represents entities and relationships: Export 5 *uses* API 3; API 3 *requires* Auth 2. Relationship labels have defined meanings, and nodes identify the items, including their versions.

**Knowledge Augmented Generation**, in [OpenSPG’s KAG work](https://arxiv.org/html/2409.13731v3), is a specific framework linking this representation to document passages. A *planner* turns a question into operations. A *solver* executes them, combining graph retrieval, text search and operations such as sorting or calculation. Retrieval remains part of KAG.

For Export 5, the path becomes inspectable: find the feature, follow its relationship to API 3, then to Auth 2, and retrieve the passages explaining the requirements. For API 2’s retirement, the search runs in the other direction, looking for items that still depend on it.

This structure can help when questions span many documents and relationships matter more than similar wording. The extra work includes defining a schema, recognizing the same entity across texts and preserving links to sources. The generated plan can also be wrong: the paper discusses errors alongside the cost of intermediate model calls.

Before extracting a new graph, though, I would inspect the existing inventory. If components, versions and dependencies already live in a reliable database, a SQL query or domain API may provide the chain directly. The assistant can then explain the result. Reconstructing a second inventory from the same information adds another system to synchronize.

### GraphRAG and LightRAG: other uses of a graph

[Microsoft GraphRAG’s Global Search](https://microsoft.github.io/graphrag/query/global_search/) primarily addresses questions about a document collection as a whole. It prepares summaries of communities — groups of connected entities — and uses them to build an overall response. That can help explore recurring issues in support reports. It is a different task from listing every component that uses API 2.

[LightRAG](https://arxiv.org/html/2410.05779v3) combines entity and relationship retrieval with vector search. It is another architecture to compare when the questions justify that preparation work. KAG, GraphRAG and LightRAG use different mechanisms; choosing “a graph” still leaves substantial design decisions open.

## Changes reveal another part of the cost

With CAG, how many questions reuse a corpus before it becomes stale? Preparing documentation once and querying it frequently has different economics from rebuilding a cache after every change.

For a cost comparison, I would use the same period and set of requests for every alternative. The period’s total, in euros, includes preparation, updates, request execution and the infrastructure needed to retain the cache. RAG also requires indexing and retrieval; a graph adds extraction and maintenance of derived data. Dividing each total by the same number of requests gives comparable average costs. Separately, record how many questions reuse each valid corpus state before a change, expiry or access revocation.

An access revocation makes the issue tangible. If a document is already in a shared cache, hiding its link in the interface still leaves its content available to the model. Summaries and extracted relationships can retain information from it too. Authorization has to cover the material actually used to answer, as well as files shown to the user. [Microsoft’s security-trimming documentation](https://learn.microsoft.com/en-us/azure/search/search-security-trimming-for-azure-search) illustrates query-time filters; identity and authorization remain application responsibilities.

For an initial comparison, I would use this worksheet:

_Changes reveal another part of the cost_

| Test | What to observe |
| --- | --- |
| Enable Export 5 | Correct versions, the API 3 → Auth 2 path and relevant sources |
| Assess API 2 retirement | The active connector is found; inventory coverage is stated |
| Omit a dependency that remains active | The system distinguishes insufficient data from an absence of dependents |
| Update a requirement | The response uses the new version and lets us identify the version used |
| Revoke access to a document | Excluded content is no longer used, including through caches or summaries |
| Compare costs | Preparation, cold and warm requests, updates and memory are counted over the same period |

For the omission test, I would retain the complete inventory as a reference and give the system a view without the connector, without marking it as retired. The expected answer is “insufficient information to authorize retirement” until a source establishes the inventory’s coverage.

For revocation, I would inspect execution traces as well as the answer: identifiers of selected documents, the corpus or cache version, and the relationships and summaries supplied to the model. This reveals which data remained available without logging their confidential contents.

I would keep the questions fixed and start with the simplest approach suited to the data: an inventory query or document retrieval. CAG or a graph would enter where that baseline encounters a specific limitation. Keeping the initial architecture is a valid outcome of the comparison.

## Open-source projects worth evaluating

This repository comparison is current to **24 October 2026** and considers licenses, releases, documentation and recent changes. For adoption, I would also examine who maintains the integrations and how data updates work. Star counts say little about either.

_Open-source projects worth evaluating_

| Project | What I would evaluate it for | Operational signal to consider |
| --- | --- | --- |
| [Haystack](https://github.com/deepset-ai/haystack), Apache-2.0 | RAG pipelines with explicit, replaceable components | [3.4.1](https://github.com/deepset-ai/haystack/releases/tag/v3.4.1), released 21 October; recent fixes and configured CI. Documentation distinguishes team-maintained and external integrations. |
| [LlamaIndex](https://github.com/run-llama/llama_index), MIT | Connectors, indexing and document management | [0.14.25](https://github.com/run-llama/llama_index/releases/tag/v0.14.25), released 21 September; core and integrations have their own versions. |
| [LightRAG](https://github.com/HKUDS/LightRAG), MIT | A focused test of entity and relationship retrieval, with an API and UI | [1.5.7](https://github.com/HKUDS/LightRAG/releases/tag/v1.5.7), released 2 September; recent fixes. Embedding migration instructions need checking against the selected version. |
| [Microsoft GraphRAG](https://github.com/microsoft/graphrag), MIT | Global search and community reports | The [3.3.0 README](https://github.com/microsoft/graphrag/blob/v3.3.0/README.md) describes the project as largely in maintenance mode, with fixes and dependency updates. |
| [OpenSPG/KAG](https://github.com/OpenSPG/KAG), Apache-2.0 | Studying the planner, solver and structured knowledge | Latest observed release [0.8.0](https://github.com/OpenSPG/KAG/releases/tag/v0.8.0), June 2025; more limited recent activity and an OpenSPG stack to operate. |
| [hhhuang/CAG](https://github.com/hhhuang/CAG), MIT | Studying corpus preloading and KV cache reuse | Research code for the paper’s experiments, with no published releases observed; the inspected material does not document a complete application service. |

Haystack and LlamaIndex would be my first candidates for a document-retrieval baseline. LightRAG enters the comparison if relationships improve the answers we need. The other repositories remain useful for understanding and experimenting with the methods, with different maintenance commitments.

In our example, the legacy connector still needs API 2. That is the constraint the assistant needs to surface, with its source, before the team plans the retirement.