Modern AI Systems Study Guide: RAG, MCP, and AI Agents

Executive Summary & Comparison Matrix

Feature	RAG (Retrieval-Augmented Generation)	MCP (Model Context Protocol)	AI Agent
Primary Role	Knowledge Retrieval & Context Grounding	Universal Connection Standard	Autonomous Problem Solving & Execution
What It Does	Pulls relevant text chunks from a vector database and feeds them to the LLM.	Standardizes how LLMs talk to external tools, databases, and APIs.	Decides what steps to take, which tools to use, and how to achieve a goal.
Analogy	Open-Book Exam: Searching a textbook before writing an answer.	USB-C Standard: Universal adapter connecting any device to any peripheral.	Autonomous Project Manager: Hires tools, delegates tasks, and double-checks work.
Directionality	Read-Only (Information Retrieval)	Two-Way (Read data + Execute actions)	Interactive Loop (Reason $\rightarrow$ Act $\rightarrow$ Observe $\rightarrow$ Repeat)
Key Components	Vector DB, Embeddings, Document Chunker	Host Application, MCP Client, MCP Server, JSON-RPC	Reasoning Engine (LLM), Memory, Planning, Tools

1. RAG (Retrieval-Augmented Generation)

What is RAG?

Retrieval-Augmented Generation (RAG) is an architectural pattern that enhances Large Language Models (LLMs) by retrieving relevant facts from an external knowledge base (like private documents, corporate wikis, or databases) before generating a response.

RAG prevents model hallucinations and bypasses static knowledge cutoffs by grounding answers in retrieved, verifiable data.

[ User Query ] 
      │
      ▼
[ Vector Search ] ──► ( Retrieve Relevant Context Chunks )
      │
      ▼
[ System Prompt ] = Context + Original Query
      │
      ▼
[ Large Language Model ] ──► [ Accurate Grounded Response ]

The 4-Step RAG Pipeline

Ingestion & Embedding: Unstructured documents (PDFs, Markdown) are split into smaller text "chunks" and converted into numerical vectors (embeddings) stored in a Vector Database (e.g., Pinecone, Chroma, Qdrant).
Retrieval: When a user asks a question, the query is converted into an embedding to perform a semantic similarity search (e.g., cosine similarity) across the vector database.
Augmentation: The top-$K$ most relevant document chunks are injected directly into the LLM's context window along with the user's prompt.
Generation: The LLM reads the injected context and synthesizes a clear, grounded answer.

Key Strengths & Limitations

Strengths: Eliminates fine-tuning costs, updates knowledge instantly when documents change, provides source citations, keeps enterprise data private.
Limitations: Read-only (cannot execute actions), limited by the LLM's context window size and vector search quality.

2. MCP (Model Context Protocol)

What is MCP?

Introduced by Anthropic, the Model Context Protocol (MCP) is an open standard designed to solve the $N \times M$ integration problem in AI development.

Historically, connecting $N$ different LLM applications to $M$ different data sources and tools required custom code for every combination. MCP acts as a universal protocol (similar to USB-C or HTTP) that allows any AI application to connect seamlessly to any tool or dataset.

┌─────────────────────────────────────────────────────────────┐
│                       MCP Host App                          │
│  (e.g., Claude Desktop, Cursor, Custom Agent Framework)     │
│                                                             │
│       ┌──────────────────┐       ┌──────────────────┐       │
│       │    MCP Client    │       │    MCP Client    │       │
│       └────────┬─────────┘       └────────┬─────────┘       │
└────────────────┼──────────────────────────┼─────────────────┘
                 │ (JSON-RPC over           │ (JSON-RPC over
                 │  stdio / SSE)    │  stdio / SSE)
                 ▼                          ▼
      ┌────────────────────┐     ┌────────────────────┐
      │     MCP Server     │     │     MCP Server     │
      │  (e.g., GitHub)    │     │  (e.g., Postgres)  │
      └────────────────────┘     └────────────────────┘

Core Architecture & Primitives

MCP relies on a Client-Server architecture communicating via JSON-RPC 2.0 messages:

MCP Host: The main AI application or environment containing the LLM (e.g., Claude Desktop, VS Code).
MCP Client: An intermediary within the host that manages dedicated connections to individual servers.
MCP Server: Lightweight programs exposing specific data or tools via three standardized primitives:
1. Resources: Passive readable data (e.g., log files, database schemas, API readouts).
2. Tools: Executable functions that can perform actions or side-effects (e.g., create_github_issue(), run_query()).
3. Prompts: Pre-configured prompt templates for standard tasks.

Key Strengths & Limitations

Strengths: Reusable integrations, standardizes tool discovery and execution, enables secure two-way read/write capabilities.
Limitations: Requires local/remote host runtime, security sandboxing must be enforced by the client.

3. AI Agents (Agentic AI)

What is an AI Agent?

An AI Agent is an autonomous software system that uses an LLM as its central reasoning engine to evaluate goals, break down complex tasks into intermediate steps, execute tools, observe outputs, and adapt its approach until a target goal is fulfilled.

Unlike a simple chat interaction, an agent operates in an interactive feedback loop (e.g., the ReAct pattern: Reason $\rightarrow$ Act $\rightarrow$ Observe).

                      ┌───────────────────────┐
                      │    User Task / Goal   │
                      └───────────┬───────────┘
                                  │
                                  ▼
                     ┌────────────────────────┐
                     │   1. Brain / Reasoning  │
                     │  (LLM Task Evaluation) │
                     └────────────┬───────────┘
                                  │
        ┌─────────────────────────┼─────────────────────────┐
        ▼                         ▼                         ▼
┌──────────────┐          ┌──────────────┐          ┌──────────────┐
│  2. Memory   │          │ 3. Planning  │          │  4. Tools    │
│(State, Context)│        │(Sub-goals)   │          │ (APIs, Code) │
└───────┬──────┘          └───────┬──────┘          └───────┬──────┘
        │                         │                         │
        └─────────────────────────┼─────────────────────────┘
                                  │
                                  ▼
                     ┌────────────────────────┐
                     │ 5. Observe & Iterate   │
                     │   (Refine Approach)    │
                     └────────────────────────┘

The 4 Structural Pillars of an Agent

Brain / Reasoning Engine: The core LLM evaluating context, making decisions, and planning.
Memory:
- Short-term: Conversation context window.
- Long-term: Vector DBs or persistent stores tracking past decisions and learned facts.
Planning & Reflection: Methods like Chain-of-Thought (CoT) and ReAct that break a major goal into structured sub-tasks and evaluate past mistakes.
Tools & Execution: Real-world interfaces (web browser, code execution sandbox, database write-access, custom APIs).

How RAG, MCP, and Agents Work Together

Rather than competing technologies, RAG, MCP, and AI Agents represent different layers of a modern AI system stack:

 ┌─────────────────────────────────────────────────────────────────┐
 │                            AI AGENT                             │
 │    (Orchestrates overall logic, reasoning, and planning)        │
 └────────────────────────────────┬────────────────────────────────┘
                                  │
                                  │ Calls tools via
                                  ▼
 ┌─────────────────────────────────────────────────────────────────┐
 │                   MODEL CONTEXT PROTOCOL (MCP)                  │
 │    (Standardized communication layer connecting Agent to Tools) │
 └──────────────────────┬──────────────────────────┬───────────────┘
                        │                          │
        ┌───────────────┘                          └───────────────┐
        ▼                                                          ▼
┌─────────────────────────┐                            ┌──────────────────────┐
│       MCP SERVER        │                            │      MCP SERVER      │
│     (RAG Pipeline)      │                            │  (Database / GitHub) │
│  Retrieves relevant DB  │                            │ Executes API calls & │
│      knowledge          │                            │    modifies state    │
└─────────────────────────┘                            └──────────────────────┘

Concrete Example: Automated Software Bug Fixing

Agent receives goal: "Fix issue #102: User login failing on production."
Agent queries an MCP Server hosting a RAG Pipeline to pull relevant architecture documents and error logs.
Agent analyzes the code, plans the fix, and uses an MCP Server for GitHub to inspect the codebase and modify files.
Agent runs unit tests using an MCP Tool and commits the fix once all tests pass.