RAG Architecture Patterns

1. Basic/Naive RAG

• Description: Simple retrieval-then-generate approach

• Flow: Query → Retrieve relevant documents → Generate response using retrieved context

• Components: Single vector database, basic embedding model, LLM for generation

• Use Case: Simple Q&A applications, proof of concepts

• Limitations: No query optimization, basic relevance matching

2. Advanced RAG

• Description: Enhanced retrieval with pre/post-processing

• Features: Query rewriting, document re-ranking, result filtering

• Components: Query optimizer, multiple retrieval strategies, re-ranking models

• Improvements: Better relevance, reduced hallucination, contextual understanding

• Use Case: Production applications requiring higher accuracy

https://aws.amazon.com/blogs/machine-learning/access-control-for-vector-stores-using-metadata-filtering-with-knowledge-bases-for-amazon-bedrock/

https://aws.amazon.com/blogs/machine-learning/streamline-rag-applications-with-intelligent-metadata-filtering-using-amazon-bedrock/

Metadata filtering and hybrid search

Use case : So lets say there are some 10000 legal documents some are case specific some are general policies some are topic specific but public some legal documents about companies internal cases ongoing or judgements are case specific and are sensitive . If we have to index these 10k docs in a vector store we need to do some pre processing before we injest these files in vector stores 

i.e  if the case file is named case_001.pdf, the metadata file should be created in same location named case_001.pdf.metadata.json with  
{

  "metadataAttributes": {

    "respondent_id": 669,

    "case_id": 1

  }

} 

we may need to classify group of files and have right metadata e.g for general policies 

{

  "metadataAttributes": {

    "document_type": "policy",

    "sensitivity": "public",

    "access_level": "public",

    "topic": "employment_law"

  }

}

Confidential files would have 

{

  "metadataAttributes": {

    "case_id": "001",

    "document_type": "case_file",

    "sensitivity": "confidential",

    "access_level": "restricted",

    "respondent_id": 669,

    "case_status": "ongoing"

  }

}

Step one : prepare and create the metadata files 

This means we have to run some kind of pipeline to understand and add these files based on our usecase. which could  need services such as Entity Extraction (Claude/Textract/Comprehend) . 

Step two : Injest to vector store 

After we add these files , then we index i.e injest into vector data store ,

Key Point: There's no "update metadata" API for existing indexed documents. You must re-ingest with metadata present in S3 at ingestion time.

Step 3 : At query time we send 

# User with case access

filter = {"equals": {"key": "case_id", "value": "001"}}

# Public documents only

filter = {"equals": {"key": "access_level", "value": "public"}}

Should we use this Metadata Filtering to design security Conrol ? 

One point to take home , is the feature is to optimize fetch at scale ,however this could be very easily used or rather misused as access control mechanism , i would like to explain why this may not be a good idea .

Technically this would work , but Consider scenario , we have a huge data set of files in thousands , we ran a pipeline to understand its content create metadata files for access contrtol .  our data store layer will not give access ( by filtering out ) docments , if we get an edge case where we need to have Temporary access grants to Access  for  a specific scenario we wont be able to do this without huge cost of reindexing just for temporary access !

Relying solely on data-layer filters for primary access control is a security anti-pattern and a key principle of poor system design because it creates a single point of failure and violates the principle of layered security

Apart from that  Relying solely on data-layer filters for primary access control is a security anti-pattern ,it creates a single point of failure and violates the principle of layered security

Scalability and Complexity Issues: Implementing complex, fine-grained access control entirely within the database or data layer often leads to a maintenance nightmare as the application grows. It is difficult to manage user permissions and roles in a scalable way within the database itself.

Lack of Context: Data-layer filters often lack the rich contextual information (e.g., time of day, user's device posture, session risk, or business process state) available at higher application layers, which is necessary for effective, adaptive access control. 

Metadata filters are NOT ideal for primary access control because:

• Role changes require re-indexing - Expensive and slow for large datasets

• Access policies are dynamic - User permissions change frequently

• Metadata is static - Baked into vectors at ingestion time

Core Principle: Vector Stores = Immutable Content Layer

What belongs IN vector store metadata:

Intrinsic document properties (won't change)

document_type: "legal_brief"

jurisdiction: "EU"

topic: "employment_law"

publication_date: "2024-01-15"

case_id: "001" (the case itself doesn't change)

What belongs OUTSIDE vector store:

Transactional/mutable access data

User roles and permissions

Case assignments (who can access case_001)

Recommended Approach

Enforce Access Controls at the Application Level: Implement a dedicated authorization service or layer in your application logic that handles access control checks. This layer should enforce policy based on the user's identity, role, and the requested action.

Principle of Least Privilege: Ensure that users, processes, and systems only have the minimum necessary access to perform their required tasks.

Use metadata for:

• Document categorization (public/internal - broad categories)

• Topic/domain filtering

• Performance optimization (reduce search scope)

Optimal Architecture:

Query Flow:

User Query 

  ↓

[Access Control Layer] ← DynamoDB/RDS (mutable permissions)

   ↓ (determines allowed case_ids, topics, etc.)

[Vector Store Query] ← Static metadata filters

   ↓

[Results Post-Filter] ← Additional security check

  ↓

Return to User

Benefits:

• Cost: No re-indexing for permission changes

• Performance: Vector store does what it's good at (semantic search)

• Flexibility: Update access rules in milliseconds, not hours

• Scalability: Independent scaling of access control vs search

• Audit: Separate permission tracking from content

This brings us to next BIG Topic of finding the right granularity.

Multiple Knowledge Bases: When to Split

Split into separate KBs when:

• Different update frequencies

• Public policies (update quarterly) ≠ Active case files (daily updates)

• Static legal precedents ≠ Ongoing litigation docs

• Different access patterns

  • • Public documents (all users) vs Confidential cases (restricted users)

    • Saves cost - don't search confidential KB for public queries

• Different retrieval characteristics

  • • Legal precedents: need high recall, search entire corpus

    • Case-specific: need precision, smaller focused search

• Operational independence

• HR legal docs managed by HR team

• Corporate litigation managed by legal team

• Independent update cycles, ownership, SLAs

Legal Use Case - Recommended Split:

KB 1: Public Legal Resources

• - General policies

• - Public case law

• - Legal guidelines

• - Update: Monthly

• - Access: All users

KB 2: Internal Policies & Procedures  

• - Company policies

• - Internal guidelines

• - Update: Quarterly

• - Access: All employees

KB 3: Active Case Files

• - Ongoing litigation

• - Sensitive case materials

• - Update: Daily/Weekly

• - Access: Case-specific RBAC

KB 4: Closed Case Archive

• - Historical cases

• - Settled matters

• - Update: Rarely (append-only)

• - Access: Legal team + authorized

Benefits of This Approach:

Cost Optimization:

• Re-index only KB 3 (active cases) frequently

• KB 4 (archive) almost never re-indexed

• KB 1, 2 on slower schedules

Performance:

• Route queries to appropriate KB based on context

• Smaller search space = faster, cheaper queries

• Public queries never hit expensive confidential KBs

Maintenance:

• Independent teams manage their KBs

• Failures isolated to one KB

• Easier troubleshooting

Access Control:

• Simpler RBAC per KB

• IAM policies per knowledge base

• Clear security boundaries

Single KB Approach - When It Makes Sense:

• All documents update at similar frequency

• Similar access patterns across all docs

• Small dataset (<10K docs)

• Unified search experience required

• Metadata filtering is sufficient

Hybrid Pattern (Often Best):

• 3-4 KBs + Smart Routing:

• Most queries hit 1 KB (fast, cheap)

• Complex queries can search multiple KBs in parallel

• Application layer orchestrates multi-KB search when needed

For our hypothetical usecase of  10,000 legal docs:

recommend 3-4 separate KBs based on:

• Update frequency (active vs archived)

• Sensitivity (public vs confidential)

• Access patterns (broad vs case-specific)

This gives you operational flexibility without over-fragmenting. You can always merge later if needed, but splitting an existing KB is harder.

Bottom line: Multiple KBs aligned with your data lifecycle and access patterns = lower cost, better performance, easier maintenance.

Recommended Architecture Patterns on AWS:

Production-Ready (Native Support):

1. Basic RAG: Bedrock Knowledge Bases + Foundation Models

2. Agentic RAG: Bedrock Agents + Lambda functions

3. Conversational RAG: Bedrock + DynamoDB + API Gateway

Advanced (Custom Implementation):

1. Hybrid RAG: Kendra + OpenSearch + custom fusion logic

2. Multi-Modal RAG: Bedrock multimodal models + S3 + Rekognition

3. Self-RAG: Custom orchestration with quality assessment

Academically Established RAG Patterns (Peer-Reviewed):

1. Basic/Naive RAG

• Paper: "Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks" (Lewis et al., 2020, NeurIPS)

• Status: Foundational paper, highly cited (~3000+ citations)

2. Self-RAG

• Paper: "Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection" (Asai et al., 2023, ICLR 2024)

• Status: Peer-reviewed, significant impact

3. Corrective RAG (CRAG)

• Paper: "Corrective Retrieval Augmented Generation" (Yan et al., 2024, ICLR 2024)

• Status: Recently accepted at top-tier venue

4. GraphRAG

• Papers: Multiple works including "Graph-RAG" papers and Microsoft's GraphRAG implementation

• Status: Mixed - concept is established, specific implementations vary

5. Fusion/Hybrid RAG

• Papers: "Precise Zero-Shot Dense Retrieval without Relevance Labels" (Gao et al., 2023) and related fusion techniques

• Status: Rank fusion methods well-established in IR literature

Industry/Practitioner Patterns 

6. Modular RAG - More of an engineering pattern from LangChain, LlamaIndex
7. Agentic RAG - Emerging from agent frameworks, not formalized academically yet
8. Hierarchical RAG - Implementation pattern, limited formal research
9. Adaptive RAG - Engineering optimization, some academic work emerging
10. Multi-Modal RAG - Active research area but fragmented approaches
Others - Mostly industry terminology or implementation patterns

Academic Research Sources:

• ACL, EMNLP, NAACL - Main NLP venues

• ICLR, NeurIPS, ICML - ML conferences with RAG research

• SIGIR, ECIR - Information retrieval conferences

• arXiv - Preprints (not peer-reviewed but influential)