/docs/pipeline/llm/rag.md

RAG

The Retrieval Augmented Generation (RAG) pipeline joins a prompt, context data store and generative model together to extract knowledge.

The data store can be an embeddings database or a similarity instance with associated input text. The generative model can be a prompt-driven large language model (LLM), an extractive question-answering model or a custom pipeline.

Example

The following shows a simple example using this pipeline.

from txtai import Embeddings, RAG

# Input data
data = [
  "US tops 5 million confirmed virus cases",
  "Canada's last fully intact ice shelf has suddenly collapsed, " +
  "forming a Manhattan-sized iceberg",
  "Beijing mobilises invasion craft along coast as Taiwan tensions escalate",
  "The National Park Service warns against sacrificing slower friends " +
  "in a bear attack",
  "Maine man wins $1M from $25 lottery ticket",
  "Make huge profits without work, earn up to $100,000 a day"
]

# Build embeddings index
embeddings = Embeddings(content=True)
embeddings.index(data)

# Create the RAG pipeline
rag = RAG(embeddings, "Qwen/Qwen3-0.6B", template="""
  Answer the following question using the provided context.

  Question:
  {question}

  Context:
  {context}
""")

# Run RAG pipeline
rag("What was won?")

# Prompts with chat templating can be directly passed
# The template format varies by model
rag = RAG(embeddings, "Qwen/Qwen3-0.6B", template="""
  <|im_start|>system
  You are a friendly assistant.<|im_end|>
  <|im_start|>user
  Answer the following question using the provided context.

  Question:
  {question}

  Context:
  {context}
  <|im_start|>assistant
  """
)
rag("What was won?")

# Inputs are automatically converted to chat messages when a
# system prompt is provided
rag = RAG(
  embeddings,
  "openai/gpt-oss-20b",
  system="You are a friendly assistant",
  template="""
  Answer the following question using the provided context.

  Question:
  {question}

  Context:
  {context}
""")
rag("What was won?")

# LLM options can be passed as additional arguments
#  - Streaming RAG response with `stream=True`
#  - String inputs are always converted to user messages with `defaultrole="user"`
#  - Thinking text is removed with `stripthink=True`
rag("What was won?", stream=True, defaultrole="user", stripThink=True)

See the Embeddings and LLM pages for additional configuration options.

Check out this RAG Quickstart Example. Additional examples are listed below.

Notebook	Description
Prompt-driven search with LLMs	Embeddings-guided and Prompt-driven search with Large Language Models (LLMs)
Build RAG pipelines with txtai ▶️	Guide on retrieval augmented generation including how to create citations
Integrate LLM frameworks	Integrate llama.cpp, LiteLLM and custom generation frameworks
Generate knowledge with Semantic Graphs and RAG	Knowledge exploration and discovery with Semantic Graphs and RAG
Advanced RAG with graph path traversal	Graph path traversal to collect complex sets of data for advanced RAG
Advanced RAG with guided generation	Retrieval Augmented and Guided Generation
RAG with llama.cpp and external API services	RAG with additional vector and LLM frameworks
How RAG with txtai works	Create RAG processes, API services and Docker instances
Speech to Speech RAG ▶️	Full cycle speech to speech workflow with RAG
Parsing the stars with txtai	Explore an astronomical knowledge graph of known stars, planets, galaxies
Chunking your data for RAG	Extract, chunk and index content for effective retrieval
Medical RAG Research with txtai	Analyze PubMed article metadata with RAG
GraphRAG with Wikipedia and GPT OSS	Deep graph search powered RAG
RAG is more than Vector Search	Context retrieval via Web, SQL and other sources

Configuration-driven example

Pipelines are run with Python or configuration. Pipelines can be instantiated in configuration using the lower case name of the pipeline. Configuration-driven pipelines are run with workflows or the API.

config.yml

# Allow documents to be indexed
writable: True

# Content is required for extractor pipeline
embeddings:
  content: True

rag:
  path: Qwen/Qwen3-0.6B
  template: |
    Answer the following question using the provided context.

    Question:
    {question}

    Context:
    {context}

workflow:
  search:
    tasks:
      - action: rag

Run with Workflows

Built in tasks make using the extractor pipeline easier.

from txtai import Application

# Create and run pipeline with workflow
app = Application("config.yml")
app.add([
  "US tops 5 million confirmed virus cases",
  "Canada's last fully intact ice shelf has suddenly collapsed, " +
  "forming a Manhattan-sized iceberg",
  "Beijing mobilises invasion craft along coast as Taiwan tensions escalate",
  "The National Park Service warns against sacrificing slower friends " +
  "in a bear attack",
  "Maine man wins $1M from $25 lottery ticket",
  "Make huge profits without work, earn up to $100,000 a day"
])
app.index()

list(app.workflow("search", ["What was won?"]))

Run with API

CONFIG=config.yml uvicorn "txtai.api:app" &

curl \
  -X POST "http://localhost:8000/workflow" \
  -H "Content-Type: application/json" \
  -d '{"name": "search", "elements": ["What was won"]}'

Methods

Python documentation for the pipeline.

::: txtai.pipeline.RAG.init

::: txtai.pipeline.RAG.call