Docs/RAG

Local RAG Implementation & Architecture

Ground open-source models with your private files. FineTuneMyAI runs on-device vector indexing and semantic retrieval without sending document tokens to external vector SaaS providers.

Local Index Storage

Vector indexes are compiled into self-contained JSON data structures stored directly in ./data/rag/indexes/. Each chunk records:

  • `id`: Unique chunk hash (e.g. `chunk_c77a11_0`)
  • `text`: Raw snippet text extracted from the source document
  • `vector`: 384-dimensional normalized float embedding vector
  • `metadata`: Source filename, character offsets, and document title

Dual-Stage Retrieval Pipeline

Vector distance alone is prone to false positives on short technical entities. FineTuneMyAI separates retrieval into two distinct phases:

Phase 1: Dense Cosine Scan

Scans all indexed chunk vectors against the query embedding using dot-product cosine similarity. Collects the top candidate set (default: candidate_k = 20).

Phase 2: Lexical Re-Ranker

Evaluates candidates using exact token density, title keyword matches, and term frequency. Selects the final highest-precision passages (default: rerank_k = 3).

Programmatic Querying via REST API

Query your local index directly via HTTP:

$ curl -X POST http://localhost:3050/api/v1/rag/query \
-H "Content-Type: application/json" \
-d '{"query": "What is QLoRA?", "candidateK": 10, "rerankK": 3}'
[+] Retrieved 3 chunks in 14.8ms with zero cloud transmission.
← Fine-Tuning HyperparametersOpen Local RAG Studio →