Skip to content
Armand Notue

RAG · Information Retrieval · LLM Applications

RAG Document Intelligence API — Grounded answers over your own documents

A document question-answering service — ingestion, chunking, vector retrieval and grounded generation — being built evaluation-first. Early stage.

In progress

At a glance

Status
In progress
Focus
Ingestion, retrieval and generation services in place · Answers generated only from retrieved passages · Evaluation planned before new features
Stack
  • Python
  • FastAPI
  • LangChain
  • ChromaDB
  • sentence-transformers
  • Together AI
Problem
Teams need answers from their own PDFs and pages, with sources — not a model's best guess. That makes retrieval quality the part that has to be measured.
Approach
A FastAPI service that loads documents, chunks and embeds them into ChromaDB, retrieves the closest passages for each question and asks an LLM to answer from them. Reranking and RAGAS evaluation come next.

01

Overview

A retrieval-augmented generation service for question answering over private documents. Documents are split into overlapping chunks, embedded, stored in a vector database, and retrieved at question time so the model answers from the source text.

Early stage. The ingestion, retrieval and LLM services are in the repository; the HTTP routes, Docker setup and evaluation are not built yet, so no quality metrics are reported.

02

Problem & motivation

RAG demos are easy; trustworthy RAG is an evaluation problem. The plan is to measure retrieval and answer quality before adding features like reranking, so every change can be justified with numbers.

03

My role

  • Independent project — design and implementation.

04

System architecture

  1. Stage 1:Ingest

    • PDF loading

      PyPDF document loader

  2. Stage 2:Chunk

    • Text splitting

      Chunks of 500 with 50 overlap

  3. Stage 3:Embed & store

    • all-MiniLM-L6-v2

      Sentence-transformer embeddings

    • ChromaDB

      Vector store

  4. Stage 4:Retrieve

    • Top-5 search

      Similarity with relevance scores

  5. Stage 5:Generate

    • Llama 3 70B

      Via Together AI, answering from retrieved passages

  6. Stage 6:PlannedNot built yet

    • Reranking

      BGE reranker on retrieved passages

    • Evaluation

      RAGAS on a fixed question set

    • API & streaming

      FastAPI routes with SSE responses

Current pipeline, with the planned stage marked.

05

Technical implementation

  1. Configuration first

    Models, chunking and retrieval depth live in one typed settings module, so experiments change configuration rather than code.

  2. Services, not scripts

    Ingestion, retrieval and generation are separate services behind a FastAPI app with a health endpoint, ready for routes to be added on top.

06

Evaluation & results

No metrics yet — by design. Results will be published here once they are measured.

  • Planned: BGE reranking and RAGAS evaluation, compared on the same question set.

07

Technologies used

  • Python
  • FastAPI
  • LangChain
  • ChromaDB
  • sentence-transformers
  • Together AI

Want the deeper walkthrough?

I'm happy to go through the code, the trade-offs and what I'd do next.

Get in touch