Local RAG App

Problem

Answer questions across PDFs, text files, and Markdown without sending document contents to a hosted AI service. That requirement shaped the whole system: document processing, embeddings, retrieval, and language-model inference all run locally.

My contribution

I built the system end to end: file ingestion and chunking, the ChromaDB retrieval layer, the FastAPI service, the Ollama integration, and Docker Compose packaging. I also evaluated answers across a mix of supported document formats.

Technical decisions

ChromaDB stores locally generated embeddings and retrieves relevant document chunks for each question. Ollama runs both the embedding model and the language model. Answers include source citations so a reader can inspect the supporting passages.

The three-service Docker Compose setup packages the application and its dependencies behind one startup command. Keeping inference local makes the machine's hardware and model choice part of the performance constraint.

Results

In a small local evaluation of 20+ questions across PDF, text, and Markdown files, about 80% of answers were correct. Observed responses took about half a second in that test environment. These are initial, environment-specific measurements, not a general accuracy or latency guarantee.