Author: Manibala Sinha Tech Stack: Python, FastAPI, OpenAI API, FAISS / ChromaDB, LangChain, Docker

Retrieval-Augmented Generation (RAG) pipeline built using FastAPI and OpenAI’s API.

It enables intelligent question-answering over custom document corpora, such as PDFs, text manuals, or engineering files — simulating how field operators or engineers can query technical data in real time.

Designed with scalable microservices principles, the system can be containerized with Docker and deployed on Kubernetes / EKS or OpenShift.

          ┌─────────────────────┐
          │  Document Loader    │  ← PDF, text, or well files
          └─────────┬───────────┘
                    │
                    ▼
          ┌─────────────────────┐
          │  Embedding Model    │  ← OpenAI / SentenceTransformer
          └─────────┬───────────┘
                    │
                    ▼
          ┌─────────────────────┐
          │  Vector Database    │  ← FAISS / ChromaDB / Milvus
          └─────────┬───────────┘
                    │
                    ▼
          ┌─────────────────────┐
          │   Retriever Layer   │
          └─────────┬───────────┘
                    │
                    ▼
          ┌─────────────────────┐
          │  OpenAI GPT (LLM)   │  ← Generates final contextual answer
          └─────────────────────┘

Document Ingestion – Upload or load domain documents (PDF, TXT, CSV). Vector Store Indexing – Store embeddings using FAISS or Chroma for fast retrieval. Contextual Q&A – Ask domain-specific questions and get concise, source-aware answers. API Endpoint – Expose a /query endpoint via FastAPI for external integration. Configurable Models – Easily switch between OpenAI, Hugging Face, or local models. Scalable & Deployable – Dockerized for deployment on any cloud (AWS, GCP, Azure).

git clone https://github.com/ManibalaSinha/OpenAI.git
cd OpenAI
git checkout feature_branch

python -m venv venv
source venv/bin/activate   # or venv\Scripts\activate on Windows
pip install -r requirements.txt

Create a .env file:

OPENAI_API_KEY=your_api_key_here
VECTOR_DB=chroma   # or faiss

uvicorn main:app --reload

curl -X POST "http://127.0.0.1:8000/query" \
     -H "Content-Type: application/json" \
     -d '{"question": "What is the procedure for pump maintenance?"}'

• Energy Operations: Ask about well files, safety manuals, or regulatory filings.
• Industrial Applications: Query process documents or equipment SOPs.
• Corporate Knowledge Base: Enable semantic Q&A across internal wikis or handbooks.

Docker Build

docker build -t openai-rag-pipeline .
docker run -p 8000:8000 openai-rag-pipeline

Kubernetes Example

kubectl apply -f k8s/deployment.yaml

• Integrate Milvus / OpenSearch for enterprise-scale retrieval
• Add GPU inference optimization via VLLM or TensorRT
• Incorporate RLHF / Agentic workflows for adaptive reasoning
• Support Azure ML / AWS SageMaker deployments

Manibala Sinha 🔗 LinkedIn | GitHub | Blog