Machine Learning Engineer @ Laboratory for Laser Energetics | Building Fine-Tuned LLMs and RAG Chatbot
❌ Stop Expecting Retrieval to Work Without Cleaning Your Data → Garbage in = hallucinations out. ❌ Stop Ignoring Metadata in Retrieval → A little filtering goes a long way when you're juggling 100s of files. ❌ Stop Acting Like Tables, Images and Equations Don’t Matter → Your model won’t “just get it” if you drop structured data as flat text. It’s time we talk about the most common—and most mishandled—problems RAG pipelines: 🔥 1. Convert PDFs to Markdown (Yes, Really) If you're not doing supervised fine-tuning, Markdown is your best friend. It preserves structure, context, and traceability. Tools I swear by: Marker by DataLab — clean markdown with metadata Docling (via LangChain) — especially solid with tabular data Nougat by Meta — OCR + LaTeX + image-aware, great for scientific PDFs 💡 Pro tip: No GPU? Use Mistral OCR — fast, efficient, and impressively accurate. 🧠 2. Handling Images in PDFs Images ≠ noise. In reports, research, or medical docs, they often carry the context. Two smart options: Convert to image embeddings (when visual layout matters) Or do what I do: run a multimodal model to generate textual descriptions and enrich your chunks with image context ✂️ 3. Stop Using Arbitrary Chunk Sizes If you're still using chunk_size=1000, chunk_overlap=100—you're leaving performance on the table. ✅ Go Semantic + Hierarchical: Break parent docs into paragraphs Group semantically similar paras into mini-chunks Map each mini-chunk back to its parent using something like ParentDocumentRetriever It’s smarter. Cleaner. Way more context-aware. 🧠 4. Smarter Retrieval Starts with Smarter Queries: i) Use chat history to understand and rewrite the query—replace vague prepositions, inject clarity, and give ambiguous terms proper names. ii) Use an LLM to reformulate the query: Generate 4–5 follow-up or sub-questions Use the answers to those to reason better and form a stronger, more accurate final response Let your retriever think, not just fetch. 📌 5. Accurate Referencing Builds Trust Citations aren’t optional—they’re essential. Markdown headers help, but if your PDF is scanned or messy, they often get lost. Here's what I do: Run a 7B model to extract the main topic or section name from each chunk Use this as the source label during generation Clean, readable, and traceable. Exactly what you want in a production-grade chatbot. ⚡ RAG is not about gluing together a retriever and a generator. It's about: ✅ Understanding your data ✅ Structuring it semantically ✅ Retrieving wisely ✅ Citing clearly If you're doing that—now you're building RAG right. What’s the biggest challenge you’ve hit while working on a RAG system? Let’s trade notes ↓