LongParser

LongParser is a document intelligence engine for retrieval-augmented generation pipelines that extracts PDFs, DOCX, PPTX, XLSX and CSV files through Docling and Marker, then chunks them using token-aware, heading-hierarchy-aware and table-aware strategies, plus embedding-based semantic chunking. A heuristic scorer, backed by dictionary and fastText validation, checks chunk quality, and a hybrid regex-and-NER pipeline redacts personally identifiable information while preserving it for human review.

A human-in-the-loop review step lets a person approve, edit or reject blocks and chunks before they are embedded, including a LangGraph-based approve, edit or reject workflow with a MongoDB checkpointer, and an async worker generates hierarchical summary chunks. The system keeps three layers of conversational memory, supports OpenAI, Gemini, Groq and OpenRouter as LLM providers, and stores vectors in Chroma, FAISS or Qdrant. It ships LangChain and LlamaIndex adapters and a production FastAPI server built on Motor, ARQ and Redis with tenant isolation, role-based access control and CORS, and all processing runs locally rather than sending data to a third party.

It installs from PyPI with optional GPU or CPU extras.