context-engine
A pure-Python library that decides what an LLM actually sees from retrieved documents: retrieval, re-ranking, memory decay and token-budget enforcement in one pipeline.
context-engine handles what comes after a basic RAG retrieve-and-stuff step: a Retriever offering keyword, TF-IDF or hybrid embedding-plus-TF-IDF search, a Re-ranker that blends tag-weighted scores to promote domain-relevant documents, a Memory component with exponential decay, auto-importance scoring and deduplication, a Compressor with truncate, sentence or extractive query-aware modes, and a TokenBudget that enforces a slot-based budget across system, history and document text.
All five components are chained through a single build() call. The core runs on the Python standard library plus numpy with no other required dependencies; installing the optional sentence-transformers package enables hybrid embedding retrieval, and without it hybrid mode falls back to random embeddings with a warning, useful for development and testing. Seven runnable demos cover each component individually and the full pipeline under tight token pressure.