COSMOS
An open-source semantic search engine for retrieving information from PDF documents, built from ingestion of text, tables and figures, ElasticSearch-based retrieval with an optional neural reranker, and a question-answering model for extracting answers from the retrieved spans.
COSMOS is built from three parts. Ingestion extracts text, tables, figures and other components commonly found in PDF documents. Retrieval searches across a stack of ingested documents using ElasticSearch, with an optional neural reranker.
Extraction adds a question-answering model that can be deployed to answer queries with subspans pulled from the documents the retrieval step surfaces.
Although it was created to automate scientific discovery and analysis over research papers, its components are described as applicable more generally to any stack of PDF documents. The project was supported by DARPA ASKE and the University of Wisconsin-Madison.