Deep Lake
A database for AI built on a storage format optimized for deep learning, storing and querying data, vectors and embeddings for LLM applications and managing datasets for model training.
Deep Lake is a database for AI, built on a storage format optimized for deep-learning applications. It stores and searches data together with vectors and embeddings for building LLM applications, and manages datasets while training deep-learning models.
It stores data of any type, including embeddings, audio, text, video, images, DICOM, PDFs and annotations, with querying and vector search, data streaming for training at scale, and data versioning and lineage. It supports multi-cloud storage across S3, GCP, Azure, Activeloop's own cloud, local or in-memory storage through one API, keeps images, audio and video in their native compression with lazy, NumPy-like indexing, and ships built-in data loaders for PyTorch and TensorFlow. It integrates with LangChain and LlamaIndex as a vector store, with Weights & Biases for lineage during training, and with MMDetection and MMSegmentation for object-detection and segmentation training, and gives access to more than 100 popular public image, video and audio datasets.
It is serverless, works with data of any size, and is written in C++.