Awesome Vector Search Engine
A curated list of awesome vector search framework/engine, library, cloud service and research papers to vector similarity search
Standalone Service
- Qdrant - Vector Similarity Search Engine with extended filtering support
- Vald - A Highly Scalable Distributed Vector Search Engine
- Milvus - An open source embedding vector similarity search engine powered by Faiss, NMSLIB and Annoy
- Weaviate - A cloud-native, real-time vector search engine
- OpenDistro Elasticsearch KNN - A machine learning plugin which supports an approximate k-NN search algorithm for Open Distro for Elasticsearch
- Elastiknn - Elasticsearch plugin for nearest neighbor search
- Vearch - A scalable distributed system for efficient similarity search of deep learning vectors
- pgANN - Fast Approximate Nearest Neighbor (ANN) searches with a PostgreSQL database
- Jina - Jina allows you to build deep learning-powered search-as-a-service.
- Aquila DB - Distribution focused k-NN search algorithm
- Redis HNSW - A redis module for similarity search based on HNSW
- Marqo - A semantic search engine which supports tensor search (sequence of vectors)
- txtai - Build semantic search applications and workflows
Library
- Faiss - A library for efficient similarity search and clustering of dense vectors
- Distributed Faiss - Work with FAISS indexes which don't fit into a single server memory
- Autofaiss - Automatically create Faiss knn indices
- ScaNN - A library efficient vector similarity search at scale.
- NMSLIB - Non-Metric Space Library, an efficient similarity search library for generic non-metric spaces
- Annoy - C++ library with Python bindings to search for points
- FLANN - Library written in C++ and contains bindings for the following languages: C, MATLAB, Python, and Ruby
- MRPT - Fast nearest neighbor search with random projection
- RPForest - Python library for approximate nearest neighbours search
- pgvector - Open-source vector similarity search extension for Postgres
- PASE - Ultra-High-Dimensional approximate nearest neighbor search extension for Postgres
- Pyserini - Toolkit for reproducible information retrieval research with sparse and dense representations
- NGT - Provides commands and a library for performing high-speed approximate nearest neighbor
- NearPy - Approximate search using different locality-sensitive hashing methods
- TOROS N2 - lightweight approximate Nearest Neighbor library
- PUFFINN - Parameterless and Universal Fast FInding of Nearest Neighbors
- SPTAG - A distributed approximate nearest neighborhood search (ANN) library
- PyNNDescent - A python nearest neighbor descent for approximate k nearest neighbors
- TarsosLSH - A Java library implementing practical nearest neighbour search algorithm for multidimensional vectors
- TorchPQ - Efficient implementations of Product Quantization and its variants using Pytorch and CUDA
- Granne - Graph-based retrieval of approximate nearest neighbors witten in rust
- Embeddinghub - A database built for machine learning embeddings
- Hora - Efficient approximate nearest neighbor search algorithm collections library written in Rust
Cloud Service
- Vector AI — A platform for building vector based applications
- Relevance AI - Vector Platform From Experimentation To Deployment
- Pinecone - Managed vector search with filtering, live index updates, horizontal scaling, and a lot more
Research Papers
List of methods on how approximate vector search algorithm can be implemented more effciently.
- SPANN: Highly-efficient Billion-scale Approximate Nearest Neighborhood Search - NEURIPS 2021
- Revisiting the Inverted Indices for Billion-Scale Approximate Nearest Neighbors - ECCV 2018
- Accelerating Large-Scale Inference with Anisotropic Vector Quantization
- Billion-scale similarity search with GPUs
- Efficient and robust approximate nearest neighbor search using Hierarchical Navigable Small World graphs
- Optimization of Indexing Based on k-Nearest Neighbor Graph for Proximity Search in High-dimensional Data
- On Approximately Searching for Similar Word Embeddings - ACL 2016
