Transformer-based Embedding Retrieval with Product Quantization for Edge Computing

Implementation of "vector database" embedded in the browser thanks to JS and Product Quantization compression. Thanks to the Transformer.js library the query is computed in the web page. The asymmetric distance computation is done also locally thanks to the codewords and pqcodes, finally, a heap sort is used to find similar vectors.

Demo : https://dev7384.dctawdl2zqela.amplifyapp.com/

Requirements

Python packages

• nanopq (version using scikit-learn for the kmeans and multithreated)

Usage

Author

• Sébastien Campion

License

• GPLv3 For other licenses, please contact the author.