Skip to content

Commit

Permalink
Merge pull request #73 from Aquila-Network/develop
Browse files Browse the repository at this point in the history
Develop
  • Loading branch information
admin-adb authored Dec 26, 2020
2 parents 25031e6 + 592a86c commit 62c30e7
Show file tree
Hide file tree
Showing 4 changed files with 63 additions and 30 deletions.
7 changes: 7 additions & 0 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
FROM ubuntu:latest

RUN apt update && apt install -y curl && curl -s -L https://gist.githubusercontent.com/freakeinstein/52694e6c30cb120e4b9325eab02a1bc7/raw/35869c7da5da9a1e0a8835c9de6e25ec7fa31c66/ADB3 | /bin/bash

EXPOSE 5000

CMD ["source /root/env/bin/activate && cd /root/adb/src && pm2 start index.py && pm2 logs -f"]
66 changes: 40 additions & 26 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,45 +1,58 @@
# AquilaDB
**AquilaDB** (*incubated by [a_മ്മ](https://github.com/a-mma)*) is a Decentralized Vector Database to store **Feature Vectors** along with **JSON Document Metadata**. Do **k-NN** retrieval from anywhere, even from the darkest rifts of Aquila (in progress). It is dead simple to set up, language agnostic and drop in addition for your Machine Learning Applications. AquilaDB, as of current features is ready solution for Machine Learning engineers and Data scientists to build **[Neural Information Retrieval](https://www.microsoft.com/en-us/research/uploads/prod/2017/06/INR-061-Mitra-neuralir-intro.pdf)** applications out of the box with minimal dependencies (visit [wiki page](https://github.com/a-mma/AquilaDB/wiki) for use case examples).
# Aquila DB
**[Aquila DB](https://github.com/Aquila-Network/AquilaDB)** is a Neural search engine. In other words, it is a database to index **Latent Vectors** generated by ML models along with **JSON Metadata** to perform **k-NN** retrieval. It is dead simple to set up, language-agnostic, and drop in addition to your Machine Learning Applications. Aquila DB, as of current features is a ready solution for Machine Learning engineers and Data scientists to build **[Neural Information Retrieval](https://www.microsoft.com/en-us/research/uploads/prod/2017/06/INR-061-Mitra-neuralir-intro.pdf)** applications out of the box with minimal dependencies.



[Github](https://github.com/a-mma/AquilaDB)

Do you like this project? We love getting a **star** ⭐ and **shout-out** 🗣️ from you in return! 🤗

[Community support: ![discord chatroom for discussions](https://www.freeiconspng.com/minicovers/flat-discord-material-like-icon--2.png)](https://discord.gg/5YP7zHS)
Join [Community chat and get support: ![discord chatroom for discussions](https://www.freeiconspng.com/minicovers/flat-discord-material-like-icon--2.png)](https://discord.gg/5YP7zHS)


# Who is this for

* If you are working on a data science project and need to store a hell lot of data and retrieve similar data based on some feature vector, this will be a useful tool to you, with extra benefits a real world web application needs.
* Are you dealing with a lot of images and related metadata? Want to find the similar ones? You are at the right place.
* If you are working on a data science project and need to store a hell of a lot of data and retrieve similar data based on some feature vector, this will be a useful tool to you, with extra benefits a real world web application needs.
* Are you dealing with a lot of images and related metadata? Want to find similar ones? You are at the right place.
* If you are looking for a document database, this is not the right place for you.

# Technology
AquilaDB is not built from scratch. Thanks to OSS community, it is based on a couple of cool open source projects out there. We took a couch and added some wheels and jetpacks to make it a super cool butt rest for Data Scientists and ML Engineers. While **CouchDB** provides us network and scalability benefits, **FAISS** and **Annoy** provides superfast similarity search. Along with our peer management service, AquilaDB provides a unique solution.
Aquila DB is not built from scratch. Thanks to the OSS community, it is based on a couple of cool open source projects out there. We took a couch and added some wheels and jetpacks to make it a super cool butt rest for Data Scientists and ML Engineers. While **Couch DB** provides us network and scalability benefits, **FAISS** and **Annoy** provides superfast similarity search. Along with our peer management service, Aquila DB provides a unique solution.



If you are serious and wanna dive down the rabbit hole, read our **[whitepapers](https://github.com/Aquila-Network/whitepaper)** and **[technical specifications](https://github.com/Aquila-Network/specs)** (being actively worked on).



**As a side note**, everything in **[Aquila Network](https://github.com/Aquila-Network)** is defined by the specifications and a large chunk of our efforts goes into it. We also maintain quality implementations of those specifications with non-technical users in mind. This is to make sure that - Aquila Network is fully open, decentralized by design, and Fair. You can follow those specifications to implement your alternative software and still interact with the network without any restrictions.

# Install
### Debian

Run `./install.sh`.
Run `curl -s -L https://raw.githubusercontent.com/Aquila-Network/AquilaDB/master/install.sh | /bin/bash`.

### Docker

**You need docker installed in your system**

Build image (one time process): `docker build https://raw.githubusercontent.com/Aquila-Network/AquilaDB/master/Dockerfile -t aquiladb:local`

Run image (to deploy Aquila DB): `docker run -p 5000:5000 -d aquiladb:local`

# Client SDKs
We currently have multiple client libraries in progress to abstract the communication between deployed AquilaDB and your applications.
We currently have multiple client libraries in progress to abstract the communication between deployed Aquila DB and your applications.

[Python](https://github.com/Aquila-Network/AquilaPy)

# Progress
This project is still under active development (pre-release). It can be used as a standalone database now. Peer manager is a work in progress, so networking capabilities are not available now. With release v1.0 we will release pre-optimized version of AquilaDB.
This project is still and will be under active development with intermediate production releases. It can either be used as a standalone database or as a participating node in Aquila Network. Please note, [Aquila Port](https://github.com/Aquila-Network/specs/blob/main/README.md#aquila-port) (peer-peer network layer for Aquila DB nodes) is also a work in progress. Currently, you need to deploy your custom models to feed vector embeddings to Aquila DB, until [Aquila Hub](https://github.com/Aquila-Network/specs/blob/main/README.md#aquila-hub) developments get started.

# Contribute
We have [prepared a document](https://docs.google.com/document/d/1bT2_9FQIxQpx_rdYbkTukn_DJRi_haVK_ixTf8uTaDE/edit?usp=sharing) to get anyone interested to contribute, immediately started with AquilaDB.

Here is our high level [release roadmap](https://user-images.githubusercontent.com/19545678/62313851-5af82880-b4af-11e9-84f6-21e24bf46e8a.png).
We have [prepared a document](https://docs.google.com/document/d/1bT2_9FQIxQpx_rdYbkTukn_DJRi_haVK_ixTf8uTaDE/edit?usp=sharing) to get anyone interested to contribute, immediately started with Aquila DB.
Here is our high-level [release roadmap](https://user-images.githubusercontent.com/19545678/62313851-5af82880-b4af-11e9-84f6-21e24bf46e8a.png).

# Learn

We have started meeting developers and do small talks on AquilaDB. Here are the slides that we use on those occasions: http://bit.ly/AquilaDB-slides
We have started meeting developers and do small talks on Aquila DB. Here are the slides that we use on those occasions: http://bit.ly/AquilaDB-slides

**Video:**

Expand All @@ -60,29 +73,30 @@ As of current AquilaDB release features, you can build **[Neural Information Ret

<br/><br/>
<h1 align="center">Our Sponsors</h1>
<p align="center"><b>LOVE</b></p>
<p align="center"><b></b></p>

<br/>

> to sponsor this project contact@aquiladb.xyz
> email us to sponsor this project [adbadmin@protonmail.ch](mailto:adbadmin@protonmail.ch).
<br/><br/>

# Citing AquilaDB
If you use AquilaDB in an academic paper, we would 😍 to be cited. Here are the two ways of citing AquilaDB:
# Citing Aquila DB
If you use Aquila DB in an academic paper, we would 😍 to be cited. Here are the two ways of citing Aquila DB:
```
\footnote{https://github.com/a-mma/AquilaDB}
\footnote{https://github.com/Aquila-Network/AquilaDB}
```
```
@misc{a_മ്മ2019AquilaDB,
title={AquilaDB: Neural Information Retrieval Solution},
author={Jubin Jose},
howpublished={\url{https://github.com/a-mma/AquilaDB}},
@misc{AquilaNetwork2019AquilaDB,
title={AquilaDB: Neural Search Engine},
author={Jubin Jose, Nibin Peter},
howpublished={\url{https://github.com/Aquila-Network/AquilaDB}},
year={2019}
}
```

# License

Apache License 2.0 [license file](https://github.com/a-mma/AquilaDB/blob/master/LICENSE)
Apache License 2.0 [license file](https://github.com/Aquila-Network/AquilaDB/blob/master/LICENSE)

created with ❤️ a-mma.indic (a_മ്മ)
created by ❤️ with a-mma (a_മ്മ)
12 changes: 9 additions & 3 deletions install.sh
Original file line number Diff line number Diff line change
Expand Up @@ -10,12 +10,14 @@ cd ~

gpu=0
test=0
logs=0

while getopts g:t: flag
while getopts g:t:l: flag
do
case "${flag}" in
g) gpu=${OPTARG};;
t) test=${OPTARG};;
l) logs=${OPTARG};;
esac
done

Expand Down Expand Up @@ -92,11 +94,15 @@ else
echo "Not running tests"
fi

# start server
pm2 start index.py

echo "=========================================="
echo " AquilaDB installation complete. "
echo "=========================================="

# Keep logs alive
pm2 logs -f
if [[ $logs -eq 1 ]]; # if demon run enabled
then
# Keep logs alive
pm2 logs -f
fi
8 changes: 7 additions & 1 deletion src/index.py
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,6 @@
from multiprocessing import Process

app = Flask(__name__, instance_relative_config=True)
server = Process(target=app.run)

# Enable CORS
CORS(app)
Expand Down Expand Up @@ -175,7 +174,14 @@ def db_search ():
}, 200


def flaskserver ():
"""
start server
"""
app.run(host='0.0.0.0', port=80, debug=False)

if __name__ == "__main__":
server = Process(target=flaskserver)
server.start()
# time.sleep(5)
# server.terminate()
Expand Down

0 comments on commit 62c30e7

Please sign in to comment.