Open Retrieval Augmented Generation

A simple shell-based tool that you can customize for any application.
Visit the Organization · Report a Bug

MAJOR UPDATE - Google Collaboratory Support

This tool has been put entirely into a Google Collaboratory notebook available at https://colab.research.google.com/drive/1vhm-s94JTGwf3_uvexzyVIPIZrfPUFUj?usp=sharing. Please use this to run and access the RAG-pipeline detailed in this codebase.

About The Project

A local Retrieval Augmented Generation method to complete any tasks. This specific implementation is built for accelerating email writing using previously written emails. A couple of changes and this can be suitable for any task!

(back to top)

Getting Started

To get a local copy up and running follow these simple steps.

Prerequisites

WSL - Windows ONLY

This software runs on Linux/Mac - run it on Windows with Windows Subsystem for Linux (WSL).

Install WSL by running the following in Powershell:

wsl --install

Run standard WSL setup.

Ollama

Download Ollama based on your system:

Windows w/ WSL or Linux

Open the Linux Comand Line and run the following:

curl -fsSL https://ollama.com/install.sh | sh

MacOS

Download Ollama from this link and run normal setup.

Anaconda

Install Anaconda within WSL, on Linux, or on Mac.

• Install Anaconda on WSL by following this link.
• Use the Anaconda webpage here for all other systems

Outlook

Please install Outlook on your machine by following this link

• Mac: Outlook Installation

• Windows: Install directly from Windows Store

Installation

Follow these steps in order to set up the repo.

1. Clone the Repo

   git clone https://github.com/Rohit-K814307/open_rag/tree/windows

2. Change Directory and Install Conda Environment

    cd open_rag
   
    conda env create -f environment.yml

3. Get Outlook Email CSV

   i) Check this webpage for more specific instructions on getting email CSV from Outlook app

   • Confirm that you are using the "old" Outlook app

   ii) Save to ./open_rag/data/raw.CSV

4. Set up Ollama Model

   i) In a separate instance of the same terminal, run

   ollama serve

   ii) Create the Ollama custom model

   ollama create email_model_qwen -f ./email_model_qwen.txt 

(back to top)

Usage

This project is an LLM-powered client.

In order to run the client, please follow these steps:

1. Open a new terminal at root
2. Allow running the generation script

   chmod +x ./open_rag/scripts/generate.sh

3. Query the LLM with your desired inputs and choose model parameters

   bash ./open_rag/scripts/generate.sh -q <query> -c <full path to csv emails> -e <your email address> ...

   • run ./open_rag/scripts/generate.sh -h for help with arguments

(back to top)

Contributing

Contributions are what make the open source community such an amazing place to learn, inspire, and create. Any contributions you make are greatly appreciated.

If you have a suggestion that would make this better, please fork the repo and create a pull request. You can also simply open an issue with the tag "enhancement". Don't forget to give the project a star! Thanks again!

1. Fork the Project
2. Create your Feature Branch (git checkout -b feature/AmazingFeature)
3. Commit your Changes (git commit -m 'Add some AmazingFeature')
4. Push to the Branch (git push origin feature/AmazingFeature)
5. Open a Pull Request

(back to top)

License

Distributed under the MIT License. See LICENSE.txt for more information.

(back to top)

Contact

Rohit Kulkarni - "kulkarni.rohitva@gmail.com"

Link to NVTEI Organization main page:

(back to top)

Acknowledgments

This project is presented to communities through the Northern Virginia Technology Education Initiative (NVTEI).

(back to top)