Create and search a vector database to get a response from the large language model that's more accurate. This is commonly referred to as "retrieval augmented generation" (RAG)! You can watch an introductory Video or read a Medium article about the program.

Back to Top

Go to the latest "release" and download the ZIP file. Extract its contents and navigate to the src folder.

• NOTE: If you simply clone this repository you will get the development version, which may or may not be stable.

Within the src folder, create a virtual environment:

python -m venv .

Activate the virtual environment:

.\Scripts\activate

Run the setup script:

Only Windows is supported for now.

python setup_windows.py

Back to Top

• More detailed instructions are being consolidated into the Ask Jeeves functionality, which can be accessed from the "Ask Jeeves" menu option. Please create an issue if Jeeves is not working.

.\Scripts\activate

python gui.py

• Download a vector/embedding model from the Models Tab.
• Select one or more files to be put into the queue.
  • 📄 Non-audio files (including images) can be selected by clicking the Choose Files button within the Create Database Tab.
  • 🎵 Audio files must be transcribed first within the Tools Tab.
  • 🖼️ Images can be selected just like other general file types.
• Click the Create Vector Database button within the Create Database Tab.

  Make sure and change the chunk size and chunk overlap settings within the Settings Tab first.

• Within the Query Database Tab, select the database you want to search.
• Select the backend that will analyze the results from the vector database and provide a response: Local Models, Kobold, LM Studio or ChatGPT.
• Click Submit Question.
  • The chunks only checkbox will display the results from the vector database.

If you use either the Kobold or LM Studio backends you must be familiar with those programs. For example, LM Studio must be running in "server mode" and handles the prompt formatting. However,Kobold automatically starts in server mode but requires you to specify the prompt formatting. This program no longer provides detailed instructions on how to use either of these two backends. For more information:

• Kobold home page, instructions, and Discord server
• LM Studio home page, instructions, and Discord server.

• In the Manage Databases Tab, select a database and click Delete Database.

Back to Top

Feel free to report bugs or request enhancements by creating an issue on github and I will respond promptly.

I welcome all suggestions - both positive and negative. You can e-mail me directly at "bbc@chintellalaw.com" or I can frequently be seen on the KoboldAI Discord server (moniker is vic49). I am always happy to answer any quesitons or discuss anything vector database related! (no formal affiliation with KoboldAI).