Create and search a vector database to get a response from the large language model that's more accurate. This is commonly referred to as "retrieval augmented generation" (RAG)! You can watch an introductory Video or read a Medium article about the program.
🐍 Python 3.11 or Python 3.12 • 📁 Git • 📁 Git LFS • 🌐 Pandoc • 🛠️ Compiler |
---|
The above link downloads Visual Studio as an example. Make sure to install the required SDKs, however.
Go to the latest "release" and download the ZIP file. Extract its contents and navigate to the src
folder.
- NOTE: If you simply clone this repository you will get the development version, which may or may not be stable.
Within the src
folder, create a virtual environment:
python -m venv .
Activate the virtual environment:
.\Scripts\activate
Run the setup script:
Only
Windows
is supported for now.
python setup_windows.py
- More detailed instructions are being consolidated into the
Ask Jeeves
functionality, which can be accessed from the "Ask Jeeves" menu option. Please create an issue if Jeeves is not working.
.\Scripts\activate
python gui.py
- Download a vector/embedding model from the
Models Tab
. - Select one or more files to be put into the
queue
.- 📄 Non-audio files (including images) can be selected by clicking the
Choose Files
button within theCreate Database Tab
. - 🎵 Audio files must be transcribed first within the
Tools Tab
. - 🖼️ Images can be selected just like other general file types.
- 📄 Non-audio files (including images) can be selected by clicking the
- Click the
Create Vector Database
button within theCreate Database Tab
.Make sure and change the
chunk size
andchunk overlap
settings within theSettings Tab
first.
- Within the
Query Database Tab
, select the database you want to search. - Select the backend that will analyze the results from the vector database and provide a response:
Local Models
,Kobold
,LM Studio
orChatGPT
. - Click
Submit Question
.- The
chunks only
checkbox will display the results from the vector database.
- The
If you use either the Kobold
or LM Studio
backends you must be familiar with those programs. For example, LM Studio
must be running in "server mode" and handles the prompt formatting. However,Kobold
automatically starts in server mode but requires you to specify the prompt formatting. This program no longer provides detailed instructions on how to use either of these two backends. For more information:
- Kobold home page, instructions, and Discord server
- LM Studio home page, instructions, and Discord server.
- In the
Manage Databases Tab
, select a database and clickDelete Database
.
Feel free to report bugs or request enhancements by creating an issue on github and I will respond promptly.
I welcome all suggestions - both positive and negative. You can e-mail me directly at "bbc@chintellalaw.com" or I can frequently be seen on the KoboldAI
Discord server (moniker is vic49
). I am always happy to answer any quesitons or discuss anything vector database related! (no formal affiliation with KoboldAI
).