Manualify/src/Manualify.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Manualify.ai\n",
    "---\n",
    "\n",
    "`Manualify.ai:` A Smart AI for Answering Questions from Tutorials: Harnessing the Power of RAG LLMs and Documents.\n",
    "\n",
    "The objective is to develop an AI capable of answering questions by harnessing information from websites, tutorials, ServiceNow, customer support resources, and transactional databases.\n",
    "\n",
    "**Author:** Amit Shukla\n",
    "\n",
    "**Connect**\n",
    "Author: Amit Shukla\n",
    "\n",
    "[GitHub](https://github.com/AmitXShukla) | [X](https://x.com/@ashuklax) | [YouTube](https://youtube.com/@Amit.Shukla) | [Medium](https://amit-shukla.medium.com)\n",
    "\n",
    "---\n",
    "\n",
    "In this blog, we will create an online manual for Python, Oracle or Julia Lang, Angular and Flutter.\n",
    "\n",
    "**Step 0:** getting started\n",
    "\n",
    "**Step 1:** build a simple web crawler - Scrapify\n",
    "    \n",
    "In this section, we will\n",
    "\n",
    "1. query LLM API and build a Q&A from LLM.\n",
    "2. read data from PDF files and then query LLM.\n",
    "3. Scape an online page and query LLM.\n",
    "4. automated crawling: build a simple `web crawler` to gather text from the website. The crawler will collect links from the given domain and then visit each link to download the associated text. \n",
    "5. explore various options for downloading data from Single Page Applications (SPAs) using web scraping techniques and libraries such as BeautifulSoup, Scrapy, and Selenium.\n",
    "6. Image data extraction:\n",
    "look into different methods for extracting data from images, including Optical Character Recognition (OCR) techniques and libraries such as Tesseract.\n",
    "7. read data from PDF files: Additionally, we will examine ways to read and extract data from PDF files using libraries such as PyPDF2 and PDFMiner.\n",
    "8. Querying Language Models (LLMs):\n",
    "Once we have extracted the data, we will then query the Language Models (LLMs) with the extracted data to generate insights and answers.\n",
    "\n",
    "**Step 2:** We will convert all PDFs to csv and simply build a Q&A prompt using Gen AI (Claude) with entire file content at once.\n",
    "\n",
    "**Step 3:** Creating Embedding from csvs and other documents to create a Vector database.\n",
    "\n",
    "**Step 4:** Using RAG and LLMs to query manual documments.\n",
    "\n",
    "**Step 5:** Using SQL queries with Functional calling.\n",
    "\n",
    "**Step 6:** creating an online app and hosting"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "At some point, you may wonder, Is it really worth investing time and resources into building a data lake and RAG database when AI models can handle massive datasets? Can't we just let machines do the heavy lifting?\n",
    "\n",
    "LLM inference is still costly and use-case dependent. Even with massive token processing capacity, it's meaningless without intelligent data. RAG and fine-tuning models will continue to thrive as long as it serve our needs.\n",
    "\n",
    "I'd rather spend time learning and building tools using these technologies than engaging in pointless debates. Building takes less time and effort than debating, so I'll focus on creating and refining my models with my own data. Then, I'll compare results to see what works best."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Steps to create Manualify\n",
    "\n",
    "---\n",
    "<!-- gitGraph TB:\n",
    "    commit id: \"query\" tag: \"build prompt\"\n",
    "    commit id: \"1\"\n",
    "    branch Query\n",
    "    commit id: \"2\" tag: \"LLM\"\n",
    "    branch PDFQuery\n",
    "    commit id: \"3\" tag: \"send PDF as sontext\"\n",
    "    branch OnlineScapping\n",
    "    commit id: \"4\" tag: \"online content as context\"\n",
    "    branch AutoScapping\n",
    "    commit id: \"5\" tag: \"auto online download\"\n",
    "    branch EmbeddingsVectorDB\n",
    "    commit id: \"6\" tag: \"embeddings Vector DB\"\n",
    "    branch RAG\n",
    "    commit id: \"7\" tag: \"query RAG\"\n",
    "    branch SQLQueries\n",
    "    commit id: \"8\" tag: \"query RDBMs\"\n",
    "    branch FunctionCalling\n",
    "    commit id: \"9\" tag: \"Function Calling\" -->\n",
    "\n",
    "`Steps to create Manualify`\n",
    "\n",
    "![Process Flow](../images/processflow.png)\n",
    "\n",
    "## Process Flow diagram\n",
    "---\n",
    "\n",
    "`Brief overview of the RAG stack : Voyage AI`\n",
    "\n",
    "![Brief overview of the RAG stack : Voyage AI](https://files.readme.io/ec25408-RAG-white.png)\n",
    "\n",
    "The diagram below illustrates the high-level architecture and data flow of this project. Please note that **not** all of these features are included in the Community version, and the Pro/Custom version may vary significantly from this diagram based on individual implementation.\n",
    "\n",
    "As depicted in the diagram, this basic web crawler accepts a URL as input and navigates through all linked sub-pages, collecting text from the specific website one page at a time.\n",
    "Our current goal is straightforward: we aim to extract relevant text information, metadata and other useful details using this crawler. In subsequent blogs, we plan to construct an embedding vector data store or a vector database composed of embeddings derived from the text of the website and other available documentation and knowledge bases.\n",
    "\n",
    "Now let's proceed to construct our simple web crawler that fetches text and pertinent information from all pages of a given website.\n",
    "\n",
    "![Process Flow](../images/process_flow.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 0: Getting Started\n",
    "---\n",
    "\n",
    "We'll discuss few different approaches.\n",
    "- **Approach 1** - Setup Anthropic LLMs\n",
    "- **Approach 2** - Setup Open AI LLMs\n",
    "- **Approach 3** - Setup Google Gemini AI\n",
    "- **Approach 4** - Setup Groq API\n",
    "- **Approach 5** - Setup Locally running models using Ollama\n",
    "\t\t\tsimilar steps can be done for other local LLMs like llama.cpp etc.\n",
    "\n",
    "The selection of a Large Language Model (LLM) is influenced by factors such as your specific business needs, financial constraints, and personal tastes.\n",
    "\n",
    "In this demo, I'll present few distinct methods for establishing connections with various LLM service providers.\n",
    "Additionally, I recommend utilizing this occasion as an avenue to evaluate Large Language Models against the backdrop of your unique inputs and industry-specific needs."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Approach 1 : Setup Anthropic LLMs\n",
    "Let's first set up our Python working environment. While we can also use Node.js, please note that for the current version, we will be using Python for development. \n",
    "\n",
    "Please signup using these links and get your own API Keys.\n",
    "- [ANTHROPIC_API_KEY](https://docs.anthropic.com/claude/reference/getting-started-with-the-api)\n",
    "- [YOUR_PINECONE_API_KEY](https://docs.pinecone.io/docs/quickstart)\n",
    "- [VOYAGE_API_KEY](https://docs.voyageai.com/docs/api-key-and-installation)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "##########################################################################\n",
    "## Although not mandatory, \n",
    "## it is highly recommended to set up a new Python working environment. ##\n",
    "##########################################################################\n",
    "## To create a virtual environment called `GenAI`, follow the steps:\n",
    "\n",
    "## On Windows: \n",
    "# !python -m venv GenAI GenAI\\Scripts\\activate\n",
    "\n",
    "## On macOS or Linux: \n",
    "# !python3 -m venv GenAI source GenAI/bin/activate\n",
    "\n",
    "## Then, install required packages using pip: \n",
    "# !pip install pandas numpy matplotlib seaborn tqdm beautifulsoup4\n",
    "\n",
    "## install only in case if you are using OpenAI\n",
    "# !pip install openai\n",
    "\n",
    "## install only in case if you are using Claude\n",
    "# !pip install anthropic datasets pinecone-client voyageai\n",
    "\n",
    "## in case if you fork this repo, just run\n",
    "# !pip install -r requirements.txt\n",
    "\n",
    "# !pip install --upgrade pip\n",
    "# !pip freeze > requirements.txt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import platform;\n",
    "print(platform.processor())\n",
    "\n",
    "import os\n",
    "\n",
    "####################################################\n",
    "## if you are using OpenAI LLMs\n",
    "## setup windows environment variable OPENAI_API_KEY\n",
    "####################################################\n",
    "# import openai\n",
    "# openai.api_key = os.getenv(\"OPENAI_API_KEY\")\n",
    "\n",
    "####################################################\n",
    "## if you are using Anthropic Claude LLMs\n",
    "## sign up for API keys & setup windows environment variable \n",
    "## ANTHROPIC_API_KEY, PINECONE_API_KEY & VOYAGE_API_KEY\n",
    "####################################################\n",
    "# import anthropic\n",
    "\n",
    "if (not os.environ.get(\"ANTHROPIC_API_KEY\")) | (not os.environ.get(\"PINECONE_API_KEY\")) | (not os.environ.get(\"VOYAGE_API_KEY\")):\n",
    "    print(\"One of the api key is missing.\")\n",
    "else:\n",
    "    print(\"All API Keys are in place.\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Test LLM API\n",
    "import anthropic\n",
    "\n",
    "client = anthropic.Anthropic(\n",
    "    # defaults to os.environ.get(\"ANTHROPIC_API_KEY\")\n",
    "    # you don't need to pass api_key explicitly\n",
    "    api_key=os.environ.get(\"ANTHROPIC_API_KEY\")\n",
    ")\n",
    "\n",
    "message = client.messages.create(\n",
    "    model=\"claude-3-opus-20240229\",\n",
    "    max_tokens=1000,\n",
    "    temperature=0.0,\n",
    "    system=\"Respond only in Yoda-speak.\",\n",
    "    messages=[\n",
    "        {\"role\": \"user\", \"content\": \"how are args and keyword arguments defined in python?\"}\n",
    "    ]\n",
    ")\n",
    "print(message.content)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Approach 2 : Setup Open AI LLMs\n",
    "\n",
    "Let's first set up our Python working environment. While we can also use Node.js, please note that for the current version, we will be using Python for development.\n",
    "\n",
    "Please signup and get your own API Keys.\n",
    "\n",
    "[Open AI API Key](https://platform.openai.com/docs/api-reference/introduction)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# !pip install openai"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Let's make sure your API keys are properly setup.\n",
    "import platform;\n",
    "print(platform.processor())\n",
    "import os\n",
    "\n",
    "if (not os.environ.get(\"OPENAI_API_KEY\")):\n",
    "    print(\"One of the api key is missing.\")\n",
    "else:\n",
    "    print(\"All API Keys are in place.\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# test LLM\n",
    "from openai import OpenAI\n",
    "client = OpenAI()\n",
    "\n",
    "response = client.chat.completions.create(\n",
    "  model=\"gpt-4o-mini-2024-07-18\",\n",
    "  messages=[{\"role\": \"user\", \"content\": \"how are args and keyword arguments defined in python?\"}],\n",
    "  temperature=1,\n",
    "  max_tokens=256,\n",
    "  top_p=1,\n",
    "  frequency_penalty=0,\n",
    "  presence_penalty=0\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Approach 3 : Setup Google Gemini LLMs\n",
    "\n",
    "Let's first set up our Python working environment. While we can also use Node.js, please note that for the current version, we will be using Python for development.\n",
    "\n",
    "Please signup and get your own API Keys.\n",
    "\n",
    "[Gemini API Key](https://ai.google.dev/gemini-api)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# !python.exe -m pip install --upgrade pip\n",
    "# !pip install -q -U google-generativeai"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# export API_KEY=<YOUR_API_KEY>\n",
    "\n",
    "import google.generativeai as genai\n",
    "import os\n",
    "\n",
    "genai.configure(api_key=os.environ[\"API_KEY\"])\n",
    "model = genai.GenerativeModel('gemini-1.5-flash')\n",
    "\n",
    "response = model.generate_content(\"Write a story about an AI and magic\")\n",
    "print(response.text)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Approach 4 - Running Groq Llama 3.2 API\n",
    "\n",
    "Let's first set up our Python working environment. While we can also use Node.js, please note that for the current version, we will be using Python for development.\n",
    "\n",
    "Please signup and get your own API Keys.\n",
    "\n",
    "[Groq API Key](https://console.groq.com/keys)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# !pip install groq"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from groq import Groq\n",
    "\n",
    "client = Groq()\n",
    "completion = client.chat.completions.create(\n",
    "    model=\"llama3-8b-8192\",\n",
    "    messages=[\n",
    "        {\n",
    "        \"role\": \"user\",\n",
    "        \"content\": \"\"\n",
    "        }\n",
    "    ],\n",
    "    temperature=1,\n",
    "    max_tokens=1024,\n",
    "    top_p=1,\n",
    "    stream=True,\n",
    "    stop=None,\n",
    ")\n",
    "\n",
    "for chunk in completion:\n",
    "    print(chunk.choices[0].delta.content or \"\", end=\"\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Approach 5 - Setup Locally running models using Ollama\n",
    "[download](https://ollama.com/download) and setup Ollama\n",
    "\n",
    "Let's first set up our Python working environment."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# !pip install graphviz Pillow networkx requests ollama pypdf beautifulsoup4 tiktoken pandas matplotlib seaborn langchain chromadb pysqlite3-binary"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#############################################################\n",
    "# locally hosted ollama models\n",
    "# install ollama binaries/exe from https://ollama.com/download\n",
    "#############################################################\n",
    "\n",
    "# run local installation then\n",
    "# on windows powershell | MacOS Terminal\n",
    "# run following commands one at a time\n",
    "#############################################################\n",
    "\n",
    "# !ollama list\n",
    "\n",
    "# ## don't install heavy size LLMs, start with small LLMs like 3B or 7B versions\n",
    "# ## phi3:mini is a great LLM model, I find it compatible to do most of the tasks\n",
    "# !ollama pull phi3:mini\n",
    "# !ollama pull llama3.2\n",
    "\n",
    "# ## don't install heavy size LLMs, start with small LLMs with 3B or 7B versions\n",
    "# ## This is an embedding model\n",
    "# ollama pull all-minilm:latest\n",
    "# ollama pull mxbai-embed-large # SOTA model - recommended\n",
    "\n",
    "# ## don't install heavy size LLMs, start with small LLMs with 3B or 7B versions\n",
    "# ## This is a code model\n",
    "# ollama pull codegemma\n",
    "\n",
    "# ollama list\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "automating ollama start"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# to setup auto start on Linux VM\n",
    "# add this to your .zshrc file\n",
    "\n",
    "# alias l3=\"ollama run llama3.2\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# run ollama LLM\n",
    "# this code assumes you have a local ollama running on your machine\n",
    "# refer to previous step if you have any issues\n",
    "\n",
    "import ollama\n",
    "response = ollama.chat(model='llama3.2', messages=[\n",
    "    {\n",
    "        'role': 'user',\n",
    "        'content': 'rephrase sentence: let us first make sure, we have a LLM/Embedding API running successfully',\n",
    "    },\n",
    "])\n",
    "\n",
    "print(response['message']['content'])\n",
    "# When the output displays, it indicates that your program has been run without errors and completed its intended tasks effectively.\n",
    "# congratulations!, you have a LLM running locally on your machine."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#######################################\n",
    "# fast API based hosted models\n",
    "# for instance llama.ccp\n",
    "#######################################\n",
    "\n",
    "# !curl -X 'GET' '<<IP.HOSTED.Models>>/v1/models' -H 'accept: application/json'\n",
    "\n",
    "import requests\n",
    "url = '<<IP.HOSTED.Models>>/v1/models'\n",
    "payload = \"models\"\n",
    "headers = {'content-type': 'application/json', 'Accept-Charset': 'UTF-8'}\n",
    "r = requests.post(url, data=payload, headers=headers)\n",
    "\n",
    "!curl -X 'GET' 'https://<<IP.HOSTED.Models>>/v1/models' -H 'accept: application/json'\n",
    "\n",
    "r = requests.get(url)\n",
    "r.json()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "With LLM models running locally or via API, we'll create flexible functions to interact with them. This approach allows us to easily switch between local, cloud-based, or other LLM models as needed.\n",
    "\n",
    "#### define generic reusable static contexts"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# define a generic function whick takes context, message and return results\n",
    "\n",
    "#########################################################\n",
    "# These contexts are temporary and will be later replaced\n",
    "# by LLM Tools | Functions automation (later sections)\n",
    "# or use langchain to change context/prompt dynamically\n",
    "#########################################################\n",
    "\n",
    "# define different versions of contexts\n",
    "# for POC purpose, these context will be used along with user prompt\n",
    "\n",
    "context_append_bool_assistant = \"respond in only true or false. \"\n",
    "context_append_one_word_assistant = \"respond in only one word. \"\n",
    "context_append_code_assistant = \"You are a helpful assistant with advance SQL coding skills. \"\n",
    "context_append_service_assistant = \"You are a helpful assistant with reasoning skills. \"\n",
    "\n",
    "context_append_output_assistant = \"\"\"Ensure that the response strictly adheres to JSON format, excluding all additional content. Attached is an example demonstrating the desired JSON structure for the expected output. \"\"\"\n",
    "\n",
    "context_append_SQL_assistant = \"return SQL or schema only and nothing else. \"\n",
    "\n",
    "context_sample_metadata = context_append_service_assistant + context_append_output_assistant + \"\"\"\n",
    "                                From the given message, retrieve, what domain user is referring to? for example\n",
    "                                {\n",
    "                                    \"Domain\": [\"Sales Order\"]\n",
    "                                }\n",
    "                                \"\"\"\n",
    "context_sample_userinfo = context_append_service_assistant + context_append_output_assistant + \"\"\"\n",
    "                                From the given message, retreive, data/user information. example Joe Biden is POTUS,\n",
    "                                his email ID will be Joe.X.Biden@whitehouse.org because, even if he does have a middle name, he doesn't use it that often.\n",
    "                                so put an X there in middle when creating his email ID.\n",
    "                                but answer only in this format.\n",
    "                                {\n",
    "                                    \"Person\": [\"Joe Biden\"],\n",
    "                                    \"EMPLOYEEID\": [\"ABC1234\"],\n",
    "                                    \"emailID\": [\"Joe.X.Biden@whitehouse.org\"]\n",
    "                                }\n",
    "                            ,\"\"\"\n",
    "\n",
    "context_sample_table_schema = context_append_code_assistant + context_append_SQL_assistant + \"\"\"retrieve table schema from this SQL\n",
    "                                SELECT name, salary\n",
    "                                FROM EMPLOYEES\n",
    "                                WHERE department = 'Engineering';\n",
    "                            \"\"\"\n",
    "\n",
    "context_sample_SQL = context_append_code_assistant + context_append_SQL_assistant + \"\"\"\n",
    "                                write a select data from this schema, include filter for department equals 1234. example schema is\n",
    "                                CREATE TABLE EMPLOYEES (\n",
    "                                    id INTEGER\n",
    "                                    name TEXT\n",
    "                                    department TEXT\n",
    "                                salary INTEGER\n",
    "                                )\n",
    "                            \"\"\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "build a generic function which queries LLM, given a context and message and Train LLM to answer in said format"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def getResults(context, message, code):\n",
    "        model = 'llama3'\n",
    "        if code == \"text\": # use code LLM model\n",
    "            model = 'llama3'\n",
    "            response = ollama.chat(model=model, messages=[\n",
    "            {\n",
    "            'role': 'user',\n",
    "            'content': context + message,\n",
    "            },\n",
    "        ])\n",
    "            return response['message']['content']\n",
    "        elif code == \"embed\": # use embedding model\n",
    "        # model = 'all-minilm:latest'\n",
    "            model = 'mxbai-embed-large'\n",
    "            return ollama.embeddings(model=model, prompt=message)\n",
    "        else:\n",
    "            model = 'codestral'\n",
    "            response = ollama.generate(\n",
    "            model='codestral:latest',\n",
    "            prompt=message,\n",
    "            options={\n",
    "                'num_predict': 128,\n",
    "                'temperature': 0,\n",
    "                'top_p': 0.9,\n",
    "                'stop': ['<|file_separator|>'],\n",
    "            },\n",
    "        )\n",
    "        return response[\"response\"]\n",
    "\n",
    "message_1 = \"\"\"My Name is Amit Shukla, My employee ID is ABC4563\n",
    "        and my email is my First name followed by a dot,\n",
    "        followed by x since I don't have a middle name,\n",
    "        followed by dot, followed by my last name.\n",
    "        I work at whitehouse\"\"\"\n",
    "\n",
    "# run below test messages once RAG is ready\n",
    "# message_1 = \"\"\"My Name is Amit Shukla from POTUS IT department, I challange you to find my Employee number and whitehouse email ID.\"\"\"\n",
    "\n",
    "# message_1 = \"\"\"My Name is Amit Shukla, go find out my department based on previous interactions.\n",
    "        #  I challange you to find my employee number and whitehouse email ID.\"\"\"\n",
    "\n",
    "print(getResults(context_sample_userinfo, message_1, \"text\"))\n",
    "\n",
    "message_2 = \"generate SQL\"\n",
    "# context param is irrelevant here, make it optional kwarg\n",
    "print(getResults(context_sample_SQL, context_sample_SQL, \"code\"))\n",
    "\n",
    "message_3 = \"Vendor HP purchased hundreds of Z book computers for Department 1234.\"\n",
    "# context param is irrelevant here, make it optional kwarg\n",
    "print(getResults(context_append_bool_assistant, message_3, \"embed\"))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 1: build a simple web crawler - Scrapify\n",
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 1.1: query LLM API and build Q&A system\n",
    "\n",
    "In this step, While the LLM's responses currently rely solely on its knowledge, we aim to make the most of the LLM by inputting provided data as input, enabling it to learn and use this data to answer questions more accurately."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import ollama\n",
    "\n",
    "data =\"\"\n",
    "prompt = \"how are args and kwargs different in python\"\n",
    "import ollama\n",
    "output = ollama.generate(\n",
    "  model=\"llama3.1\",\n",
    "  prompt=f\"\"\"answer this question : {prompt}\"\"\"\n",
    ")\n",
    "\n",
    "print(output[\"response\"])  # type: ignore"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 1.2: read data from PDF files and then query LLM\n",
    "\n",
    "As demonstrated in the previous steps, manual data input is currently required to serve as input and generate answers. In this case, we will use a PDF such as manual or tutorial as reference to query."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# !pip install pypdf\n",
    "# !curl -O https://github.com/AmitXShukla/RPA/blob/main/SampleData/The%20Ultimate%20Guide%20to%20Data%20Wrangling%20with%20Python%20-%20Rust%20Polars%20Data%20Frame.pdf"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from pypdf import PdfReader\n",
    "\n",
    "reader = PdfReader(\"../downloads/Python - understanding functions.pdf\")\n",
    "number_of_pages = len(reader.pages)\n",
    "text = ''.join([page.extract_text() for page in reader.pages])\n",
    "print(text[:2155])\n",
    "\n",
    "import ollama\n",
    "\n",
    "data =\"\"\n",
    "prompt = \"how are args and kwargs different in python\"\n",
    "import ollama\n",
    "\n",
    "def get_completion(prompt):\n",
    "    output = ollama.generate(\n",
    "        model=\"llama3.1\",\n",
    "        prompt=f\"\"\"answer this question : {prompt}\"\"\"\n",
    "        )\n",
    "    return output[\"response\"]  # type: ignore\n",
    "\n",
    "completion = get_completion(\n",
    "    f\"\"\"Here is a local guide: <guide>{text}</guide>    \n",
    "\n",
    "Please do the following:\n",
    "1. Summarize the abstract about Python args \n",
    "and keyword args understanding at a kindergarten reading level. (In <kindergarten_abstract> tags.)\n",
    "2. Write the Methods section as a recipe from the Moosewood Cookbook. (In <moosewood_methods> tags.)\n",
    "\"\"\"\n",
    ")\n",
    "print(completion)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 1.3: Scrape an online page and query LLM\n",
    "\n",
    "In this step, we will construct a basic web crawler that will download text from a specified URL, using this downloaded text as input. By automating this process, we aim to eliminate the need for manual data input.\n",
    "\n",
    "These approaches are very useful in automation, for example, you want to run Assistant based on some online search or SQL results. You can achieve full automation while using these codes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# !pip install anthropic requests beautifulsoup4"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "USER_QUESTION=\"how are args and keyword arguments defined in python?\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "GENERATE_QUERIES=f\"\"\"\\n\\nHuman: You are an expert at Python programmer. \n",
    "Your proficiency in Python programming is exceptional.\n",
    "You have the ability to craft code, author blogs, and create tutorials. \n",
    "Typically, when a question is posed to you, your response is comprehensive and often includes illustrative code examples.\n",
    "\n",
    "User question: {USER_QUESTION}\n",
    "\n",
    "Format: {{\"queries\": [\"query_1\", \"query_2\", \"query_3\"]}}\\n\\nAssistant: {{\"\"\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import ollama\n",
    "\n",
    "# define generic function to query LLM\n",
    "def get_completion(prompt: str):\n",
    "    output = ollama.generate(\n",
    "    model=\"llama3.1\",\n",
    "    prompt=f\"\"\"answer this question : {prompt}\"\"\"\n",
    "    )\n",
    "    return output[\"response\"]  # type: ignore\n",
    "\n",
    "# query LLM based on fixed text\n",
    "queries_json = \"{\" + get_completion(GENERATE_QUERIES)\n",
    "print(queries_json)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "let's write code to scrape an online page."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import requests\n",
    "from bs4 import BeautifulSoup\n",
    "\n",
    "url = \"https://docs.python.org/3/tutorial/controlflow.html#more-on-defining-functions\"\n",
    "\n",
    "def get_page_content(url : str) -> str:\n",
    "    html = requests.get(url).text\n",
    "    soup = BeautifulSoup(html, 'html.parser')\n",
    "    return soup.get_text(strip=True, separator='\\n')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "formatted_search_results = get_page_content(url)\n",
    "print(formatted_search_results)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# inject search results into prompt\n",
    "ANSWER_QUESTION = f\"\"\"\\n\\nHuman: I have provided you with the following search results:\n",
    "{formatted_search_results}\n",
    "\n",
    "Please answer the user's question using only information from the search results. Reference the relevant search result urls within your answer as links. Keep your answer concise.\n",
    "\n",
    "User's question: {USER_QUESTION} \\n\\nAssistant:\n",
    "\"\"\"\n",
    "print(ANSWER_QUESTION)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(get_completion(ANSWER_QUESTION))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 1.4: Scrapify: build a simple `web crawler`\n",
    "\n",
    "Now, since We can build and automate text retreival from one HTML page, we will build a simple `web crawler` to gather text from the website. The crawler will collect links from the given domain and then visit each link to download the associated text."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#############################################################################\n",
    "# credit: majority of this code is reference through OpenAI documentation\n",
    "# however, it's not necessary to use OpenAI API\n",
    "# like in this blog, we will use local ollama instead\n",
    "# https://platform.openai.com/docs/tutorials/web-qa-embeddings\n",
    "#############################################################################\n",
    "\n",
    "# install dependencies\n",
    "# !pip install requests pandas beautifulsoup4 tiktoken openai\n",
    "\n",
    "# some of embedding packages were not working so tried downgrading openai version\n",
    "# from openai.embeddings_utils import distances_from_embeddings, cosine_similarity\n",
    "# !pip install openai==0.27.7\n",
    "\n",
    "# !pip show openai"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# import all packages\n",
    "import requests\n",
    "import re\n",
    "import urllib.request\n",
    "from bs4 import BeautifulSoup\n",
    "from collections import deque\n",
    "from html.parser import HTMLParser\n",
    "from urllib.parse import urlparse\n",
    "import os\n",
    "import pandas as pd\n",
    "# import tiktoken\n",
    "# import openai\n",
    "import numpy as np\n",
    "# from openai.embeddings_utils import distances_from_embeddings, cosine_similarity\n",
    "from ast import literal_eval"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Regex pattern to match a URL\n",
    "HTTP_URL_PATTERN = r'^http[s]{0,1}://.+$'\n",
    "\n",
    "# Regex pattern to match a Phone number\n",
    "PHONE_PATTERN = r'^http[s]{0,1}://.+$'\n",
    "\n",
    "# Regex pattern to match an email\n",
    "EMAIL_PATTERN = r'^http[s]{0,1}://.+$'\n",
    "\n",
    "# Define OpenAI api_key\n",
    "# openai.api_key = '<Your API Key>'\n",
    "\n",
    "# Define root domain to crawl\n",
    "domain = \"oracle.com\"\n",
    "full_url = \"https://docs.oracle.com/en/cloud/saas/financials/24b/books.html\"\n",
    "\n",
    "# Create a class to parse the HTML and get the hyperlinks\n",
    "class HyperlinkParser(HTMLParser):\n",
    "    def __init__(self):\n",
    "        super().__init__()\n",
    "        # Create a list to store the hyperlinks\n",
    "        self.hyperlinks = []\n",
    "\n",
    "    # Override the HTMLParser's handle_starttag method to get the hyperlinks\n",
    "    def handle_starttag(self, tag, attrs):\n",
    "        attrs = dict(attrs)\n",
    "\n",
    "        # If the tag is an anchor tag and it has an href attribute, add the href attribute to the list of hyperlinks\n",
    "        if tag == \"a\" and \"href\" in attrs:\n",
    "            self.hyperlinks.append(attrs[\"href\"])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Function to get the hyperlinks from a URL\n",
    "def get_hyperlinks(url):\n",
    "    \n",
    "    # Try to open the URL and read the HTML\n",
    "    try:\n",
    "        # Open the URL and read the HTML\n",
    "        with urllib.request.urlopen(url) as response:\n",
    "\n",
    "            # If the response is not HTML, return an empty list\n",
    "            if not response.info().get('Content-Type').startswith(\"text/html\"):\n",
    "                return []\n",
    "            \n",
    "            # Decode the HTML\n",
    "            html = response.read().decode('utf-8')\n",
    "    except Exception as e:\n",
    "        print(e)\n",
    "        return []\n",
    "\n",
    "    # Create the HTML Parser and then Parse the HTML to get hyperlinks\n",
    "    parser = HyperlinkParser()\n",
    "    parser.feed(html)\n",
    "\n",
    "    return parser.hyperlinks"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Function to get the hyperlinks from a URL that are within the same domain\n",
    "def get_domain_hyperlinks(local_domain, url):\n",
    "    clean_links = []\n",
    "    for link in set(get_hyperlinks(url)):\n",
    "        clean_link = None\n",
    "\n",
    "        # If the link is a URL, check if it is within the same domain\n",
    "        if re.search(HTTP_URL_PATTERN, link):\n",
    "            # Parse the URL and check if the domain is the same\n",
    "            url_obj = urlparse(link)\n",
    "            if url_obj.netloc == local_domain:\n",
    "                clean_link = link\n",
    "\n",
    "        # If the link is not a URL, check if it is a relative link\n",
    "        else:\n",
    "            if link.startswith(\"/\"):\n",
    "                link = link[1:]\n",
    "            elif (\n",
    "                link.startswith(\"#\")\n",
    "                or link.startswith(\"mailto:\")\n",
    "                or link.startswith(\"tel:\")\n",
    "            ):\n",
    "                continue\n",
    "            clean_link = \"https://\" + local_domain + \"/\" + link\n",
    "\n",
    "        if clean_link is not None:\n",
    "            if clean_link.endswith(\"/\"):\n",
    "                clean_link = clean_link[:-1]\n",
    "            clean_links.append(clean_link)\n",
    "\n",
    "    # Return the list of hyperlinks that are within the same domain\n",
    "    return list(set(clean_links))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def crawl(url):\n",
    "    # Parse the URL and get the domain\n",
    "    local_domain = urlparse(url).netloc\n",
    "\n",
    "    # Create a queue to store the URLs to crawl\n",
    "    queue = deque([url])\n",
    "\n",
    "    # Create a set to store the URLs that have already been seen (no duplicates)\n",
    "    seen = set([url])\n",
    "\n",
    "    # Create a directory to store the text files\n",
    "    if not os.path.exists(\"text/\"):\n",
    "            os.mkdir(\"text/\")\n",
    "\n",
    "    if not os.path.exists(\"text/\"+local_domain+\"/\"):\n",
    "            os.mkdir(\"text/\" + local_domain + \"/\")\n",
    "\n",
    "    # Create a directory to store the csv files\n",
    "    if not os.path.exists(\"processed\"):\n",
    "            os.mkdir(\"processed\")\n",
    "\n",
    "    # While the queue is not empty, continue crawling\n",
    "    while queue:\n",
    "\n",
    "        # Get the next URL from the queue\n",
    "        url = queue.pop()\n",
    "        print(url) # for debugging and to see the progress\n",
    "        \n",
    "        # Try extracting the text from the link, if failed proceed with the next item in the queue\n",
    "        try:\n",
    "            # Save text from the url to a <url>.txt file\n",
    "            with open('text/'+local_domain+'/'+url[8:].replace(\"/\", \"_\") + \".txt\", \"w\", encoding=\"UTF-8\") as f:\n",
    "\n",
    "                # Get the text from the URL using BeautifulSoup\n",
    "                soup = BeautifulSoup(requests.get(url).text, \"html.parser\")\n",
    "\n",
    "                # Get the text but remove the tags\n",
    "                text = soup.get_text()\n",
    "\n",
    "                # If the crawler gets to a page that requires JavaScript, it will stop the crawl\n",
    "                if (\"You need to enable JavaScript to run this app.\" in text):\n",
    "                    print(\"Unable to parse page \" + url + \" due to JavaScript being required\")\n",
    "            \n",
    "                # Otherwise, write the text to the file in the text directory\n",
    "                f.write(text)\n",
    "        except Exception as e:\n",
    "            print(\"Unable to parse page \" + url)\n",
    "\n",
    "        # Get the hyperlinks from the URL and add them to the queue\n",
    "        for link in get_domain_hyperlinks(local_domain, url):\n",
    "            if link not in seen:\n",
    "                queue.append(link)\n",
    "                seen.add(link)\n",
    "\n",
    "crawl(full_url)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 1.5: Scrapify: Crawling SPAs as screenshots\n",
    "\n",
    "Although previous steps works fine for static content website, it often fails to scrape data from dynamic and single page app (SPAs) webpages. In this case, we will convert these pages to screenshot.\n",
    "\n",
    "First step is to [download chrome web-driver](https://chromedriver.chromium.org/downloads). Please make sure, web-driver version matches with your chrome version.\n",
    "\n",
    "(Open Chrome -> Help -> About chrome -> check version).\n",
    "\n",
    "download appropriate version depending on machine OS and unzip/extract to a local folder."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# !pip install Pillow selenium"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from selenium import webdriver\n",
    "from PIL import Image\n",
    "\n",
    "# Define the URL of the web page we want to screenshot\n",
    "\n",
    "url='/https://finance.yahoo.com/quote/AAPL?p=AAPL&.tsrc=fin-srch'\n",
    "\n",
    "# Define the path to the webdriver executable (e.g., chromedriver.exe)\n",
    "\n",
    "# webdriver_path = '/path/to/webdriver/executable'\n",
    "webdriver_path = r'C:\\amit.la\\WIP\\RPA\\downloads\\chromedriver.exe'\n",
    "\n",
    "# Set up the webdriver\n",
    "\n",
    "options = webdriver.ChromeOptions()\n",
    "options.headless = True # type: ignore # Run the browser in headless mode to prevent a window from popping up\n",
    "driver = webdriver.Chrome(options=options) # type: ignore\n",
    "\n",
    "# Load the web page\n",
    "\n",
    "driver.get(url)\n",
    "\n",
    "# Take a screenshot of the entire page\n",
    "\n",
    "# screenshot = driver.find_element_by_tag_name('body').screenshot_as_png\n",
    "screenshot = driver.save_screenshot('../downloads/screenshot.png')\n",
    "\n",
    "# Close the webdriver\n",
    "\n",
    "driver.quit()\n",
    "\n",
    "# Save the screenshot to a file\n",
    "\n",
    "# with open('../SampleData/screenshot.png', 'wb') as file:\n",
    "#     file.write(screenshot)\n",
    "\n",
    "# Open the screenshot with Pillow to display it (optional)\n",
    "\n",
    "img = Image.open('../downloads/screenshot.png')\n",
    "img.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "urls = {\n",
    "        \"AAPL.png\": \"https://finance.yahoo.com/quote/AAPL?p=AAPL&.tsrc=fin-srch\",\n",
    "        \"ORCL.png\": \"https://finance.yahoo.com/quote/ORCL?p=ORCL&.tsrc=fin-srch\",\n",
    "        \"TSLA.png\": \"https://finance.yahoo.com/quote/TSLA?p=TSLA&.tsrc=fin-srch\",\n",
    "        \"GOOG.png\": \"https://finance.yahoo.com/quote/GOOG?p=GOOG&.tsrc=fin-srch\",\n",
    "        \"MSFT.png\": \"https://finance.yahoo.com/quote/MSFT?p=MSFT&.tsrc=fin-srch\"\n",
    "    }"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def takeScreenshots(outputFileName, url):\n",
    "    driver.get(url)\n",
    "    driver.save_screenshot(os.path.join('../downloads/',outputFileName))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# take multiple screen shots\n",
    "# automate this script to autodownload data\n",
    "\n",
    "for key,value in urls.items():\n",
    "    takeScreenshots(key, value)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 1.6: Scrapify: read Image and query LLM\n",
    "\n",
    "In this step, we will finally learn to read data from images and build our knowledge base.\n",
    "\n",
    "To read text from images using Tesseract OCR in Python, we can use the pytesseract library, which is a Python wrapper for the Tesseract OCR engine. Here's an example code snippet:\n",
    "\n",
    "[download tesseract here](https://tesseract-ocr.github.io/tessdoc/#binaries)\n",
    "\n",
    "`Note that Tesseract OCR is not perfect and may not be able to extract text accurately from all images.`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# py -m pip install pytesseract PIL"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from PIL import Image\n",
    "img = Image.open('../downloads/AAPL.png')\n",
    "img.show()\n",
    "\n",
    "# make sure, you have tesseract included in your environment path\n",
    "\n",
    "import os\n",
    "os.getenv(\"tesseract\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pytesseract\n",
    "from PIL import Image\n",
    "\n",
    "##############################################################################\n",
    "# in case if tesseract is not included in PATH\n",
    "pytesseract.pytesseract.tesseract_cmd = r'C:\\amit.la\\WIP\\RPA\\downloads\\ts\\tesseract.exe'\n",
    "##############################################################################\n",
    "\n",
    "def read_image_text(image_path):\n",
    "    \"\"\"\n",
    "    Reads text from an image file using Tesseract OCR.\n",
    "\n",
    "    Args:\n",
    "        image_path (str): The file path to the input image.\n",
    "\n",
    "    Returns:\n",
    "        str: The extracted text from the image.\n",
    "    \"\"\"\n",
    "    # Load the image file\n",
    "    image = Image.open(image_path)\n",
    "\n",
    "    # Use Tesseract OCR to extract the text from the image\n",
    "    text = pytesseract.image_to_string(image)\n",
    "\n",
    "    return text\n",
    "\n",
    "# Example usage\n",
    "image_path = \"../downloads/APPL.png\"\n",
    "text = read_image_text(image_path)\n",
    "print(text)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "images = {\n",
    "        \"AAPL\": \"../downloads/AAPL.png\",\n",
    "        \"ORCL\": \"../downloads/ORCL.png\",\n",
    "        \"TSLA\": \"../downloads/TSLA.png\",\n",
    "        \"GOOG\": \"../downloads/GOOG.png\",\n",
    "        \"MSFT\": \"../downloads/MSFT.png\"\n",
    "    }\n",
    "\n",
    "# automate reading images and creating text from these images\n",
    "# you can further store these texts into a database\n",
    "\n",
    "for key,value in images.items():\n",
    "    # print(key, value)\n",
    "    text = read_image_text(value)\n",
    "    print(text)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 2: convert all PDFs | texts to one csv\n",
    "---\n",
    "In this step, We will convert all PDFs/text files to one csv and simply build a Q&A prompt using Gen AI (Claude) with entire file content at once. As one big csv can be too much data to input, we will split big csv into smaller csvs."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# remove new lines function converts all new line and tab chars to space\n",
    "def remove_newlinechars(txt):\n",
    "    txt = txt.str.replace('\\n', ' ')\n",
    "    txt = txt.str.replace('\\\\n', ' ')\n",
    "    txt = txt.str.replace('  ', ' ')\n",
    "    txt = txt.str.replace('  ', ' ')\n",
    "    return txt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "# Create a list to store the text files\n",
    "texts=[]\n",
    "\n",
    "# Get all the text files in the text directory\n",
    "for file in os.listdir(\"../downloads/texts/\"):\n",
    "\n",
    "    # Open the file and read the text\n",
    "    with open(\"../downloads/texts/\" + file, \"r\", encoding=\"UTF-8\") as f:\n",
    "        text = f.read()\n",
    "\n",
    "        # Omit the first 11 lines and the last 4 lines, then replace -, _, and #update with spaces.\n",
    "        texts.append((file[11:-4].replace('-',' ').replace('_', ' ').replace('#update',''), text))\n",
    "\n",
    "# Create a dataframe from the list of texts\n",
    "df = pd.DataFrame(texts, columns = ['fname', 'text'])\n",
    "\n",
    "# Set the text column to be the raw text with the newlines removed\n",
    "df['text'] = df.fname + \". \" + remove_newlinechars(df.text)\n",
    "df.to_csv('../downloads/scraped.csv')\n",
    "df.head()\n",
    "\n",
    "# as you can see, we created one row in csv per txt file \n",
    "# i.e. chapter 3, 4 and 5 each has one row in csv"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "os.listdir(\"../downloads/\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 3: Creating Embedding from csvs and other documents to create a Vector database.\n",
    "---\n",
    "\n",
    "Most of the API limit number of input tokens for embeddings. In this step, we will split rows into tokens."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Step 3.1: Creating tokens from text"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# !pip install tiktoken"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "########################\n",
    "# visualize text tokens\n",
    "########################\n",
    "import pandas as pd\n",
    "import tiktoken\n",
    "\n",
    "# Load the cl100k_base tokenizer which is designed to work with the ada-002 model\n",
    "tokenizer = tiktoken.get_encoding(\"cl100k_base\")\n",
    "\n",
    "df = pd.read_csv('../downloads/scraped.csv', index_col=0)\n",
    "df.columns = ['title', 'text']\n",
    "\n",
    "# Tokenize the text and save the number of tokens to a new column\n",
    "df['n_tokens'] = df.text.apply(lambda x: len(tokenizer.encode(x)))\n",
    "\n",
    "print(df)\n",
    "# Visualize the distribution of the number of tokens per row using a histogram\n",
    "df.n_tokens.hist()\n",
    "\n",
    "# as you can see in below results\n",
    "# Chapter 3, 4 & 5 has appx 4.2, 8 and 5.7k tokens\n",
    "# this is expectecd, as Chapter as longest text\n",
    "\n",
    "# now, we will need to further split these rows based on number of tokens\n",
    "# because most of the vector databases have upper limits of # of tokens that can be stored"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Step 3.2: Creating equal tokens from text"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#####################################################################\n",
    "# let's say we want to split csv into chunks of 500 tokens\n",
    "#####################################################################\n",
    "\n",
    "max_tokens = 500\n",
    "\n",
    "# Function to split the text into chunks of a maximum number of tokens\n",
    "def split_into_many(text, max_tokens = max_tokens):\n",
    "\n",
    "    # Split the text into sentences\n",
    "    sentences = text.split('. ')\n",
    "\n",
    "    # Get the number of tokens for each sentence\n",
    "    n_tokens = [len(tokenizer.encode(\" \" + sentence)) for sentence in sentences]\n",
    "\n",
    "    chunks = []\n",
    "    tokens_so_far = 0\n",
    "    chunk = []\n",
    "\n",
    "    # Loop through the sentences and tokens joined together in a tuple\n",
    "    for sentence, token in zip(sentences, n_tokens):\n",
    "\n",
    "        # If the number of tokens so far plus the number of tokens in the current sentence is greater\n",
    "        # than the max number of tokens, then add the chunk to the list of chunks and reset\n",
    "        # the chunk and tokens so far\n",
    "        if tokens_so_far + token > max_tokens:\n",
    "            chunks.append(\". \".join(chunk) + \".\")\n",
    "            chunk = []\n",
    "            tokens_so_far = 0\n",
    "\n",
    "        # If the number of tokens in the current sentence is greater than the max number of\n",
    "        # tokens, go to the next sentence\n",
    "        if token > max_tokens:\n",
    "            continue\n",
    "\n",
    "        # Otherwise, add the sentence to the chunk and add the number of tokens to the total\n",
    "        chunk.append(sentence)\n",
    "        tokens_so_far += token + 1\n",
    "\n",
    "    return chunks\n",
    "\n",
    "\n",
    "shortened = []\n",
    "\n",
    "# Loop through the dataframe\n",
    "for row in df.iterrows():\n",
    "\n",
    "    # If the text is None, go to the next row\n",
    "    if row[1]['text'] is None:\n",
    "        continue\n",
    "\n",
    "    # If the number of tokens is greater than the max number of tokens, split the text into chunks\n",
    "    if row[1]['n_tokens'] > max_tokens:\n",
    "        shortened += split_into_many(row[1]['text'])\n",
    "\n",
    "    # Otherwise, add the text to the list of shortened texts\n",
    "    else:\n",
    "        shortened.append( row[1]['text'] )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#####################################################################\n",
    "# Visualizing the updated histogram again can help to confirm\n",
    "# if the rows were successfully split into shortened sections.\n",
    "#####################################################################\n",
    "\n",
    "df = pd.DataFrame(shortened, columns = ['text'])\n",
    "df['n_tokens'] = df.text.apply(lambda x: len(tokenizer.encode(x)))\n",
    "print(df.head())\n",
    "print(df.shape) # we had appx 18k tokens earlier, so we should expect ~35+ distributions of 500 each\n",
    "df.n_tokens.hist()\n",
    "\n",
    "#####################################################################\n",
    "# as you can see from histogram\n",
    "# most of rows have about 450-500 tokens each"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Step 3.3: Creating ChromaDB #trychroma vector DB"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# !pip install pysqlite3-binary\n",
    "# !pip show chromadb\n",
    "# version 0.3.29\n",
    "# there might be issues due to sqlite library, if so,\n",
    "# replace sqlite with pysqlite3-binary"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# if you wish to just experiment without have chromadb as persistent DB, \n",
    "# run this code and ignore next cell\n",
    "\n",
    "# store sample documents in chromadb\n",
    "\n",
    "# import ollama\n",
    "# __import__('pysqlite3')\n",
    "# import sys\n",
    "# sys.modules['sqlite3'] = sys.modules.pop('pysqlite3')\n",
    "# import chromadb\n",
    "# client = chromadb.Client() # will create only a temp, non-persistent db"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# running prod chromadb as persistent db\n",
    "# store tokens|docs in a persisten db so that you don't need to store vectors everytime\n",
    "\n",
    "# first make sure chromadb in installed, if not\n",
    "# !pip install chromadb\n",
    "\n",
    "# do not run this here, instead run this command on terminal\n",
    "# !chroma run --host localhost --port 8080 --path ./OLVectorDB\n",
    "\n",
    "# if you see errors, add below code to your ___init__.py file\n",
    "# import ollama\n",
    "# __import__('pysqlite3')\n",
    "# import sys\n",
    "# sys.modules['sqlite3'] = sys.modules.pop('pysqlite3')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# store sample documents in chromadb\n",
    "\n",
    "import ollama\n",
    "__import__('pysqlite3')\n",
    "import sys\n",
    "sys.modules['sqlite3'] = sys.modules.pop('pysqlite3')\n",
    "import chromadb\n",
    "# use this client version to start a non-persistent experimental chromadb instance\n",
    "# client = chromadb.Client() # will create only a temp, non-persistent db\n",
    "\n",
    "# use this client version to start persistent chromadb instance\n",
    "# from chromadb.config import DEFAULT_TENANT, DEFAULT_DATABASE, Settings\n",
    "# client = chromadb.PersistentClient(\n",
    "#     path=\"./vectordb\",\n",
    "#     settings=Settings(),\n",
    "#     tenant=DEFAULT_TENANT,\n",
    "#     database=\"OL\",\n",
    "# )\n",
    "\n",
    "client = chromadb.PersistentClient(path=\"./OLVectorDB\")\n",
    "\n",
    "# use this client version to start chromadb http instance\n",
    "# client = chromadb.HttpClient(\n",
    "#     host=\"localhost\",\n",
    "#     port=8080,\n",
    "#     ssl=False,\n",
    "#     headers=None,\n",
    "#     # settings=Settings(),\n",
    "#     # tenant=DEFAULT_TENANT,\n",
    "#     database=\"./vectordb\",\n",
    "# )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "client.list_collections()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# uncomment this code,\n",
    "# if collection is already created, do not create it over and over again\n",
    "\n",
    "documents = df[\"text\"].to_list()\n",
    "\n",
    "# collection = client.create_collection(name=\"docs\")\n",
    "# # store each document in a vector embedding database\n",
    "# for i, d in enumerate(documents):\n",
    "#   response = ollama.embeddings(model=\"mxbai-embed-large\", prompt=d)\n",
    "#   embedding = response[\"embedding\"]\n",
    "#   collection.add(\n",
    "#     ids=[str(i)],\n",
    "#     embeddings=[embedding],\n",
    "#     documents=[d]\n",
    "#   )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Step 3.4: Creating SQLLite vector DB"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# !pip install sqlite_vec"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# this code is copied form sqlite-vec GitRepo\n",
    "import sqlite3\n",
    "import sqlite_vec\n",
    "\n",
    "from typing import List\n",
    "import struct\n",
    "\n",
    "def serialize_f32(vector: List[float]) -> bytes:\n",
    "    \"\"\"serializes a list of floats into a compact \"raw bytes\" format\"\"\"\n",
    "    return struct.pack(\"%sf\" % len(vector), *vector)\n",
    "\n",
    "db = sqlite3.connect(\":memory:\")\n",
    "db.enable_load_extension(True)\n",
    "sqlite_vec.load(db)\n",
    "db.enable_load_extension(False)\n",
    "\n",
    "sqlite_version, vec_version = db.execute(\n",
    "    \"select sqlite_version(), vec_version()\"\n",
    ").fetchone()\n",
    "print(f\"sqlite_version={sqlite_version}, vec_version={vec_version}\")\n",
    "\n",
    "items = [\n",
    "    (1, [0.1, 0.1, 0.1, 0.1]),\n",
    "    (2, [0.2, 0.2, 0.2, 0.2]),\n",
    "    (3, [0.3, 0.3, 0.3, 0.3]),\n",
    "    (4, [0.4, 0.4, 0.4, 0.4]),\n",
    "    (5, [0.5, 0.5, 0.5, 0.5]),\n",
    "]\n",
    "query = [0.3, 0.3, 0.3, 0.3]\n",
    "\n",
    "db.execute(\"CREATE VIRTUAL TABLE vec_items USING vec0(embedding float[4])\")\n",
    "\n",
    "with db:\n",
    "    for item in items:\n",
    "        db.execute(\n",
    "            \"INSERT INTO vec_items(rowid, embedding) VALUES (?, ?)\",\n",
    "            [item[0], serialize_f32(item[1])],\n",
    "        )\n",
    "\n",
    "rows = db.execute(\n",
    "    \"\"\"\n",
    "      SELECT\n",
    "        rowid,\n",
    "        distance\n",
    "      FROM vec_items\n",
    "      WHERE embedding MATCH ?\n",
    "      and k=3\n",
    "      ORDER BY distance\n",
    "    \"\"\",\n",
    "    [serialize_f32(query)],\n",
    ").fetchall()\n",
    "\n",
    "print(rows)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 4: Using RAG and LLMs to query manual documents.\n",
    "---\n",
    "\n",
    "Retrieval-Augmented Generation using trychroma ChromaDB\n",
    "\n",
    "query a embedding collection"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "collection = client.get_collection(name=\"docs\")\n",
    "# retreive data from vector store\n",
    "prompt = \"how are args and keyword arguments defined in python?\"\n",
    "\n",
    "# generate an embedding for the prompt and retrieve the most relevant doc\n",
    "response = ollama.embeddings(\n",
    "  prompt=prompt,\n",
    "  model=\"mxbai-embed-large\"\n",
    ")\n",
    "results = collection.query(\n",
    "  query_embeddings=[response[\"embedding\"]],\n",
    "  n_results=1\n",
    ")\n",
    "data = results['documents'][0][0]\n",
    "data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "response = ollama.chat(model='phi3:mini', messages=[\n",
    "  {\n",
    "    'role': 'user',\n",
    "    'content': prompt,\n",
    "  },\n",
    "])\n",
    "print(response['message']['content']) # type: ignore"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "output = ollama.generate(\n",
    "  model=\"llama3\",\n",
    "  prompt=f\"Using this data: {data}. Respond to this prompt: {prompt}\"\n",
    ")\n",
    "\n",
    "print(output['response']) # type: ignore"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 5: Using SQL queries | API with Tools | Functional calling\n",
    "---\n",
    "\n",
    "in later usecases, we will work with advance data topics and use function/tools calling extensively."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# !pip install ollama"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "89"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import requests\n",
    "\n",
    "def get_current_weather(city):\n",
    "  # https://api.weather.gov/gridpoints/TOP/32,81/forecast\n",
    "  # The API endpoint\n",
    "  url = \"https://api.weather.gov/gridpoints/TOP/32,81/forecast\"\n",
    "  response = requests.get(url)\n",
    "  return response.json()[\"properties\"][\"periods\"][0][\"temperature\"]\n",
    "\n",
    "get_current_weather(\"Los Angeles\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(1, 'Alice', 30)\n",
      "(2, 'Alice Wonder', 30)\n",
      "(3, 'Alice Wonder', 30)\n",
      "(4, 'Alice Wonder', 30)\n",
      "(5, 'Alice Wonder', 30)\n",
      "(6, 'Alice Wonder', 30)\n",
      "(7, 'Alice Wonder', 30)\n",
      "(8, 'Alice Wonder', 30)\n",
      "(9, 'Alice Wonder', 30)\n",
      "(10, 'Alice Wonder', 30)\n",
      "(11, 'Alice Wonder', 30)\n",
      "(12, 'Alice Wonder', 30)\n",
      "(13, 'Alice Wonder', 30)\n",
      "(14, 'Alice Wonder', 30)\n",
      "(15, 'Alice Wonder', 30)\n",
      "(16, 'Alice Wonder', 30)\n"
     ]
    }
   ],
   "source": [
    "import sqlite3\n",
    "\n",
    "# Connect to the database (or create it)\n",
    "conn = sqlite3.connect('example.db')\n",
    "cursor = conn.cursor()\n",
    "\n",
    "# Create a table\n",
    "cursor.execute('''\n",
    "CREATE TABLE IF NOT EXISTS users (\n",
    "    id INTEGER PRIMARY KEY,\n",
    "    name TEXT,\n",
    "    age INTEGER\n",
    ")\n",
    "''')\n",
    "conn.commit()\n",
    "\n",
    "# Insert a record\n",
    "cursor.execute('''\n",
    "INSERT INTO users (name, age) VALUES (?, ?)\n",
    "''', ('Alice Wonder', 30))\n",
    "conn.commit()\n",
    "\n",
    "# Retrieve records\n",
    "cursor.execute('SELECT * FROM users')\n",
    "rows = cursor.fetchall()\n",
    "for row in rows:\n",
    "    print(row)\n",
    "\n",
    "# Close the connection\n",
    "conn.close()\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "SELECT * FROM users where name = Alice\n",
      "[(1, 'Alice', 30)]\n"
     ]
    }
   ],
   "source": [
    "conn = sqlite3.connect('example.db')\n",
    "cursor = conn.cursor()\n",
    "def get_user(employee):\n",
    "  # Insert a record\n",
    "  # cursor.execute('''\n",
    "  # INSERT INTO users (name, age) VALUES (?, ?)\n",
    "  # ''', ('Alice Wonder', 30))\n",
    "  # conn.commit()\n",
    "  print(f\"SELECT * FROM users where name = {employee}\")\n",
    "  cursor.execute(f\"SELECT * FROM users where name = '{employee}'\")\n",
    "  rows = cursor.fetchall()\n",
    "  # for row in rows:\n",
    "  #   print(row)\n",
    "  return rows\n",
    "\n",
    "print(get_user(\"Alice\"))\n",
    "\n",
    "# Close the connection\n",
    "# conn.close()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "SELECT * FROM users where name = Alice\n",
      "[(1, 'Alice', 30)]\n"
     ]
    }
   ],
   "source": [
    "import ollama\n",
    "\n",
    "tools = [{\n",
    "      'type': 'function',\n",
    "      'function': {\n",
    "        'name': 'get_current_weather',\n",
    "        'description': 'Get the current weather for a city',\n",
    "        'parameters': {\n",
    "          'type': 'object',\n",
    "          'properties': {\n",
    "            'city': {\n",
    "              'type': 'string',\n",
    "              'description': 'The name of the city',\n",
    "            },\n",
    "          },\n",
    "          'required': ['city'],\n",
    "        },\n",
    "      },\n",
    "    },\n",
    "    {\n",
    "      'type': 'function',\n",
    "      'function': {\n",
    "        'name': 'get_user',\n",
    "        'description': 'Get the current age of employee',\n",
    "        'parameters': {\n",
    "          'type': 'object',\n",
    "          'properties': {\n",
    "            'employee': {\n",
    "              'type': 'string',\n",
    "              'description': 'The name of the employee',\n",
    "            },\n",
    "          },\n",
    "          'required': ['employee'],\n",
    "        },\n",
    "      },\n",
    "    },\n",
    "  ]\n",
    "\n",
    "# creating a generic function to call appropriate tool based on tool input\n",
    "def process_tool_call(tool_name, tool_input):\n",
    "    if tool_name == \"get_current_weather\":\n",
    "        return get_current_weather(tool_input[\"city\"])\n",
    "    if tool_name == \"get_user\":\n",
    "        return get_user(tool_input[\"employee\"])\n",
    "  \n",
    "# print(process_tool_call('get_current_weather', {'city': 'Los Angeles CA'}))\n",
    "print(process_tool_call('get_user', {'employee': 'Alice'}))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Initial Response:\n",
      "Tool called: {'function': {'name': 'get_user', 'arguments': {'employee': 'Alice'}}}\n",
      "Tool name: get_user\n",
      "Tool param: {'employee': 'Alice'}\n",
      "Stop Reason: stop\n",
      "Content: \n"
     ]
    }
   ],
   "source": [
    "response = ollama.chat(\n",
    "        model='llama3.2',\n",
    "        messages=[{'role': 'user', 'content': \n",
    "            'how old is Alice?'}],\n",
    "\t\t    # provide a weather checking tool to the model\n",
    "        tools=tools # type: ignore\n",
    "    )\n",
    "\n",
    "# response \n",
    "print(f\"\\nInitial Response:\")\n",
    "print(f\"Tool called: {response[\"message\"][\"tool_calls\"][0]}\")\n",
    "print(f\"Tool name: {response[\"message\"][\"tool_calls\"][0][\"function\"][\"name\"]}\")\n",
    "print(f\"Tool param: {response[\"message\"][\"tool_calls\"][0][\"function\"][\"arguments\"]}\")\n",
    "print(f\"Stop Reason: {response[\"done_reason\"]}\")\n",
    "print(f\"Content: {response[\"message\"][\"content\"]}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Initial Response:\n",
      "Tool called: {'function': {'name': 'get_current_weather', 'arguments': {'city': 'Los Angeles, CA'}}}\n",
      "Tool name: get_current_weather\n",
      "Tool param: {'city': 'Los Angeles, CA'}\n",
      "Stop Reason: stop\n",
      "Content: \n"
     ]
    }
   ],
   "source": [
    "response = ollama.chat(\n",
    "        model='llama3.2',\n",
    "        messages=[{'role': 'user', 'content': \n",
    "            'What is the weather in Los Angeles CA today?'}],\n",
    "\t\t    # provide a weather checking tool to the model\n",
    "        tools=tools # type: ignore\n",
    "    )\n",
    "\n",
    "# response \n",
    "print(f\"\\nInitial Response:\")\n",
    "print(f\"Tool called: {response[\"message\"][\"tool_calls\"][0]}\")\n",
    "print(f\"Tool name: {response[\"message\"][\"tool_calls\"][0][\"function\"][\"name\"]}\")\n",
    "print(f\"Tool param: {response[\"message\"][\"tool_calls\"][0][\"function\"][\"arguments\"]}\")\n",
    "print(f\"Stop Reason: {response[\"done_reason\"]}\")\n",
    "print(f\"Content: {response[\"message\"][\"content\"]}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [],
   "source": [
    "def chatBot(user_message):\n",
    "    print(f\"\\n{'='*50}\\nUser Message: {user_message}\\n{'='*50}\")\n",
    "    response = ollama.chat(\n",
    "        model='llama3.2',\n",
    "        messages=[{'role': 'user', 'content': user_message}],\n",
    "\t\t    # provide a weather checking tool to the model\n",
    "        tools=tools # type: ignore\n",
    "    )\n",
    "    print(f\"\\nInitial Response:\")\n",
    "    print(f\"Tool called: {response[\"message\"][\"tool_calls\"][0]}\")\n",
    "    print(f\"Stop Reason: {response[\"done_reason\"]}\")\n",
    "    print(f\"Content: {response[\"message\"][\"content\"]}\")\n",
    "\n",
    "    if response[\"done_reason\"] == \"stop\":\n",
    "        # tool_use = next(block for block in response.content if block.type == \"tool_use\")\n",
    "        tool_name = response[\"message\"][\"tool_calls\"][0][\"function\"][\"name\"]\n",
    "        tool_input = response[\"message\"][\"tool_calls\"][0][\"function\"][\"arguments\"]\n",
    "        tool_content = response[\"message\"][\"content\"]\n",
    "\n",
    "        tool_result = process_tool_call(tool_name, tool_input)\n",
    "        print(f\"Tool Result: {tool_result}\")\n",
    "\n",
    "        response = ollama.chat(\n",
    "                model='llama3.2',\n",
    "                messages=[\n",
    "                    {\"role\": \"user\", \"content\": user_message},\n",
    "                    # {\"role\": \"assistant\", \"content\": f\"as per results from tools API, current data is {str(tool_result)} , based on this data, please answer this {user_message}.\"},\n",
    "                    {\n",
    "                    \"role\": \"tool\",\n",
    "                    \"content\": str(tool_result) # type: ignore\n",
    "                    },\n",
    "                ],\n",
    "                tools=tools # type: ignore\n",
    "                )\n",
    "        print(response)\n",
    "    return response"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "==================================================\n",
      "User Message: How is the weather in San Francisco today?\n",
      "==================================================\n",
      "\n",
      "Initial Response:\n",
      "Tool called: {'function': {'name': 'get_current_weather', 'arguments': {'city': 'San Francisco'}}}\n",
      "Stop Reason: stop\n",
      "Content: \n",
      "Tool Result: 54\n",
      "{'model': 'llama3.2', 'created_at': '2024-10-03T05:37:55.7531787Z', 'message': {'role': 'assistant', 'content': 'The current temperature in San Francisco is 54°F. Would you like to know more about the weather forecast for San Francisco or any specific time of day?'}, 'done_reason': 'stop', 'done': True, 'total_duration': 6176065000, 'load_duration': 4932965100, 'prompt_eval_count': 77, 'prompt_eval_duration': 373857000, 'eval_count': 32, 'eval_duration': 866191000}\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "{'model': 'llama3.2',\n",
       " 'created_at': '2024-10-03T05:37:55.7531787Z',\n",
       " 'message': {'role': 'assistant',\n",
       "  'content': 'The current temperature in San Francisco is 54°F. Would you like to know more about the weather forecast for San Francisco or any specific time of day?'},\n",
       " 'done_reason': 'stop',\n",
       " 'done': True,\n",
       " 'total_duration': 6176065000,\n",
       " 'load_duration': 4932965100,\n",
       " 'prompt_eval_count': 77,\n",
       " 'prompt_eval_duration': 373857000,\n",
       " 'eval_count': 32,\n",
       " 'eval_duration': 866191000}"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "chatBot(\"How is the weather in San Francisco today?\")\n",
    "# chatBot(\"How old is my employee name Alice?\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 6: build, host online UI app with llama3.2\n",
    "## ollama ChromaDB SQLite RAG Q&A\n",
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}