notebooks/official/tabnet/ai-explanations-tabnet-algorithm.ipynb

{
  "cells": [
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "ur8xi4C7S06n"
      },
      "outputs": [],
      "source": [
        "# Copyright 2022 Google LLC\n",
        "#\n",
        "# Licensed under the Apache License, Version 2.0 (the \"License\");\n",
        "# you may not use this file except in compliance with the License.\n",
        "# You may obtain a copy of the License at\n",
        "#\n",
        "#     https://www.apache.org/licenses/LICENSE-2.0\n",
        "#\n",
        "# Unless required by applicable law or agreed to in writing, software\n",
        "# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
        "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
        "# See the License for the specific language governing permissions and\n",
        "# limitations under the License."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "JAPoU8Sm5E6e"
      },
      "source": [
        "# Vertex AI Explanations with TabNet models\n",
        "\n",
        "<table align=\"left\">\n",
        "\n",
        "  <td style=\"text-align: center\">\n",
        "    <a href=\"https://colab.research.google.com/github/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/tabnet/ai-explanations-tabnet-algorithm.ipynb\">\n",
        "      <img src=\"https://cloud.google.com/ml-engine/images/colab-logo-32px.png\" alt=\"Colab logo\"><br> Run in Colab\n",
        "    </a>\n",
        "  </td>\n",
        "  <td style=\"text-align: center\">\n",
        "    <a href=\"https://console.cloud.google.com/vertex-ai/colab/import/https:%2F%2Fraw.githubusercontent.com%2FGoogleCloudPlatform%2Fvertex-ai-samples%2Fmain%2Fnotebooks%2Fofficial%2Ftabnet%2Fai-explanations-tabnet-algorithm.ipynb\">\n",
        "      <img width=\"32px\" src=\"https://cloud.google.com/ml-engine/images/colab-enterprise-logo-32px.png\" alt=\"Google Cloud Colab Enterprise logo\"><br> Open in Colab Enterprise\n",
        "    </a>\n",
        "  </td>\n",
        "  <td style=\"text-align: center\">\n",
        "    <a href=\"https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/tabnet/ai-explanations-tabnet-algorithm.ipynb\">\n",
        "      <img src=\"https://cloud.google.com/ml-engine/images/github-logo-32px.png\" alt=\"GitHub logo\"><br>\n",
        "      View on GitHub\n",
        "    </a>\n",
        "  </td>\n",
        "  <td style=\"text-align: center\">\n",
        "    <a href=\"https://console.cloud.google.com/vertex-ai/workbench/deploy-notebook?download_url=https://raw.githubusercontent.com/GoogleCloudPlatform/vertex-ai-samples/main/notebooks/official/tabnet/ai-explanations-tabnet-algorithm.ipynb\">\n",
        "      <img src=\"https://lh3.googleusercontent.com/UiNooY4LUgW_oTvpsNhPpQzsstV5W8F7rYgxgGBD85cWJoLmrOzhVs_ksK_vgx40SHs7jCqkTkCk=e14-rj-sc0xffffff-h130-w32\" alt=\"Vertex AI logo\"><br>\n",
        "      Open in Vertex AI Workbench\n",
        "    </a>\n",
        "  </td>                                                                                               \n",
        "</table>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ce49f48f9bb2"
      },
      "source": [
        "## Overview\n",
        "\n",
        "Vertex AI provides a algorithm called on [TabNet](https://arxiv.org/abs/1908.07442). TabNet is an interpretable deep learning architecture for tabular (structured) data, the most common data type among enterprises. TabNet combines the best of two worlds: it's explainable, like simpler tree-based models, and can achieve the high accuracy of complex black-box models and ensembles, meaning it's precise without obscuring how the model works. This makes TabNet well-suited for a wide range of tabular data tasks where model explainability is just as important as accuracy.\n",
        "\n",
        "The goal of the tutorial is to provide a sample plotting tool to visualize the output of TabNet, which is helpful in explaining the algorithm.\n",
        "\n",
        "Learn more about [Tabular Workflow for TabNet](https://cloud.google.com/vertex-ai/docs/tabular-data/tabular-workflows/tabnet)."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "tvgnzT1CKxrO"
      },
      "source": [
        "### Objective\n",
        "\n",
        "In this tutorial, you learn how to provide a sample plotting tool to visualize the output of TabNet, which is helpful in explaining the algorithm. \n",
        "\n",
        "\n",
        "\n",
        "This tutorial uses the following Google Cloud ML services and resources:\n",
        "\n",
        "- Vertex Explainable AI\n",
        "- TabNet builtin algorithm\n",
        "\n",
        "The steps performed are:\n",
        "* Setup the project.\n",
        "* Download the prediction data of pretrain model on Syn2 data.\n",
        "* Visualize and understand the feature importance based on the masks output.\n",
        "* Clean up the resource created by this tutorial."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "3d631370325e"
      },
      "source": [
        "### Dataset\n",
        "\n",
        "This tutorial uses Synthetic_2 (Syn2) data, described in Section 4.1 of the [Learning to Explain](https://arxiv.org/pdf/1802.07814.pdf) paper. The input feature X is generated from a 10-dimensional standard Gaussian. The response variable Y is generated from feature X[3:6] only. The data has been split into training and prediction sets and has been uploaded to Google Cloud Storage:\n",
        "* Training data: gs://cloud-samples-data/ai-platform-unified/datasets/tabnet/tab_net_input/syn2_train.csv.\n",
        "* Prediction output data: gs://cloud-samples-data/ai-platform-unified/datasets/tabnet/tab_net_output/syn2\n",
        "\n",
        "At this time, the TabNet pre-trained model file is not publicly available."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5e2eba58ad71"
      },
      "source": [
        "### Costs \n",
        "\n",
        "This tutorial uses billable components of Google Cloud:\n",
        "\n",
        "* Vertex AI\n",
        "* Cloud Storage\n",
        "\n",
        "Learn about [Vertex AI pricing](https://cloud.google.com/vertex-ai/pricing) and [Cloud Storage pricing](https://cloud.google.com/storage/pricing), and use the [Pricing Calculator](https://cloud.google.com/products/calculator/) to generate a cost estimate based on your projected usage."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "f0316df526f8"
      },
      "source": [
        "## Get started"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "xOMNWzTbftDr"
      },
      "source": [
        "### Install Vertex AI SDK for Python and other required packages"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "9eb4e79842bf"
      },
      "outputs": [],
      "source": [
        "! pip3 install --upgrade --quiet google-cloud-aiplatform tensorflow"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ff555b32bab8"
      },
      "source": [
        "### Restart runtime (Colab only)\n",
        "\n",
        "To use the newly installed packages, you must restart the runtime on Google Colab."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "f09b4dff629a"
      },
      "outputs": [],
      "source": [
        "import sys\n",
        "\n",
        "if \"google.colab\" in sys.modules:\n",
        "\n",
        "    import IPython\n",
        "\n",
        "    app = IPython.Application.instance()\n",
        "    app.kernel.do_shutdown(True)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "4a2b7b59bbf7"
      },
      "source": [
        "<div class=\"alert alert-block alert-warning\">\n",
        "<b>⚠️ The kernel is going to restart. Wait until it's finished before continuing to the next step. ⚠️</b>\n",
        "</div>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "restart"
      },
      "source": [
        "### Authenticate your notebook environment (Colab only)\n",
        "\n",
        "Authenticate your environment on Google Colab."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "bzPxhxS5lugp"
      },
      "outputs": [],
      "source": [
        "import sys\n",
        "\n",
        "if \"google.colab\" in sys.modules:\n",
        "\n",
        "    from google.colab import auth\n",
        "\n",
        "    auth.authenticate_user()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "d2qpIurSjmpT"
      },
      "source": [
        "### Set Google Cloud project information\n",
        "\n",
        "Learn more about [setting up a project and a development environment](https://cloud.google.com/vertex-ai/docs/start/cloud-environment)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "wsePm9c4jmpT"
      },
      "outputs": [],
      "source": [
        "PROJECT_ID = \"[your-project-id]\"  # @param {type:\"string\"}"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "a54f9d7c1876"
      },
      "source": [
        "#### Location\n",
        "\n",
        "You can also change the `LOCATION` variable used by Vertex AI. Learn more about [Vertex AI locations](https://cloud.google.com/vertex-ai/docs/general/locations)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "3aaadaaf9b30"
      },
      "outputs": [],
      "source": [
        "LOCATION = \"us-central1\"  # @param {type: \"string\"}"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "e663bd062c6f"
      },
      "source": [
        "#### UUID\n",
        "\n",
        "If you are in a live tutorial session, you might be using a shared test account or project. To avoid name collisions between users on resources created, you create a uuid for each instance session, and append it onto the name of resources you create in this tutorial."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "953fa6e5ddda"
      },
      "outputs": [],
      "source": [
        "import random\n",
        "import string\n",
        "\n",
        "\n",
        "# Generate a uuid of a specifed length(default=8)\n",
        "def generate_uuid(length: int = 8) -> str:\n",
        "    return \"\".join(random.choices(string.ascii_lowercase + string.digits, k=length))\n",
        "\n",
        "\n",
        "UUID = generate_uuid()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "bucket:custom"
      },
      "source": [
        "### Create a Cloud Storage bucket\n",
        "\n",
        "Create a storage bucket to store intermediate artifacts such as datasets."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "bucket"
      },
      "outputs": [],
      "source": [
        "BUCKET_URI = f\"gs://your-bucket-name-{PROJECT_ID}-unique\"  # @param {type:\"string\"}"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "create_bucket"
      },
      "source": [
        "**If your bucket doesn't already exist**: Run the following cell to create your Cloud Storage bucket."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "Oz8J0vmSlugt"
      },
      "outputs": [],
      "source": [
        "! gsutil mb -l $LOCATION -p $PROJECT_ID $BUCKET_URI"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "XoEqT2Y4DJmf"
      },
      "source": [
        "### Import libraries and define constants"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "pRUOFELefqf1"
      },
      "outputs": [],
      "source": [
        "import json\n",
        "\n",
        "import matplotlib.cm as cm\n",
        "import matplotlib.pyplot as plt\n",
        "import numpy as np\n",
        "from google.cloud import storage\n",
        "\n",
        "%matplotlib inline"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "06f6abd8c40e"
      },
      "source": [
        "## Reading a sample TabNet prediction on syn2 data\n",
        "\n",
        "After training and serving your model, you upload the output to Google Cloud Storage. \n",
        "\n",
        "Sample prediction data is stored on Google Cloud at gs://cloud-samples-data/ai-platform-unified/datasets/tabnet/tab_net_output/syn2. You can use your own set of prediction data, but you must ensure that the format of the prediction data is the same as the format of the training data.\n",
        "\n",
        "Each prediction in TabNet contains a mask that is used to explain the predictions. The mask is stored in an **aggregated_mask_values** field.\n",
        "\n",
        "Information about the training set and the model are better suited for the Dataset section at the previous section of the notebook.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "0f55ca32f9b3"
      },
      "outputs": [],
      "source": [
        "!gsutil cp gs://cloud-samples-data/ai-platform-unified/datasets/tabnet/tab_net_output/syn2 $BUCKET_URI\n",
        "\n",
        "# Replace your the BUCKET_URI and PREDICTION_FILE\n",
        "# BUCKET_NAME = \"[<your-bucket-name>]\"\n",
        "# PREDICTION_FILE = \"[<your-prediction-file>]\"\n",
        "\n",
        "BUCKET_NAME = BUCKET_URI[5:]\n",
        "PREDICTION_FILE = \"syn2\"\n",
        "\n",
        "MASK_KEY = \"aggregated_mask_values\"\n",
        "\n",
        "HEADER = [(\"feat_\" + str(i)) for i in range(1, 12)]\n",
        "HEADER"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "c4782ae3d628"
      },
      "source": [
        "### Download and preprocess the predictions."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "4aeaa7143830"
      },
      "outputs": [],
      "source": [
        "storage_client = storage.Client()\n",
        "bucket = storage_client.get_bucket(BUCKET_NAME)\n",
        "blob = bucket.blob(PREDICTION_FILE)\n",
        "f = blob.download_as_string(client=None).decode(\"utf-8\").strip()\n",
        "predictions = f.split(\"\\n\")\n",
        "predictions[:1]"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "3f9f6b27280a"
      },
      "source": [
        "## Parse the mask values in prediction. Then, concatenate the mask values.\n",
        "The output is a matrix having Nxk (N is the number of outputs, k is the size of each mask). Concatenating mask values are used to visualize the feature importance."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "7f5b5c11f3dc"
      },
      "outputs": [],
      "source": [
        "masks = []\n",
        "for prediction in predictions:\n",
        "    prediction = json.loads(prediction)\n",
        "    masks.append(prediction[MASK_KEY])\n",
        "masks = np.matrix(masks)\n",
        "masks.shape"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8455a22d412c"
      },
      "source": [
        "## Visualize the mask value matrix.\n",
        "The lighter color indicates more important feature. For example, only features 3-6 are meaningful in prediction output in Syn2 data. In the plot, the column 3-6 have light color."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "ab63558c2593"
      },
      "outputs": [],
      "source": [
        "fig = plt.figure(figsize=(20, 10))\n",
        "ax = fig.add_subplot(121)\n",
        "ax.imshow(masks[:50, :], interpolation=\"bilinear\", cmap=cm.Greys_r)\n",
        "ax.set_xlabel(\"Features\")\n",
        "ax.set_ylabel(\"Sample index\")\n",
        "ax.xaxis.set_ticks(np.arange(len(HEADER)))\n",
        "ax.set_xticklabels(HEADER, rotation=\"vertical\")\n",
        "plt.show()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "TpV-iwP9qw9c"
      },
      "source": [
        "## Cleaning up\n",
        "\n",
        "To clean up all Google Cloud resources used in this project, you can [delete the Google Cloud\n",
        "project](https://cloud.google.com/resource-manager/docs/creating-managing-projects#shutting_down_projects) you used for the tutorial.\n",
        "\n",
        "Otherwise, you can delete the individual resources you created in this tutorial."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "sx_vKniMq9ZX"
      },
      "outputs": [],
      "source": [
        "# Delete Cloud Storage that were created\n",
        "! gsutil -m rm -r $BUCKET_URI"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "cb0f06c7ba0d"
      },
      "source": [
        "## What's next?\n",
        "\n",
        "To learn more about TabNet, check out the resources here.\n",
        "\n",
        "* [TabNet: Attentive Interpretable Tabular Learning](https://arxiv.org/abs/1908.07442)"
      ]
    }
  ],
  "metadata": {
    "colab": {
      "collapsed_sections": [],
      "name": "ai-explanations-tabnet-algorithm.ipynb",
      "toc_visible": true
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}