Drug_Clincial_Trial_Data_Assessing.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "provenance": [],
      "gpuType": "T4"
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "code",
      "source": [
        "!git\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "NvmoYfxbpdet",
        "outputId": "f2c15d96-11f9-49a6-fa8f-cd1ff53a76d0"
      },
      "execution_count": 194,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "usage: git [--version] [--help] [-C <path>] [-c <name>=<value>]\n",
            "           [--exec-path[=<path>]] [--html-path] [--man-path] [--info-path]\n",
            "           [-p | --paginate | -P | --no-pager] [--no-replace-objects] [--bare]\n",
            "           [--git-dir=<path>] [--work-tree=<path>] [--namespace=<name>]\n",
            "           [--super-prefix=<path>] [--config-env=<name>=<envvar>]\n",
            "           <command> [<args>]\n",
            "\n",
            "These are common Git commands used in various situations:\n",
            "\n",
            "start a working area (see also: git help tutorial)\n",
            "   clone     Clone a repository into a new directory\n",
            "   init      Create an empty Git repository or reinitialize an existing one\n",
            "\n",
            "work on the current change (see also: git help everyday)\n",
            "   add       Add file contents to the index\n",
            "   mv        Move or rename a file, a directory, or a symlink\n",
            "   restore   Restore working tree files\n",
            "   rm        Remove files from the working tree and from the index\n",
            "\n",
            "examine the history and state (see also: git help revisions)\n",
            "   bisect    Use binary search to find the commit that introduced a bug\n",
            "   diff      Show changes between commits, commit and working tree, etc\n",
            "   grep      Print lines matching a pattern\n",
            "   log       Show commit logs\n",
            "   show      Show various types of objects\n",
            "   status    Show the working tree status\n",
            "\n",
            "grow, mark and tweak your common history\n",
            "   branch    List, create, or delete branches\n",
            "   commit    Record changes to the repository\n",
            "   merge     Join two or more development histories together\n",
            "   rebase    Reapply commits on top of another base tip\n",
            "   reset     Reset current HEAD to the specified state\n",
            "   switch    Switch branches\n",
            "   tag       Create, list, delete or verify a tag object signed with GPG\n",
            "\n",
            "collaborate (see also: git help workflows)\n",
            "   fetch     Download objects and refs from another repository\n",
            "   pull      Fetch from and integrate with another repository or a local branch\n",
            "   push      Update remote refs along with associated objects\n",
            "\n",
            "'git help -a' and 'git help -g' list available subcommands and some\n",
            "concept guides. See 'git help <command>' or 'git help <concept>'\n",
            "to read about a specific subcommand or concept.\n",
            "See 'git help git' for an overview of the system.\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "!git init Data-Assessing\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "PhqhNhunpffm",
        "outputId": "97da6a46-a9d0-45dd-b9ea-dd517bd6c18b"
      },
      "execution_count": 196,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "\u001b[33mhint: Using 'master' as the name for the initial branch. This default branch name\u001b[m\n",
            "\u001b[33mhint: is subject to change. To configure the initial branch name to use in all\u001b[m\n",
            "\u001b[33mhint: of your new repositories, which will suppress this warning, call:\u001b[m\n",
            "\u001b[33mhint: \u001b[m\n",
            "\u001b[33mhint: \tgit config --global init.defaultBranch <name>\u001b[m\n",
            "\u001b[33mhint: \u001b[m\n",
            "\u001b[33mhint: Names commonly chosen instead of 'master' are 'main', 'trunk' and\u001b[m\n",
            "\u001b[33mhint: 'development'. The just-created branch can be renamed via this command:\u001b[m\n",
            "\u001b[33mhint: \u001b[m\n",
            "\u001b[33mhint: \tgit branch -m <name>\u001b[m\n",
            "Initialized empty Git repository in /content/Data-Assessing/.git/\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Data Analysis Process\n",
        "\n",
        "1. Asking Questions\n",
        "2. Data Wrangling<br>\n",
        "    a. Gathering Data\n",
        "        - i. CSV files\n",
        "        - ii. APIs\n",
        "        - iii. Web Scraping\n",
        "        - iv. Databases\n",
        "    b. Assessing Data<br>\n",
        "    c. Cleaning Data\n",
        "3. Exploratory Data Analysis\n",
        "4. Drawing Conclusion\n",
        "5. Comunicating Results"
      ],
      "metadata": {
        "id": "F-wE8bjqKaEc"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "### 1. Data Wrangling\n",
        "\n",
        "- Data Gathering\n",
        "- Data Accessing\n",
        "- Data Cleaning"
      ],
      "metadata": {
        "id": "HkCsfWWWL7Zi"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "### 1b. Data Accessing\n",
        "\n",
        "In this step, the data is to be understood more deeply. Before implementing methods to clean it."
      ],
      "metadata": {
        "id": "iHJBl3yVMIpw"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "Dirty Data (Data with Quality issues): Dirty data, also known as low quality data. Low quality data has content issues.\n",
        "\n",
        "Duplicated data\n",
        "Missing Data\n",
        "Corrupt Data\n",
        "Inaccurate Data\n",
        "Messy Data (Data with tidiness issues): Messy data, also known as untidy data. Untidy data has structural issues.Tidy data has the following properties:\n",
        "\n",
        "Each variable forms a column\n",
        "Each observation forms a row\n",
        "Each observational unit forms a table"
      ],
      "metadata": {
        "id": "LQ63y6d7lUBk"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "import pandas as pd\n",
        "import numpy as np"
      ],
      "metadata": {
        "id": "4NsoxauVKZTX"
      },
      "execution_count": 151,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "patients = pd.read_csv('patients.csv')\n",
        "treatments = pd.read_csv('treatments.csv')\n",
        "adverse_reactions = pd.read_csv('adverse_reactions.csv')\n",
        "treatments_cut = pd.read_csv('treatments_cut.csv')"
      ],
      "metadata": {
        "id": "jM051KScAOhz"
      },
      "execution_count": 152,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# view datasets\n",
        "patients.head()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 293
        },
        "id": "rxzIKAu2Eyd9",
        "outputId": "4fa61b39-4d16-4412-8c8d-90ca232c4ace"
      },
      "execution_count": 153,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "   patient_id assigned_sex given_name  surname                    address  \\\n",
              "0           1       female        Zoe  Wellish       576 Brown Bear Drive   \n",
              "1           2       female     Pamela     Hill  2370 University Hill Road   \n",
              "2           3         male        Jae   Debord      1493 Poling Farm Road   \n",
              "3           4         male       Liêm     Phan        2335 Webster Street   \n",
              "4           5         male        Tim  Neudorf       1428 Turkey Pen Lane   \n",
              "\n",
              "                city       state  zip_code        country  \\\n",
              "0  Rancho California  California   92390.0  United States   \n",
              "1          Armstrong    Illinois   61812.0  United States   \n",
              "2               York    Nebraska   68467.0  United States   \n",
              "3         Woodbridge          NJ    7095.0  United States   \n",
              "4             Dothan          AL   36303.0  United States   \n",
              "\n",
              "                                      contact  birthdate  weight  height   bmi  \n",
              "0        951-719-9170ZoeWellish@superrito.com  7/10/1976   121.7      66  19.6  \n",
              "1       PamelaSHill@cuvox.de+1 (217) 569-3204   4/3/1967   118.8      66  19.2  \n",
              "2            402-363-6804JaeMDebord@gustr.com  2/19/1980   177.8      71  24.8  \n",
              "3  PhanBaLiem@jourrapide.com+1 (732) 636-8246  7/26/1951   220.9      70  31.7  \n",
              "4             334-515-7487TimNeudorf@cuvox.de  2/18/1928   192.3      27  26.1  "
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-d50308e8-079f-4e4f-b049-c326387f21f9\" class=\"colab-df-container\">\n",
              "    <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>patient_id</th>\n",
              "      <th>assigned_sex</th>\n",
              "      <th>given_name</th>\n",
              "      <th>surname</th>\n",
              "      <th>address</th>\n",
              "      <th>city</th>\n",
              "      <th>state</th>\n",
              "      <th>zip_code</th>\n",
              "      <th>country</th>\n",
              "      <th>contact</th>\n",
              "      <th>birthdate</th>\n",
              "      <th>weight</th>\n",
              "      <th>height</th>\n",
              "      <th>bmi</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>1</td>\n",
              "      <td>female</td>\n",
              "      <td>Zoe</td>\n",
              "      <td>Wellish</td>\n",
              "      <td>576 Brown Bear Drive</td>\n",
              "      <td>Rancho California</td>\n",
              "      <td>California</td>\n",
              "      <td>92390.0</td>\n",
              "      <td>United States</td>\n",
              "      <td>951-719-9170ZoeWellish@superrito.com</td>\n",
              "      <td>7/10/1976</td>\n",
              "      <td>121.7</td>\n",
              "      <td>66</td>\n",
              "      <td>19.6</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>2</td>\n",
              "      <td>female</td>\n",
              "      <td>Pamela</td>\n",
              "      <td>Hill</td>\n",
              "      <td>2370 University Hill Road</td>\n",
              "      <td>Armstrong</td>\n",
              "      <td>Illinois</td>\n",
              "      <td>61812.0</td>\n",
              "      <td>United States</td>\n",
              "      <td>PamelaSHill@cuvox.de+1 (217) 569-3204</td>\n",
              "      <td>4/3/1967</td>\n",
              "      <td>118.8</td>\n",
              "      <td>66</td>\n",
              "      <td>19.2</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>3</td>\n",
              "      <td>male</td>\n",
              "      <td>Jae</td>\n",
              "      <td>Debord</td>\n",
              "      <td>1493 Poling Farm Road</td>\n",
              "      <td>York</td>\n",
              "      <td>Nebraska</td>\n",
              "      <td>68467.0</td>\n",
              "      <td>United States</td>\n",
              "      <td>402-363-6804JaeMDebord@gustr.com</td>\n",
              "      <td>2/19/1980</td>\n",
              "      <td>177.8</td>\n",
              "      <td>71</td>\n",
              "      <td>24.8</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>4</td>\n",
              "      <td>male</td>\n",
              "      <td>Liêm</td>\n",
              "      <td>Phan</td>\n",
              "      <td>2335 Webster Street</td>\n",
              "      <td>Woodbridge</td>\n",
              "      <td>NJ</td>\n",
              "      <td>7095.0</td>\n",
              "      <td>United States</td>\n",
              "      <td>PhanBaLiem@jourrapide.com+1 (732) 636-8246</td>\n",
              "      <td>7/26/1951</td>\n",
              "      <td>220.9</td>\n",
              "      <td>70</td>\n",
              "      <td>31.7</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>5</td>\n",
              "      <td>male</td>\n",
              "      <td>Tim</td>\n",
              "      <td>Neudorf</td>\n",
              "      <td>1428 Turkey Pen Lane</td>\n",
              "      <td>Dothan</td>\n",
              "      <td>AL</td>\n",
              "      <td>36303.0</td>\n",
              "      <td>United States</td>\n",
              "      <td>334-515-7487TimNeudorf@cuvox.de</td>\n",
              "      <td>2/18/1928</td>\n",
              "      <td>192.3</td>\n",
              "      <td>27</td>\n",
              "      <td>26.1</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "    <div class=\"colab-df-buttons\">\n",
              "\n",
              "  <div class=\"colab-df-container\">\n",
              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-d50308e8-079f-4e4f-b049-c326387f21f9')\"\n",
              "            title=\"Convert this dataframe to an interactive table.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    .colab-df-buttons div {\n",
              "      margin-bottom: 4px;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const buttonEl =\n",
              "        document.querySelector('#df-d50308e8-079f-4e4f-b049-c326387f21f9 button.colab-df-convert');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function convertToInteractive(key) {\n",
              "        const element = document.querySelector('#df-d50308e8-079f-4e4f-b049-c326387f21f9');\n",
              "        const dataTable =\n",
              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                    [key], {});\n",
              "        if (!dataTable) return;\n",
              "\n",
              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "          + ' to learn more about interactive tables.';\n",
              "        element.innerHTML = '';\n",
              "        dataTable['output_type'] = 'display_data';\n",
              "        await google.colab.output.renderOutput(dataTable, element);\n",
              "        const docLink = document.createElement('div');\n",
              "        docLink.innerHTML = docLinkHtml;\n",
              "        element.appendChild(docLink);\n",
              "      }\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "\n",
              "<div id=\"df-77ab0002-1305-4ecc-931e-52b832aff2bf\">\n",
              "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-77ab0002-1305-4ecc-931e-52b832aff2bf')\"\n",
              "            title=\"Suggest charts\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "     width=\"24px\">\n",
              "    <g>\n",
              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "    </g>\n",
              "</svg>\n",
              "  </button>\n",
              "\n",
              "<style>\n",
              "  .colab-df-quickchart {\n",
              "      --bg-color: #E8F0FE;\n",
              "      --fill-color: #1967D2;\n",
              "      --hover-bg-color: #E2EBFA;\n",
              "      --hover-fill-color: #174EA6;\n",
              "      --disabled-fill-color: #AAA;\n",
              "      --disabled-bg-color: #DDD;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart {\n",
              "      --bg-color: #3B4455;\n",
              "      --fill-color: #D2E3FC;\n",
              "      --hover-bg-color: #434B5C;\n",
              "      --hover-fill-color: #FFFFFF;\n",
              "      --disabled-bg-color: #3B4455;\n",
              "      --disabled-fill-color: #666;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart {\n",
              "    background-color: var(--bg-color);\n",
              "    border: none;\n",
              "    border-radius: 50%;\n",
              "    cursor: pointer;\n",
              "    display: none;\n",
              "    fill: var(--fill-color);\n",
              "    height: 32px;\n",
              "    padding: 0;\n",
              "    width: 32px;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart:hover {\n",
              "    background-color: var(--hover-bg-color);\n",
              "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "    fill: var(--button-hover-fill-color);\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart-complete:disabled,\n",
              "  .colab-df-quickchart-complete:disabled:hover {\n",
              "    background-color: var(--disabled-bg-color);\n",
              "    fill: var(--disabled-fill-color);\n",
              "    box-shadow: none;\n",
              "  }\n",
              "\n",
              "  .colab-df-spinner {\n",
              "    border: 2px solid var(--fill-color);\n",
              "    border-color: transparent;\n",
              "    border-bottom-color: var(--fill-color);\n",
              "    animation:\n",
              "      spin 1s steps(1) infinite;\n",
              "  }\n",
              "\n",
              "  @keyframes spin {\n",
              "    0% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "      border-left-color: var(--fill-color);\n",
              "    }\n",
              "    20% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    30% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    40% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    60% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    80% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "    90% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "  }\n",
              "</style>\n",
              "\n",
              "  <script>\n",
              "    async function quickchart(key) {\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#' + key + ' button');\n",
              "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
              "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
              "      try {\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'suggestCharts', [key], {});\n",
              "      } catch (error) {\n",
              "        console.error('Error during call to suggestCharts:', error);\n",
              "      }\n",
              "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
              "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
              "    }\n",
              "    (() => {\n",
              "      let quickchartButtonEl =\n",
              "        document.querySelector('#df-77ab0002-1305-4ecc-931e-52b832aff2bf button');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "    })();\n",
              "  </script>\n",
              "</div>\n",
              "\n",
              "    </div>\n",
              "  </div>\n"
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "dataframe",
              "variable_name": "patients",
              "summary": "{\n  \"name\": \"patients\",\n  \"rows\": 503,\n  \"fields\": [\n    {\n      \"column\": \"patient_id\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 145,\n        \"min\": 1,\n        \"max\": 503,\n        \"num_unique_values\": 503,\n        \"samples\": [\n          269,\n          74,\n          290\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"assigned_sex\",\n      \"properties\": {\n        \"dtype\": \"category\",\n        \"num_unique_values\": 2,\n        \"samples\": [\n          \"male\",\n          \"female\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"given_name\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 470,\n        \"samples\": [\n          \"Anco\",\n          \"Leo\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"surname\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 466,\n        \"samples\": [\n          \"Koldenhof\",\n          \"Ferreira\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"address\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 483,\n        \"samples\": [\n          \"1962 Cabell Avenue\",\n          \"3958 Liberty Avenue\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"city\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 349,\n        \"samples\": [\n          \"Monroe\",\n          \"Lafayette\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"state\",\n      \"properties\": {\n        \"dtype\": \"category\",\n        \"num_unique_values\": 54,\n        \"samples\": [\n          \"IL\",\n          \"OR\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"zip_code\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 30265.80744180444,\n        \"min\": 1002.0,\n        \"max\": 99701.0,\n        \"num_unique_values\": 430,\n        \"samples\": [\n          91505.0,\n          2048.0\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"country\",\n      \"properties\": {\n        \"dtype\": \"category\",\n        \"num_unique_values\": 1,\n        \"samples\": [\n          \"United States\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"contact\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 483,\n        \"samples\": [\n          \"703-547-0551SunKo@einrot.com\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"birthdate\",\n      \"properties\": {\n        \"dtype\": \"object\",\n        \"num_unique_values\": 493,\n        \"samples\": [\n          \"10/12/1980\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"weight\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 33.91674139657365,\n        \"min\": 48.8,\n        \"max\": 255.9,\n        \"num_unique_values\": 347,\n        \"samples\": [\n          239.1\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"height\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 4,\n        \"min\": 27,\n        \"max\": 79,\n        \"num_unique_values\": 22,\n        \"samples\": [\n          66\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"bmi\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 5.2764378079801695,\n        \"min\": 17.1,\n        \"max\": 37.7,\n        \"num_unique_values\": 183,\n        \"samples\": [\n          24.7\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    }\n  ]\n}"
            }
          },
          "metadata": {},
          "execution_count": 153
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "treatments.head()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 206
        },
        "id": "HzejqbNKaqYO",
        "outputId": "99d5866d-7f02-4e3f-b35f-886ddedad7bd"
      },
      "execution_count": 154,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "  given_name     surname    auralin    novodra  hba1c_start  hba1c_end  \\\n",
              "0   veronika    jindrová  41u - 48u          -         7.63       7.20   \n",
              "1     elliot  richardson          -  40u - 45u         7.56       7.09   \n",
              "2   yukitaka    takenaka          -  39u - 36u         7.68       7.25   \n",
              "3       skye  gormanston  33u - 36u          -         7.97       7.62   \n",
              "4     alissa      montez          -  33u - 29u         7.78       7.46   \n",
              "\n",
              "   hba1c_change  \n",
              "0           NaN  \n",
              "1          0.97  \n",
              "2           NaN  \n",
              "3          0.35  \n",
              "4          0.32  "
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-0fda7282-d202-4634-b3cf-b032ecd0bba8\" class=\"colab-df-container\">\n",
              "    <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>given_name</th>\n",
              "      <th>surname</th>\n",
              "      <th>auralin</th>\n",
              "      <th>novodra</th>\n",
              "      <th>hba1c_start</th>\n",
              "      <th>hba1c_end</th>\n",
              "      <th>hba1c_change</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>veronika</td>\n",
              "      <td>jindrová</td>\n",
              "      <td>41u - 48u</td>\n",
              "      <td>-</td>\n",
              "      <td>7.63</td>\n",
              "      <td>7.20</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>elliot</td>\n",
              "      <td>richardson</td>\n",
              "      <td>-</td>\n",
              "      <td>40u - 45u</td>\n",
              "      <td>7.56</td>\n",
              "      <td>7.09</td>\n",
              "      <td>0.97</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>yukitaka</td>\n",
              "      <td>takenaka</td>\n",
              "      <td>-</td>\n",
              "      <td>39u - 36u</td>\n",
              "      <td>7.68</td>\n",
              "      <td>7.25</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>skye</td>\n",
              "      <td>gormanston</td>\n",
              "      <td>33u - 36u</td>\n",
              "      <td>-</td>\n",
              "      <td>7.97</td>\n",
              "      <td>7.62</td>\n",
              "      <td>0.35</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>alissa</td>\n",
              "      <td>montez</td>\n",
              "      <td>-</td>\n",
              "      <td>33u - 29u</td>\n",
              "      <td>7.78</td>\n",
              "      <td>7.46</td>\n",
              "      <td>0.32</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "    <div class=\"colab-df-buttons\">\n",
              "\n",
              "  <div class=\"colab-df-container\">\n",
              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-0fda7282-d202-4634-b3cf-b032ecd0bba8')\"\n",
              "            title=\"Convert this dataframe to an interactive table.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    .colab-df-buttons div {\n",
              "      margin-bottom: 4px;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const buttonEl =\n",
              "        document.querySelector('#df-0fda7282-d202-4634-b3cf-b032ecd0bba8 button.colab-df-convert');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function convertToInteractive(key) {\n",
              "        const element = document.querySelector('#df-0fda7282-d202-4634-b3cf-b032ecd0bba8');\n",
              "        const dataTable =\n",
              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                    [key], {});\n",
              "        if (!dataTable) return;\n",
              "\n",
              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "          + ' to learn more about interactive tables.';\n",
              "        element.innerHTML = '';\n",
              "        dataTable['output_type'] = 'display_data';\n",
              "        await google.colab.output.renderOutput(dataTable, element);\n",
              "        const docLink = document.createElement('div');\n",
              "        docLink.innerHTML = docLinkHtml;\n",
              "        element.appendChild(docLink);\n",
              "      }\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "\n",
              "<div id=\"df-52b1d9ff-efed-44fd-a3bb-adf1843140a4\">\n",
              "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-52b1d9ff-efed-44fd-a3bb-adf1843140a4')\"\n",
              "            title=\"Suggest charts\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "     width=\"24px\">\n",
              "    <g>\n",
              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "    </g>\n",
              "</svg>\n",
              "  </button>\n",
              "\n",
              "<style>\n",
              "  .colab-df-quickchart {\n",
              "      --bg-color: #E8F0FE;\n",
              "      --fill-color: #1967D2;\n",
              "      --hover-bg-color: #E2EBFA;\n",
              "      --hover-fill-color: #174EA6;\n",
              "      --disabled-fill-color: #AAA;\n",
              "      --disabled-bg-color: #DDD;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart {\n",
              "      --bg-color: #3B4455;\n",
              "      --fill-color: #D2E3FC;\n",
              "      --hover-bg-color: #434B5C;\n",
              "      --hover-fill-color: #FFFFFF;\n",
              "      --disabled-bg-color: #3B4455;\n",
              "      --disabled-fill-color: #666;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart {\n",
              "    background-color: var(--bg-color);\n",
              "    border: none;\n",
              "    border-radius: 50%;\n",
              "    cursor: pointer;\n",
              "    display: none;\n",
              "    fill: var(--fill-color);\n",
              "    height: 32px;\n",
              "    padding: 0;\n",
              "    width: 32px;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart:hover {\n",
              "    background-color: var(--hover-bg-color);\n",
              "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "    fill: var(--button-hover-fill-color);\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart-complete:disabled,\n",
              "  .colab-df-quickchart-complete:disabled:hover {\n",
              "    background-color: var(--disabled-bg-color);\n",
              "    fill: var(--disabled-fill-color);\n",
              "    box-shadow: none;\n",
              "  }\n",
              "\n",
              "  .colab-df-spinner {\n",
              "    border: 2px solid var(--fill-color);\n",
              "    border-color: transparent;\n",
              "    border-bottom-color: var(--fill-color);\n",
              "    animation:\n",
              "      spin 1s steps(1) infinite;\n",
              "  }\n",
              "\n",
              "  @keyframes spin {\n",
              "    0% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "      border-left-color: var(--fill-color);\n",
              "    }\n",
              "    20% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    30% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    40% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    60% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    80% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "    90% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "  }\n",
              "</style>\n",
              "\n",
              "  <script>\n",
              "    async function quickchart(key) {\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#' + key + ' button');\n",
              "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
              "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
              "      try {\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'suggestCharts', [key], {});\n",
              "      } catch (error) {\n",
              "        console.error('Error during call to suggestCharts:', error);\n",
              "      }\n",
              "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
              "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
              "    }\n",
              "    (() => {\n",
              "      let quickchartButtonEl =\n",
              "        document.querySelector('#df-52b1d9ff-efed-44fd-a3bb-adf1843140a4 button');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "    })();\n",
              "  </script>\n",
              "</div>\n",
              "\n",
              "    </div>\n",
              "  </div>\n"
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "dataframe",
              "variable_name": "treatments",
              "summary": "{\n  \"name\": \"treatments\",\n  \"rows\": 280,\n  \"fields\": [\n    {\n      \"column\": \"given_name\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 273,\n        \"samples\": [\n          \"szilveszter\",\n          \"javier\",\n          \"avdei\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"surname\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 272,\n        \"samples\": [\n          \"totth\",\n          \"silva\",\n          \"tikhonov\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"auralin\",\n      \"properties\": {\n        \"dtype\": \"category\",\n        \"num_unique_values\": 121,\n        \"samples\": [\n          \"53u - 64u\",\n          \"39u - 43u\",\n          \"31u - 38u\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"novodra\",\n      \"properties\": {\n        \"dtype\": \"category\",\n        \"num_unique_values\": 113,\n        \"samples\": [\n          \"31u - 36u\",\n          \"42u - 44u\",\n          \"34u - 32u\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"hba1c_start\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0.5686384372332098,\n        \"min\": 7.5,\n        \"max\": 9.95,\n        \"num_unique_values\": 97,\n        \"samples\": [\n          8.19,\n          7.58,\n          9.62\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"hba1c_end\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0.5696715306576079,\n        \"min\": 7.01,\n        \"max\": 9.58,\n        \"num_unique_values\": 109,\n        \"samples\": [\n          7.45,\n          7.19,\n          7.46\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"hba1c_change\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0.2795552760918328,\n        \"min\": 0.2,\n        \"max\": 0.99,\n        \"num_unique_values\": 30,\n        \"samples\": [\n          0.26,\n          0.3,\n          0.28\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    }\n  ]\n}"
            }
          },
          "metadata": {},
          "execution_count": 154
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "treatments_cut.shape"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "x8yAHH6_a0xZ",
        "outputId": "e07bc811-4ec0-4d1a-85fc-98180c52ae6b"
      },
      "execution_count": 155,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(70, 7)"
            ]
          },
          "metadata": {},
          "execution_count": 155
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "adverse_reactions"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000
        },
        "id": "WraIcwiua9Qu",
        "outputId": "d94d2df2-733a-4349-c252-cb38f8e4af52"
      },
      "execution_count": 156,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "     given_name     surname           adverse_reaction\n",
              "0         berta  napolitani  injection site discomfort\n",
              "1          lena        baer               hypoglycemia\n",
              "2        joseph         day               hypoglycemia\n",
              "3        flavia  fiorentino                      cough\n",
              "4       manouck     wubbels          throat irritation\n",
              "5       jasmine       sykes               hypoglycemia\n",
              "6        louise     johnson               hypoglycemia\n",
              "7       albinca     komavec               hypoglycemia\n",
              "8           noe      aranda               hypoglycemia\n",
              "9         sofia   hermansen  injection site discomfort\n",
              "10        tegan     johnson                   headache\n",
              "11         abel     yonatan                      cough\n",
              "12    abdul-nur         isa               hypoglycemia\n",
              "13         leon      scholz  injection site discomfort\n",
              "14     gabriele     saenger               hypoglycemia\n",
              "15       jia li        teng                     nausea\n",
              "16        jakob    jakobsen               hypoglycemia\n",
              "17  christopher    woodward                     nausea\n",
              "18          ole    petersen               hypoglycemia\n",
              "19       finley    chandler                   headache\n",
              "20     anenechi       chidi               hypoglycemia\n",
              "21     miłosław  wiśniewski  injection site discomfort\n",
              "22        lixue       hsueh  injection site discomfort\n",
              "23        merci      leroux               hypoglycemia\n",
              "24         kang         mai  injection site discomfort\n",
              "25       elliot  richardson               hypoglycemia\n",
              "26      clinton      miller          throat irritation\n",
              "27       idalia       moore               hypoglycemia\n",
              "28       xiuxiu       chang               hypoglycemia\n",
              "29         alex    crawford               hypoglycemia\n",
              "30       monika      lončar               hypoglycemia\n",
              "31       steven         roy                   headache\n",
              "32      cecilie      nilsen               hypoglycemia\n",
              "33    krisztina      magyar               hypoglycemia"
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-f737fa0f-94df-4155-8e7d-2d1a0bfcfbe7\" class=\"colab-df-container\">\n",
              "    <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>given_name</th>\n",
              "      <th>surname</th>\n",
              "      <th>adverse_reaction</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>berta</td>\n",
              "      <td>napolitani</td>\n",
              "      <td>injection site discomfort</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>lena</td>\n",
              "      <td>baer</td>\n",
              "      <td>hypoglycemia</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>joseph</td>\n",
              "      <td>day</td>\n",
              "      <td>hypoglycemia</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>flavia</td>\n",
              "      <td>fiorentino</td>\n",
              "      <td>cough</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>manouck</td>\n",
              "      <td>wubbels</td>\n",
              "      <td>throat irritation</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>5</th>\n",
              "      <td>jasmine</td>\n",
              "      <td>sykes</td>\n",
              "      <td>hypoglycemia</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>6</th>\n",
              "      <td>louise</td>\n",
              "      <td>johnson</td>\n",
              "      <td>hypoglycemia</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>7</th>\n",
              "      <td>albinca</td>\n",
              "      <td>komavec</td>\n",
              "      <td>hypoglycemia</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>8</th>\n",
              "      <td>noe</td>\n",
              "      <td>aranda</td>\n",
              "      <td>hypoglycemia</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>9</th>\n",
              "      <td>sofia</td>\n",
              "      <td>hermansen</td>\n",
              "      <td>injection site discomfort</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>10</th>\n",
              "      <td>tegan</td>\n",
              "      <td>johnson</td>\n",
              "      <td>headache</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>11</th>\n",
              "      <td>abel</td>\n",
              "      <td>yonatan</td>\n",
              "      <td>cough</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>12</th>\n",
              "      <td>abdul-nur</td>\n",
              "      <td>isa</td>\n",
              "      <td>hypoglycemia</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>13</th>\n",
              "      <td>leon</td>\n",
              "      <td>scholz</td>\n",
              "      <td>injection site discomfort</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>14</th>\n",
              "      <td>gabriele</td>\n",
              "      <td>saenger</td>\n",
              "      <td>hypoglycemia</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>15</th>\n",
              "      <td>jia li</td>\n",
              "      <td>teng</td>\n",
              "      <td>nausea</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>16</th>\n",
              "      <td>jakob</td>\n",
              "      <td>jakobsen</td>\n",
              "      <td>hypoglycemia</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>17</th>\n",
              "      <td>christopher</td>\n",
              "      <td>woodward</td>\n",
              "      <td>nausea</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>18</th>\n",
              "      <td>ole</td>\n",
              "      <td>petersen</td>\n",
              "      <td>hypoglycemia</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>19</th>\n",
              "      <td>finley</td>\n",
              "      <td>chandler</td>\n",
              "      <td>headache</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>20</th>\n",
              "      <td>anenechi</td>\n",
              "      <td>chidi</td>\n",
              "      <td>hypoglycemia</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>21</th>\n",
              "      <td>miłosław</td>\n",
              "      <td>wiśniewski</td>\n",
              "      <td>injection site discomfort</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>22</th>\n",
              "      <td>lixue</td>\n",
              "      <td>hsueh</td>\n",
              "      <td>injection site discomfort</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>23</th>\n",
              "      <td>merci</td>\n",
              "      <td>leroux</td>\n",
              "      <td>hypoglycemia</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>24</th>\n",
              "      <td>kang</td>\n",
              "      <td>mai</td>\n",
              "      <td>injection site discomfort</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>25</th>\n",
              "      <td>elliot</td>\n",
              "      <td>richardson</td>\n",
              "      <td>hypoglycemia</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>26</th>\n",
              "      <td>clinton</td>\n",
              "      <td>miller</td>\n",
              "      <td>throat irritation</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>27</th>\n",
              "      <td>idalia</td>\n",
              "      <td>moore</td>\n",
              "      <td>hypoglycemia</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>28</th>\n",
              "      <td>xiuxiu</td>\n",
              "      <td>chang</td>\n",
              "      <td>hypoglycemia</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>29</th>\n",
              "      <td>alex</td>\n",
              "      <td>crawford</td>\n",
              "      <td>hypoglycemia</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>30</th>\n",
              "      <td>monika</td>\n",
              "      <td>lončar</td>\n",
              "      <td>hypoglycemia</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>31</th>\n",
              "      <td>steven</td>\n",
              "      <td>roy</td>\n",
              "      <td>headache</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>32</th>\n",
              "      <td>cecilie</td>\n",
              "      <td>nilsen</td>\n",
              "      <td>hypoglycemia</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>33</th>\n",
              "      <td>krisztina</td>\n",
              "      <td>magyar</td>\n",
              "      <td>hypoglycemia</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "    <div class=\"colab-df-buttons\">\n",
              "\n",
              "  <div class=\"colab-df-container\">\n",
              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-f737fa0f-94df-4155-8e7d-2d1a0bfcfbe7')\"\n",
              "            title=\"Convert this dataframe to an interactive table.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    .colab-df-buttons div {\n",
              "      margin-bottom: 4px;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const buttonEl =\n",
              "        document.querySelector('#df-f737fa0f-94df-4155-8e7d-2d1a0bfcfbe7 button.colab-df-convert');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function convertToInteractive(key) {\n",
              "        const element = document.querySelector('#df-f737fa0f-94df-4155-8e7d-2d1a0bfcfbe7');\n",
              "        const dataTable =\n",
              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                    [key], {});\n",
              "        if (!dataTable) return;\n",
              "\n",
              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "          + ' to learn more about interactive tables.';\n",
              "        element.innerHTML = '';\n",
              "        dataTable['output_type'] = 'display_data';\n",
              "        await google.colab.output.renderOutput(dataTable, element);\n",
              "        const docLink = document.createElement('div');\n",
              "        docLink.innerHTML = docLinkHtml;\n",
              "        element.appendChild(docLink);\n",
              "      }\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "\n",
              "<div id=\"df-84d1c0f6-d506-4db4-98b5-7cc9fb34e921\">\n",
              "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-84d1c0f6-d506-4db4-98b5-7cc9fb34e921')\"\n",
              "            title=\"Suggest charts\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "     width=\"24px\">\n",
              "    <g>\n",
              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "    </g>\n",
              "</svg>\n",
              "  </button>\n",
              "\n",
              "<style>\n",
              "  .colab-df-quickchart {\n",
              "      --bg-color: #E8F0FE;\n",
              "      --fill-color: #1967D2;\n",
              "      --hover-bg-color: #E2EBFA;\n",
              "      --hover-fill-color: #174EA6;\n",
              "      --disabled-fill-color: #AAA;\n",
              "      --disabled-bg-color: #DDD;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart {\n",
              "      --bg-color: #3B4455;\n",
              "      --fill-color: #D2E3FC;\n",
              "      --hover-bg-color: #434B5C;\n",
              "      --hover-fill-color: #FFFFFF;\n",
              "      --disabled-bg-color: #3B4455;\n",
              "      --disabled-fill-color: #666;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart {\n",
              "    background-color: var(--bg-color);\n",
              "    border: none;\n",
              "    border-radius: 50%;\n",
              "    cursor: pointer;\n",
              "    display: none;\n",
              "    fill: var(--fill-color);\n",
              "    height: 32px;\n",
              "    padding: 0;\n",
              "    width: 32px;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart:hover {\n",
              "    background-color: var(--hover-bg-color);\n",
              "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "    fill: var(--button-hover-fill-color);\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart-complete:disabled,\n",
              "  .colab-df-quickchart-complete:disabled:hover {\n",
              "    background-color: var(--disabled-bg-color);\n",
              "    fill: var(--disabled-fill-color);\n",
              "    box-shadow: none;\n",
              "  }\n",
              "\n",
              "  .colab-df-spinner {\n",
              "    border: 2px solid var(--fill-color);\n",
              "    border-color: transparent;\n",
              "    border-bottom-color: var(--fill-color);\n",
              "    animation:\n",
              "      spin 1s steps(1) infinite;\n",
              "  }\n",
              "\n",
              "  @keyframes spin {\n",
              "    0% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "      border-left-color: var(--fill-color);\n",
              "    }\n",
              "    20% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    30% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    40% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    60% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    80% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "    90% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "  }\n",
              "</style>\n",
              "\n",
              "  <script>\n",
              "    async function quickchart(key) {\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#' + key + ' button');\n",
              "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
              "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
              "      try {\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'suggestCharts', [key], {});\n",
              "      } catch (error) {\n",
              "        console.error('Error during call to suggestCharts:', error);\n",
              "      }\n",
              "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
              "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
              "    }\n",
              "    (() => {\n",
              "      let quickchartButtonEl =\n",
              "        document.querySelector('#df-84d1c0f6-d506-4db4-98b5-7cc9fb34e921 button');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "    })();\n",
              "  </script>\n",
              "</div>\n",
              "\n",
              "  <div id=\"id_5ef2f403-7d35-4705-a9c1-1a64f15cad2d\">\n",
              "    <style>\n",
              "      .colab-df-generate {\n",
              "        background-color: #E8F0FE;\n",
              "        border: none;\n",
              "        border-radius: 50%;\n",
              "        cursor: pointer;\n",
              "        display: none;\n",
              "        fill: #1967D2;\n",
              "        height: 32px;\n",
              "        padding: 0 0 0 0;\n",
              "        width: 32px;\n",
              "      }\n",
              "\n",
              "      .colab-df-generate:hover {\n",
              "        background-color: #E2EBFA;\n",
              "        box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "        fill: #174EA6;\n",
              "      }\n",
              "\n",
              "      [theme=dark] .colab-df-generate {\n",
              "        background-color: #3B4455;\n",
              "        fill: #D2E3FC;\n",
              "      }\n",
              "\n",
              "      [theme=dark] .colab-df-generate:hover {\n",
              "        background-color: #434B5C;\n",
              "        box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "        filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "        fill: #FFFFFF;\n",
              "      }\n",
              "    </style>\n",
              "    <button class=\"colab-df-generate\" onclick=\"generateWithVariable('adverse_reactions')\"\n",
              "            title=\"Generate code using this dataframe.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M7,19H8.4L18.45,9,17,7.55,7,17.6ZM5,21V16.75L18.45,3.32a2,2,0,0,1,2.83,0l1.4,1.43a1.91,1.91,0,0,1,.58,1.4,1.91,1.91,0,0,1-.58,1.4L9.25,21ZM18.45,9,17,7.55Zm-12,3A5.31,5.31,0,0,0,4.9,8.1,5.31,5.31,0,0,0,1,6.5,5.31,5.31,0,0,0,4.9,4.9,5.31,5.31,0,0,0,6.5,1,5.31,5.31,0,0,0,8.1,4.9,5.31,5.31,0,0,0,12,6.5,5.46,5.46,0,0,0,6.5,12Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "    <script>\n",
              "      (() => {\n",
              "      const buttonEl =\n",
              "        document.querySelector('#id_5ef2f403-7d35-4705-a9c1-1a64f15cad2d button.colab-df-generate');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      buttonEl.onclick = () => {\n",
              "        google.colab.notebook.generateWithVariable('adverse_reactions');\n",
              "      }\n",
              "      })();\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "    </div>\n",
              "  </div>\n"
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "dataframe",
              "variable_name": "adverse_reactions",
              "summary": "{\n  \"name\": \"adverse_reactions\",\n  \"rows\": 34,\n  \"fields\": [\n    {\n      \"column\": \"given_name\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 34,\n        \"samples\": [\n          \"jia li\",\n          \"finley\",\n          \"idalia\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"surname\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 33,\n        \"samples\": [\n          \"nilsen\",\n          \"jakobsen\",\n          \"moore\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"adverse_reaction\",\n      \"properties\": {\n        \"dtype\": \"category\",\n        \"num_unique_values\": 6,\n        \"samples\": [\n          \"injection site discomfort\",\n          \"hypoglycemia\",\n          \"nausea\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    }\n  ]\n}"
            }
          },
          "metadata": {},
          "execution_count": 156
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "### 1. Write a summary for your data\n",
        "\n",
        "This is a dataset about 500 patients of which 350 patients participated in a clinical trial. None of the patients were using Novodra (a popular injectable insulin) or Auralin (the oral insulin being researched) as their primary source of insulin before. All were experiencing elevated HbA1c levels.\n",
        "\n",
        "All 350 patients were treated with Novodra to establish a baseline HbA1c level and insulin dose. After 4 weeks, which isn’t enough time to capture all the change in HbA1c that can be attributed by the switch to Auralin or Novodra:\n",
        "\n",
        "- 175 patients switched to Auralin for 24 weeks\n",
        "- 175 patients continued using Novodra for 24 weeks\n",
        "\n",
        "Data about patients feeling some adverse effects is also recorded."
      ],
      "metadata": {
        "id": "hX-2YwTZU-Rz"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "### 2. Write Column descriptions\n",
        "\n",
        "#### **Table** -> `patients`:\n",
        "\n",
        "- `patient_id`: the unique identifier for each patient in the Master Patient Index (i.e. patient database) of the pharmaceutical company that is producing Auralin\n",
        "- `assigned_sex`: the assigned sex of each patient at birth (male or female)\n",
        "- `given_name`: the given name (i.e. first name) of each patient\n",
        "- `surname`: the surname (i.e. last name) of each patient\n",
        "- `address`: the main address for each patient\n",
        "- `city`: the corresponding city for the main address of each patient\n",
        "- `state`: the corresponding state for the main address of each patient\n",
        "- `zip_code`: the corresponding zip code for the main address of each patient\n",
        "- `country`: the corresponding country for the main address of each patient (all United states for this clinical trial)\n",
        "- `contact`: phone number and email information for each patient\n",
        "- `birthdate`: the date of birth of each patient (month/day/year). The inclusion criteria for this clinical trial is age >= 18 (there is no maximum age because diabetes is a growing problem among the elderly population)\n",
        "- `weight`: the weight of each patient in pounds (lbs)\n",
        "- `height`: the height of each patient in inches (in)\n",
        "- `bmi`: the Body Mass Index (BMI) of each patient. BMI is a simple calculation using a person's height and weight. The formula is BMI = kg/m2 where kg is a person's weight in kilograms and m2 is their height in metres squared. A BMI of 25.0 or more is overweight, while the healthy range is 18.5 to 24.9. The inclusion criteria for this clinical trial is 16 >= BMI >= 38."
      ],
      "metadata": {
        "id": "Kf37wVIMVzGY"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "#### **Table** -> `treatments` and `treatment_cut`:\n",
        "\n",
        "- `given_name`: the given name of each patient in the Master Patient Index that took part in the clinical trial\n",
        "- `surname`: the surname of each patient in the Master Patient Index that took part in the clinical trial\n",
        "- `auralin`: the baseline median daily dose of insulin from the week prior to switching to Auralin (the number before the dash) and the ending median daily dose of insulin at the end of the 24 weeks of treatment measured over the 24th week of treatment (the number after the dash). Both are measured in units (shortform 'u'), which is the international unit of measurement and the standard measurement for insulin.\n",
        "- `novodra`: same as above, except for patients that continued treatment with Novodra\n",
        "- `hba1c_start`: the patient's HbA1c level at the beginning of the first week of treatment. HbA1c stands for Hemoglobin A1c. The HbA1c test measures what the average blood sugar has been over the past three months. It is thus a powerful way to get an overall sense of how well diabetes has been controlled. Everyone with diabetes should have this test 2 to 4 times per year. Measured in %.\n",
        "- `hba1c_end`: the patient's HbA1c level at the end of the last week of treatment\n",
        "- `hba1c_change`: the change in the patient's HbA1c level from the start of treatment to the end, i.e., hba1c_start - hba1c_end. For Auralin to be deemed effective, it must be \"noninferior\" to Novodra, the current standard for insulin. This \"noninferiority\" is statistically defined as the upper bound of the 95% confidence interval being less than 0.4% for the difference between the mean HbA1c changes for Novodra and Auralin (i.e. Novodra minus Auralin)."
      ],
      "metadata": {
        "id": "upnvbnvSWh3u"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "#### **Table** -> `adverse_reactions`\n",
        "\n",
        "- `given_name`: the given name of each patient in the Master Patient Index that took part in the clinical trial and had an adverse reaction (includes both patients treated Auralin and Novodra)\n",
        "- `surname`: the surname of each patient in the Master Patient Index that took part in the clinical trial and had an adverse reaction (includes both patients treated Auralin and Novodra)\n",
        "- `adverse_reaction`: the adverse reaction reported by the patient"
      ],
      "metadata": {
        "id": "ATIbzZc3XCoX"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "### 3. Add any additional information\n",
        "\n",
        "Additional useful information:\n",
        "\n",
        "- Insulin resistance varies person to person, which is why both starting median daily dose and ending median daily dose are required, i.e., to calculate change in dose.\n",
        "- It is important to test drugs and medical products in the people they are meant to help. People of different age, race, sex, and ethnic group must be included in clinical trials. This diversity is reflected in the patients table."
      ],
      "metadata": {
        "id": "5E4go7hDXpNQ"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Types of Assessment\n",
        "There are 2 types of assessment styles\n",
        "\n",
        "- `Manual` - Looking through the data manually in google sheets\n",
        "- `Programmatic` - By using pandas functions such as info(), describe() or sample()\n",
        "\n",
        "### Steps in Assessment\n",
        "There are 2 steps involved in Assessment\n",
        "\n",
        "- Discover\n",
        "- Document"
      ],
      "metadata": {
        "id": "jgbn6X0fdi3R"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# export data for manual assessment\n",
        "\n",
        "with pd.ExcelWriter('clinical_trials.xlsx') as writer:\n",
        "  patients.to_excel(writer,sheet_name='patients')\n",
        "  treatments.to_excel(writer,sheet_name='treatments')\n",
        "  treatments_cut.to_excel(writer,sheet_name='treatment_cut')\n",
        "  adverse_reactions.to_excel(writer,sheet_name='adverse_reactions')"
      ],
      "metadata": {
        "id": "Cy3xG6SUVyTB"
      },
      "execution_count": 157,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Issues with the dataset\n",
        "\n",
        "1. Dirty Data\n",
        "\n",
        "  Table - `Patients`\n",
        "\n",
        "    - patient_id = 9 has misspelled name 'Dsvid' instead of David `accuracy`\n",
        "    - state col sometimes contain full name and some times abbrivietation `consistency`\n",
        "    - zip code col has entries with 4 digit `validity`\n",
        "    - data missing for 12 patients in address,city,\tstate,zip_code\t,country,\tcontact `completion`\n",
        "    - incorrect data type assigned to sex, zip code, birthdate `validity`\n",
        "    - duplicate entries by the name of John Doe `accuracy`\n",
        "    - one patient has weight = 48 pounds `accuracy`\n",
        "    - one patient has height = 27 inches `accuracy`\n",
        "\n",
        "  Table - `Treatments` & `Treatments_cut`\n",
        "\n",
        "    - given_name and surname col is is all lower case `consistency`\n",
        "    - remove u from Auralin and Novadra cols `validity`\n",
        "    - '-' in novadra and Auralin col treated as nan `validity`\n",
        "    - missing values in hba1c_change col `completion`\n",
        "    - 1 duplicate entry by the name Joseph day `accuracy`\n",
        "    - in hba1c_change 9 instead of 4 `accuracy`\n",
        "\n",
        "  Table - `Adverse_reactions`\n",
        "\n",
        "    - given_name and surname are all in lower case `consistency`\n",
        "\n",
        "2. Messy Data\n",
        "\n",
        "  Table - `Patients`\n",
        "\n",
        "    - contact col contains both phone and email\n",
        "\n",
        "  Table - `Treatments` & `Treatments_cut`\n",
        "\n",
        "    - Auralin and Novadra col should be split into 2 cols start and end dose\n",
        "    - merge both the tables\n",
        "\n",
        "  Table - `Adverse_reactions`\n",
        "\n",
        "    - This table should not exist independently"
      ],
      "metadata": {
        "id": "yiq4Dv63PlGO"
      }
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "Q3kGLT_fCJsJ"
      },
      "execution_count": 157,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "#Manual Assessment\n"
      ],
      "metadata": {
        "id": "POj-jeWSCH4J"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Issues with the dataset\n",
        "1. Dirty Data Table - 'Patients\n",
        "patient_id = 9 has misspelled name 'Dsvid' instead of David\n",
        "\n",
        "state col sometimes contain full name and some times abbrivietation\n",
        "\n",
        "zip code col has entries with 4 digit\n",
        "\n",
        "data missing for 12 patients in address, city, state, zip_code country, contact\n",
        "\n",
        "Table - \"Treatments' & 'Treatments_cut*\n",
        "\n",
        "given_name and surname col is is all lower case\n",
        "\n",
        "remove u from Auralin and Novadra cols\n",
        "\n",
        "'-' in novadra and Auralin col treated as nan\n",
        "\n",
        "missing values in hba1c_change col\n",
        "\n",
        "Table - Adverse_reactions\"\n",
        "\n",
        "2. Messy Data\n",
        "\n",
        "Table - Patients\n",
        "\n",
        "• contact col contains both phone and email\n",
        "\n",
        "Table - Treatments & Treatments\n",
        "\n",
        "• Auralin and Novadra col should be split into 2 cols start and end dose\n",
        "\n",
        "Table - Adverse_reactions\n",
        "\n",
        "• This table should not exist independently"
      ],
      "metadata": {
        "id": "Vs_Bxf_mCIpG"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Automatic Assessment\n",
        "\n",
        "- head and tail\n",
        "- sample\n",
        "- info\n",
        "- isnull\n",
        "- duplicated\n",
        "- describe"
      ],
      "metadata": {
        "id": "9u8OLVp6htxf"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "patients.sample(5) #Gives 5 random samples from dataset"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 293
        },
        "id": "CNE6hrnCAnqg",
        "outputId": "8e7442ea-4159-4b3e-ea27-bba9ca60fddc"
      },
      "execution_count": 158,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "     patient_id assigned_sex  given_name   surname                 address  \\\n",
              "212         213         male        Jens   Poulsen        1942 Harry Place   \n",
              "373         374       female      Johana      Hrdá     541 Green Hill Road   \n",
              "361         362         male  Hans Peter  Radislav       208 Birch  Street   \n",
              "468         469         male        Tosh    Jensen  1536 John Daniel Drive   \n",
              "502         503         male         Pat   Gersten       2778 North Avenue   \n",
              "\n",
              "           city     state  zip_code        country  \\\n",
              "212   Charlotte        NC   28202.0  United States   \n",
              "373  Springdale        AR   72762.0  United States   \n",
              "361     El Paso        TX   79905.0  United States   \n",
              "468      Argyle        MO   65001.0  United States   \n",
              "502        Burr  Nebraska   68324.0  United States   \n",
              "\n",
              "                                     contact   birthdate  weight  height   bmi  \n",
              "212      JensPoulsen@fleckens.hu704-805-5094   12/9/1963   168.1      71  23.4  \n",
              "373        JohanaHrda@dayrep.com479-248-5663    8/4/1952   216.9      66  35.0  \n",
              "361  HansPeterRadislav@gustr.com915-612-1462  10/28/1973   169.0      70  24.2  \n",
              "468    573-728-3861ToshCJensen@superrito.com  12/19/1969   194.7      68  29.6  \n",
              "502     PatrickGersten@rhyta.com402-848-4923    5/3/1954   138.2      71  19.3  "
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-115d9592-c26d-4eea-b0aa-0774248cc08f\" class=\"colab-df-container\">\n",
              "    <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>patient_id</th>\n",
              "      <th>assigned_sex</th>\n",
              "      <th>given_name</th>\n",
              "      <th>surname</th>\n",
              "      <th>address</th>\n",
              "      <th>city</th>\n",
              "      <th>state</th>\n",
              "      <th>zip_code</th>\n",
              "      <th>country</th>\n",
              "      <th>contact</th>\n",
              "      <th>birthdate</th>\n",
              "      <th>weight</th>\n",
              "      <th>height</th>\n",
              "      <th>bmi</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>212</th>\n",
              "      <td>213</td>\n",
              "      <td>male</td>\n",
              "      <td>Jens</td>\n",
              "      <td>Poulsen</td>\n",
              "      <td>1942 Harry Place</td>\n",
              "      <td>Charlotte</td>\n",
              "      <td>NC</td>\n",
              "      <td>28202.0</td>\n",
              "      <td>United States</td>\n",
              "      <td>JensPoulsen@fleckens.hu704-805-5094</td>\n",
              "      <td>12/9/1963</td>\n",
              "      <td>168.1</td>\n",
              "      <td>71</td>\n",
              "      <td>23.4</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>373</th>\n",
              "      <td>374</td>\n",
              "      <td>female</td>\n",
              "      <td>Johana</td>\n",
              "      <td>Hrdá</td>\n",
              "      <td>541 Green Hill Road</td>\n",
              "      <td>Springdale</td>\n",
              "      <td>AR</td>\n",
              "      <td>72762.0</td>\n",
              "      <td>United States</td>\n",
              "      <td>JohanaHrda@dayrep.com479-248-5663</td>\n",
              "      <td>8/4/1952</td>\n",
              "      <td>216.9</td>\n",
              "      <td>66</td>\n",
              "      <td>35.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>361</th>\n",
              "      <td>362</td>\n",
              "      <td>male</td>\n",
              "      <td>Hans Peter</td>\n",
              "      <td>Radislav</td>\n",
              "      <td>208 Birch  Street</td>\n",
              "      <td>El Paso</td>\n",
              "      <td>TX</td>\n",
              "      <td>79905.0</td>\n",
              "      <td>United States</td>\n",
              "      <td>HansPeterRadislav@gustr.com915-612-1462</td>\n",
              "      <td>10/28/1973</td>\n",
              "      <td>169.0</td>\n",
              "      <td>70</td>\n",
              "      <td>24.2</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>468</th>\n",
              "      <td>469</td>\n",
              "      <td>male</td>\n",
              "      <td>Tosh</td>\n",
              "      <td>Jensen</td>\n",
              "      <td>1536 John Daniel Drive</td>\n",
              "      <td>Argyle</td>\n",
              "      <td>MO</td>\n",
              "      <td>65001.0</td>\n",
              "      <td>United States</td>\n",
              "      <td>573-728-3861ToshCJensen@superrito.com</td>\n",
              "      <td>12/19/1969</td>\n",
              "      <td>194.7</td>\n",
              "      <td>68</td>\n",
              "      <td>29.6</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>502</th>\n",
              "      <td>503</td>\n",
              "      <td>male</td>\n",
              "      <td>Pat</td>\n",
              "      <td>Gersten</td>\n",
              "      <td>2778 North Avenue</td>\n",
              "      <td>Burr</td>\n",
              "      <td>Nebraska</td>\n",
              "      <td>68324.0</td>\n",
              "      <td>United States</td>\n",
              "      <td>PatrickGersten@rhyta.com402-848-4923</td>\n",
              "      <td>5/3/1954</td>\n",
              "      <td>138.2</td>\n",
              "      <td>71</td>\n",
              "      <td>19.3</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "    <div class=\"colab-df-buttons\">\n",
              "\n",
              "  <div class=\"colab-df-container\">\n",
              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-115d9592-c26d-4eea-b0aa-0774248cc08f')\"\n",
              "            title=\"Convert this dataframe to an interactive table.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    .colab-df-buttons div {\n",
              "      margin-bottom: 4px;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const buttonEl =\n",
              "        document.querySelector('#df-115d9592-c26d-4eea-b0aa-0774248cc08f button.colab-df-convert');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function convertToInteractive(key) {\n",
              "        const element = document.querySelector('#df-115d9592-c26d-4eea-b0aa-0774248cc08f');\n",
              "        const dataTable =\n",
              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                    [key], {});\n",
              "        if (!dataTable) return;\n",
              "\n",
              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "          + ' to learn more about interactive tables.';\n",
              "        element.innerHTML = '';\n",
              "        dataTable['output_type'] = 'display_data';\n",
              "        await google.colab.output.renderOutput(dataTable, element);\n",
              "        const docLink = document.createElement('div');\n",
              "        docLink.innerHTML = docLinkHtml;\n",
              "        element.appendChild(docLink);\n",
              "      }\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "\n",
              "<div id=\"df-546ee181-a329-418e-bf3a-c026118a9a5d\">\n",
              "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-546ee181-a329-418e-bf3a-c026118a9a5d')\"\n",
              "            title=\"Suggest charts\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "     width=\"24px\">\n",
              "    <g>\n",
              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "    </g>\n",
              "</svg>\n",
              "  </button>\n",
              "\n",
              "<style>\n",
              "  .colab-df-quickchart {\n",
              "      --bg-color: #E8F0FE;\n",
              "      --fill-color: #1967D2;\n",
              "      --hover-bg-color: #E2EBFA;\n",
              "      --hover-fill-color: #174EA6;\n",
              "      --disabled-fill-color: #AAA;\n",
              "      --disabled-bg-color: #DDD;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart {\n",
              "      --bg-color: #3B4455;\n",
              "      --fill-color: #D2E3FC;\n",
              "      --hover-bg-color: #434B5C;\n",
              "      --hover-fill-color: #FFFFFF;\n",
              "      --disabled-bg-color: #3B4455;\n",
              "      --disabled-fill-color: #666;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart {\n",
              "    background-color: var(--bg-color);\n",
              "    border: none;\n",
              "    border-radius: 50%;\n",
              "    cursor: pointer;\n",
              "    display: none;\n",
              "    fill: var(--fill-color);\n",
              "    height: 32px;\n",
              "    padding: 0;\n",
              "    width: 32px;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart:hover {\n",
              "    background-color: var(--hover-bg-color);\n",
              "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "    fill: var(--button-hover-fill-color);\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart-complete:disabled,\n",
              "  .colab-df-quickchart-complete:disabled:hover {\n",
              "    background-color: var(--disabled-bg-color);\n",
              "    fill: var(--disabled-fill-color);\n",
              "    box-shadow: none;\n",
              "  }\n",
              "\n",
              "  .colab-df-spinner {\n",
              "    border: 2px solid var(--fill-color);\n",
              "    border-color: transparent;\n",
              "    border-bottom-color: var(--fill-color);\n",
              "    animation:\n",
              "      spin 1s steps(1) infinite;\n",
              "  }\n",
              "\n",
              "  @keyframes spin {\n",
              "    0% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "      border-left-color: var(--fill-color);\n",
              "    }\n",
              "    20% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    30% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    40% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    60% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    80% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "    90% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "  }\n",
              "</style>\n",
              "\n",
              "  <script>\n",
              "    async function quickchart(key) {\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#' + key + ' button');\n",
              "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
              "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
              "      try {\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'suggestCharts', [key], {});\n",
              "      } catch (error) {\n",
              "        console.error('Error during call to suggestCharts:', error);\n",
              "      }\n",
              "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
              "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
              "    }\n",
              "    (() => {\n",
              "      let quickchartButtonEl =\n",
              "        document.querySelector('#df-546ee181-a329-418e-bf3a-c026118a9a5d button');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "    })();\n",
              "  </script>\n",
              "</div>\n",
              "\n",
              "    </div>\n",
              "  </div>\n"
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "dataframe",
              "repr_error": "0"
            }
          },
          "metadata": {},
          "execution_count": 158
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "adverse_reactions.info() #Gives missing values\n",
        "patients.info()\n",
        "treatments.info()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "arQVdf5Tjh_S",
        "outputId": "da40e28f-a80c-43cb-9aa5-b5e4b5ee00ac"
      },
      "execution_count": 159,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "<class 'pandas.core.frame.DataFrame'>\n",
            "RangeIndex: 34 entries, 0 to 33\n",
            "Data columns (total 3 columns):\n",
            " #   Column            Non-Null Count  Dtype \n",
            "---  ------            --------------  ----- \n",
            " 0   given_name        34 non-null     object\n",
            " 1   surname           34 non-null     object\n",
            " 2   adverse_reaction  34 non-null     object\n",
            "dtypes: object(3)\n",
            "memory usage: 944.0+ bytes\n",
            "<class 'pandas.core.frame.DataFrame'>\n",
            "RangeIndex: 503 entries, 0 to 502\n",
            "Data columns (total 14 columns):\n",
            " #   Column        Non-Null Count  Dtype  \n",
            "---  ------        --------------  -----  \n",
            " 0   patient_id    503 non-null    int64  \n",
            " 1   assigned_sex  503 non-null    object \n",
            " 2   given_name    503 non-null    object \n",
            " 3   surname       503 non-null    object \n",
            " 4   address       491 non-null    object \n",
            " 5   city          491 non-null    object \n",
            " 6   state         491 non-null    object \n",
            " 7   zip_code      491 non-null    float64\n",
            " 8   country       491 non-null    object \n",
            " 9   contact       491 non-null    object \n",
            " 10  birthdate     503 non-null    object \n",
            " 11  weight        503 non-null    float64\n",
            " 12  height        503 non-null    int64  \n",
            " 13  bmi           503 non-null    float64\n",
            "dtypes: float64(3), int64(2), object(9)\n",
            "memory usage: 55.1+ KB\n",
            "<class 'pandas.core.frame.DataFrame'>\n",
            "RangeIndex: 280 entries, 0 to 279\n",
            "Data columns (total 7 columns):\n",
            " #   Column        Non-Null Count  Dtype  \n",
            "---  ------        --------------  -----  \n",
            " 0   given_name    280 non-null    object \n",
            " 1   surname       280 non-null    object \n",
            " 2   auralin       280 non-null    object \n",
            " 3   novodra       280 non-null    object \n",
            " 4   hba1c_start   280 non-null    float64\n",
            " 5   hba1c_end     280 non-null    float64\n",
            " 6   hba1c_change  171 non-null    float64\n",
            "dtypes: float64(3), object(4)\n",
            "memory usage: 15.4+ KB\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "* assigned_sex - must be a category type\n",
        "* zipcode must be int type\n",
        "* birthdate must be datetime"
      ],
      "metadata": {
        "id": "sLxMj0NiDy5u"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "patients[patients['address'].isnull()] # data missing in 12 patients in these columns."
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 425
        },
        "id": "8fZRxjmjkSb8",
        "outputId": "02ef3ed2-143b-4715-9dda-9c80055ae112"
      },
      "execution_count": 160,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "     patient_id assigned_sex given_name      surname address city state  \\\n",
              "209         210       female     Lalita  Eldarkhanov     NaN  NaN   NaN   \n",
              "219         220         male         Mỹ        Quynh     NaN  NaN   NaN   \n",
              "230         231       female  Elisabeth      Knudsen     NaN  NaN   NaN   \n",
              "234         235       female    Martina    Tománková     NaN  NaN   NaN   \n",
              "242         243         male       John      O'Brian     NaN  NaN   NaN   \n",
              "249         250         male   Benjamin       Mehler     NaN  NaN   NaN   \n",
              "257         258         male        Jin         Kung     NaN  NaN   NaN   \n",
              "264         265       female   Wafiyyah       Asfour     NaN  NaN   NaN   \n",
              "269         270       female     Flavia   Fiorentino     NaN  NaN   NaN   \n",
              "278         279       female   Generosa        Cabán     NaN  NaN   NaN   \n",
              "286         287         male      Lewis         Webb     NaN  NaN   NaN   \n",
              "296         297       female        Chỉ          Lâm     NaN  NaN   NaN   \n",
              "\n",
              "     zip_code country contact   birthdate  weight  height   bmi  \n",
              "209       NaN     NaN     NaN   8/14/1950   143.4      62  26.2  \n",
              "219       NaN     NaN     NaN    4/9/1978   237.8      69  35.1  \n",
              "230       NaN     NaN     NaN   9/23/1976   165.9      63  29.4  \n",
              "234       NaN     NaN     NaN    4/7/1936   199.5      65  33.2  \n",
              "242       NaN     NaN     NaN   2/25/1957   205.3      74  26.4  \n",
              "249       NaN     NaN     NaN  10/30/1951   146.5      69  21.6  \n",
              "257       NaN     NaN     NaN   5/17/1995   231.7      69  34.2  \n",
              "264       NaN     NaN     NaN   11/3/1989   158.6      63  28.1  \n",
              "269       NaN     NaN     NaN   10/9/1937   175.2      61  33.1  \n",
              "278       NaN     NaN     NaN  12/16/1962   124.3      69  18.4  \n",
              "286       NaN     NaN     NaN    4/1/1979   155.3      68  23.6  \n",
              "296       NaN     NaN     NaN   5/14/1990   181.1      63  32.1  "
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-9b054ff6-dc52-4d2e-ae36-8922785f5486\" class=\"colab-df-container\">\n",
              "    <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>patient_id</th>\n",
              "      <th>assigned_sex</th>\n",
              "      <th>given_name</th>\n",
              "      <th>surname</th>\n",
              "      <th>address</th>\n",
              "      <th>city</th>\n",
              "      <th>state</th>\n",
              "      <th>zip_code</th>\n",
              "      <th>country</th>\n",
              "      <th>contact</th>\n",
              "      <th>birthdate</th>\n",
              "      <th>weight</th>\n",
              "      <th>height</th>\n",
              "      <th>bmi</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>209</th>\n",
              "      <td>210</td>\n",
              "      <td>female</td>\n",
              "      <td>Lalita</td>\n",
              "      <td>Eldarkhanov</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>8/14/1950</td>\n",
              "      <td>143.4</td>\n",
              "      <td>62</td>\n",
              "      <td>26.2</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>219</th>\n",
              "      <td>220</td>\n",
              "      <td>male</td>\n",
              "      <td>Mỹ</td>\n",
              "      <td>Quynh</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>4/9/1978</td>\n",
              "      <td>237.8</td>\n",
              "      <td>69</td>\n",
              "      <td>35.1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>230</th>\n",
              "      <td>231</td>\n",
              "      <td>female</td>\n",
              "      <td>Elisabeth</td>\n",
              "      <td>Knudsen</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>9/23/1976</td>\n",
              "      <td>165.9</td>\n",
              "      <td>63</td>\n",
              "      <td>29.4</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>234</th>\n",
              "      <td>235</td>\n",
              "      <td>female</td>\n",
              "      <td>Martina</td>\n",
              "      <td>Tománková</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>4/7/1936</td>\n",
              "      <td>199.5</td>\n",
              "      <td>65</td>\n",
              "      <td>33.2</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>242</th>\n",
              "      <td>243</td>\n",
              "      <td>male</td>\n",
              "      <td>John</td>\n",
              "      <td>O'Brian</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>2/25/1957</td>\n",
              "      <td>205.3</td>\n",
              "      <td>74</td>\n",
              "      <td>26.4</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>249</th>\n",
              "      <td>250</td>\n",
              "      <td>male</td>\n",
              "      <td>Benjamin</td>\n",
              "      <td>Mehler</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>10/30/1951</td>\n",
              "      <td>146.5</td>\n",
              "      <td>69</td>\n",
              "      <td>21.6</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>257</th>\n",
              "      <td>258</td>\n",
              "      <td>male</td>\n",
              "      <td>Jin</td>\n",
              "      <td>Kung</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>5/17/1995</td>\n",
              "      <td>231.7</td>\n",
              "      <td>69</td>\n",
              "      <td>34.2</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>264</th>\n",
              "      <td>265</td>\n",
              "      <td>female</td>\n",
              "      <td>Wafiyyah</td>\n",
              "      <td>Asfour</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>11/3/1989</td>\n",
              "      <td>158.6</td>\n",
              "      <td>63</td>\n",
              "      <td>28.1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>269</th>\n",
              "      <td>270</td>\n",
              "      <td>female</td>\n",
              "      <td>Flavia</td>\n",
              "      <td>Fiorentino</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>10/9/1937</td>\n",
              "      <td>175.2</td>\n",
              "      <td>61</td>\n",
              "      <td>33.1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>278</th>\n",
              "      <td>279</td>\n",
              "      <td>female</td>\n",
              "      <td>Generosa</td>\n",
              "      <td>Cabán</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>12/16/1962</td>\n",
              "      <td>124.3</td>\n",
              "      <td>69</td>\n",
              "      <td>18.4</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>286</th>\n",
              "      <td>287</td>\n",
              "      <td>male</td>\n",
              "      <td>Lewis</td>\n",
              "      <td>Webb</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>4/1/1979</td>\n",
              "      <td>155.3</td>\n",
              "      <td>68</td>\n",
              "      <td>23.6</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>296</th>\n",
              "      <td>297</td>\n",
              "      <td>female</td>\n",
              "      <td>Chỉ</td>\n",
              "      <td>Lâm</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>5/14/1990</td>\n",
              "      <td>181.1</td>\n",
              "      <td>63</td>\n",
              "      <td>32.1</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "    <div class=\"colab-df-buttons\">\n",
              "\n",
              "  <div class=\"colab-df-container\">\n",
              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-9b054ff6-dc52-4d2e-ae36-8922785f5486')\"\n",
              "            title=\"Convert this dataframe to an interactive table.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    .colab-df-buttons div {\n",
              "      margin-bottom: 4px;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const buttonEl =\n",
              "        document.querySelector('#df-9b054ff6-dc52-4d2e-ae36-8922785f5486 button.colab-df-convert');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function convertToInteractive(key) {\n",
              "        const element = document.querySelector('#df-9b054ff6-dc52-4d2e-ae36-8922785f5486');\n",
              "        const dataTable =\n",
              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                    [key], {});\n",
              "        if (!dataTable) return;\n",
              "\n",
              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "          + ' to learn more about interactive tables.';\n",
              "        element.innerHTML = '';\n",
              "        dataTable['output_type'] = 'display_data';\n",
              "        await google.colab.output.renderOutput(dataTable, element);\n",
              "        const docLink = document.createElement('div');\n",
              "        docLink.innerHTML = docLinkHtml;\n",
              "        element.appendChild(docLink);\n",
              "      }\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "\n",
              "<div id=\"df-e8ae31e8-f7c2-41c5-b72a-2577db0fd1c8\">\n",
              "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-e8ae31e8-f7c2-41c5-b72a-2577db0fd1c8')\"\n",
              "            title=\"Suggest charts\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "     width=\"24px\">\n",
              "    <g>\n",
              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "    </g>\n",
              "</svg>\n",
              "  </button>\n",
              "\n",
              "<style>\n",
              "  .colab-df-quickchart {\n",
              "      --bg-color: #E8F0FE;\n",
              "      --fill-color: #1967D2;\n",
              "      --hover-bg-color: #E2EBFA;\n",
              "      --hover-fill-color: #174EA6;\n",
              "      --disabled-fill-color: #AAA;\n",
              "      --disabled-bg-color: #DDD;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart {\n",
              "      --bg-color: #3B4455;\n",
              "      --fill-color: #D2E3FC;\n",
              "      --hover-bg-color: #434B5C;\n",
              "      --hover-fill-color: #FFFFFF;\n",
              "      --disabled-bg-color: #3B4455;\n",
              "      --disabled-fill-color: #666;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart {\n",
              "    background-color: var(--bg-color);\n",
              "    border: none;\n",
              "    border-radius: 50%;\n",
              "    cursor: pointer;\n",
              "    display: none;\n",
              "    fill: var(--fill-color);\n",
              "    height: 32px;\n",
              "    padding: 0;\n",
              "    width: 32px;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart:hover {\n",
              "    background-color: var(--hover-bg-color);\n",
              "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "    fill: var(--button-hover-fill-color);\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart-complete:disabled,\n",
              "  .colab-df-quickchart-complete:disabled:hover {\n",
              "    background-color: var(--disabled-bg-color);\n",
              "    fill: var(--disabled-fill-color);\n",
              "    box-shadow: none;\n",
              "  }\n",
              "\n",
              "  .colab-df-spinner {\n",
              "    border: 2px solid var(--fill-color);\n",
              "    border-color: transparent;\n",
              "    border-bottom-color: var(--fill-color);\n",
              "    animation:\n",
              "      spin 1s steps(1) infinite;\n",
              "  }\n",
              "\n",
              "  @keyframes spin {\n",
              "    0% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "      border-left-color: var(--fill-color);\n",
              "    }\n",
              "    20% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    30% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    40% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    60% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    80% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "    90% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "  }\n",
              "</style>\n",
              "\n",
              "  <script>\n",
              "    async function quickchart(key) {\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#' + key + ' button');\n",
              "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
              "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
              "      try {\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'suggestCharts', [key], {});\n",
              "      } catch (error) {\n",
              "        console.error('Error during call to suggestCharts:', error);\n",
              "      }\n",
              "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
              "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
              "    }\n",
              "    (() => {\n",
              "      let quickchartButtonEl =\n",
              "        document.querySelector('#df-e8ae31e8-f7c2-41c5-b72a-2577db0fd1c8 button');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "    })();\n",
              "  </script>\n",
              "</div>\n",
              "\n",
              "    </div>\n",
              "  </div>\n"
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "dataframe",
              "repr_error": "0"
            }
          },
          "metadata": {},
          "execution_count": 160
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "patients['patient_id'].duplicated().sum()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Wy32jL4yJTHX",
        "outputId": "3593a059-6c6b-4e18-a679-a6e599f076b2"
      },
      "execution_count": 161,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0"
            ]
          },
          "metadata": {},
          "execution_count": 161
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "patients[patients.duplicated(subset=['given_name', 'surname'])]"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 206
        },
        "id": "iTaNhMrwKb_6",
        "outputId": "bac82e0f-ae11-4fcf-d0db-d6564fc9fa8e"
      },
      "execution_count": 162,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "     patient_id assigned_sex given_name surname          address      city  \\\n",
              "229         230         male       John     Doe  123 Main Street  New York   \n",
              "237         238         male       John     Doe  123 Main Street  New York   \n",
              "244         245         male       John     Doe  123 Main Street  New York   \n",
              "251         252         male       John     Doe  123 Main Street  New York   \n",
              "277         278         male       John     Doe  123 Main Street  New York   \n",
              "\n",
              "    state  zip_code        country                      contact birthdate  \\\n",
              "229    NY   12345.0  United States  johndoe@email.com1234567890  1/1/1975   \n",
              "237    NY   12345.0  United States  johndoe@email.com1234567890  1/1/1975   \n",
              "244    NY   12345.0  United States  johndoe@email.com1234567890  1/1/1975   \n",
              "251    NY   12345.0  United States  johndoe@email.com1234567890  1/1/1975   \n",
              "277    NY   12345.0  United States  johndoe@email.com1234567890  1/1/1975   \n",
              "\n",
              "     weight  height   bmi  \n",
              "229   180.0      72  24.4  \n",
              "237   180.0      72  24.4  \n",
              "244   180.0      72  24.4  \n",
              "251   180.0      72  24.4  \n",
              "277   180.0      72  24.4  "
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-3892482e-52cd-4d99-b3e1-1444ee266435\" class=\"colab-df-container\">\n",
              "    <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>patient_id</th>\n",
              "      <th>assigned_sex</th>\n",
              "      <th>given_name</th>\n",
              "      <th>surname</th>\n",
              "      <th>address</th>\n",
              "      <th>city</th>\n",
              "      <th>state</th>\n",
              "      <th>zip_code</th>\n",
              "      <th>country</th>\n",
              "      <th>contact</th>\n",
              "      <th>birthdate</th>\n",
              "      <th>weight</th>\n",
              "      <th>height</th>\n",
              "      <th>bmi</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>229</th>\n",
              "      <td>230</td>\n",
              "      <td>male</td>\n",
              "      <td>John</td>\n",
              "      <td>Doe</td>\n",
              "      <td>123 Main Street</td>\n",
              "      <td>New York</td>\n",
              "      <td>NY</td>\n",
              "      <td>12345.0</td>\n",
              "      <td>United States</td>\n",
              "      <td>johndoe@email.com1234567890</td>\n",
              "      <td>1/1/1975</td>\n",
              "      <td>180.0</td>\n",
              "      <td>72</td>\n",
              "      <td>24.4</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>237</th>\n",
              "      <td>238</td>\n",
              "      <td>male</td>\n",
              "      <td>John</td>\n",
              "      <td>Doe</td>\n",
              "      <td>123 Main Street</td>\n",
              "      <td>New York</td>\n",
              "      <td>NY</td>\n",
              "      <td>12345.0</td>\n",
              "      <td>United States</td>\n",
              "      <td>johndoe@email.com1234567890</td>\n",
              "      <td>1/1/1975</td>\n",
              "      <td>180.0</td>\n",
              "      <td>72</td>\n",
              "      <td>24.4</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>244</th>\n",
              "      <td>245</td>\n",
              "      <td>male</td>\n",
              "      <td>John</td>\n",
              "      <td>Doe</td>\n",
              "      <td>123 Main Street</td>\n",
              "      <td>New York</td>\n",
              "      <td>NY</td>\n",
              "      <td>12345.0</td>\n",
              "      <td>United States</td>\n",
              "      <td>johndoe@email.com1234567890</td>\n",
              "      <td>1/1/1975</td>\n",
              "      <td>180.0</td>\n",
              "      <td>72</td>\n",
              "      <td>24.4</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>251</th>\n",
              "      <td>252</td>\n",
              "      <td>male</td>\n",
              "      <td>John</td>\n",
              "      <td>Doe</td>\n",
              "      <td>123 Main Street</td>\n",
              "      <td>New York</td>\n",
              "      <td>NY</td>\n",
              "      <td>12345.0</td>\n",
              "      <td>United States</td>\n",
              "      <td>johndoe@email.com1234567890</td>\n",
              "      <td>1/1/1975</td>\n",
              "      <td>180.0</td>\n",
              "      <td>72</td>\n",
              "      <td>24.4</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>277</th>\n",
              "      <td>278</td>\n",
              "      <td>male</td>\n",
              "      <td>John</td>\n",
              "      <td>Doe</td>\n",
              "      <td>123 Main Street</td>\n",
              "      <td>New York</td>\n",
              "      <td>NY</td>\n",
              "      <td>12345.0</td>\n",
              "      <td>United States</td>\n",
              "      <td>johndoe@email.com1234567890</td>\n",
              "      <td>1/1/1975</td>\n",
              "      <td>180.0</td>\n",
              "      <td>72</td>\n",
              "      <td>24.4</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "    <div class=\"colab-df-buttons\">\n",
              "\n",
              "  <div class=\"colab-df-container\">\n",
              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-3892482e-52cd-4d99-b3e1-1444ee266435')\"\n",
              "            title=\"Convert this dataframe to an interactive table.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    .colab-df-buttons div {\n",
              "      margin-bottom: 4px;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const buttonEl =\n",
              "        document.querySelector('#df-3892482e-52cd-4d99-b3e1-1444ee266435 button.colab-df-convert');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function convertToInteractive(key) {\n",
              "        const element = document.querySelector('#df-3892482e-52cd-4d99-b3e1-1444ee266435');\n",
              "        const dataTable =\n",
              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                    [key], {});\n",
              "        if (!dataTable) return;\n",
              "\n",
              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "          + ' to learn more about interactive tables.';\n",
              "        element.innerHTML = '';\n",
              "        dataTable['output_type'] = 'display_data';\n",
              "        await google.colab.output.renderOutput(dataTable, element);\n",
              "        const docLink = document.createElement('div');\n",
              "        docLink.innerHTML = docLinkHtml;\n",
              "        element.appendChild(docLink);\n",
              "      }\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "\n",
              "<div id=\"df-22962a55-bc0c-4c5d-9855-b27fb775d63d\">\n",
              "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-22962a55-bc0c-4c5d-9855-b27fb775d63d')\"\n",
              "            title=\"Suggest charts\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "     width=\"24px\">\n",
              "    <g>\n",
              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "    </g>\n",
              "</svg>\n",
              "  </button>\n",
              "\n",
              "<style>\n",
              "  .colab-df-quickchart {\n",
              "      --bg-color: #E8F0FE;\n",
              "      --fill-color: #1967D2;\n",
              "      --hover-bg-color: #E2EBFA;\n",
              "      --hover-fill-color: #174EA6;\n",
              "      --disabled-fill-color: #AAA;\n",
              "      --disabled-bg-color: #DDD;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart {\n",
              "      --bg-color: #3B4455;\n",
              "      --fill-color: #D2E3FC;\n",
              "      --hover-bg-color: #434B5C;\n",
              "      --hover-fill-color: #FFFFFF;\n",
              "      --disabled-bg-color: #3B4455;\n",
              "      --disabled-fill-color: #666;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart {\n",
              "    background-color: var(--bg-color);\n",
              "    border: none;\n",
              "    border-radius: 50%;\n",
              "    cursor: pointer;\n",
              "    display: none;\n",
              "    fill: var(--fill-color);\n",
              "    height: 32px;\n",
              "    padding: 0;\n",
              "    width: 32px;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart:hover {\n",
              "    background-color: var(--hover-bg-color);\n",
              "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "    fill: var(--button-hover-fill-color);\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart-complete:disabled,\n",
              "  .colab-df-quickchart-complete:disabled:hover {\n",
              "    background-color: var(--disabled-bg-color);\n",
              "    fill: var(--disabled-fill-color);\n",
              "    box-shadow: none;\n",
              "  }\n",
              "\n",
              "  .colab-df-spinner {\n",
              "    border: 2px solid var(--fill-color);\n",
              "    border-color: transparent;\n",
              "    border-bottom-color: var(--fill-color);\n",
              "    animation:\n",
              "      spin 1s steps(1) infinite;\n",
              "  }\n",
              "\n",
              "  @keyframes spin {\n",
              "    0% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "      border-left-color: var(--fill-color);\n",
              "    }\n",
              "    20% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    30% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    40% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    60% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    80% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "    90% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "  }\n",
              "</style>\n",
              "\n",
              "  <script>\n",
              "    async function quickchart(key) {\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#' + key + ' button');\n",
              "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
              "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
              "      try {\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'suggestCharts', [key], {});\n",
              "      } catch (error) {\n",
              "        console.error('Error during call to suggestCharts:', error);\n",
              "      }\n",
              "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
              "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
              "    }\n",
              "    (() => {\n",
              "      let quickchartButtonEl =\n",
              "        document.querySelector('#df-22962a55-bc0c-4c5d-9855-b27fb775d63d button');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "    })();\n",
              "  </script>\n",
              "</div>\n",
              "\n",
              "    </div>\n",
              "  </div>\n"
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "dataframe",
              "repr_error": "0"
            }
          },
          "metadata": {},
          "execution_count": 162
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "* there are multiple duplicate entries by the name of john doe - 'consistency'"
      ],
      "metadata": {
        "id": "pM8H3h8eKx4j"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "treatments[treatments.duplicated()]"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 81
        },
        "id": "gERedpODkSon",
        "outputId": "c54e1159-4956-493b-f3d3-9bb52a97ddcf"
      },
      "execution_count": 163,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "    given_name surname    auralin novodra  hba1c_start  hba1c_end  \\\n",
              "136     joseph     day  29u - 36u       -          7.7       7.19   \n",
              "\n",
              "     hba1c_change  \n",
              "136           NaN  "
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-e68450c1-af38-44b0-b9c2-ee618d933901\" class=\"colab-df-container\">\n",
              "    <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>given_name</th>\n",
              "      <th>surname</th>\n",
              "      <th>auralin</th>\n",
              "      <th>novodra</th>\n",
              "      <th>hba1c_start</th>\n",
              "      <th>hba1c_end</th>\n",
              "      <th>hba1c_change</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>136</th>\n",
              "      <td>joseph</td>\n",
              "      <td>day</td>\n",
              "      <td>29u - 36u</td>\n",
              "      <td>-</td>\n",
              "      <td>7.7</td>\n",
              "      <td>7.19</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "    <div class=\"colab-df-buttons\">\n",
              "\n",
              "  <div class=\"colab-df-container\">\n",
              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-e68450c1-af38-44b0-b9c2-ee618d933901')\"\n",
              "            title=\"Convert this dataframe to an interactive table.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    .colab-df-buttons div {\n",
              "      margin-bottom: 4px;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const buttonEl =\n",
              "        document.querySelector('#df-e68450c1-af38-44b0-b9c2-ee618d933901 button.colab-df-convert');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function convertToInteractive(key) {\n",
              "        const element = document.querySelector('#df-e68450c1-af38-44b0-b9c2-ee618d933901');\n",
              "        const dataTable =\n",
              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                    [key], {});\n",
              "        if (!dataTable) return;\n",
              "\n",
              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "          + ' to learn more about interactive tables.';\n",
              "        element.innerHTML = '';\n",
              "        dataTable['output_type'] = 'display_data';\n",
              "        await google.colab.output.renderOutput(dataTable, element);\n",
              "        const docLink = document.createElement('div');\n",
              "        docLink.innerHTML = docLinkHtml;\n",
              "        element.appendChild(docLink);\n",
              "      }\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "\n",
              "    </div>\n",
              "  </div>\n"
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "dataframe",
              "summary": "{\n  \"name\": \"treatments[treatments\",\n  \"rows\": 1,\n  \"fields\": [\n    {\n      \"column\": \"given_name\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 1,\n        \"samples\": [\n          \"joseph\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"surname\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 1,\n        \"samples\": [\n          \"day\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"auralin\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 1,\n        \"samples\": [\n          \"29u - 36u\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"novodra\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 1,\n        \"samples\": [\n          \"-\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"hba1c_start\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": null,\n        \"min\": 7.7,\n        \"max\": 7.7,\n        \"num_unique_values\": 1,\n        \"samples\": [\n          7.7\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"hba1c_end\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": null,\n        \"min\": 7.19,\n        \"max\": 7.19,\n        \"num_unique_values\": 1,\n        \"samples\": [\n          7.19\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"hba1c_change\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": null,\n        \"min\": null,\n        \"max\": null,\n        \"num_unique_values\": 0,\n        \"samples\": [],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    }\n  ]\n}"
            }
          },
          "metadata": {},
          "execution_count": 163
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "treatments[treatments.duplicated(subset=['given_name'\t,'surname'])]"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 81
        },
        "id": "Uvu6qhPxkSv_",
        "outputId": "b667ab4e-1cb2-4435-8212-a2332a9b7c1d"
      },
      "execution_count": 164,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "    given_name surname    auralin novodra  hba1c_start  hba1c_end  \\\n",
              "136     joseph     day  29u - 36u       -          7.7       7.19   \n",
              "\n",
              "     hba1c_change  \n",
              "136           NaN  "
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-e18f46b4-a1fc-4e64-ab58-9ad01679f412\" class=\"colab-df-container\">\n",
              "    <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>given_name</th>\n",
              "      <th>surname</th>\n",
              "      <th>auralin</th>\n",
              "      <th>novodra</th>\n",
              "      <th>hba1c_start</th>\n",
              "      <th>hba1c_end</th>\n",
              "      <th>hba1c_change</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>136</th>\n",
              "      <td>joseph</td>\n",
              "      <td>day</td>\n",
              "      <td>29u - 36u</td>\n",
              "      <td>-</td>\n",
              "      <td>7.7</td>\n",
              "      <td>7.19</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "    <div class=\"colab-df-buttons\">\n",
              "\n",
              "  <div class=\"colab-df-container\">\n",
              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-e18f46b4-a1fc-4e64-ab58-9ad01679f412')\"\n",
              "            title=\"Convert this dataframe to an interactive table.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    .colab-df-buttons div {\n",
              "      margin-bottom: 4px;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const buttonEl =\n",
              "        document.querySelector('#df-e18f46b4-a1fc-4e64-ab58-9ad01679f412 button.colab-df-convert');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function convertToInteractive(key) {\n",
              "        const element = document.querySelector('#df-e18f46b4-a1fc-4e64-ab58-9ad01679f412');\n",
              "        const dataTable =\n",
              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                    [key], {});\n",
              "        if (!dataTable) return;\n",
              "\n",
              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "          + ' to learn more about interactive tables.';\n",
              "        element.innerHTML = '';\n",
              "        dataTable['output_type'] = 'display_data';\n",
              "        await google.colab.output.renderOutput(dataTable, element);\n",
              "        const docLink = document.createElement('div');\n",
              "        docLink.innerHTML = docLinkHtml;\n",
              "        element.appendChild(docLink);\n",
              "      }\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "\n",
              "    </div>\n",
              "  </div>\n"
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "dataframe",
              "summary": "{\n  \"name\": \"treatments[treatments\",\n  \"rows\": 1,\n  \"fields\": [\n    {\n      \"column\": \"given_name\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 1,\n        \"samples\": [\n          \"joseph\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"surname\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 1,\n        \"samples\": [\n          \"day\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"auralin\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 1,\n        \"samples\": [\n          \"29u - 36u\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"novodra\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 1,\n        \"samples\": [\n          \"-\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"hba1c_start\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": null,\n        \"min\": 7.7,\n        \"max\": 7.7,\n        \"num_unique_values\": 1,\n        \"samples\": [\n          7.7\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"hba1c_end\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": null,\n        \"min\": 7.19,\n        \"max\": 7.19,\n        \"num_unique_values\": 1,\n        \"samples\": [\n          7.19\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"hba1c_change\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": null,\n        \"min\": null,\n        \"max\": null,\n        \"num_unique_values\": 0,\n        \"samples\": [],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    }\n  ]\n}"
            }
          },
          "metadata": {},
          "execution_count": 164
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "treatments_cut[treatments_cut.duplicated(subset=['given_name'\t,'surname'])]"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 53
        },
        "id": "brDwgr0DkS2j",
        "outputId": "a7f7c6ec-5d84-4c54-c7ca-6410df40d455"
      },
      "execution_count": 165,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "Empty DataFrame\n",
              "Columns: [given_name, surname, auralin, novodra, hba1c_start, hba1c_end, hba1c_change]\n",
              "Index: []"
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-0e17804d-30e8-4940-a060-d6b847b7e404\" class=\"colab-df-container\">\n",
              "    <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>given_name</th>\n",
              "      <th>surname</th>\n",
              "      <th>auralin</th>\n",
              "      <th>novodra</th>\n",
              "      <th>hba1c_start</th>\n",
              "      <th>hba1c_end</th>\n",
              "      <th>hba1c_change</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "    <div class=\"colab-df-buttons\">\n",
              "\n",
              "  <div class=\"colab-df-container\">\n",
              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-0e17804d-30e8-4940-a060-d6b847b7e404')\"\n",
              "            title=\"Convert this dataframe to an interactive table.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    .colab-df-buttons div {\n",
              "      margin-bottom: 4px;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const buttonEl =\n",
              "        document.querySelector('#df-0e17804d-30e8-4940-a060-d6b847b7e404 button.colab-df-convert');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function convertToInteractive(key) {\n",
              "        const element = document.querySelector('#df-0e17804d-30e8-4940-a060-d6b847b7e404');\n",
              "        const dataTable =\n",
              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                    [key], {});\n",
              "        if (!dataTable) return;\n",
              "\n",
              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "          + ' to learn more about interactive tables.';\n",
              "        element.innerHTML = '';\n",
              "        dataTable['output_type'] = 'display_data';\n",
              "        await google.colab.output.renderOutput(dataTable, element);\n",
              "        const docLink = document.createElement('div');\n",
              "        docLink.innerHTML = docLinkHtml;\n",
              "        element.appendChild(docLink);\n",
              "      }\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "\n",
              "    </div>\n",
              "  </div>\n"
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "dataframe",
              "repr_error": "Out of range float values are not JSON compliant: nan"
            }
          },
          "metadata": {},
          "execution_count": 165
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "adverse_reactions.duplicated().sum()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "PGLuqOo1kS8l",
        "outputId": "a06cb43c-47c6-442b-ebd2-05a5cb56d1d1"
      },
      "execution_count": 166,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0"
            ]
          },
          "metadata": {},
          "execution_count": 166
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "patients.describe()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 300
        },
        "id": "qN41NTPnkTCg",
        "outputId": "cff2a83f-4b06-4e47-f9e2-77cd9c6a5665"
      },
      "execution_count": 167,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "       patient_id      zip_code      weight      height         bmi\n",
              "count  503.000000    491.000000  503.000000  503.000000  503.000000\n",
              "mean   252.000000  49084.118126  173.434990   66.634195   27.483897\n",
              "std    145.347859  30265.807442   33.916741    4.411297    5.276438\n",
              "min      1.000000   1002.000000   48.800000   27.000000   17.100000\n",
              "25%    126.500000  21920.500000  149.300000   63.000000   23.300000\n",
              "50%    252.000000  48057.000000  175.300000   67.000000   27.200000\n",
              "75%    377.500000  75679.000000  199.500000   70.000000   31.750000\n",
              "max    503.000000  99701.000000  255.900000   79.000000   37.700000"
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-c326d492-82e0-4466-9309-dd4648287e36\" class=\"colab-df-container\">\n",
              "    <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>patient_id</th>\n",
              "      <th>zip_code</th>\n",
              "      <th>weight</th>\n",
              "      <th>height</th>\n",
              "      <th>bmi</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>count</th>\n",
              "      <td>503.000000</td>\n",
              "      <td>491.000000</td>\n",
              "      <td>503.000000</td>\n",
              "      <td>503.000000</td>\n",
              "      <td>503.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>mean</th>\n",
              "      <td>252.000000</td>\n",
              "      <td>49084.118126</td>\n",
              "      <td>173.434990</td>\n",
              "      <td>66.634195</td>\n",
              "      <td>27.483897</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>std</th>\n",
              "      <td>145.347859</td>\n",
              "      <td>30265.807442</td>\n",
              "      <td>33.916741</td>\n",
              "      <td>4.411297</td>\n",
              "      <td>5.276438</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>min</th>\n",
              "      <td>1.000000</td>\n",
              "      <td>1002.000000</td>\n",
              "      <td>48.800000</td>\n",
              "      <td>27.000000</td>\n",
              "      <td>17.100000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>25%</th>\n",
              "      <td>126.500000</td>\n",
              "      <td>21920.500000</td>\n",
              "      <td>149.300000</td>\n",
              "      <td>63.000000</td>\n",
              "      <td>23.300000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>50%</th>\n",
              "      <td>252.000000</td>\n",
              "      <td>48057.000000</td>\n",
              "      <td>175.300000</td>\n",
              "      <td>67.000000</td>\n",
              "      <td>27.200000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>75%</th>\n",
              "      <td>377.500000</td>\n",
              "      <td>75679.000000</td>\n",
              "      <td>199.500000</td>\n",
              "      <td>70.000000</td>\n",
              "      <td>31.750000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>max</th>\n",
              "      <td>503.000000</td>\n",
              "      <td>99701.000000</td>\n",
              "      <td>255.900000</td>\n",
              "      <td>79.000000</td>\n",
              "      <td>37.700000</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "    <div class=\"colab-df-buttons\">\n",
              "\n",
              "  <div class=\"colab-df-container\">\n",
              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-c326d492-82e0-4466-9309-dd4648287e36')\"\n",
              "            title=\"Convert this dataframe to an interactive table.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    .colab-df-buttons div {\n",
              "      margin-bottom: 4px;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const buttonEl =\n",
              "        document.querySelector('#df-c326d492-82e0-4466-9309-dd4648287e36 button.colab-df-convert');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function convertToInteractive(key) {\n",
              "        const element = document.querySelector('#df-c326d492-82e0-4466-9309-dd4648287e36');\n",
              "        const dataTable =\n",
              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                    [key], {});\n",
              "        if (!dataTable) return;\n",
              "\n",
              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "          + ' to learn more about interactive tables.';\n",
              "        element.innerHTML = '';\n",
              "        dataTable['output_type'] = 'display_data';\n",
              "        await google.colab.output.renderOutput(dataTable, element);\n",
              "        const docLink = document.createElement('div');\n",
              "        docLink.innerHTML = docLinkHtml;\n",
              "        element.appendChild(docLink);\n",
              "      }\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "\n",
              "<div id=\"df-a6157e5f-fd35-4b55-b546-a0c1ea06515e\">\n",
              "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-a6157e5f-fd35-4b55-b546-a0c1ea06515e')\"\n",
              "            title=\"Suggest charts\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "     width=\"24px\">\n",
              "    <g>\n",
              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "    </g>\n",
              "</svg>\n",
              "  </button>\n",
              "\n",
              "<style>\n",
              "  .colab-df-quickchart {\n",
              "      --bg-color: #E8F0FE;\n",
              "      --fill-color: #1967D2;\n",
              "      --hover-bg-color: #E2EBFA;\n",
              "      --hover-fill-color: #174EA6;\n",
              "      --disabled-fill-color: #AAA;\n",
              "      --disabled-bg-color: #DDD;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart {\n",
              "      --bg-color: #3B4455;\n",
              "      --fill-color: #D2E3FC;\n",
              "      --hover-bg-color: #434B5C;\n",
              "      --hover-fill-color: #FFFFFF;\n",
              "      --disabled-bg-color: #3B4455;\n",
              "      --disabled-fill-color: #666;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart {\n",
              "    background-color: var(--bg-color);\n",
              "    border: none;\n",
              "    border-radius: 50%;\n",
              "    cursor: pointer;\n",
              "    display: none;\n",
              "    fill: var(--fill-color);\n",
              "    height: 32px;\n",
              "    padding: 0;\n",
              "    width: 32px;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart:hover {\n",
              "    background-color: var(--hover-bg-color);\n",
              "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "    fill: var(--button-hover-fill-color);\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart-complete:disabled,\n",
              "  .colab-df-quickchart-complete:disabled:hover {\n",
              "    background-color: var(--disabled-bg-color);\n",
              "    fill: var(--disabled-fill-color);\n",
              "    box-shadow: none;\n",
              "  }\n",
              "\n",
              "  .colab-df-spinner {\n",
              "    border: 2px solid var(--fill-color);\n",
              "    border-color: transparent;\n",
              "    border-bottom-color: var(--fill-color);\n",
              "    animation:\n",
              "      spin 1s steps(1) infinite;\n",
              "  }\n",
              "\n",
              "  @keyframes spin {\n",
              "    0% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "      border-left-color: var(--fill-color);\n",
              "    }\n",
              "    20% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    30% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    40% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    60% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    80% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "    90% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "  }\n",
              "</style>\n",
              "\n",
              "  <script>\n",
              "    async function quickchart(key) {\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#' + key + ' button');\n",
              "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
              "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
              "      try {\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'suggestCharts', [key], {});\n",
              "      } catch (error) {\n",
              "        console.error('Error during call to suggestCharts:', error);\n",
              "      }\n",
              "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
              "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
              "    }\n",
              "    (() => {\n",
              "      let quickchartButtonEl =\n",
              "        document.querySelector('#df-a6157e5f-fd35-4b55-b546-a0c1ea06515e button');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "    })();\n",
              "  </script>\n",
              "</div>\n",
              "\n",
              "    </div>\n",
              "  </div>\n"
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "dataframe",
              "summary": "{\n  \"name\": \"patients\",\n  \"rows\": 8,\n  \"fields\": [\n    {\n      \"column\": \"patient_id\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 180.97894164379872,\n        \"min\": 1.0,\n        \"max\": 503.0,\n        \"num_unique_values\": 6,\n        \"samples\": [\n          503.0,\n          252.0,\n          377.5\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"zip_code\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 34812.27029878351,\n        \"min\": 491.0,\n        \"max\": 99701.0,\n        \"num_unique_values\": 8,\n        \"samples\": [\n          49084.118126272915,\n          48057.0,\n          491.0\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"weight\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 145.79957601759185,\n        \"min\": 33.91674139657365,\n        \"max\": 503.0,\n        \"num_unique_values\": 8,\n        \"samples\": [\n          173.43499005964213,\n          175.3,\n          503.0\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"height\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 160.79559646498527,\n        \"min\": 4.4112974866528925,\n        \"max\": 503.0,\n        \"num_unique_values\": 8,\n        \"samples\": [\n          66.63419483101391,\n          67.0,\n          503.0\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"bmi\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 169.54244289734334,\n        \"min\": 5.2764378079801695,\n        \"max\": 503.0,\n        \"num_unique_values\": 8,\n        \"samples\": [\n          27.48389662027833,\n          27.2,\n          503.0\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    }\n  ]\n}"
            }
          },
          "metadata": {},
          "execution_count": 167
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "patients[patients['weight']==48.8]"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 98
        },
        "id": "Asz9is3oLujp",
        "outputId": "15865b62-e99a-4397-dc83-4c284fa66dd0"
      },
      "execution_count": 168,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "     patient_id assigned_sex given_name   surname              address  \\\n",
              "210         211       female    Camilla  Zaitseva  4689 Briarhill Lane   \n",
              "\n",
              "        city state  zip_code        country  \\\n",
              "210  Wooster    OH   44691.0  United States   \n",
              "\n",
              "                                       contact   birthdate  weight  height  \\\n",
              "210  330-202-2145CamillaZaitseva@superrito.com  11/26/1938    48.8      63   \n",
              "\n",
              "      bmi  \n",
              "210  19.1  "
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-de38e5ae-339f-4a6a-93ed-a5a2750f8d55\" class=\"colab-df-container\">\n",
              "    <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>patient_id</th>\n",
              "      <th>assigned_sex</th>\n",
              "      <th>given_name</th>\n",
              "      <th>surname</th>\n",
              "      <th>address</th>\n",
              "      <th>city</th>\n",
              "      <th>state</th>\n",
              "      <th>zip_code</th>\n",
              "      <th>country</th>\n",
              "      <th>contact</th>\n",
              "      <th>birthdate</th>\n",
              "      <th>weight</th>\n",
              "      <th>height</th>\n",
              "      <th>bmi</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>210</th>\n",
              "      <td>211</td>\n",
              "      <td>female</td>\n",
              "      <td>Camilla</td>\n",
              "      <td>Zaitseva</td>\n",
              "      <td>4689 Briarhill Lane</td>\n",
              "      <td>Wooster</td>\n",
              "      <td>OH</td>\n",
              "      <td>44691.0</td>\n",
              "      <td>United States</td>\n",
              "      <td>330-202-2145CamillaZaitseva@superrito.com</td>\n",
              "      <td>11/26/1938</td>\n",
              "      <td>48.8</td>\n",
              "      <td>63</td>\n",
              "      <td>19.1</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "    <div class=\"colab-df-buttons\">\n",
              "\n",
              "  <div class=\"colab-df-container\">\n",
              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-de38e5ae-339f-4a6a-93ed-a5a2750f8d55')\"\n",
              "            title=\"Convert this dataframe to an interactive table.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    .colab-df-buttons div {\n",
              "      margin-bottom: 4px;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const buttonEl =\n",
              "        document.querySelector('#df-de38e5ae-339f-4a6a-93ed-a5a2750f8d55 button.colab-df-convert');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function convertToInteractive(key) {\n",
              "        const element = document.querySelector('#df-de38e5ae-339f-4a6a-93ed-a5a2750f8d55');\n",
              "        const dataTable =\n",
              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                    [key], {});\n",
              "        if (!dataTable) return;\n",
              "\n",
              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "          + ' to learn more about interactive tables.';\n",
              "        element.innerHTML = '';\n",
              "        dataTable['output_type'] = 'display_data';\n",
              "        await google.colab.output.renderOutput(dataTable, element);\n",
              "        const docLink = document.createElement('div');\n",
              "        docLink.innerHTML = docLinkHtml;\n",
              "        element.appendChild(docLink);\n",
              "      }\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "\n",
              "    </div>\n",
              "  </div>\n"
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "dataframe",
              "repr_error": "0"
            }
          },
          "metadata": {},
          "execution_count": 168
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "* one patient has weight equals 48 pounds - 'accuracy'\n"
      ],
      "metadata": {
        "id": "zXx2rcJmMSv7"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "patients[patients['height'] == 27]"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 81
        },
        "id": "_P_B7ICikTIj",
        "outputId": "03702be6-fd1e-4c48-f2a9-de15ad2d2a7a"
      },
      "execution_count": 169,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "   patient_id assigned_sex given_name  surname               address    city  \\\n",
              "4           5         male        Tim  Neudorf  1428 Turkey Pen Lane  Dothan   \n",
              "\n",
              "  state  zip_code        country                          contact  birthdate  \\\n",
              "4    AL   36303.0  United States  334-515-7487TimNeudorf@cuvox.de  2/18/1928   \n",
              "\n",
              "   weight  height   bmi  \n",
              "4   192.3      27  26.1  "
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-e7d7e2c2-a21f-44ed-b3de-dd26d38744a3\" class=\"colab-df-container\">\n",
              "    <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>patient_id</th>\n",
              "      <th>assigned_sex</th>\n",
              "      <th>given_name</th>\n",
              "      <th>surname</th>\n",
              "      <th>address</th>\n",
              "      <th>city</th>\n",
              "      <th>state</th>\n",
              "      <th>zip_code</th>\n",
              "      <th>country</th>\n",
              "      <th>contact</th>\n",
              "      <th>birthdate</th>\n",
              "      <th>weight</th>\n",
              "      <th>height</th>\n",
              "      <th>bmi</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>5</td>\n",
              "      <td>male</td>\n",
              "      <td>Tim</td>\n",
              "      <td>Neudorf</td>\n",
              "      <td>1428 Turkey Pen Lane</td>\n",
              "      <td>Dothan</td>\n",
              "      <td>AL</td>\n",
              "      <td>36303.0</td>\n",
              "      <td>United States</td>\n",
              "      <td>334-515-7487TimNeudorf@cuvox.de</td>\n",
              "      <td>2/18/1928</td>\n",
              "      <td>192.3</td>\n",
              "      <td>27</td>\n",
              "      <td>26.1</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "    <div class=\"colab-df-buttons\">\n",
              "\n",
              "  <div class=\"colab-df-container\">\n",
              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-e7d7e2c2-a21f-44ed-b3de-dd26d38744a3')\"\n",
              "            title=\"Convert this dataframe to an interactive table.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    .colab-df-buttons div {\n",
              "      margin-bottom: 4px;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const buttonEl =\n",
              "        document.querySelector('#df-e7d7e2c2-a21f-44ed-b3de-dd26d38744a3 button.colab-df-convert');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function convertToInteractive(key) {\n",
              "        const element = document.querySelector('#df-e7d7e2c2-a21f-44ed-b3de-dd26d38744a3');\n",
              "        const dataTable =\n",
              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                    [key], {});\n",
              "        if (!dataTable) return;\n",
              "\n",
              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "          + ' to learn more about interactive tables.';\n",
              "        element.innerHTML = '';\n",
              "        dataTable['output_type'] = 'display_data';\n",
              "        await google.colab.output.renderOutput(dataTable, element);\n",
              "        const docLink = document.createElement('div');\n",
              "        docLink.innerHTML = docLinkHtml;\n",
              "        element.appendChild(docLink);\n",
              "      }\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "\n",
              "    </div>\n",
              "  </div>\n"
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "dataframe",
              "repr_error": "0"
            }
          },
          "metadata": {},
          "execution_count": 169
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "* one patient has height of 27 inches - 'accuracy'"
      ],
      "metadata": {
        "id": "9vGbLEgoMjPv"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "treatments_cut.describe()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 300
        },
        "id": "K_-fLnNvkTO_",
        "outputId": "aaa84fad-18f9-4d7d-b20b-0d05882096af"
      },
      "execution_count": 170,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "       hba1c_start  hba1c_end  hba1c_change\n",
              "count    70.000000  70.000000     42.000000\n",
              "mean      7.838000   7.443143      0.518810\n",
              "std       0.423007   0.418706      0.270719\n",
              "min       7.510000   7.020000      0.280000\n",
              "25%       7.640000   7.232500      0.340000\n",
              "50%       7.730000   7.345000      0.370000\n",
              "75%       7.860000   7.467500      0.907500\n",
              "max       9.910000   9.460000      0.970000"
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-cd58efe8-d640-48ac-8c67-7462759e15f3\" class=\"colab-df-container\">\n",
              "    <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>hba1c_start</th>\n",
              "      <th>hba1c_end</th>\n",
              "      <th>hba1c_change</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>count</th>\n",
              "      <td>70.000000</td>\n",
              "      <td>70.000000</td>\n",
              "      <td>42.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>mean</th>\n",
              "      <td>7.838000</td>\n",
              "      <td>7.443143</td>\n",
              "      <td>0.518810</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>std</th>\n",
              "      <td>0.423007</td>\n",
              "      <td>0.418706</td>\n",
              "      <td>0.270719</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>min</th>\n",
              "      <td>7.510000</td>\n",
              "      <td>7.020000</td>\n",
              "      <td>0.280000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>25%</th>\n",
              "      <td>7.640000</td>\n",
              "      <td>7.232500</td>\n",
              "      <td>0.340000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>50%</th>\n",
              "      <td>7.730000</td>\n",
              "      <td>7.345000</td>\n",
              "      <td>0.370000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>75%</th>\n",
              "      <td>7.860000</td>\n",
              "      <td>7.467500</td>\n",
              "      <td>0.907500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>max</th>\n",
              "      <td>9.910000</td>\n",
              "      <td>9.460000</td>\n",
              "      <td>0.970000</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "    <div class=\"colab-df-buttons\">\n",
              "\n",
              "  <div class=\"colab-df-container\">\n",
              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-cd58efe8-d640-48ac-8c67-7462759e15f3')\"\n",
              "            title=\"Convert this dataframe to an interactive table.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    .colab-df-buttons div {\n",
              "      margin-bottom: 4px;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const buttonEl =\n",
              "        document.querySelector('#df-cd58efe8-d640-48ac-8c67-7462759e15f3 button.colab-df-convert');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function convertToInteractive(key) {\n",
              "        const element = document.querySelector('#df-cd58efe8-d640-48ac-8c67-7462759e15f3');\n",
              "        const dataTable =\n",
              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                    [key], {});\n",
              "        if (!dataTable) return;\n",
              "\n",
              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "          + ' to learn more about interactive tables.';\n",
              "        element.innerHTML = '';\n",
              "        dataTable['output_type'] = 'display_data';\n",
              "        await google.colab.output.renderOutput(dataTable, element);\n",
              "        const docLink = document.createElement('div');\n",
              "        docLink.innerHTML = docLinkHtml;\n",
              "        element.appendChild(docLink);\n",
              "      }\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "\n",
              "<div id=\"df-fac3ad5e-90de-4aac-8d5c-cc255aa0f2e6\">\n",
              "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-fac3ad5e-90de-4aac-8d5c-cc255aa0f2e6')\"\n",
              "            title=\"Suggest charts\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "     width=\"24px\">\n",
              "    <g>\n",
              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "    </g>\n",
              "</svg>\n",
              "  </button>\n",
              "\n",
              "<style>\n",
              "  .colab-df-quickchart {\n",
              "      --bg-color: #E8F0FE;\n",
              "      --fill-color: #1967D2;\n",
              "      --hover-bg-color: #E2EBFA;\n",
              "      --hover-fill-color: #174EA6;\n",
              "      --disabled-fill-color: #AAA;\n",
              "      --disabled-bg-color: #DDD;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart {\n",
              "      --bg-color: #3B4455;\n",
              "      --fill-color: #D2E3FC;\n",
              "      --hover-bg-color: #434B5C;\n",
              "      --hover-fill-color: #FFFFFF;\n",
              "      --disabled-bg-color: #3B4455;\n",
              "      --disabled-fill-color: #666;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart {\n",
              "    background-color: var(--bg-color);\n",
              "    border: none;\n",
              "    border-radius: 50%;\n",
              "    cursor: pointer;\n",
              "    display: none;\n",
              "    fill: var(--fill-color);\n",
              "    height: 32px;\n",
              "    padding: 0;\n",
              "    width: 32px;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart:hover {\n",
              "    background-color: var(--hover-bg-color);\n",
              "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "    fill: var(--button-hover-fill-color);\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart-complete:disabled,\n",
              "  .colab-df-quickchart-complete:disabled:hover {\n",
              "    background-color: var(--disabled-bg-color);\n",
              "    fill: var(--disabled-fill-color);\n",
              "    box-shadow: none;\n",
              "  }\n",
              "\n",
              "  .colab-df-spinner {\n",
              "    border: 2px solid var(--fill-color);\n",
              "    border-color: transparent;\n",
              "    border-bottom-color: var(--fill-color);\n",
              "    animation:\n",
              "      spin 1s steps(1) infinite;\n",
              "  }\n",
              "\n",
              "  @keyframes spin {\n",
              "    0% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "      border-left-color: var(--fill-color);\n",
              "    }\n",
              "    20% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    30% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    40% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    60% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    80% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "    90% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "  }\n",
              "</style>\n",
              "\n",
              "  <script>\n",
              "    async function quickchart(key) {\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#' + key + ' button');\n",
              "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
              "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
              "      try {\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'suggestCharts', [key], {});\n",
              "      } catch (error) {\n",
              "        console.error('Error during call to suggestCharts:', error);\n",
              "      }\n",
              "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
              "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
              "    }\n",
              "    (() => {\n",
              "      let quickchartButtonEl =\n",
              "        document.querySelector('#df-fac3ad5e-90de-4aac-8d5c-cc255aa0f2e6 button');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "    })();\n",
              "  </script>\n",
              "</div>\n",
              "\n",
              "    </div>\n",
              "  </div>\n"
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "dataframe",
              "summary": "{\n  \"name\": \"treatments_cut\",\n  \"rows\": 8,\n  \"fields\": [\n    {\n      \"column\": \"hba1c_start\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 22.45199649273293,\n        \"min\": 0.4230071777922547,\n        \"max\": 70.0,\n        \"num_unique_values\": 8,\n        \"samples\": [\n          7.837999999999999,\n          7.73,\n          70.0\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"hba1c_end\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 22.561446280050365,\n        \"min\": 0.41870588361906946,\n        \"max\": 70.0,\n        \"num_unique_values\": 8,\n        \"samples\": [\n          7.443142857142857,\n          7.345,\n          70.0\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"hba1c_change\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 14.667102529820683,\n        \"min\": 0.270719027964801,\n        \"max\": 42.0,\n        \"num_unique_values\": 8,\n        \"samples\": [\n          0.5188095238095238,\n          0.37,\n          42.0\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    }\n  ]\n}"
            }
          },
          "metadata": {},
          "execution_count": 170
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "treatments.sort_values('hba1c_change',na_position='first')"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 423
        },
        "id": "Lec8pKINoyH3",
        "outputId": "39fb218f-8a05-4e70-a6a2-8fa206180e8f"
      },
      "execution_count": 171,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "    given_name      surname    auralin    novodra  hba1c_start  hba1c_end  \\\n",
              "0     veronika     jindrová  41u - 48u          -         7.63       7.20   \n",
              "2     yukitaka     takenaka          -  39u - 36u         7.68       7.25   \n",
              "8        saber       ménard          -  54u - 54u         8.08       7.70   \n",
              "9         asia      woźniak  30u - 36u          -         7.76       7.37   \n",
              "10      joseph          day  29u - 36u          -         7.70       7.19   \n",
              "..         ...          ...        ...        ...          ...        ...   \n",
              "49     jackson      addison          -  42u - 42u         7.99       7.51   \n",
              "17        gina         cain          -  36u - 36u         7.88       7.40   \n",
              "32       laura  ehrlichmann          -  43u - 40u         7.95       7.46   \n",
              "245         wu         sung          -  47u - 48u         7.61       7.12   \n",
              "138    giovana        rocha          -  23u - 21u         7.87       7.38   \n",
              "\n",
              "     hba1c_change  \n",
              "0             NaN  \n",
              "2             NaN  \n",
              "8             NaN  \n",
              "9             NaN  \n",
              "10            NaN  \n",
              "..            ...  \n",
              "49           0.98  \n",
              "17           0.98  \n",
              "32           0.99  \n",
              "245          0.99  \n",
              "138          0.99  \n",
              "\n",
              "[280 rows x 7 columns]"
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-5ca64f59-0eaa-4ef0-8620-532008b73669\" class=\"colab-df-container\">\n",
              "    <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>given_name</th>\n",
              "      <th>surname</th>\n",
              "      <th>auralin</th>\n",
              "      <th>novodra</th>\n",
              "      <th>hba1c_start</th>\n",
              "      <th>hba1c_end</th>\n",
              "      <th>hba1c_change</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>veronika</td>\n",
              "      <td>jindrová</td>\n",
              "      <td>41u - 48u</td>\n",
              "      <td>-</td>\n",
              "      <td>7.63</td>\n",
              "      <td>7.20</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>yukitaka</td>\n",
              "      <td>takenaka</td>\n",
              "      <td>-</td>\n",
              "      <td>39u - 36u</td>\n",
              "      <td>7.68</td>\n",
              "      <td>7.25</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>8</th>\n",
              "      <td>saber</td>\n",
              "      <td>ménard</td>\n",
              "      <td>-</td>\n",
              "      <td>54u - 54u</td>\n",
              "      <td>8.08</td>\n",
              "      <td>7.70</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>9</th>\n",
              "      <td>asia</td>\n",
              "      <td>woźniak</td>\n",
              "      <td>30u - 36u</td>\n",
              "      <td>-</td>\n",
              "      <td>7.76</td>\n",
              "      <td>7.37</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>10</th>\n",
              "      <td>joseph</td>\n",
              "      <td>day</td>\n",
              "      <td>29u - 36u</td>\n",
              "      <td>-</td>\n",
              "      <td>7.70</td>\n",
              "      <td>7.19</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>...</th>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>49</th>\n",
              "      <td>jackson</td>\n",
              "      <td>addison</td>\n",
              "      <td>-</td>\n",
              "      <td>42u - 42u</td>\n",
              "      <td>7.99</td>\n",
              "      <td>7.51</td>\n",
              "      <td>0.98</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>17</th>\n",
              "      <td>gina</td>\n",
              "      <td>cain</td>\n",
              "      <td>-</td>\n",
              "      <td>36u - 36u</td>\n",
              "      <td>7.88</td>\n",
              "      <td>7.40</td>\n",
              "      <td>0.98</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>32</th>\n",
              "      <td>laura</td>\n",
              "      <td>ehrlichmann</td>\n",
              "      <td>-</td>\n",
              "      <td>43u - 40u</td>\n",
              "      <td>7.95</td>\n",
              "      <td>7.46</td>\n",
              "      <td>0.99</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>245</th>\n",
              "      <td>wu</td>\n",
              "      <td>sung</td>\n",
              "      <td>-</td>\n",
              "      <td>47u - 48u</td>\n",
              "      <td>7.61</td>\n",
              "      <td>7.12</td>\n",
              "      <td>0.99</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>138</th>\n",
              "      <td>giovana</td>\n",
              "      <td>rocha</td>\n",
              "      <td>-</td>\n",
              "      <td>23u - 21u</td>\n",
              "      <td>7.87</td>\n",
              "      <td>7.38</td>\n",
              "      <td>0.99</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>280 rows × 7 columns</p>\n",
              "</div>\n",
              "    <div class=\"colab-df-buttons\">\n",
              "\n",
              "  <div class=\"colab-df-container\">\n",
              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-5ca64f59-0eaa-4ef0-8620-532008b73669')\"\n",
              "            title=\"Convert this dataframe to an interactive table.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    .colab-df-buttons div {\n",
              "      margin-bottom: 4px;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const buttonEl =\n",
              "        document.querySelector('#df-5ca64f59-0eaa-4ef0-8620-532008b73669 button.colab-df-convert');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function convertToInteractive(key) {\n",
              "        const element = document.querySelector('#df-5ca64f59-0eaa-4ef0-8620-532008b73669');\n",
              "        const dataTable =\n",
              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                    [key], {});\n",
              "        if (!dataTable) return;\n",
              "\n",
              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "          + ' to learn more about interactive tables.';\n",
              "        element.innerHTML = '';\n",
              "        dataTable['output_type'] = 'display_data';\n",
              "        await google.colab.output.renderOutput(dataTable, element);\n",
              "        const docLink = document.createElement('div');\n",
              "        docLink.innerHTML = docLinkHtml;\n",
              "        element.appendChild(docLink);\n",
              "      }\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "\n",
              "<div id=\"df-74761eb9-a8c3-4c2d-a5c0-d4958aa944f6\">\n",
              "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-74761eb9-a8c3-4c2d-a5c0-d4958aa944f6')\"\n",
              "            title=\"Suggest charts\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "     width=\"24px\">\n",
              "    <g>\n",
              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "    </g>\n",
              "</svg>\n",
              "  </button>\n",
              "\n",
              "<style>\n",
              "  .colab-df-quickchart {\n",
              "      --bg-color: #E8F0FE;\n",
              "      --fill-color: #1967D2;\n",
              "      --hover-bg-color: #E2EBFA;\n",
              "      --hover-fill-color: #174EA6;\n",
              "      --disabled-fill-color: #AAA;\n",
              "      --disabled-bg-color: #DDD;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart {\n",
              "      --bg-color: #3B4455;\n",
              "      --fill-color: #D2E3FC;\n",
              "      --hover-bg-color: #434B5C;\n",
              "      --hover-fill-color: #FFFFFF;\n",
              "      --disabled-bg-color: #3B4455;\n",
              "      --disabled-fill-color: #666;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart {\n",
              "    background-color: var(--bg-color);\n",
              "    border: none;\n",
              "    border-radius: 50%;\n",
              "    cursor: pointer;\n",
              "    display: none;\n",
              "    fill: var(--fill-color);\n",
              "    height: 32px;\n",
              "    padding: 0;\n",
              "    width: 32px;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart:hover {\n",
              "    background-color: var(--hover-bg-color);\n",
              "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "    fill: var(--button-hover-fill-color);\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart-complete:disabled,\n",
              "  .colab-df-quickchart-complete:disabled:hover {\n",
              "    background-color: var(--disabled-bg-color);\n",
              "    fill: var(--disabled-fill-color);\n",
              "    box-shadow: none;\n",
              "  }\n",
              "\n",
              "  .colab-df-spinner {\n",
              "    border: 2px solid var(--fill-color);\n",
              "    border-color: transparent;\n",
              "    border-bottom-color: var(--fill-color);\n",
              "    animation:\n",
              "      spin 1s steps(1) infinite;\n",
              "  }\n",
              "\n",
              "  @keyframes spin {\n",
              "    0% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "      border-left-color: var(--fill-color);\n",
              "    }\n",
              "    20% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    30% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    40% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    60% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    80% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "    90% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "  }\n",
              "</style>\n",
              "\n",
              "  <script>\n",
              "    async function quickchart(key) {\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#' + key + ' button');\n",
              "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
              "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
              "      try {\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'suggestCharts', [key], {});\n",
              "      } catch (error) {\n",
              "        console.error('Error during call to suggestCharts:', error);\n",
              "      }\n",
              "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
              "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
              "    }\n",
              "    (() => {\n",
              "      let quickchartButtonEl =\n",
              "        document.querySelector('#df-74761eb9-a8c3-4c2d-a5c0-d4958aa944f6 button');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "    })();\n",
              "  </script>\n",
              "</div>\n",
              "\n",
              "    </div>\n",
              "  </div>\n"
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "dataframe",
              "summary": "{\n  \"name\": \"treatments\",\n  \"rows\": 280,\n  \"fields\": [\n    {\n      \"column\": \"given_name\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 273,\n        \"samples\": [\n          \"mette\",\n          \"johana\",\n          \"katrine\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"surname\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 272,\n        \"samples\": [\n          \"sandgreen\",\n          \"mata\",\n          \"touma\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"auralin\",\n      \"properties\": {\n        \"dtype\": \"category\",\n        \"num_unique_values\": 121,\n        \"samples\": [\n          \"51u - 60u\",\n          \"39u - 50u\",\n          \"29u - 38u\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"novodra\",\n      \"properties\": {\n        \"dtype\": \"category\",\n        \"num_unique_values\": 113,\n        \"samples\": [\n          \"56u - 58u\",\n          \"43u - 47u\",\n          \"51u - 54u\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"hba1c_start\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0.5686384372332098,\n        \"min\": 7.5,\n        \"max\": 9.95,\n        \"num_unique_values\": 97,\n        \"samples\": [\n          7.99,\n          7.86,\n          8.51\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"hba1c_end\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0.5696715306576079,\n        \"min\": 7.01,\n        \"max\": 9.58,\n        \"num_unique_values\": 109,\n        \"samples\": [\n          7.48,\n          7.94,\n          7.19\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"hba1c_change\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0.27955527609183284,\n        \"min\": 0.2,\n        \"max\": 0.99,\n        \"num_unique_values\": 30,\n        \"samples\": [\n          0.97,\n          0.39,\n          0.93\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    }\n  ]\n}"
            }
          },
          "metadata": {},
          "execution_count": 171
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "* the hba1c change has wrong values - some values should be 0.49 and are changed to 0.99"
      ],
      "metadata": {
        "id": "KPahLBp5NiwC"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Note - Assessing Data is an Iterative Process"
      ],
      "metadata": {
        "id": "hQHkTcMzsJ1S"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Data Quality Dimensions\n",
        "\n",
        "- Completeness -> is data missing?\n",
        "- Validity -> is data invalid -> negative height -> duplicate patient id\n",
        "- Accuracy -> data is valid but not accurate -> weight -> 1kg\n",
        "- Consistency -> both valid and accurate but written differently -> New Youk and NY"
      ],
      "metadata": {
        "id": "eeTObK6-sPdP"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Order of severity\n",
        "\n",
        "Completeness <- Validity <- Accuracy <- Consistency"
      ],
      "metadata": {
        "id": "FByfAkZpywyy"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Data Cleaning Order\n",
        "\n",
        "1. Quality -> Completeness\n",
        "2. Tidiness\n",
        "3. Quality -> Validity\n",
        "4. Quality -> Accuracy\n",
        "5. Quality -> Consistency\n",
        "\n",
        "#### Steps involved in Data cleaning\n",
        "- Define\n",
        "- Code\n",
        "- Test\n",
        "\n",
        "`Always make sure to create a copy of your pandas dataframe before you start the cleaning process`"
      ],
      "metadata": {
        "id": "Ff-K97yd0wMc"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "patients_df = patients.copy()\n",
        "treatments_df = treatments.copy()\n",
        "treatments_cut_df = treatments_cut.copy()\n",
        "adverse_reactions_df = adverse_reactions.copy()"
      ],
      "metadata": {
        "id": "CKIZnF8YtjdY"
      },
      "execution_count": 172,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Define\n",
        "\n",
        "- replace all missing values of patients df with no data\n",
        "- subtract hba1c_start from hba1c_end to get all the change values\n",
        "- in patients table we will use regex to separate email and phone"
      ],
      "metadata": {
        "id": "S3qLmJTmuBmZ"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "\n",
        "Code & Solutions"
      ],
      "metadata": {
        "id": "lLeIsOj-T0qB"
      }
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "qejOwUN8UYYU"
      },
      "execution_count": 172,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# code\n",
        "patients_df.fillna('No data',inplace=True)"
      ],
      "metadata": {
        "id": "mY839W2qtjpp"
      },
      "execution_count": 173,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# test\n",
        "patients_df.info()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Q8QWP4DMtjzm",
        "outputId": "44ed41ac-3dac-4616-9681-3b4070b83663"
      },
      "execution_count": 174,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "<class 'pandas.core.frame.DataFrame'>\n",
            "RangeIndex: 503 entries, 0 to 502\n",
            "Data columns (total 14 columns):\n",
            " #   Column        Non-Null Count  Dtype  \n",
            "---  ------        --------------  -----  \n",
            " 0   patient_id    503 non-null    int64  \n",
            " 1   assigned_sex  503 non-null    object \n",
            " 2   given_name    503 non-null    object \n",
            " 3   surname       503 non-null    object \n",
            " 4   address       503 non-null    object \n",
            " 5   city          503 non-null    object \n",
            " 6   state         503 non-null    object \n",
            " 7   zip_code      503 non-null    object \n",
            " 8   country       503 non-null    object \n",
            " 9   contact       503 non-null    object \n",
            " 10  birthdate     503 non-null    object \n",
            " 11  weight        503 non-null    float64\n",
            " 12  height        503 non-null    int64  \n",
            " 13  bmi           503 non-null    float64\n",
            "dtypes: float64(2), int64(2), object(10)\n",
            "memory usage: 55.1+ KB\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "treatments.head()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 206
        },
        "id": "6wgQNBrVtj9J",
        "outputId": "0f4afc4d-4f05-4b2d-ce6f-48b09b3f9b3e"
      },
      "execution_count": 175,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "  given_name     surname    auralin    novodra  hba1c_start  hba1c_end  \\\n",
              "0   veronika    jindrová  41u - 48u          -         7.63       7.20   \n",
              "1     elliot  richardson          -  40u - 45u         7.56       7.09   \n",
              "2   yukitaka    takenaka          -  39u - 36u         7.68       7.25   \n",
              "3       skye  gormanston  33u - 36u          -         7.97       7.62   \n",
              "4     alissa      montez          -  33u - 29u         7.78       7.46   \n",
              "\n",
              "   hba1c_change  \n",
              "0           NaN  \n",
              "1          0.97  \n",
              "2           NaN  \n",
              "3          0.35  \n",
              "4          0.32  "
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-75d1f814-063f-4185-b890-e97efd206efc\" class=\"colab-df-container\">\n",
              "    <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>given_name</th>\n",
              "      <th>surname</th>\n",
              "      <th>auralin</th>\n",
              "      <th>novodra</th>\n",
              "      <th>hba1c_start</th>\n",
              "      <th>hba1c_end</th>\n",
              "      <th>hba1c_change</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>veronika</td>\n",
              "      <td>jindrová</td>\n",
              "      <td>41u - 48u</td>\n",
              "      <td>-</td>\n",
              "      <td>7.63</td>\n",
              "      <td>7.20</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>elliot</td>\n",
              "      <td>richardson</td>\n",
              "      <td>-</td>\n",
              "      <td>40u - 45u</td>\n",
              "      <td>7.56</td>\n",
              "      <td>7.09</td>\n",
              "      <td>0.97</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>yukitaka</td>\n",
              "      <td>takenaka</td>\n",
              "      <td>-</td>\n",
              "      <td>39u - 36u</td>\n",
              "      <td>7.68</td>\n",
              "      <td>7.25</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>skye</td>\n",
              "      <td>gormanston</td>\n",
              "      <td>33u - 36u</td>\n",
              "      <td>-</td>\n",
              "      <td>7.97</td>\n",
              "      <td>7.62</td>\n",
              "      <td>0.35</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>alissa</td>\n",
              "      <td>montez</td>\n",
              "      <td>-</td>\n",
              "      <td>33u - 29u</td>\n",
              "      <td>7.78</td>\n",
              "      <td>7.46</td>\n",
              "      <td>0.32</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "    <div class=\"colab-df-buttons\">\n",
              "\n",
              "  <div class=\"colab-df-container\">\n",
              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-75d1f814-063f-4185-b890-e97efd206efc')\"\n",
              "            title=\"Convert this dataframe to an interactive table.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    .colab-df-buttons div {\n",
              "      margin-bottom: 4px;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const buttonEl =\n",
              "        document.querySelector('#df-75d1f814-063f-4185-b890-e97efd206efc button.colab-df-convert');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function convertToInteractive(key) {\n",
              "        const element = document.querySelector('#df-75d1f814-063f-4185-b890-e97efd206efc');\n",
              "        const dataTable =\n",
              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                    [key], {});\n",
              "        if (!dataTable) return;\n",
              "\n",
              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "          + ' to learn more about interactive tables.';\n",
              "        element.innerHTML = '';\n",
              "        dataTable['output_type'] = 'display_data';\n",
              "        await google.colab.output.renderOutput(dataTable, element);\n",
              "        const docLink = document.createElement('div');\n",
              "        docLink.innerHTML = docLinkHtml;\n",
              "        element.appendChild(docLink);\n",
              "      }\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "\n",
              "<div id=\"df-83b95399-f54c-4809-a18d-267563768cf0\">\n",
              "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-83b95399-f54c-4809-a18d-267563768cf0')\"\n",
              "            title=\"Suggest charts\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "     width=\"24px\">\n",
              "    <g>\n",
              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "    </g>\n",
              "</svg>\n",
              "  </button>\n",
              "\n",
              "<style>\n",
              "  .colab-df-quickchart {\n",
              "      --bg-color: #E8F0FE;\n",
              "      --fill-color: #1967D2;\n",
              "      --hover-bg-color: #E2EBFA;\n",
              "      --hover-fill-color: #174EA6;\n",
              "      --disabled-fill-color: #AAA;\n",
              "      --disabled-bg-color: #DDD;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart {\n",
              "      --bg-color: #3B4455;\n",
              "      --fill-color: #D2E3FC;\n",
              "      --hover-bg-color: #434B5C;\n",
              "      --hover-fill-color: #FFFFFF;\n",
              "      --disabled-bg-color: #3B4455;\n",
              "      --disabled-fill-color: #666;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart {\n",
              "    background-color: var(--bg-color);\n",
              "    border: none;\n",
              "    border-radius: 50%;\n",
              "    cursor: pointer;\n",
              "    display: none;\n",
              "    fill: var(--fill-color);\n",
              "    height: 32px;\n",
              "    padding: 0;\n",
              "    width: 32px;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart:hover {\n",
              "    background-color: var(--hover-bg-color);\n",
              "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "    fill: var(--button-hover-fill-color);\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart-complete:disabled,\n",
              "  .colab-df-quickchart-complete:disabled:hover {\n",
              "    background-color: var(--disabled-bg-color);\n",
              "    fill: var(--disabled-fill-color);\n",
              "    box-shadow: none;\n",
              "  }\n",
              "\n",
              "  .colab-df-spinner {\n",
              "    border: 2px solid var(--fill-color);\n",
              "    border-color: transparent;\n",
              "    border-bottom-color: var(--fill-color);\n",
              "    animation:\n",
              "      spin 1s steps(1) infinite;\n",
              "  }\n",
              "\n",
              "  @keyframes spin {\n",
              "    0% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "      border-left-color: var(--fill-color);\n",
              "    }\n",
              "    20% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    30% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    40% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    60% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    80% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "    90% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "  }\n",
              "</style>\n",
              "\n",
              "  <script>\n",
              "    async function quickchart(key) {\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#' + key + ' button');\n",
              "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
              "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
              "      try {\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'suggestCharts', [key], {});\n",
              "      } catch (error) {\n",
              "        console.error('Error during call to suggestCharts:', error);\n",
              "      }\n",
              "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
              "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
              "    }\n",
              "    (() => {\n",
              "      let quickchartButtonEl =\n",
              "        document.querySelector('#df-83b95399-f54c-4809-a18d-267563768cf0 button');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "    })();\n",
              "  </script>\n",
              "</div>\n",
              "\n",
              "    </div>\n",
              "  </div>\n"
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "dataframe",
              "variable_name": "treatments",
              "summary": "{\n  \"name\": \"treatments\",\n  \"rows\": 280,\n  \"fields\": [\n    {\n      \"column\": \"given_name\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 273,\n        \"samples\": [\n          \"szilveszter\",\n          \"javier\",\n          \"avdei\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"surname\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 272,\n        \"samples\": [\n          \"totth\",\n          \"silva\",\n          \"tikhonov\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"auralin\",\n      \"properties\": {\n        \"dtype\": \"category\",\n        \"num_unique_values\": 121,\n        \"samples\": [\n          \"53u - 64u\",\n          \"39u - 43u\",\n          \"31u - 38u\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"novodra\",\n      \"properties\": {\n        \"dtype\": \"category\",\n        \"num_unique_values\": 113,\n        \"samples\": [\n          \"31u - 36u\",\n          \"42u - 44u\",\n          \"34u - 32u\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"hba1c_start\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0.5686384372332098,\n        \"min\": 7.5,\n        \"max\": 9.95,\n        \"num_unique_values\": 97,\n        \"samples\": [\n          8.19,\n          7.58,\n          9.62\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"hba1c_end\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0.5696715306576079,\n        \"min\": 7.01,\n        \"max\": 9.58,\n        \"num_unique_values\": 109,\n        \"samples\": [\n          7.45,\n          7.19,\n          7.46\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"hba1c_change\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0.2795552760918328,\n        \"min\": 0.2,\n        \"max\": 0.99,\n        \"num_unique_values\": 30,\n        \"samples\": [\n          0.26,\n          0.3,\n          0.28\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    }\n  ]\n}"
            }
          },
          "metadata": {},
          "execution_count": 175
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# code\n",
        "treatments_df['hba1c_change'] = treatments_df['hba1c_start'] - treatments_df['hba1c_end']\n",
        "treatments_cut_df['hba1c_change'] = treatments_cut_df['hba1c_start'] - treatments_cut_df['hba1c_end']"
      ],
      "metadata": {
        "id": "nEedHCbiu4jq"
      },
      "execution_count": 176,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# test\n",
        "treatments_cut_df.info()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "crgiIhg7u40W",
        "outputId": "e5b93764-89cf-4511-b0db-00279a4d410c"
      },
      "execution_count": 177,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "<class 'pandas.core.frame.DataFrame'>\n",
            "RangeIndex: 70 entries, 0 to 69\n",
            "Data columns (total 7 columns):\n",
            " #   Column        Non-Null Count  Dtype  \n",
            "---  ------        --------------  -----  \n",
            " 0   given_name    70 non-null     object \n",
            " 1   surname       70 non-null     object \n",
            " 2   auralin       70 non-null     object \n",
            " 3   novodra       70 non-null     object \n",
            " 4   hba1c_start   70 non-null     float64\n",
            " 5   hba1c_end     70 non-null     float64\n",
            " 6   hba1c_change  70 non-null     float64\n",
            "dtypes: float64(3), object(4)\n",
            "memory usage: 4.0+ KB\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "patients.head()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 293
        },
        "id": "KskC9le2u5K1",
        "outputId": "f6ffd340-ce78-4d1b-a526-bbb4570c1ae6"
      },
      "execution_count": 178,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "   patient_id assigned_sex given_name  surname                    address  \\\n",
              "0           1       female        Zoe  Wellish       576 Brown Bear Drive   \n",
              "1           2       female     Pamela     Hill  2370 University Hill Road   \n",
              "2           3         male        Jae   Debord      1493 Poling Farm Road   \n",
              "3           4         male       Liêm     Phan        2335 Webster Street   \n",
              "4           5         male        Tim  Neudorf       1428 Turkey Pen Lane   \n",
              "\n",
              "                city       state  zip_code        country  \\\n",
              "0  Rancho California  California   92390.0  United States   \n",
              "1          Armstrong    Illinois   61812.0  United States   \n",
              "2               York    Nebraska   68467.0  United States   \n",
              "3         Woodbridge          NJ    7095.0  United States   \n",
              "4             Dothan          AL   36303.0  United States   \n",
              "\n",
              "                                      contact  birthdate  weight  height   bmi  \n",
              "0        951-719-9170ZoeWellish@superrito.com  7/10/1976   121.7      66  19.6  \n",
              "1       PamelaSHill@cuvox.de+1 (217) 569-3204   4/3/1967   118.8      66  19.2  \n",
              "2            402-363-6804JaeMDebord@gustr.com  2/19/1980   177.8      71  24.8  \n",
              "3  PhanBaLiem@jourrapide.com+1 (732) 636-8246  7/26/1951   220.9      70  31.7  \n",
              "4             334-515-7487TimNeudorf@cuvox.de  2/18/1928   192.3      27  26.1  "
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-cd5ae5a9-e48f-4715-be1e-618b7607e4b4\" class=\"colab-df-container\">\n",
              "    <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>patient_id</th>\n",
              "      <th>assigned_sex</th>\n",
              "      <th>given_name</th>\n",
              "      <th>surname</th>\n",
              "      <th>address</th>\n",
              "      <th>city</th>\n",
              "      <th>state</th>\n",
              "      <th>zip_code</th>\n",
              "      <th>country</th>\n",
              "      <th>contact</th>\n",
              "      <th>birthdate</th>\n",
              "      <th>weight</th>\n",
              "      <th>height</th>\n",
              "      <th>bmi</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>1</td>\n",
              "      <td>female</td>\n",
              "      <td>Zoe</td>\n",
              "      <td>Wellish</td>\n",
              "      <td>576 Brown Bear Drive</td>\n",
              "      <td>Rancho California</td>\n",
              "      <td>California</td>\n",
              "      <td>92390.0</td>\n",
              "      <td>United States</td>\n",
              "      <td>951-719-9170ZoeWellish@superrito.com</td>\n",
              "      <td>7/10/1976</td>\n",
              "      <td>121.7</td>\n",
              "      <td>66</td>\n",
              "      <td>19.6</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>2</td>\n",
              "      <td>female</td>\n",
              "      <td>Pamela</td>\n",
              "      <td>Hill</td>\n",
              "      <td>2370 University Hill Road</td>\n",
              "      <td>Armstrong</td>\n",
              "      <td>Illinois</td>\n",
              "      <td>61812.0</td>\n",
              "      <td>United States</td>\n",
              "      <td>PamelaSHill@cuvox.de+1 (217) 569-3204</td>\n",
              "      <td>4/3/1967</td>\n",
              "      <td>118.8</td>\n",
              "      <td>66</td>\n",
              "      <td>19.2</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>3</td>\n",
              "      <td>male</td>\n",
              "      <td>Jae</td>\n",
              "      <td>Debord</td>\n",
              "      <td>1493 Poling Farm Road</td>\n",
              "      <td>York</td>\n",
              "      <td>Nebraska</td>\n",
              "      <td>68467.0</td>\n",
              "      <td>United States</td>\n",
              "      <td>402-363-6804JaeMDebord@gustr.com</td>\n",
              "      <td>2/19/1980</td>\n",
              "      <td>177.8</td>\n",
              "      <td>71</td>\n",
              "      <td>24.8</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>4</td>\n",
              "      <td>male</td>\n",
              "      <td>Liêm</td>\n",
              "      <td>Phan</td>\n",
              "      <td>2335 Webster Street</td>\n",
              "      <td>Woodbridge</td>\n",
              "      <td>NJ</td>\n",
              "      <td>7095.0</td>\n",
              "      <td>United States</td>\n",
              "      <td>PhanBaLiem@jourrapide.com+1 (732) 636-8246</td>\n",
              "      <td>7/26/1951</td>\n",
              "      <td>220.9</td>\n",
              "      <td>70</td>\n",
              "      <td>31.7</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>5</td>\n",
              "      <td>male</td>\n",
              "      <td>Tim</td>\n",
              "      <td>Neudorf</td>\n",
              "      <td>1428 Turkey Pen Lane</td>\n",
              "      <td>Dothan</td>\n",
              "      <td>AL</td>\n",
              "      <td>36303.0</td>\n",
              "      <td>United States</td>\n",
              "      <td>334-515-7487TimNeudorf@cuvox.de</td>\n",
              "      <td>2/18/1928</td>\n",
              "      <td>192.3</td>\n",
              "      <td>27</td>\n",
              "      <td>26.1</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "    <div class=\"colab-df-buttons\">\n",
              "\n",
              "  <div class=\"colab-df-container\">\n",
              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-cd5ae5a9-e48f-4715-be1e-618b7607e4b4')\"\n",
              "            title=\"Convert this dataframe to an interactive table.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    .colab-df-buttons div {\n",
              "      margin-bottom: 4px;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const buttonEl =\n",
              "        document.querySelector('#df-cd5ae5a9-e48f-4715-be1e-618b7607e4b4 button.colab-df-convert');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function convertToInteractive(key) {\n",
              "        const element = document.querySelector('#df-cd5ae5a9-e48f-4715-be1e-618b7607e4b4');\n",
              "        const dataTable =\n",
              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                    [key], {});\n",
              "        if (!dataTable) return;\n",
              "\n",
              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "          + ' to learn more about interactive tables.';\n",
              "        element.innerHTML = '';\n",
              "        dataTable['output_type'] = 'display_data';\n",
              "        await google.colab.output.renderOutput(dataTable, element);\n",
              "        const docLink = document.createElement('div');\n",
              "        docLink.innerHTML = docLinkHtml;\n",
              "        element.appendChild(docLink);\n",
              "      }\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "\n",
              "<div id=\"df-f16cfb2b-19a1-4d0a-8364-396e8c4a4210\">\n",
              "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-f16cfb2b-19a1-4d0a-8364-396e8c4a4210')\"\n",
              "            title=\"Suggest charts\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "     width=\"24px\">\n",
              "    <g>\n",
              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "    </g>\n",
              "</svg>\n",
              "  </button>\n",
              "\n",
              "<style>\n",
              "  .colab-df-quickchart {\n",
              "      --bg-color: #E8F0FE;\n",
              "      --fill-color: #1967D2;\n",
              "      --hover-bg-color: #E2EBFA;\n",
              "      --hover-fill-color: #174EA6;\n",
              "      --disabled-fill-color: #AAA;\n",
              "      --disabled-bg-color: #DDD;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart {\n",
              "      --bg-color: #3B4455;\n",
              "      --fill-color: #D2E3FC;\n",
              "      --hover-bg-color: #434B5C;\n",
              "      --hover-fill-color: #FFFFFF;\n",
              "      --disabled-bg-color: #3B4455;\n",
              "      --disabled-fill-color: #666;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart {\n",
              "    background-color: var(--bg-color);\n",
              "    border: none;\n",
              "    border-radius: 50%;\n",
              "    cursor: pointer;\n",
              "    display: none;\n",
              "    fill: var(--fill-color);\n",
              "    height: 32px;\n",
              "    padding: 0;\n",
              "    width: 32px;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart:hover {\n",
              "    background-color: var(--hover-bg-color);\n",
              "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "    fill: var(--button-hover-fill-color);\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart-complete:disabled,\n",
              "  .colab-df-quickchart-complete:disabled:hover {\n",
              "    background-color: var(--disabled-bg-color);\n",
              "    fill: var(--disabled-fill-color);\n",
              "    box-shadow: none;\n",
              "  }\n",
              "\n",
              "  .colab-df-spinner {\n",
              "    border: 2px solid var(--fill-color);\n",
              "    border-color: transparent;\n",
              "    border-bottom-color: var(--fill-color);\n",
              "    animation:\n",
              "      spin 1s steps(1) infinite;\n",
              "  }\n",
              "\n",
              "  @keyframes spin {\n",
              "    0% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "      border-left-color: var(--fill-color);\n",
              "    }\n",
              "    20% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    30% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    40% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    60% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    80% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "    90% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "  }\n",
              "</style>\n",
              "\n",
              "  <script>\n",
              "    async function quickchart(key) {\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#' + key + ' button');\n",
              "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
              "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
              "      try {\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'suggestCharts', [key], {});\n",
              "      } catch (error) {\n",
              "        console.error('Error during call to suggestCharts:', error);\n",
              "      }\n",
              "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
              "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
              "    }\n",
              "    (() => {\n",
              "      let quickchartButtonEl =\n",
              "        document.querySelector('#df-f16cfb2b-19a1-4d0a-8364-396e8c4a4210 button');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "    })();\n",
              "  </script>\n",
              "</div>\n",
              "\n",
              "    </div>\n",
              "  </div>\n"
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "dataframe",
              "variable_name": "patients",
              "summary": "{\n  \"name\": \"patients\",\n  \"rows\": 503,\n  \"fields\": [\n    {\n      \"column\": \"patient_id\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 145,\n        \"min\": 1,\n        \"max\": 503,\n        \"num_unique_values\": 503,\n        \"samples\": [\n          269,\n          74,\n          290\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"assigned_sex\",\n      \"properties\": {\n        \"dtype\": \"category\",\n        \"num_unique_values\": 2,\n        \"samples\": [\n          \"male\",\n          \"female\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"given_name\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 470,\n        \"samples\": [\n          \"Anco\",\n          \"Leo\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"surname\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 466,\n        \"samples\": [\n          \"Koldenhof\",\n          \"Ferreira\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"address\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 483,\n        \"samples\": [\n          \"1962 Cabell Avenue\",\n          \"3958 Liberty Avenue\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"city\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 349,\n        \"samples\": [\n          \"Monroe\",\n          \"Lafayette\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"state\",\n      \"properties\": {\n        \"dtype\": \"category\",\n        \"num_unique_values\": 54,\n        \"samples\": [\n          \"IL\",\n          \"OR\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"zip_code\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 30265.80744180444,\n        \"min\": 1002.0,\n        \"max\": 99701.0,\n        \"num_unique_values\": 430,\n        \"samples\": [\n          91505.0,\n          2048.0\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"country\",\n      \"properties\": {\n        \"dtype\": \"category\",\n        \"num_unique_values\": 1,\n        \"samples\": [\n          \"United States\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"contact\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 483,\n        \"samples\": [\n          \"703-547-0551SunKo@einrot.com\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"birthdate\",\n      \"properties\": {\n        \"dtype\": \"object\",\n        \"num_unique_values\": 493,\n        \"samples\": [\n          \"10/12/1980\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"weight\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 33.91674139657365,\n        \"min\": 48.8,\n        \"max\": 255.9,\n        \"num_unique_values\": 347,\n        \"samples\": [\n          239.1\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"height\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 4,\n        \"min\": 27,\n        \"max\": 79,\n        \"num_unique_values\": 22,\n        \"samples\": [\n          66\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"bmi\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 5.2764378079801695,\n        \"min\": 17.1,\n        \"max\": 37.7,\n        \"num_unique_values\": 183,\n        \"samples\": [\n          24.7\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    }\n  ]\n}"
            }
          },
          "metadata": {},
          "execution_count": 178
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "import pandas as pd\n",
        "import re\n",
        "\n",
        "def find_contact_details(text: str) -> tuple:\n",
        "  \"\"\"\n",
        "  Extracts phone numbers and potential email addresses from text.\n",
        "\n",
        "  Args:\n",
        "      text: The text string to search.\n",
        "\n",
        "  Returns:\n",
        "      A tuple containing the extracted phone number (or np.nan) and\n",
        "      the potential email address (or None).\n",
        "  \"\"\"\n",
        "  # Handle missing values\n",
        "  if pd.isna(text):\n",
        "    return np.nan, None\n",
        "\n",
        "  # Phone number pattern (consider international formats)\n",
        "  phone_number_pattern = re.compile(r\"(\\+[\\d]{1,3}\\s)?(\\(?[\\d]{3}\\)?\\s?-?[\\d]{3}\\s?-?[\\d]{4})\")\n",
        "  phone_number_matches = re.findall(phone_number_pattern, text)\n",
        "\n",
        "  # Extract phone number (handle cases with/without country code)\n",
        "  phone_number = None\n",
        "  if phone_number_matches:\n",
        "    if len(phone_number_matches) >= 2:\n",
        "      # Country code present, extract actual phone number\n",
        "      phone_number = phone_number_matches[1]\n",
        "    else:\n",
        "      # No country code, use the first match\n",
        "      phone_number = phone_number_matches[0]\n",
        "\n",
        "  # Potential email address extraction (consider basic validation)\n",
        "  possible_email_add = re.sub(phone_number_pattern, \"\", text).strip()\n",
        "  email_pattern = re.compile(r\"([a-zA-Z0-9_.+-]+@[a-zA-Z0-9-]+\\.[a-zA-Z0-9-.]+)\")\n",
        "  # Check if remaining string has a valid email format (basic check)\n",
        "  if email_pattern.match(possible_email_add):\n",
        "    email_address = possible_email_add\n",
        "  else:\n",
        "    email_address = None\n",
        "\n",
        "  return phone_number, email_address\n",
        "\n",
        "# Apply function to DataFrame column (consider vectorized approach for large datasets)\n",
        "patients_df['phone'] = patients_df[\"contact\"].apply(lambda x: find_contact_details(x)[0])\n",
        "patients_df['email'] = patients_df[\"contact\"].apply(lambda x: find_contact_details(x)[1])\n"
      ],
      "metadata": {
        "id": "jNRbSM2bu5Xo"
      },
      "execution_count": 179,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "patients_df"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 860
        },
        "id": "6RQ9rI0aouRX",
        "outputId": "09732482-3120-4793-8f79-f3b2db4af9ae"
      },
      "execution_count": 193,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "     patient_id assigned_sex given_name      surname  \\\n",
              "0             1       female        Zoe      Wellish   \n",
              "1             2       female     Pamela         Hill   \n",
              "2             3         male        Jae       Debord   \n",
              "3             4         male       Liêm         Phan   \n",
              "4             5         male        Tim      Neudorf   \n",
              "..          ...          ...        ...          ...   \n",
              "498         499         male    Mustafa    Lindström   \n",
              "499         500         male      Ruman      Bisliev   \n",
              "500         501       female      Jinke    de Keizer   \n",
              "501         502       female    Chidalu  Onyekaozulu   \n",
              "502         503         male        Pat      Gersten   \n",
              "\n",
              "                       address               city       state zip_code  \\\n",
              "0         576 Brown Bear Drive  Rancho California  California  92390.0   \n",
              "1    2370 University Hill Road          Armstrong    Illinois  61812.0   \n",
              "2        1493 Poling Farm Road               York    Nebraska  68467.0   \n",
              "3          2335 Webster Street         Woodbridge          NJ   7095.0   \n",
              "4         1428 Turkey Pen Lane             Dothan          AL  36303.0   \n",
              "..                         ...                ...         ...      ...   \n",
              "498        2530 Victoria Court       Milton Mills          ME   3852.0   \n",
              "499   494 Clarksburg Park Road             Sedona          AZ  86341.0   \n",
              "500          649 Nutter Street      Overland Park          MO  64110.0   \n",
              "501   3652 Boone Crockett Lane            Seattle          WA  98109.0   \n",
              "502          2778 North Avenue               Burr    Nebraska  68324.0   \n",
              "\n",
              "           country  birthdate  weight  height   bmi                  phone  \\\n",
              "0    United States  7/10/1976   121.7      66  19.6       (, 951-719-9170)   \n",
              "1    United States   4/3/1967   118.8      66  19.2  (+1 , (217) 569-3204)   \n",
              "2    United States  2/19/1980   177.8      71  24.8       (, 402-363-6804)   \n",
              "3    United States  7/26/1951   220.9      70  31.7  (+1 , (732) 636-8246)   \n",
              "4    United States  2/18/1928   192.3      27  26.1       (, 334-515-7487)   \n",
              "..             ...        ...     ...     ...   ...                    ...   \n",
              "498  United States  4/10/1959   181.1      72  24.6       (, 207-477-0579)   \n",
              "499  United States  3/26/1948   239.6      70  34.4       (, 928-284-4492)   \n",
              "500  United States  1/13/1971   171.2      67  26.8       (, 816-223-6007)   \n",
              "501  United States  2/13/1952   176.9      67  27.7       (, 360 443 2060)   \n",
              "502  United States   5/3/1954   138.2      71  19.3       (, 402-848-4923)   \n",
              "\n",
              "                                  email  \n",
              "0              ZoeWellish@superrito.com  \n",
              "1                  PamelaSHill@cuvox.de  \n",
              "2                  JaeMDebord@gustr.com  \n",
              "3             PhanBaLiem@jourrapide.com  \n",
              "4                   TimNeudorf@cuvox.de  \n",
              "..                                  ...  \n",
              "498     MustafaLindstrom@jourrapide.com  \n",
              "499              RumanBisliev@gustr.com  \n",
              "500           JinkedeKeizer@teleworm.us  \n",
              "501  ChidaluOnyekaozulu@jourrapide.com1  \n",
              "502            PatrickGersten@rhyta.com  \n",
              "\n",
              "[503 rows x 15 columns]"
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-83677866-41fd-4506-bdc3-9edf1ecde935\" class=\"colab-df-container\">\n",
              "    <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>patient_id</th>\n",
              "      <th>assigned_sex</th>\n",
              "      <th>given_name</th>\n",
              "      <th>surname</th>\n",
              "      <th>address</th>\n",
              "      <th>city</th>\n",
              "      <th>state</th>\n",
              "      <th>zip_code</th>\n",
              "      <th>country</th>\n",
              "      <th>birthdate</th>\n",
              "      <th>weight</th>\n",
              "      <th>height</th>\n",
              "      <th>bmi</th>\n",
              "      <th>phone</th>\n",
              "      <th>email</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>1</td>\n",
              "      <td>female</td>\n",
              "      <td>Zoe</td>\n",
              "      <td>Wellish</td>\n",
              "      <td>576 Brown Bear Drive</td>\n",
              "      <td>Rancho California</td>\n",
              "      <td>California</td>\n",
              "      <td>92390.0</td>\n",
              "      <td>United States</td>\n",
              "      <td>7/10/1976</td>\n",
              "      <td>121.7</td>\n",
              "      <td>66</td>\n",
              "      <td>19.6</td>\n",
              "      <td>(, 951-719-9170)</td>\n",
              "      <td>ZoeWellish@superrito.com</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>2</td>\n",
              "      <td>female</td>\n",
              "      <td>Pamela</td>\n",
              "      <td>Hill</td>\n",
              "      <td>2370 University Hill Road</td>\n",
              "      <td>Armstrong</td>\n",
              "      <td>Illinois</td>\n",
              "      <td>61812.0</td>\n",
              "      <td>United States</td>\n",
              "      <td>4/3/1967</td>\n",
              "      <td>118.8</td>\n",
              "      <td>66</td>\n",
              "      <td>19.2</td>\n",
              "      <td>(+1 , (217) 569-3204)</td>\n",
              "      <td>PamelaSHill@cuvox.de</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>3</td>\n",
              "      <td>male</td>\n",
              "      <td>Jae</td>\n",
              "      <td>Debord</td>\n",
              "      <td>1493 Poling Farm Road</td>\n",
              "      <td>York</td>\n",
              "      <td>Nebraska</td>\n",
              "      <td>68467.0</td>\n",
              "      <td>United States</td>\n",
              "      <td>2/19/1980</td>\n",
              "      <td>177.8</td>\n",
              "      <td>71</td>\n",
              "      <td>24.8</td>\n",
              "      <td>(, 402-363-6804)</td>\n",
              "      <td>JaeMDebord@gustr.com</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>4</td>\n",
              "      <td>male</td>\n",
              "      <td>Liêm</td>\n",
              "      <td>Phan</td>\n",
              "      <td>2335 Webster Street</td>\n",
              "      <td>Woodbridge</td>\n",
              "      <td>NJ</td>\n",
              "      <td>7095.0</td>\n",
              "      <td>United States</td>\n",
              "      <td>7/26/1951</td>\n",
              "      <td>220.9</td>\n",
              "      <td>70</td>\n",
              "      <td>31.7</td>\n",
              "      <td>(+1 , (732) 636-8246)</td>\n",
              "      <td>PhanBaLiem@jourrapide.com</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>5</td>\n",
              "      <td>male</td>\n",
              "      <td>Tim</td>\n",
              "      <td>Neudorf</td>\n",
              "      <td>1428 Turkey Pen Lane</td>\n",
              "      <td>Dothan</td>\n",
              "      <td>AL</td>\n",
              "      <td>36303.0</td>\n",
              "      <td>United States</td>\n",
              "      <td>2/18/1928</td>\n",
              "      <td>192.3</td>\n",
              "      <td>27</td>\n",
              "      <td>26.1</td>\n",
              "      <td>(, 334-515-7487)</td>\n",
              "      <td>TimNeudorf@cuvox.de</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>...</th>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>498</th>\n",
              "      <td>499</td>\n",
              "      <td>male</td>\n",
              "      <td>Mustafa</td>\n",
              "      <td>Lindström</td>\n",
              "      <td>2530 Victoria Court</td>\n",
              "      <td>Milton Mills</td>\n",
              "      <td>ME</td>\n",
              "      <td>3852.0</td>\n",
              "      <td>United States</td>\n",
              "      <td>4/10/1959</td>\n",
              "      <td>181.1</td>\n",
              "      <td>72</td>\n",
              "      <td>24.6</td>\n",
              "      <td>(, 207-477-0579)</td>\n",
              "      <td>MustafaLindstrom@jourrapide.com</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>499</th>\n",
              "      <td>500</td>\n",
              "      <td>male</td>\n",
              "      <td>Ruman</td>\n",
              "      <td>Bisliev</td>\n",
              "      <td>494 Clarksburg Park Road</td>\n",
              "      <td>Sedona</td>\n",
              "      <td>AZ</td>\n",
              "      <td>86341.0</td>\n",
              "      <td>United States</td>\n",
              "      <td>3/26/1948</td>\n",
              "      <td>239.6</td>\n",
              "      <td>70</td>\n",
              "      <td>34.4</td>\n",
              "      <td>(, 928-284-4492)</td>\n",
              "      <td>RumanBisliev@gustr.com</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>500</th>\n",
              "      <td>501</td>\n",
              "      <td>female</td>\n",
              "      <td>Jinke</td>\n",
              "      <td>de Keizer</td>\n",
              "      <td>649 Nutter Street</td>\n",
              "      <td>Overland Park</td>\n",
              "      <td>MO</td>\n",
              "      <td>64110.0</td>\n",
              "      <td>United States</td>\n",
              "      <td>1/13/1971</td>\n",
              "      <td>171.2</td>\n",
              "      <td>67</td>\n",
              "      <td>26.8</td>\n",
              "      <td>(, 816-223-6007)</td>\n",
              "      <td>JinkedeKeizer@teleworm.us</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>501</th>\n",
              "      <td>502</td>\n",
              "      <td>female</td>\n",
              "      <td>Chidalu</td>\n",
              "      <td>Onyekaozulu</td>\n",
              "      <td>3652 Boone Crockett Lane</td>\n",
              "      <td>Seattle</td>\n",
              "      <td>WA</td>\n",
              "      <td>98109.0</td>\n",
              "      <td>United States</td>\n",
              "      <td>2/13/1952</td>\n",
              "      <td>176.9</td>\n",
              "      <td>67</td>\n",
              "      <td>27.7</td>\n",
              "      <td>(, 360 443 2060)</td>\n",
              "      <td>ChidaluOnyekaozulu@jourrapide.com1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>502</th>\n",
              "      <td>503</td>\n",
              "      <td>male</td>\n",
              "      <td>Pat</td>\n",
              "      <td>Gersten</td>\n",
              "      <td>2778 North Avenue</td>\n",
              "      <td>Burr</td>\n",
              "      <td>Nebraska</td>\n",
              "      <td>68324.0</td>\n",
              "      <td>United States</td>\n",
              "      <td>5/3/1954</td>\n",
              "      <td>138.2</td>\n",
              "      <td>71</td>\n",
              "      <td>19.3</td>\n",
              "      <td>(, 402-848-4923)</td>\n",
              "      <td>PatrickGersten@rhyta.com</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>503 rows × 15 columns</p>\n",
              "</div>\n",
              "    <div class=\"colab-df-buttons\">\n",
              "\n",
              "  <div class=\"colab-df-container\">\n",
              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-83677866-41fd-4506-bdc3-9edf1ecde935')\"\n",
              "            title=\"Convert this dataframe to an interactive table.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    .colab-df-buttons div {\n",
              "      margin-bottom: 4px;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const buttonEl =\n",
              "        document.querySelector('#df-83677866-41fd-4506-bdc3-9edf1ecde935 button.colab-df-convert');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function convertToInteractive(key) {\n",
              "        const element = document.querySelector('#df-83677866-41fd-4506-bdc3-9edf1ecde935');\n",
              "        const dataTable =\n",
              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                    [key], {});\n",
              "        if (!dataTable) return;\n",
              "\n",
              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "          + ' to learn more about interactive tables.';\n",
              "        element.innerHTML = '';\n",
              "        dataTable['output_type'] = 'display_data';\n",
              "        await google.colab.output.renderOutput(dataTable, element);\n",
              "        const docLink = document.createElement('div');\n",
              "        docLink.innerHTML = docLinkHtml;\n",
              "        element.appendChild(docLink);\n",
              "      }\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "\n",
              "<div id=\"df-0dc805c5-0c09-4cc1-b407-ca366aaed864\">\n",
              "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-0dc805c5-0c09-4cc1-b407-ca366aaed864')\"\n",
              "            title=\"Suggest charts\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "     width=\"24px\">\n",
              "    <g>\n",
              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "    </g>\n",
              "</svg>\n",
              "  </button>\n",
              "\n",
              "<style>\n",
              "  .colab-df-quickchart {\n",
              "      --bg-color: #E8F0FE;\n",
              "      --fill-color: #1967D2;\n",
              "      --hover-bg-color: #E2EBFA;\n",
              "      --hover-fill-color: #174EA6;\n",
              "      --disabled-fill-color: #AAA;\n",
              "      --disabled-bg-color: #DDD;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart {\n",
              "      --bg-color: #3B4455;\n",
              "      --fill-color: #D2E3FC;\n",
              "      --hover-bg-color: #434B5C;\n",
              "      --hover-fill-color: #FFFFFF;\n",
              "      --disabled-bg-color: #3B4455;\n",
              "      --disabled-fill-color: #666;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart {\n",
              "    background-color: var(--bg-color);\n",
              "    border: none;\n",
              "    border-radius: 50%;\n",
              "    cursor: pointer;\n",
              "    display: none;\n",
              "    fill: var(--fill-color);\n",
              "    height: 32px;\n",
              "    padding: 0;\n",
              "    width: 32px;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart:hover {\n",
              "    background-color: var(--hover-bg-color);\n",
              "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "    fill: var(--button-hover-fill-color);\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart-complete:disabled,\n",
              "  .colab-df-quickchart-complete:disabled:hover {\n",
              "    background-color: var(--disabled-bg-color);\n",
              "    fill: var(--disabled-fill-color);\n",
              "    box-shadow: none;\n",
              "  }\n",
              "\n",
              "  .colab-df-spinner {\n",
              "    border: 2px solid var(--fill-color);\n",
              "    border-color: transparent;\n",
              "    border-bottom-color: var(--fill-color);\n",
              "    animation:\n",
              "      spin 1s steps(1) infinite;\n",
              "  }\n",
              "\n",
              "  @keyframes spin {\n",
              "    0% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "      border-left-color: var(--fill-color);\n",
              "    }\n",
              "    20% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    30% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    40% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    60% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    80% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "    90% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "  }\n",
              "</style>\n",
              "\n",
              "  <script>\n",
              "    async function quickchart(key) {\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#' + key + ' button');\n",
              "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
              "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
              "      try {\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'suggestCharts', [key], {});\n",
              "      } catch (error) {\n",
              "        console.error('Error during call to suggestCharts:', error);\n",
              "      }\n",
              "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
              "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
              "    }\n",
              "    (() => {\n",
              "      let quickchartButtonEl =\n",
              "        document.querySelector('#df-0dc805c5-0c09-4cc1-b407-ca366aaed864 button');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "    })();\n",
              "  </script>\n",
              "</div>\n",
              "\n",
              "  <div id=\"id_8776d755-5711-40e4-95d7-c32123d6409d\">\n",
              "    <style>\n",
              "      .colab-df-generate {\n",
              "        background-color: #E8F0FE;\n",
              "        border: none;\n",
              "        border-radius: 50%;\n",
              "        cursor: pointer;\n",
              "        display: none;\n",
              "        fill: #1967D2;\n",
              "        height: 32px;\n",
              "        padding: 0 0 0 0;\n",
              "        width: 32px;\n",
              "      }\n",
              "\n",
              "      .colab-df-generate:hover {\n",
              "        background-color: #E2EBFA;\n",
              "        box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "        fill: #174EA6;\n",
              "      }\n",
              "\n",
              "      [theme=dark] .colab-df-generate {\n",
              "        background-color: #3B4455;\n",
              "        fill: #D2E3FC;\n",
              "      }\n",
              "\n",
              "      [theme=dark] .colab-df-generate:hover {\n",
              "        background-color: #434B5C;\n",
              "        box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "        filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "        fill: #FFFFFF;\n",
              "      }\n",
              "    </style>\n",
              "    <button class=\"colab-df-generate\" onclick=\"generateWithVariable('patients_df')\"\n",
              "            title=\"Generate code using this dataframe.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M7,19H8.4L18.45,9,17,7.55,7,17.6ZM5,21V16.75L18.45,3.32a2,2,0,0,1,2.83,0l1.4,1.43a1.91,1.91,0,0,1,.58,1.4,1.91,1.91,0,0,1-.58,1.4L9.25,21ZM18.45,9,17,7.55Zm-12,3A5.31,5.31,0,0,0,4.9,8.1,5.31,5.31,0,0,0,1,6.5,5.31,5.31,0,0,0,4.9,4.9,5.31,5.31,0,0,0,6.5,1,5.31,5.31,0,0,0,8.1,4.9,5.31,5.31,0,0,0,12,6.5,5.46,5.46,0,0,0,6.5,12Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "    <script>\n",
              "      (() => {\n",
              "      const buttonEl =\n",
              "        document.querySelector('#id_8776d755-5711-40e4-95d7-c32123d6409d button.colab-df-generate');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      buttonEl.onclick = () => {\n",
              "        google.colab.notebook.generateWithVariable('patients_df');\n",
              "      }\n",
              "      })();\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "    </div>\n",
              "  </div>\n"
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "dataframe",
              "variable_name": "patients_df",
              "summary": "{\n  \"name\": \"patients_df\",\n  \"rows\": 503,\n  \"fields\": [\n    {\n      \"column\": \"patient_id\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 145,\n        \"min\": 1,\n        \"max\": 503,\n        \"num_unique_values\": 503,\n        \"samples\": [\n          269,\n          74,\n          290\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"assigned_sex\",\n      \"properties\": {\n        \"dtype\": \"category\",\n        \"num_unique_values\": 2,\n        \"samples\": [\n          \"male\",\n          \"female\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"given_name\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 470,\n        \"samples\": [\n          \"Anco\",\n          \"Leo\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"surname\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 466,\n        \"samples\": [\n          \"Koldenhof\",\n          \"Ferreira\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"address\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 484,\n        \"samples\": [\n          \"1962 Cabell Avenue\",\n          \"3958 Liberty Avenue\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"city\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 350,\n        \"samples\": [\n          \"Monroe\",\n          \"Ottawa\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"state\",\n      \"properties\": {\n        \"dtype\": \"category\",\n        \"num_unique_values\": 55,\n        \"samples\": [\n          \"NC\",\n          \"Florida\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"zip_code\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 431,\n        \"samples\": [\n          91505.0,\n          2048.0\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"country\",\n      \"properties\": {\n        \"dtype\": \"category\",\n        \"num_unique_values\": 2,\n        \"samples\": [\n          \"No data\",\n          \"United States\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"birthdate\",\n      \"properties\": {\n        \"dtype\": \"object\",\n        \"num_unique_values\": 493,\n        \"samples\": [\n          \"10/12/1980\",\n          \"1/20/1926\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"weight\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 33.91674139657365,\n        \"min\": 48.8,\n        \"max\": 255.9,\n        \"num_unique_values\": 347,\n        \"samples\": [\n          239.1,\n          137.1\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"height\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 4,\n        \"min\": 27,\n        \"max\": 79,\n        \"num_unique_values\": 22,\n        \"samples\": [\n          66,\n          63\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"bmi\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 5.2764378079801695,\n        \"min\": 17.1,\n        \"max\": 37.7,\n        \"num_unique_values\": 183,\n        \"samples\": [\n          24.7,\n          19.4\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"phone\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 483,\n        \"samples\": [\n          [\n            \"\",\n            \"703-547-0551\"\n          ],\n          [\n            \"\",\n            \"714-496-2264\"\n          ]\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"email\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 483,\n        \"samples\": [\n          \"SunKo@einrot.com\",\n          \"TarPoldi@superrito.com\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    }\n  ]\n}"
            }
          },
          "metadata": {},
          "execution_count": 193
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "patients_df.drop(columns='contact',inplace=True)"
      ],
      "metadata": {
        "id": "JwTXJ83Vu5zD"
      },
      "execution_count": 181,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "baFyR4bju5_Z"
      },
      "execution_count": 181,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "U0jWyJbZu6Le"
      },
      "execution_count": 181,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "\n",
        "import re\n",
        "import pandas as pd\n",
        "\n",
        "# Assuming 'patients' is a DataFrame with a 'contact' column\n",
        "df = pd.DataFrame(columns=['phone', 'email'])\n",
        "\n",
        "# Consider vectorized approach using str.extract for larger datasets\n",
        "df[['phone', 'email']] = patients['contact'].str.extract(\n",
        "     r'(\\d{3}[-\\.\\s]??\\d{3}[-\\.\\s]??\\d{4}|\\(\\d{3}\\)\\s*\\d{3}[-\\.\\s]??\\d{4}|\\d{3}[-\\.\\s]??\\d{4})|'\n",
        "     r'([a-zA-Z0-9_.+-]+@[a-zA-Z0-9-]+\\.[a-zA-Z0-9-.]+)', expand=True)\n",
        "\n",
        "# Loop approach (can be replaced with vectorized approach above)\n",
        "'''\n",
        "for item in patients['contact']:\n",
        "  phone_match = re.search(r'(\\d{3}[-\\.\\s]??\\d{3}[-\\.\\s]??\\d{4}|\\(\\d{3}\\)\\s*\\d{3}[-\\.\\s]??\\d{4}|\\d{3}[-\\.\\s]??\\d{4})', item)\n",
        "  email_match = re.search(r'([a-zA-Z0-9_.+-]+@[a-zA-Z0-9-]+\\.[a-zA-Z0-9-.]+)', item)\n",
        "  phone = phone_match.group(0) if phone_match else None\n",
        "  email = email_match.group(1) if email_match else None\n",
        "  df = df.append({'phone': phone, 'email': email}, ignore_index=True)\n",
        "'''\n",
        "print(df)\n",
        "\n",
        "\n",
        "\n"
      ],
      "metadata": {
        "id": "knkJcn3Q5U3V",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "cfc838ff-a4bf-4a4a-9a7b-da33fa8e3cb2"
      },
      "execution_count": 182,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "            phone                                 email\n",
            "0    951-719-9170                                   NaN\n",
            "1             NaN                  PamelaSHill@cuvox.de\n",
            "2    402-363-6804                                   NaN\n",
            "3             NaN             PhanBaLiem@jourrapide.com\n",
            "4    334-515-7487                                   NaN\n",
            "..            ...                                   ...\n",
            "498  207-477-0579                                   NaN\n",
            "499  928-284-4492                                   NaN\n",
            "500  816-223-6007                                   NaN\n",
            "501           NaN    ChidaluOnyekaozulu@jourrapide.com1\n",
            "502           NaN  PatrickGersten@rhyta.com402-848-4923\n",
            "\n",
            "[503 rows x 2 columns]\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "treatments_df = pd.concat([treatments_df,treatments_cut_df])"
      ],
      "metadata": {
        "id": "DeJ24dEOBI0o"
      },
      "execution_count": 183,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "treatments_df = treatments_df.melt(id_vars=['given_name', 'surname' ,'hba1c_start', 'hba1c_end','hba1c_change'],var_name='type',value_name='dosage_range')"
      ],
      "metadata": {
        "id": "hSXaUjvPx2ME"
      },
      "execution_count": 184,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "treatments_df = treatments_df[treatments_df['dosage_range'] != '-']\n"
      ],
      "metadata": {
        "id": "vh1bRtP0yGak"
      },
      "execution_count": 185,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "treatments_df['dosage_start'] = treatments_df['dosage_range'].str.split('-').str.get(0)\n",
        "treatments_df['dosage_end'] = treatments_df['dosage_range'].str.split('-').str.get(1)"
      ],
      "metadata": {
        "id": "Ahy1SDVAy3v4"
      },
      "execution_count": 186,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "treatments_df.drop(columns='dosage_range',inplace=True)"
      ],
      "metadata": {
        "id": "BoomrglszKsn"
      },
      "execution_count": 187,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "treatments_df['dosage_start'] = treatments_df['dosage_start'].str.replace('u','')\n",
        "treatments_df['dosage_end'] = treatments_df['dosage_end'].str.replace('u','')"
      ],
      "metadata": {
        "id": "XbQNKXqGzfDI"
      },
      "execution_count": 188,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "treatments_df['dosage_start'] = treatments_df['dosage_start'].astype('int')\n",
        "treatments_df['dosage_end'] = treatments_df['dosage_end'].astype('int')"
      ],
      "metadata": {
        "id": "-_7SsSi-znMi"
      },
      "execution_count": 189,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "treatments_df"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 423
        },
        "id": "uYBsQmxjz5uR",
        "outputId": "cbc6f6e0-401d-471d-82bf-7f3e3c4e7d8a"
      },
      "execution_count": 190,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "      given_name     surname  hba1c_start  hba1c_end  hba1c_change     type  \\\n",
              "0       veronika    jindrová         7.63       7.20          0.43  auralin   \n",
              "3           skye  gormanston         7.97       7.62          0.35  auralin   \n",
              "6         sophia      haugen         7.65       7.27          0.38  auralin   \n",
              "7          eddie      archer         7.89       7.55          0.34  auralin   \n",
              "9           asia     woźniak         7.76       7.37          0.39  auralin   \n",
              "..           ...         ...          ...        ...           ...      ...   \n",
              "688  christopher    woodward         7.51       7.06          0.45  novodra   \n",
              "690        maret    sultygov         7.67       7.30          0.37  novodra   \n",
              "694        lixue       hsueh         9.21       8.80          0.41  novodra   \n",
              "696        jakob    jakobsen         7.96       7.51          0.45  novodra   \n",
              "698        berta  napolitani         7.68       7.21          0.47  novodra   \n",
              "\n",
              "     dosage_start  dosage_end  \n",
              "0              41          48  \n",
              "3              33          36  \n",
              "6              37          42  \n",
              "7              31          38  \n",
              "9              30          36  \n",
              "..            ...         ...  \n",
              "688            55          51  \n",
              "690            26          23  \n",
              "694            22          23  \n",
              "696            28          26  \n",
              "698            42          44  \n",
              "\n",
              "[350 rows x 8 columns]"
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-bc66d213-bdfa-44b1-96a3-58e462af5e7b\" class=\"colab-df-container\">\n",
              "    <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>given_name</th>\n",
              "      <th>surname</th>\n",
              "      <th>hba1c_start</th>\n",
              "      <th>hba1c_end</th>\n",
              "      <th>hba1c_change</th>\n",
              "      <th>type</th>\n",
              "      <th>dosage_start</th>\n",
              "      <th>dosage_end</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>veronika</td>\n",
              "      <td>jindrová</td>\n",
              "      <td>7.63</td>\n",
              "      <td>7.20</td>\n",
              "      <td>0.43</td>\n",
              "      <td>auralin</td>\n",
              "      <td>41</td>\n",
              "      <td>48</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>skye</td>\n",
              "      <td>gormanston</td>\n",
              "      <td>7.97</td>\n",
              "      <td>7.62</td>\n",
              "      <td>0.35</td>\n",
              "      <td>auralin</td>\n",
              "      <td>33</td>\n",
              "      <td>36</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>6</th>\n",
              "      <td>sophia</td>\n",
              "      <td>haugen</td>\n",
              "      <td>7.65</td>\n",
              "      <td>7.27</td>\n",
              "      <td>0.38</td>\n",
              "      <td>auralin</td>\n",
              "      <td>37</td>\n",
              "      <td>42</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>7</th>\n",
              "      <td>eddie</td>\n",
              "      <td>archer</td>\n",
              "      <td>7.89</td>\n",
              "      <td>7.55</td>\n",
              "      <td>0.34</td>\n",
              "      <td>auralin</td>\n",
              "      <td>31</td>\n",
              "      <td>38</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>9</th>\n",
              "      <td>asia</td>\n",
              "      <td>woźniak</td>\n",
              "      <td>7.76</td>\n",
              "      <td>7.37</td>\n",
              "      <td>0.39</td>\n",
              "      <td>auralin</td>\n",
              "      <td>30</td>\n",
              "      <td>36</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>...</th>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>688</th>\n",
              "      <td>christopher</td>\n",
              "      <td>woodward</td>\n",
              "      <td>7.51</td>\n",
              "      <td>7.06</td>\n",
              "      <td>0.45</td>\n",
              "      <td>novodra</td>\n",
              "      <td>55</td>\n",
              "      <td>51</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>690</th>\n",
              "      <td>maret</td>\n",
              "      <td>sultygov</td>\n",
              "      <td>7.67</td>\n",
              "      <td>7.30</td>\n",
              "      <td>0.37</td>\n",
              "      <td>novodra</td>\n",
              "      <td>26</td>\n",
              "      <td>23</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>694</th>\n",
              "      <td>lixue</td>\n",
              "      <td>hsueh</td>\n",
              "      <td>9.21</td>\n",
              "      <td>8.80</td>\n",
              "      <td>0.41</td>\n",
              "      <td>novodra</td>\n",
              "      <td>22</td>\n",
              "      <td>23</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>696</th>\n",
              "      <td>jakob</td>\n",
              "      <td>jakobsen</td>\n",
              "      <td>7.96</td>\n",
              "      <td>7.51</td>\n",
              "      <td>0.45</td>\n",
              "      <td>novodra</td>\n",
              "      <td>28</td>\n",
              "      <td>26</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>698</th>\n",
              "      <td>berta</td>\n",
              "      <td>napolitani</td>\n",
              "      <td>7.68</td>\n",
              "      <td>7.21</td>\n",
              "      <td>0.47</td>\n",
              "      <td>novodra</td>\n",
              "      <td>42</td>\n",
              "      <td>44</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>350 rows × 8 columns</p>\n",
              "</div>\n",
              "    <div class=\"colab-df-buttons\">\n",
              "\n",
              "  <div class=\"colab-df-container\">\n",
              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-bc66d213-bdfa-44b1-96a3-58e462af5e7b')\"\n",
              "            title=\"Convert this dataframe to an interactive table.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    .colab-df-buttons div {\n",
              "      margin-bottom: 4px;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const buttonEl =\n",
              "        document.querySelector('#df-bc66d213-bdfa-44b1-96a3-58e462af5e7b button.colab-df-convert');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function convertToInteractive(key) {\n",
              "        const element = document.querySelector('#df-bc66d213-bdfa-44b1-96a3-58e462af5e7b');\n",
              "        const dataTable =\n",
              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                    [key], {});\n",
              "        if (!dataTable) return;\n",
              "\n",
              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "          + ' to learn more about interactive tables.';\n",
              "        element.innerHTML = '';\n",
              "        dataTable['output_type'] = 'display_data';\n",
              "        await google.colab.output.renderOutput(dataTable, element);\n",
              "        const docLink = document.createElement('div');\n",
              "        docLink.innerHTML = docLinkHtml;\n",
              "        element.appendChild(docLink);\n",
              "      }\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "\n",
              "<div id=\"df-68635aef-8499-4c1f-bd4a-37565593867d\">\n",
              "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-68635aef-8499-4c1f-bd4a-37565593867d')\"\n",
              "            title=\"Suggest charts\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "     width=\"24px\">\n",
              "    <g>\n",
              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "    </g>\n",
              "</svg>\n",
              "  </button>\n",
              "\n",
              "<style>\n",
              "  .colab-df-quickchart {\n",
              "      --bg-color: #E8F0FE;\n",
              "      --fill-color: #1967D2;\n",
              "      --hover-bg-color: #E2EBFA;\n",
              "      --hover-fill-color: #174EA6;\n",
              "      --disabled-fill-color: #AAA;\n",
              "      --disabled-bg-color: #DDD;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart {\n",
              "      --bg-color: #3B4455;\n",
              "      --fill-color: #D2E3FC;\n",
              "      --hover-bg-color: #434B5C;\n",
              "      --hover-fill-color: #FFFFFF;\n",
              "      --disabled-bg-color: #3B4455;\n",
              "      --disabled-fill-color: #666;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart {\n",
              "    background-color: var(--bg-color);\n",
              "    border: none;\n",
              "    border-radius: 50%;\n",
              "    cursor: pointer;\n",
              "    display: none;\n",
              "    fill: var(--fill-color);\n",
              "    height: 32px;\n",
              "    padding: 0;\n",
              "    width: 32px;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart:hover {\n",
              "    background-color: var(--hover-bg-color);\n",
              "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "    fill: var(--button-hover-fill-color);\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart-complete:disabled,\n",
              "  .colab-df-quickchart-complete:disabled:hover {\n",
              "    background-color: var(--disabled-bg-color);\n",
              "    fill: var(--disabled-fill-color);\n",
              "    box-shadow: none;\n",
              "  }\n",
              "\n",
              "  .colab-df-spinner {\n",
              "    border: 2px solid var(--fill-color);\n",
              "    border-color: transparent;\n",
              "    border-bottom-color: var(--fill-color);\n",
              "    animation:\n",
              "      spin 1s steps(1) infinite;\n",
              "  }\n",
              "\n",
              "  @keyframes spin {\n",
              "    0% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "      border-left-color: var(--fill-color);\n",
              "    }\n",
              "    20% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    30% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    40% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    60% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    80% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "    90% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "  }\n",
              "</style>\n",
              "\n",
              "  <script>\n",
              "    async function quickchart(key) {\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#' + key + ' button');\n",
              "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
              "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
              "      try {\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'suggestCharts', [key], {});\n",
              "      } catch (error) {\n",
              "        console.error('Error during call to suggestCharts:', error);\n",
              "      }\n",
              "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
              "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
              "    }\n",
              "    (() => {\n",
              "      let quickchartButtonEl =\n",
              "        document.querySelector('#df-68635aef-8499-4c1f-bd4a-37565593867d button');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "    })();\n",
              "  </script>\n",
              "</div>\n",
              "\n",
              "  <div id=\"id_e0da16ba-8b32-4c4a-9bef-3d60e9fa0ffa\">\n",
              "    <style>\n",
              "      .colab-df-generate {\n",
              "        background-color: #E8F0FE;\n",
              "        border: none;\n",
              "        border-radius: 50%;\n",
              "        cursor: pointer;\n",
              "        display: none;\n",
              "        fill: #1967D2;\n",
              "        height: 32px;\n",
              "        padding: 0 0 0 0;\n",
              "        width: 32px;\n",
              "      }\n",
              "\n",
              "      .colab-df-generate:hover {\n",
              "        background-color: #E2EBFA;\n",
              "        box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "        fill: #174EA6;\n",
              "      }\n",
              "\n",
              "      [theme=dark] .colab-df-generate {\n",
              "        background-color: #3B4455;\n",
              "        fill: #D2E3FC;\n",
              "      }\n",
              "\n",
              "      [theme=dark] .colab-df-generate:hover {\n",
              "        background-color: #434B5C;\n",
              "        box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "        filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "        fill: #FFFFFF;\n",
              "      }\n",
              "    </style>\n",
              "    <button class=\"colab-df-generate\" onclick=\"generateWithVariable('treatments_df')\"\n",
              "            title=\"Generate code using this dataframe.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M7,19H8.4L18.45,9,17,7.55,7,17.6ZM5,21V16.75L18.45,3.32a2,2,0,0,1,2.83,0l1.4,1.43a1.91,1.91,0,0,1,.58,1.4,1.91,1.91,0,0,1-.58,1.4L9.25,21ZM18.45,9,17,7.55Zm-12,3A5.31,5.31,0,0,0,4.9,8.1,5.31,5.31,0,0,0,1,6.5,5.31,5.31,0,0,0,4.9,4.9,5.31,5.31,0,0,0,6.5,1,5.31,5.31,0,0,0,8.1,4.9,5.31,5.31,0,0,0,12,6.5,5.46,5.46,0,0,0,6.5,12Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "    <script>\n",
              "      (() => {\n",
              "      const buttonEl =\n",
              "        document.querySelector('#id_e0da16ba-8b32-4c4a-9bef-3d60e9fa0ffa button.colab-df-generate');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      buttonEl.onclick = () => {\n",
              "        google.colab.notebook.generateWithVariable('treatments_df');\n",
              "      }\n",
              "      })();\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "    </div>\n",
              "  </div>\n"
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "dataframe",
              "variable_name": "treatments_df",
              "summary": "{\n  \"name\": \"treatments_df\",\n  \"rows\": 350,\n  \"fields\": [\n    {\n      \"column\": \"given_name\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 338,\n        \"samples\": [\n          \"cintia\",\n          \"christopher\",\n          \"bao\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"surname\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 333,\n        \"samples\": [\n          \"zhdanov\",\n          \"villalpando\",\n          \"tucker\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"hba1c_start\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0.5453279522653312,\n        \"min\": 7.5,\n        \"max\": 9.95,\n        \"num_unique_values\": 101,\n        \"samples\": [\n          8.51,\n          7.88,\n          9.64\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"hba1c_end\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0.5454563528170605,\n        \"min\": 7.01,\n        \"max\": 9.58,\n        \"num_unique_values\": 113,\n        \"samples\": [\n          7.92,\n          7.37,\n          8.46\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"hba1c_change\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0.059283482640262015,\n        \"min\": 0.1999999999999993,\n        \"max\": 0.54,\n        \"num_unique_values\": 59,\n        \"samples\": [\n          0.4299999999999997,\n          0.5099999999999998,\n          0.2699999999999996\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"type\",\n      \"properties\": {\n        \"dtype\": \"category\",\n        \"num_unique_values\": 2,\n        \"samples\": [\n          \"novodra\",\n          \"auralin\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"dosage_start\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 9,\n        \"min\": 19,\n        \"max\": 63,\n        \"num_unique_values\": 44,\n        \"samples\": [\n          26,\n          25\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"dosage_end\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 10,\n        \"min\": 21,\n        \"max\": 75,\n        \"num_unique_values\": 49,\n        \"samples\": [\n          43,\n          21\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    }\n  ]\n}"
            }
          },
          "metadata": {},
          "execution_count": 190
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "treatments_df = treatments_df.merge(adverse_reactions_df, how ='left', on=['given_name','surname'])"
      ],
      "metadata": {
        "id": "LQCCAS0V0IoU"
      },
      "execution_count": 191,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "treatments_df"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 423
        },
        "id": "iXVn_t8e0Vmb",
        "outputId": "056e240a-53b9-4f0a-b7a0-287673ddfb6d"
      },
      "execution_count": 192,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "      given_name     surname  hba1c_start  hba1c_end  hba1c_change     type  \\\n",
              "0       veronika    jindrová         7.63       7.20          0.43  auralin   \n",
              "1           skye  gormanston         7.97       7.62          0.35  auralin   \n",
              "2         sophia      haugen         7.65       7.27          0.38  auralin   \n",
              "3          eddie      archer         7.89       7.55          0.34  auralin   \n",
              "4           asia     woźniak         7.76       7.37          0.39  auralin   \n",
              "..           ...         ...          ...        ...           ...      ...   \n",
              "345  christopher    woodward         7.51       7.06          0.45  novodra   \n",
              "346        maret    sultygov         7.67       7.30          0.37  novodra   \n",
              "347        lixue       hsueh         9.21       8.80          0.41  novodra   \n",
              "348        jakob    jakobsen         7.96       7.51          0.45  novodra   \n",
              "349        berta  napolitani         7.68       7.21          0.47  novodra   \n",
              "\n",
              "     dosage_start  dosage_end           adverse_reaction  \n",
              "0              41          48                        NaN  \n",
              "1              33          36                        NaN  \n",
              "2              37          42                        NaN  \n",
              "3              31          38                        NaN  \n",
              "4              30          36                        NaN  \n",
              "..            ...         ...                        ...  \n",
              "345            55          51                     nausea  \n",
              "346            26          23                        NaN  \n",
              "347            22          23  injection site discomfort  \n",
              "348            28          26               hypoglycemia  \n",
              "349            42          44  injection site discomfort  \n",
              "\n",
              "[350 rows x 9 columns]"
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-3bb9a86f-0a54-430d-afc4-b2c6d97dc03d\" class=\"colab-df-container\">\n",
              "    <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>given_name</th>\n",
              "      <th>surname</th>\n",
              "      <th>hba1c_start</th>\n",
              "      <th>hba1c_end</th>\n",
              "      <th>hba1c_change</th>\n",
              "      <th>type</th>\n",
              "      <th>dosage_start</th>\n",
              "      <th>dosage_end</th>\n",
              "      <th>adverse_reaction</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>veronika</td>\n",
              "      <td>jindrová</td>\n",
              "      <td>7.63</td>\n",
              "      <td>7.20</td>\n",
              "      <td>0.43</td>\n",
              "      <td>auralin</td>\n",
              "      <td>41</td>\n",
              "      <td>48</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>skye</td>\n",
              "      <td>gormanston</td>\n",
              "      <td>7.97</td>\n",
              "      <td>7.62</td>\n",
              "      <td>0.35</td>\n",
              "      <td>auralin</td>\n",
              "      <td>33</td>\n",
              "      <td>36</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>sophia</td>\n",
              "      <td>haugen</td>\n",
              "      <td>7.65</td>\n",
              "      <td>7.27</td>\n",
              "      <td>0.38</td>\n",
              "      <td>auralin</td>\n",
              "      <td>37</td>\n",
              "      <td>42</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>eddie</td>\n",
              "      <td>archer</td>\n",
              "      <td>7.89</td>\n",
              "      <td>7.55</td>\n",
              "      <td>0.34</td>\n",
              "      <td>auralin</td>\n",
              "      <td>31</td>\n",
              "      <td>38</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>asia</td>\n",
              "      <td>woźniak</td>\n",
              "      <td>7.76</td>\n",
              "      <td>7.37</td>\n",
              "      <td>0.39</td>\n",
              "      <td>auralin</td>\n",
              "      <td>30</td>\n",
              "      <td>36</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>...</th>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>345</th>\n",
              "      <td>christopher</td>\n",
              "      <td>woodward</td>\n",
              "      <td>7.51</td>\n",
              "      <td>7.06</td>\n",
              "      <td>0.45</td>\n",
              "      <td>novodra</td>\n",
              "      <td>55</td>\n",
              "      <td>51</td>\n",
              "      <td>nausea</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>346</th>\n",
              "      <td>maret</td>\n",
              "      <td>sultygov</td>\n",
              "      <td>7.67</td>\n",
              "      <td>7.30</td>\n",
              "      <td>0.37</td>\n",
              "      <td>novodra</td>\n",
              "      <td>26</td>\n",
              "      <td>23</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>347</th>\n",
              "      <td>lixue</td>\n",
              "      <td>hsueh</td>\n",
              "      <td>9.21</td>\n",
              "      <td>8.80</td>\n",
              "      <td>0.41</td>\n",
              "      <td>novodra</td>\n",
              "      <td>22</td>\n",
              "      <td>23</td>\n",
              "      <td>injection site discomfort</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>348</th>\n",
              "      <td>jakob</td>\n",
              "      <td>jakobsen</td>\n",
              "      <td>7.96</td>\n",
              "      <td>7.51</td>\n",
              "      <td>0.45</td>\n",
              "      <td>novodra</td>\n",
              "      <td>28</td>\n",
              "      <td>26</td>\n",
              "      <td>hypoglycemia</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>349</th>\n",
              "      <td>berta</td>\n",
              "      <td>napolitani</td>\n",
              "      <td>7.68</td>\n",
              "      <td>7.21</td>\n",
              "      <td>0.47</td>\n",
              "      <td>novodra</td>\n",
              "      <td>42</td>\n",
              "      <td>44</td>\n",
              "      <td>injection site discomfort</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>350 rows × 9 columns</p>\n",
              "</div>\n",
              "    <div class=\"colab-df-buttons\">\n",
              "\n",
              "  <div class=\"colab-df-container\">\n",
              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-3bb9a86f-0a54-430d-afc4-b2c6d97dc03d')\"\n",
              "            title=\"Convert this dataframe to an interactive table.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    .colab-df-buttons div {\n",
              "      margin-bottom: 4px;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const buttonEl =\n",
              "        document.querySelector('#df-3bb9a86f-0a54-430d-afc4-b2c6d97dc03d button.colab-df-convert');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function convertToInteractive(key) {\n",
              "        const element = document.querySelector('#df-3bb9a86f-0a54-430d-afc4-b2c6d97dc03d');\n",
              "        const dataTable =\n",
              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                    [key], {});\n",
              "        if (!dataTable) return;\n",
              "\n",
              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "          + ' to learn more about interactive tables.';\n",
              "        element.innerHTML = '';\n",
              "        dataTable['output_type'] = 'display_data';\n",
              "        await google.colab.output.renderOutput(dataTable, element);\n",
              "        const docLink = document.createElement('div');\n",
              "        docLink.innerHTML = docLinkHtml;\n",
              "        element.appendChild(docLink);\n",
              "      }\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "\n",
              "<div id=\"df-e72f5ac0-4cb6-429b-8030-3977ce365340\">\n",
              "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-e72f5ac0-4cb6-429b-8030-3977ce365340')\"\n",
              "            title=\"Suggest charts\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "     width=\"24px\">\n",
              "    <g>\n",
              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "    </g>\n",
              "</svg>\n",
              "  </button>\n",
              "\n",
              "<style>\n",
              "  .colab-df-quickchart {\n",
              "      --bg-color: #E8F0FE;\n",
              "      --fill-color: #1967D2;\n",
              "      --hover-bg-color: #E2EBFA;\n",
              "      --hover-fill-color: #174EA6;\n",
              "      --disabled-fill-color: #AAA;\n",
              "      --disabled-bg-color: #DDD;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart {\n",
              "      --bg-color: #3B4455;\n",
              "      --fill-color: #D2E3FC;\n",
              "      --hover-bg-color: #434B5C;\n",
              "      --hover-fill-color: #FFFFFF;\n",
              "      --disabled-bg-color: #3B4455;\n",
              "      --disabled-fill-color: #666;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart {\n",
              "    background-color: var(--bg-color);\n",
              "    border: none;\n",
              "    border-radius: 50%;\n",
              "    cursor: pointer;\n",
              "    display: none;\n",
              "    fill: var(--fill-color);\n",
              "    height: 32px;\n",
              "    padding: 0;\n",
              "    width: 32px;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart:hover {\n",
              "    background-color: var(--hover-bg-color);\n",
              "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "    fill: var(--button-hover-fill-color);\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart-complete:disabled,\n",
              "  .colab-df-quickchart-complete:disabled:hover {\n",
              "    background-color: var(--disabled-bg-color);\n",
              "    fill: var(--disabled-fill-color);\n",
              "    box-shadow: none;\n",
              "  }\n",
              "\n",
              "  .colab-df-spinner {\n",
              "    border: 2px solid var(--fill-color);\n",
              "    border-color: transparent;\n",
              "    border-bottom-color: var(--fill-color);\n",
              "    animation:\n",
              "      spin 1s steps(1) infinite;\n",
              "  }\n",
              "\n",
              "  @keyframes spin {\n",
              "    0% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "      border-left-color: var(--fill-color);\n",
              "    }\n",
              "    20% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    30% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    40% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    60% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    80% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "    90% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "  }\n",
              "</style>\n",
              "\n",
              "  <script>\n",
              "    async function quickchart(key) {\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#' + key + ' button');\n",
              "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
              "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
              "      try {\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'suggestCharts', [key], {});\n",
              "      } catch (error) {\n",
              "        console.error('Error during call to suggestCharts:', error);\n",
              "      }\n",
              "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
              "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
              "    }\n",
              "    (() => {\n",
              "      let quickchartButtonEl =\n",
              "        document.querySelector('#df-e72f5ac0-4cb6-429b-8030-3977ce365340 button');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "    })();\n",
              "  </script>\n",
              "</div>\n",
              "\n",
              "  <div id=\"id_e7cbe80e-2517-4548-a6d5-13c87bfa5675\">\n",
              "    <style>\n",
              "      .colab-df-generate {\n",
              "        background-color: #E8F0FE;\n",
              "        border: none;\n",
              "        border-radius: 50%;\n",
              "        cursor: pointer;\n",
              "        display: none;\n",
              "        fill: #1967D2;\n",
              "        height: 32px;\n",
              "        padding: 0 0 0 0;\n",
              "        width: 32px;\n",
              "      }\n",
              "\n",
              "      .colab-df-generate:hover {\n",
              "        background-color: #E2EBFA;\n",
              "        box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "        fill: #174EA6;\n",
              "      }\n",
              "\n",
              "      [theme=dark] .colab-df-generate {\n",
              "        background-color: #3B4455;\n",
              "        fill: #D2E3FC;\n",
              "      }\n",
              "\n",
              "      [theme=dark] .colab-df-generate:hover {\n",
              "        background-color: #434B5C;\n",
              "        box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "        filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "        fill: #FFFFFF;\n",
              "      }\n",
              "    </style>\n",
              "    <button class=\"colab-df-generate\" onclick=\"generateWithVariable('treatments_df')\"\n",
              "            title=\"Generate code using this dataframe.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M7,19H8.4L18.45,9,17,7.55,7,17.6ZM5,21V16.75L18.45,3.32a2,2,0,0,1,2.83,0l1.4,1.43a1.91,1.91,0,0,1,.58,1.4,1.91,1.91,0,0,1-.58,1.4L9.25,21ZM18.45,9,17,7.55Zm-12,3A5.31,5.31,0,0,0,4.9,8.1,5.31,5.31,0,0,0,1,6.5,5.31,5.31,0,0,0,4.9,4.9,5.31,5.31,0,0,0,6.5,1,5.31,5.31,0,0,0,8.1,4.9,5.31,5.31,0,0,0,12,6.5,5.46,5.46,0,0,0,6.5,12Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "    <script>\n",
              "      (() => {\n",
              "      const buttonEl =\n",
              "        document.querySelector('#id_e7cbe80e-2517-4548-a6d5-13c87bfa5675 button.colab-df-generate');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      buttonEl.onclick = () => {\n",
              "        google.colab.notebook.generateWithVariable('treatments_df');\n",
              "      }\n",
              "      })();\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "    </div>\n",
              "  </div>\n"
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "dataframe",
              "variable_name": "treatments_df",
              "summary": "{\n  \"name\": \"treatments_df\",\n  \"rows\": 350,\n  \"fields\": [\n    {\n      \"column\": \"given_name\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 338,\n        \"samples\": [\n          \"cintia\",\n          \"christopher\",\n          \"bao\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"surname\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 333,\n        \"samples\": [\n          \"zhdanov\",\n          \"villalpando\",\n          \"tucker\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"hba1c_start\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0.5453279522653312,\n        \"min\": 7.5,\n        \"max\": 9.95,\n        \"num_unique_values\": 101,\n        \"samples\": [\n          8.51,\n          7.88,\n          9.64\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"hba1c_end\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0.5454563528170605,\n        \"min\": 7.01,\n        \"max\": 9.58,\n        \"num_unique_values\": 113,\n        \"samples\": [\n          7.92,\n          7.37,\n          8.46\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"hba1c_change\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0.059283482640262015,\n        \"min\": 0.1999999999999993,\n        \"max\": 0.54,\n        \"num_unique_values\": 59,\n        \"samples\": [\n          0.4299999999999997,\n          0.5099999999999998,\n          0.2699999999999996\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"type\",\n      \"properties\": {\n        \"dtype\": \"category\",\n        \"num_unique_values\": 2,\n        \"samples\": [\n          \"novodra\",\n          \"auralin\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"dosage_start\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 9,\n        \"min\": 19,\n        \"max\": 63,\n        \"num_unique_values\": 44,\n        \"samples\": [\n          26,\n          25\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"dosage_end\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 10,\n        \"min\": 21,\n        \"max\": 75,\n        \"num_unique_values\": 49,\n        \"samples\": [\n          43,\n          21\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"adverse_reaction\",\n      \"properties\": {\n        \"dtype\": \"category\",\n        \"num_unique_values\": 6,\n        \"samples\": [\n          \"hypoglycemia\",\n          \"throat irritation\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    }\n  ]\n}"
            }
          },
          "metadata": {},
          "execution_count": 192
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "Mk-VoNsf0mIG"
      },
      "execution_count": 192,
      "outputs": []
    }
  ]
}