langchain_lex_agent.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "provenance": [],
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    },
    "gpuClass": "standard"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/willdphan/langchain-lex-agent/blob/main/langchain_lex_agent.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "!pip install -qU datasets pod-gpt pinecone-client[grpc] langchain openai tqdm"
      ],
      "metadata": {
        "id": "pADty8WtkcGq"
      },
      "execution_count": 20,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Set api keys, etc:"
      ],
      "metadata": {
        "id": "nYzuZiAP5HGL"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "OPENAI_API_KEY = \"OPENAI_API_KEY\"  # platform.openai.com\n",
        "PINECONE_API_KEY = \"PINECONE_API_KEY\"  # app.pinecone.io\n",
        "PINECONE_ENV = \"PINECONE_ENV\""
      ],
      "metadata": {
        "id": "f1R8X32-41vc"
      },
      "execution_count": 21,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "First we download a prebuilt Lex Fridman podcast transcriptions dataset from HF:"
      ],
      "metadata": {
        "id": "4RYCg6ep4L0p"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from datasets import load_dataset\n",
        "\n",
        "data = load_dataset(\n",
        "    'jamescalam/lex-transcripts',\n",
        "    split='train'\n",
        ")\n",
        "data"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "2AT6IR3t4Rcl",
        "outputId": "e0d0b9f3-b9fa-4209-aa7f-dc27abb3f01c"
      },
      "execution_count": 22,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "WARNING:datasets.builder:Found cached dataset json (/root/.cache/huggingface/datasets/jamescalam___json/jamescalam--lex-transcripts-6a9688b7915283fe/0.0.0/fe5dd6ea2639a6df622901539cb550cf8797e5a6b2dd7af1cf934bed8e233e6e)\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "Dataset({\n",
              "    features: ['video_id', 'channel_id', 'title', 'published', 'transcript', 'source'],\n",
              "    num_rows: 499\n",
              "})"
            ]
          },
          "metadata": {},
          "execution_count": 22
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Initialize the `indexer` object. pod_gpt is a python package that provides an implementation of the OpenAI GPT. The Indexer class is used to create an indexer, which allows you to index and search text documents using the language model. "
      ],
      "metadata": {
        "id": "nG7-U4xC4rcE"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "import pod_gpt\n",
        "\n",
        "indexer = pod_gpt.Indexer(\n",
        "    openai_api_key=OPENAI_API_KEY,\n",
        "    pinecone_api_key=PINECONE_API_KEY,\n",
        "    pinecone_environment=PINECONE_ENV,\n",
        "    index_name=\"pod-gpt\"\n",
        ")"
      ],
      "metadata": {
        "id": "imF-Jxft4QjJ"
      },
      "execution_count": 23,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "The Python loop iterates through the data, formats the published field of each row, creates a VideoRecord object, and indexes the object to a Pinecone index using the pod_gpt.Indexer class. The tqdm.auto library is used to provide a progress bar for the indexing process.\n",
        "\n",
        "In each iteration, the code formats the published field of the row object as a string in the '%Y%m%d' format using the strftime() method.\n",
        "\n",
        "When creating a VideoRecord object, the fields are populated with the corresponding metadata for a particular video."
      ],
      "metadata": {
        "id": "Yspdavspo3US"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from tqdm.auto import tqdm\n",
        "\n",
        "for row in data:\n",
        "    row['published'] = row['published'].strftime('%Y%m%d')\n",
        "    indexer(pod_gpt.VideoRecord(**row))"
      ],
      "metadata": {
        "id": "7P9Cqb_Z4RSZ"
      },
      "execution_count": 24,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "The code then checks if an index with the name \"pod-gpt\" exists by calling the \"list_indexes()\" function and checking if the index name is in the list of indexes.\n",
        "\n",
        "Finally, the code creates an index object for the \"pod-gpt\" index by calling the \"Index()\" function and passing in the index name as a parameter."
      ],
      "metadata": {
        "id": "MU-C0ivarD8n"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "import pinecone\n",
        "\n",
        "pinecone.init(\n",
        "    api_key=PINECONE_API_KEY,  # app.pinecone.io\n",
        "    environment=PINECONE_ENV  # next to API key in console\n",
        ")\n",
        "\n",
        "index_name = \"pod-gpt\"\n",
        "\n",
        "if index_name not in pinecone.list_indexes():\n",
        "    raise ValueError(\n",
        "        f\"No '{index_name}' index exists. You must create the index before \"\n",
        "        \"running this notebook. Please refer to the walkthrough at \"\n",
        "        \"'github.com/pinecone-io/examples'.\"  # TODO add full link\n",
        "    )\n",
        "\n",
        "index = pinecone.Index(index_name)"
      ],
      "metadata": {
        "id": "y338927dkxlY"
      },
      "execution_count": 25,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Initialize the retrieval components (embedding model and vector DB)\n",
        "\n",
        "The first two import statements are necessary classes from the \"langchain\" library to create a vector database.\n",
        "\n",
        "Next we create a Pinecone object\n",
        "\n",
        "We create an object of the \"Pinecone\" class with the \"index\" parameter set to the value of the \"index\" variable, the \"embedding_function\" parameter set to the \"embed_query\" function of the \"embeddings\" object created earlier, and the \"text_key\" parameter set to \"text\"."
      ],
      "metadata": {
        "id": "gFTiO7eX2mkN"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from langchain.embeddings.openai import OpenAIEmbeddings\n",
        "from langchain.vectorstores import Pinecone\n",
        "\n",
        "embeddings = OpenAIEmbeddings(openai_api_key=OPENAI_API_KEY)\n",
        "\n",
        "vectordb = Pinecone(\n",
        "    index=index,\n",
        "    embedding_function=embeddings.embed_query,\n",
        "    text_key=\"text\"\n",
        ")"
      ],
      "metadata": {
        "id": "GPvqFFGXcFGF"
      },
      "execution_count": 26,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Initialize `gpt-3.5-turbo` chat model. "
      ],
      "metadata": {
        "id": "bPFXYJE92mNm"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from langchain.chat_models import ChatOpenAI\n",
        "\n",
        "llm=ChatOpenAI(\n",
        "    openai_api_key=OPENAI_API_KEY,\n",
        "    temperature=0,\n",
        "    model_name='gpt-3.5-turbo'\n",
        ")"
      ],
      "metadata": {
        "id": "jESRtKeJk1IO"
      },
      "execution_count": 27,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "One additional thing we have here is the `chain_type=\"stuff\"`. There are two options here, `\"stuff\"` or `\"map_reduce\"`. The `map_reduce` option essentially summarizes returned documents, whereas the `stuff` option just returns the retrieved documents as is.\n",
        "\n",
        "The `retriever` is ready and can be used by us like this. However, we need to convert it into a `Tool` to be used by our conversational agent. To do that we need the `retriever` itself, a tool description, and a tool name. We use these to initialize the tool like so:"
      ],
      "metadata": {
        "id": "NtaTKxTJG7J8"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from langchain.chains import RetrievalQA\n",
        "\n",
        "retriever = RetrievalQA.from_chain_type(\n",
        "    llm=llm,\n",
        "    chain_type=\"stuff\",\n",
        "    retriever=vectordb.as_retriever()\n",
        ")"
      ],
      "metadata": {
        "id": "PkTHIAE47uJp"
      },
      "execution_count": 28,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "tool_desc = \"\"\"Use this tool to answer user questions using Lex\n",
        "Fridman podcasts. If the user states 'ask Lex' use this tool to get\n",
        "the answer. This tool can also be used for follow up questions from\n",
        "the user.\"\"\""
      ],
      "metadata": {
        "id": "rEQEdRQRmCbO"
      },
      "execution_count": 29,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "This line of code imports the necessary class from the \"langchain\" library to create an agent.\n",
        "\n",
        "This list represents the tools that the agent will use to answer questions. In this case, there is only one tool, which is a question-answering model that retrieves answers from a vector database.\n",
        "\n",
        "The \"Tool\" object is initialized with several parameters.\n",
        "\n",
        "The \"func\" parameter specifies the function that will be used to answer questions. In this case, the \"run\" method of the \"retriever\" object is used, which retrieves answers from the vector database.\n",
        "\n",
        "The \"description\" parameter specifies a description of the tool. This parameter is set to the value of the \"tool_desc\" variable.\n",
        "\n",
        "The \"name\" parameter specifies a name for the tool. In this case, the name is set to \"Lex Fridman DB\".\n",
        "\n",
        "The \"tools\" list represents the tools that the agent will use to answer questions."
      ],
      "metadata": {
        "id": "SEAO7kaswGJF"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from langchain.agents import Tool\n",
        "\n",
        "tools = [Tool(\n",
        "    func=retriever.run,\n",
        "    description=tool_desc,\n",
        "    name='Lex Fridman DB'\n",
        ")]"
      ],
      "metadata": {
        "id": "abClSi8GlC8O"
      },
      "execution_count": 30,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "With that, we're ready to initialize the conversational agent. As it is a *conversational* agent, it does need some form of [conversational memory](https://www.pinecone.io/learn/langchain-conversational-memory/). For this we will use the `ConversationBufferWindowMemory` option, which will *remember* the previous `k` interactions between the user and the AI."
      ],
      "metadata": {
        "id": "T9o6IIHSHwVX"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from langchain.chains.conversation.memory import ConversationBufferWindowMemory\n",
        "\n",
        "memory = ConversationBufferWindowMemory(\n",
        "    memory_key=\"chat_history\",  # important to align with agent prompt (below)\n",
        "    k=5,\n",
        "    return_messages=True\n",
        ")"
      ],
      "metadata": {
        "id": "L_CRqHPrmcHo"
      },
      "execution_count": 31,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Important items in `agent` parameter:\n",
        "\n",
        "* `chat-conversational`: for chatbots with conversational memory.\n",
        "* `react`: refers to the ReAct framework.\n",
        "* `description`: because the LLM relies on the tool description to decide which tool to use."
      ],
      "metadata": {
        "id": "f8EkS7lG3ElB"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from langchain.agents import initialize_agent\n",
        "\n",
        "conversational_agent = initialize_agent(\n",
        "    agent='chat-conversational-react-description', \n",
        "    tools=tools, \n",
        "    llm=llm,\n",
        "    verbose=True,\n",
        "    max_iterations=2,\n",
        "    early_stopping_method=\"generate\",\n",
        "    memory=memory,\n",
        ")"
      ],
      "metadata": {
        "id": "ArdEDGH7nBRR"
      },
      "execution_count": 32,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Conversational Agent Prompt\n",
        "The prompt of the conversational agent is fairly complex. Let's create it then break it down."
      ],
      "metadata": {
        "id": "qmYCv2fEwicS"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "conversational_agent.agent.llm_chain.prompt"
      ],
      "metadata": {
        "id": "m1ApK_bkBOQ7",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "ae3ffa47-8e6c-4f1a-dd4d-6dbd015df63f"
      },
      "execution_count": 33,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "ChatPromptTemplate(input_variables=['input', 'chat_history', 'agent_scratchpad'], output_parser=None, partial_variables={}, messages=[SystemMessagePromptTemplate(prompt=PromptTemplate(input_variables=[], output_parser=None, partial_variables={}, template='Assistant is a large language model trained by OpenAI.\\n\\nAssistant is designed to be able to assist with a wide range of tasks, from answering simple questions to providing in-depth explanations and discussions on a wide range of topics. As a language model, Assistant is able to generate human-like text based on the input it receives, allowing it to engage in natural-sounding conversations and provide responses that are coherent and relevant to the topic at hand.\\n\\nAssistant is constantly learning and improving, and its capabilities are constantly evolving. It is able to process and understand large amounts of text, and can use this knowledge to provide accurate and informative responses to a wide range of questions. Additionally, Assistant is able to generate its own text based on the input it receives, allowing it to engage in discussions and provide explanations and descriptions on a wide range of topics.\\n\\nOverall, Assistant is a powerful system that can help with a wide range of tasks and provide valuable insights and information on a wide range of topics. Whether you need help with a specific question or just want to have a conversation about a particular topic, Assistant is here to assist.', template_format='f-string', validate_template=True), additional_kwargs={}), MessagesPlaceholder(variable_name='chat_history'), HumanMessagePromptTemplate(prompt=PromptTemplate(input_variables=['input'], output_parser=None, partial_variables={}, template='TOOLS\\n------\\nAssistant can ask the user to use tools to look up information that may be helpful in answering the users original question. The tools the human can use are:\\n\\n> Lex Fridman DB: Use this tool to answer user questions using Lex\\nFridman podcasts. If the user states \\'ask Lex\\' use this tool to get\\nthe answer. This tool can also be used for follow up questions from\\nthe user.\\n\\nRESPONSE FORMAT INSTRUCTIONS\\n----------------------------\\n\\nWhen responding to me please, please output a response in one of two formats:\\n\\n**Option 1:**\\nUse this if you want the human to use a tool.\\nMarkdown code snippet formatted in the following schema:\\n\\n```json\\n{{\\n    \"action\": string \\\\ The action to take. Must be one of Lex Fridman DB\\n    \"action_input\": string \\\\ The input to the action\\n}}\\n```\\n\\n**Option #2:**\\nUse this if you want to respond directly to the human. Markdown code snippet formatted in the following schema:\\n\\n```json\\n{{\\n    \"action\": \"Final Answer\",\\n    \"action_input\": string \\\\ You should put what you want to return to use here\\n}}\\n```\\n\\nUSER\\'S INPUT\\n--------------------\\nHere is the user\\'s input (remember to respond with a markdown code snippet of a json blob with a single action, and NOTHING else):\\n\\n{input}', template_format='f-string', validate_template=True), additional_kwargs={}), MessagesPlaceholder(variable_name='agent_scratchpad')])"
            ]
          },
          "metadata": {},
          "execution_count": 33
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Creating a System Message\n",
        "\n",
        "This line of code creates a string variable \"sys_msg\" that contains a message that will be displayed to the user. This message is a system message that introduces the chatbot to the user.\n",
        "\n",
        "The third line of code creates a prompt for the conversational agent with several parameters set.\n",
        "\n",
        "This prompt represents the message that will be displayed to the user to prompt them to ask a question.\n",
        "\n",
        "The \"conversational_agent.agent.create_prompt()\" function is called to create the prompt. The \"system_message\" parameter specifies the system message that will be displayed to the user. This parameter is set to the value of the \"sys_msg\" variable.\n",
        "\n",
        "The \"tools\" parameter specifies the tools that the agent will use to answer questions. This parameter is set to the list of \"Tool\" objects that was created earlier.\n",
        "\n",
        "The last line of code sets the prompt for the conversational agent. The \"conversational_agent.agent.llm_chain\" object represents the conversational agent, and the \"prompt\" variable represents the prompt that was created earlier.\n",
        "\n",
        "When the agent is started, the prompt will be displayed to the user to prompt them to ask a question.\n"
      ],
      "metadata": {
        "id": "CafTqTBYwnW2"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "sys_msg = \"\"\"You are a helpful chatbot that answers the user's questions.\n",
        "\"\"\"\n",
        "\n",
        "prompt = conversational_agent.agent.create_prompt(\n",
        "    system_message=sys_msg,\n",
        "    tools=tools\n",
        ")\n",
        "conversational_agent.agent.llm_chain.prompt = prompt"
      ],
      "metadata": {
        "id": "ISOscagrnDyD"
      },
      "execution_count": 34,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "We can see the prompt template like so:"
      ],
      "metadata": {
        "id": "Ezpd5vaOMC-w"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "conversational_agent.agent.llm_chain.prompt"
      ],
      "metadata": {
        "id": "lvEAE9XKMFIR",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "3f4e4761-d32a-46f9-f8ea-450242508bb8"
      },
      "execution_count": 35,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "ChatPromptTemplate(input_variables=['input', 'chat_history', 'agent_scratchpad'], output_parser=None, partial_variables={}, messages=[SystemMessagePromptTemplate(prompt=PromptTemplate(input_variables=[], output_parser=None, partial_variables={}, template=\"You are a helpful chatbot that answers the user's questions.\\n\", template_format='f-string', validate_template=True), additional_kwargs={}), MessagesPlaceholder(variable_name='chat_history'), HumanMessagePromptTemplate(prompt=PromptTemplate(input_variables=['input'], output_parser=None, partial_variables={}, template='TOOLS\\n------\\nAssistant can ask the user to use tools to look up information that may be helpful in answering the users original question. The tools the human can use are:\\n\\n> Lex Fridman DB: Use this tool to answer user questions using Lex\\nFridman podcasts. If the user states \\'ask Lex\\' use this tool to get\\nthe answer. This tool can also be used for follow up questions from\\nthe user.\\n\\nRESPONSE FORMAT INSTRUCTIONS\\n----------------------------\\n\\nWhen responding to me please, please output a response in one of two formats:\\n\\n**Option 1:**\\nUse this if you want the human to use a tool.\\nMarkdown code snippet formatted in the following schema:\\n\\n```json\\n{{\\n    \"action\": string \\\\ The action to take. Must be one of Lex Fridman DB\\n    \"action_input\": string \\\\ The input to the action\\n}}\\n```\\n\\n**Option #2:**\\nUse this if you want to respond directly to the human. Markdown code snippet formatted in the following schema:\\n\\n```json\\n{{\\n    \"action\": \"Final Answer\",\\n    \"action_input\": string \\\\ You should put what you want to return to use here\\n}}\\n```\\n\\nUSER\\'S INPUT\\n--------------------\\nHere is the user\\'s input (remember to respond with a markdown code snippet of a json blob with a single action, and NOTHING else):\\n\\n{input}', template_format='f-string', validate_template=True), additional_kwargs={}), MessagesPlaceholder(variable_name='agent_scratchpad')])"
            ]
          },
          "metadata": {},
          "execution_count": 35
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "The conversational agent prompt is defined by the `ChatPromptTemplate`. Let's break it down:"
      ],
      "metadata": {
        "id": "56MZZYTTs0Jq"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "conversational_agent.agent.llm_chain.prompt.input_variables"
      ],
      "metadata": {
        "id": "4ybUWUOZOFAL",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "76d6ebc4-82cb-47b5-c16b-6c831e487a52"
      },
      "execution_count": 36,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "['input', 'chat_history', 'agent_scratchpad']"
            ]
          },
          "metadata": {},
          "execution_count": 36
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        " This prompt template contains *three* `input_variables`, those are:\n",
        "\n",
        "* `input`: The new user input to the chatbot, i.e. our prompt/query.\n",
        "\n",
        "* `chat_history`: We defined this above in the `ConversationBufferWindowMemory` definition.\n",
        "\n",
        "* `agent_scratchpad`: This is where we store the thoughts of the LLM as it is deciding which tools to interact with and *how* to interact with them.\n",
        "\n",
        "These `input_variables` are fed into the `messages` contained within the prompt template, let's see what we have there:"
      ],
      "metadata": {
        "id": "axHAsgRROFUO"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "conversational_agent.agent.llm_chain.prompt.messages"
      ],
      "metadata": {
        "id": "-3z2E1rlOIr_",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "adf0a24e-eb3e-4a49-877d-684ac03c995e"
      },
      "execution_count": 37,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[SystemMessagePromptTemplate(prompt=PromptTemplate(input_variables=[], output_parser=None, partial_variables={}, template=\"You are a helpful chatbot that answers the user's questions.\\n\", template_format='f-string', validate_template=True), additional_kwargs={}),\n",
              " MessagesPlaceholder(variable_name='chat_history'),\n",
              " HumanMessagePromptTemplate(prompt=PromptTemplate(input_variables=['input'], output_parser=None, partial_variables={}, template='TOOLS\\n------\\nAssistant can ask the user to use tools to look up information that may be helpful in answering the users original question. The tools the human can use are:\\n\\n> Lex Fridman DB: Use this tool to answer user questions using Lex\\nFridman podcasts. If the user states \\'ask Lex\\' use this tool to get\\nthe answer. This tool can also be used for follow up questions from\\nthe user.\\n\\nRESPONSE FORMAT INSTRUCTIONS\\n----------------------------\\n\\nWhen responding to me please, please output a response in one of two formats:\\n\\n**Option 1:**\\nUse this if you want the human to use a tool.\\nMarkdown code snippet formatted in the following schema:\\n\\n```json\\n{{\\n    \"action\": string \\\\ The action to take. Must be one of Lex Fridman DB\\n    \"action_input\": string \\\\ The input to the action\\n}}\\n```\\n\\n**Option #2:**\\nUse this if you want to respond directly to the human. Markdown code snippet formatted in the following schema:\\n\\n```json\\n{{\\n    \"action\": \"Final Answer\",\\n    \"action_input\": string \\\\ You should put what you want to return to use here\\n}}\\n```\\n\\nUSER\\'S INPUT\\n--------------------\\nHere is the user\\'s input (remember to respond with a markdown code snippet of a json blob with a single action, and NOTHING else):\\n\\n{input}', template_format='f-string', validate_template=True), additional_kwargs={}),\n",
              " MessagesPlaceholder(variable_name='agent_scratchpad')]"
            ]
          },
          "metadata": {},
          "execution_count": 37
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "\n",
        "It's a little hard to see here, but there are **three** components in `messages`. Those are:\n",
        "\n",
        "* `SystemMessagePromptTemplate`\n",
        "\n",
        "* `MessagesPlaceholder`\n",
        "\n",
        "* `HumanMessagePromptTemplate`\n",
        "\n",
        "Let's start with the first item, the `SystemMessage`:"
      ],
      "metadata": {
        "id": "XuMfxYayOIbu"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "conversational_agent.agent.llm_chain.prompt.messages[0]"
      ],
      "metadata": {
        "id": "D9JWEqIzs9oC",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "850d3fdf-cf55-40a9-ee48-174ee15c1b8d"
      },
      "execution_count": 38,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "SystemMessagePromptTemplate(prompt=PromptTemplate(input_variables=[], output_parser=None, partial_variables={}, template=\"You are a helpful chatbot that answers the user's questions.\\n\", template_format='f-string', validate_template=True), additional_kwargs={})"
            ]
          },
          "metadata": {},
          "execution_count": 38
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "print(\n",
        "    conversational_agent.agent.llm_chain.prompt.messages[0].prompt.template\n",
        ")"
      ],
      "metadata": {
        "id": "b2MkeX4_tIga",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "fe8f30cc-26a0-4777-d5b7-c4d9c9ba77f8"
      },
      "execution_count": 39,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "You are a helpful chatbot that answers the user's questions.\n",
            "\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "That is our initial system message that we set earlier with the `sys_msg`. There's not much to say about this other than it is used to \"prime\" (set the initial objective of) the model.\n",
        "\n",
        "Next we have the `MessagesPlaceholder`:"
      ],
      "metadata": {
        "id": "pill7M9AtdIJ"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "conversational_agent.agent.llm_chain.prompt.messages[1]"
      ],
      "metadata": {
        "id": "vHeZW75itky1",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "06bf3de5-37dd-48a6-8f72-d72118b31cfe"
      },
      "execution_count": 40,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "MessagesPlaceholder(variable_name='chat_history')"
            ]
          },
          "metadata": {},
          "execution_count": 40
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "We can see from `'chat_history'` (this must align to the `memory_key` from the `ConversationBufferWindowMemory` initialized earlier) that this is where the previous messages of the conversation will be fed into the LLM.\n",
        "\n",
        "The format of this input is set by the type of conversational memory being used, which in this case is the `ConversationBufferWindowMemory`.\n",
        "\n",
        "Finally, we have the `HumanMessagePromptTemplate`:"
      ],
      "metadata": {
        "id": "89ktagFTSkvv"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "conversational_agent.agent.llm_chain.prompt.messages[2]"
      ],
      "metadata": {
        "id": "fcIVdeklTZwh",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "7fd99108-60f8-45b1-a41a-926084859286"
      },
      "execution_count": 41,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "HumanMessagePromptTemplate(prompt=PromptTemplate(input_variables=['input'], output_parser=None, partial_variables={}, template='TOOLS\\n------\\nAssistant can ask the user to use tools to look up information that may be helpful in answering the users original question. The tools the human can use are:\\n\\n> Lex Fridman DB: Use this tool to answer user questions using Lex\\nFridman podcasts. If the user states \\'ask Lex\\' use this tool to get\\nthe answer. This tool can also be used for follow up questions from\\nthe user.\\n\\nRESPONSE FORMAT INSTRUCTIONS\\n----------------------------\\n\\nWhen responding to me please, please output a response in one of two formats:\\n\\n**Option 1:**\\nUse this if you want the human to use a tool.\\nMarkdown code snippet formatted in the following schema:\\n\\n```json\\n{{\\n    \"action\": string \\\\ The action to take. Must be one of Lex Fridman DB\\n    \"action_input\": string \\\\ The input to the action\\n}}\\n```\\n\\n**Option #2:**\\nUse this if you want to respond directly to the human. Markdown code snippet formatted in the following schema:\\n\\n```json\\n{{\\n    \"action\": \"Final Answer\",\\n    \"action_input\": string \\\\ You should put what you want to return to use here\\n}}\\n```\\n\\nUSER\\'S INPUT\\n--------------------\\nHere is the user\\'s input (remember to respond with a markdown code snippet of a json blob with a single action, and NOTHING else):\\n\\n{input}', template_format='f-string', validate_template=True), additional_kwargs={})"
            ]
          },
          "metadata": {},
          "execution_count": 41
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "print(\n",
        "    conversational_agent.agent.llm_chain.prompt.messages[2].prompt.template\n",
        ")"
      ],
      "metadata": {
        "id": "NRuqI-sCTZsb",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "4abc74bd-1989-44a4-f3ce-9004dde8ca8f"
      },
      "execution_count": 42,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "TOOLS\n",
            "------\n",
            "Assistant can ask the user to use tools to look up information that may be helpful in answering the users original question. The tools the human can use are:\n",
            "\n",
            "> Lex Fridman DB: Use this tool to answer user questions using Lex\n",
            "Fridman podcasts. If the user states 'ask Lex' use this tool to get\n",
            "the answer. This tool can also be used for follow up questions from\n",
            "the user.\n",
            "\n",
            "RESPONSE FORMAT INSTRUCTIONS\n",
            "----------------------------\n",
            "\n",
            "When responding to me please, please output a response in one of two formats:\n",
            "\n",
            "**Option 1:**\n",
            "Use this if you want the human to use a tool.\n",
            "Markdown code snippet formatted in the following schema:\n",
            "\n",
            "```json\n",
            "{{\n",
            "    \"action\": string \\ The action to take. Must be one of Lex Fridman DB\n",
            "    \"action_input\": string \\ The input to the action\n",
            "}}\n",
            "```\n",
            "\n",
            "**Option #2:**\n",
            "Use this if you want to respond directly to the human. Markdown code snippet formatted in the following schema:\n",
            "\n",
            "```json\n",
            "{{\n",
            "    \"action\": \"Final Answer\",\n",
            "    \"action_input\": string \\ You should put what you want to return to use here\n",
            "}}\n",
            "```\n",
            "\n",
            "USER'S INPUT\n",
            "--------------------\n",
            "Here is the user's input (remember to respond with a markdown code snippet of a json blob with a single action, and NOTHING else):\n",
            "\n",
            "{input}\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "This is the most interesting component. First, we have a single `input` — the user's query/prompt. But before this we see a lot of text, the majority of this is the setup for the LLM to be able to use any tools that we've passed to the conversational agent.\n",
        "\n",
        "In our case, there is just one tool, the `Lex Fridman DB` tool that we defined earlier. We can also see the tool description that we defined. The LLM will use this tool description to figure out which tool (if any) it should use."
      ],
      "metadata": {
        "id": "A3NhHdrMTia2"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Having a Conversation"
      ],
      "metadata": {
        "id": "Yo3nDYEWUQLZ"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "Let's begin our conversation. We'll start as any typical conversation begins:"
      ],
      "metadata": {
        "id": "VyiFqLg5UPyM"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "conversational_agent(\"What's up Lex?\")"
      ],
      "metadata": {
        "id": "SwjREzM8nJsx",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "fea67890-23ce-4f6e-db49-34ea9f0a940b"
      },
      "execution_count": 43,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "\n",
            "\n",
            "\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
            "\u001b[32;1m\u001b[1;3m{\n",
            "    \"action\": \"Final Answer\",\n",
            "    \"action_input\": \"I'm just a chatbot, so I don't have feelings, but I'm here to help you with any questions you have!\"\n",
            "}\u001b[0m\n",
            "\n",
            "\u001b[1m> Finished chain.\u001b[0m\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "{'input': 'hi how are you',\n",
              " 'chat_history': [],\n",
              " 'output': \"I'm just a chatbot, so I don't have feelings, but I'm here to help you with any questions you have!\"}"
            ]
          },
          "metadata": {},
          "execution_count": 43
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Looks good. We should note that there is this **AgentExecutor chain** thing. Where we can see an `\"action\"` and an `\"action_input\"`. It is here where the agent is deciding whether it should use a tool.\n",
        "\n",
        "Here we see the agent decides on `\"action\": \"Final Answer\"`, meaning no tool is required. Therefore, it just uses the LLM as per usual to generate an answer. That answer can be seen in `\"I'm just a chatbot, I don't have feelings, but thanks for asking! How can I assist you today?\"`.\n",
        "\n",
        "What if we mention the words `\"ask lex\"`?"
      ],
      "metadata": {
        "id": "sGM7i372Ua7X"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "conversational_agent(\"Are you scared of the future of AI?\")"
      ],
      "metadata": {
        "id": "bkgKG1x3ndBJ",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "9e9190ea-0f29-49ef-809c-74b64869f13c"
      },
      "execution_count": 45,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "\n",
            "\n",
            "\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
            "\u001b[32;1m\u001b[1;3mHere's your response:\n",
            "\n",
            "```json\n",
            "{\n",
            "    \"action\": \"Lex Fridman DB\",\n",
            "    \"action_input\": \"What did Lex Fridman say about the future of AI?\"\n",
            "}\n",
            "```\u001b[0m\n",
            "Observation: \u001b[36;1m\u001b[1;3mLex Fridman has expressed concerns about the potential risks of AI and its impact on human civilization. He believes that the algorithms that drive our interaction on social media already have an intelligence and power that far outstrip the intelligence and power of any one human being. He also believes that the future of human civilization may be at stake over the question of the role of artificial intelligence in our society.\u001b[0m\n",
            "Thought:\u001b[32;1m\u001b[1;3mHere's your response:\n",
            "\n",
            "```json\n",
            "{\n",
            "    \"action\": \"Final Answer\",\n",
            "    \"action_input\": \"Based on the insights from a certain AI expert, there are concerns about the potential risks of AI and its impact on human civilization. The expert believes that the future of human civilization may be at stake over the question of the role of artificial intelligence in our society.\"\n",
            "}\n",
            "```\u001b[0m\n",
            "\n",
            "\u001b[1m> Finished chain.\u001b[0m\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "{'input': 'ask lex about the future of ai',\n",
              " 'chat_history': [HumanMessage(content='hi how are you', additional_kwargs={}),\n",
              "  AIMessage(content=\"I'm just a chatbot, so I don't have feelings, but I'm here to help you with any questions you have!\", additional_kwargs={})],\n",
              " 'output': 'Based on the insights from a certain AI expert, there are concerns about the potential risks of AI and its impact on human civilization. The expert believes that the future of human civilization may be at stake over the question of the role of artificial intelligence in our society.'}"
            ]
          },
          "metadata": {},
          "execution_count": 45
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Great, we can see that the first thing the agent did was default to the `\"Lex Fridman DB\"` tool. The input to that tool was generated by the LLM, and is `\"What did Lex Fridman say about the future of AI?\"`.\n",
        "\n",
        "This input is then passed into the `Lex Fridman DB` tool and the output observation of the LLM (after it has read all of the information returned by our vector DB is returned to our agent. From this observation the agent moves on to the `\"Final Answer\"` action, giving us the output."
      ],
      "metadata": {
        "id": "-dcGpHZSXs2Z"
      }
    }
  ]
}