notebooks/llm-agent-functioncall/llm-agent-functioncall-qwen.ipynb

{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "b3c04f1a",
   "metadata": {},
   "source": [
    "# Create Function-calling Agent using OpenVINO and Qwen-Agent\n",
    "\n",
    "LLM are limited to the knowledge on which they have been trained and the additional knowledge provided as context, as a result, if a useful piece of information is missing the provided knowledge, the model cannot “go around” and try to find it in other sources. This is the reason why we need to introduce the concept of Agents.\n",
    "\n",
    "The core idea of agents is to use a language model to choose a sequence of actions to take. In agents, a language model is used as a reasoning engine to determine which actions to take and in which order. Agents can be seen as applications powered by LLMs and integrated with a set of tools like search engines, databases, websites, and so on. Within an agent, the LLM is the reasoning engine that, based on the user input, is able to plan and execute a set of actions that are needed to fulfill the request.\n",
    "\n",
    "![agent](https://github.com/openvinotoolkit/openvino_notebooks/assets/91237924/22fa5396-8381-400f-a78f-97e25d57d807)\n",
    "\n",
    "[Qwen-Agent](https://github.com/QwenLM/Qwen-Agent) is a framework for developing LLM applications based on the instruction following, tool usage, planning, and memory capabilities of Qwen. It also comes with example applications such as Browser Assistant, Code Interpreter, and Custom Assistant.\n",
    "\n",
    "This notebook explores how to create a Function calling Agent step by step using OpenVINO and Qwen-Agent.\n",
    "\n",
    "#### Table of contents:\n",
    "\n",
    "- [Prerequisites](#Prerequisites)\n",
    "- [Create a Function calling agent](#Create-a-Function-calling-agent)\n",
    "  - [Create functions](#Create-functions)\n",
    "  - [Download model](#Download-model)\n",
    "  - [Select inference device for LLM](#Select-inference-device-for-LLM)\n",
    "  - [Create LLM for Qwen-Agent](#Create-LLM-for-Qwen-Agent)\n",
    "  - [Create Function-calling pipeline](#Create-Function-calling-pipeline)\n",
    "- [Interactive Demo](#Interactive-Demo)\n",
    "  - [Create tools](#Create-tools)\n",
    "  - [Create AI agent demo with Qwen-Agent and Gradio UI](#Create-AI-agent-demo-with-Qwen-Agent-and-Gradio-UI)\n",
    "\n",
    "### Installation Instructions\n",
    "\n",
    "This is a self-contained example that relies solely on its own code.\n",
    "\n",
    "We recommend  running the notebook in a virtual environment. You only need a Jupyter server to start.\n",
    "For details, please refer to [Installation Guide](https://github.com/openvinotoolkit/openvino_notebooks/blob/latest/README.md#-installation-guide).\n",
    "\n",
    "<img referrerpolicy=\"no-referrer-when-downgrade\" src=\"https://static.scarf.sh/a.png?x-pxid=5b5a4db0-7875-4bfb-bdbd-01698b5b1a77&file=notebooks/llm-agent-functioncall/llm-agent-functioncall-qwen.ipynb\" />\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "f7bb0a67",
   "metadata": {},
   "source": [
    "## Prerequisites\n",
    "\n",
    "[back to top ⬆️](#Table-of-contents:)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "47d43de7-9946-482d-84cb-222294c1cda8",
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "from pathlib import Path\n",
    "import requests\n",
    "\n",
    "os.environ[\"GIT_CLONE_PROTECTION_ACTIVE\"] = \"false\"\n",
    "\n",
    "%pip install -Uq pip\n",
    "%pip uninstall -q -y optimum optimum-intel\n",
    "%pip install --pre -Uq \"openvino>=2024.2.0\" openvino-tokenizers[transformers] --extra-index-url https://storage.openvinotoolkit.org/simple/wheels/nightly\n",
    "%pip install -q --extra-index-url https://download.pytorch.org/whl/cpu \\\n",
    "\"torch>=2.1\" \\\n",
    "\"datasets\" \\\n",
    "\"accelerate\" \\\n",
    "\"qwen-agent[gui]==0.0.7\" \"transformers>=4.38.1\" \"gradio==4.21.0\", \"modelscope-studio>=0.4.0,<1.0.0\" \"langchain>=0.2.3\" \"langchain-community>=0.2.4\" \"wikipedia\"\n",
    "%pip install -q --extra-index-url https://download.pytorch.org/whl/cpu \\\n",
    "\"git+https://github.com/huggingface/optimum-intel.git\" \\\n",
    "\"git+https://github.com/openvinotoolkit/nncf.git\"\n",
    "\n",
    "utility_files = [\"notebook_utils.py\", \"cmd_helper.py\"]\n",
    "\n",
    "for utility in utility_files:\n",
    "    local_path = Path(utility)\n",
    "    if not local_path.exists():\n",
    "        r = requests.get(\n",
    "            url=f\"https://raw.githubusercontent.com/openvinotoolkit/openvino_notebooks/latest/utils/{local_path.name}\",\n",
    "        )\n",
    "        with local_path.open(\"w\") as f:\n",
    "            f.write(r.text)\n",
    "\n",
    "# Read more about telemetry collection at https://github.com/openvinotoolkit/openvino_notebooks?tab=readme-ov-file#-telemetry\n",
    "from notebook_utils import collect_telemetry\n",
    "\n",
    "collect_telemetry(\"llm-agent-functioncall-qwen.ipynb\")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "625a2e19-56be-478c-87cf-4d906b08a20f",
   "metadata": {},
   "source": [
    "## Create a Function calling agent\n",
    "\n",
    "[back to top ⬆️](#Table-of-contents:)\n",
    "\n",
    "Function calling allows a model to detect when one or more tools should be called and respond with the inputs that should be passed to those tools. In an API call, you can describe tools and have the model intelligently choose to output a structured object like JSON containing arguments to call these tools. The goal of tools APIs is to more reliably return valid and useful tool calls than what can be done using a generic text completion or chat API.\n",
    "\n",
    "We can take advantage of this structured output, combined with the fact that you can bind multiple tools to a tool calling chat model and allow the model to choose which one to call, to create an agent that repeatedly calls tools and receives results until a query is resolved."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "c722d565",
   "metadata": {},
   "source": [
    "### Create a function\n",
    "\n",
    "[back to top ⬆️](#Table-of-contents:)\n",
    "\n",
    "First, we need to create a example function/tool for getting the information of current weather."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "e8bfe609-1823-4df7-9a68-f210a58a0d38",
   "metadata": {},
   "outputs": [],
   "source": [
    "import json\n",
    "\n",
    "\n",
    "def get_current_weather(location, unit=\"fahrenheit\"):\n",
    "    \"\"\"Get the current weather in a given location\"\"\"\n",
    "    if \"tokyo\" in location.lower():\n",
    "        return json.dumps({\"location\": \"Tokyo\", \"temperature\": \"10\", \"unit\": \"celsius\"})\n",
    "    elif \"san francisco\" in location.lower():\n",
    "        return json.dumps({\"location\": \"San Francisco\", \"temperature\": \"72\", \"unit\": \"fahrenheit\"})\n",
    "    elif \"paris\" in location.lower():\n",
    "        return json.dumps({\"location\": \"Paris\", \"temperature\": \"22\", \"unit\": \"celsius\"})\n",
    "    else:\n",
    "        return json.dumps({\"location\": location, \"temperature\": \"unknown\"})"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "31041310-cc81-4989-b8c1-c83face90b2c",
   "metadata": {},
   "source": [
    "Wrap the function's name and description into a json list, and it will help LLM to find out which function should be called for current task."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "5ea4ce13",
   "metadata": {},
   "outputs": [],
   "source": [
    "functions = [\n",
    "    {\n",
    "        \"name\": \"get_current_weather\",\n",
    "        \"description\": \"Get the current weather in a given location\",\n",
    "        \"parameters\": {\n",
    "            \"type\": \"object\",\n",
    "            \"properties\": {\n",
    "                \"location\": {\n",
    "                    \"type\": \"string\",\n",
    "                    \"description\": \"The city and state, e.g. San Francisco, CA\",\n",
    "                },\n",
    "                \"unit\": {\"type\": \"string\", \"enum\": [\"celsius\", \"fahrenheit\"]},\n",
    "            },\n",
    "            \"required\": [\"location\"],\n",
    "        },\n",
    "    }\n",
    "]"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "259e1f0d",
   "metadata": {},
   "source": [
    "### Download model\n",
    "\n",
    "[back to top ⬆️](#Table-of-contents:)\n",
    "\n",
    "Large Language Models (LLMs) are a core component of Agent. In this example, we will demonstrate how to create a OpenVINO LLM model in Qwen-Agent framework. Since Qwen2 can support function calling during text generation, we select `Qwen/Qwen2-7B-Instruct` as LLM in agent pipeline.\n",
    "\n",
    "* **Qwen/Qwen2-7B-Instruct** - Qwen2 is the new series of Qwen large language models. Compared with the state-of-the-art open source language models, including the previous released Qwen1.5, Qwen2 has generally surpassed most open source models and demonstrated competitiveness against proprietary models across a series of benchmarks targeting for language understanding, language generation, multilingual capability, coding, mathematics, reasoning, etc. [Model Card](https://huggingface.co/Qwen/Qwen2-7B-Instruct)\n",
    "\n",
    "To run LLM locally, we have to download the model in the first step. It is possible to [export your model](https://github.com/huggingface/optimum-intel?tab=readme-ov-file#export) to the OpenVINO IR format with the CLI, and load the model from local folder.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "981df8fe-cfcf-455a-919e-dda36f3b5dfb",
   "metadata": {
    "test_replace": {
     "Qwen/Qwen2-7B-Instruct": "Qwen/Qwen2-0.5B-Instruct"
    }
   },
   "outputs": [],
   "source": [
    "from pathlib import Path\n",
    "from cmd_helper import optimum_cli\n",
    "\n",
    "model_id = \"Qwen/Qwen2-7B-Instruct\"\n",
    "model_path = \"Qwen2-7B-Instruct-ov\"\n",
    "\n",
    "if not Path(model_path).exists():\n",
    "    optimum_cli(model_id, model_path, additional_args={\"task\": \"text-generation-with-past\", \"trust-remote-code\": \"\", \"weight-format\": \"int4\", \"ratio\": \"0.72\"})"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "6cfdbbae",
   "metadata": {},
   "source": [
    "### Select inference device for LLM\n",
    "\n",
    "[back to top ⬆️](#Table-of-contents:)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "a1ea3bdb-f97c-4374-880a-2b62abb5baaa",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "f981d582698c4400ac8997e9108c5558",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Dropdown(description='Device:', options=('CPU', 'AUTO'), value='CPU')"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from notebook_utils import device_widget\n",
    "\n",
    "device = device_widget(\"CPU\", [\"NPU\"])"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "77244c52",
   "metadata": {},
   "source": [
    "### Create LLM for Qwen-Agent\n",
    "\n",
    "[back to top ⬆️](#Table-of-contents:)\n",
    "\n",
    "OpenVINO has been integrated into the `Qwen-Agent` framework. You can use following method to create a OpenVINO based LLM for a `Qwen-Agent` pipeline."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "abfaab28-fd5b-46cd-88ad-b60ea5a3cdd6",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Compiling the model to CPU ...\n",
      "Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.\n"
     ]
    }
   ],
   "source": [
    "from qwen_agent.llm import get_chat_model\n",
    "\n",
    "import openvino.properties as props\n",
    "import openvino.properties.hint as hints\n",
    "import openvino.properties.streams as streams\n",
    "\n",
    "\n",
    "ov_config = {hints.performance_mode(): hints.PerformanceMode.LATENCY, streams.num(): \"1\", props.cache_dir(): \"\"}\n",
    "llm_cfg = {\n",
    "    \"ov_model_dir\": model_path,\n",
    "    \"model_type\": \"openvino\",\n",
    "    \"device\": device.value,\n",
    "    \"ov_config\": ov_config,\n",
    "    # (Optional) LLM hyperparameters for generation:\n",
    "    \"generate_cfg\": {\"top_p\": 0.8},\n",
    "}\n",
    "llm = get_chat_model(llm_cfg)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "d70905e2",
   "metadata": {},
   "source": [
    "You can get additional inference speed improvement with [Dynamic Quantization of activations and KV-cache quantization on CPU](https://docs.openvino.ai/2024/learn-openvino/llm_inference_guide/llm-inference-hf.html#enabling-openvino-runtime-optimizations). These options can be enabled with `ov_config` as follows:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "ee406254",
   "metadata": {},
   "outputs": [],
   "source": [
    "ov_config = {\n",
    "    \"KV_CACHE_PRECISION\": \"u8\",\n",
    "    \"DYNAMIC_QUANTIZATION_GROUP_SIZE\": \"32\",\n",
    "    hints.performance_mode(): hints.PerformanceMode.LATENCY,\n",
    "    streams.num(): \"\",\n",
    "    props.cache_dir(): \"\",\n",
    "}"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "52a9a190",
   "metadata": {},
   "source": [
    "## Create Function-calling pipeline\n",
    "\n",
    "[back to top ⬆️](#Table-of-contents:)\n",
    "\n",
    "After defining the functions and LLM, we can build the agent pipeline with capability of function calling.\n",
    "\n",
    "![functioncalling](https://github.com/openvinotoolkit/openvino_notebooks/assets/91237924/3170ca30-23af-4a1a-a655-1d0d67df2ded)\n",
    "\n",
    "The workflow of Qwen2 function calling consists of several steps:\n",
    "\n",
    "1. Role `user` sending the request.\n",
    "2. Check if the model wanted to call a function, and call the function if needed\n",
    "3. Get the observation from `function`'s results.\n",
    "4. Consolidate the observation into final response of `assistant`.\n",
    "\n",
    "A typical multi-turn dialogue structure is as follows:\n",
    "\n",
    "- **Query**: \n",
    "```{'role': 'user', 'content': 'create a picture of cute cat'},```\n",
    "\n",
    "- **Function calling**: \n",
    "```{'role': 'assistant', 'content': '', 'function_call': {'name': 'my_image_gen', 'arguments': '{\"prompt\": \"a cute cat\"}'}},```\n",
    "\n",
    "- **Observation**: \n",
    "```{'role': 'function', 'content': '{\"image_url\": \"https://image.pollinations.ai/prompt/a%20cute%20cat\"}', 'name': 'my_image_gen'}```\n",
    "\n",
    "- **Final Response**: \n",
    "```{'role': 'assistant', 'content': \"Here is the image of a cute cat based on your description:\\n\\n![](https://image.pollinations.ai/prompt/a%20cute%20cat).\"}```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "4799540b-eee0-491f-a5b6-5bae68c22af9",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "# User question:\n",
      "[{'role': 'user', 'content': \"What's the weather like in San Francisco?\"}]\n",
      "# Assistant Response 1:\n",
      "[{'role': 'assistant', 'content': '', 'function_call': {'name': 'get_current_weather', 'arguments': '{\"location\": \"San Francisco, CA\"}'}}]\n",
      "# Function Response:\n",
      "{\"location\": \"San Francisco\", \"temperature\": \"72\", \"unit\": \"fahrenheit\"}\n",
      "# Assistant Response 2:\n",
      "[{'role': 'assistant', 'content': 'The current weather in San Francisco is 72 degrees Fahrenheit.'}]\n"
     ]
    }
   ],
   "source": [
    "print(\"# User question:\")\n",
    "messages = [{\"role\": \"user\", \"content\": \"What's the weather like in San Francisco?\"}]\n",
    "print(messages)\n",
    "\n",
    "print(\"# Assistant Response 1:\")\n",
    "responses = []\n",
    "\n",
    "# Step 1: Role `user` sending the request\n",
    "responses = llm.chat(\n",
    "    messages=messages,\n",
    "    functions=functions,\n",
    "    stream=False,\n",
    ")\n",
    "print(responses)\n",
    "\n",
    "messages.extend(responses)\n",
    "\n",
    "# Step 2: check if the model wanted to call a function, and call the function if needed\n",
    "last_response = messages[-1]\n",
    "if last_response.get(\"function_call\", None):\n",
    "    available_functions = {\n",
    "        \"get_current_weather\": get_current_weather,\n",
    "    }  # only one function in this example, but you can have multiple\n",
    "    function_name = last_response[\"function_call\"][\"name\"]\n",
    "    function_to_call = available_functions[function_name]\n",
    "    function_args = json.loads(last_response[\"function_call\"][\"arguments\"])\n",
    "    function_response = function_to_call(\n",
    "        location=function_args.get(\"location\"),\n",
    "    )\n",
    "    print(\"# Function Response:\")\n",
    "    print(function_response)\n",
    "\n",
    "    # Step 3: Get the observation from `function`'s results\n",
    "    messages.append(\n",
    "        {\n",
    "            \"role\": \"function\",\n",
    "            \"name\": function_name,\n",
    "            \"content\": function_response,\n",
    "        }\n",
    "    )\n",
    "\n",
    "    print(\"# Assistant Response 2:\")\n",
    "    # Step 4: Consolidate the observation from function into final response\n",
    "    responses = llm.chat(\n",
    "        messages=messages,\n",
    "        functions=functions,\n",
    "        stream=False,\n",
    "    )\n",
    "    print(responses)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "688ced57",
   "metadata": {},
   "source": [
    "## Interactive Demo\n",
    "\n",
    "[back to top ⬆️](#Table-of-contents:)\n",
    "\n",
    "Let's create a interactive agent using [Gradio](https://www.gradio.app/).\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "ce623516",
   "metadata": {},
   "source": [
    "### Create tools\n",
    "\n",
    "[back to top ⬆️](#Table-of-contents:)\n",
    "\n",
    "Qwen-Agent provides a mechanism for [registering tools](https://github.com/QwenLM/Qwen-Agent/blob/main/docs/tool.md). For example, to register your own image generation tool:\n",
    "\n",
    "- Specify the tool’s name, description, and parameters. Note that the string passed to `@register_tool('my_image_gen')` is automatically added as the `.name` attribute of the class and will serve as the unique identifier for the tool.\n",
    "- Implement the `call(...)` function.\n",
    "\n",
    "In this notebook, we will create 3 tools as examples:\n",
    "- **image_generation**: AI painting (image generation) service, input text description, and return the image URL drawn based on text information.\n",
    "- **get_current_weather**: Get the current weather in a given city name.\n",
    "- **wikipedia**: A wrapper around Wikipedia. Useful for when you need to answer general questions about people, places, companies, facts, historical events, or other subjects."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "1f64b67c-1259-4fe6-bfc3-af317bfe04f6",
   "metadata": {},
   "outputs": [],
   "source": [
    "import urllib.parse\n",
    "import json5\n",
    "import requests\n",
    "from qwen_agent.tools.base import BaseTool, register_tool\n",
    "\n",
    "\n",
    "@register_tool(\"image_generation\")\n",
    "class ImageGeneration(BaseTool):\n",
    "    description = \"AI painting (image generation) service, input text description, and return the image URL drawn based on text information.\"\n",
    "    parameters = [{\"name\": \"prompt\", \"type\": \"string\", \"description\": \"Detailed description of the desired image content, in English\", \"required\": True}]\n",
    "\n",
    "    def call(self, params: str, **kwargs) -> str:\n",
    "        prompt = json5.loads(params)[\"prompt\"]\n",
    "        prompt = urllib.parse.quote(prompt)\n",
    "        return json5.dumps({\"image_url\": f\"https://image.pollinations.ai/prompt/{prompt}\"}, ensure_ascii=False)\n",
    "\n",
    "\n",
    "@register_tool(\"get_current_weather\")\n",
    "class GetCurrentWeather(BaseTool):\n",
    "    description = \"Get the current weather in a given city name.\"\n",
    "    parameters = [{\"name\": \"city_name\", \"type\": \"string\", \"description\": \"The city and state, e.g. San Francisco, CA\", \"required\": True}]\n",
    "\n",
    "    def call(self, params: str, **kwargs) -> str:\n",
    "        # `params` are the arguments generated by the LLM agent.\n",
    "        city_name = json5.loads(params)[\"city_name\"]\n",
    "        key_selection = {\n",
    "            \"current_condition\": [\n",
    "                \"temp_C\",\n",
    "                \"FeelsLikeC\",\n",
    "                \"humidity\",\n",
    "                \"weatherDesc\",\n",
    "                \"observation_time\",\n",
    "            ],\n",
    "        }\n",
    "        resp = requests.get(f\"https://wttr.in/{city_name}?format=j1\")\n",
    "        resp.raise_for_status()\n",
    "        resp = resp.json()\n",
    "        ret = {k: {_v: resp[k][0][_v] for _v in v} for k, v in key_selection.items()}\n",
    "        return str(ret)\n",
    "\n",
    "\n",
    "@register_tool(\"wikipedia\")\n",
    "class Wikipedia(BaseTool):\n",
    "    description = \"A wrapper around Wikipedia. Useful for when you need to answer general questions about people, places, companies, facts, historical events, or other subjects.\"\n",
    "    parameters = [{\"name\": \"query\", \"type\": \"string\", \"description\": \"Query to look up on wikipedia\", \"required\": True}]\n",
    "\n",
    "    def call(self, params: str, **kwargs) -> str:\n",
    "        # `params` are the arguments generated by the LLM agent.\n",
    "        from langchain.tools import WikipediaQueryRun\n",
    "        from langchain_community.utilities import WikipediaAPIWrapper\n",
    "\n",
    "        query = json5.loads(params)[\"query\"]\n",
    "        wikipedia = WikipediaQueryRun(api_wrapper=WikipediaAPIWrapper(top_k_results=2, doc_content_chars_max=1000))\n",
    "        resutlt = wikipedia.run(query)\n",
    "        return str(resutlt)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "ea649d2f-ab4b-4228-8f32-75b01a98d9a7",
   "metadata": {},
   "outputs": [],
   "source": [
    "tools = [\"image_generation\", \"get_current_weather\", \"wikipedia\"]"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "148b7c9d-2b8b-45fa-9fc1-c37ebdc1e5fb",
   "metadata": {},
   "source": [
    "### Create AI agent demo with Qwen-Agent and Gradio UI\n",
    "\n",
    "[back to top ⬆️](#Table-of-contents:)\n",
    "\n",
    "The Agent class serves as a higher-level interface for Qwen-Agent, where an Agent object integrates the interfaces for tool calls and LLM (Large Language Model). The Agent receives a list of messages as input and produces a generator that yields a list of messages, effectively providing a stream of output messages.\n",
    "\n",
    "Qwen-Agent offers a generic Agent class: the `Assistant` class, which, when directly instantiated, can handle the majority of Single-Agent tasks. Features:\n",
    "\n",
    "- It supports role-playing.\n",
    "- It provides automatic planning and tool calls abilities.\n",
    "- RAG (Retrieval-Augmented Generation): It accepts documents input, and can use an integrated RAG strategy to parse the documents."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "52aed08c-b515-40b5-b371-16e27fb89e60",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Compiling the model to CPU ...\n",
      "Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.\n"
     ]
    }
   ],
   "source": [
    "from qwen_agent.agents import Assistant\n",
    "\n",
    "bot = Assistant(llm=llm_cfg, function_list=tools, name=\"OpenVINO Agent\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "752e052d-74d7-4df7-81eb-bbf317196ab1",
   "metadata": {},
   "outputs": [],
   "source": [
    "if not Path(\"gradio_helper.py\").exists():\n",
    "    r = requests.get(url=\"https://raw.githubusercontent.com/openvinotoolkit/openvino_notebooks/latest/notebooks/llm-agent-functioncall/gradio_helper.py\")\n",
    "    open(\"gradio_helper.py\", \"w\").write(r.text)\n",
    "\n",
    "from gradio_helper import make_demo\n",
    "\n",
    "demo = make_demo(bot=bot)\n",
    "\n",
    "try:\n",
    "    demo.run()\n",
    "except Exception:\n",
    "    demo.run(share=True)\n",
    "# If you are launching remotely, specify server_name and server_port\n",
    "# EXAMPLE: `demo.launch(server_name='your server name', server_port='server port in int')`\n",
    "# To learn more please refer to the Gradio docs: https://gradio.app/docs/"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a9f9bbfb-a2e9-469a-892b-fc6617d08eff",
   "metadata": {},
   "outputs": [],
   "source": [
    "# please uncomment and run this cell for stopping gradio interface\n",
    "# demo.close()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.4"
  },
  "openvino_notebooks": {
   "imageUrl": "https://github.com/openvinotoolkit/openvino_notebooks/assets/91237924/202cddac-dbbb-493b-ae79-0d45798f75c1",
   "tags": {
    "categories": [
     "Model Demos",
     "AI Trends"
    ],
    "libraries": [],
    "other": [
     "LLM"
    ],
    "tasks": [
     "Text Generation"
    ]
   }
  },
  "widgets": {
   "application/vnd.jupyter.widget-state+json": {
    "state": {},
    "version_major": 2,
    "version_minor": 0
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}