LLM_001_role_system_user_prompts.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/micah-shull/LLMs/blob/main/LLM_001_role_system_user_prompts.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## **\"role,\" \"system,\" and \"user\"** prompts in the API code\n",
        "\n",
        "In LLM APIs, like OpenAI's and similar services, these roles are crucial because they help guide the conversation flow and context for the language model. Let me explain each role and its importance:\n",
        "\n",
        "1. **\"system\" role**:\n",
        "   - This sets the context for the conversation. The system message is used to provide instructions on how the assistant should behave throughout the conversation.\n",
        "   - It's like setting up the \"personality\" or purpose of the model.\n",
        "   - For example, `\"role\": \"system\", \"content\": \"You are a helpful assistant that answers questions in a friendly and concise manner.\"` would make the LLM take on that friendly assistant persona. This influences all the subsequent messages and responses.\n",
        "\n",
        "2. **\"user\" role**:\n",
        "   - This represents the input from the user interacting with the LLM.\n",
        "   - The content here is essentially the user's question, request, or input that the model will respond to.\n",
        "   - For example, `\"role\": \"user\", \"content\": \"What is the capital of France?\"` tells the model what the user wants to know. The model processes this input to generate a response.\n",
        "\n",
        "3. **\"assistant\" role**:\n",
        "   - This is where the LLM's response is specified.\n",
        "   - It is used to maintain context and keep track of what has already been said in the conversation.\n",
        "   - For instance, `\"role\": \"assistant\", \"content\": \"The capital of France is Paris.\"` provides the model's answer.\n",
        "\n",
        "The **importance** of these roles lies in:\n",
        "\n",
        "- **Structuring Conversations**: The user and assistant roles help the API differentiate between who is asking and who is responding, maintaining a coherent flow of information.\n",
        "- **Maintaining State and Continuity**: These roles are also used to maintain the conversational history, allowing the LLM to generate context-aware responses as the conversation progresses.\n",
        "\n",
        "Overall, these roles form the foundational structure for interactive, contextual communication between users and the language model. By appropriately structuring these roles, developers can make the LLM perform tasks more efficiently and maintain an interactive, context-rich conversation.\n",
        "\n",
        "### Example 1: FAQ Chatbot for Customer Support\n",
        "- **System Prompt**:\n",
        "  ```json\n",
        "  {\"role\": \"system\", \"content\": \"You are a helpful customer support assistant for an e-commerce store. You answer questions about orders, refunds, and products.\"}\n",
        "  ```\n",
        "- **User Prompt**:\n",
        "  ```json\n",
        "  {\"role\": \"user\", \"content\": \"How can I return a product?\"}\n",
        "  ```\n",
        "- **Assistant Response**:\n",
        "  ```json\n",
        "  {\"role\": \"assistant\", \"content\": \"You can return a product within 30 days of purchase. Please visit our returns page and fill out the return request form. You'll get a prepaid label to ship the product back.\"}\n",
        "  ```\n",
        "\n",
        "### Example 2: Friendly Conversational Assistant\n",
        "- **System Prompt**:\n",
        "  ```json\n",
        "  {\"role\": \"system\", \"content\": \"You are a friendly assistant who loves to chat about movies and recommend great films.\"}\n",
        "  ```\n",
        "- **User Prompt**:\n",
        "  ```json\n",
        "  {\"role\": \"user\", \"content\": \"Can you recommend a good science fiction movie?\"}\n",
        "  ```\n",
        "- **Assistant Response**:\n",
        "  ```json\n",
        "  {\"role\": \"assistant\", \"content\": \"Sure! If you're into science fiction, I recommend watching 'Interstellar' by Christopher Nolan. It's an epic story about space exploration and the survival of humanity, with stunning visuals and a great storyline.\"}\n",
        "  ```\n",
        "\n",
        "### Example 3: Technical Assistant for Coding Help\n",
        "- **System Prompt**:\n",
        "  ```json\n",
        "  {\"role\": \"system\", \"content\": \"You are a technical assistant that provides programming guidance, particularly for Python.\"}\n",
        "  ```\n",
        "- **User Prompt**:\n",
        "  ```json\n",
        "  {\"role\": \"user\", \"content\": \"How do I reverse a list in Python?\"}\n",
        "  ```\n",
        "- **Assistant Response**:\n",
        "  ```json\n",
        "  {\"role\": \"assistant\", \"content\": \"You can reverse a list in Python using the `reverse()` method or by using slicing. For example:\\n\\n```\\nmy_list = [1, 2, 3, 4, 5]\\nmy_list.reverse()  # In-place reversal\\nprint(my_list)  # Output: [5, 4, 3, 2, 1]\\n\\n# Or using slicing:\\nreversed_list = my_list[::-1]\\nprint(reversed_list)  # Output: [5, 4, 3, 2, 1]\\n```\"}\n",
        "  ```\n",
        "\n",
        "### Example 4: Encouraging Study Buddy\n",
        "- **System Prompt**:\n",
        "  ```json\n",
        "  {\"role\": \"system\", \"content\": \"You are an encouraging study buddy who motivates the user to keep up with their studies.\"}\n",
        "  ```\n",
        "- **User Prompt**:\n",
        "  ```json\n",
        "  {\"role\": \"user\", \"content\": \"I'm feeling overwhelmed with my math homework.\"}\n",
        "  ```\n",
        "- **Assistant Response**:\n",
        "  ```json\n",
        "  {\"role\": \"assistant\", \"content\": \"I totally understand! Math can be challenging, but remember, you’re capable of doing hard things. Take it one problem at a time, and don’t be afraid to ask for help if you get stuck. You've got this!\"}\n",
        "  ```\n",
        "\n",
        "### Example 5: Travel Planner Assistant\n",
        "- **System Prompt**:\n",
        "  ```json\n",
        "  {\"role\": \"system\", \"content\": \"You are a travel planner assistant who helps users create itineraries for their trips.\"}\n",
        "  ```\n",
        "- **User Prompt**:\n",
        "  ```json\n",
        "  {\"role\": \"user\", \"content\": \"Can you help me plan a 3-day trip to Paris?\"}\n",
        "  ```\n",
        "- **Assistant Response**:\n",
        "  ```json\n",
        "  {\"role\": \"assistant\", \"content\": \"Absolutely! Here’s a suggested itinerary for a 3-day trip to Paris:\\n\\n**Day 1**: Visit the Eiffel Tower, stroll along the Seine River, and explore the Louvre Museum.\\n\\n**Day 2**: Start the day at Notre-Dame Cathedral, then head to Montmartre to see the Sacré-Cœur Basilica. In the evening, enjoy a dinner cruise on the Seine.\\n\\n**Day 3**: Spend the morning at the Palace of Versailles, and later, visit the Musée d'Orsay. Wrap up your trip by relaxing in the Luxembourg Gardens.\\n\\nLet me know if you'd like more details or any adjustments!\"}\n",
        "  ```\n",
        "\n",
        "### Example 6: Health & Wellness Assistant\n",
        "- **System Prompt**:\n",
        "  ```json\n",
        "  {\"role\": \"system\", \"content\": \"You are a knowledgeable health and wellness coach providing advice on fitness, diet, and mental health.\"}\n",
        "  ```\n",
        "- **User Prompt**:\n",
        "  ```json\n",
        "  {\"role\": \"user\", \"content\": \"What's a good way to manage stress?\"}\n",
        "  ```\n",
        "- **Assistant Response**:\n",
        "  ```json\n",
        "  {\"role\": \"assistant\", \"content\": \"Managing stress effectively can involve several strategies. Deep breathing exercises, meditation, and regular physical activity like yoga or jogging are great ways to reduce stress. Make sure to get enough sleep, and consider keeping a journal to express your thoughts and emotions. Also, staying connected with supportive friends and family can make a big difference.\"}\n",
        "  ```\n",
        "\n",
        "### Summary of Roles:\n",
        "- **System**: Sets the assistant's behavior and tone for the conversation.\n",
        "- **User**: Represents the queries or instructions given by the user.\n",
        "- **Assistant**: Contains the responses generated by the LLM, based on the system prompt and user inputs.\n",
        "\n",
        "These examples should help you see how you can use different system prompts to adjust the assistant’s behavior and make interactions tailored to specific tasks or purposes."
      ],
      "metadata": {
        "id": "jTjDmaGpk-_X"
      },
      "id": "jTjDmaGpk-_X"
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Install Libraries"
      ],
      "metadata": {
        "id": "NrJwnntWLwcj"
      },
      "id": "NrJwnntWLwcj"
    },
    {
      "cell_type": "code",
      "source": [
        "# !pip install python-dotenv\n",
        "# !pip install openai\n",
        "# !pip install google-generativeai\n",
        "# !pip install anthropic"
      ],
      "metadata": {
        "id": "VENi5mnfLmdA"
      },
      "id": "VENi5mnfLmdA",
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Import Libraries"
      ],
      "metadata": {
        "id": "ahuJXJzqL2CJ"
      },
      "id": "ahuJXJzqL2CJ"
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "de23bb9e-37c5-4377-9a82-d7b6c648eeb6",
      "metadata": {
        "id": "de23bb9e-37c5-4377-9a82-d7b6c648eeb6"
      },
      "outputs": [],
      "source": [
        "import os\n",
        "from dotenv import load_dotenv\n",
        "from openai import OpenAI\n",
        "import google.generativeai\n",
        "import anthropic\n",
        "from IPython.display import Markdown, display, update_display"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Load Environment Variables"
      ],
      "metadata": {
        "id": "-DBMss9vQ1Iy"
      },
      "id": "-DBMss9vQ1Iy"
    },
    {
      "cell_type": "code",
      "source": [
        "# Load the environment variables from the .env file\n",
        "load_dotenv('/content/API_KEYS.env')  # Ensure this is the correct path to your file\n",
        "\n",
        "# Get the API keys from the environment\n",
        "openai_api_key = os.getenv(\"OPENAI_API_KEY\")\n",
        "anthropic_api_key = os.getenv(\"ANTHROPIC_API_KEY\")\n",
        "google_api_key = os.getenv(\"GOOGLE_API_KEY\")\n",
        "\n",
        "# Check if the keys are loaded correctly and print a portion of them\n",
        "if openai_api_key:\n",
        "    print(f\"OpenAI API Key loaded: {openai_api_key[0:10]}...\")  # Only print part of the key\n",
        "else:\n",
        "    print(\"OpenAI API key not loaded correctly.\")\n",
        "\n",
        "if anthropic_api_key:\n",
        "    print(f\"Anthropic API Key loaded: {anthropic_api_key[0:10]}...\")\n",
        "else:\n",
        "    print(\"Anthropic API key not loaded correctly.\")\n",
        "\n",
        "if google_api_key:\n",
        "    print(f\"Google API Key loaded: {google_api_key[0:10]}...\")\n",
        "else:\n",
        "    print(\"Google API key not loaded correctly.\")"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "GHozblbHf8Ke",
        "outputId": "db8b3293-bab6-480a-ddd7-8707101a15fa"
      },
      "id": "GHozblbHf8Ke",
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "OpenAI API Key loaded: sk-proj-mf...\n",
            "Anthropic API Key loaded: sk-ant-api...\n",
            "Google API Key loaded: AIzaSyDh3a...\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Connect to OpenAI, Anthropic and Google"
      ],
      "metadata": {
        "id": "MyD28eFXPD7v"
      },
      "id": "MyD28eFXPD7v"
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "1179b4c5-cd1f-4131-a876-4c9f3f38d2ba",
      "metadata": {
        "id": "1179b4c5-cd1f-4131-a876-4c9f3f38d2ba"
      },
      "outputs": [],
      "source": [
        "import openai\n",
        "import anthropic\n",
        "import google.generativeai\n",
        "\n",
        "# Connect to OpenAI\n",
        "openai.api_key = openai_api_key  # Set OpenAI API key\n",
        "\n",
        "# Connect to Anthropic (Claude)\n",
        "claude = anthropic.Anthropic(api_key=anthropic_api_key)  # Set Anthropic API key\n",
        "\n",
        "# Connect to Google Generative AI\n",
        "google.generativeai.configure(api_key=google_api_key)  # Set Google API key"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Setting up a Connection Explained\n",
        "\n",
        "The code you provided initializes instances of different APIs — **OpenAI**, **Anthropic**, and **Google Generative AI**. Each line is setting up a connection or configuration for the specific API, enabling you to interact with their services in your code. Let's break down what each line does:\n",
        "\n",
        "### 1. **`openai = OpenAI()`**:\n",
        "   - This line **initializes the OpenAI API client**.\n",
        "   - The `OpenAI()` function (assuming you're using the OpenAI SDK) connects your code to OpenAI’s API, allowing you to interact with models like GPT-3, GPT-4, etc.\n",
        "   - To work correctly, it needs your **API key**, which is usually set via environment variables or passed during initialization.\n",
        "   - Once the client is initialized, you can use it to send requests for things like **text generation**, **completions**, or other tasks supported by OpenAI's models.\n",
        "\n",
        "   **What it does**: Prepares your code to interact with the OpenAI API, which might be used later to send requests for language models, embeddings, or other functionalities.\n",
        "\n",
        "---\n",
        "\n",
        "### 2. **`claude = anthropic.Anthropic()`**:\n",
        "   - This line **initializes the Anthropic API client**, specifically for accessing Anthropic's **Claude** language model.\n",
        "   - `anthropic.Anthropic()` is part of Anthropic's Python SDK, and it connects your code to their API, allowing you to interact with models like **Claude**.\n",
        "\n",
        "---\n",
        "\n",
        "### General Flow:\n",
        "1. **Initialization**: Each of these lines initializes the **API clients** for their respective services — OpenAI, Anthropic (Claude), and Google Generative AI.\n",
        "2. **Access to Models**: Once these clients are initialized, you can use them to interact with their models and services. For example, you might make API calls to generate text, complete prompts, or perform other AI-driven tasks.\n",
        "3. **Authentication**: Typically, behind the scenes, these clients require **API keys** or other credentials, which are often stored in environment variables or configured during the setup process. This allows your code to securely interact with the respective APIs.\n",
        "\n"
      ],
      "metadata": {
        "id": "G_L8NPcARZJM"
      },
      "id": "G_L8NPcARZJM"
    },
    {
      "cell_type": "markdown",
      "id": "42f77b59-2fb1-462a-b90d-78994e4cef33",
      "metadata": {
        "id": "42f77b59-2fb1-462a-b90d-78994e4cef33"
      },
      "source": [
        "### Asking LLMs to tell a joke\n",
        "\n",
        "### Main Functions of the Code:\n",
        "- **System Message Setup**:\n",
        "  - Defines the behavior or tone of the assistant using a `system_message`.\n",
        "  - Example: `\"You are an assistant that is great at telling jokes\"`.\n",
        "\n",
        "- **User Prompt Creation**:\n",
        "  - Creates a specific user prompt for the assistant.\n",
        "  - Example: `\"Tell a bawdy joke for an audience of professional comedians\"`.\n",
        "\n",
        "- **Prompt List Creation**:\n",
        "  - Combines the `system_message` and `user_prompt` into a structured list called `prompts`.\n",
        "  - This list contains dictionaries with a role (`system` or `user`) and the corresponding content.\n",
        "  \n",
        "- **API Request**:\n",
        "  - Uses the `openai.chat.completions.create()` method to generate a response from the model.\n",
        "  - Specifies the model (`gpt-4o-mini`) and sends the `prompts` as the input messages.\n",
        "  \n",
        "- **Printing the Model's Response**:\n",
        "  - Accesses the response using `completion.choices[0].message.content` to print the content generated by the model.\n",
        "\n",
        "### Focus Points for Learning:\n",
        "- **Roles (`system` and `user`)**:\n",
        "  - Understand the importance of defining different roles to set up the assistant's behavior (`system`) and provide specific prompts (`user`).\n",
        "  - Learn how the system prompt influences the entire conversation's tone and purpose.\n",
        "\n",
        "- **Prompt Engineering**:\n",
        "  - Practice structuring prompts to elicit desired responses from the model.\n",
        "  - Learn how to use multiple roles to guide the assistant more effectively.\n",
        "\n",
        "- **API Method Usage**:\n",
        "  - Get familiar with the **`openai.chat.completions.create()`** method to interact with the model.\n",
        "  - Understand how to pass the **model**, **messages**, and other relevant parameters to the API.\n",
        "\n",
        "- **Accessing the Response**:\n",
        "  - Understand how to access the generated response through `completion.choices`.\n",
        "  - Learn how to interpret and extract the content from the response object to use it in your application.\n",
        "\n",
        "These key functions and concepts will help you become more adept at structuring interactions with OpenAI's models, focusing on prompt engineering and using the API effectively."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "378a0296-59a2-45c6-82eb-941344d3eeff",
      "metadata": {
        "id": "378a0296-59a2-45c6-82eb-941344d3eeff"
      },
      "outputs": [],
      "source": [
        "system_message = \"You are an assistant that is great at telling jokes\"\n",
        "user_prompt = \"Tell a bawdy joke for an audience of professional comedians\"\n",
        "\n",
        "prompts = [\n",
        "    {\"role\": \"system\", \"content\": system_message},\n",
        "    {\"role\": \"user\", \"content\": user_prompt}\n",
        "  ]\n",
        "\n",
        "completion = openai.chat.completions.create(\n",
        "    model='gpt-4o-mini',\n",
        "    messages=prompts\n",
        ")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "3b3879b6-9a55-4fed-a18c-1ea2edfaf397",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "3b3879b6-9a55-4fed-a18c-1ea2edfaf397",
        "outputId": "27309f59-7111-4d9a-a138-2cd6c4edd73d"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Why did the scarecrow win an award?\n",
            "\n",
            "Because he was outstanding in his field... but he still couldn’t figure out how to make hay while the sun shined on his mishaps in the barn!\n"
          ]
        }
      ],
      "source": [
        "# send request to openAI\n",
        "\n",
        "print(completion.choices[0].message.content)"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "#### Response Object\n",
        "\n",
        "This is an example of a response object returned by the OpenAI API after making a chat completion request. Let me break down the key parts of this JSON object:\n",
        "\n",
        "```json\n",
        "{\n",
        "    \"id\": \"chatcmpl-abc123\",\n",
        "    \"object\": \"chat.completion\",\n",
        "    \"created\": 1677858242,\n",
        "    \"model\": \"gpt-4o-mini\",\n",
        "    \"usage\": {\n",
        "        \"prompt_tokens\": 13,\n",
        "        \"completion_tokens\": 7,\n",
        "        \"total_tokens\": 20,\n",
        "        \"completion_tokens_details\": {\n",
        "            \"reasoning_tokens\": 0\n",
        "        }\n",
        "    },\n",
        "    \"choices\": [\n",
        "        {\n",
        "            \"message\": {\n",
        "                \"role\": \"assistant\",\n",
        "                \"content\": \"\\n\\nThis is a test!\"\n",
        "            },\n",
        "            \"logprobs\": null,\n",
        "            \"finish_reason\": \"stop\",\n",
        "            \"index\": 0\n",
        "        }\n",
        "    ]\n",
        "}\n",
        "```\n",
        "\n",
        "### Breakdown of Key Elements:\n",
        "\n",
        "1. **`id`**: `\"chatcmpl-abc123\"`\n",
        "   - This is a unique identifier for the response.\n",
        "   - It can be useful for tracking or logging purposes.\n",
        "\n",
        "2. **`object`**: `\"chat.completion\"`\n",
        "   - This specifies the type of response object.\n",
        "   - Here, it indicates that the object is a chat completion.\n",
        "\n",
        "3. **`created`**: `1677858242`\n",
        "   - This is a timestamp representing when the completion was created.\n",
        "   - It’s in Unix time format, which is the number of seconds since January 1, 1970.\n",
        "\n",
        "4. **`model`**: `\"gpt-4o-mini\"`\n",
        "   - This specifies the model that generated the response.\n",
        "   - In this case, it’s `\"gpt-4o-mini\"`.\n",
        "\n",
        "5. **`usage`**: Contains token usage information.\n",
        "   - **`prompt_tokens`**: `13`\n",
        "     - The number of tokens used for the input (the user's prompts).\n",
        "   - **`completion_tokens`**: `7`\n",
        "     - The number of tokens used for the model's generated response.\n",
        "   - **`total_tokens`**: `20`\n",
        "     - The total number of tokens used for both the prompt and the response.\n",
        "   - **`completion_tokens_details`**:\n",
        "     - **`reasoning_tokens`**: `0`\n",
        "       - This provides additional details about the types of tokens used in the response. It might be specific to this model, for tracking certain aspects of the generated completion.\n",
        "\n",
        "6. **`choices`**: A list of response choices.\n",
        "   - This list contains one or more possible completions from the model. Each item in this list represents a different generated completion.\n",
        "   - In this example, there is only one completion (index `0`).\n",
        "   \n",
        "   **Within each choice**:\n",
        "   - **`message`**:\n",
        "     - **`role`**: `\"assistant\"`\n",
        "       - This indicates that this part of the response is from the assistant.\n",
        "     - **`content`**: `\"This is a test!\"`\n",
        "       - This is the actual response generated by the assistant.\n",
        "\n",
        "   - **`logprobs`**: `null`\n",
        "     - This field may contain log probabilities of the tokens generated, which can be used for analyzing how confident the model was about each token. In this example, it is `null`.\n",
        "\n",
        "   - **`finish_reason`**: `\"stop\"`\n",
        "     - This indicates why the completion ended. The value `\"stop\"` means that the model stopped because it reached a logical stopping point, based on its internal heuristics.\n",
        "     - Other reasons could include `\"length\"` if it reached a token limit, or `\"max_tokens\"` if it hit a specific limit defined by the request.\n",
        "\n",
        "   - **`index`**: `0`\n",
        "     - This is the index of the completion in the list of possible choices.\n",
        "     - Useful if you request multiple completions and need to differentiate between them.\n",
        "\n",
        "### Summary:\n",
        "- The JSON response provides a complete breakdown of what the model returned, including metadata about the request, token usage details, and the actual response message(s).\n",
        "- The `usage` information is particularly useful for tracking the cost of using the API, as many OpenAI models charge based on the number of tokens used.\n",
        "- The `choices` list contains the generated response content and other details about how and why the response was generated.\n",
        "\n",
        "This structure is designed to be detailed, allowing you to understand exactly how the model responded and the resources used in the process."
      ],
      "metadata": {
        "id": "OVhV6pQhpO8E"
      },
      "id": "OVhV6pQhpO8E"
    },
    {
      "cell_type": "code",
      "source": [
        "import json  # Import the json module to pretty-print the response\n",
        "\n",
        "completion = openai.chat.completions.create(\n",
        "    model='gpt-4o-mini',\n",
        "    messages=prompts\n",
        ")\n",
        "\n",
        "print(completion.choices[0].message.content)\n",
        "print('\\n #---------  Response Object ---------#\\n')\n",
        "# Convert the response to a dictionary and print it\n",
        "completion_dict = completion.to_dict()\n",
        "print(json.dumps(completion_dict, indent=2))"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "DKzYjx1lq2kk",
        "outputId": "35552f26-73a0-4708-dbcf-1c5e1a2c2c3e"
      },
      "id": "DKzYjx1lq2kk",
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Sure, here’s a more risqué joke for a crowd of professional comedians:\n",
            "\n",
            "Why did the comedian bring a ladder to the bar?\n",
            "\n",
            "Because he heard the drinks were on the house—but he wanted to make sure he could reach the top shelf for some \"adult\" entertainment!\n",
            "\n",
            " #---------  Response Object ---------#\n",
            "\n",
            "{\n",
            "  \"id\": \"chatcmpl-ALrzSQmRhYvvAvwpyMkZSRl8FS2eu\",\n",
            "  \"choices\": [\n",
            "    {\n",
            "      \"finish_reason\": \"stop\",\n",
            "      \"index\": 0,\n",
            "      \"logprobs\": null,\n",
            "      \"message\": {\n",
            "        \"content\": \"Sure, here\\u2019s a more risqu\\u00e9 joke for a crowd of professional comedians:\\n\\nWhy did the comedian bring a ladder to the bar?\\n\\nBecause he heard the drinks were on the house\\u2014but he wanted to make sure he could reach the top shelf for some \\\"adult\\\" entertainment!\",\n",
            "        \"refusal\": null,\n",
            "        \"role\": \"assistant\"\n",
            "      }\n",
            "    }\n",
            "  ],\n",
            "  \"created\": 1729775790,\n",
            "  \"model\": \"gpt-4o-mini-2024-07-18\",\n",
            "  \"object\": \"chat.completion\",\n",
            "  \"system_fingerprint\": \"fp_482c22a7bc\",\n",
            "  \"usage\": {\n",
            "    \"completion_tokens\": 55,\n",
            "    \"prompt_tokens\": 32,\n",
            "    \"total_tokens\": 87,\n",
            "    \"completion_tokens_details\": {\n",
            "      \"reasoning_tokens\": 0\n",
            "    },\n",
            "    \"prompt_tokens_details\": {\n",
            "      \"cached_tokens\": 0\n",
            "    }\n",
            "  }\n",
            "}\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "### OpenAI LLM APIs\n",
        "\n",
        "To effectively work with OpenAI LLM APIs, there are several key concepts you'll want to understand. These concepts help you use the APIs effectively, control the output, and understand what the model can do. Here’s a list of important areas to focus on:\n",
        "\n",
        "### 1. **API Basics**\n",
        "   - **Endpoints**: Understand the different API endpoints available, such as the `ChatCompletion` endpoint for chat models.\n",
        "   - **Authentication**: Learn how to use API keys securely to authenticate your requests.\n",
        "   - **API Limits and Pricing**: Be aware of rate limits, token usage, and how pricing is calculated.\n",
        "\n",
        "### 2. **Roles and Messages**\n",
        "   - **Message Structure**: Understand how messages are structured with `\"role\"` (e.g., `system`, `user`, `assistant`) and `\"content\"`. These roles help define the context of the conversation and guide the model's behavior.\n",
        "   - **System, User, Assistant Roles**: Grasp how each role contributes to a structured conversation and how to use them to control the assistant's behavior.\n",
        "\n",
        "### 3. **Parameters and Controls**\n",
        "   - **Temperature**: Controls the randomness of responses. Higher values make the output more creative, while lower values make it more deterministic.\n",
        "   - **Top-p (Nucleus Sampling)**: Similar to temperature, this parameter influences response generation by limiting the choice of next words to a certain cumulative probability.\n",
        "   - **Max Tokens**: Understand how to control the length of responses to manage costs and ensure concise answers.\n",
        "   - **Frequency Penalty & Presence Penalty**: These parameters control repetition and encourage the introduction of new ideas, helping to make responses more interesting or stay focused.\n",
        "\n",
        "### 4. **Tokens and Usage**\n",
        "   - **Tokenization**: Learn how text is broken into tokens. Tokens can be as short as one character or as long as one word. Understanding how tokenization works is crucial for managing the input and output efficiently.\n",
        "   - **Prompt Tokens vs Completion Tokens**: Understand the difference between tokens used for the input prompt and those generated as a response. Both affect the API usage costs.\n",
        "\n",
        "### 5. **Prompt Engineering**\n",
        "   - **System Prompts**: Learn how to design system prompts that set up the assistant's behavior effectively for a given task.\n",
        "   - **Few-Shot Learning**: Practice giving examples in the prompt to teach the model how to answer in a specific format.\n",
        "   - **Prompt Design**: Understand how to craft effective user prompts to elicit desired responses from the model.\n",
        "\n",
        "### 6. **Managing Context**\n",
        "   - **Conversation History**: Learn how to maintain conversation history by passing previous messages in the `messages` parameter to create coherent, context-aware interactions.\n",
        "   - **Token Limits**: Understand the token limit of the models (e.g., 8,000 or 32,000 tokens depending on the version) and how to manage context within these limits.\n",
        "\n",
        "### 7. **Handling Responses**\n",
        "   - **Response Parsing**: Be comfortable working with the JSON response object. Learn how to access different parts of the response, such as choices and usage statistics.\n",
        "   - **Error Handling**: Learn to handle common errors (e.g., rate limits, token limit exceeded) gracefully, using retry logic or catching exceptions.\n",
        "\n",
        "### 8. **Use Cases and Model Limitations**\n",
        "   - **Understand Use Cases**: Be familiar with how LLMs are used—like content generation, summarization, answering questions, writing code, and more.\n",
        "   - **Limitations**: Understand what the model can and cannot do. For instance, the model may generate incorrect information confidently. It’s important to know when to validate responses or use other tools.\n",
        "\n",
        "### 9. **Streaming Responses**\n",
        "   - **Response Streaming**: Learn about streaming responses (like typing indicators in chat) to improve user experience, especially for long responses.\n",
        "\n",
        "### 10. **Rate Limits and Best Practices**\n",
        "   - **Rate Limits**: Understand the rate limits associated with your API key and how to optimize requests to stay within these limits.\n",
        "   - **Best Practices**: Learn best practices, such as batching requests, optimizing prompts to reduce tokens, and ensuring API keys are kept secure.\n",
        "\n",
        "### 11. **Advanced Techniques**\n",
        "   - **Fine-Tuning**: Learn about fine-tuning models on specific datasets to improve their performance for a particular task.\n",
        "   - **Tool Use with Plugins**: In some advanced cases, LLMs can interact with external tools or plugins (e.g., for code execution or searching the web). Understanding how these plugins work can greatly enhance what your assistant can accomplish.\n",
        "\n",
        "### 12. **APIs in Practice**\n",
        "   - **Experimentation**: Experiment with different types of system and user prompts to see how they influence the assistant’s responses.\n",
        "   - **Integration**: Learn to integrate the API into applications using libraries like Python’s `requests` or dedicated SDKs (`openai` library). Build simple projects such as chatbots, question-answering systems, or content generators to practice.\n",
        "\n",
        "Understanding these fundamental concepts will give you the ability to use OpenAI's LLM APIs effectively, creating more intelligent and contextually-aware applications while optimizing for performance and cost."
      ],
      "metadata": {
        "id": "EgqPnXQDr2Sh"
      },
      "id": "EgqPnXQDr2Sh"
    },
    {
      "cell_type": "markdown",
      "source": [
        "### **Temperature**\n",
        "\n",
        "The **`temperature`** parameter controls the randomness or creativity of the model's responses.\n",
        "\n",
        "- **`temperature=0.7`**:\n",
        "  - The **temperature** value ranges typically from `0` to `1` (though it can sometimes go higher).\n",
        "  - **Lower values (e.g., `0` to `0.3`)** make the model more deterministic and focused. It tends to generate more predictable, often fact-based responses.\n",
        "  - **Higher values (e.g., `0.7` to `1`)** make the model more creative and diverse. It allows for more varied responses, but this can sometimes lead to less consistent or off-topic content.\n",
        "\n",
        "- By setting **`temperature=0.7`**, you are asking the model to provide responses that strike a balance between creativity and reliability. It won't be too rigid or repetitive, but it also won't be completely unpredictable. It's a good middle-ground setting that allows for some creativity without being too random."
      ],
      "metadata": {
        "id": "m_WEThuPoZ_b"
      },
      "id": "m_WEThuPoZ_b"
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "3d2d6beb-1b81-466f-8ed1-40bf51e7adbf",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "3d2d6beb-1b81-466f-8ed1-40bf51e7adbf",
        "outputId": "6141eef8-8fb8-4cb7-a339-1033b6220b2a"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Sure, here's one for a seasoned crowd:\n",
            "\n",
            "Why did the scarecrow win an award?\n",
            "\n",
            "Because he was outstanding in his field... until the crows started asking for a raise! \n",
            "\n",
            "(Feel free to adjust the delivery to match your style—nothing like a little playful innuendo to spice up the punchline!)\n"
          ]
        }
      ],
      "source": [
        "# Temperature setting controls creativity\n",
        "\n",
        "completion = openai.chat.completions.create(\n",
        "    model='gpt-4o-mini',\n",
        "    messages=prompts,\n",
        "    temperature=0.9\n",
        ")\n",
        "print(completion.choices[0].message.content)"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "### **Top-p (Nucleus Sampling)**\n",
        "The `top_p` parameter controls the diversity of the response by limiting the possible tokens to a subset based on cumulative probability. A value between `0` and `1` is used:\n",
        "\n",
        "- **Lower Value (e.g., `top_p=0.3`)**: Limits the response to the most probable words, making the response more focused and predictable.\n",
        "- **Higher Value (e.g., `top_p=0.9`)**: Allows for more diversity in the response.\n",
        "\n",
        "In this example, setting `top_p=0.85` means that the model will only consider tokens that collectively make up 85% of the probability distribution, ensuring some level of diversity but keeping it within a manageable range."
      ],
      "metadata": {
        "id": "XEsBYeyVsZlB"
      },
      "id": "XEsBYeyVsZlB"
    },
    {
      "cell_type": "code",
      "source": [
        "completion = openai.chat.completions.create(\n",
        "    model='gpt-4o-mini',\n",
        "    messages=prompts,\n",
        "    temperature=0.9,      # Controls randomness\n",
        "    top_p=0.85            # Controls diversity of the response\n",
        ")\n",
        "print(completion.choices[0].message.content)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "0Juy4shIszDK",
        "outputId": "f51511a8-5c7f-4aec-8be3-0144d29024a4"
      },
      "id": "0Juy4shIszDK",
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Sure, here’s a cheeky one for a crowd of pros:\n",
            "\n",
            "Why did the comedian bring a ladder to the bar?\n",
            "\n",
            "Because they heard the drinks were on the house, but they were really just trying to raise the stakes!\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "### **Max Tokens**\n",
        "The `max_tokens` parameter controls the length of the generated response. It limits the number of tokens (words or characters) in the response:\n",
        "\n",
        "- **Lower Value (e.g., `max_tokens=50`)**: Keeps the response short.\n",
        "- **Higher Value (e.g., `max_tokens=200`)**: Allows for longer, more detailed responses.\n",
        "\n",
        "In this example, setting `max_tokens=100` ensures that the response is concise, useful for scenarios like short answers or summaries.\n"
      ],
      "metadata": {
        "id": "ZYONm-7DscZf"
      },
      "id": "ZYONm-7DscZf"
    },
    {
      "cell_type": "code",
      "source": [
        "completion = openai.chat.completions.create(\n",
        "    model='gpt-4o-mini',\n",
        "    messages=prompts,\n",
        "    temperature=0.7,      # Balanced creativity\n",
        "    max_tokens=100        # Limit the response to a maximum of 100 tokens\n",
        ")\n",
        "print(completion.choices[0].message.content)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "T2unZk-ws7Th",
        "outputId": "ab81f62d-1139-423d-aaca-fb9a494afbd2"
      },
      "id": "T2unZk-ws7Th",
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Sure, here’s a classic with a cheeky twist:\n",
            "\n",
            "Why did the scarecrow win an award?\n",
            "\n",
            "Because he was outstanding in his field... and he really knew how to bury the competition under a stack of corny puns! \n",
            "\n",
            "But seriously, folks, it’s hard to find good help these days. I mean, my last assistant was so bad, I had to fire him… he kept trying to plant ideas instead of seeds!\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "### **Frequency Penalty**\n",
        "The `frequency_penalty` parameter helps prevent the model from repeating words or phrases, making the response more varied:\n",
        "\n",
        "- **Lower Value (e.g., `frequency_penalty=0.0`)**: No penalty, which can lead to repetition if the model tends to be repetitive.\n",
        "- **Higher Value (e.g., `frequency_penalty=1.0`)**: Encourages more diverse responses by penalizing repeated words.\n",
        "\n",
        "In this example, setting `frequency_penalty=0.8` discourages the model from repeating itself, which can be useful when generating creative content where redundancy is undesirable.\n"
      ],
      "metadata": {
        "id": "N_SgqaU1sgN-"
      },
      "id": "N_SgqaU1sgN-"
    },
    {
      "cell_type": "code",
      "source": [
        "completion = openai.chat.completions.create(\n",
        "    model='gpt-4o-mini',\n",
        "    messages=prompts,\n",
        "    temperature=0.7,            # Balanced creativity\n",
        "    frequency_penalty=0.8       # Penalize repeated phrases or words\n",
        ")\n",
        "print(completion.choices[0].message.content)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Vojv1hTntC6N",
        "outputId": "d736816f-c3d5-4529-8b4a-5cbc61833112"
      },
      "id": "Vojv1hTntC6N",
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Sure, here’s a bawdy joke for you:\n",
            "\n",
            "Why did the scarecrow win an award?\n",
            "\n",
            "Because he was outstanding in his field... but rumor has it, he also had a reputation for getting straw-hatted after dark! \n",
            "\n",
            "(Just remember to keep it playful and know your audience!)\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Putting It All Together:\n",
        "You can combine these parameters to fine-tune the model’s response characteristics based on your requirements:\n",
        "\n",
        "- **`temperature=0.9`**: Promotes creativity.\n",
        "- **`top_p=0.9`**: Adds diversity while maintaining focus.\n",
        "- **`max_tokens=150`**: Caps response length.\n",
        "- **`frequency_penalty=0.5`**: Encourages avoiding repetition but not overly strict.\n",
        "\n",
        "These parameters provide a lot of control over how the model responds, allowing you to optimize it for specific use cases, whether it's more deterministic or creative, concise or elaborate."
      ],
      "metadata": {
        "id": "_PFrH6xosUw7"
      },
      "id": "_PFrH6xosUw7"
    },
    {
      "cell_type": "code",
      "source": [
        "completion = openai.chat.completions.create(\n",
        "    model='gpt-4o-mini',\n",
        "    messages=prompts,\n",
        "    temperature=0.9,          # Makes the response creative\n",
        "    top_p=0.9,                # Allows for diverse responses\n",
        "    max_tokens=150,           # Limit response length to 150 tokens\n",
        "    frequency_penalty=0.5     # Avoid repeating words too frequently\n",
        ")\n",
        "print(completion.choices[0].message.content)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "UqrJThunsUd5",
        "outputId": "9c59518f-d80f-47a8-8a78-fccb86236e47"
      },
      "id": "UqrJThunsUd5",
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Sure, here’s a bawdy joke that should resonate with a crowd of professional comedians:\n",
            "\n",
            "Why did the scarecrow win an award?\n",
            "\n",
            "Because he was outstanding in his field… and had plenty of straw to wrap around his “crops”! \n",
            "\n",
            "(Okay, that’s not as naughty as you might expect—maybe you have to \"harvest\" the punchline! 😏)\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Claude 3.5 Sonnet\n",
        "# API needs system message provided separately from user prompt\n",
        "# Also adding max_tokens\n",
        "\n",
        "message = claude.messages.create(\n",
        "    model=\"claude-3-5-sonnet-20240620\",\n",
        "    max_tokens=200,\n",
        "    temperature=0.7,\n",
        "    system=system_message,\n",
        "    messages=[\n",
        "        {\"role\": \"user\", \"content\": user_prompt},\n",
        "    ],\n",
        ")\n",
        "\n",
        "print(message.content[0].text)"
      ],
      "metadata": {
        "id": "CHB6qTzVtTHq"
      },
      "id": "CHB6qTzVtTHq",
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# Claude 3.5 Sonnet again\n",
        "# Now let's add in streaming back results\n",
        "\n",
        "result = claude.messages.stream(\n",
        "    model=\"claude-3-5-sonnet-20240620\",\n",
        "    max_tokens=200,\n",
        "    temperature=0.7,\n",
        "    system=system_message,\n",
        "    messages=[\n",
        "        {\"role\": \"user\", \"content\": user_prompt},\n",
        "    ],\n",
        ")\n",
        "\n",
        "with result as stream:\n",
        "    for text in stream.text_stream:\n",
        "            print(text, end=\"\", flush=True)"
      ],
      "metadata": {
        "id": "nIuyhwGWtVgQ"
      },
      "id": "nIuyhwGWtVgQ",
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "6df48ce5-70f8-4643-9a50-b0b5bfdb66ad",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 191
        },
        "id": "6df48ce5-70f8-4643-9a50-b0b5bfdb66ad",
        "outputId": "eb4aaeea-04df-412a-a492-0aab138d192d"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "A guy walks into a bar with a tiny piano under his arm. He sits down, places the piano on the counter, and starts playing a beautiful tune. \n",
            "\n",
            "The bartender, impressed, asks, \"Wow, where did you get that little piano?\"\n",
            "\n",
            "The guy replies, \"It's a special piano. It shrinks every time I play it.\"\n",
            "\n",
            "The bartender, intrigued, asks, \"Really? How does that work?\"\n",
            "\n",
            "The guy leans in and whispers, \"I just gotta keep my fingers crossed.\" \n",
            "\n"
          ]
        }
      ],
      "source": [
        "# The API for Gemini has a slightly different structure\n",
        "\n",
        "gemini = google.generativeai.GenerativeModel(\n",
        "    model_name='gemini-1.5-flash',\n",
        "    system_instruction=system_message\n",
        ")\n",
        "response = gemini.generate_content(user_prompt)\n",
        "print(response.text)"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Stream, Reply, Display\n",
        "\n",
        "This code snippet demonstrates how to stream responses from OpenAI's API and update the display dynamically, such as in an interactive notebook like Jupyter. Let’s break it down step by step:\n",
        "\n",
        "### Overview\n",
        "- The code streams the response from OpenAI's language model and displays it incrementally in a Jupyter-like environment.\n",
        "- This approach is particularly useful when generating long responses, as it allows you to start displaying output before the full response has been generated.\n",
        "\n",
        "### Detailed Breakdown\n",
        "\n",
        "1. **Streaming the Response**\n",
        "   ```python\n",
        "   stream = openai.chat.completions.create(\n",
        "       model='gpt-4o',\n",
        "       messages=prompts,\n",
        "       temperature=0.7,\n",
        "       stream=True\n",
        "   )\n",
        "   ```\n",
        "   - **`stream=True`**: This tells the API to send the response in chunks, as it is being generated.\n",
        "   - **`stream`**: This variable represents the iterable response from the OpenAI API. Each item in `stream` contains a part of the model’s response (called a \"chunk\").\n",
        "  \n",
        "2. **Initializing `reply` and Display Handle**\n",
        "   ```python\n",
        "   reply = \"\"\n",
        "   display_handle = display(Markdown(\"\"), display_id=True)\n",
        "   ```\n",
        "   - **`reply = \"\"`**: Initializes an empty string to store the complete reply as it is constructed chunk by chunk.\n",
        "   - **`display_handle = display(Markdown(\"\"), display_id=True)`**:\n",
        "     - **`display()`**: This function is used to output content to a notebook cell. The initial `Markdown(\"\")` creates an empty Markdown output, which is later updated.\n",
        "     - **`display_id=True`**: Creates a display with an identifier, allowing for updates to the same output instead of creating new outputs each time.\n",
        "     - **`display_handle`**: Stores a reference to the display so it can be updated with new content as the response streams in.\n",
        "\n",
        "3. **Streaming the Response and Updating the Display**\n",
        "   ```python\n",
        "   for chunk in stream:\n",
        "       reply += chunk.choices[0].delta.content or ''\n",
        "       reply = reply.replace(\"```\", \"\").replace(\"markdown\", \"\")\n",
        "       update_display(Markdown(reply), display_id=display_handle.display_id)\n",
        "   ```\n",
        "   - **`for chunk in stream:`**: Iterates over each chunk of the streamed response.\n",
        "   \n",
        "   - **`reply += chunk.choices[0].delta.content or ''`**:\n",
        "     - **`chunk`**: Each chunk contains a partial response.\n",
        "     - **`chunk.choices[0].delta.content`**: This retrieves the content of the chunk. The `delta` object represents a partial update (like an incremental addition to the response).\n",
        "     - **`or ''`**: This ensures that if `content` is `None`, it appends an empty string, preventing errors.\n",
        "     - **`reply += ...`**: Appends the newly received content to the existing `reply` string.\n",
        "\n",
        "   - **`reply = reply.replace(\"```\",\"\").replace(\"markdown\",\"\")`**:\n",
        "     - **`.replace(\"```\", \"\")`**: Removes backticks (`\\`\\`\\``), often used to format code blocks, from the streamed response.\n",
        "     - **`.replace(\"markdown\", \"\")`**: Removes occurrences of the word \"markdown\" from the response, likely to avoid unnecessary formatting instructions.\n",
        "     - These replacements help keep the output clean and avoid unintentional formatting issues in the displayed content.\n",
        "\n",
        "   - **`update_display(Markdown(reply), display_id=display_handle.display_id)`**:\n",
        "     - **`Markdown(reply)`**: Converts the `reply` string to a Markdown object so it is properly formatted when displayed.\n",
        "     - **`update_display()`**: Updates the existing display (referenced by `display_handle.display_id`) with the new content of `reply`. This avoids creating new outputs for every chunk and instead progressively updates the original display.\n",
        "\n",
        "### Summary\n",
        "- The code uses streaming to receive the response in parts and constructs the full response incrementally.\n",
        "- **`reply`** stores the combined response.\n",
        "- **`display_handle`** allows updating the output in real-time in a notebook, providing an interactive experience.\n",
        "- The loop iteratively adds each chunk to the reply and updates the display, ensuring that users see the response being built as it is generated.\n",
        "\n",
        "This approach creates a seamless experience for users, where they can see the assistant's response being generated step by step, which is especially helpful for longer outputs or when reducing waiting time in interactive environments."
      ],
      "metadata": {
        "id": "eFa0y8HWt6Oa"
      },
      "id": "eFa0y8HWt6Oa"
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "83ddb483-4f57-4668-aeea-2aade3a9e573",
      "metadata": {
        "id": "83ddb483-4f57-4668-aeea-2aade3a9e573"
      },
      "outputs": [],
      "source": [
        "# To be serious! GPT-4o-mini with the original question\n",
        "\n",
        "prompts = [\n",
        "    {\"role\": \"system\", \"content\": \"You are a helpful assistant\"},\n",
        "    {\"role\": \"user\", \"content\": \"How do I decide if a business problem is suitable for an LLM solution?\"}\n",
        "  ]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "749f50ab-8ccd-4502-a521-895c3f0808a2",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 749
        },
        "id": "749f50ab-8ccd-4502-a521-895c3f0808a2",
        "outputId": "b67c504f-553c-4db3-e62b-14bc8ec93aeb"
      },
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ],
            "text/markdown": "Determining whether a business problem is suitable for a language model solution, such as a large language model (LLM) like GPT-3 or similar, involves evaluating several key factors. Here are some steps and considerations to help you make that decision:\n\n1. **Nature of the Problem:**\n   - **Textual Data:** LLMs are particularly effective for problems involving natural language processing (NLP), such as text generation, summarization, translation, sentiment analysis, entity recognition, and question-answering.\n   - **Creativity and Variability:** Problems that require creativity, generating varied outputs, or handling ambiguous instructions are well-suited for LLMs.\n\n2. **Complexity of Language Understanding:**\n   - **Contextual Understanding:** If the problem requires understanding context, nuances, or conversational dynamics, an LLM might be appropriate.\n   - **Complex Language Tasks:** Tasks that involve complex language understanding, such as drafting detailed reports, generating conversational agents, or understanding nuanced customer feedback, can benefit from an LLM.\n\n3. **Scalability:**\n   - **Volume of Text:** If the problem involves processing large volumes of text efficiently, LLMs can be useful due to their ability to handle vast datasets.\n\n4. **Data Availability:**\n   - **Quality and Quantity of Data:** Ensure you have access to sufficient high-quality text data for training or fine-tuning if needed. Pre-trained LLMs can handle a wide range of tasks, but specific applications may require additional data.\n\n5. **Business Value:**\n   - **Impact on Business Goals:** Evaluate whether the LLM solution aligns with your business goals and can deliver tangible value, such as improving customer satisfaction, increasing efficiency, or reducing costs.\n   - **Cost-Benefit Analysis:** Consider the cost of implementing an LLM solution versus the expected benefits.\n\n6. **Ethical and Compliance Considerations:**\n   - **Bias and Fairness:** Assess the potential for bias in outputs and ensure the LLM solution complies with ethical standards and regulations.\n   - **Data Privacy:** Ensure that using LLMs adheres to data privacy laws and that sensitive data is protected.\n\n7. **Technical Feasibility:**\n   - **Integration with Existing Systems:** Determine if the LLM solution can be integrated into your current technology stack.\n   - **Resource Requirements:** Consider the computational resources required to deploy and maintain an LLM solution.\n\n8. **Alternatives and Complementary Solutions:**\n   - **Explore Alternatives:** Assess whether simpler models or rule-based systems could solve the problem effectively.\n   - **Hybrid Approaches:** Sometimes, combining LLMs with other techniques (e.g., structured data analysis, machine learning models) provides a more robust solution.\n\n9. **Expertise and Support:**\n   - **Availability of Expertise:** Ensure you have access to the necessary expertise to implement and maintain the LLM solution.\n   - **Vendor Support:** Consider the level of support and documentation available from the LLM provider.\n\nBy considering these factors, you can determine if an LLM is a suitable solution for your business problem. If the problem meets most of these criteria, it is likely a good candidate for an LLM-based approach."
          },
          "metadata": {}
        }
      ],
      "source": [
        "# Have it stream back results in markdown\n",
        "\n",
        "stream = openai.chat.completions.create(\n",
        "    model='gpt-4o',\n",
        "    messages=prompts,\n",
        "    temperature=0.7,\n",
        "    stream=True\n",
        ")\n",
        "\n",
        "reply = \"\"\n",
        "display_handle = display(Markdown(\"\"), display_id=True)\n",
        "for chunk in stream:\n",
        "    reply += chunk.choices[0].delta.content or ''\n",
        "    reply = reply.replace(\"```\",\"\").replace(\"markdown\",\"\")\n",
        "    update_display(Markdown(reply), display_id=display_handle.display_id)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "2618c3fa-9b8e-4280-a070-d039361b8918",
      "metadata": {
        "id": "2618c3fa-9b8e-4280-a070-d039361b8918"
      },
      "outputs": [],
      "source": []
    }
  ],
  "metadata": {
    "kernelspec": {
      "display_name": "Python 3 (ipykernel)",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.11.10"
    },
    "colab": {
      "provenance": [],
      "include_colab_link": true
    }
  },
  "nbformat": 4,
  "nbformat_minor": 5
}