01_Basic_Prompt_Structure.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Chapter 1: Basic Prompt Structure\n",
    "\n",
    "- [Lesson](#lesson)\n",
    "- [Exercises](#exercises)\n",
    "- [Example Playground](#example-playground)\n",
    "\n",
    "## Setup\n",
    "\n",
    "Run the following setup cell to load your API key and establish the `get_completion` helper function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "%pip install -qU pip\n",
    "%pip install -qUr requirements.txt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Import python's built-in regular expression library\n",
    "import re\n",
    "import boto3\n",
    "from botocore.exceptions import ClientError\n",
    "import json\n",
    "\n",
    "# Import the hints module from the utils package\n",
    "from utils import hints\n",
    "\n",
    "# Retrieve the MODEL_NAME variable from the IPython store\n",
    "%store -r modelId\n",
    "%store -r region\n",
    "\n",
    "bedrock_client = boto3.client(service_name='bedrock-runtime', region_name=region)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_completion(prompt, system_prompt=None):\n",
    "    # Define the inference configuration\n",
    "    inference_config = {\n",
    "        \"temperature\": 0.0,  # Set the temperature for generating diverse responses\n",
    "        \"maxTokens\": 200,  # Set the maximum number of tokens to generate\n",
    "        \"topP\": 1,  # Set the top_p value for nucleus sampling\n",
    "    }\n",
    "    # Create the converse method parameters\n",
    "    converse_api_params = {\n",
    "        \"modelId\": modelId,  # Specify the model ID to use\n",
    "        \"messages\": [{\"role\": \"user\", \"content\": [{\"text\": prompt}]}],  # Provide the user's prompt\n",
    "        \"inferenceConfig\": inference_config,  # Pass the inference configuration\n",
    "    }\n",
    "    # Check if system_text is provided\n",
    "    if system_prompt:\n",
    "        # If system_text is provided, add the system parameter to the converse_params dictionary\n",
    "        converse_api_params[\"system\"] = [{\"text\": system_prompt}]\n",
    "\n",
    "    # Send a request to the Bedrock client to generate a response\n",
    "    try:\n",
    "        response = bedrock_client.converse(**converse_api_params)\n",
    "\n",
    "        # Extract the generated text content from the response\n",
    "        text_content = response['output']['message']['content'][0]['text']\n",
    "\n",
    "        # Return the generated text content\n",
    "        return text_content\n",
    "\n",
    "    except ClientError as err:\n",
    "        message = err.response['Error']['Message']\n",
    "        print(f\"A client error occured: {message}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "## Lesson\n",
    "\n",
    "Amazon Bedrock offers three APIs that can be used with Anthropic Claude models, the legacy [Text Completions API](https://docs.aws.amazon.com/bedrock/latest/userguide/model-parameters-anthropic-claude-text-completion.html) , the [Messages API](https://docs.aws.amazon.com/bedrock/latest/userguide/model-parameters-anthropic-claude-messages.html) and the current [Converse API](https://docs.aws.amazon.com/bedrock/latest/userguide/conversation-inference.html). For this tutorial, we will be exclusively using the Converse API.\n",
    "\n",
    "At minimum, a call to Claude using the Converse API requires the following parameters:\n",
    "- `modelId`: the [API model name](https://docs.aws.amazon.com/bedrock/latest/userguide/model-ids.html#model-ids-arns) of the model that you intend to call\n",
    "\n",
    "- `messages`: an array of input messages. Claude 3 models are trained to operate on alternating `user` and `assistant` conversational turns. When creating a new `Message`, you specify the prior conversational turns with the messages parameter, and the model then generates the next `Message` in the conversation.\n",
    "  - Each input message must be an object with a `role` and `content`. You can specify a single `user`-role message, or you can include multiple `user` and `assistant` messages (they must alternate, if so). **The first message must always use the `user` role.**\n",
    "  \n",
    "  You store the content for the message in the `content` field of a [(ContentBlock)](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_ContentBlock.html). Specify text in the `text` field, or if supported by the model, you can also pass the raw bytes for an image in the `image` field of an [(ImageBlock)](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_ImageBlock.html). The other fields in ContentBlock are for [tool use](https://docs.aws.amazon.com/bedrock/latest/userguide/tool-use.html).\n",
    "\n",
    "There are also optional parameters, such as:\n",
    "- `system`: the system prompt - more on this below.\n",
    "  \n",
    "- `temperature`: the degree of variability in Claude's response. For these lessons and exercises, we have set `temperature` to 0.\n",
    "\n",
    "- `max_tokens`: the maximum number of tokens to generate before stopping. Note that Claude may stop before reaching this maximum. This parameter only specifies the absolute maximum number of tokens to generate. Furthermore, this is a *hard* stop, meaning that it may cause Claude to stop generating mid-word or mid-sentence.\n",
    "\n",
    "For a complete list of all API parameters, visit our [API documentation](https://docs.aws.amazon.com/bedrock/latest/userguide/conversation-inference.html)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Examples\n",
    "\n",
    "Let's take a look at how Claude responds to some correctly-formatted prompts. For each of the following cells, run the cell (`shift+enter`), and Claude's response will appear below the block."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Prompt\n",
    "PROMPT = \"Hi Claude, how are you?\"\n",
    "\n",
    "# Print Claude's response\n",
    "print(get_completion(PROMPT))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Prompt\n",
    "PROMPT = \"Can you tell me the color of the ocean?\"\n",
    "\n",
    "# Print Claude's response\n",
    "print(get_completion(PROMPT))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Prompt\n",
    "PROMPT = \"What year was Celine Dion born in?\"\n",
    "\n",
    "# Print Claude's response\n",
    "print(get_completion(PROMPT))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now let's take a look at some prompts that do not include the correct Converse API formatting. For these malformatted prompts, the Converse API returns an error.\n",
    "\n",
    "First, we have an example of a Converse API call that lacks `role` and `content` fields in the `messages` array."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> ⚠️ **Warning:** Due to the incorrect formatting of the messages parameter in the prompt, the following cell will return an error. This is expected behavior."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Get Claude's response\n",
    "inference_config = {\n",
    "    \"temperature\": 0.0, # Set the temperature for generating diverse responses\n",
    "    \"maxTokens\": 200 # Set the maximum number of tokens to generate\n",
    "}\n",
    "\n",
    "converse_api_params = {\n",
    "    \"modelId\": modelId,\n",
    "    \"messages\": [{\"text\":\"Hi Claude, how are you?\"}], # Provide the user's prompt\n",
    "    \"inferenceConfig\": inference_config # Pass the inference configuration\n",
    "}\n",
    "\n",
    "response = bedrock_client.converse(**converse_api_params)\n",
    "\n",
    "# Print Claude's response\n",
    "print(response['output']['message']['content'][0]['text'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here's a prompt that fails to alternate between the `user` and `assistant` roles."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> ⚠️ **Warning:** Due to the lack of alternation between `user` and `assistant` roles, Claude will return an error message. This is expected behavior."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "inference_config = {\n",
    "    \"temperature\": 0.5,\n",
    "    \"maxTokens\": 200\n",
    "}\n",
    "# Create the converse method parameters\n",
    "converse_api_params = {\n",
    "    \"modelId\": modelId,\n",
    "    \"messages\": [\n",
    "        {\"role\": \"user\", \"content\": [{\"text\": \"What year was Celine Dion born in?\"}]},\n",
    "        {\"role\": \"user\", \"content\": [{\"text\":\"Also, can you tell me some other facts about her?\"}]}\n",
    "    ],\n",
    "    \"inferenceConfig\": inference_config,\n",
    "}\n",
    "\n",
    "response = bedrock_client.converse(**converse_api_params)\n",
    "\n",
    "# Print Claude's response\n",
    "print(response['output']['message']['content'][0]['text'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`user` and `assistant` messages **MUST alternate**, and messages **MUST start with a `user` turn**. You can have multiple `user` & `assistant` pairs in a prompt (as if simulating a multi-turn conversation). You can also put words into a terminal `assistant` message for Claude to continue from where you left off (more on that in later chapters).\n",
    "\n",
    "#### System Prompts\n",
    "\n",
    "You can also use **system prompts**. A system prompt is a way to **provide context, instructions, and guidelines to Claude** before presenting it with a question or task in the \"User\" turn. \n",
    "\n",
    "Structurally, system prompts exist separately from the list of `user` & `assistant` messages, and thus belong in a separate `system` parameter (take a look at the structure of the `get_completion` helper function in the [Setup](#setup) section of the notebook). \n",
    "\n",
    "Within this tutorial, wherever we might utilize a system prompt, we have provided you a `system` field in your completions function. Should you not want to use a system prompt, simply set the `SYSTEM_PROMPT` variable to an empty string."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### System Prompt Example"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# System prompt\n",
    "SYSTEM_PROMPT = \"Your answer should always be a series of critical thinking questions that further the conversation (do not provide answers to your questions). Do not actually answer the user question.\"\n",
    "\n",
    "# Prompt\n",
    "PROMPT = \"Why is the sky blue?\"\n",
    "\n",
    "# Print Claude's response\n",
    "print(get_completion(PROMPT, SYSTEM_PROMPT))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Why use a system prompt? A **well-written system prompt can improve Claude's performance** in a variety of ways, such as increasing Claude's ability to follow rules and instructions. For more information, visit Anthropic's documentation on [how to use system prompts](https://docs.anthropic.com/claude/docs/how-to-use-system-prompts) with Claude.\n",
    "\n",
    "Now we'll dive into some exercises. If you would like to experiment with the lesson prompts without changing any content above, scroll all the way to the bottom of the lesson notebook to visit the [**Example Playground**](#example-playground)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "## Exercises\n",
    "- [Exercise 1.1 - Counting to Three](#exercise-11---counting-to-three)\n",
    "- [Exercise 1.2 - System Prompt](#exercise-12---system-prompt)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise 1.1 - Counting to Three\n",
    "Using proper `user` / `assistant` formatting, edit the `PROMPT` below to get Claude to **count to three.** The output will also indicate whether your solution is correct."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Prompt - this is the only field you should change\n",
    "PROMPT = \"[Replace this text]\"\n",
    "\n",
    "# Get Claude's response\n",
    "response = get_completion(PROMPT)\n",
    "\n",
    "# Function to grade exercise correctness\n",
    "def grade_exercise(text):\n",
    "    pattern = re.compile(r'^(?=.*1)(?=.*2)(?=.*3).*$', re.DOTALL)\n",
    "    return bool(pattern.match(text))\n",
    "\n",
    "# Print Claude's response and the corresponding grade\n",
    "print(response)\n",
    "print(\"\\n--------------------------- GRADING ---------------------------\")\n",
    "print(\"This exercise has been correctly solved:\", grade_exercise(response))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "❓ If you want a hint, run the cell below!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(hints.exercise_1_1_hint)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise 1.2 - System Prompt\n",
    "\n",
    "Modify the `SYSTEM_PROMPT` to make Claude respond like it's a 3 year old child."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# System prompt - this is the only field you should change\n",
    "SYSTEM_PROMPT = \"[Replace this text]\"\n",
    "\n",
    "# Prompt\n",
    "PROMPT = \"How big is the sky?\"\n",
    "\n",
    "# Get Claude's response\n",
    "response = get_completion(PROMPT, SYSTEM_PROMPT)\n",
    "\n",
    "# Function to grade exercise correctness\n",
    "def grade_exercise(text):\n",
    "    return bool(re.search(r\"giggles\", text) or re.search(r\"soo\", text))\n",
    "\n",
    "# Print Claude's response and the corresponding grade\n",
    "print(response)\n",
    "print(\"\\n--------------------------- GRADING ---------------------------\")\n",
    "print(\"This exercise has been correctly solved:\", grade_exercise(response))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "❓ If you want a hint, run the cell below!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(hints.exercise_1_2_hint)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Congrats!\n",
    "\n",
    "If you've solved all exercises up until this point, you're ready to move to the next chapter. Happy prompting!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "## Example Playground\n",
    "\n",
    "This is an area for you to experiment freely with the prompt examples shown in this lesson and tweak prompts to see how it may affect Claude's responses."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Prompt\n",
    "PROMPT = \"Hi Claude, how are you?\"\n",
    "\n",
    "# Print Claude's response\n",
    "print(get_completion(PROMPT))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Prompt\n",
    "PROMPT = \"Can you tell me the color of the ocean?\"\n",
    "\n",
    "# Print Claude's response\n",
    "print(get_completion(PROMPT))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Prompt\n",
    "PROMPT = \"What year was Celine Dion born in?\"\n",
    "\n",
    "# Print Claude's response\n",
    "print(get_completion(PROMPT))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Get Claude's response\n",
    "inference_config = {\n",
    "    \"temperature\": 0.0\n",
    "}\n",
    "additional_model_fields = {\n",
    "    \"max_tokens\": 200\n",
    "}\n",
    "\n",
    "converse_api_params = {\n",
    "    \"modelId\": modelId,\n",
    "    \"messages\": [{\"text\":\"Hi Claude, how are you?\"}],\n",
    "    \"inferenceConfig\": inference_config,\n",
    "    \"additionalModelRequestFields\": additional_model_fields\n",
    "}\n",
    "\n",
    "response = bedrock_client.converse(**converse_api_params)\n",
    "\n",
    "# Print Claude's response\n",
    "print(response['output']['message']['content'][0]['text'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "inference_config = {\n",
    "    \"temperature\": 0.0\n",
    "}\n",
    "additional_model_fields = {\n",
    "    \"top_p\": 1,\n",
    "    \"max_tokens\": 200\n",
    "}\n",
    "converse_api_params = {\n",
    "    \"modelId\": modelId,\n",
    "    \"messages\": [\n",
    "        {\"role\": \"user\", \"content\": [{\"text\": \"What year was Celine Dion born in?\"}]},\n",
    "        {\"role\": \"user\", \"content\": [{\"text\":\"Also, can you tell me some other facts about her?\"}]}\n",
    "    ],\n",
    "    \"inferenceConfig\": inference_config,\n",
    "    \"additionalModelRequestFields\": additional_model_fields\n",
    "}\n",
    "\n",
    "response = bedrock_client.converse(**converse_api_params)\n",
    "\n",
    "# Print Claude's response\n",
    "print(response['output']['message']['content'][0]['text'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# System prompt\n",
    "SYSTEM_PROMPT = \"Your answer should always be a series of critical thinking questions that further the conversation (do not provide answers to your questions). Do not actually answer the user question.\"\n",
    "\n",
    "# Prompt\n",
    "PROMPT = \"Why is the sky blue?\"\n",
    "\n",
    "# Print Claude's response\n",
    "print(get_completion(PROMPT, SYSTEM_PROMPT))"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "",
   "name": ""
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.14"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}