Predibase documentation (#266)

* Predibase documentation * Added Predibase integration to Github repo Readme * Updated Predibase integration
comet-ml · Sep 17, 2024 · 929a199 · 929a199
1 parent 4553b5c
commit 929a199
Show file tree

Hide file tree

Showing 8 changed files with 500 additions and 7 deletions.
diff --git a/README.md b/README.md
@@ -116,6 +116,7 @@ The easiest way to get started is to use one of our integrations. Opik supports:
 | OpenAI | Log traces for all OpenAI LLM calls | [Documentation](https://www.comet.com//docs/opik/tracing/integrations/openai/?utm_source=opik&utm_medium=github&utm_content=openai_link) | [![Open Quickstart In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/comet-ml/opik/blob/master/apps/opik-documentation/documentation/docs/cookbook/openai.ipynb) |
 | LangChain | Log traces for all LangChain LLM calls | [Documentation](https://www.comet.com/docs/opik/tracing/integrations/langchain/?utm_source=opik&utm_medium=github&utm_content=langchain_link) | [![Open Quickstart In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/comet-ml/opik/blob/master/apps/opik-documentation/documentation/docs/cookbook/langchain.ipynb) |
 | LlamaIndex | Log traces for all LlamaIndex LLM calls | [Documentation](https://www.comet.com/docs/opik/tracing/integrations/llama_index?utm_source=opik&utm_medium=github&utm_content=llama_index_link) | [![Open Quickstart In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/comet-ml/opik/blob/master/apps/opik-documentation/documentation/docs/cookbook/llama-index.ipynb) |
+| Predibase | Fine-tune and serve open-source Large Language Models | [Documentation](https://www.comet.com/docs/opik/tracing/integrations/predibase?utm_source=opik&utm_medium=github&utm_content=predibase_link) | [![Open Quickstart In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/comet-ml/opik/blob/master/apps/opik-documentation/documentation/docs/cookbook/predibase.ipynb) |
 
 > [!TIP]  
 > If the framework you are using is not listed above, feel free to [open an issue](https://github.com/comet-ml/opik/issues) or submit a PR with the integration.

diff --git a/apps/opik-documentation/documentation/docs/cookbook/predibase.ipynb b/apps/opik-documentation/documentation/docs/cookbook/predibase.ipynb
@@ -0,0 +1,254 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Using Opik with Predibase\n",
+    "\n",
+    "This notebook demonstrates how to use Predibase as an LLM provider with LangChain, and how to integrate Comet for tracking and logging.\n",
+    "\n",
+    "## Setup\n",
+    "\n",
+    "First, let's install the necessary packages and set up our environment variables."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "%pip install --upgrade --quiet predibase  opik"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "We will now configure Opik and Predibase:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Configure Opik\n",
+    "import opik\n",
+    "import os\n",
+    "\n",
+    "opik.configure(use_local=False)\n",
+    "\n",
+    "# Configure predibase\n",
+    "import getpass\n",
+    "os.environ[\"PREDIBASE_API_TOKEN\"] = getpass.getpass(\"Enter your Predibase API token\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Creating the Opik Tracer\n",
+    "\n",
+    "In order to log traces to Opik, we will be using the OpikTracer from the LangChain integration."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Import Comet tracer\n",
+    "from opik.integrations.langchain import OpikTracer\n",
+    "\n",
+    "# Initialize Comet tracer\n",
+    "opik_tracer = OpikTracer(\n",
+    "    tags=[\"predibase\", \"langchain\"],\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Initial Call\n",
+    "\n",
+    "Let's set up our Predibase model and make an initial call."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain_community.llms import Predibase\n",
+    "import os\n",
+    "\n",
+    "model = Predibase(\n",
+    "    model=\"mistral-7b\",\n",
+    "    predibase_api_key=os.environ.get(\"PREDIBASE_API_TOKEN\"),\n",
+    ")\n",
+    "\n",
+    "# Test the model with Comet tracing\n",
+    "response = model.invoke(\n",
+    "    \"Can you recommend me a nice dry wine?\",\n",
+    "    config={\n",
+    "        \"temperature\": 0.5,\n",
+    "        \"max_new_tokens\": 1024,\n",
+    "        \"callbacks\": [opik_tracer]\n",
+    "    }\n",
+    ")\n",
+    "print(response)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "In addition to passing the OpikTracer to the invoke method, you can also define it during the creation of the `Predibase` object:\n",
+    "\n",
+    "```python\n",
+    "model = Predibase(\n",
+    "    model=\"mistral-7b\",\n",
+    "    predibase_api_key=os.environ.get(\"PREDIBASE_API_TOKEN\"),\n",
+    ").with_config({\"callbacks\": [opik_tracer]})\n",
+    "```"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## SequentialChain\n",
+    "\n",
+    "Now, let's create a more complex chain and run it with Comet tracing."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain.chains import LLMChain, SimpleSequentialChain\n",
+    "from langchain_core.prompts import PromptTemplate\n",
+    "\n",
+    "# Synopsis chain\n",
+    "template = \"\"\"You are a playwright. Given the title of play, it is your job to write a synopsis for that title.\n",
+    "\n",
+    "Title: {title}\n",
+    "Playwright: This is a synopsis for the above play:\"\"\"\n",
+    "prompt_template = PromptTemplate(input_variables=[\"title\"], template=template)\n",
+    "synopsis_chain = LLMChain(llm=model, prompt=prompt_template)\n",
+    "\n",
+    "# Review chain\n",
+    "template = \"\"\"You are a play critic from the New York Times. Given the synopsis of play, it is your job to write a review for that play.\n",
+    "\n",
+    "Play Synopsis:\n",
+    "{synopsis}\n",
+    "Review from a New York Times play critic of the above play:\"\"\"\n",
+    "prompt_template = PromptTemplate(input_variables=[\"synopsis\"], template=template)\n",
+    "review_chain = LLMChain(llm=model, prompt=prompt_template)\n",
+    "\n",
+    "# Overall chain\n",
+    "overall_chain = SimpleSequentialChain(\n",
+    "    chains=[synopsis_chain, review_chain], verbose=True\n",
+    ")\n",
+    "\n",
+    "# Run the chain with Comet tracing\n",
+    "review = overall_chain.run(\"Tragedy at sunset on the beach\", callbacks=[opik_tracer])\n",
+    "print(review)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Accessing Logged Traces\n",
+    "\n",
+    "We can access the trace IDs collected by the Comet tracer."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "traces = opik_tracer.created_traces()\n",
+    "print(\"Collected trace IDs:\", [trace.id for trace in traces])\n",
+    "\n",
+    "# Flush traces to ensure all data is logged\n",
+    "opik_tracer.flush()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Fine-tuned LLM Example\n",
+    "\n",
+    "Finally, let's use a fine-tuned model with Comet tracing.\n",
+    "\n",
+    "**Note:** In order to use a fine-tuned model, you will need to have access to the model and the correct model ID. The code below will return a `NotFoundError` unless the `model` and `adapter_id` are updated."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "fine_tuned_model = Predibase(\n",
+    "    model=\"my-base-LLM\",\n",
+    "    predibase_api_key=os.environ.get(\"PREDIBASE_API_TOKEN\"),\n",
+    "    predibase_sdk_version=None,\n",
+    "    adapter_id=\"my-finetuned-adapter-id\",\n",
+    "    adapter_version=1,\n",
+    "    **{\n",
+    "        \"api_token\": os.environ.get(\"HUGGING_FACE_HUB_TOKEN\"),\n",
+    "        \"max_new_tokens\": 5,\n",
+    "    },\n",
+    ")\n",
+    "\n",
+    "# Configure the Comet tracer\n",
+    "fine_tuned_model = fine_tuned_model.with_config({\"callbacks\": [opik_tracer]})\n",
+    "\n",
+    "# Invode the fine-tuned model\n",
+    "response = fine_tuned_model.invoke(\n",
+    "    \"Can you help categorize the following emails into positive, negative, and neutral?\",\n",
+    "    **{\"temperature\": 0.5, \"max_new_tokens\": 1024}\n",
+    ")\n",
+    "print(response)\n",
+    "\n",
+    "# Final flush to ensure all traces are logged\n",
+    "opik_tracer.flush()"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "py312_llm_eval",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.12.4"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}