diff --git a/notebook/agentchat_openlit.ipynb b/notebook/agentchat_openlit.ipynb new file mode 100644 index 0000000000..ba0838a28b --- /dev/null +++ b/notebook/agentchat_openlit.ipynb @@ -0,0 +1,454 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": { + "id": "DRFDEL14XTiN" + }, + "source": [ + "# Agent Observability with OpenLIT\n" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "PqhddfHZXgVH" + }, + "source": [ + "\n", + "\n", + "\"OpenLIT" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "dmNFkiaGYXIv" + }, + "source": [ + "[OpenLIT](https://github.com/openlit/openlit) an open source product that helps developers build and manage AI agents in production, effectively helping them improve accuracy. As a self-hosted solution, it enables developers to experiment with LLMs, manage and version prompts, securely manage API keys, and provide safeguards against prompt injection and jailbreak attempts. It also includes built-in OpenTelemetry-native observability and evaluation for the complete GenAI stack (LLMs, Agents, vector databases, and GPUs).\n", + "\n", + "For more info, check out the [OpenLIT Repo](https://github.com/openlit/openlit)\n", + "\n", + "![](https://github.com/openlit/.github/blob/main/profile/assets/openlit-client-1.png?raw=true)\n", + "![](https://github.com/openlit/.github/blob/main/profile/assets/openlit-client-2.png?raw=true)\n" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "hePjFDc6Yu6G" + }, + "source": [ + "## Adding OpenLIT to an existing AutoGen (Now AG2) service\n", + "To get started, you'll need to install the OpenLIT library\n", + "\n", + "OpenLIT uses OpenTelemetry to automatically intrument the AI Agent app when it's initialized meaning your agent observability data like execution traces and metrics will be tracked in just one line of code." + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "metadata": { + "collapsed": true, + "id": "kyb8oFmCZdg8", + "colab": { + "base_uri": "https://localhost:8080/" + }, + "outputId": "3e34e1ef-039b-4d38-f66b-67f145cc5a24" + }, + "outputs": [ + { + "output_type": "stream", + "name": "stdout", + "text": [ + "Collecting ag2\n", + " Downloading ag2-0.4-py3-none-any.whl.metadata (24 kB)\n", + "Collecting openlit\n", + " Downloading openlit-1.32.4-py3-none-any.whl.metadata (22 kB)\n", + "Collecting diskcache (from ag2)\n", + " Downloading diskcache-5.6.3-py3-none-any.whl.metadata (20 kB)\n", + "Collecting docker (from ag2)\n", + " Downloading docker-7.1.0-py3-none-any.whl.metadata (3.8 kB)\n", + "Collecting flaml (from ag2)\n", + " Downloading FLAML-2.3.2-py3-none-any.whl.metadata (16 kB)\n", + "Requirement already satisfied: openai>=1.3 in /usr/local/lib/python3.10/dist-packages (from ag2) (1.54.4)\n", + "Requirement already satisfied: packaging in /usr/local/lib/python3.10/dist-packages (from ag2) (24.2)\n", + "Requirement already satisfied: pydantic!=2.6.0,<3,>=1.10 in /usr/local/lib/python3.10/dist-packages (from ag2) (2.9.2)\n", + "Collecting python-dotenv (from ag2)\n", + " Downloading python_dotenv-1.0.1-py3-none-any.whl.metadata (23 kB)\n", + "Requirement already satisfied: termcolor in /usr/local/lib/python3.10/dist-packages (from ag2) (2.5.0)\n", + "Collecting tiktoken (from ag2)\n", + " Downloading tiktoken-0.8.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (6.6 kB)\n", + "Requirement already satisfied: numpy<2.0.0,>=1.24.0 in /usr/local/lib/python3.10/dist-packages (from ag2) (1.26.4)\n", + "Collecting anthropic<0.22.0,>=0.21.0 (from openlit)\n", + " Downloading anthropic-0.21.3-py3-none-any.whl.metadata (17 kB)\n", + "Collecting boto3<2.0.0,>=1.34.0 (from openlit)\n", + " Downloading boto3-1.35.69-py3-none-any.whl.metadata (6.7 kB)\n", + "Collecting botocore<2.0.0,>=1.34.0 (from openlit)\n", + " Downloading botocore-1.35.69-py3-none-any.whl.metadata (5.7 kB)\n", + "Requirement already satisfied: opentelemetry-api<2.0.0,>=1.27.0 in /usr/local/lib/python3.10/dist-packages (from openlit) (1.28.2)\n", + "Collecting opentelemetry-exporter-otlp<2.0.0,>=1.27.0 (from openlit)\n", + " Downloading opentelemetry_exporter_otlp-1.28.2-py3-none-any.whl.metadata (2.3 kB)\n", + "Collecting opentelemetry-instrumentation<0.49,>=0.48b0 (from openlit)\n", + " Downloading opentelemetry_instrumentation-0.48b0-py3-none-any.whl.metadata (6.1 kB)\n", + "Requirement already satisfied: opentelemetry-sdk<2.0.0,>=1.27.0 in /usr/local/lib/python3.10/dist-packages (from openlit) (1.28.2)\n", + "Requirement already satisfied: requests<3.0.0,>=2.26.0 in /usr/local/lib/python3.10/dist-packages (from openlit) (2.32.3)\n", + "Collecting schedule<2.0.0,>=1.2.2 (from openlit)\n", + " Downloading schedule-1.2.2-py3-none-any.whl.metadata (3.8 kB)\n", + "Collecting tiktoken (from ag2)\n", + " Downloading tiktoken-0.7.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (6.6 kB)\n", + "Collecting xmltodict<0.14.0,>=0.13.0 (from openlit)\n", + " Downloading xmltodict-0.13.0-py2.py3-none-any.whl.metadata (7.7 kB)\n", + "Requirement already satisfied: anyio<5,>=3.5.0 in /usr/local/lib/python3.10/dist-packages (from anthropic<0.22.0,>=0.21.0->openlit) (3.7.1)\n", + "Requirement already satisfied: distro<2,>=1.7.0 in /usr/local/lib/python3.10/dist-packages (from anthropic<0.22.0,>=0.21.0->openlit) (1.9.0)\n", + "Requirement already satisfied: httpx<1,>=0.23.0 in /usr/local/lib/python3.10/dist-packages (from anthropic<0.22.0,>=0.21.0->openlit) (0.27.2)\n", + "Requirement already satisfied: sniffio in /usr/local/lib/python3.10/dist-packages (from anthropic<0.22.0,>=0.21.0->openlit) (1.3.1)\n", + "Requirement already satisfied: tokenizers>=0.13.0 in /usr/local/lib/python3.10/dist-packages (from anthropic<0.22.0,>=0.21.0->openlit) (0.20.3)\n", + "Requirement already satisfied: typing-extensions<5,>=4.7 in /usr/local/lib/python3.10/dist-packages (from anthropic<0.22.0,>=0.21.0->openlit) (4.12.2)\n", + "Collecting jmespath<2.0.0,>=0.7.1 (from boto3<2.0.0,>=1.34.0->openlit)\n", + " Downloading jmespath-1.0.1-py3-none-any.whl.metadata (7.6 kB)\n", + "Collecting s3transfer<0.11.0,>=0.10.0 (from boto3<2.0.0,>=1.34.0->openlit)\n", + " Downloading s3transfer-0.10.4-py3-none-any.whl.metadata (1.7 kB)\n", + "Requirement already satisfied: python-dateutil<3.0.0,>=2.1 in /usr/local/lib/python3.10/dist-packages (from botocore<2.0.0,>=1.34.0->openlit) (2.8.2)\n", + "Requirement already satisfied: urllib3!=2.2.0,<3,>=1.25.4 in /usr/local/lib/python3.10/dist-packages (from botocore<2.0.0,>=1.34.0->openlit) (2.2.3)\n", + "Requirement already satisfied: jiter<1,>=0.4.0 in /usr/local/lib/python3.10/dist-packages (from openai>=1.3->ag2) (0.7.1)\n", + "Requirement already satisfied: tqdm>4 in /usr/local/lib/python3.10/dist-packages (from openai>=1.3->ag2) (4.66.6)\n", + "Requirement already satisfied: deprecated>=1.2.6 in /usr/local/lib/python3.10/dist-packages (from opentelemetry-api<2.0.0,>=1.27.0->openlit) (1.2.15)\n", + "Requirement already satisfied: importlib-metadata<=8.5.0,>=6.0 in /usr/local/lib/python3.10/dist-packages (from opentelemetry-api<2.0.0,>=1.27.0->openlit) (8.5.0)\n", + "Collecting opentelemetry-exporter-otlp-proto-grpc==1.28.2 (from opentelemetry-exporter-otlp<2.0.0,>=1.27.0->openlit)\n", + " Downloading opentelemetry_exporter_otlp_proto_grpc-1.28.2-py3-none-any.whl.metadata (2.2 kB)\n", + "Collecting opentelemetry-exporter-otlp-proto-http==1.28.2 (from opentelemetry-exporter-otlp<2.0.0,>=1.27.0->openlit)\n", + " Downloading opentelemetry_exporter_otlp_proto_http-1.28.2-py3-none-any.whl.metadata (2.2 kB)\n", + "Requirement already satisfied: googleapis-common-protos~=1.52 in /usr/local/lib/python3.10/dist-packages (from opentelemetry-exporter-otlp-proto-grpc==1.28.2->opentelemetry-exporter-otlp<2.0.0,>=1.27.0->openlit) (1.66.0)\n", + "Requirement already satisfied: grpcio<2.0.0,>=1.63.2 in /usr/local/lib/python3.10/dist-packages (from opentelemetry-exporter-otlp-proto-grpc==1.28.2->opentelemetry-exporter-otlp<2.0.0,>=1.27.0->openlit) (1.68.0)\n", + "Collecting opentelemetry-exporter-otlp-proto-common==1.28.2 (from opentelemetry-exporter-otlp-proto-grpc==1.28.2->opentelemetry-exporter-otlp<2.0.0,>=1.27.0->openlit)\n", + " Downloading opentelemetry_exporter_otlp_proto_common-1.28.2-py3-none-any.whl.metadata (1.8 kB)\n", + "Collecting opentelemetry-proto==1.28.2 (from opentelemetry-exporter-otlp-proto-grpc==1.28.2->opentelemetry-exporter-otlp<2.0.0,>=1.27.0->openlit)\n", + " Downloading opentelemetry_proto-1.28.2-py3-none-any.whl.metadata (2.3 kB)\n", + "Collecting protobuf<6.0,>=5.0 (from opentelemetry-proto==1.28.2->opentelemetry-exporter-otlp-proto-grpc==1.28.2->opentelemetry-exporter-otlp<2.0.0,>=1.27.0->openlit)\n", + " Downloading protobuf-5.28.3-cp38-abi3-manylinux2014_x86_64.whl.metadata (592 bytes)\n", + "Requirement already satisfied: setuptools>=16.0 in /usr/local/lib/python3.10/dist-packages (from opentelemetry-instrumentation<0.49,>=0.48b0->openlit) (75.1.0)\n", + "Requirement already satisfied: wrapt<2.0.0,>=1.0.0 in /usr/local/lib/python3.10/dist-packages (from opentelemetry-instrumentation<0.49,>=0.48b0->openlit) (1.16.0)\n", + "Requirement already satisfied: opentelemetry-semantic-conventions==0.49b2 in /usr/local/lib/python3.10/dist-packages (from opentelemetry-sdk<2.0.0,>=1.27.0->openlit) (0.49b2)\n", + "Requirement already satisfied: annotated-types>=0.6.0 in /usr/local/lib/python3.10/dist-packages (from pydantic!=2.6.0,<3,>=1.10->ag2) (0.7.0)\n", + "Requirement already satisfied: pydantic-core==2.23.4 in /usr/local/lib/python3.10/dist-packages (from pydantic!=2.6.0,<3,>=1.10->ag2) (2.23.4)\n", + "Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests<3.0.0,>=2.26.0->openlit) (3.4.0)\n", + "Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests<3.0.0,>=2.26.0->openlit) (3.10)\n", + "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests<3.0.0,>=2.26.0->openlit) (2024.8.30)\n", + "Requirement already satisfied: regex>=2022.1.18 in /usr/local/lib/python3.10/dist-packages (from tiktoken->ag2) (2024.9.11)\n", + "Requirement already satisfied: exceptiongroup in /usr/local/lib/python3.10/dist-packages (from anyio<5,>=3.5.0->anthropic<0.22.0,>=0.21.0->openlit) (1.2.2)\n", + "Requirement already satisfied: httpcore==1.* in /usr/local/lib/python3.10/dist-packages (from httpx<1,>=0.23.0->anthropic<0.22.0,>=0.21.0->openlit) (1.0.7)\n", + "Requirement already satisfied: h11<0.15,>=0.13 in /usr/local/lib/python3.10/dist-packages (from httpcore==1.*->httpx<1,>=0.23.0->anthropic<0.22.0,>=0.21.0->openlit) (0.14.0)\n", + "Requirement already satisfied: zipp>=3.20 in /usr/local/lib/python3.10/dist-packages (from importlib-metadata<=8.5.0,>=6.0->opentelemetry-api<2.0.0,>=1.27.0->openlit) (3.21.0)\n", + "Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.10/dist-packages (from python-dateutil<3.0.0,>=2.1->botocore<2.0.0,>=1.34.0->openlit) (1.16.0)\n", + "Requirement already satisfied: huggingface-hub<1.0,>=0.16.4 in /usr/local/lib/python3.10/dist-packages (from tokenizers>=0.13.0->anthropic<0.22.0,>=0.21.0->openlit) (0.26.2)\n", + "Requirement already satisfied: filelock in /usr/local/lib/python3.10/dist-packages (from huggingface-hub<1.0,>=0.16.4->tokenizers>=0.13.0->anthropic<0.22.0,>=0.21.0->openlit) (3.16.1)\n", + "Requirement already satisfied: fsspec>=2023.5.0 in /usr/local/lib/python3.10/dist-packages (from huggingface-hub<1.0,>=0.16.4->tokenizers>=0.13.0->anthropic<0.22.0,>=0.21.0->openlit) (2024.10.0)\n", + "Requirement already satisfied: pyyaml>=5.1 in /usr/local/lib/python3.10/dist-packages (from huggingface-hub<1.0,>=0.16.4->tokenizers>=0.13.0->anthropic<0.22.0,>=0.21.0->openlit) (6.0.2)\n", + "Downloading ag2-0.4-py3-none-any.whl (366 kB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m366.1/366.1 kB\u001b[0m \u001b[31m8.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hDownloading openlit-1.32.4-py3-none-any.whl (229 kB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m229.1/229.1 kB\u001b[0m \u001b[31m15.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hDownloading anthropic-0.21.3-py3-none-any.whl (851 kB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m851.6/851.6 kB\u001b[0m \u001b[31m8.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hDownloading boto3-1.35.69-py3-none-any.whl (139 kB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m139.2/139.2 kB\u001b[0m \u001b[31m9.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hDownloading botocore-1.35.69-py3-none-any.whl (13.0 MB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m13.0/13.0 MB\u001b[0m \u001b[31m57.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hDownloading opentelemetry_exporter_otlp-1.28.2-py3-none-any.whl (7.0 kB)\n", + "Downloading opentelemetry_exporter_otlp_proto_grpc-1.28.2-py3-none-any.whl (18 kB)\n", + "Downloading opentelemetry_exporter_otlp_proto_http-1.28.2-py3-none-any.whl (17 kB)\n", + "Downloading opentelemetry_exporter_otlp_proto_common-1.28.2-py3-none-any.whl (18 kB)\n", + "Downloading opentelemetry_proto-1.28.2-py3-none-any.whl (55 kB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m55.8/55.8 kB\u001b[0m \u001b[31m4.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hDownloading opentelemetry_instrumentation-0.48b0-py3-none-any.whl (29 kB)\n", + "Downloading schedule-1.2.2-py3-none-any.whl (12 kB)\n", + "Downloading tiktoken-0.7.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.1 MB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.1/1.1 MB\u001b[0m \u001b[31m42.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hDownloading xmltodict-0.13.0-py2.py3-none-any.whl (10.0 kB)\n", + "Downloading diskcache-5.6.3-py3-none-any.whl (45 kB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m45.5/45.5 kB\u001b[0m \u001b[31m3.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hDownloading docker-7.1.0-py3-none-any.whl (147 kB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m147.8/147.8 kB\u001b[0m \u001b[31m8.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hDownloading FLAML-2.3.2-py3-none-any.whl (313 kB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m313.9/313.9 kB\u001b[0m \u001b[31m19.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hDownloading python_dotenv-1.0.1-py3-none-any.whl (19 kB)\n", + "Downloading jmespath-1.0.1-py3-none-any.whl (20 kB)\n", + "Downloading s3transfer-0.10.4-py3-none-any.whl (83 kB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m83.2/83.2 kB\u001b[0m \u001b[31m6.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hDownloading protobuf-5.28.3-cp38-abi3-manylinux2014_x86_64.whl (316 kB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m316.6/316.6 kB\u001b[0m \u001b[31m21.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hInstalling collected packages: xmltodict, schedule, python-dotenv, protobuf, jmespath, flaml, diskcache, tiktoken, opentelemetry-proto, docker, botocore, s3transfer, opentelemetry-instrumentation, opentelemetry-exporter-otlp-proto-common, boto3, anthropic, ag2, opentelemetry-exporter-otlp-proto-http, opentelemetry-exporter-otlp-proto-grpc, opentelemetry-exporter-otlp, openlit\n", + " Attempting uninstall: protobuf\n", + " Found existing installation: protobuf 4.25.5\n", + " Uninstalling protobuf-4.25.5:\n", + " Successfully uninstalled protobuf-4.25.5\n", + "\u001b[31mERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.\n", + "tensorflow 2.17.1 requires protobuf!=4.21.0,!=4.21.1,!=4.21.2,!=4.21.3,!=4.21.4,!=4.21.5,<5.0.0dev,>=3.20.3, but you have protobuf 5.28.3 which is incompatible.\n", + "tensorflow-metadata 1.13.1 requires protobuf<5,>=3.20.3, but you have protobuf 5.28.3 which is incompatible.\u001b[0m\u001b[31m\n", + "\u001b[0mSuccessfully installed ag2-0.4 anthropic-0.21.3 boto3-1.35.69 botocore-1.35.69 diskcache-5.6.3 docker-7.1.0 flaml-2.3.2 jmespath-1.0.1 openlit-1.32.4 opentelemetry-exporter-otlp-1.28.2 opentelemetry-exporter-otlp-proto-common-1.28.2 opentelemetry-exporter-otlp-proto-grpc-1.28.2 opentelemetry-exporter-otlp-proto-http-1.28.2 opentelemetry-instrumentation-0.48b0 opentelemetry-proto-1.28.2 protobuf-5.28.3 python-dotenv-1.0.1 s3transfer-0.10.4 schedule-1.2.2 tiktoken-0.7.0 xmltodict-0.13.0\n" + ] + } + ], + "source": [ + "pip install ag2 openlit" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "pxawniRGYyj2" + }, + "outputs": [], + "source": [ + "import openlit\n", + "\n", + "from autogen import AssistantAgent, UserProxyAgent\n", + "\n", + "openlit.init()" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "hK4KL2JLZwLg" + }, + "source": [ + "OpenLIT will now start automatically tracking\n", + "\n", + "- LLM prompts and completions\n", + "- Token usage and costs\n", + "- Agent names and actions\n", + "- Tool usage\n", + "- Errors\n", + "\n", + "\n", + "## Lets look at a simple chat example\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "Z_5QJzEqa7I0" + }, + "outputs": [], + "source": [ + "import openlit\n", + "\n", + "openlit.init()" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "2CKZPle2aal6", + "outputId": "7f3346e9-7e5b-4c47-f111-32f8d06313a0" + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "user_proxy (to assistant):\n", + "\n", + "Tell me a joke about NVDA and TESLA stock prices.\n", + "\n", + "--------------------------------------------------------------------------------\n", + "assistant (to user_proxy):\n", + "\n", + "Why don't NVDA and TESLA stock prices ever get coffee together?\n", + "\n", + "Because whenever they heat up, they always take a steep drop before they can cool down! \n", + "\n", + "I hope this brings a smile to your face. Investing in stocks can be a rollercoaster sometimes. Please note that this is humor and doesn't reflect the actual dynamics of these companies' stock prices. TERMINATE.\n", + "\n", + "--------------------------------------------------------------------------------\n", + "Replying as user_proxy. Provide feedback to assistant. Press enter to skip and use auto-reply, or type 'exit' to end the conversation: hi\n", + "user_proxy (to assistant):\n", + "\n", + "hi\n", + "\n", + "--------------------------------------------------------------------------------\n", + "{\n", + " \"name\": \"openai.chat.completions\",\n", + " \"context\": {\n", + " \"trace_id\": \"0x35ae5952626492e432ae25af5bf92daa\",\n", + " \"span_id\": \"0x44103383aa51d1b1\",\n", + " \"trace_state\": \"[]\"\n", + " },\n", + " \"kind\": \"SpanKind.CLIENT\",\n", + " \"parent_id\": null,\n", + " \"start_time\": \"2024-11-21T01:53:47.597241Z\",\n", + " \"end_time\": \"2024-11-21T01:53:48.506758Z\",\n", + " \"status\": {\n", + " \"status_code\": \"OK\"\n", + " },\n", + " \"attributes\": {\n", + " \"telemetry.sdk.name\": \"openlit\",\n", + " \"gen_ai.system\": \"openai\",\n", + " \"gen_ai.operation.name\": \"chat\",\n", + " \"gen_ai.endpoint\": \"openai.chat.completions\",\n", + " \"gen_ai.response.id\": \"chatcmpl-AVqg7QSYB8CpEN1I5PYT1laBcds9S\",\n", + " \"gen_ai.environment\": \"default\",\n", + " \"gen_ai.application_name\": \"default\",\n", + " \"gen_ai.request.model\": \"gpt-4\",\n", + " \"gen_ai.request.top_p\": 1.0,\n", + " \"gen_ai.request.max_tokens\": -1,\n", + " \"gen_ai.request.user\": \"\",\n", + " \"gen_ai.request.temperature\": 1.0,\n", + " \"gen_ai.request.presence_penalty\": 0.0,\n", + " \"gen_ai.request.frequency_penalty\": 0.0,\n", + " \"gen_ai.request.seed\": \"\",\n", + " \"gen_ai.request.is_stream\": false,\n", + " \"gen_ai.usage.input_tokens\": 580,\n", + " \"gen_ai.usage.output_tokens\": 9,\n", + " \"gen_ai.usage.total_tokens\": 589,\n", + " \"gen_ai.response.finish_reasons\": [\n", + " \"stop\"\n", + " ],\n", + " \"gen_ai.usage.cost\": 0.017939999999999998\n", + " },\n", + " \"events\": [\n", + " {\n", + " \"name\": \"gen_ai.content.prompt\",\n", + " \"timestamp\": \"2024-11-21T01:53:48.506257Z\",\n", + " \"attributes\": {\n", + " \"gen_ai.prompt\": \"system: You are a helpful AI assistant.\\nSolve tasks using your coding and language skills.\\nIn the following cases, suggest python code (in a python coding block) or shell script (in a sh coding block) for the user to execute.\\n 1. When you need to collect info, use the code to output the info you need, for example, browse or search the web, download/read a file, print the content of a webpage or a file, get the current date/time, check the operating system. After sufficient info is printed and the task is ready to be solved based on your language skill, you can solve the task by yourself.\\n 2. When you need to perform some task with code, use the code to perform the task and output the result. Finish the task smartly.\\nSolve the task step by step if you need to. If a plan is not provided, explain your plan first. Be clear which step uses code, and which step uses your language skill.\\nWhen using code, you must indicate the script type in the code block. The user cannot provide any other feedback or perform any other action beyond executing the code you suggest. The user can't modify your code. So do not suggest incomplete code which requires users to modify. Don't use a code block if it's not intended to be executed by the user.\\nIf you want the user to save the code in a file before executing it, put # filename: inside the code block as the first line. Don't include multiple code blocks in one response. Do not ask users to copy and paste the result. Instead, use 'print' function for the output when relevant. Check the execution result returned by the user.\\nIf the result indicates there is an error, fix the error and output the code again. Suggest the full code instead of partial code or code changes. If the error can't be fixed or if the task is not solved even after the code is executed successfully, analyze the problem, revisit your assumption, collect additional info you need, and think of a different approach to try.\\nWhen you find an answer, verify the answer carefully. Include verifiable evidence in your response if possible.\\nReply \\\"TERMINATE\\\" in the end when everything is done.\\n \\nuser: Tell me a joke about NVDA and TESLA stock prices.\\nassistant: Why don't NVDA and TESLA stock prices ever get coffee together?\\n\\nBecause whenever they heat up, they always take a steep drop before they can cool down! \\n\\nI hope this brings a smile to your face. Investing in stocks can be a rollercoaster sometimes. Please note that this is humor and doesn't reflect the actual dynamics of these companies' stock prices. TERMINATE.\\nuser: hi\"\n", + " }\n", + " },\n", + " {\n", + " \"name\": \"gen_ai.content.completion\",\n", + " \"timestamp\": \"2024-11-21T01:53:48.506314Z\",\n", + " \"attributes\": {\n", + " \"gen_ai.completion\": \"Hello! How can I assist you today?\"\n", + " }\n", + " }\n", + " ],\n", + " \"links\": [],\n", + " \"resource\": {\n", + " \"attributes\": {\n", + " \"telemetry.sdk.language\": \"python\",\n", + " \"telemetry.sdk.name\": \"openlit\",\n", + " \"telemetry.sdk.version\": \"1.28.2\",\n", + " \"service.name\": \"default\",\n", + " \"deployment.environment\": \"default\"\n", + " },\n", + " \"schema_url\": \"\"\n", + " }\n", + "}\n", + "assistant (to user_proxy):\n", + "\n", + "Hello! How can I assist you today?\n", + "\n", + "--------------------------------------------------------------------------------\n", + "Replying as user_proxy. Provide feedback to assistant. Press enter to skip and use auto-reply, or type 'exit' to end the conversation: exit\n" + ] + }, + { + "data": { + "text/plain": [ + "ChatResult(chat_id=None, chat_history=[{'content': 'Tell me a joke about NVDA and TESLA stock prices.', 'role': 'assistant', 'name': 'user_proxy'}, {'content': \"Why don't NVDA and TESLA stock prices ever get coffee together?\\n\\nBecause whenever they heat up, they always take a steep drop before they can cool down! \\n\\nI hope this brings a smile to your face. Investing in stocks can be a rollercoaster sometimes. Please note that this is humor and doesn't reflect the actual dynamics of these companies' stock prices. TERMINATE.\", 'role': 'user', 'name': 'assistant'}, {'content': 'hi', 'role': 'assistant', 'name': 'user_proxy'}, {'content': 'Hello! How can I assist you today?', 'role': 'user', 'name': 'assistant'}], summary='Hello! How can I assist you today?', cost={'usage_including_cached_inference': {'total_cost': 0.03731999999999999, 'gpt-4-0613': {'cost': 0.03731999999999999, 'prompt_tokens': 1068, 'completion_tokens': 88, 'total_tokens': 1156}}, 'usage_excluding_cached_inference': {'total_cost': 0.017939999999999998, 'gpt-4-0613': {'cost': 0.017939999999999998, 'prompt_tokens': 580, 'completion_tokens': 9, 'total_tokens': 589}}}, human_input=['hi', 'exit'])" + ] + }, + "execution_count": 3, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "import os\n", + "\n", + "llm_config = {\"model\": \"gpt-4\", \"api_key\": os.environ[\"OPENAI_API_KEY\"]}\n", + "assistant = AssistantAgent(\"assistant\", llm_config=llm_config)\n", + "user_proxy = UserProxyAgent(\"user_proxy\", code_execution_config=False)\n", + "\n", + "# Start the chat\n", + "user_proxy.initiate_chat(\n", + " assistant,\n", + " message=\"Tell me a joke about NVDA and TESLA stock prices.\",\n", + ")" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "7_gNWze0bYdO" + }, + "source": [ + "# Sending Traces and metrics to OpenLIT\n", + "\n", + "By default, OpenLIT generates OpenTelemetry traces and metrics that are logged to your console. To set up a detailed monitoring environment, this guide outlines how to deploy OpenLIT and direct all traces and metrics there. You also have the flexibility to send the telemetry data to any OpenTelemetry-compatible endpoint, such as Grafana, Jaeger, or DataDog.\n", + "\n", + "## Deploy OpenLIT Stack\n", + "\n", + "1. Clone the OpenLIT Repository\n", + "\n", + " Open your terminal or command line and execute:\n", + "\n", + " ```shell\n", + " git clone git@github.com:openlit/openlit.git\n", + " ```\n", + "\n", + "2. Host it Yourself with Docker\n", + "\n", + " Deploy and start OpenLIT using the command:\n", + "\n", + " ```shell\n", + " docker compose up -d\n", + " ```\n", + "\n", + "> For instructions on installing in Kubernetes using Helm, refer to the [Kubernetes Helm installation guide](https://docs.openlit.io/latest/installation#kubernetes).\n", + "\n", + "Configure the telemetry data destination as follows:\n", + "\n", + "| Purpose | Parameter/Environment Variable | For Sending to OpenLIT |\n", + "|-------------------------------------------|--------------------------------------------------|--------------------------------|\n", + "| Send data to an HTTP OTLP endpoint | `otlp_endpoint` or `OTEL_EXPORTER_OTLP_ENDPOINT` | `\"http://127.0.0.1:4318\"` |\n", + "| Authenticate telemetry backends | `otlp_headers` or `OTEL_EXPORTER_OTLP_HEADERS` | Not required by default |\n", + "\n", + "> 💡 Info: If the `otlp_endpoint` or `OTEL_EXPORTER_OTLP_ENDPOINT` is not provided, the OpenLIT SDK will output traces directly to your console, which is recommended during the development phase.\n", + "\n", + "## Visualize and Optimize!\n", + "\n", + "With the Observability data now being collected and sent to OpenLIT, the next step is to visualize and analyze this data to get insights into your AI application's performance, behavior, and identify areas of improvement.\n", + "\n", + "Just head over to OpenLIT at `127.0.0.1:3000` on your browser to start exploring. You can login using the default credentials\n", + " - **Email**: `user@openlit.io`\n", + " - **Password**: `openlituser`\n", + "\n", + "![](https://github.com/openlit/.github/blob/main/profile/assets/openlit-ag2-1.png?raw=true)\n", + "![](https://github.com/openlit/.github/blob/main/profile/assets/openlit-ag2-2.png?raw=true)" + ] + } + ], + "metadata": { + "colab": { + "provenance": [] + }, + "front_matter": { + "description": "Use OpenLIT to easily monitor AI agents in production with OpenTelemetry.", + "tags": [ + "integration", + "monitoring", + "observability", + "debugging" + ] + }, + "kernelspec": { + "display_name": "Python 3", + "name": "python3" + }, + "language_info": { + "name": "python" + } + }, + "nbformat": 4, + "nbformat_minor": 0 +} diff --git a/test/agentchat/test_async.py b/test/agentchat/test_async.py index 0d04c076c3..a40d4441f4 100755 --- a/test/agentchat/test_async.py +++ b/test/agentchat/test_async.py @@ -23,8 +23,8 @@ def get_market_news(ind, ind_upper): data = { "feed": [ { - "title": "Palantir CEO Says Our Generation's Atomic Bomb Could Be AI Weapon - And Arrive Sooner Than You Think - Palantir Technologies ( NYSE:PLTR ) ", - "summary": "Christopher Nolan's blockbuster movie \"Oppenheimer\" has reignited the public discourse surrounding the United States' use of an atomic bomb on Japan at the end of World War II.", + "title": "Palantir CEO Says Our Generation's Main Challenge Could Be AI Against Humanity - And Arrive Sooner Than You Think - Palantir Technologies ( NYSE:PLTR ) ", + "summary": "Christopher Nolan's blockbuster movie \"Oppenheimer\" has reignited the public discourse surrounding the United States' use of a weapon on Japan at the end of World War II.", "overall_sentiment_score": 0.009687, }, { @@ -63,7 +63,7 @@ def get_market_news(ind, ind_upper): @pytest.mark.skipif(skip_openai, reason=reason) @pytest.mark.asyncio async def test_async_groupchat(): - config_list = autogen.config_list_from_json(OAI_CONFIG_LIST, KEY_LOC, filter_dict={"tags": ["gpt-3.5-turbo"]}) + config_list = autogen.config_list_from_json(OAI_CONFIG_LIST, KEY_LOC, filter_dict={"tags": ["gpt-4o-mini"]}) # create an AssistantAgent instance named "assistant" assistant = autogen.AssistantAgent( @@ -97,7 +97,7 @@ async def test_async_groupchat(): @pytest.mark.skipif(skip_openai, reason=reason) @pytest.mark.asyncio async def test_stream(): - config_list = autogen.config_list_from_json(OAI_CONFIG_LIST, KEY_LOC, filter_dict={"tags": ["gpt-3.5-turbo"]}) + config_list = autogen.config_list_from_json(OAI_CONFIG_LIST, KEY_LOC, filter_dict={"tags": ["gpt-4o-mini"]}) data = asyncio.Future() async def add_stock_price_data(): diff --git a/website/docs/topics/llm-observability.md b/website/docs/topics/llm-observability.md index f80b55ea09..dd0e2cc57b 100644 --- a/website/docs/topics/llm-observability.md +++ b/website/docs/topics/llm-observability.md @@ -8,7 +8,8 @@ AutoGen supports advanced LLM agent observability and monitoring through built-i AutoGen's SQLite and File Logger - [Tutorial Notebook](/docs/notebooks/agentchat_logging) ### Full-Service Partner Integrations -AutoGen partners with [AgentOps](https://agentops.ai) to provide multi-agent tracking, metrics, and monitoring - [Tutorial Notebook](/docs/notebooks/agentchat_agentops) +- AutoGen partners with [AgentOps](https://agentops.ai) to provide multi-agent tracking, metrics, and monitoring - [Tutorial Notebook](/docs/notebooks/agentchat_agentops) +- AutoGen partners with [OpenLIT](https://github.com/openlit/openlit) to provide OpenTelemetry-native agent observability, which includes full execution traces and metrics - [Tutorial Notebook](https://github.com/ag2ai/ag2/blob/main/notebook/agentchat_openlit.ipynb) ## What is Observability? diff --git a/website/docusaurus.config.js b/website/docusaurus.config.js index 28f9cb940f..c7d85eeeaa 100644 --- a/website/docusaurus.config.js +++ b/website/docusaurus.config.js @@ -24,7 +24,7 @@ module.exports = { projectName: "ag2", // Usually your repo name. scripts: [ { - src: "/autogen/js/custom.js", + src: "/js/custom.js", async: true, defer: true, }, @@ -247,7 +247,7 @@ module.exports = { routeBasePath: 'talks', path: './talks', showReadingTime: true, - postsPerPage: 'ALL', + blogSidebarCount: "ALL", }, ], [ diff --git a/website/src/components/GalleryPage.js b/website/src/components/GalleryPage.js index ca6e4f4b7f..11406aa29c 100644 --- a/website/src/components/GalleryPage.js +++ b/website/src/components/GalleryPage.js @@ -70,8 +70,8 @@ const GalleryPage = (props) => { item.image ? item.image.includes("http") ? item.image - : `/autogen/img/gallery/${item.image}` - : `/autogen/img/gallery/default.png` + : `../../static/img/gallery/${item.image}` + : `../../static/img/gallery/default.png` } style={{ height: 150, diff --git a/website/talks/2024-11-25/index.mdx b/website/talks/2024-11-25/index.mdx new file mode 100644 index 0000000000..a524600a32 --- /dev/null +++ b/website/talks/2024-11-25/index.mdx @@ -0,0 +1,13 @@ +--- +title: Investigating Group Decision-Making Mechanism in Decentralized Multi-Agent Collaboration - Nov 25, 2024 +--- + +### Speakers: Young-Min Cho + +### Biography of the speakers: + +Young-Min Cho (known as Jeffrey) is a Ph.D. student in Computer and Information Science at the University of Pennsylvania, specializing in Natural Language Processing and Artificial Intelligence. As a member of Penn NLP, WWBP, and CSL, he is advised by Dr. Lyle Ungar and Dr. Sharath Chandra Guntuku. His research focuses on controlling conversations in Multi-Agent Systems and Conversational Agents, with an emphasis on achieving consensus in agent collaboration, managing conversational behavior traits, and generating effective clarifying questions. He is also exploring social and psychological insights through language-based assessments, including applications such as mental health chatbots and analyses of cultural differences in emotional expressions. + +### Abstract: + +This study investigates the efficacy of Multi-Agent Systems in eliciting cross-agent communication and enhancing collective intelligence through group decision-making in a decentralized setting. Unlike centralized mechanisms, where a fixed hierarchy governs social choice, decentralized group decision-making allows agents to engage in joint deliberation. Our research focuses on the dynamics of communication and decision-making within various social choice methods. By applying different voting rules in various environments, we find that moderate decision flexibility yields better outcomes. Additionally, exploring the linguistic features of agent-to-agent conversations reveals indicators of effective collaboration, offering insights into communication patterns that facilitate or hinder collaboration. Finally, we propose various methods for determining the optimal stopping point in multi-agent collaborations based on linguistic cues. Our findings contribute to a deeper understanding of how decentralized decision-making and group conversation shape multi-agent collaboration, with implications for the design of more effective MAS environments. diff --git a/website/talks/2024-11-28/index.mdx b/website/talks/2024-11-28/index.mdx new file mode 100644 index 0000000000..956a7a4418 --- /dev/null +++ b/website/talks/2024-11-28/index.mdx @@ -0,0 +1,13 @@ +--- +title: Mosaia - The AI community’s platform for creating, sharing and deploying AI agents in a serverless cloud environment - Nov 28, 2024 +--- + +### Speakers: Aaron Wong-Ellis + +### Biography of the speakers: + +Aaron Wong-Ellis is the co-founder and CTO at Mosaia. His several years of experience in the field of AI, IOT and building enterprise platforms has equipped him with the right skill set to build Mosaia. Aaron has worked as an application architect and engineer for small startups and large Fortune 100 companies like AWS. His recent work focuses on developing a platform for creating, sharing and running LLM agents in a scalable serverless cloud infrastructure. + +### Abstract: + +Running multiple AI agents reliably on the cloud, can encounter numerous challenges. At Mosaia we faced the challenges head on and created a way to do this in a scalable serverless cloud environment. Allowing people to run their agents with little to no code at all. Just write up your prompts and construct your groups of agents through a browser based UI. Being able to do this, opened up many possibilities to construct and share agents with others to use on Mosaia or run locally using Autogen. Mosaia was created as a platform to not only run agents but allow prompt engineers to host and share these agents with others. Fostering a community of collaboration and creativity around building AI agents. diff --git a/website/talks/future_talks/index.mdx b/website/talks/future_talks/index.mdx index 5f5b0d23f3..65dad7c997 100644 --- a/website/talks/future_talks/index.mdx +++ b/website/talks/future_talks/index.mdx @@ -2,21 +2,6 @@ title: Upcoming Talks --- -## Mosaia - The AI community’s platform for creating, sharing and deploying AI agents in a serverless cloud environment - Nov 28, 2024 - -### Speakers: Aaron Wong-Ellis - -### Biography of the speakers: - -Aaron Wong-Ellis is the co-founder and CTO at Mosaia. His several years of experience in the field of AI, IOT and building enterprise platforms has equipped him with the right skill set to build Mosaia. Aaron has worked as an application architect and engineer for small startups and large Fortune 100 companies like AWS. His recent work focuses on developing a platform for creating, sharing and running LLM agents in a scalable serverless cloud infrastructure. - -### Abstract: - -Running multiple AI agents reliably on the cloud, can encounter numerous challenges. At Mosaia we faced the challenges head on and created a way to do this in a scalable serverless cloud environment. Allowing people to run their agents with little to no code at all. Just write up your prompts and construct your groups of agents through a browser based UI. Being able to do this, opened up many possibilities to construct and share agents with others to use on Mosaia or run locally using Autogen. Mosaia was created as a platform to not only run agents but allow prompt engineers to host and share these agents with others. Fostering a community of collaboration and creativity around building AI agents. - -### Sign Up: https://discord.gg/NrNP5ZAx?event=1308232124062503012 - - ## Make AI Agents Collaborate: Drag, Drop, and Orchestrate with Waldiez - Dec 9, 2024 ### Speakers: Panagiotis Kasnesis @@ -45,19 +30,6 @@ In today's fast-paced digital landscape, businesses need tools that go beyond co ### Sign Up: https://discord.gg/NrNP5ZAx?event=1308497335768059974 -## Investigating Group Decision-Making Mechanism in Decentralized Multi-Agent Collaboration - Nov 25 - -### Speakers: Young-Min Cho - -### Biography of the speakers: - -Young-Min Cho (known as Jeffrey) is a Ph.D. student in Computer and Information Science at the University of Pennsylvania, specializing in Natural Language Processing and Artificial Intelligence. As a member of Penn NLP, WWBP, and CSL, he is advised by Dr. Lyle Ungar and Dr. Sharath Chandra Guntuku. His research focuses on controlling conversations in Multi-Agent Systems and Conversational Agents, with an emphasis on achieving consensus in agent collaboration, managing conversational behavior traits, and generating effective clarifying questions. He is also exploring social and psychological insights through language-based assessments, including applications such as mental health chatbots and analyses of cultural differences in emotional expressions. - -### Abstract: - -This study investigates the efficacy of Multi-Agent Systems in eliciting cross-agent communication and enhancing collective intelligence through group decision-making in a decentralized setting. Unlike centralized mechanisms, where a fixed hierarchy governs social choice, decentralized group decision-making allows agents to engage in joint deliberation. Our research focuses on the dynamics of communication and decision-making within various social choice methods. By applying different voting rules in various environments, we find that moderate decision flexibility yields better outcomes. Additionally, exploring the linguistic features of agent-to-agent conversations reveals indicators of effective collaboration, offering insights into communication patterns that facilitate or hinder collaboration. Finally, we propose various methods for determining the optimal stopping point in multi-agent collaborations based on linguistic cues. Our findings contribute to a deeper understanding of how decentralized decision-making and group conversation shape multi-agent collaboration, with implications for the design of more effective MAS environments. - -### Sign Up: https://discord.gg/Fjfp46uG?event=1309713358391349288 ## How to follow up with the latest talks?