From 065a2e463c9162b6885d8c511aecd8e7d7f340c1 Mon Sep 17 00:00:00 2001 From: innovation64 Date: Wed, 27 Mar 2024 21:54:08 +0800 Subject: [PATCH 01/31] docs: update rag_llamaindex_librarian chinese version --- .../zh-CN/rag_llamaindex_librarian.ipynb | 339 ++++++++++++++++++ 1 file changed, 339 insertions(+) create mode 100644 notebooks/zh-CN/rag_llamaindex_librarian.ipynb diff --git a/notebooks/zh-CN/rag_llamaindex_librarian.ipynb b/notebooks/zh-CN/rag_llamaindex_librarian.ipynb new file mode 100644 index 00000000..c7a79bda --- /dev/null +++ b/notebooks/zh-CN/rag_llamaindex_librarian.ipynb @@ -0,0 +1,339 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# 用 LlamaIndex 构建一个 RAG 电子书库智能助手\n", + "\n", + "_作者: [Jonathan Jin](https://huggingface.co/jinnovation)_" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 简介\n", + "\n", + "这个 notebook 展示了如何快速构建一个基于 RAG 的电子图书助手，用于你的本地电子书库。\n", + "就像图书馆的图书管理员帮你找书一样，这个助手也能帮你从你的电子书里找到你需要的书。\n", + "\n", + "## 要求\n", + "这个助手要做得**轻巧**，**尽量在本地运行**，而且**不要用太多其他的东西**。我们会尽量用免费的开源软件，选择那种在**本地普通电脑上，比如 M1 型号的 MacBook 上就能运行的模型**。\n", + "\n", + "## 组件\n", + "我们的解决方案将包括以下组件：\n", + "- [LlamaIndex]，一个基于LLM的应用数据框架，与 [LangChain] 不同，它是专门为 RAG 设计的；\n", + "- [Ollama]，一个简单易用的工具，可以让你在本地运行语言模型，比如Llama 2；\n", + "- [`BAAI/bge-base-en-v1.5`](https://huggingface.co/BAAI/bge-base-en-v1.5) 嵌入模型，它的表现[相当好，并且大小适中](https://huggingface.co/spaces/mteb/leaderboard)；\n", + "- [Llama 2]，我们将通过 [Ollama] 运行它。\n", + "\n", + "[LlamaIndex]: https://docs.llamaindex.ai/en/stable/index.html\n", + "[LangChain]: https://python.langchain.com/docs/get_started/introduction\n", + "[Ollama]: https://ollama.com/\n", + "[Llama 2]: https://ollama.com/library/llama2\n", + "\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 依赖\n", + "\n", + "首先安装依赖" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "%pip install -q \\\n", + " llama-index \\\n", + " EbookLib \\\n", + " html2text \\\n", + " llama-index-embeddings-huggingface \\\n", + " llama-index-llms-ollama" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "!brew install ollama" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 文本库初始化设置\n", + "\n", + "我们接下来要弄个测试用的“书库”。\n", + "\n", + "简单点说，我们的“书库”就是一个放有 `.epub` 格式电子书文件的**文件夹**。这个方法很容易就能扩展到像 Calibre 那种带有个 `metadata.db` 数据库文件的书库。怎么扩展这个问题，我们留给读者自己思考。😇\n", + "\n", + "现在，我们先从[古腾堡计划网站](https://www.gutenberg.org/)下载两本`.epub`格式的电子书放到我们的书库里。\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "!mkdir -p \".test/library/jane-austen\"\n", + "!mkdir -p \".test/library/victor-hugo\"\n", + "!wget https://www.gutenberg.org/ebooks/1342.epub.noimages -O \".test/library/jane-austen/pride-and-prejudice.epub\"\n", + "!wget https://www.gutenberg.org/ebooks/135.epub.noimages -O \".test/library/victor-hugo/les-miserables.epub\"" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 用 LlamaIndex 构建 RAG\n", + "\n", + "使用 LlamaIndex 的 RAG 主要包括以下三个阶段：\n", + "\n", + "1. **加载**，在这个阶段你告诉 LlamaIndex 你的数据在哪里以及如何加载它；\n", + "2. **索引**，在这个阶段你扩充加载的数据以方便查询，例如使用向量嵌入；\n", + "3. **查询**，在这个阶段你配置一个 LLM 作为你索引数据的查询接口。\n", + "\n", + "这个解释只是触及了 LlamaIndex 的皮毛。要想了解更多深入细节，我强烈推荐阅读 LlamaIndex 文档中的[\"高级概念\"页面](https://docs.llamaindex.ai/en/stable/getting_started/concepts.html)。\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### 加载\n", + "\n", + "好的，我们首先从**加载**阶段开始。\n", + "\n", + "之前我说过，LlamaIndex 是专为 RAG 这种混合检索生成模型设计的。这一点从它的[`SimpleDirectoryReader`](https://docs.llamaindex.ai/en/stable/module_guides/loading/simpledirectoryreader.html)功能就可以明显看出，它能**神奇地**免费支持很多种文件类型。对我们来说很方便的是，`.epub`这种电子书格式也是它支持的。\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "from llama_index.core import SimpleDirectoryReader\n", + "\n", + "loader = SimpleDirectoryReader(\n", + " input_dir=\"./.test/\",\n", + " recursive=True,\n", + " required_exts=[\".epub\"],\n", + ")\n", + "\n", + "documents = loader.load_data()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "`SimpleDirectoryReader.load_data()` 将我们的电子书转换成一组 LlamaIndex 可以处理的 [`Document`s](https://docs.llamaindex.ai/en/stable/api/llama_index.core.schema.Document.html)。\n", + "\n", + "这里有一个重要的事情要注意，就是这个阶段的文档**还没有被分块**——这将在索引阶段进行。继续往下看...\n", + "\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### 索引\n", + "\n", + "\n", + "在把数据**加载**进来之后，接下来我们要做的是**建立索引**。这样我们的 RAG 系统就能找到与用户查询相关的信息，然后把这些信息传给语言模型（LLM），以便它能够**增强**回答的内容。同时，这一步也会把文档分成一块一块的。\n", + "\n", + "在 LlamaIndex 中，[`VectorStoreIndex`](https://docs.llamaindex.ai/en/stable/module_guides/indexing/vector_store_index.html) 是用来建立索引的一个“默认”工具。这个工具默认使用一个简单、基于内存的字典来保存索引，但随着你的使用规模扩大，LlamaIndex 还支持\n", + "[多种向量存储解决方案](https://docs.llamaindex.ai/en/stable/module_guides/storing/vector_stores.html)。\n", + "\n", + "\n", + "LlamaIndex 默认的块大小是 1024 个字符，块与块之间有 20 个字符的重叠。如果需要了解更多细节，可以查看 [LlamaIndex 的文档](https://docs.llamaindex.ai/en/stable/optimizing/basic_strategies/basic_strategies.html#chunk-sizes)。\n", + "\n", + "\n", + "我们之前提到过，会用\n", + "[`BAAI/bge-small-en-v1.5`](https://huggingface.co/BAAI/bge-base-en-v1.5) 这个模型来生成文本的向量表示。LlamaIndex 默认使用 OpenAI 的服务（特别是 `gpt-3.5-turbo` 这个模型），但因为我们要的是一个轻量级、能在本地运行的端到端解决方案，所以不想用 OpenAI。\n", + "\n", + "好消息是，LlamaIndex 支持通过 `HuggingFaceEmbedding` 这个类来使用 Hugging Face 上的模型，所以我们这儿就打算用这个方法。\n" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": {}, + "outputs": [], + "source": [ + "from llama_index.embeddings.huggingface import HuggingFaceEmbedding\n", + "\n", + "embedding_model = HuggingFaceEmbedding(model_name=\"BAAI/bge-small-en-v1.5\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "我们将把这个模型传递给 `VectorStoreIndex`，作为我们的嵌入模型，以绕过 OpenAI 的默认行为。" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": {}, + "outputs": [], + "source": [ + "from llama_index.core import VectorStoreIndex\n", + "\n", + "index = VectorStoreIndex.from_documents(\n", + " documents,\n", + " embed_model=embedding_model,\n", + ")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### 查询\n", + "\n", + "现在我们要完成 RAG 智能助手的最后一部分——设置查询接口。\n", + "\n", + "在这个教程中，我们将使用 Llama 2 作为语言模型，但我建议你试试不同的模型，看看哪个能给出最好的回答。\n", + "\n", + "首先，我们需要在一个新的终端窗口中启动 Ollama 服务器。不过，[Ollama 的 Python 客户端](https://github.com/ollama/ollama-python)不支持直接启动和关闭服务器，所以我们需要在 Python 环境之外操作。\n", + "\n", + "打开一个新的终端窗口，输入命令：`ollama serve`。等我们这里操作完成后，别忘了关闭服务器！\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "现在，让我们将 Llama 2 连接到 LlamaIndex，并使用它作为我们查询引擎的基础。" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": {}, + "outputs": [], + "source": [ + "from llama_index.llms.ollama import Ollama\n", + "\n", + "llama = Ollama(\n", + " model=\"llama2\",\n", + " request_timeout=40.0,\n", + ")\n", + "\n", + "query_engine = index.as_query_engine(llm=llama)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 最终结果 \n", + "\n", + "有了这些，我们的基本的 RAG 电子书库智能助手就设置好了，我们可以开始询问有关我们电子书库的问题了。例如：\n" + ] + }, + { + "cell_type": "code", + "execution_count": 29, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Based on the context provided, there are two books available:\n", + "\n", + "1. \"Pride and Prejudice\" by Jane Austen\n", + "2. \"Les Misérables\" by Victor Hugo\n", + "\n", + "The context used to derive this answer includes:\n", + "\n", + "* The file path for each book, which provides information about the location of the book files on the computer.\n", + "* The titles of the books, which are mentioned in the context as being available for reading.\n", + "* A list of words associated with each book, such as \"epub\" and \"notebooks\", which provide additional information about the format and storage location of each book.\n" + ] + } + ], + "source": [ + "print(query_engine.query(\"What are the titles of all the books available? Show me the context used to derive your answer.\"))" + ] + }, + { + "cell_type": "code", + "execution_count": 31, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "The main character of 'Pride and Prejudice' is Elizabeth Bennet.\n" + ] + } + ], + "source": [ + "print(query_engine.query(\"Who is the main character of 'Pride and Prejudice'?\"))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 总结和未来可能的提升\n", + "\n", + "\n", + "我们成功地展示了如何创建一个完全在本地运行的基本 RAG 的电子书库智能助手，甚至在苹果的 Apple silicon Macs 上也能运行。在这个过程中，我们还全面了解了 LlamaIndex 是如何帮助我们简化建立基于 RAG 的应用程序的。\n", + "\n", + "尽管如此，我们其实只是接触到了一些皮毛。下面是一些关于如何改进和在这个基础上进一步发展的想法。\n", + "\n", + "### 强制引用\n", + "\n", + "为了防止我们的书库助手胡编乱造，我们怎样才能要求它为其所说的每件事都提供引用呢？\n", + "\n", + "### 使用扩充的元数据\n", + "\n", + "像 [Calibre](https://calibre-ebook.com/) 这样的电子书库管理工具会为电子书创建更多的元数据。这些元数据可以提供一些在书中文本里找不到的信息，比如出版商或版本。我们怎样才能扩展我们的 RAG 流程，使其也能利用那些不是 .epub 文件的额外信息源呢？\n", + "\n", + "\n", + "### 高效索引\n", + "\n", + "如果我们把这里做的所有东西写成一个脚本或可执行程序，那么每次运行这个脚本时，它都会重新索引我们的图书馆。对于只有两个文件的微型测试库来说，这样还行，但对于稍大一点的图书馆来说，每次都重新索引会让用户感到非常烦恼。我们怎样才能让索引持久化，并且只在图书馆内容有重要变化时，比如添加了新书，才去更新索引呢？" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.11.8" + } + }, + "nbformat": 4, + "nbformat_minor": 2 +} From 44851bc113938fdbb847a847809e4ecfe4396157 Mon Sep 17 00:00:00 2001 From: innovation64 Date: Wed, 27 Mar 2024 21:55:17 +0800 Subject: [PATCH 02/31] docs: update rag_with_HF_gemma_Mongodb chinese version --- .../rag_with_hugging_face_gemma_mongodb.ipynb | 4031 +++++++++++++++++ 1 file changed, 4031 insertions(+) create mode 100644 notebooks/zh-CN/rag_with_hugging_face_gemma_mongodb.ipynb diff --git a/notebooks/zh-CN/rag_with_hugging_face_gemma_mongodb.ipynb b/notebooks/zh-CN/rag_with_hugging_face_gemma_mongodb.ipynb new file mode 100644 index 00000000..fde99ec5 --- /dev/null +++ b/notebooks/zh-CN/rag_with_hugging_face_gemma_mongodb.ipynb @@ -0,0 +1,4031 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# 用 Gemma, MongoDB 和开源模型构建 RAG 系统\n", + "\n", + "作者: [Richmond Alake](https://huggingface.co/RichmondMongo)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "\n", + "## 第一步：安装库\n", + "\n", + "这些命令是用来安装一些软件包的，这些软件包可以帮助你使用和操作 LLMs，处理数据，并且和数据库进行交流。这些库简化了RAG系统的开发，将复杂性减少到少量的代码：\n", + "- PyMongo：一个用于与 MongoDB 交互的 Python 库，它提供了连接到集群和查询存储在集合和文档中的数据的功能。\n", + "- Pandas：提供了一个数据结构，用于使用 Python 进行高效的数据处理和分析。\n", + "- Hugging Face datasets：包含音频、视觉和文本数据集。\n", + "- Hugging Face Accelerate：抽象了编写利用硬件加速器（如GPU）的代码的复杂性。在实现中，利用 Accelerate 在 GPU 资源上利用 Gemma 模型。\n", + "- Hugging Face Transformers：访问大量预训练模型。\n", + "- Hugging Face Sentence Transformers：提供对句子、文本和图像嵌入的访问。" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "gVSo_nNOUsdn", + "outputId": "907f4738-a3b0-4c0f-b293-eff65c665c07" + }, + "outputs": [], + "source": [ + "!pip install datasets pandas pymongo sentence_transformers\n", + "!pip install -U transformers\n", + "# Install below if using GPU\n", + "!pip install accelerate" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 第二步：数据源选择和准备\n", + "\n", + "\n", + "本教程使用的数据来源于 Hugging Face datasets，具体是 [AIatMongoDB/embedded_movies 数据集](https://huggingface.co/datasets/AIatMongoDB/embedded_movies)。\n", + "\n" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 747 + }, + "id": "5gCzss27UwWw", + "outputId": "212cca18-a0d7-4289-bce0-ee6259fc2dba" + }, + "outputs": [ + { + "data": { + "application/vnd.google.colaboratory.intrinsic+json": { + "summary": "{\n \"name\": \"dataset_df\",\n \"rows\": 1500,\n \"fields\": [\n {\n \"column\": \"num_mflix_comments\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 27,\n \"min\": 0,\n \"max\": 158,\n \"num_unique_values\": 40,\n \"samples\": [\n 117,\n 134,\n 124\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"genres\",\n \"properties\": {\n \"dtype\": \"object\",\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"countries\",\n \"properties\": {\n \"dtype\": \"object\",\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"directors\",\n \"properties\": {\n \"dtype\": \"object\",\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"fullplot\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 1409,\n \"samples\": [\n \"An undercover cop infiltrates a gang of thieves who plan to rob a jewelry store.\",\n \"Godzilla returns in a brand-new movie that ignores all preceding movies except for the original with a brand new look and a powered up atomic ray. This time he battles a mysterious UFO that later transforms into a mysterious kaiju dubbed Orga. They meet up for the final showdown in the city of Shinjuku.\",\n \"Relationships become entangled in an emotional web.\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"writers\",\n \"properties\": {\n \"dtype\": \"object\",\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"awards\",\n \"properties\": {\n \"dtype\": \"object\",\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"runtime\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 42.09038552453906,\n \"min\": 6.0,\n \"max\": 1256.0,\n \"num_unique_values\": 139,\n \"samples\": [\n 152.0,\n 127.0,\n 96.0\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"type\",\n \"properties\": {\n \"dtype\": \"category\",\n \"num_unique_values\": 2,\n \"samples\": [\n \"series\",\n \"movie\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"rated\",\n \"properties\": {\n \"dtype\": \"category\",\n \"num_unique_values\": 12,\n \"samples\": [\n \"TV-MA\",\n \"TV-14\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"metacritic\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 16.861995960390892,\n \"min\": 9.0,\n \"max\": 97.0,\n \"num_unique_values\": 83,\n \"samples\": [\n 50.0,\n 97.0\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"poster\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 1368,\n \"samples\": [\n \"https://m.media-amazon.com/images/M/MV5BNWE5MzAwMjQtNzI1YS00YjZhLTkxNDItM2JjNjM3ZjI5NzBjXkEyXkFqcGdeQXVyMTQxNzMzNDI@._V1_SY1000_SX677_AL_.jpg\",\n \"https://m.media-amazon.com/images/M/MV5BMTgwNjIyNTczMF5BMl5BanBnXkFtZTcwODI5MDkyMQ@@._V1_SY1000_SX677_AL_.jpg\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"languages\",\n \"properties\": {\n \"dtype\": \"object\",\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"imdb\",\n \"properties\": {\n \"dtype\": \"object\",\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"plot\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 1429,\n \"samples\": [\n \"A New York City architect becomes a one-man vigilante squad after his wife is murdered by street punks in which he randomly goes out and kills would-be muggers on the mean streets after dark.\",\n \"As the daring thief Ars\\u00e8ne Lupin (Duris) ransacks the homes of wealthy Parisians, the police, with a secret weapon in their arsenal, attempt to ferret him out.\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"cast\",\n \"properties\": {\n \"dtype\": \"object\",\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"plot_embedding\",\n \"properties\": {\n \"dtype\": \"object\",\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"title\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 1435,\n \"samples\": [\n \"Turbo: A Power Rangers Movie\",\n \"Neon Genesis Evangelion: Death & Rebirth\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n }\n ]\n}", + "type": "dataframe", + "variable_name": "dataset_df" + }, + "text/html": [ + "\n", + "
\n", + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
num_mflix_commentsgenrescountriesdirectorsfullplotwritersawardsruntimetyperatedmetacriticposterlanguagesimdbplotcastplot_embeddingtitle
00[Action][USA][Louis J. Gasnier, Donald MacKenzie]Young Pauline is left a lot of money when her ...[Charles W. Goddard (screenplay), Basil Dickey...{'nominations': 0, 'text': '1 win.', 'wins': 1}199.0movieNoneNaNhttps://m.media-amazon.com/images/M/MV5BMzgxOD...[English]{'id': 4465, 'rating': 7.6, 'votes': 744}Young Pauline is left a lot of money when her ...[Pearl White, Crane Wilbur, Paul Panzer, Edwar...[0.00072939653, -0.026834568, 0.013515796, -0....The Perils of Pauline
10[Comedy, Short, Action][USA][Alfred J. Goulding, Hal Roach]As a penniless man worries about how he will m...[H.M. Walker (titles)]{'nominations': 1, 'text': '1 nomination.', 'w...22.0movieTV-GNaNhttps://m.media-amazon.com/images/M/MV5BNzE1OW...[English]{'id': 10146, 'rating': 7.0, 'votes': 639}A penniless young man tries to save an heiress...[Harold Lloyd, Mildred Davis, 'Snub' Pollard, ...[-0.022837115, -0.022941574, 0.014937485, -0.0...From Hand to Mouth
20[Action, Adventure, Drama][USA][Herbert Brenon]Michael \"Beau\" Geste leaves England in disgrac...[Herbert Brenon (adaptation), John Russell (ad...{'nominations': 0, 'text': '1 win.', 'wins': 1}101.0movieNoneNaNNone[English]{'id': 16634, 'rating': 6.9, 'votes': 222}Michael \"Beau\" Geste leaves England in disgrac...[Ronald Colman, Neil Hamilton, Ralph Forbes, A...[0.00023330493, -0.028511643, 0.014653289, -0....Beau Geste
31[Adventure, Action][USA][Albert Parker]A nobleman vows to avenge the death of his fat...[Douglas Fairbanks (story), Jack Cunningham (a...{'nominations': 0, 'text': '1 win.', 'wins': 1}88.0movieNoneNaNhttps://m.media-amazon.com/images/M/MV5BMzU0ND...None{'id': 16654, 'rating': 7.2, 'votes': 1146}Seeking revenge, an athletic young man joins t...[Billie Dove, Tempe Pigott, Donald Crisp, Sam ...[-0.005927917, -0.033394486, 0.0015323418, -0....The Black Pirate
40[Action, Comedy, Romance][USA][Sam Taylor]The Uptown Boy, J. Harold Manners (Lloyd) is a...[Ted Wilde (story), John Grey (story), Clyde B...{'nominations': 1, 'text': '1 nomination.', 'w...58.0moviePASSEDNaNhttps://m.media-amazon.com/images/M/MV5BMTcxMT...[English]{'id': 16895, 'rating': 7.6, 'votes': 918}An irresponsible young millionaire changes his...[Harold Lloyd, Jobyna Ralston, Noah Young, Jim...[-0.0059373598, -0.026604708, -0.0070914757, -...For Heaven's Sake
\n", + "
\n", + "
\n", + "\n", + "
\n", + " \n", + "\n", + " \n", + "\n", + " \n", + "
\n", + "\n", + "\n", + "
\n", + " \n", + "\n", + "\n", + "\n", + " \n", + "
\n", + "
\n", + "
\n" + ], + "text/plain": [ + " num_mflix_comments genres countries \\\n", + "0 0 [Action] [USA] \n", + "1 0 [Comedy, Short, Action] [USA] \n", + "2 0 [Action, Adventure, Drama] [USA] \n", + "3 1 [Adventure, Action] [USA] \n", + "4 0 [Action, Comedy, Romance] [USA] \n", + "\n", + " directors \\\n", + "0 [Louis J. Gasnier, Donald MacKenzie] \n", + "1 [Alfred J. Goulding, Hal Roach] \n", + "2 [Herbert Brenon] \n", + "3 [Albert Parker] \n", + "4 [Sam Taylor] \n", + "\n", + " fullplot \\\n", + "0 Young Pauline is left a lot of money when her ... \n", + "1 As a penniless man worries about how he will m... \n", + "2 Michael \"Beau\" Geste leaves England in disgrac... \n", + "3 A nobleman vows to avenge the death of his fat... \n", + "4 The Uptown Boy, J. Harold Manners (Lloyd) is a... \n", + "\n", + " writers \\\n", + "0 [Charles W. Goddard (screenplay), Basil Dickey... \n", + "1 [H.M. Walker (titles)] \n", + "2 [Herbert Brenon (adaptation), John Russell (ad... \n", + "3 [Douglas Fairbanks (story), Jack Cunningham (a... \n", + "4 [Ted Wilde (story), John Grey (story), Clyde B... \n", + "\n", + " awards runtime type rated \\\n", + "0 {'nominations': 0, 'text': '1 win.', 'wins': 1} 199.0 movie None \n", + "1 {'nominations': 1, 'text': '1 nomination.', 'w... 22.0 movie TV-G \n", + "2 {'nominations': 0, 'text': '1 win.', 'wins': 1} 101.0 movie None \n", + "3 {'nominations': 0, 'text': '1 win.', 'wins': 1} 88.0 movie None \n", + "4 {'nominations': 1, 'text': '1 nomination.', 'w... 58.0 movie PASSED \n", + "\n", + " metacritic poster languages \\\n", + "0 NaN https://m.media-amazon.com/images/M/MV5BMzgxOD... [English] \n", + "1 NaN https://m.media-amazon.com/images/M/MV5BNzE1OW... [English] \n", + "2 NaN None [English] \n", + "3 NaN https://m.media-amazon.com/images/M/MV5BMzU0ND... None \n", + "4 NaN https://m.media-amazon.com/images/M/MV5BMTcxMT... [English] \n", + "\n", + " imdb \\\n", + "0 {'id': 4465, 'rating': 7.6, 'votes': 744} \n", + "1 {'id': 10146, 'rating': 7.0, 'votes': 639} \n", + "2 {'id': 16634, 'rating': 6.9, 'votes': 222} \n", + "3 {'id': 16654, 'rating': 7.2, 'votes': 1146} \n", + "4 {'id': 16895, 'rating': 7.6, 'votes': 918} \n", + "\n", + " plot \\\n", + "0 Young Pauline is left a lot of money when her ... \n", + "1 A penniless young man tries to save an heiress... \n", + "2 Michael \"Beau\" Geste leaves England in disgrac... \n", + "3 Seeking revenge, an athletic young man joins t... \n", + "4 An irresponsible young millionaire changes his... \n", + "\n", + " cast \\\n", + "0 [Pearl White, Crane Wilbur, Paul Panzer, Edwar... \n", + "1 [Harold Lloyd, Mildred Davis, 'Snub' Pollard, ... \n", + "2 [Ronald Colman, Neil Hamilton, Ralph Forbes, A... \n", + "3 [Billie Dove, Tempe Pigott, Donald Crisp, Sam ... \n", + "4 [Harold Lloyd, Jobyna Ralston, Noah Young, Jim... \n", + "\n", + " plot_embedding title \n", + "0 [0.00072939653, -0.026834568, 0.013515796, -0.... The Perils of Pauline \n", + "1 [-0.022837115, -0.022941574, 0.014937485, -0.0... From Hand to Mouth \n", + "2 [0.00023330493, -0.028511643, 0.014653289, -0.... Beau Geste \n", + "3 [-0.005927917, -0.033394486, 0.0015323418, -0.... The Black Pirate \n", + "4 [-0.0059373598, -0.026604708, -0.0070914757, -... For Heaven's Sake " + ] + }, + "execution_count": 2, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# Load Dataset\n", + "from datasets import load_dataset\n", + "import pandas as pd\n", + "\n", + "# https://huggingface.co/datasets/AIatMongoDB/embedded_movies\n", + "dataset = load_dataset(\"AIatMongoDB/embedded_movies\")\n", + "\n", + "# Convert the dataset to a pandas dataframe\n", + "dataset_df = pd.DataFrame(dataset[\"train\"])\n", + "\n", + "dataset_df.head(5)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "以下代码片段中的操作侧重于确保数据的完整性和质量。\n", + "1. 第一个过程确保每个数据点的 `fullplot` 属性不为空，因为这是我们嵌入过程中主要使用的数据。\n", + "2. 这一步还确保我们移除所有数据点的 `plot_embedding` 属性，因为这将被一个不同的嵌入模型 `gte-large` 创建的新嵌入所替换。" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 1000 + }, + "id": "ARdz6j7SUxqi", + "outputId": "c53c458a-512d-4b7e-93b4-514f6de9d497" + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\n", + "Number of missing values in each column after removal:\n", + "num_mflix_comments 0\n", + "genres 0\n", + "countries 0\n", + "directors 12\n", + "fullplot 0\n", + "writers 13\n", + "awards 0\n", + "runtime 14\n", + "type 0\n", + "rated 279\n", + "metacritic 893\n", + "poster 78\n", + "languages 1\n", + "imdb 0\n", + "plot 0\n", + "cast 1\n", + "plot_embedding 1\n", + "title 0\n", + "dtype: int64\n" + ] + }, + { + "data": { + "application/vnd.google.colaboratory.intrinsic+json": { + "summary": "{\n \"name\": \"dataset_df\",\n \"rows\": 1452,\n \"fields\": [\n {\n \"column\": \"num_mflix_comments\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 27,\n \"min\": 0,\n \"max\": 158,\n \"num_unique_values\": 40,\n \"samples\": [\n 117,\n 134,\n 124\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"genres\",\n \"properties\": {\n \"dtype\": \"object\",\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"countries\",\n \"properties\": {\n \"dtype\": \"object\",\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"directors\",\n \"properties\": {\n \"dtype\": \"object\",\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"fullplot\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 1409,\n \"samples\": [\n \"An undercover cop infiltrates a gang of thieves who plan to rob a jewelry store.\",\n \"Godzilla returns in a brand-new movie that ignores all preceding movies except for the original with a brand new look and a powered up atomic ray. This time he battles a mysterious UFO that later transforms into a mysterious kaiju dubbed Orga. They meet up for the final showdown in the city of Shinjuku.\",\n \"Relationships become entangled in an emotional web.\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"writers\",\n \"properties\": {\n \"dtype\": \"object\",\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"awards\",\n \"properties\": {\n \"dtype\": \"object\",\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"runtime\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 42.5693352357647,\n \"min\": 6.0,\n \"max\": 1256.0,\n \"num_unique_values\": 137,\n \"samples\": [\n 60.0,\n 151.0,\n 110.0\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"type\",\n \"properties\": {\n \"dtype\": \"category\",\n \"num_unique_values\": 2,\n \"samples\": [\n \"series\",\n \"movie\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"rated\",\n \"properties\": {\n \"dtype\": \"category\",\n \"num_unique_values\": 12,\n \"samples\": [\n \"TV-MA\",\n \"TV-14\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"metacritic\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 16.855402595666057,\n \"min\": 9.0,\n \"max\": 97.0,\n \"num_unique_values\": 83,\n \"samples\": [\n 50.0,\n 97.0\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"poster\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 1332,\n \"samples\": [\n \"https://m.media-amazon.com/images/M/MV5BMTQ2NTMxODEyNV5BMl5BanBnXkFtZTcwMDgxMjA0MQ@@._V1_SY1000_SX677_AL_.jpg\",\n \"https://m.media-amazon.com/images/M/MV5BMTY5OTg1ODk0MV5BMl5BanBnXkFtZTcwMTEwMjU1MQ@@._V1_SY1000_SX677_AL_.jpg\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"languages\",\n \"properties\": {\n \"dtype\": \"object\",\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"imdb\",\n \"properties\": {\n \"dtype\": \"object\",\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"plot\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 1409,\n \"samples\": [\n \"An undercover cop infiltrates a gang of thieves who plan to rob a jewelry store.\",\n \"Godzilla saves Tokyo from a flying saucer that transforms into the beast Orga.\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"cast\",\n \"properties\": {\n \"dtype\": \"object\",\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"title\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 1391,\n \"samples\": [\n \"Superhero Movie\",\n \"Hooper\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n }\n ]\n}", + "type": "dataframe", + "variable_name": "dataset_df" + }, + "text/html": [ + "\n", + "
\n", + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
num_mflix_commentsgenrescountriesdirectorsfullplotwritersawardsruntimetyperatedmetacriticposterlanguagesimdbplotcasttitle
00[Action][USA][Louis J. Gasnier, Donald MacKenzie]Young Pauline is left a lot of money when her ...[Charles W. Goddard (screenplay), Basil Dickey...{'nominations': 0, 'text': '1 win.', 'wins': 1}199.0movieNoneNaNhttps://m.media-amazon.com/images/M/MV5BMzgxOD...[English]{'id': 4465, 'rating': 7.6, 'votes': 744}Young Pauline is left a lot of money when her ...[Pearl White, Crane Wilbur, Paul Panzer, Edwar...The Perils of Pauline
10[Comedy, Short, Action][USA][Alfred J. Goulding, Hal Roach]As a penniless man worries about how he will m...[H.M. Walker (titles)]{'nominations': 1, 'text': '1 nomination.', 'w...22.0movieTV-GNaNhttps://m.media-amazon.com/images/M/MV5BNzE1OW...[English]{'id': 10146, 'rating': 7.0, 'votes': 639}A penniless young man tries to save an heiress...[Harold Lloyd, Mildred Davis, 'Snub' Pollard, ...From Hand to Mouth
20[Action, Adventure, Drama][USA][Herbert Brenon]Michael \"Beau\" Geste leaves England in disgrac...[Herbert Brenon (adaptation), John Russell (ad...{'nominations': 0, 'text': '1 win.', 'wins': 1}101.0movieNoneNaNNone[English]{'id': 16634, 'rating': 6.9, 'votes': 222}Michael \"Beau\" Geste leaves England in disgrac...[Ronald Colman, Neil Hamilton, Ralph Forbes, A...Beau Geste
31[Adventure, Action][USA][Albert Parker]A nobleman vows to avenge the death of his fat...[Douglas Fairbanks (story), Jack Cunningham (a...{'nominations': 0, 'text': '1 win.', 'wins': 1}88.0movieNoneNaNhttps://m.media-amazon.com/images/M/MV5BMzU0ND...None{'id': 16654, 'rating': 7.2, 'votes': 1146}Seeking revenge, an athletic young man joins t...[Billie Dove, Tempe Pigott, Donald Crisp, Sam ...The Black Pirate
40[Action, Comedy, Romance][USA][Sam Taylor]The Uptown Boy, J. Harold Manners (Lloyd) is a...[Ted Wilde (story), John Grey (story), Clyde B...{'nominations': 1, 'text': '1 nomination.', 'w...58.0moviePASSEDNaNhttps://m.media-amazon.com/images/M/MV5BMTcxMT...[English]{'id': 16895, 'rating': 7.6, 'votes': 918}An irresponsible young millionaire changes his...[Harold Lloyd, Jobyna Ralston, Noah Young, Jim...For Heaven's Sake
\n", + "
\n", + "
\n", + "\n", + "
\n", + " \n", + "\n", + " \n", + "\n", + " \n", + "
\n", + "\n", + "\n", + "
\n", + " \n", + "\n", + "\n", + "\n", + " \n", + "
\n", + "
\n", + "
\n" + ], + "text/plain": [ + " num_mflix_comments genres countries \\\n", + "0 0 [Action] [USA] \n", + "1 0 [Comedy, Short, Action] [USA] \n", + "2 0 [Action, Adventure, Drama] [USA] \n", + "3 1 [Adventure, Action] [USA] \n", + "4 0 [Action, Comedy, Romance] [USA] \n", + "\n", + " directors \\\n", + "0 [Louis J. Gasnier, Donald MacKenzie] \n", + "1 [Alfred J. Goulding, Hal Roach] \n", + "2 [Herbert Brenon] \n", + "3 [Albert Parker] \n", + "4 [Sam Taylor] \n", + "\n", + " fullplot \\\n", + "0 Young Pauline is left a lot of money when her ... \n", + "1 As a penniless man worries about how he will m... \n", + "2 Michael \"Beau\" Geste leaves England in disgrac... \n", + "3 A nobleman vows to avenge the death of his fat... \n", + "4 The Uptown Boy, J. Harold Manners (Lloyd) is a... \n", + "\n", + " writers \\\n", + "0 [Charles W. Goddard (screenplay), Basil Dickey... \n", + "1 [H.M. Walker (titles)] \n", + "2 [Herbert Brenon (adaptation), John Russell (ad... \n", + "3 [Douglas Fairbanks (story), Jack Cunningham (a... \n", + "4 [Ted Wilde (story), John Grey (story), Clyde B... \n", + "\n", + " awards runtime type rated \\\n", + "0 {'nominations': 0, 'text': '1 win.', 'wins': 1} 199.0 movie None \n", + "1 {'nominations': 1, 'text': '1 nomination.', 'w... 22.0 movie TV-G \n", + "2 {'nominations': 0, 'text': '1 win.', 'wins': 1} 101.0 movie None \n", + "3 {'nominations': 0, 'text': '1 win.', 'wins': 1} 88.0 movie None \n", + "4 {'nominations': 1, 'text': '1 nomination.', 'w... 58.0 movie PASSED \n", + "\n", + " metacritic poster languages \\\n", + "0 NaN https://m.media-amazon.com/images/M/MV5BMzgxOD... [English] \n", + "1 NaN https://m.media-amazon.com/images/M/MV5BNzE1OW... [English] \n", + "2 NaN None [English] \n", + "3 NaN https://m.media-amazon.com/images/M/MV5BMzU0ND... None \n", + "4 NaN https://m.media-amazon.com/images/M/MV5BMTcxMT... [English] \n", + "\n", + " imdb \\\n", + "0 {'id': 4465, 'rating': 7.6, 'votes': 744} \n", + "1 {'id': 10146, 'rating': 7.0, 'votes': 639} \n", + "2 {'id': 16634, 'rating': 6.9, 'votes': 222} \n", + "3 {'id': 16654, 'rating': 7.2, 'votes': 1146} \n", + "4 {'id': 16895, 'rating': 7.6, 'votes': 918} \n", + "\n", + " plot \\\n", + "0 Young Pauline is left a lot of money when her ... \n", + "1 A penniless young man tries to save an heiress... \n", + "2 Michael \"Beau\" Geste leaves England in disgrac... \n", + "3 Seeking revenge, an athletic young man joins t... \n", + "4 An irresponsible young millionaire changes his... \n", + "\n", + " cast title \n", + "0 [Pearl White, Crane Wilbur, Paul Panzer, Edwar... The Perils of Pauline \n", + "1 [Harold Lloyd, Mildred Davis, 'Snub' Pollard, ... From Hand to Mouth \n", + "2 [Ronald Colman, Neil Hamilton, Ralph Forbes, A... Beau Geste \n", + "3 [Billie Dove, Tempe Pigott, Donald Crisp, Sam ... The Black Pirate \n", + "4 [Harold Lloyd, Jobyna Ralston, Noah Young, Jim... For Heaven's Sake " + ] + }, + "execution_count": 3, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# Data Preparation\n", + "\n", + "# Remove data point where plot coloumn is missing\n", + "dataset_df = dataset_df.dropna(subset=[\"fullplot\"])\n", + "print(\"\\nNumber of missing values in each column after removal:\")\n", + "print(dataset_df.isnull().sum())\n", + "\n", + "# Remove the plot_embedding from each data point in the dataset as we are going to create new embeddings with an open source embedding model from Hugging Face\n", + "dataset_df = dataset_df.drop(columns=[\"plot_embedding\"])\n", + "dataset_df.head(5)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 第三步：生成嵌入\n", + "\n", + "**代码片段中的步骤如下：**\n", + "\n", + "1. 导入 `SentenceTransformer` 类以访问嵌入模型。\n", + "2. 使用 `SentenceTransformer` 构造函数加载嵌入模型，以实例化 `gte-large` 嵌入模型。\n", + "3. 定义 `get_embedding` 函数，该函数接受一个文本字符串作为输入，并返回一个代表嵌入的浮点数列表。该函数首先检查输入文本是否为空（去除空白后）。如果文本为空，则返回一个空列表。否则，它使用加载的模型生成嵌入。\n", + "4. 通过将 `get_embedding` 函数应用于 `dataset_df` DataFrame 的 \"fullplot\" 列，为每个电影的剧情生成嵌入。生成的嵌入列表被分配到一个名为 embedding 的新列中。\n", + "\n", + "*注意：由于我们可以确保文本长度保持在可管理的范围内，因此不需要对完整剧情文本进行分块处理。*\n", + "\n", + "\n" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 747 + }, + "id": "ZX8zJNN5UzPK", + "outputId": "81bc1a57-7d96-4311-ba94-4748c34c20e3" + }, + "outputs": [ + { + "data": { + "application/vnd.google.colaboratory.intrinsic+json": { + "summary": "{\n \"name\": \"dataset_df\",\n \"rows\": 1452,\n \"fields\": [\n {\n \"column\": \"num_mflix_comments\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 27,\n \"min\": 0,\n \"max\": 158,\n \"num_unique_values\": 40,\n \"samples\": [\n 117,\n 134,\n 124\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"genres\",\n \"properties\": {\n \"dtype\": \"object\",\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"countries\",\n \"properties\": {\n \"dtype\": \"object\",\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"directors\",\n \"properties\": {\n \"dtype\": \"object\",\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"fullplot\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 1409,\n \"samples\": [\n \"An undercover cop infiltrates a gang of thieves who plan to rob a jewelry store.\",\n \"Godzilla returns in a brand-new movie that ignores all preceding movies except for the original with a brand new look and a powered up atomic ray. This time he battles a mysterious UFO that later transforms into a mysterious kaiju dubbed Orga. They meet up for the final showdown in the city of Shinjuku.\",\n \"Relationships become entangled in an emotional web.\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"writers\",\n \"properties\": {\n \"dtype\": \"object\",\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"awards\",\n \"properties\": {\n \"dtype\": \"object\",\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"runtime\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 42.5693352357647,\n \"min\": 6.0,\n \"max\": 1256.0,\n \"num_unique_values\": 137,\n \"samples\": [\n 60.0,\n 151.0,\n 110.0\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"type\",\n \"properties\": {\n \"dtype\": \"category\",\n \"num_unique_values\": 2,\n \"samples\": [\n \"series\",\n \"movie\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"rated\",\n \"properties\": {\n \"dtype\": \"category\",\n \"num_unique_values\": 12,\n \"samples\": [\n \"TV-MA\",\n \"TV-14\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"metacritic\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 16.855402595666057,\n \"min\": 9.0,\n \"max\": 97.0,\n \"num_unique_values\": 83,\n \"samples\": [\n 50.0,\n 97.0\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"poster\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 1332,\n \"samples\": [\n \"https://m.media-amazon.com/images/M/MV5BMTQ2NTMxODEyNV5BMl5BanBnXkFtZTcwMDgxMjA0MQ@@._V1_SY1000_SX677_AL_.jpg\",\n \"https://m.media-amazon.com/images/M/MV5BMTY5OTg1ODk0MV5BMl5BanBnXkFtZTcwMTEwMjU1MQ@@._V1_SY1000_SX677_AL_.jpg\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"languages\",\n \"properties\": {\n \"dtype\": \"object\",\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"imdb\",\n \"properties\": {\n \"dtype\": \"object\",\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"plot\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 1409,\n \"samples\": [\n \"An undercover cop infiltrates a gang of thieves who plan to rob a jewelry store.\",\n \"Godzilla saves Tokyo from a flying saucer that transforms into the beast Orga.\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"cast\",\n \"properties\": {\n \"dtype\": \"object\",\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"title\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 1391,\n \"samples\": [\n \"Superhero Movie\",\n \"Hooper\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"embedding\",\n \"properties\": {\n \"dtype\": \"object\",\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n }\n ]\n}", + "type": "dataframe", + "variable_name": "dataset_df" + }, + "text/html": [ + "\n", + "
\n", + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
num_mflix_commentsgenrescountriesdirectorsfullplotwritersawardsruntimetyperatedmetacriticposterlanguagesimdbplotcasttitleembedding
00[Action][USA][Louis J. Gasnier, Donald MacKenzie]Young Pauline is left a lot of money when her ...[Charles W. Goddard (screenplay), Basil Dickey...{'nominations': 0, 'text': '1 win.', 'wins': 1}199.0movieNoneNaNhttps://m.media-amazon.com/images/M/MV5BMzgxOD...[English]{'id': 4465, 'rating': 7.6, 'votes': 744}Young Pauline is left a lot of money when her ...[Pearl White, Crane Wilbur, Paul Panzer, Edwar...The Perils of Pauline[-0.009285838343203068, -0.005062104668468237,...
10[Comedy, Short, Action][USA][Alfred J. Goulding, Hal Roach]As a penniless man worries about how he will m...[H.M. Walker (titles)]{'nominations': 1, 'text': '1 nomination.', 'w...22.0movieTV-GNaNhttps://m.media-amazon.com/images/M/MV5BNzE1OW...[English]{'id': 10146, 'rating': 7.0, 'votes': 639}A penniless young man tries to save an heiress...[Harold Lloyd, Mildred Davis, 'Snub' Pollard, ...From Hand to Mouth[-0.0024393785279244184, 0.02309592440724373, ...
20[Action, Adventure, Drama][USA][Herbert Brenon]Michael \"Beau\" Geste leaves England in disgrac...[Herbert Brenon (adaptation), John Russell (ad...{'nominations': 0, 'text': '1 win.', 'wins': 1}101.0movieNoneNaNNone[English]{'id': 16634, 'rating': 6.9, 'votes': 222}Michael \"Beau\" Geste leaves England in disgrac...[Ronald Colman, Neil Hamilton, Ralph Forbes, A...Beau Geste[0.012204292230308056, -0.01145575474947691, -...
31[Adventure, Action][USA][Albert Parker]A nobleman vows to avenge the death of his fat...[Douglas Fairbanks (story), Jack Cunningham (a...{'nominations': 0, 'text': '1 win.', 'wins': 1}88.0movieNoneNaNhttps://m.media-amazon.com/images/M/MV5BMzU0ND...None{'id': 16654, 'rating': 7.2, 'votes': 1146}Seeking revenge, an athletic young man joins t...[Billie Dove, Tempe Pigott, Donald Crisp, Sam ...The Black Pirate[0.004541348200291395, -0.0006100579630583525,...
40[Action, Comedy, Romance][USA][Sam Taylor]The Uptown Boy, J. Harold Manners (Lloyd) is a...[Ted Wilde (story), John Grey (story), Clyde B...{'nominations': 1, 'text': '1 nomination.', 'w...58.0moviePASSEDNaNhttps://m.media-amazon.com/images/M/MV5BMTcxMT...[English]{'id': 16895, 'rating': 7.6, 'votes': 918}An irresponsible young millionaire changes his...[Harold Lloyd, Jobyna Ralston, Noah Young, Jim...For Heaven's Sake[-0.0022256041411310434, 0.011567804962396622,...
\n", + "
\n", + "
\n", + "\n", + "
\n", + " \n", + "\n", + " \n", + "\n", + " \n", + "
\n", + "\n", + "\n", + "
\n", + " \n", + "\n", + "\n", + "\n", + " \n", + "
\n", + "
\n", + "
\n" + ], + "text/plain": [ + " num_mflix_comments genres countries \\\n", + "0 0 [Action] [USA] \n", + "1 0 [Comedy, Short, Action] [USA] \n", + "2 0 [Action, Adventure, Drama] [USA] \n", + "3 1 [Adventure, Action] [USA] \n", + "4 0 [Action, Comedy, Romance] [USA] \n", + "\n", + " directors \\\n", + "0 [Louis J. Gasnier, Donald MacKenzie] \n", + "1 [Alfred J. Goulding, Hal Roach] \n", + "2 [Herbert Brenon] \n", + "3 [Albert Parker] \n", + "4 [Sam Taylor] \n", + "\n", + " fullplot \\\n", + "0 Young Pauline is left a lot of money when her ... \n", + "1 As a penniless man worries about how he will m... \n", + "2 Michael \"Beau\" Geste leaves England in disgrac... \n", + "3 A nobleman vows to avenge the death of his fat... \n", + "4 The Uptown Boy, J. Harold Manners (Lloyd) is a... \n", + "\n", + " writers \\\n", + "0 [Charles W. Goddard (screenplay), Basil Dickey... \n", + "1 [H.M. Walker (titles)] \n", + "2 [Herbert Brenon (adaptation), John Russell (ad... \n", + "3 [Douglas Fairbanks (story), Jack Cunningham (a... \n", + "4 [Ted Wilde (story), John Grey (story), Clyde B... \n", + "\n", + " awards runtime type rated \\\n", + "0 {'nominations': 0, 'text': '1 win.', 'wins': 1} 199.0 movie None \n", + "1 {'nominations': 1, 'text': '1 nomination.', 'w... 22.0 movie TV-G \n", + "2 {'nominations': 0, 'text': '1 win.', 'wins': 1} 101.0 movie None \n", + "3 {'nominations': 0, 'text': '1 win.', 'wins': 1} 88.0 movie None \n", + "4 {'nominations': 1, 'text': '1 nomination.', 'w... 58.0 movie PASSED \n", + "\n", + " metacritic poster languages \\\n", + "0 NaN https://m.media-amazon.com/images/M/MV5BMzgxOD... [English] \n", + "1 NaN https://m.media-amazon.com/images/M/MV5BNzE1OW... [English] \n", + "2 NaN None [English] \n", + "3 NaN https://m.media-amazon.com/images/M/MV5BMzU0ND... None \n", + "4 NaN https://m.media-amazon.com/images/M/MV5BMTcxMT... [English] \n", + "\n", + " imdb \\\n", + "0 {'id': 4465, 'rating': 7.6, 'votes': 744} \n", + "1 {'id': 10146, 'rating': 7.0, 'votes': 639} \n", + "2 {'id': 16634, 'rating': 6.9, 'votes': 222} \n", + "3 {'id': 16654, 'rating': 7.2, 'votes': 1146} \n", + "4 {'id': 16895, 'rating': 7.6, 'votes': 918} \n", + "\n", + " plot \\\n", + "0 Young Pauline is left a lot of money when her ... \n", + "1 A penniless young man tries to save an heiress... \n", + "2 Michael \"Beau\" Geste leaves England in disgrac... \n", + "3 Seeking revenge, an athletic young man joins t... \n", + "4 An irresponsible young millionaire changes his... \n", + "\n", + " cast title \\\n", + "0 [Pearl White, Crane Wilbur, Paul Panzer, Edwar... The Perils of Pauline \n", + "1 [Harold Lloyd, Mildred Davis, 'Snub' Pollard, ... From Hand to Mouth \n", + "2 [Ronald Colman, Neil Hamilton, Ralph Forbes, A... Beau Geste \n", + "3 [Billie Dove, Tempe Pigott, Donald Crisp, Sam ... The Black Pirate \n", + "4 [Harold Lloyd, Jobyna Ralston, Noah Young, Jim... For Heaven's Sake \n", + "\n", + " embedding \n", + "0 [-0.009285838343203068, -0.005062104668468237,... \n", + "1 [-0.0024393785279244184, 0.02309592440724373, ... \n", + "2 [0.012204292230308056, -0.01145575474947691, -... \n", + "3 [0.004541348200291395, -0.0006100579630583525,... \n", + "4 [-0.0022256041411310434, 0.011567804962396622,... " + ] + }, + "execution_count": 4, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "from sentence_transformers import SentenceTransformer\n", + "\n", + "# https://huggingface.co/thenlper/gte-large\n", + "embedding_model = SentenceTransformer(\"thenlper/gte-large\")\n", + "\n", + "\n", + "def get_embedding(text: str) -> list[float]:\n", + " if not text.strip():\n", + " print(\"Attempted to get embedding for empty text.\")\n", + " return []\n", + "\n", + " embedding = embedding_model.encode(text)\n", + "\n", + " return embedding.tolist()\n", + "\n", + "\n", + "dataset_df[\"embedding\"] = dataset_df[\"fullplot\"].apply(get_embedding)\n", + "\n", + "dataset_df.head()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 第4步：数据库设置和连接\n", + "\n", + "MongoDB 既是一个操作数据库，也是一个向量数据库。它提供了一个数据库解决方案，有效地存储、查询和检索向量嵌入。其优势在于数据库维护、管理和成本的简单性。\n", + "\n", + "**创建新的 MongoDB 数据库，设置数据库集群：**\n", + "\n", + "1. 前往MongoDB官网，注册一个[免费的 MongoDB Atlas 账户](https://www.mongodb.com/cloud/atlas/register?utm_campaign=devrel&utm_source=community&utm_medium=cta&utm_content=Partner%20Cookbook&utm_term=richmond.alake)，或者对于现有用户，[登录 MongoDB Atlas](https://account.mongodb.com/account/login?utm_campaign=devrel&utm_source=community&utm_medium=cta&utm_content=Partner%20Cookbook&utm_term=richmond.alakee)。\n", + "\n", + "2. 在左侧窗格中选择 'Database' 选项，这将导航到数据库部署页面，你可以在其中查看任何现有集群的部署规格。点击 \"+Create\" 按钮，创建一个新的数据库集群。\n", + "\n", + "3. 选择适用于数据库集群的所有配置。选择所有配置选项后，点击 “Create Cluster” 按钮以部署新创建的集群。MongoDB 还在 “Shared Tab” 上启用了免费集群的创建。\n", + "\n", + " *注意：创建概念证明时，不要忘记将 Python 主机的 IP 列入白名单，或设置 0.0.0.0/0 用于任何IP。*\n", + "\n", + "4. 成功创建和部署集群后，集群将在 ‘Database Deployment’ 页面中变得可访问。\n", + "\n", + "5. 点击集群的 “Connect” 按钮，查看通过各种语言驱动程序设置与集群的连接的选项。\n", + "\n", + "6. 本教程只需要集群的 URI（唯一资源标识符）。获取 URI 并将其复制到 Google Colabs Secrets 环境中的名为 `MONGO_URI` 的变量中，或者将其放入 .env 文件或等效文件中。\n", + "\n", + "### 4.1 数据库和集合设置\n", + "\n", + "在继续之前，请确保满足以下先决条件\n", + "\n", + "- 在 MongoDB Atlas 上设置数据库集群\n", + "- 获取到你的集群的 URI\n", + "\n", + "有关数据库集群设置和获取 URI 的帮助，请参阅我们的指南：[设置 MongoDB 集群](https://www.mongodb.com/docs/guides/atlas/cluster/)和[获取你的连接字符串](https://www.mongodb.com/docs/guides/atlas/connection-string/)\n", + "\n", + "创建集群后，通过在集群概览页面点击+创建数据库，在 MongoDB Atlas 集群中创建数据库和集合。\n", + "\n", + "这里有关于[创建数据库和集合](https://www.mongodb.com/basics/create-database)的指南\n", + "**数据库将被命名为 `movies`。**\n", + "**集合将被命名为 `movie_collection_2`。**\n", + "\n", + "\n", + "\n", + "\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "\n", + "## 第5步：创建向量搜索索引\n", + "\n", + "在这一点上，请确保你的向量索引是通过 MongoDB Atlas 创建的。\n", + "\n", + "接下来，你必须做一个非常重要的步骤，那就是在 `movie_collection_2` 这个数据库的文档里，为那些用来表示电影特点的向量建立一个特殊的搜索索引。这个索引就像是图书馆里的图书索引卡，它帮助计算机快速准确地找到与你的搜索最相似的电影向量。没有这个索引，计算机就得一篇一篇地翻找，效率会非常低。所以，建立这个索引是为了让搜索变得又快又准。\n", + "\n", + "点击此处了解更多关于[ MongoDB 向量搜索索引](https://www.mongodb.com/docs/atlas/atlas-search/field-types/knn-vector/)的信息。\n", + "\n", + "```\n", + "{\n", + " \"fields\": [{\n", + " \"numDimensions\": 1024,\n", + " \"path\": \"embedding\",\n", + " \"similarity\": \"cosine\",\n", + " \"type\": \"vector\"\n", + " }]\n", + "}\n", + "```\n", + "\n", + "`numDimensions` 字段的 `1024` 值对应于由 gte-large 嵌入模型生成的向量的维度。如果你使用 `gte-base` 或 `gte-small` 嵌入模型，向量搜索索引中的 numDimensions 值必须分别设置为 768 和 384。\n", + "\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 第6步：建立数据连接\n", + "\n", + "下面的代码片段还使用了 PyMongo 来创建一个 MongoDB 客户端对象，该对象代表与集群的连接，并允许访问其数据库和集合。\n", + "\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "Oi0l9POtU0iP", + "outputId": "d3fe3cc4-8c08-4435-ddfc-8cfcc5ada572" + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Connection to MongoDB successful\n" + ] + } + ], + "source": [ + "import pymongo\n", + "from google.colab import userdata\n", + "\n", + "\n", + "def get_mongo_client(mongo_uri):\n", + " \"\"\"Establish connection to the MongoDB.\"\"\"\n", + " try:\n", + " client = pymongo.MongoClient(mongo_uri)\n", + " print(\"Connection to MongoDB successful\")\n", + " return client\n", + " except pymongo.errors.ConnectionFailure as e:\n", + " print(f\"Connection failed: {e}\")\n", + " return None\n", + "\n", + "\n", + "mongo_uri = userdata.get(\"MONGO_URI\")\n", + "if not mongo_uri:\n", + " print(\"MONGO_URI not set in environment variables\")\n", + "\n", + "mongo_client = get_mongo_client(mongo_uri)\n", + "\n", + "# Ingest data into MongoDB\n", + "db = mongo_client[\"movies\"]\n", + "collection = db[\"movie_collection_2\"]" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "F7XXXa-OU1u9", + "outputId": "7bd1eb43-e933-4150-990a-fa20bad84e9a" + }, + "outputs": [ + { + "data": { + "text/plain": [ + "DeleteResult({'n': 1452, 'electionId': ObjectId('7fffffff000000000000000c'), 'opTime': {'ts': Timestamp(1708554945, 1452), 't': 12}, 'ok': 1.0, '$clusterTime': {'clusterTime': Timestamp(1708554945, 1452), 'signature': {'hash': b'\\x99\\x89\\xc0\\x00Cn!\\xd6\\xaf\\xb3\\x96\\xdf\\xc3\\xda\\x88\\x11\\xf5\\t\\xbd\\xc0', 'keyId': 7320226449804230661}}, 'operationTime': Timestamp(1708554945, 1452)}, acknowledged=True)" + ] + }, + "execution_count": 6, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# Delete any existing records in the collection\n", + "collection.delete_many({})" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "从 pandas DataFrame 中将数据导入 MongoDB 集合是一个简单的过程，可以通过将 DataFrame 转换为字典，然后在集合上使用 `insert_many` 方法来传递转换后的数据集记录，从而高效完成。\n" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "XrfMY4QBU2-l", + "outputId": "e2b5c534-2ba0-4ffa-bca8-1e96bef14c54" + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Data ingestion into MongoDB completed\n" + ] + } + ], + "source": [ + "documents = dataset_df.to_dict(\"records\")\n", + "collection.insert_many(documents)\n", + "\n", + "print(\"Data ingestion into MongoDB completed\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 第7步：对用户查询执行向量搜索\n", + "\n", + "下一步实现了一个函数，该函数通过生成查询嵌入并定义一个 MongoDB 聚合流水线来返回一个向量搜索结果。\n", + "\n", + "该流水线包括 `$vectorSearch` 和 `$project` 阶段，它使用生成的向量执行查询，并格式化结果以仅包括所需信息，如剧情、标题和类型，同时为每个结果引入一个搜索分数。\n" + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "metadata": { + "id": "kWucnQBEU35k" + }, + "outputs": [], + "source": [ + "def vector_search(user_query, collection):\n", + " \"\"\"\n", + " Perform a vector search in the MongoDB collection based on the user query.\n", + "\n", + " Args:\n", + " user_query (str): The user's query string.\n", + " collection (MongoCollection): The MongoDB collection to search.\n", + "\n", + " Returns:\n", + " list: A list of matching documents.\n", + " \"\"\"\n", + "\n", + " # Generate embedding for the user query\n", + " query_embedding = get_embedding(user_query)\n", + "\n", + " if query_embedding is None:\n", + " return \"Invalid query or embedding generation failed.\"\n", + "\n", + " # Define the vector search pipeline\n", + " pipeline = [\n", + " {\n", + " \"$vectorSearch\": {\n", + " \"index\": \"vector_index\",\n", + " \"queryVector\": query_embedding,\n", + " \"path\": \"embedding\",\n", + " \"numCandidates\": 150, # Number of candidate matches to consider\n", + " \"limit\": 4, # Return top 4 matches\n", + " }\n", + " },\n", + " {\n", + " \"$project\": {\n", + " \"_id\": 0, # Exclude the _id field\n", + " \"fullplot\": 1, # Include the plot field\n", + " \"title\": 1, # Include the title field\n", + " \"genres\": 1, # Include the genres field\n", + " \"score\": {\"$meta\": \"vectorSearchScore\"}, # Include the search score\n", + " }\n", + " },\n", + " ]\n", + "\n", + " # Execute the search\n", + " results = collection.aggregate(pipeline)\n", + " return list(results)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 第 8 步：处理用户查询和加载 Gemma\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "0ka4WLTmU5L4" + }, + "outputs": [], + "source": [ + "def get_search_result(query, collection):\n", + "\n", + " get_knowledge = vector_search(query, collection)\n", + "\n", + " search_result = \"\"\n", + " for result in get_knowledge:\n", + " search_result += f\"Title: {result.get('title', 'N/A')}, Plot: {result.get('fullplot', 'N/A')}\\n\"\n", + "\n", + " return search_result" + ] + }, + { + "cell_type": "code", + "execution_count": 13, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "Z4L4SfueU6PY", + "outputId": "11ea30ca-8cac-4e4c-9ab6-780e043c6345" + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Query: What is the best romantic movie to watch and why?\n", + "Continue to answer the query by using the Search Results:\n", + "Title: Shut Up and Kiss Me!, Plot: Ryan and Pete are 27-year old best friends in Miami, born on the same day and each searching for the perfect woman. Ryan is a rookie stockbroker living with his psychic Mom. Pete is a slick surfer dude yet to find commitment. Each meets the women of their dreams on the same day. Ryan knocks heads in an elevator with the gorgeous Jessica, passing out before getting her number. Pete falls for the insatiable Tiara, but Tiara's uncle is mob boss Vincent Bublione, charged with her protection. This high-energy romantic comedy asks to what extent will you go for true love?\n", + "Title: Pearl Harbor, Plot: Pearl Harbor is a classic tale of romance set during a war that complicates everything. It all starts when childhood friends Rafe and Danny become Army Air Corps pilots and meet Evelyn, a Navy nurse. Rafe falls head over heels and next thing you know Evelyn and Rafe are hooking up. Then Rafe volunteers to go fight in Britain and Evelyn and Danny get transferred to Pearl Harbor. While Rafe is off fighting everything gets completely whack and next thing you know everybody is in the middle of an air raid we now know as \"Pearl Harbor.\"\n", + "Title: Titanic, Plot: The plot focuses on the romances of two couples upon the doomed ship's maiden voyage. Isabella Paradine (Catherine Zeta-Jones) is a wealthy woman mourning the loss of her aunt, who reignites a romance with former flame Wynn Park (Peter Gallagher). Meanwhile, a charming ne'er-do-well named Jamie Perse (Mike Doyle) steals a ticket for the ship, and falls for a sweet innocent Irish girl on board. But their romance is threatened by the villainous Simon Doonan (Tim Curry), who has discovered about the ticket and makes Jamie his unwilling accomplice, as well as having sinister plans for the girl.\n", + "Title: China Girl, Plot: A modern day Romeo & Juliet story is told in New York when an Italian boy and a Chinese girl become lovers, causing a tragic conflict between ethnic gangs.\n", + ".\n" + ] + } + ], + "source": [ + "# Conduct query with retrival of sources\n", + "query = \"What is the best romantic movie to watch and why?\"\n", + "source_information = get_search_result(query, collection)\n", + "combined_information = f\"Query: {query}\\nContinue to answer the query by using the Search Results:\\n{source_information}.\"\n", + "\n", + "print(combined_information)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 209, + "referenced_widgets": [ + "60c4d6d5e7a84fa493f101cc47dadef9", + "fa0c528cca744cff8da0a4fa21fdb4b5", + "d7d4a9f444fe4ebb9135035e2166a3a5", + "4e62b9ec821348cc94b34cfbc010c2a4", + "9d9a247c6569458abd0dcd6e0d717079", + "e5a6d300bbf441b8904aa9afb89e6f31", + "88226abe35534278bbd427d8eff0f5f8", + "2e081a17ddc04104882893a30c902265", + "938f8f60901442f2902eb51e86c27961", + "ce6a3a655d2f4ce2ab351c766568bed5", + "19fd13ad5b2740aa8be2a7d62488fdaf", + "c7c0c34a71954d6ea976c774573c49c5", + "0d6ec3bab579406fa4e6fc2b3d6b6998", + "a37f2164e11d4e5f851a4a09a12c663c", + "ef32431228f24a5498810a36b9cf6506", + "c06e354cb8294e66a3d7590a576571e0", + "e2998c2c6b1f4d489a5e39f2076838e4", + "4300755179d9465db871b14ae78dabc6", + "f14106c7f60f411199acf47f530443fd", + "bf88ee6dc83648d8a61d75bb4466b1e3", + "5776e818d9d34e009f95833056522876", + "06b1a069317041c8a9174c14fdc867bc", + "0e27bfa4f64f427d9996de0451e9edd9", + "d5e9f339fe7e4ab9955531cc125f071e", + "fa9cf3e72280417d8711ef7227a95d34", + "c3a1b520140444fbb40b7ac789f7ac0e", + "2c84bc5c158641f49f421a7d28da1518", + "6a15f1cf54a141fc9d6bb790366c6bdd", + "8813b56cd89744b58ace2787206e1501", + "edc37210db734d01a8afce596698bb27", + "eba6048eb694485693656fcbf4a4f297", + "30885be6a7c84f0f9f02bc2ea11679bc", + "29178e51df9e47489fff623763b130ed", + "5266bebcf8bb4b0798b14831a38e2a8c", + "7c638aaf734c423fbe54daddff97040f", + "4c6736981923464db2f754339c60cd0d", + "57383c03fc854a92a2ff732cbdd80a70", + "8a302ae0412b4393a17b50861afe36b5", + "b2fdc502d6ee491bb826fd616e35d407", + "2677891ce891475b8dc7c2ae287c58d7", + "fddbae43ce7f477cafaff89b81e47fc7", + "592d069be51e43f99212d47be9c11dcf", + "9a4c90a767c746659ea535d7c36d40a5", + "43fcf04b360f4a75be9fb99ab69fbe38", + "b7c439aa6d584c5784b46980050e503d", + "8aa8805651d34181b1851d302ccc47e2", + "713f1d91e445411288e565a33ce4b271", + "55941e08c602404c9342d00b7ee26918", + "87da02f5606d404ea242c3bd1f9ac38c", + "947f9b7e19dc4be4bd21b1b021e91f9d", + "0b7f3d233b8f4912bef4deae2e395001", + "6ccbd7b9ae924b5e843efd3114dfb2c5", + "9e0bccbc6072461fbf96482b870ed8d5", + "d7a00f1f114e4f008f4d5a48c1c69f53", + "faf25fd219f24bdbaa2e3202548c97d9", + "a0996675df13484aaa519e6ff45c5476", + "0bfb4937ed5547b3ba464ca47ac77f1a", + "7f59906980724a8b840dec85ce400f89", + "80f3d29327bf429481ad191b1abe556f", + "6d7c024126ac4c34825fae522234ebca", + "a0600fb407034c2d8df6ae5830d601db", + "c1d37ab1952b4d268d9786b74b6902d7", + "e7f471604a5a42e095d35d8ad399c6fe", + "feb438afda6b4c148a3a62ee7e03da74", + "e68cf53b04a845ac9d6f4047600ebc21", + "33fef11f829f49e2aa9555201d4a0e42" + ] + }, + "id": "OYGmKVv9mm8g", + "outputId": "ff41bfed-daa0-4ed8-8cc4-0aa138e697a1" + }, + "outputs": [], + "source": [ + "from transformers import AutoTokenizer, AutoModelForCausalLM\n", + "\n", + "tokenizer = AutoTokenizer.from_pretrained(\"google/gemma-2b-it\")\n", + "# CPU Enabled uncomment below 👇🏽\n", + "# model = AutoModelForCausalLM.from_pretrained(\"google/gemma-2b-it\")\n", + "# GPU Enabled use below 👇🏽\n", + "model = AutoModelForCausalLM.from_pretrained(\"google/gemma-2b-it\", device_map=\"auto\")" + ] + }, + { + "cell_type": "code", + "execution_count": 16, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "wDA9SdXhsFyM", + "outputId": "c3300fa5-586c-48bd-9abb-b12a4390a294" + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Query: What is the best romantic movie to watch and why?\n", + "Continue to answer the query by using the Search Results:\n", + "Title: Shut Up and Kiss Me!, Plot: Ryan and Pete are 27-year old best friends in Miami, born on the same day and each searching for the perfect woman. Ryan is a rookie stockbroker living with his psychic Mom. Pete is a slick surfer dude yet to find commitment. Each meets the women of their dreams on the same day. Ryan knocks heads in an elevator with the gorgeous Jessica, passing out before getting her number. Pete falls for the insatiable Tiara, but Tiara's uncle is mob boss Vincent Bublione, charged with her protection. This high-energy romantic comedy asks to what extent will you go for true love?\n", + "Title: Pearl Harbor, Plot: Pearl Harbor is a classic tale of romance set during a war that complicates everything. It all starts when childhood friends Rafe and Danny become Army Air Corps pilots and meet Evelyn, a Navy nurse. Rafe falls head over heels and next thing you know Evelyn and Rafe are hooking up. Then Rafe volunteers to go fight in Britain and Evelyn and Danny get transferred to Pearl Harbor. While Rafe is off fighting everything gets completely whack and next thing you know everybody is in the middle of an air raid we now know as \"Pearl Harbor.\"\n", + "Title: Titanic, Plot: The plot focuses on the romances of two couples upon the doomed ship's maiden voyage. Isabella Paradine (Catherine Zeta-Jones) is a wealthy woman mourning the loss of her aunt, who reignites a romance with former flame Wynn Park (Peter Gallagher). Meanwhile, a charming ne'er-do-well named Jamie Perse (Mike Doyle) steals a ticket for the ship, and falls for a sweet innocent Irish girl on board. But their romance is threatened by the villainous Simon Doonan (Tim Curry), who has discovered about the ticket and makes Jamie his unwilling accomplice, as well as having sinister plans for the girl.\n", + "Title: China Girl, Plot: A modern day Romeo & Juliet story is told in New York when an Italian boy and a Chinese girl become lovers, causing a tragic conflict between ethnic gangs.\n", + ".\n", + "\n", + "Based on the search results, the best romantic movie to watch is **Shut Up and Kiss Me!** because it is a romantic comedy that explores the complexities of love and relationships. The movie is funny, heartwarming, and thought-provoking.\n" + ] + } + ], + "source": [ + "# Moving tensors to GPU\n", + "input_ids = tokenizer(combined_information, return_tensors=\"pt\").to(\"cuda\")\n", + "response = model.generate(**input_ids, max_new_tokens=500)\n", + "print(tokenizer.decode(response[0]))" + ] + }, + { + "cell_type": "code", + "execution_count": 12, + "metadata": { + "id": "FhMmFmUBwBcy" + }, + "outputs": [], + "source": [] + } + ], + "metadata": { + "accelerator": "GPU", + "colab": { + "gpuType": "A100", + "machine_shape": "hm", + "provenance": [] + }, + "kernelspec": { + "display_name": "Python 3", + "name": "python3" + }, + "language_info": { + "name": "python" + }, + "widgets": { + "application/vnd.jupyter.widget-state+json": { + "06b1a069317041c8a9174c14fdc867bc": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "DescriptionStyleModel", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "0b7f3d233b8f4912bef4deae2e395001": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "DescriptionStyleModel", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "0bfb4937ed5547b3ba464ca47ac77f1a": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "HTMLModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_a0600fb407034c2d8df6ae5830d601db", + "placeholder": "​", + "style": "IPY_MODEL_c1d37ab1952b4d268d9786b74b6902d7", + "value": "generation_config.json: 100%" + } + }, + "0d6ec3bab579406fa4e6fc2b3d6b6998": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "HTMLModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_e2998c2c6b1f4d489a5e39f2076838e4", + "placeholder": "​", + "style": "IPY_MODEL_4300755179d9465db871b14ae78dabc6", + "value": "Downloading shards: 100%" + } + }, + "0e27bfa4f64f427d9996de0451e9edd9": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "HBoxModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HBoxModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HBoxView", + "box_style": "", + "children": [ + "IPY_MODEL_d5e9f339fe7e4ab9955531cc125f071e", + "IPY_MODEL_fa9cf3e72280417d8711ef7227a95d34", + "IPY_MODEL_c3a1b520140444fbb40b7ac789f7ac0e" + ], + "layout": "IPY_MODEL_2c84bc5c158641f49f421a7d28da1518" + } + }, + "19fd13ad5b2740aa8be2a7d62488fdaf": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "DescriptionStyleModel", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "2677891ce891475b8dc7c2ae287c58d7": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "DescriptionStyleModel", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "29178e51df9e47489fff623763b130ed": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "DescriptionStyleModel", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "2c84bc5c158641f49f421a7d28da1518": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "2e081a17ddc04104882893a30c902265": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "30885be6a7c84f0f9f02bc2ea11679bc": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "33fef11f829f49e2aa9555201d4a0e42": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "DescriptionStyleModel", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "4300755179d9465db871b14ae78dabc6": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "DescriptionStyleModel", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "43fcf04b360f4a75be9fb99ab69fbe38": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "DescriptionStyleModel", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "4c6736981923464db2f754339c60cd0d": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "FloatProgressModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "FloatProgressModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "ProgressView", + "bar_style": "success", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_fddbae43ce7f477cafaff89b81e47fc7", + "max": 67121608, + "min": 0, + "orientation": "horizontal", + "style": "IPY_MODEL_592d069be51e43f99212d47be9c11dcf", + "value": 67121608 + } + }, + "4e62b9ec821348cc94b34cfbc010c2a4": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "HTMLModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_ce6a3a655d2f4ce2ab351c766568bed5", + "placeholder": "​", + "style": "IPY_MODEL_19fd13ad5b2740aa8be2a7d62488fdaf", + "value": " 13.5k/13.5k [00:00<00:00, 1.10MB/s]" + } + }, + "5266bebcf8bb4b0798b14831a38e2a8c": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "HBoxModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HBoxModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HBoxView", + "box_style": "", + "children": [ + "IPY_MODEL_7c638aaf734c423fbe54daddff97040f", + "IPY_MODEL_4c6736981923464db2f754339c60cd0d", + "IPY_MODEL_57383c03fc854a92a2ff732cbdd80a70" + ], + "layout": "IPY_MODEL_8a302ae0412b4393a17b50861afe36b5" + } + }, + "55941e08c602404c9342d00b7ee26918": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "HTMLModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_d7a00f1f114e4f008f4d5a48c1c69f53", + "placeholder": "​", + "style": "IPY_MODEL_faf25fd219f24bdbaa2e3202548c97d9", + "value": " 2/2 [00:04<00:00,  1.94s/it]" + } + }, + "57383c03fc854a92a2ff732cbdd80a70": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "HTMLModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_9a4c90a767c746659ea535d7c36d40a5", + "placeholder": "​", + "style": "IPY_MODEL_43fcf04b360f4a75be9fb99ab69fbe38", + "value": " 67.1M/67.1M [00:00<00:00, 465MB/s]" + } + }, + "5776e818d9d34e009f95833056522876": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "592d069be51e43f99212d47be9c11dcf": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "ProgressStyleModel", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "ProgressStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "bar_color": null, + "description_width": "" + } + }, + "60c4d6d5e7a84fa493f101cc47dadef9": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "HBoxModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HBoxModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HBoxView", + "box_style": "", + "children": [ + "IPY_MODEL_fa0c528cca744cff8da0a4fa21fdb4b5", + "IPY_MODEL_d7d4a9f444fe4ebb9135035e2166a3a5", + "IPY_MODEL_4e62b9ec821348cc94b34cfbc010c2a4" + ], + "layout": "IPY_MODEL_9d9a247c6569458abd0dcd6e0d717079" + } + }, + "6a15f1cf54a141fc9d6bb790366c6bdd": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "6ccbd7b9ae924b5e843efd3114dfb2c5": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "6d7c024126ac4c34825fae522234ebca": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "713f1d91e445411288e565a33ce4b271": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "FloatProgressModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "FloatProgressModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "ProgressView", + "bar_style": "success", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_6ccbd7b9ae924b5e843efd3114dfb2c5", + "max": 2, + "min": 0, + "orientation": "horizontal", + "style": "IPY_MODEL_9e0bccbc6072461fbf96482b870ed8d5", + "value": 2 + } + }, + "7c638aaf734c423fbe54daddff97040f": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "HTMLModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_b2fdc502d6ee491bb826fd616e35d407", + "placeholder": "​", + "style": "IPY_MODEL_2677891ce891475b8dc7c2ae287c58d7", + "value": "model-00002-of-00002.safetensors: 100%" + } + }, + "7f59906980724a8b840dec85ce400f89": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "FloatProgressModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "FloatProgressModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "ProgressView", + "bar_style": "success", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_e7f471604a5a42e095d35d8ad399c6fe", + "max": 137, + "min": 0, + "orientation": "horizontal", + "style": "IPY_MODEL_feb438afda6b4c148a3a62ee7e03da74", + "value": 137 + } + }, + "80f3d29327bf429481ad191b1abe556f": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "HTMLModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_e68cf53b04a845ac9d6f4047600ebc21", + "placeholder": "​", + "style": "IPY_MODEL_33fef11f829f49e2aa9555201d4a0e42", + "value": " 137/137 [00:00<00:00, 11.9kB/s]" + } + }, + "87da02f5606d404ea242c3bd1f9ac38c": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "8813b56cd89744b58ace2787206e1501": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "DescriptionStyleModel", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "88226abe35534278bbd427d8eff0f5f8": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "DescriptionStyleModel", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "8a302ae0412b4393a17b50861afe36b5": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "8aa8805651d34181b1851d302ccc47e2": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "HTMLModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_947f9b7e19dc4be4bd21b1b021e91f9d", + "placeholder": "​", + "style": "IPY_MODEL_0b7f3d233b8f4912bef4deae2e395001", + "value": "Loading checkpoint shards: 100%" + } + }, + "938f8f60901442f2902eb51e86c27961": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "ProgressStyleModel", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "ProgressStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "bar_color": null, + "description_width": "" + } + }, + "947f9b7e19dc4be4bd21b1b021e91f9d": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "9a4c90a767c746659ea535d7c36d40a5": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "9d9a247c6569458abd0dcd6e0d717079": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "9e0bccbc6072461fbf96482b870ed8d5": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "ProgressStyleModel", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "ProgressStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "bar_color": null, + "description_width": "" + } + }, + "a0600fb407034c2d8df6ae5830d601db": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "a0996675df13484aaa519e6ff45c5476": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "HBoxModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HBoxModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HBoxView", + "box_style": "", + "children": [ + "IPY_MODEL_0bfb4937ed5547b3ba464ca47ac77f1a", + "IPY_MODEL_7f59906980724a8b840dec85ce400f89", + "IPY_MODEL_80f3d29327bf429481ad191b1abe556f" + ], + "layout": "IPY_MODEL_6d7c024126ac4c34825fae522234ebca" + } + }, + "a37f2164e11d4e5f851a4a09a12c663c": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "FloatProgressModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "FloatProgressModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "ProgressView", + "bar_style": "success", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_f14106c7f60f411199acf47f530443fd", + "max": 2, + "min": 0, + "orientation": "horizontal", + "style": "IPY_MODEL_bf88ee6dc83648d8a61d75bb4466b1e3", + "value": 2 + } + }, + "b2fdc502d6ee491bb826fd616e35d407": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "b7c439aa6d584c5784b46980050e503d": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "HBoxModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HBoxModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HBoxView", + "box_style": "", + "children": [ + "IPY_MODEL_8aa8805651d34181b1851d302ccc47e2", + "IPY_MODEL_713f1d91e445411288e565a33ce4b271", + "IPY_MODEL_55941e08c602404c9342d00b7ee26918" + ], + "layout": "IPY_MODEL_87da02f5606d404ea242c3bd1f9ac38c" + } + }, + "bf88ee6dc83648d8a61d75bb4466b1e3": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "ProgressStyleModel", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "ProgressStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "bar_color": null, + "description_width": "" + } + }, + "c06e354cb8294e66a3d7590a576571e0": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "c1d37ab1952b4d268d9786b74b6902d7": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "DescriptionStyleModel", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "c3a1b520140444fbb40b7ac789f7ac0e": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "HTMLModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_30885be6a7c84f0f9f02bc2ea11679bc", + "placeholder": "​", + "style": "IPY_MODEL_29178e51df9e47489fff623763b130ed", + "value": " 4.95G/4.95G [00:16<00:00, 216MB/s]" + } + }, + "c7c0c34a71954d6ea976c774573c49c5": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "HBoxModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HBoxModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HBoxView", + "box_style": "", + "children": [ + "IPY_MODEL_0d6ec3bab579406fa4e6fc2b3d6b6998", + "IPY_MODEL_a37f2164e11d4e5f851a4a09a12c663c", + "IPY_MODEL_ef32431228f24a5498810a36b9cf6506" + ], + "layout": "IPY_MODEL_c06e354cb8294e66a3d7590a576571e0" + } + }, + "ce6a3a655d2f4ce2ab351c766568bed5": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "d5e9f339fe7e4ab9955531cc125f071e": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "HTMLModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_6a15f1cf54a141fc9d6bb790366c6bdd", + "placeholder": "​", + "style": "IPY_MODEL_8813b56cd89744b58ace2787206e1501", + "value": "model-00001-of-00002.safetensors: 100%" + } + }, + "d7a00f1f114e4f008f4d5a48c1c69f53": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "d7d4a9f444fe4ebb9135035e2166a3a5": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "FloatProgressModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "FloatProgressModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "ProgressView", + "bar_style": "success", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_2e081a17ddc04104882893a30c902265", + "max": 13489, + "min": 0, + "orientation": "horizontal", + "style": "IPY_MODEL_938f8f60901442f2902eb51e86c27961", + "value": 13489 + } + }, + "e2998c2c6b1f4d489a5e39f2076838e4": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "e5a6d300bbf441b8904aa9afb89e6f31": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "e68cf53b04a845ac9d6f4047600ebc21": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "e7f471604a5a42e095d35d8ad399c6fe": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "eba6048eb694485693656fcbf4a4f297": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "ProgressStyleModel", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "ProgressStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "bar_color": null, + "description_width": "" + } + }, + "edc37210db734d01a8afce596698bb27": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "ef32431228f24a5498810a36b9cf6506": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "HTMLModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_5776e818d9d34e009f95833056522876", + "placeholder": "​", + "style": "IPY_MODEL_06b1a069317041c8a9174c14fdc867bc", + "value": " 2/2 [00:17<00:00,  7.35s/it]" + } + }, + "f14106c7f60f411199acf47f530443fd": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "fa0c528cca744cff8da0a4fa21fdb4b5": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "HTMLModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_e5a6d300bbf441b8904aa9afb89e6f31", + "placeholder": "​", + "style": "IPY_MODEL_88226abe35534278bbd427d8eff0f5f8", + "value": "model.safetensors.index.json: 100%" + } + }, + "fa9cf3e72280417d8711ef7227a95d34": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "FloatProgressModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "FloatProgressModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "ProgressView", + "bar_style": "success", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_edc37210db734d01a8afce596698bb27", + "max": 4945242264, + "min": 0, + "orientation": "horizontal", + "style": "IPY_MODEL_eba6048eb694485693656fcbf4a4f297", + "value": 4945242264 + } + }, + "faf25fd219f24bdbaa2e3202548c97d9": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "DescriptionStyleModel", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "fddbae43ce7f477cafaff89b81e47fc7": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "feb438afda6b4c148a3a62ee7e03da74": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "ProgressStyleModel", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "ProgressStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "bar_color": null, + "description_width": "" + } + } + } + } + }, + "nbformat": 4, + "nbformat_minor": 0 +} From 0a43b0d0e4a4eec8c16b1fe83b304ed0627ffa05 Mon Sep 17 00:00:00 2001 From: innovation64 Date: Wed, 27 Mar 2024 21:58:41 +0800 Subject: [PATCH 03/31] =?UTF-8?q?chore=EF=BC=9A=20update=20=5Ftoctree.yal?= =?UTF-8?q?=20in=20zh-CN?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- notebooks/zh-CN/_toctree.yml | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/notebooks/zh-CN/_toctree.yml b/notebooks/zh-CN/_toctree.yml index b17917d8..37dfa53a 100644 --- a/notebooks/zh-CN/_toctree.yml +++ b/notebooks/zh-CN/_toctree.yml @@ -2,6 +2,8 @@ sections: - local: index title: 开源 AI 指南 (Cookbook) + - local: rag_with_hugging_face_gemma_mongodb + title: 用 Gemma, MongoDB 和开源模型构建 RAG 系统 - local: automatic_embedding_tei_inference_endpoints title: 通过推理端点使用 TEI 自动嵌入 - local: faiss_with_hf_datasets_and_clip @@ -10,6 +12,8 @@ title: 在单个 GPU 上针对自定义代码微调代码 LLM - local: rag_zephyr_langchain title: 用 Hugging Face Zephyr 和 LangChain 针对 Github issues 构建简单的 RAG + - local: rag_llamaindex_librarian + title: 用 LlamaIndex 构建一个 RAG 电子书库智能助手 - local: advanced_rag title: 使用 LangChain 在 HuggingFace 文档上构建高级 RAG - local: rag_evaluation From 15d227cbfef3ccce1ddfaeea59f993895ab5dee2 Mon Sep 17 00:00:00 2001 From: innovation64 Date: Thu, 28 Mar 2024 10:53:51 +0800 Subject: [PATCH 04/31] docs: update semantic_cache_vector_database in zh-CN --- ...emantic_cache_chroma_vector_database.ipynb | 1424 +++++++++++++++++ 1 file changed, 1424 insertions(+) create mode 100644 notebooks/zh-CN/semantic_cache_chroma_vector_database.ipynb diff --git a/notebooks/zh-CN/semantic_cache_chroma_vector_database.ipynb b/notebooks/zh-CN/semantic_cache_chroma_vector_database.ipynb new file mode 100644 index 00000000..2ff7561f --- /dev/null +++ b/notebooks/zh-CN/semantic_cache_chroma_vector_database.ipynb @@ -0,0 +1,1424 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": { + "id": "AVv_M1Dz9TDz" + }, + "source": [ + "# 通过引入语义缓存到 FAISS 中以增强 RAG 系统的性能\n", + "\n", + "_作者:[Pere Martra](https://github.com/peremartra)_\n", + "\n", + "在这个 notebook 中，我们将使用一个现成的模型和 Chroma 数据库来搭建一个常见的 RAG 系统。**但我们会加入一个新功能，就是一个语义缓存系统，它会保存用户的各种问题，并决定是直接用数据库的信息来回答问题，还是用之前保存的问题答案。**\n", + "\n", + "这个语义缓存系统的目的是找出用户提出的问题中哪些是相似的或者是一样的。如果找到了一个之前问过的问题，系统就会直接用缓存里的答案来回答，这样就不用再去数据库里找了。\n", + "\n", + "因为这个系统会考虑问题的实际意思，所以即使问题表达的方式不同，或者有些小错误，比如拼写或句子结构不对，系统也能识别出用户其实是在问同一个问题。\n", + "\n", + "比如，像 **法国的首都是什么？**、**告诉我法国的首都叫什么？** 和 **法国的首都是什么？** 这样的问题，虽然问法不一样，但都是在问同一个事情。\n", + "\n", + "虽然根据问题的不同，模型的回答可能会有点不一样，但基本上从数据库里拿到的信息应该是相同的。这就是为什么我们把缓存系统放在用户和数据库之间，而不是用户和语言模型之间。\n", + "\n", + "\n", + "\n", + "\n" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "5gtBERjX1vFd" + }, + "source": [ + "大多数教程指导你创建一个 RAG 系统，这些教程都是为单个用户设计的，用于在测试环境中运行。换句话说，就是在笔记本中与本地向量数据库交互，以及进行 API 调用或使用本地存储的模型。\n", + "\n", + "当尝试将其中一种模型过渡到生产环境时，这种架构很快就显得不够用了，在生产环境中，它们可能会遇到从几十到成千上万次的重复请求。\n", + "\n", + "提高性能的一种方法是通过一个或多个语义缓存。这个缓存保留了以前请求的结果，并且在解决新请求之前，它会检查是否之前收到过类似的请求。如果是这样，它就不会重新执行过程，而是从缓存中检索信息。\n", + "\n", + "在 RAG 系统中，有两个耗时的点：\n", + "\n", + "* 检索用于构建丰富提示的信息：\n", + "* 调用大型语言模型以获得响应。\n", + "\n", + "在这两点上，都可以实现语义缓存系统，我们甚至可以有两个缓存，每个点一个。\n", + "\n", + "将缓存系统放在模型的响应点可能会导致对获得响应的影响减少。我们的缓存系统可能会将\"用 10 个词解释法国大革命\"和\"用 100 个词解释法国大革命\"视为相同的查询。如果我们的缓存系统存储模型响应，用户可能会认为他们的指令没有被准确地遵循。\n", + "\n", + "但是，两个请求都需要相同的信息来丰富提示。这就是我选择将语义缓存系统放置在用户请求和从向量数据库检索信息之间的主要原因。\n", + "\n", + "然而，这是一个设计决策。根据响应类型和系统请求的不同，它可以被放置在一个点或另一个点。很明显，缓存模型响应会节省最多的时间，但正如我已经解释过的，这样做会牺牲用户对响应的影响。\n", + "\n", + "\n", + "\n" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "uizxY8679TDz" + }, + "source": [ + "# 导入并加载库。\n", + "首先，我们需要安装必要的 Python 包。\n", + "* **[sentence transformers](http:/www.sbert.net/)**。这个库用于将句子转换为固定长度的向量，也称为嵌入。\n", + "* **[xformers](https://github.com/facebookresearch/xformers)**。这是一个提供库和工具的包，以便与 transformers 模型一起使用。我们需要安装它，以避免在处理模型和嵌入时出现错误。\n", + "* **[chromadb](https://www.trychroma.com/)**。这是我们的向量数据库。ChromaDB 易于使用且开源，可能是用于存储嵌入的最常用的向量数据库。\n", + "* **[accelerate](https://github.com/huggingface/accelerate)**。在 GPU 上运行模型的必要条件。\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "execution": { + "iopub.execute_input": "2024-02-29T17:30:10.787688Z", + "iopub.status.busy": "2024-02-29T17:30:10.787382Z", + "iopub.status.idle": "2024-02-29T17:34:12.804579Z", + "shell.execute_reply": "2024-02-29T17:34:12.80338Z", + "shell.execute_reply.started": "2024-02-29T17:30:10.787657Z" + }, + "id": "r1nUzd1u9TD0", + "trusted": true + }, + "outputs": [], + "source": [ + "!pip install -q transformers==4.38.1\n", + "!pip install -q accelerate==0.27.2\n", + "!pip install -q sentence-transformers==2.5.1\n", + "!pip install -q xformers==0.0.24\n", + "!pip install -q chromadb==0.4.24\n", + "!pip install -q datasets==2.17.1" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": { + "execution": { + "iopub.execute_input": "2024-02-29T17:35:23.197598Z", + "iopub.status.busy": "2024-02-29T17:35:23.197205Z", + "iopub.status.idle": "2024-02-29T17:35:23.202259Z", + "shell.execute_reply": "2024-02-29T17:35:23.201404Z", + "shell.execute_reply.started": "2024-02-29T17:35:23.197556Z" + }, + "id": "5jUwC_eE9TD0", + "trusted": true + }, + "outputs": [], + "source": [ + "import numpy as np\n", + "import pandas as pd" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "9P-kYtc79TD1" + }, + "source": [ + "# 加载数据集\n", + "\n", + "由于我们在一个免费且有限的空间中工作，并且只能使用几 GB 的内存，我通过变量 `MAX_ROWS` 限制了从数据集中使用的行数。" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "xZsN8yzUvfjN" + }, + "outputs": [], + "source": [ + "#Login to Hugging Face. It is mandatory to use the Gemma Model,\n", + "#and recommended to acces public models and Datasets.\n", + "from getpass import getpass\n", + "if 'hf_key' not in locals():\n", + " hf_key = getpass(\"Your Hugging Face API Key: \")\n", + "!huggingface-cli login --token $hf_key" + ] + }, + { + "cell_type": "code", + "execution_count": 47, + "metadata": { + "id": "9IVxu-uxtCTw" + }, + "outputs": [], + "source": [ + "from datasets import load_dataset\n", + "\n", + "data = load_dataset(\"keivalya/MedQuad-MedicalQnADataset\", split='train')" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "hmor-i1j9TD1" + }, + "source": [ + "ChromaDB 要求数据具有唯一的标识符。我们可以使用这个语句来创建一个名为**Id**的新列。\n" + ] + }, + { + "cell_type": "code", + "execution_count": 48, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 536 + }, + "id": "WbLf8c7_yHwy", + "outputId": "492eac81-2f7b-4063-f444-405bf489d08e" + }, + "outputs": [ + { + "data": { + "application/vnd.google.colaboratory.intrinsic+json": { + "summary": "{\n \"name\": \"data\",\n \"rows\": 16407,\n \"fields\": [\n {\n \"column\": \"qtype\",\n \"properties\": {\n \"dtype\": \"category\",\n \"num_unique_values\": 16,\n \"samples\": [\n \"susceptibility\",\n \"symptoms\",\n \"information\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"Question\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 14979,\n \"samples\": [\n \"What are the symptoms of Danon disease ?\",\n \"What is (are) Dowling-Degos disease ?\",\n \"What are the genetic changes related to Pearson marrow-pancreas syndrome ?\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"Answer\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 15817,\n \"samples\": [\n \"These resources address the diagnosis or management of glycogen storage disease type III: - Gene Review: Gene Review: Glycogen Storage Disease Type III - Genetic Testing Registry: Glycogen storage disease type III These resources from MedlinePlus offer information about the diagnosis and management of various health conditions: - Diagnostic Tests - Drug Therapy - Surgery and Rehabilitation - Genetic Counseling - Palliative Care\",\n \"Diagnostic Challenges\\n \\nFor doctors, diagnosing chronic fatigue syndrome (CFS) can be complicated by a number of factors:\\n \\n - There's no lab test or biomarker for CFS.\\n - Fatigue and other symptoms of CFS are common to many illnesses.\\n - For some CFS patients, it may not be obvious to doctors that they are ill.\\n - The illness has a pattern of remission and relapse.\\n - Symptoms vary from person to person in type, number, and severity.\\n \\n \\nThese factors have contributed to a low diagnosis rate. Of the one to four million Americans who have CFS, less than 20% have been diagnosed.\\n Exams and Screening Tests for CFS\\n \\nBecause there is no blood test, brain scan, or other lab test to diagnose CFS, the doctor should first rule out other possible causes.\\n \\nIf a patient has had 6 or more consecutive months of severe fatigue that is reported to be unrelieved by sufficient bed rest and that is accompanied by nonspecific symptoms, including flu-like symptoms, generalized pain, and memory problems, the doctor should consider the possibility that the patient may have CFS. Further exams and tests are needed before a diagnosis can be made:\\n \\n - A detailed medical history will be needed and should include a review of medications that could be causing the fatigue and symptoms\\n - A thorough physical and mental status examination will also be needed\\n - A battery of laboratory screening tests will be needed to help identify or rule out other possible causes of the symptoms that could be treated\\n - The doctor may also order additional tests to follow up on results of the initial screening tests\\n \\n \\nA CFS diagnosis requires that the patient has been fatigued for 6 months or more and has 4 of the 8 symptoms for CFS for 6 months or more. If, however, the patient has been fatigued for 6 months or more but does not have four of the eight symptoms, the diagnosis may be idiopathic fatigue.\\n \\nThe complete process for diagnosing CFS can be found here.\\n \\nAdditional information for healthcare professionals on use of tests can be found here.\",\n \"Eating, diet, and nutrition have not been shown to play a role in causing or preventing simple kidney cysts.\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"id\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 4736,\n \"min\": 0,\n \"max\": 16406,\n \"num_unique_values\": 16407,\n \"samples\": [\n 3634,\n 15104,\n 4395\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n }\n ]\n}", + "type": "dataframe", + "variable_name": "data" + }, + "text/html": [ + "\n", + "
\n", + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
qtypeQuestionAnswerid
0susceptibilityWho is at risk for Lymphocytic Choriomeningiti...LCMV infections can occur after exposure to fr...0
1symptomsWhat are the symptoms of Lymphocytic Choriomen...LCMV is most commonly recognized as causing ne...1
2susceptibilityWho is at risk for Lymphocytic Choriomeningiti...Individuals of all ages who come into contact ...2
3exams and testsHow to diagnose Lymphocytic Choriomeningitis (...During the first phase of the disease, the mos...3
4treatmentWhat are the treatments for Lymphocytic Chorio...Aseptic meningitis, encephalitis, or meningoen...4
5preventionHow to prevent Lymphocytic Choriomeningitis (L...LCMV infection can be prevented by avoiding co...5
6informationWhat is (are) Parasites - Cysticercosis ?Cysticercosis is an infection caused by the la...6
7susceptibilityWho is at risk for Parasites - Cysticercosis? ?Cysticercosis is an infection caused by the la...7
8exams and testsHow to diagnose Parasites - Cysticercosis ?If you think that you may have cysticercosis, ...8
9treatmentWhat are the treatments for Parasites - Cystic...Some people with cysticercosis do not need to ...9
\n", + "
\n", + "
\n", + "\n", + "
\n", + " \n", + "\n", + " \n", + "\n", + " \n", + "
\n", + "\n", + "\n", + "
\n", + " \n", + "\n", + "\n", + "\n", + " \n", + "
\n", + "
\n", + "
\n" + ], + "text/plain": [ + " qtype Question \\\n", + "0 susceptibility Who is at risk for Lymphocytic Choriomeningiti... \n", + "1 symptoms What are the symptoms of Lymphocytic Choriomen... \n", + "2 susceptibility Who is at risk for Lymphocytic Choriomeningiti... \n", + "3 exams and tests How to diagnose Lymphocytic Choriomeningitis (... \n", + "4 treatment What are the treatments for Lymphocytic Chorio... \n", + "5 prevention How to prevent Lymphocytic Choriomeningitis (L... \n", + "6 information What is (are) Parasites - Cysticercosis ? \n", + "7 susceptibility Who is at risk for Parasites - Cysticercosis? ? \n", + "8 exams and tests How to diagnose Parasites - Cysticercosis ? \n", + "9 treatment What are the treatments for Parasites - Cystic... \n", + "\n", + " Answer id \n", + "0 LCMV infections can occur after exposure to fr... 0 \n", + "1 LCMV is most commonly recognized as causing ne... 1 \n", + "2 Individuals of all ages who come into contact ... 2 \n", + "3 During the first phase of the disease, the mos... 3 \n", + "4 Aseptic meningitis, encephalitis, or meningoen... 4 \n", + "5 LCMV infection can be prevented by avoiding co... 5 \n", + "6 Cysticercosis is an infection caused by the la... 6 \n", + "7 Cysticercosis is an infection caused by the la... 7 \n", + "8 If you think that you may have cysticercosis, ... 8 \n", + "9 Some people with cysticercosis do not need to ... 9 " + ] + }, + "execution_count": 48, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "data = data.to_pandas()\n", + "data[\"id\"]=data.index\n", + "data.head(10)" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": { + "execution": { + "iopub.execute_input": "2024-02-29T17:35:25.528374Z", + "iopub.status.busy": "2024-02-29T17:35:25.527688Z", + "iopub.status.idle": "2024-02-29T17:35:25.709895Z", + "shell.execute_reply": "2024-02-29T17:35:25.709127Z", + "shell.execute_reply.started": "2024-02-29T17:35:25.528341Z" + }, + "id": "DZf0zCI29TD1", + "trusted": true + }, + "outputs": [], + "source": [ + "MAX_ROWS = 15000\n", + "DOCUMENT=\"Answer\"\n", + "TOPIC=\"qtype\"" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "metadata": { + "execution": { + "iopub.execute_input": "2024-02-29T17:35:29.184342Z", + "iopub.status.busy": "2024-02-29T17:35:29.183979Z", + "iopub.status.idle": "2024-02-29T17:35:29.189229Z", + "shell.execute_reply": "2024-02-29T17:35:29.1881Z", + "shell.execute_reply.started": "2024-02-29T17:35:29.184313Z" + }, + "id": "Mkoj9IrZ9TD1", + "trusted": true + }, + "outputs": [], + "source": [ + "#Because it is just a sample we select a small portion of News.\n", + "subset_data = data.head(MAX_ROWS)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "rZHg_Qh69TD1" + }, + "source": [ + "# 导入并配置向量数据库\n", + "\n", + "为了存储信息，我选择使用 ChromaDB，这是最知名且广泛使用的开源向量数据库之一。\n", + "\n", + "首先我们需要导入 ChromaDB。\n" + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "metadata": { + "execution": { + "iopub.execute_input": "2024-02-29T17:35:31.849551Z", + "iopub.status.busy": "2024-02-29T17:35:31.849199Z", + "iopub.status.idle": "2024-02-29T17:35:32.31736Z", + "shell.execute_reply": "2024-02-29T17:35:32.316617Z", + "shell.execute_reply.started": "2024-02-29T17:35:31.849525Z" + }, + "id": "npJhuZQw9TD1", + "trusted": true + }, + "outputs": [], + "source": [ + "import chromadb" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "8okox5C89TD1" + }, + "source": [ + "现在我们只需要指定存储向量数据库的路径。" + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "metadata": { + "execution": { + "iopub.execute_input": "2024-02-29T17:35:34.410646Z", + "iopub.status.busy": "2024-02-29T17:35:34.410268Z", + "iopub.status.idle": "2024-02-29T17:35:34.872817Z", + "shell.execute_reply": "2024-02-29T17:35:34.872039Z", + "shell.execute_reply.started": "2024-02-29T17:35:34.410614Z" + }, + "id": "9yK6y0hm9TD1", + "trusted": true + }, + "outputs": [], + "source": [ + "chroma_client = chromadb.PersistentClient(path=\"/path/to/persist/directory\")" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "7MhMwk3J9TD1" + }, + "source": [ + "# 填充和查询 ChromaDB 数据库\n", + "\n", + "ChromaDB 中的数据存储在集合中。如果集合已存在，我们需要删除它。\n", + "在接下来的行中，我们通过调用上面创建的 `chroma_client` 中的 `create_collection` 函数来创建集合。\n" + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "metadata": { + "execution": { + "iopub.execute_input": "2024-02-29T17:35:36.116012Z", + "iopub.status.busy": "2024-02-29T17:35:36.1156Z", + "iopub.status.idle": "2024-02-29T17:35:36.16922Z", + "shell.execute_reply": "2024-02-29T17:35:36.168504Z", + "shell.execute_reply.started": "2024-02-29T17:35:36.115977Z" + }, + "id": "kRCsunE19TD1", + "trusted": true + }, + "outputs": [], + "source": [ + "collection_name = \"news_collection\"\n", + "if len(chroma_client.list_collections()) > 0 and collection_name in [chroma_client.list_collections()[0].name]:\n", + " chroma_client.delete_collection(name=collection_name)\n", + "\n", + "collection = chroma_client.create_collection(name=collection_name)\n" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "rdEtcETr9TD2" + }, + "source": [ + "现在我们准备好使用 `add` 函数将数据添加到集合中。这个函数需要三个关键信息：\n", + "\n", + "* 在 **文档** 中，我们存储数据集中 `Answer` 列的内容。\n", + "* 在 **元数据** 中，我们可以提供一个主题列表。我使用了 `qtype` 列中的值。\n", + "* 在 **id** 中，我们需要为每一行提供一个唯一的标识符。我使用 `MAX_ROWS` 的范围来创建ID。" + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "execution": { + "iopub.execute_input": "2024-02-29T17:35:38.051601Z", + "iopub.status.busy": "2024-02-29T17:35:38.051179Z", + "iopub.status.idle": "2024-02-29T17:36:38.612836Z", + "shell.execute_reply": "2024-02-29T17:36:38.611814Z", + "shell.execute_reply.started": "2024-02-29T17:35:38.051569Z" + }, + "id": "4dDoqJE79TD2", + "outputId": "36f579dc-ec60-48b1-807a-1e68113cc9f4", + "trusted": true + }, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "/root/.cache/chroma/onnx_models/all-MiniLM-L6-v2/onnx.tar.gz: 100%|██████████| 79.3M/79.3M [00:01<00:00, 68.1MiB/s]\n" + ] + } + ], + "source": [ + "collection.add(\n", + " documents=subset_data[DOCUMENT].tolist(),\n", + " metadatas=[{TOPIC: topic} for topic in subset_data[TOPIC].tolist()],\n", + " ids=[f\"id{x}\" for x in range(MAX_ROWS)],\n", + ")" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "du6-iuUisRkM" + }, + "source": [ + "一旦我们在数据库中有了信息，我们就可以查询它，并请求符合我们需求的数据。搜索是在文档内容内部进行的，它不会查找确切的单词或短语。结果将基于搜索词与文档内容之间的相似性。\n", + "\n", + "元数据在初始搜索过程中并不直接参与，它可以在检索后用于过滤或细化结果，从而实现进一步的定制和精确性。\n", + "\n", + "让我们定义一个函数来查询 ChromaDB 数据库。\n" + ] + }, + { + "cell_type": "code", + "execution_count": 12, + "metadata": { + "execution": { + "iopub.execute_input": "2024-02-29T17:36:38.616047Z", + "iopub.status.busy": "2024-02-29T17:36:38.615302Z", + "iopub.status.idle": "2024-02-29T17:36:38.620516Z", + "shell.execute_reply": "2024-02-29T17:36:38.619561Z", + "shell.execute_reply.started": "2024-02-29T17:36:38.616008Z" + }, + "id": "UjdhZ4MJ9TD2", + "trusted": true + }, + "outputs": [], + "source": [ + "def query_database(query_text, n_results=10):\n", + " results = collection.query(query_texts=query_text, n_results=n_results )\n", + " return results" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "CL0Crl3x9TD2" + }, + "source": [ + "## 创建语义缓存系统\n", + "为了实现缓存系统，我们将使用 Faiss 库，该库允许在内存中存储嵌入。这和 Chroma 做的事情很相似，但没有其持久性。\n", + "\n", + "为此，我们将创建一个名为 `semantic_cache` 的类，它将使用自己的编码器，并为用户提供执行查询所需的函数。\n", + "\n", + "在这个类中，我们首先查询使用 Faiss 实现的缓存，其中包含以前的请求，如果返回的结果超过了一个指定的阈值，它将返回缓存的内容。否则，它将从 Chroma 数据库获取结果。\n", + "缓存存储在一个 .json 文件中。\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "execution": { + "iopub.execute_input": "2024-02-29T17:36:38.621968Z", + "iopub.status.busy": "2024-02-29T17:36:38.621655Z", + "iopub.status.idle": "2024-02-29T17:36:51.313356Z", + "shell.execute_reply": "2024-02-29T17:36:51.312232Z", + "shell.execute_reply.started": "2024-02-29T17:36:38.621936Z" + }, + "id": "6OzUbRUe9TD2", + "trusted": true + }, + "outputs": [], + "source": [ + "!pip install -q faiss-cpu==1.8.0" + ] + }, + { + "cell_type": "code", + "execution_count": 14, + "metadata": { + "id": "0yGE4cTEp3QJ" + }, + "outputs": [], + "source": [ + "import faiss\n", + "from sentence_transformers import SentenceTransformer\n", + "import time\n", + "import json" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "yi_riXHhcLy0" + }, + "source": [ + "下面的 `init_cache()` 函数初始化了语义缓存。\n", + "\n", + "它使用了 FlatLS 索引，这可能不是最快的，但对于小数据集来说是理想的。如果我们需要根据数据的具体内容和大小来选择缓存（临时存储）数据的方式，我们还可以考虑使用其他的索引方法，比如 HNSW 或 IVF。\n", + "\n", + "我选择这个索引是因为它与示例非常契合。它可以用于高维向量，消耗的内存最少，并且在小数据集上表现良好。\n", + "\n", + "下面概述了 Faiss 可用的各种索引的关键特性。\n", + "\n", + "* FlatL2 或 FlatIP。非常适合小数据集，可能不是最快的，但其内存消耗并不过分。\n", + "* LSH。它在小数据集上工作效果很好，并且推荐用于最多 128 维的向量。\n", + "* HNSW。非常快，但需要大量的 RAM。\n", + "* IVF。在大数据集上工作良好，而且不会消耗太多内存或影响性能。\n", + "\n", + "关于 Faiss 可用的不同索引的更多信息可以在以下链接中找到：https://github.com/facebookresearch/faiss/wiki/Guidelines-to-choose-an-index\n", + "\n" + ] + }, + { + "cell_type": "code", + "execution_count": 15, + "metadata": { + "id": "9poNBxbPl7xE" + }, + "outputs": [], + "source": [ + "def init_cache():\n", + " index = faiss.IndexFlatL2(768)\n", + " if index.is_trained:\n", + " print('Index trained')\n", + "\n", + " # Initialize Sentence Transformer model\n", + " encoder = SentenceTransformer('all-mpnet-base-v2')\n", + "\n", + " return index, encoder" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "_uZzX60odo1U" + }, + "source": [ + "在 `retrieve_cache` 函数中，.json 文件从磁盘中被检索出来，以便在需要跨会话重用缓存时使用。" + ] + }, + { + "cell_type": "code", + "execution_count": 16, + "metadata": { + "id": "FDJJ86TSp5CO" + }, + "outputs": [], + "source": [ + "def retrieve_cache(json_file):\n", + " try:\n", + " with open(json_file, 'r') as file:\n", + " cache = json.load(file)\n", + " except FileNotFoundError:\n", + " cache = {'questions': [], 'embeddings': [], 'answers': [], 'response_text': []}\n", + "\n", + " return cache" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "3uO-12UIdtSD" + }, + "source": [ + "`store_cache` 函数将包含缓存数据的文件保存到磁盘上。" + ] + }, + { + "cell_type": "code", + "execution_count": 17, + "metadata": { + "id": "jx1CiKOcwKGn" + }, + "outputs": [], + "source": [ + "def store_cache(json_file, cache):\n", + " with open(json_file, 'w') as file:\n", + " json.dump(cache, file)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "t9AdmnhQd2E8" + }, + "source": [ + "这些函数将在 `SemanticCache` 类中使用，该类包括搜索函数及其初始化函数。\n", + "\n", + "尽管 `ask` 函数的代码量相当大，但它的目的非常直接。它在缓存中查找与用户刚刚提出的问题最接近的问题。\n", + "\n", + "然后，检查它是否在指定的阈值内。如果是肯定的，它直接从缓存中返回响应；否则，它调用 `query_database` 函数从 ChromaDB 检索数据。\n", + "\n", + "我使用了欧几里得距离而不是广泛应用于向量比较的余弦距离。这个选择是基于欧几里得距离是 Faiss 默认使用的度量标准。尽管也可以计算余弦距离，但这样做会增加复杂性，可能不会显著有助于最终结果。\n" + ] + }, + { + "cell_type": "code", + "execution_count": 51, + "metadata": { + "execution": { + "iopub.execute_input": "2024-02-29T17:36:51.31678Z", + "iopub.status.busy": "2024-02-29T17:36:51.316449Z", + "iopub.status.idle": "2024-02-29T17:36:55.197427Z", + "shell.execute_reply": "2024-02-29T17:36:55.196616Z", + "shell.execute_reply.started": "2024-02-29T17:36:51.316746Z" + }, + "id": "t_HVtwww9TD2", + "trusted": true + }, + "outputs": [], + "source": [ + "class semantic_cache:\n", + " def __init__(self, json_file=\"cache_file.json\", thresold=0.35):\n", + " # Initialize Faiss index with Euclidean distance\n", + " self.index, self.encoder = init_cache()\n", + "\n", + " # Set Euclidean distance threshold\n", + " # a distance of 0 means identicals sentences\n", + " # We only return from cache sentences under this thresold\n", + " self.euclidean_threshold = thresold\n", + "\n", + " self.json_file = json_file\n", + " self.cache = retrieve_cache(self.json_file)\n", + "\n", + " def ask(self, question: str) -> str:\n", + " # Method to retrieve an answer from the cache or generate a new one\n", + " start_time = time.time()\n", + " try:\n", + " #First we obtain the embeddings corresponding to the user question\n", + " embedding = self.encoder.encode([question])\n", + "\n", + " # Search for the nearest neighbor in the index\n", + " self.index.nprobe = 8\n", + " D, I = self.index.search(embedding, 1)\n", + "\n", + " if D[0] >= 0:\n", + " if I[0][0] >= 0 and D[0][0] <= self.euclidean_threshold:\n", + " row_id = int(I[0][0])\n", + "\n", + " print('Answer recovered from Cache. ')\n", + " print(f'{D[0][0]:.3f} smaller than {self.euclidean_threshold}')\n", + " print(f'Found cache in row: {row_id} with score {D[0][0]:.3f}')\n", + " print(f'response_text: ' + self.cache['response_text'][row_id])\n", + "\n", + " end_time = time.time()\n", + " elapsed_time = end_time - start_time\n", + " print(f\"Time taken: {elapsed_time:.3f} seconds\")\n", + " return self.cache['response_text'][row_id]\n", + "\n", + " # Handle the case when there are not enough results\n", + " # or Euclidean distance is not met, asking to chromaDB.\n", + " answer = query_database([question], 1)\n", + " response_text = answer['documents'][0][0]\n", + "\n", + " self.cache['questions'].append(question)\n", + " self.cache['embeddings'].append(embedding[0].tolist())\n", + " self.cache['answers'].append(answer)\n", + " self.cache['response_text'].append(response_text)\n", + "\n", + " print('Answer recovered from ChromaDB. ')\n", + " print(f'response_text: {response_text}')\n", + "\n", + " self.index.add(embedding)\n", + " store_cache(self.json_file, self.cache)\n", + " end_time = time.time()\n", + " elapsed_time = end_time - start_time\n", + " print(f\"Time taken: {elapsed_time:.3f} seconds\")\n", + "\n", + " return response_text\n", + " except Exception as e:\n", + " raise RuntimeError(f\"Error during 'ask' method: {e}\")\n" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "UBWTqGM7i71N" + }, + "source": [ + "### 测试 semantic_cache 类。\n" + ] + }, + { + "cell_type": "code", + "execution_count": 52, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "JH8s8eUtCMIS", + "outputId": "c613bbfc-9f84-4a96-cd39-45972e69c15b" + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Index trained\n" + ] + } + ], + "source": [ + "# Initialize the cache.\n", + "cache = semantic_cache('4cache.json')" + ] + }, + { + "cell_type": "code", + "execution_count": 53, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "mKqKLfDe_8bC", + "outputId": "8a92ed95-c822-4382-c6db-d9de289341af" + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Answer recovered from ChromaDB. \n", + "response_text: Summary : Shots may hurt a little, but the diseases they can prevent are a lot worse. Some are even life-threatening. Immunization shots, or vaccinations, are essential. They protect against things like measles, mumps, rubella, hepatitis B, polio, tetanus, diphtheria, and pertussis (whooping cough). Immunizations are important for adults as well as children. Your immune system helps your body fight germs by producing substances to combat them. Once it does, the immune system \"remembers\" the germ and can fight it again. Vaccines contain germs that have been killed or weakened. When given to a healthy person, the vaccine triggers the immune system to respond and thus build immunity. Before vaccines, people became immune only by actually getting a disease and surviving it. Immunizations are an easier and less risky way to become immune. NIH: National Institute of Allergy and Infectious Diseases\n", + "Time taken: 0.057 seconds\n" + ] + } + ], + "source": [ + "results = cache.ask(\"How do vaccines work?\")" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "dP7H6TypknLN" + }, + "source": [ + "正如预期的那样，这个响应是从 ChromaDB 获取的。然后，该类将其存储在缓存中。\n", + "\n", + "现在，如果我们发送一个完全不同的问题，响应也应该从 ChromaDB 中检索。这是因为先前存储的问题与当前问题如此不同，以至于它在欧几里得距离方面会超过指定的阈值。\n" + ] + }, + { + "cell_type": "code", + "execution_count": 54, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "execution": { + "iopub.execute_input": "2024-02-29T17:37:15.335593Z", + "iopub.status.busy": "2024-02-29T17:37:15.335288Z", + "iopub.status.idle": "2024-02-29T17:37:17.320691Z", + "shell.execute_reply": "2024-02-29T17:37:17.319671Z", + "shell.execute_reply.started": "2024-02-29T17:37:15.335566Z" + }, + "id": "CvJykqVf9TD2", + "outputId": "7137919e-e417-47b3-a638-18026b3edfe6", + "trusted": true + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Answer recovered from ChromaDB. \n", + "response_text: Sydenham chorea (SD) is a neurological disorder of childhood resulting from infection via Group A beta-hemolytic streptococcus (GABHS), the bacterium that causes rheumatic fever. SD is characterized by rapid, irregular, and aimless involuntary movements of the arms and legs, trunk, and facial muscles. It affects girls more often than boys and typically occurs between 5 and 15 years of age. Some children will have a sore throat several weeks before the symptoms begin, but the disorder can also strike up to 6 months after the fever or infection has cleared. Symptoms can appear gradually or all at once, and also may include uncoordinated movements, muscular weakness, stumbling and falling, slurred speech, difficulty concentrating and writing, and emotional instability. The symptoms of SD can vary from a halting gait and slight grimacing to involuntary movements that are frequent and severe enough to be incapacitating. The random, writhing movements of chorea are caused by an auto-immune reaction to the bacterium that interferes with the normal function of a part of the brain (the basal ganglia) that controls motor movements. Due to better sanitary conditions and the use of antibiotics to treat streptococcal infections, rheumatic fever, and consequently SD, are rare in North America and Europe. The disease can still be found in developing nations.\n", + "Time taken: 0.082 seconds\n" + ] + } + ], + "source": [ + "\n", + "results = cache.ask(\"Explain briefly what is a Sydenham chorea\")" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "8aPWvU64lxOU" + }, + "source": [ + "完美，语义缓存系统正如预期那样运行。\n", + "\n", + "让我们继续用一个非常类似于我们刚才问的问题来测试它。\n", + "\n", + "在这种情况下，响应应该直接来自缓存，而不需要访问 ChromaDB 数据库。\n" + ] + }, + { + "cell_type": "code", + "execution_count": 55, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "execution": { + "iopub.execute_input": "2024-02-29T17:37:17.328926Z", + "iopub.status.busy": "2024-02-29T17:37:17.32865Z", + "iopub.status.idle": "2024-02-29T17:37:17.463363Z", + "shell.execute_reply": "2024-02-29T17:37:17.462397Z", + "shell.execute_reply.started": "2024-02-29T17:37:17.328902Z" + }, + "id": "9_5IcGB-9TD2", + "outputId": "13563a7d-01f7-47d1-c345-6ad128f303c3", + "trusted": true + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Answer recovered from Cache. \n", + "0.028 smaller than 0.35\n", + "Found cache in row: 1 with score 0.028\n", + "response_text: Sydenham chorea (SD) is a neurological disorder of childhood resulting from infection via Group A beta-hemolytic streptococcus (GABHS), the bacterium that causes rheumatic fever. SD is characterized by rapid, irregular, and aimless involuntary movements of the arms and legs, trunk, and facial muscles. It affects girls more often than boys and typically occurs between 5 and 15 years of age. Some children will have a sore throat several weeks before the symptoms begin, but the disorder can also strike up to 6 months after the fever or infection has cleared. Symptoms can appear gradually or all at once, and also may include uncoordinated movements, muscular weakness, stumbling and falling, slurred speech, difficulty concentrating and writing, and emotional instability. The symptoms of SD can vary from a halting gait and slight grimacing to involuntary movements that are frequent and severe enough to be incapacitating. The random, writhing movements of chorea are caused by an auto-immune reaction to the bacterium that interferes with the normal function of a part of the brain (the basal ganglia) that controls motor movements. Due to better sanitary conditions and the use of antibiotics to treat streptococcal infections, rheumatic fever, and consequently SD, are rare in North America and Europe. The disease can still be found in developing nations.\n", + "Time taken: 0.019 seconds\n" + ] + } + ], + "source": [ + "results = cache.ask(\"Briefly explain me what is a Sydenham chorea.\")" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "M4H8RoXFqdwE" + }, + "source": [ + "这两个问题非常相似，它们的欧几里得距离非常小，几乎就像它们是相同的。\n", + "\n", + "现在，让我们尝试另一个问题，这次稍微有些不同，观察系统的表现。" + ] + }, + { + "cell_type": "code", + "execution_count": 56, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "ysj5P_MBCqju", + "outputId": "d4639f73-dc7e-4c25-93ba-2a8c66dc7c61" + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Answer recovered from Cache. \n", + "0.228 smaller than 0.35\n", + "Found cache in row: 1 with score 0.228\n", + "response_text: Sydenham chorea (SD) is a neurological disorder of childhood resulting from infection via Group A beta-hemolytic streptococcus (GABHS), the bacterium that causes rheumatic fever. SD is characterized by rapid, irregular, and aimless involuntary movements of the arms and legs, trunk, and facial muscles. It affects girls more often than boys and typically occurs between 5 and 15 years of age. Some children will have a sore throat several weeks before the symptoms begin, but the disorder can also strike up to 6 months after the fever or infection has cleared. Symptoms can appear gradually or all at once, and also may include uncoordinated movements, muscular weakness, stumbling and falling, slurred speech, difficulty concentrating and writing, and emotional instability. The symptoms of SD can vary from a halting gait and slight grimacing to involuntary movements that are frequent and severe enough to be incapacitating. The random, writhing movements of chorea are caused by an auto-immune reaction to the bacterium that interferes with the normal function of a part of the brain (the basal ganglia) that controls motor movements. Due to better sanitary conditions and the use of antibiotics to treat streptococcal infections, rheumatic fever, and consequently SD, are rare in North America and Europe. The disease can still be found in developing nations.\n", + "Time taken: 0.016 seconds\n" + ] + } + ], + "source": [ + "question_def = \"Write in 20 words what is a Sydenham chorea.\"\n", + "results = cache.ask(question_def)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "MFzXsQwB9TD3" + }, + "source": [ + "我们观察到欧几里得距离已经增加，但它仍然在指定的阈值范围内。因此，它继续直接从缓存中返回响应。" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "Ot3wrq0p9TD3" + }, + "source": [ + "# 加载模型并创建提示\n", + "\n", + "是时候使用 **transformers** 库了，这是[ hugging face ](https://huggingface.co/)最著名的库，用于处理语言模型。\n", + "\n", + "我们将导入：\n", + "* **Autotokenizer**：这是一个实用程序类，用于标记化与各种预训练语言模型兼容的文本输入。\n", + "* **AutoModelForCausalLM**：它提供了一个接口，用于预训练的语言模型，特别适用于使用因果语言建模（例如，GPT 模型）的语言生成任务，或者是这个 Notebook 中使用的模型 [Gemma-2b-it](https://huggingface.co/google/gemma-2b-it)。\n", + "请随意测试 [不同的模型](https://huggingface.co/models?pipeline_tag=text-generation&sort=trending)，你需要搜索训练用于文本生成的 NLP 模型。\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "execution": { + "iopub.execute_input": "2024-02-29T17:40:32.797669Z", + "iopub.status.busy": "2024-02-29T17:40:32.797334Z", + "iopub.status.idle": "2024-02-29T17:40:44.152114Z", + "shell.execute_reply": "2024-02-29T17:40:44.151056Z", + "shell.execute_reply.started": "2024-02-29T17:40:32.797635Z" + }, + "id": "tdxiKqjT9TD3", + "trusted": true + }, + "outputs": [], + "source": [ + "!pip install torch" + ] + }, + { + "cell_type": "code", + "execution_count": 25, + "metadata": { + "execution": { + "iopub.execute_input": "2024-02-29T17:40:44.15434Z", + "iopub.status.busy": "2024-02-29T17:40:44.153914Z", + "iopub.status.idle": "2024-02-29T17:40:44.160144Z", + "shell.execute_reply": "2024-02-29T17:40:44.159154Z", + "shell.execute_reply.started": "2024-02-29T17:40:44.154292Z" + }, + "id": "pIDMTCnH9TD7", + "trusted": true + }, + "outputs": [], + "source": [ + "from torch import cuda, torch\n", + "#In a MAC Silicon the device must be 'mps'\n", + "# device = torch.device('mps') #to use with MAC Silicon\n", + "device = f'cuda:{cuda.current_device()}' if cuda.is_available() else 'cpu'" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "execution": { + "iopub.execute_input": "2024-02-29T17:41:25.628804Z", + "iopub.status.busy": "2024-02-29T17:41:25.628412Z", + "iopub.status.idle": "2024-02-29T17:41:30.202141Z", + "shell.execute_reply": "2024-02-29T17:41:30.200774Z", + "shell.execute_reply.started": "2024-02-29T17:41:25.628766Z" + }, + "id": "CU2T4lp-9TD7", + "trusted": true + }, + "outputs": [], + "source": [ + "from transformers import AutoTokenizer, AutoModelForCausalLM\n", + "\n", + "model_id = \"google/gemma-2b-it\"\n", + "tokenizer = AutoTokenizer.from_pretrained(model_id)\n", + "model = AutoModelForCausalLM.from_pretrained(model_id,\n", + " device_map=\"cuda\",\n", + " torch_dtype=torch.bfloat16)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "GzHuFrAX9TD7" + }, + "source": [ + "## 创建扩展提示\n", + "\n", + "为了创建提示，我们使用从查询 'semantic_cache' 类得到的结果以及用户提出的问题。\n", + "\n", + "提示有两部分，**相关上下文**是从数据库中恢复的信息，以及**用户的问题**。\n", + "\n", + "我们只需要将这两部分放在一起来创建提示，然后将其发送给模型。" + ] + }, + { + "cell_type": "code", + "execution_count": 44, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 209 + }, + "id": "TdjbfAHhFuhS", + "outputId": "4090da66-328e-478e-c2d7-1957597f8786" + }, + "outputs": [ + { + "data": { + "application/vnd.google.colaboratory.intrinsic+json": { + "type": "string" + }, + "text/plain": [ + "\"Relevant context: Sydenham chorea (SD) is a neurological disorder of childhood resulting from infection via Group A beta-hemolytic streptococcus (GABHS), the bacterium that causes rheumatic fever. SD is characterized by rapid, irregular, and aimless involuntary movements of the arms and legs, trunk, and facial muscles. It affects girls more often than boys and typically occurs between 5 and 15 years of age. Some children will have a sore throat several weeks before the symptoms begin, but the disorder can also strike up to 6 months after the fever or infection has cleared. Symptoms can appear gradually or all at once, and also may include uncoordinated movements, muscular weakness, stumbling and falling, slurred speech, difficulty concentrating and writing, and emotional instability. The symptoms of SD can vary from a halting gait and slight grimacing to involuntary movements that are frequent and severe enough to be incapacitating. The random, writhing movements of chorea are caused by an auto-immune reaction to the bacterium that interferes with the normal function of a part of the brain (the basal ganglia) that controls motor movements. Due to better sanitary conditions and the use of antibiotics to treat streptococcal infections, rheumatic fever, and consequently SD, are rare in North America and Europe. The disease can still be found in developing nations.\\n\\n The user's question: Write in 20 words what is a Sydenham chorea.\"" + ] + }, + "execution_count": 44, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "prompt_template = f\"Relevant context: {results}\\n\\n The user's question: {question_def}\"\n", + "prompt_template" + ] + }, + { + "cell_type": "code", + "execution_count": 45, + "metadata": { + "id": "DmYAcXEEECnz" + }, + "outputs": [], + "source": [ + "input_ids = tokenizer(prompt_template, return_tensors=\"pt\").to(\"cuda\")" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "S-QXeuJ09TD8" + }, + "source": [ + "现在剩下的就是将提示发送给模型，等待它的响应！\n" + ] + }, + { + "cell_type": "code", + "execution_count": 46, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "lheL8vHpEMDD", + "outputId": "b646d648-b88d-4a29-ab30-427d00296255" + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Relevant context: Sydenham chorea (SD) is a neurological disorder of childhood resulting from infection via Group A beta-hemolytic streptococcus (GABHS), the bacterium that causes rheumatic fever. SD is characterized by rapid, irregular, and aimless involuntary movements of the arms and legs, trunk, and facial muscles. It affects girls more often than boys and typically occurs between 5 and 15 years of age. Some children will have a sore throat several weeks before the symptoms begin, but the disorder can also strike up to 6 months after the fever or infection has cleared. Symptoms can appear gradually or all at once, and also may include uncoordinated movements, muscular weakness, stumbling and falling, slurred speech, difficulty concentrating and writing, and emotional instability. The symptoms of SD can vary from a halting gait and slight grimacing to involuntary movements that are frequent and severe enough to be incapacitating. The random, writhing movements of chorea are caused by an auto-immune reaction to the bacterium that interferes with the normal function of a part of the brain (the basal ganglia) that controls motor movements. Due to better sanitary conditions and the use of antibiotics to treat streptococcal infections, rheumatic fever, and consequently SD, are rare in North America and Europe. The disease can still be found in developing nations.\n", + "\n", + " The user's question: Write in 20 words what is a Sydenham chorea.\n", + "\n", + "Sure, here is a 20-word answer:\n", + "\n", + "Sydenham chorea is a neurological disorder of childhood resulting from infection via Group A beta-hemolytic streptococcus (GABHS).\n" + ] + } + ], + "source": [ + "outputs = model.generate(**input_ids,\n", + " max_new_tokens=256)\n", + "print(tokenizer.decode(outputs[0]))" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "execution": { + "iopub.execute_input": "2023-07-12T22:01:56.993351Z", + "iopub.status.busy": "2023-07-12T22:01:56.992775Z", + "iopub.status.idle": "2023-07-12T22:01:57.001309Z", + "shell.execute_reply": "2023-07-12T22:01:56.999431Z", + "shell.execute_reply.started": "2023-07-12T22:01:56.993305Z" + }, + "id": "Uo7lGXBV9TD8" + }, + "source": [ + "# 结论\n", + "\n", + "在访问 ChromaDB 和直接访问缓存之间，数据检索时间减少了 50%。然而，在更大的项目中，这种差异会增加，导致性能提升达到 90-95%。\n", + "\n", + "我们在 Chroma 中的数据非常少，只有一个缓存类的实例。通常，缓存系统背后的数据要大得多，可能不仅仅是对向量数据库的查询，而是来自各种来源。\n", + "\n", + "通常会有多个缓存类的实例，通常基于用户类型，因为共享共同特征的用户之间的问题往往更容易重复。\n", + "\n", + "总之，我们创建了一个非常简单的 RAG 系统，并通过在用户的问题和获取创建丰富提示所需信息之间增加一个语义缓存层来增强它。\n" + ] + } + ], + "metadata": { + "accelerator": "GPU", + "colab": { + "gpuType": "T4", + "machine_shape": "hm", + "provenance": [] + }, + "kaggle": { + "accelerator": "gpu", + "dataSources": [ + { + "datasetId": 3496946, + "sourceId": 6104553, + "sourceType": "datasetVersion" + } + ], + "dockerImageVersionId": 30527, + "isGpuEnabled": true, + "isInternetEnabled": true, + "language": "python", + "sourceType": "notebook" + }, + "kernelspec": { + "display_name": "Python 3", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.10.12" + } + }, + "nbformat": 4, + "nbformat_minor": 0 +} From 8a47b73024af81db6a0975f3a21582c9bcd9bc9c Mon Sep 17 00:00:00 2001 From: innovation64 Date: Thu, 28 Mar 2024 14:38:51 +0800 Subject: [PATCH 05/31] docs: update issues_in_text_dataset in zh-CN --- notebooks/zh-CN/issues_in_text_dataset.ipynb | 3364 ++++++++++++++++++ 1 file changed, 3364 insertions(+) create mode 100644 notebooks/zh-CN/issues_in_text_dataset.ipynb diff --git a/notebooks/zh-CN/issues_in_text_dataset.ipynb b/notebooks/zh-CN/issues_in_text_dataset.ipynb new file mode 100644 index 00000000..f7c05c46 --- /dev/null +++ b/notebooks/zh-CN/issues_in_text_dataset.ipynb @@ -0,0 +1,3364 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": { + "id": "pw6cvzTocw4G" + }, + "source": [ + "# 使用 Cleanlab 检测文本数据集中的问题\n", + "\n" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "0yPBE0Xccw4J" + }, + "source": [ + "作者: [Aravind Putrevu](https://huggingface.co/aravindputrevu)\n", + "\n", + "在这个 5 分钟的快速入门教程中，我们将使用 Cleanlab 检测一个由在线银行（文本）客户服务请求组成的意图分类数据集中的各种问题。我们考虑的是 [Banking77-OOS数据集](https://arxiv.org/abs/2106.04564) 的一个子集，包含 1,000 个客户服务请求，根据它们的意图被分类为 10 个类别（你可以在任何文本分类数据集上运行相同的代码）。[Cleanlab](https://github.com/cleanlab/cleanlab)自动识别我们数据集中的坏例子，包括错误标记的数据、范围外的示例（离群值）或其他模糊不清的示例。在深入建模你的数据之前，请考虑过滤或更正这样的坏例子！\n", + "\n", + "**本教程我们将要做的事情概述：**\n", + "\n", + "- 使用预训练的 transformer 模型从客户服务请求中提取文本嵌入\n", + "\n", + "- 在文本嵌入上训练一个简单的逻辑回归模型，以计算样本外的预测概率\n", + "\n", + "- 使用这些预测和嵌入运行 Cleanlab 的 `Datalab` 审核，以识别数据集中的问题，如：标签问题、离群值和近重复项。" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "o__pRLFYcw4K" + }, + "source": [ + "## 快速入门\n", + "\n", + "已经有一个模型在现有标签集上训练得到的（样本外）`pred_probs` 了吗？也许你还有一些数值`特征`？运行下面的代码来查找数据集中的任何潜在标签错误。\n", + "\n", + "**注意：** 如果在 Colab 上运行，可能需要使用 GPU（选择：Runtime > Change runtime type > Hardware accelerator > GPU）\n", + "\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "qaZA0cFs1fW4" + }, + "outputs": [], + "source": [ + "from cleanlab import Datalab\n", + "\n", + "lab = Datalab(data=your_dataset, label_name=\"column_name_of_labels\")\n", + "lab.find_issues(pred_probs=your_pred_probs, features=your_features)\n", + "\n", + "lab.report()\n", + "lab.get_issues()\n" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "dp4lpApmcw4K" + }, + "source": [ + "## 安装需要的依赖\n" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "DjoWBgGAcw4K" + }, + "source": [ + "你可以使用 `pip` 按照以下方式安装本教程所需的所有包：\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "fRsBIj3L_RUb" + }, + "outputs": [], + "source": [ + "!pip install -U scikit-learn sentence-transformers datasets\n", + "!pip install -U \"cleanlab[datalab]\"" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "execution": { + "iopub.execute_input": "2024-02-16T06:26:13.467211Z", + "iopub.status.busy": "2024-02-16T06:26:13.466877Z", + "iopub.status.idle": "2024-02-16T06:26:13.470222Z", + "shell.execute_reply": "2024-02-16T06:26:13.469761Z" + }, + "id": "zgezWF-2cw4L" + }, + "outputs": [], + "source": [ + "import re\n", + "import string\n", + "import pandas as pd\n", + "from sklearn.metrics import accuracy_score, log_loss\n", + "from sklearn.model_selection import cross_val_predict\n", + "from sklearn.linear_model import LogisticRegression\n", + "from sentence_transformers import SentenceTransformer\n", + "\n", + "from cleanlab import Datalab" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "execution": { + "iopub.execute_input": "2024-02-16T06:26:13.472374Z", + "iopub.status.busy": "2024-02-16T06:26:13.471951Z", + "iopub.status.idle": "2024-02-16T06:26:13.475065Z", + "shell.execute_reply": "2024-02-16T06:26:13.474625Z" + }, + "id": "mO3pnA1ncw4L", + "nbsphinx": "hidden" + }, + "outputs": [], + "source": [ + "import random\n", + "import numpy as np\n", + "\n", + "pd.set_option(\"display.max_colwidth\", None)\n", + "\n", + "SEED = 123456 # for reproducibility\n", + "np.random.seed(SEED)\n", + "random.seed(SEED)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "yj_5JcO1cw4L" + }, + "source": [ + "## 加载和格式化文本数据集" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 206 + }, + "execution": { + "iopub.execute_input": "2024-02-16T06:26:13.476949Z", + "iopub.status.busy": "2024-02-16T06:26:13.476773Z", + "iopub.status.idle": "2024-02-16T06:26:13.502278Z", + "shell.execute_reply": "2024-02-16T06:26:13.501755Z" + }, + "id": "HztO4qU9cw4L", + "outputId": "c6ff9e95-6326-413e-a72f-6f3c05af1055" + }, + "outputs": [ + { + "data": { + "application/vnd.google.colaboratory.intrinsic+json": { + "summary": "{\n \"name\": \"data\",\n \"rows\": 1000,\n \"fields\": [\n {\n \"column\": \"text\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 1000,\n \"samples\": [\n \"I made an international purchase, but the exchange rate was wrong\",\n \"I would like to know why a withdraw I made for some cash shows up as pending.\",\n \"I tried to get cash out of the ATM but it is taking too long\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"label\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 12,\n \"min\": 11,\n \"max\": 46,\n \"num_unique_values\": 7,\n \"samples\": [\n 11,\n 13,\n 46\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n }\n ]\n}", + "type": "dataframe", + "variable_name": "data" + }, + "text/html": [ + "\n", + "
\n", + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
textlabel
0I am still waiting on my card?11
1What can I do if my card still hasn't arrived after 2 weeks?11
2I have been waiting over a week. Is the card still coming?11
3Can I track my card while it is in the process of delivery?11
4How do I know if I will get my card, or if it is lost?11
\n", + "
\n", + "
\n", + "\n", + "
\n", + " \n", + "\n", + " \n", + "\n", + " \n", + "
\n", + "\n", + "\n", + "
\n", + " \n", + "\n", + "\n", + "\n", + " \n", + "
\n", + "
\n", + "
\n" + ], + "text/plain": [ + " text label\n", + "0 I am still waiting on my card? 11\n", + "1 What can I do if my card still hasn't arrived after 2 weeks? 11\n", + "2 I have been waiting over a week. Is the card still coming? 11\n", + "3 Can I track my card while it is in the process of delivery? 11\n", + "4 How do I know if I will get my card, or if it is lost? 11" + ] + }, + "execution_count": 24, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "from datasets import load_dataset\n", + "\n", + "dataset = load_dataset(\"PolyAI/banking77\", split=\"train\")\n", + "data = pd.DataFrame(dataset[:1000])\n", + "data.head()" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "execution": { + "iopub.execute_input": "2024-02-16T06:26:13.504463Z", + "iopub.status.busy": "2024-02-16T06:26:13.504049Z", + "iopub.status.idle": "2024-02-16T06:26:13.508243Z", + "shell.execute_reply": "2024-02-16T06:26:13.507706Z" + }, + "id": "Ujp0luqRcw4M", + "outputId": "b438fed5-aa75-450d-dc84-0b3398960487" + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "This dataset has 7 classes.\n", + "Classes: {32, 34, 36, 11, 13, 46, 17}\n" + ] + } + ], + "source": [ + "raw_texts, labels = data[\"text\"].values, data[\"label\"].values\n", + "num_classes = len(set(labels))\n", + "\n", + "print(f\"This dataset has {num_classes} classes.\")\n", + "print(f\"Classes: {set(labels)}\")" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "PVza57cecw4M" + }, + "source": [ + "让我们查看数据集中的第 i 个示例：" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "execution": { + "iopub.execute_input": "2024-02-16T06:26:13.510435Z", + "iopub.status.busy": "2024-02-16T06:26:13.510163Z", + "iopub.status.idle": "2024-02-16T06:26:13.513358Z", + "shell.execute_reply": "2024-02-16T06:26:13.512906Z" + }, + "id": "lXHi90Kecw4M", + "outputId": "af8a9b19-986f-44fe-c564-dd83e400309e" + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Example Label: 11\n", + "Example Text: What can I do if my card still hasn't arrived after 2 weeks?\n" + ] + } + ], + "source": [ + "i = 1 # change this to view other examples from the dataset\n", + "print(f\"Example Label: {labels[i]}\")\n", + "print(f\"Example Text: {raw_texts[i]}\")" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "JH7UU9Wscw4M" + }, + "source": [ + "数据以两个 numpy 数组的形式存储：\n", + "1. `raw_texts` 以文本格式存储客户服务请求的话语\n", + "2. `labels` 存储每个示例的意图类别（标签）" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "T0d80apCcw4M" + }, + "source": [ + "
\n", + "\n", + "自有数据？\n", + "\n", + "你可以轻松地将上述内容替换为你自己的文本数据集，并继续进行教程的其余部分。\n", + "\n", + "\n", + "
" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "YLDeD09Ncw4M" + }, + "source": [ + "接下来，我们将文本字符串转换为更适合作为机器学习模型输入的向量。\n", + "\n", + "我们将使用预训练的 Transformer 模型提供的数值表示作为我们文本的嵌入。[Sentence Transformers](https://huggingface.co/docs/hub/sentence-transformers) 库提供了计算文本数据嵌入的简单方法。在这里，我们加载了预训练的 `electra-small-discriminator` 模型，然后通过网络运行我们的数据，以提取每个示例的向量嵌入。\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "execution": { + "iopub.execute_input": "2024-02-16T06:26:13.515306Z", + "iopub.status.busy": "2024-02-16T06:26:13.515126Z", + "iopub.status.idle": "2024-02-16T06:26:18.244024Z", + "shell.execute_reply": "2024-02-16T06:26:18.243354Z" + }, + "id": "DbDb6Ni6cw4M" + }, + "outputs": [], + "source": [ + "transformer = SentenceTransformer('google/electra-small-discriminator')\n", + "text_embeddings = transformer.encode(raw_texts)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "Moz0KJvzcw4M" + }, + "source": [ + "我们后续的机器学习模型将直接在 `text_embeddings` 的元素上操作，以便对客户服务请求进行分类。" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "4FK2Q72gcw4M" + }, + "source": [ + "## 定义一个分类模型并计算样本外的预测概率" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "yaicOGrhcw4N" + }, + "source": [ + " 为了利用预训练网络进行特定的分类任务，一种典型的方法是添加一个线性输出层，并在新数据上微调网络参数。然而，这可能需要大量的计算资源。另一种方法是冻结网络的预训练权重，只训练输出层，而不依赖于 GPU。在这里，我们通过在提取的嵌入顶部拟合一个 scikit-learn 线性模型来方便地实现这一点。\n", + "\n", + " 为了识别标签问题，cleanlab 需要你的模型为每个数据点提供概率预测。然而，对于模型之前训练过的数据点，这些预测将是过拟合的（因此不可靠）。cleanlab 旨在仅与**样本外**的预测类概率一起使用，即在模型训练期间保持不变的数据点。\n", + "\n", + " 在这里，我们使用带有交叉验证的逻辑回归模型来获得数据集中每个示例的样本外预测类概率。\n", + " 确保你的 `pred_probs` 列根据类的排序正确排序，对于 Datalab 来说，是：按类名字典顺序排序。" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "execution": { + "iopub.execute_input": "2024-02-16T06:26:18.247142Z", + "iopub.status.busy": "2024-02-16T06:26:18.246652Z", + "iopub.status.idle": "2024-02-16T06:26:19.133641Z", + "shell.execute_reply": "2024-02-16T06:26:19.132953Z" + }, + "id": "tiIqp1arcw4N", + "scrolled": true + }, + "outputs": [], + "source": [ + "model = LogisticRegression(max_iter=400)\n", + "\n", + "pred_probs = cross_val_predict(model, text_embeddings, labels, method=\"predict_proba\")" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "9s0pcMk1cw4N" + }, + "source": [ + "## 使用 Cleanlab 查找数据集中的问题" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "qa8ltsx9cw4N" + }, + "source": [ + "在给定来自你拥有的任何模型的特征嵌入和（样本外）预测类概率的情况下，cleanlab 可以帮助你快速识别数据中的低质量示例。\n", + "\n", + "在这里，我们使用 Cleanlab 的 `Datalab` 来查找数据中的问题。Datalab 提供了几种加载数据的方式；我们将简单地在字典中包装训练特征和噪声标签。" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "execution": { + "iopub.execute_input": "2024-02-16T06:26:19.136722Z", + "iopub.status.busy": "2024-02-16T06:26:19.136482Z", + "iopub.status.idle": "2024-02-16T06:26:19.139419Z", + "shell.execute_reply": "2024-02-16T06:26:19.138870Z" + }, + "id": "UNj4rWW2cw4N" + }, + "outputs": [], + "source": [ + "data_dict = {\"texts\": raw_texts, \"labels\": labels}" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "IpNmBc_Lcw4N" + }, + "source": [ + "审核你的数据所需的全部操作就是调用 `find_issues()`。我们传入上面获得的预测概率和特征嵌入，但你不一定需要提供所有这些信息，具体取决于你对哪些类型的问题感兴趣。你提供的输入越多，`Datalab` 就能在你的数据中检测到更多类型的问题。使用更好的模型来生成这些输入将确保 cleanlab 更准确地估计问题。\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "execution": { + "iopub.execute_input": "2024-02-16T06:26:19.141893Z", + "iopub.status.busy": "2024-02-16T06:26:19.141673Z", + "iopub.status.idle": "2024-02-16T06:26:20.809087Z", + "shell.execute_reply": "2024-02-16T06:26:20.808461Z" + }, + "id": "R0xuUDRWcw4N", + "scrolled": true + }, + "outputs": [], + "source": [ + "lab = Datalab(data_dict, label_name=\"labels\")\n", + "lab.find_issues(pred_probs=pred_probs, features=text_embeddings)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "d6Iqy0vGq7w9" + }, + "source": [ + "输出看起来如下：\n", + "\n", + "```bash\n", + "Finding null issues ...\n", + "Finding label issues ...\n", + "Finding outlier issues ...\n", + "Fitting OOD estimator based on provided features ...\n", + "Finding near_duplicate issues ...\n", + "Finding non_iid issues ...\n", + "Finding class_imbalance issues ...\n", + "Finding underperforming_group issues ...\n", + "\n", + "Audit complete. 62 issues found in the dataset.\n", + "```" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "4aitesJccw4N" + }, + "source": [ + "审计完成后，使用 `report` 方法来查看审计结果。" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "execution": { + "iopub.execute_input": "2024-02-16T06:26:20.813057Z", + "iopub.status.busy": "2024-02-16T06:26:20.811515Z", + "iopub.status.idle": "2024-02-16T06:26:20.838760Z", + "shell.execute_reply": "2024-02-16T06:26:20.838088Z" + }, + "id": "ALXu32nzcw4N", + "outputId": "733d2ed4-5bcd-49e6-93a7-285f3d66278c", + "scrolled": true + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Here is a summary of the different kinds of issues found in the data:\n", + "\n", + " issue_type num_issues\n", + " outlier 37\n", + "near_duplicate 14\n", + " label 10\n", + " non_iid 1\n", + "\n", + "Dataset Information: num_examples: 1000, num_classes: 7\n", + "\n", + "\n", + "---------------------- outlier issues ----------------------\n", + "\n", + "About this issue:\n", + "\tExamples that are very different from the rest of the dataset \n", + " (i.e. potentially out-of-distribution or rare/anomalous instances).\n", + " \n", + "\n", + "Number of examples with this issue: 37\n", + "Overall dataset quality in terms of this issue: 0.3671\n", + "\n", + "Examples representing most severe instances of this issue:\n", + " is_outlier_issue outlier_score\n", + "791 True 0.024866\n", + "601 True 0.031162\n", + "863 True 0.060738\n", + "355 True 0.064199\n", + "157 True 0.065075\n", + "\n", + "\n", + "------------------ near_duplicate issues -------------------\n", + "\n", + "About this issue:\n", + "\tA (near) duplicate issue refers to two or more examples in\n", + " a dataset that are extremely similar to each other, relative\n", + " to the rest of the dataset. The examples flagged with this issue\n", + " may be exactly duplicated, or lie atypically close together when\n", + " represented as vectors (i.e. feature embeddings).\n", + " \n", + "\n", + "Number of examples with this issue: 14\n", + "Overall dataset quality in terms of this issue: 0.5961\n", + "\n", + "Examples representing most severe instances of this issue:\n", + " is_near_duplicate_issue near_duplicate_score near_duplicate_sets distance_to_nearest_neighbor\n", + "459 True 0.009544 [429] 0.000566\n", + "429 True 0.009544 [459] 0.000566\n", + "501 True 0.046044 [412, 517] 0.002781\n", + "412 True 0.046044 [501] 0.002781\n", + "698 True 0.054626 [607] 0.003314\n", + "\n", + "\n", + "----------------------- label issues -----------------------\n", + "\n", + "About this issue:\n", + "\tExamples whose given label is estimated to be potentially incorrect\n", + " (e.g. due to annotation error) are flagged as having label issues.\n", + " \n", + "\n", + "Number of examples with this issue: 10\n", + "Overall dataset quality in terms of this issue: 0.9930\n", + "\n", + "Examples representing most severe instances of this issue:\n", + " is_label_issue label_score given_label predicted_label\n", + "379 False 0.025486 32 11\n", + "100 False 0.032102 11 36\n", + "300 False 0.037742 32 46\n", + "485 True 0.057666 17 34\n", + "159 True 0.059408 13 11\n", + "\n", + "\n", + "---------------------- non_iid issues ----------------------\n", + "\n", + "About this issue:\n", + "\tWhether the dataset exhibits statistically significant\n", + " violations of the IID assumption like:\n", + " changepoints or shift, drift, autocorrelation, etc.\n", + " The specific violation considered is whether the\n", + " examples are ordered such that almost adjacent examples\n", + " tend to have more similar feature values.\n", + " \n", + "\n", + "Number of examples with this issue: 1\n", + "Overall dataset quality in terms of this issue: 0.0000\n", + "\n", + "Examples representing most severe instances of this issue:\n", + " is_non_iid_issue non_iid_score\n", + "988 True 0.563774\n", + "975 False 0.570179\n", + "997 False 0.571891\n", + "967 False 0.572357\n", + "956 False 0.577413\n", + "\n", + "Additional Information: \n", + "p-value: 0.0\n" + ] + } + ], + "source": [ + "lab.report()" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "sAuLE6Macw4N" + }, + "source": [ + "### 标签问题\n", + "\n", + "报告显示 cleanlab 在我们的数据集中识别出了许多标签问题。我们可以使用 `get_issues` 方法来查看哪些示例被标记为可能标签错误，以及每个示例的标签质量分数，通过指定 `label` 作为参数来关注数据中的标签问题。" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 206 + }, + "execution": { + "iopub.execute_input": "2024-02-16T06:26:20.843083Z", + "iopub.status.busy": "2024-02-16T06:26:20.842045Z", + "iopub.status.idle": "2024-02-16T06:26:20.852505Z", + "shell.execute_reply": "2024-02-16T06:26:20.852016Z" + }, + "id": "6gATaXWscw4N", + "outputId": "0d0e70c5-1548-4fe6-b67e-668c8dfedf0e", + "scrolled": true + }, + "outputs": [ + { + "data": { + "application/vnd.google.colaboratory.intrinsic+json": { + "summary": "{\n \"name\": \"label_issues\",\n \"rows\": 1000,\n \"fields\": [\n {\n \"column\": \"is_label_issue\",\n \"properties\": {\n \"dtype\": \"boolean\",\n \"num_unique_values\": 2,\n \"samples\": [\n true,\n false\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"label_score\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 0.2150390046430028,\n \"min\": 0.025486333476725527,\n \"max\": 0.999751760644687,\n \"num_unique_values\": 1000,\n \"samples\": [\n 0.98954913626076,\n 0.44264330724848383\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"given_label\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 12,\n \"min\": 11,\n \"max\": 46,\n \"num_unique_values\": 7,\n \"samples\": [\n 11,\n 13\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"predicted_label\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 12,\n \"min\": 11,\n \"max\": 46,\n \"num_unique_values\": 7,\n \"samples\": [\n 11,\n 13\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n }\n ]\n}", + "type": "dataframe", + "variable_name": "label_issues" + }, + "text/html": [ + "\n", + "
\n", + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
is_label_issuelabel_scoregiven_labelpredicted_label
0False0.9039261111
1False0.8605441111
2False0.6583091111
3False0.6970851111
4False0.4349341111
\n", + "
\n", + "
\n", + "\n", + "
\n", + " \n", + "\n", + " \n", + "\n", + " \n", + "
\n", + "\n", + "\n", + "
\n", + " \n", + "\n", + "\n", + "\n", + " \n", + "
\n", + "
\n", + "
\n" + ], + "text/plain": [ + " is_label_issue label_score given_label predicted_label\n", + "0 False 0.903926 11 11\n", + "1 False 0.860544 11 11\n", + "2 False 0.658309 11 11\n", + "3 False 0.697085 11 11\n", + "4 False 0.434934 11 11" + ] + }, + "execution_count": 32, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "label_issues = lab.get_issues(\"label\")\n", + "label_issues.head()" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "eBLFyMMcs5NT" + }, + "source": [ + "| | is_label_issue | label_score | given_label | predicted_label |\n", + "|----------------|-------------|-------------|-----------------|-----------------|\n", + "| 0 | False | 0.903926 | 11 | 11 |\n", + "| 1 | False | 0.860544 | 11 | 11 |\n", + "| 2 | False | 0.658309 | 11 | 11 |\n", + "| 3 | False | 0.697085 | 11 | 11 |\n", + "| 4 | False | 0.434934 | 11 | 11 |\n" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "-tYlhmKYcw4N" + }, + "source": [ + "此方法返回一个包含每个示例的标签质量分数的数据框。这些数值分数介于 0 和 1 之间，其中较低的分数表示更可能是错误标记的示例。数据框还包含一个布尔列，指定是否将每个示例识别为具有标签问题（表明它可能是错误标记的）。" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "XcD-oCLlcw4N" + }, + "source": [ + "我们可以获取标记有标签问题的示例的子集，并且还可以按标签质量分数排序，以找到数据集中最可能错误标记的 5 个示例的索引。" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "execution": { + "iopub.execute_input": "2024-02-16T06:26:20.854743Z", + "iopub.status.busy": "2024-02-16T06:26:20.854394Z", + "iopub.status.idle": "2024-02-16T06:26:20.858961Z", + "shell.execute_reply": "2024-02-16T06:26:20.858409Z" + }, + "id": "QtloV-NBcw4N", + "outputId": "86c32e99-7dc8-470c-b102-f0f5acc13855" + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "cleanlab found 10 potential label errors in the dataset.\n", + "Here are indices of the top 5 most likely errors: \n", + " [379 100 300 485 159]\n" + ] + } + ], + "source": [ + "identified_label_issues = label_issues[label_issues[\"is_label_issue\"] == True]\n", + "lowest_quality_labels = label_issues[\"label_score\"].argsort()[:5].to_numpy()\n", + "\n", + "print(\n", + " f\"cleanlab found {len(identified_label_issues)} potential label errors in the dataset.\\n\"\n", + " f\"Here are indices of the top 5 most likely errors: \\n {lowest_quality_labels}\"\n", + ")" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "8J49bTeocw4N" + }, + "source": [ + "让我们查看一些最可能的标签错误。\n", + "\n", + "这里我们展示了数据集中被识别为最可能的标签错误的前 5 个示例，以及它们的给定（原始）标签和 cleanlab 提供的建议替代标签。\n", + "\n", + "\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 276 + }, + "execution": { + "iopub.execute_input": "2024-02-16T06:26:20.861048Z", + "iopub.status.busy": "2024-02-16T06:26:20.860742Z", + "iopub.status.idle": "2024-02-16T06:26:20.867443Z", + "shell.execute_reply": "2024-02-16T06:26:20.866904Z" + }, + "id": "c-niFVJvcw4N", + "outputId": "5bbc5217-3581-4e2e-8b56-7a1fc77cc427" + }, + "outputs": [ + { + "data": { + "application/vnd.google.colaboratory.intrinsic+json": { + "summary": "{\n \"name\": \"data_with_suggested_labels\",\n \"rows\": 5,\n \"fields\": [\n {\n \"column\": \"text\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 5,\n \"samples\": [\n \"can you share card tracking number?\",\n \"Is there any way to see my card in the app?\",\n \"If I need to cash foreign transfers, how does that work?\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"given_label\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 10,\n \"min\": 11,\n \"max\": 32,\n \"num_unique_values\": 4,\n \"samples\": [\n 11,\n 13,\n 32\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"suggested_label\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 15,\n \"min\": 11,\n \"max\": 46,\n \"num_unique_values\": 4,\n \"samples\": [\n 36,\n 34,\n 11\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n }\n ]\n}", + "type": "dataframe" + }, + "text/html": [ + "\n", + "
\n", + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
textgiven_labelsuggested_label
379Is there a specific source that the exchange rate for the transfer I'm planning on making is pulled from?3211
100can you share card tracking number?1136
300If I need to cash foreign transfers, how does that work?3246
485Was I charged more than I should of been for a currency exchange?1734
159Is there any way to see my card in the app?1311
\n", + "
\n", + "
\n", + "\n", + "
\n", + " \n", + "\n", + " \n", + "\n", + " \n", + "
\n", + "\n", + "\n", + "
\n", + " \n", + "\n", + "\n", + "\n", + " \n", + "
\n", + "
\n", + "
\n" + ], + "text/plain": [ + " text \\\n", + "379 Is there a specific source that the exchange rate for the transfer I'm planning on making is pulled from? \n", + "100 can you share card tracking number? \n", + "300 If I need to cash foreign transfers, how does that work? \n", + "485 Was I charged more than I should of been for a currency exchange? \n", + "159 Is there any way to see my card in the app? \n", + "\n", + " given_label suggested_label \n", + "379 32 11 \n", + "100 11 36 \n", + "300 32 46 \n", + "485 17 34 \n", + "159 13 11 " + ] + }, + "execution_count": 18, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "data_with_suggested_labels = pd.DataFrame(\n", + " {\"text\": raw_texts, \"given_label\": labels, \"suggested_label\": label_issues[\"predicted_label\"]}\n", + ")\n", + "data_with_suggested_labels.iloc[lowest_quality_labels]" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "g2dvMySPtkbL" + }, + "source": [ + "上面命令的输出如下所示：\n", + " \n", + "| | text | given_label | suggested_label |\n", + "|------|-----------------------------------------------------------------------------------------------------------|----------------|-----------------|\n", + "| 379 | Is there a specific source that the exchange rate for the transfer I'm planning on making is pulled from? | 32 | 11 |\n", + "| 100 | can you share card tracking number? | 11 | 36 |\n", + "| 300 | If I need to cash foreign transfers, how does that work? | 32 | 46 |\n", + "| 485 | Was I charged more than I should of been for a currency exchange? | 17 | 34 |\n", + "| 159 | Is there any way to see my card in the app? | 13 | 11 |\n" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "eH8ltGj0cw4O", + "scrolled": true + }, + "source": [ + "这些是 cleanlab 在此数据中识别的非常清晰的标签错误！请注意，`given_label` 并没有正确反映这些请求的意图，无论谁制作了这个数据集，在建模数据之前都需要解决许多错误。\n" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "ULFeD3bzcw4O" + }, + "source": [ + "### 离群值问题\n", + "\n", + "根据报告，我们的数据集中包含一些离群值。\n", + "\n", + "我们可以通过 `get_issues` 查看哪些示例是离群值（以及一个数值质量分数，量化每个示例看起来有多么典型）。我们将结果数据框按照 cleanlab 的离群值质量分数排序，以查看数据集中最严重的离群值。\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 206 + }, + "execution": { + "iopub.execute_input": "2024-02-16T06:26:20.869718Z", + "iopub.status.busy": "2024-02-16T06:26:20.869251Z", + "iopub.status.idle": "2024-02-16T06:26:20.876386Z", + "shell.execute_reply": "2024-02-16T06:26:20.875851Z" + }, + "id": "jBLuqUXBcw4O", + "outputId": "d5d2dbc6-c708-4750-e3ea-6dcd5c24a64d" + }, + "outputs": [ + { + "data": { + "application/vnd.google.colaboratory.intrinsic+json": { + "summary": "{\n \"name\": \"outlier_issues\",\n \"rows\": 5,\n \"fields\": [\n {\n \"column\": \"is_outlier_issue\",\n \"properties\": {\n \"dtype\": \"boolean\",\n \"num_unique_values\": 1,\n \"samples\": [\n true\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"outlier_score\",\n \"properties\": {\n \"dtype\": \"float32\",\n \"num_unique_values\": 5,\n \"samples\": [\n 0.03116183541715145\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n }\n ]\n}", + "type": "dataframe" + }, + "text/html": [ + "\n", + "
\n", + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
is_outlier_issueoutlier_score
791True0.024866
601True0.031162
863True0.060738
355True0.064199
157True0.065075
\n", + "
\n", + "
\n", + "\n", + "
\n", + " \n", + "\n", + " \n", + "\n", + " \n", + "
\n", + "\n", + "\n", + "
\n", + " \n", + "\n", + "\n", + "\n", + " \n", + "
\n", + "
\n", + "
\n" + ], + "text/plain": [ + " is_outlier_issue outlier_score\n", + "791 True 0.024866\n", + "601 True 0.031162\n", + "863 True 0.060738\n", + "355 True 0.064199\n", + "157 True 0.065075" + ] + }, + "execution_count": 20, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "outlier_issues = lab.get_issues(\"outlier\")\n", + "outlier_issues.sort_values(\"outlier_score\").head()" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "F7Z2VJQAujui" + }, + "source": [ + "输出如下所示：\n", + "\n", + "| | is_outlier_issue | outlier_score |\n", + "|---| ----------------|---------------|\n", + "| 791 | True | 0.024866 |\n", + "| 601 | True | 0.031162 |\n", + "| 863 | True | 0.060738 |\n", + "| 355 | True | 0.064199 |\n", + "| 157 | True | 0.065075 |" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 246 + }, + "execution": { + "iopub.execute_input": "2024-02-16T06:26:20.878435Z", + "iopub.status.busy": "2024-02-16T06:26:20.878117Z", + "iopub.status.idle": "2024-02-16T06:26:20.884073Z", + "shell.execute_reply": "2024-02-16T06:26:20.883533Z" + }, + "id": "Kjn-muLGcw4O", + "outputId": "a5ae0a32-cac4-442d-89fc-8f7f64da9dfc" + }, + "outputs": [ + { + "data": { + "application/vnd.google.colaboratory.intrinsic+json": { + "summary": "{\n \"name\": \"data\",\n \"rows\": 5,\n \"fields\": [\n {\n \"column\": \"text\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 5,\n \"samples\": [\n \"$1 charge in transaction.\",\n \"lost card found, want to put it back in app\",\n \"My atm withdraw is stillpending\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"label\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 13,\n \"min\": 13,\n \"max\": 46,\n \"num_unique_values\": 4,\n \"samples\": [\n 34,\n 13,\n 46\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n }\n ]\n}", + "type": "dataframe" + }, + "text/html": [ + "\n", + "
\n", + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
textlabel
791withdrawal pending meaning?46
601$1 charge in transaction.34
863My atm withdraw is stillpending46
355explain the interbank exchange rate32
157lost card found, want to put it back in app13
\n", + "
\n", + "
\n", + "\n", + "
\n", + " \n", + "\n", + " \n", + "\n", + " \n", + "
\n", + "\n", + "\n", + "
\n", + " \n", + "\n", + "\n", + "\n", + " \n", + "
\n", + "
\n", + "
\n" + ], + "text/plain": [ + " text label\n", + "791 withdrawal pending meaning? 46\n", + "601 $1 charge in transaction. 34\n", + "863 My atm withdraw is stillpending 46\n", + "355 explain the interbank exchange rate 32\n", + "157 lost card found, want to put it back in app 13" + ] + }, + "execution_count": 34, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "lowest_quality_outliers = outlier_issues[\"outlier_score\"].argsort()[:5]\n", + "\n", + "data.iloc[lowest_quality_outliers]" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "kuZMsLPZYARL" + }, + "source": [ + "对于质量最低的离群值，样本输出将如下所示：\n", + "\n", + "|index|text|label|\n", + "|---|---|---|\n", + "|791|withdrawal pending meaning?|46|\n", + "|601|$1 charge in transaction\\.|34|\n", + "|863|My atm withdraw is stillpending|46|\n", + "|355|explain the interbank exchange rate|32|\n", + "|157|lost card found, want to put it back in app|13|\n" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "sBal-KDrcw4R" + }, + "source": [ + "我们看到 cleanlab 已经识别出这个数据集中的条目，这些条目看起来并不是正确的客户请求。此数据集中的离群值似乎是不在范围内的客户请求和其他对意图分类没有意义的非语义文本。仔细考虑这些离群值是否可能对你的数据建模产生不利影响，如果有可能的话，考虑从数据集中移除它们。\n" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "ch71b_0qcw4S" + }, + "source": [ + "### 近重复问题\n", + "\n", + "根据报告，我们的数据集中包含一些几乎重复的示例集。\n", + "我们可以通过 `get_issues` 查看哪些示例是（几乎）重复的（以及一个数值质量分数，量化每个示例与数据集中最近邻的相似程度）。我们将结果数据框按照 cleanlab 的近重复质量分数排序，以查看数据集中最接近重复的文本示例。" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 226 + }, + "execution": { + "iopub.execute_input": "2024-02-16T06:26:20.886079Z", + "iopub.status.busy": "2024-02-16T06:26:20.885805Z", + "iopub.status.idle": "2024-02-16T06:26:20.894466Z", + "shell.execute_reply": "2024-02-16T06:26:20.893919Z" + }, + "id": "TbI49Rdccw4S", + "outputId": "1978cdb5-02c2-4f82-e7d5-553ad1b6dca9" + }, + "outputs": [ + { + "data": { + "application/vnd.google.colaboratory.intrinsic+json": { + "summary": "{\n \"name\": \"duplicate_issues\",\n \"rows\": 5,\n \"fields\": [\n {\n \"column\": \"is_near_duplicate_issue\",\n \"properties\": {\n \"dtype\": \"boolean\",\n \"num_unique_values\": 1,\n \"samples\": [\n true\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"near_duplicate_score\",\n \"properties\": {\n \"dtype\": \"float32\",\n \"num_unique_values\": 3,\n \"samples\": [\n 0.00954437255859375\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"near_duplicate_sets\",\n \"properties\": {\n \"dtype\": \"object\",\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"distance_to_nearest_neighbor\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 0.0013286758192588926,\n \"min\": 0.0005658268928527832,\n \"max\": 0.0033143162727355957,\n \"num_unique_values\": 3,\n \"samples\": [\n 0.0005658268928527832\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n }\n ]\n}", + "type": "dataframe" + }, + "text/html": [ + "\n", + "
\n", + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
is_near_duplicate_issuenear_duplicate_scorenear_duplicate_setsdistance_to_nearest_neighbor
459True0.009544[429]0.000566
429True0.009544[459]0.000566
501True0.046044[412, 517]0.002781
412True0.046044[501]0.002781
698True0.054626[607]0.003314
\n", + "
\n", + "
\n", + "\n", + "
\n", + " \n", + "\n", + " \n", + "\n", + " \n", + "
\n", + "\n", + "\n", + "
\n", + " \n", + "\n", + "\n", + "\n", + " \n", + "
\n", + "
\n", + "
\n" + ], + "text/plain": [ + " is_near_duplicate_issue near_duplicate_score near_duplicate_sets \\\n", + "459 True 0.009544 [429] \n", + "429 True 0.009544 [459] \n", + "501 True 0.046044 [412, 517] \n", + "412 True 0.046044 [501] \n", + "698 True 0.054626 [607] \n", + "\n", + " distance_to_nearest_neighbor \n", + "459 0.000566 \n", + "429 0.000566 \n", + "501 0.002781 \n", + "412 0.002781 \n", + "698 0.003314 " + ] + }, + "execution_count": 35, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "duplicate_issues = lab.get_issues(\"near_duplicate\")\n", + "duplicate_issues.sort_values(\"near_duplicate_score\").head()" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "EawP0y1Lcw4S" + }, + "source": [ + "上面的结果显示了 cleanlab 认为哪些示例是近重复的（`is_near_duplicate_issue == True` 的行）。在这里，我们看到示例 459 和 429 是近重复的，示例 501 和 412 也是近重复的。\n", + "\n", + "让我们查看这些示例，看看它们有多么相似。\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 182 + }, + "execution": { + "iopub.execute_input": "2024-02-16T06:26:20.896501Z", + "iopub.status.busy": "2024-02-16T06:26:20.896175Z", + "iopub.status.idle": "2024-02-16T06:26:20.901983Z", + "shell.execute_reply": "2024-02-16T06:26:20.901420Z" + }, + "id": "0TEW5igFcw4S", + "outputId": "86343985-26bb-44ce-f27b-610357f43030" + }, + "outputs": [ + { + "data": { + "application/vnd.google.colaboratory.intrinsic+json": { + "summary": "{\n \"name\": \"data\",\n \"rows\": 2,\n \"fields\": [\n {\n \"column\": \"text\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 2,\n \"samples\": [\n \"I purchased something overseas and the incorrect exchange rate was applied.\",\n \"I purchased something abroad and the incorrect exchange rate was applied.\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"label\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 0,\n \"min\": 17,\n \"max\": 17,\n \"num_unique_values\": 1,\n \"samples\": [\n 17\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n }\n ]\n}", + "type": "dataframe" + }, + "text/html": [ + "\n", + "
\n", + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
textlabel
459I purchased something abroad and the incorrect exchange rate was applied.17
429I purchased something overseas and the incorrect exchange rate was applied.17
\n", + "
\n", + "
\n", + "\n", + "
\n", + " \n", + "\n", + " \n", + "\n", + " \n", + "
\n", + "\n", + "\n", + "
\n", + " \n", + "\n", + "\n", + "\n", + " \n", + "
\n", + "
\n", + "
\n" + ], + "text/plain": [ + " text \\\n", + "459 I purchased something abroad and the incorrect exchange rate was applied. \n", + "429 I purchased something overseas and the incorrect exchange rate was applied. \n", + "\n", + " label \n", + "459 17 \n", + "429 17 " + ] + }, + "execution_count": 38, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "data.iloc[[459, 429]]" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "DoAyD-FZpsSm" + }, + "source": [ + "样本输出：\n", + "\n", + "|index|text|label|\n", + "|---|---|---|\n", + "|459|I purchased something abroad and the incorrect exchange rate was applied\\.|17|\n", + "|429|I purchased something overseas and the incorrect exchange rate was applied\\.|17|" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 198 + }, + "execution": { + "iopub.execute_input": "2024-02-16T06:26:20.904159Z", + "iopub.status.busy": "2024-02-16T06:26:20.903821Z", + "iopub.status.idle": "2024-02-16T06:26:20.909681Z", + "shell.execute_reply": "2024-02-16T06:26:20.909160Z" + }, + "id": "VnbIBYaHcw4S", + "outputId": "8b00bb96-0d9d-43f6-b85f-c41e437d41b5" + }, + "outputs": [ + { + "data": { + "application/vnd.google.colaboratory.intrinsic+json": { + "summary": "{\n \"name\": \"data\",\n \"rows\": 2,\n \"fields\": [\n {\n \"column\": \"text\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 2,\n \"samples\": [\n \"The exchange rate you are using is bad.This can't be the official interbank exchange rate.\",\n \"The exchange rate you are using is really bad.This can't be the official interbank exchange rate.\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"label\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 0,\n \"min\": 17,\n \"max\": 17,\n \"num_unique_values\": 1,\n \"samples\": [\n 17\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n }\n ]\n}", + "type": "dataframe" + }, + "text/html": [ + "\n", + "
\n", + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
textlabel
501The exchange rate you are using is really bad.This can't be the official interbank exchange rate.17
412The exchange rate you are using is bad.This can't be the official interbank exchange rate.17
\n", + "
\n", + "
\n", + "\n", + "
\n", + " \n", + "\n", + " \n", + "\n", + " \n", + "
\n", + "\n", + "\n", + "
\n", + " \n", + "\n", + "\n", + "\n", + " \n", + "
\n", + "
\n", + "
\n" + ], + "text/plain": [ + " text \\\n", + "501 The exchange rate you are using is really bad.This can't be the official interbank exchange rate. \n", + "412 The exchange rate you are using is bad.This can't be the official interbank exchange rate. \n", + "\n", + " label \n", + "501 17 \n", + "412 17 " + ] + }, + "execution_count": 39, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "data.iloc[[501, 412]]" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "Y4QD35-dqeGg" + }, + "source": [ + "样本输出：\n", + "\n", + "|index|text|label|\n", + "|---|---|---|\n", + "|501|The exchange rate you are using is really bad\\.This can't be the official interbank exchange rate\\.|17|\n", + "|412|The exchange rate you are using is bad\\.This can't be the official interbank exchange rate\\.|17|" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "UG8xfTa5cw4S" + }, + "source": [ + "我们看到这两组请求确实非常相似！在数据集中包含近重复项可能会对模型产生意想不到的影响，并且要小心不要将它们分割到训练/测试集中。从[常见问题解答](../faq.html#How-to-handle-near-duplicate-data-identified-by-cleanlab?)中了解更多关于处理数据集中的近重复数据的信息。" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "iefctl3rcw4S" + }, + "source": [ + "### 非独立同分布问题（数据漂移）\n", + "根据报告，我们的数据集似乎不是独立同分布的（IID）。数据集的整体非 IID 分数（如下所示）对应于一个统计测试的 `p 值`，该测试用于判断数据集中样本的排序是否与它们特征值之间的相似性有关。一个低的 `p 值`强烈表明数据集违反了 IID 假设，这是从数据集产生的结论（模型）推广到更大总体所需的关键假设。" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "execution": { + "iopub.execute_input": "2024-02-16T06:26:20.911817Z", + "iopub.status.busy": "2024-02-16T06:26:20.911434Z", + "iopub.status.idle": "2024-02-16T06:26:20.915049Z", + "shell.execute_reply": "2024-02-16T06:26:20.914501Z" + }, + "id": "oEMWOQQPcw4S", + "outputId": "18eca4cd-2451-4850-960c-0bf1e35d9729" + }, + "outputs": [ + { + "data": { + "text/plain": [ + "0.0" + ] + }, + "execution_count": 40, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "p_value = lab.get_info('non_iid')['p-value']\n", + "p_value" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "c6swPCnncw4S" + }, + "source": [ + "在这里，我们的数据集被标记为非 IID，因为原始数据中的行恰好是按类别标签排序的。如果我们记得在模型训练和数据拆分之前打乱行，这可能是不重要的。但是，如果你不知道为什么你的数据被标记为非IID，那么你应该担心可能的数据漂移或数据点之间的意外交互（它们的价值可能不是统计独立的）。仔细考虑未来的测试数据可能看起来如何（以及你的数据是否代表你关心的人群）。在非 IID 测试运行之前，你不应该打乱数据（这将使结论无效）。\n" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "uCoKXqBrcw4S" + }, + "source": [ + " 如上所示，cleanlab 可以自动筛选出数据集中最可能的问题，帮助你更好地为后续建模整理数据集。有了这个短名单，你可以选择修复这些标签问题，或者从数据集中移除非语义或重复的示例，以获得更高质量的数据集来训练你的下一个机器学习模型。cleanlab 的问题检测可以与你最初训练的*任何*类型的模型的输出一起运行。\n", + "\n" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "qnncoRWUcw4S" + }, + "source": [ + "### Cleanlab 开源项目\n", + "\n", + "[Cleanlab](https://github.com/cleanlab/cleanlab) 是一个标准的以数据为中心的人工智能包，旨在解决混乱的现实世界数据的质量问题。\n", + "\n", + "请考虑给 Cleanlab Github 仓库一个星标，如果你有兴趣，也可以参与到这个[项目](https://github.com/cleanlab/cleanlab/issues?q=is:issue+is:open+label:%22good+first+issue%22)中来，比如帮助解决一些简单的问题。。\n" + ] + } + ], + "metadata": { + "colab": { + "provenance": [], + "toc_visible": true + }, + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.11.8" + } + }, + "nbformat": 4, + "nbformat_minor": 0 +} From a8c14a847a601583afc46dd92cd08e378e03bd1e Mon Sep 17 00:00:00 2001 From: innovation64 Date: Thu, 28 Mar 2024 14:53:26 +0800 Subject: [PATCH 06/31] chore: update _toctree.yal in zh-CN --- notebooks/zh-CN/_toctree.yml | 2 ++ 1 file changed, 2 insertions(+) diff --git a/notebooks/zh-CN/_toctree.yml b/notebooks/zh-CN/_toctree.yml index 37dfa53a..0ee17f08 100644 --- a/notebooks/zh-CN/_toctree.yml +++ b/notebooks/zh-CN/_toctree.yml @@ -2,6 +2,8 @@ sections: - local: index title: 开源 AI 指南 (Cookbook) + - local: issues_in_text_dataset + title: 使用 Cleanlab 检测文本数据集中的问题 - local: rag_with_hugging_face_gemma_mongodb title: 用 Gemma, MongoDB 和开源模型构建 RAG 系统 - local: automatic_embedding_tei_inference_endpoints From 4bfc3bd5aa8db7df1e9e3c4ed98d88dca3f0c284 Mon Sep 17 00:00:00 2001 From: innovation64 Date: Thu, 28 Mar 2024 14:57:11 +0800 Subject: [PATCH 07/31] chore: update _toctree.yal in zh-CN --- notebooks/zh-CN/_toctree.yml | 2 ++ 1 file changed, 2 insertions(+) diff --git a/notebooks/zh-CN/_toctree.yml b/notebooks/zh-CN/_toctree.yml index 0ee17f08..465fc091 100644 --- a/notebooks/zh-CN/_toctree.yml +++ b/notebooks/zh-CN/_toctree.yml @@ -20,3 +20,5 @@ title: 使用 LangChain 在 HuggingFace 文档上构建高级 RAG - local: rag_evaluation title: 使用合成数据和 LLM 作为裁判评估 RAG + - local: semantic_cache_chroma_vector_database + title: 通过引入语义缓存到 FAISS 中以增强 RAG 系统的性能 \ No newline at end of file From e8678f44d94a9b8c367d161eef1b66fded941438 Mon Sep 17 00:00:00 2001 From: innovation64 Date: Thu, 28 Mar 2024 16:37:53 +0800 Subject: [PATCH 08/31] docs: update tgi_messages_api in chinese version --- notebooks/zh-CN/tgi_messages_api_demo.ipynb | 520 ++++++++++++++++++++ 1 file changed, 520 insertions(+) create mode 100644 notebooks/zh-CN/tgi_messages_api_demo.ipynb diff --git a/notebooks/zh-CN/tgi_messages_api_demo.ipynb b/notebooks/zh-CN/tgi_messages_api_demo.ipynb new file mode 100644 index 00000000..9c29db08 --- /dev/null +++ b/notebooks/zh-CN/tgi_messages_api_demo.ipynb @@ -0,0 +1,520 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# 使用 TGI 的消息 API 从 OpenAI 迁移到 Open LLMs\n", + "\n", + "_作者: [Andrew Reed](https://huggingface.co/andrewrreed)_\n", + "\n", + "这个 notebook 展示了如何轻松地从 OpenAI 模型过渡到 Open LLMs，而无需重构任何现有代码。\n", + "\n", + "[文本生成推理（TGI）](https://github.com/huggingface/text-generation-inference)现在提供了一个[消息 API](https://huggingface.co/blog/tgi-messages-api)，使其与 OpenAI 的聊天完成 API 的直接兼容。这意味着任何使用 OpenAI 的模型（通过 OpenAI 客户端库或像 LangChain 或 LlamaIndex 这样的第三方工具）的现有脚本都可以直接替换为使用运行在 TGI 端点上的任何开源 LLM！\n", + "\n", + "这允许你快速测试并受益于开源模型提供的众多优势。例如：\n", + "\n", + "- 对模型和数据的完全控制和透明度\n", + "\n", + "- 不再担心速率限制\n", + "\n", + "- 能够根据你的具体需求完全定制系统\n", + "\n", + "在这个 notebook 中，我们将向你展示具体流程：\n", + "\n", + "1. [使用 TGI 创建推理端点来部署模型](#section_1)\n", + "2. [使用 OpenAI 客户端库查询推理端点](#section_2)\n", + "3. [将端点与 LangChain 和 LlamaIndex 工作流程集成](#section_3)\n", + "\n", + "**让我们开始吧！**" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 初始化设置\n", + "\n", + "首先，我们需要安装依赖项和设置一个 HF API 密钥。" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "!pip install --upgrade -q huggingface_hub langchain langchain-community langchainhub langchain-openai llama-index chromadb bs4 sentence_transformers torch" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": {}, + "outputs": [], + "source": [ + "import os\n", + "import getpass\n", + "\n", + "# enter API key\n", + "os.environ[\"HUGGINGFACEHUB_API_TOKEN\"] = HF_API_KEY = getpass.getpass()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "\n", + "\n", + "## 1. 创建一个推理端点\n", + "\n", + "一开始，让我们使用 TGI 将[Nous-Hermes-2-Mixtral-8x7B-DPO](https://huggingface.co/NousResearch/Nous-Hermes-2-Mixtral-8x7B-DPO)，一个微调的 Mixtral 模型，部署到推理端点。\n", + "\n", + "我们只需通过 UI 的[几次点击](https://ui.endpoints.huggingface.co/new?vendor=aws&repository=NousResearch%2FNous-Hermes-2-Mixtral-8x7B-DPO&tgi_max_total_tokens=32000&tgi=true&tgi_max_input_length=1024&task=text-generation&instance_size=2xlarge&tgi_max_batch_prefill_tokens=2048&tgi_max_batch_total_tokens=1024000&no_suggested_compute=true&accelerator=gpu®ion=us-east-1)，就可以部署模型，或者利用 `huggingface_hub` Python 库以编程方式创建和管理推理端点。\n", + "\n", + "在这里，我们将使用 Hub 库，通过指定端点名称和模型仓库，以及 `text-generation` 任务。在这个例子中，我们使用 `protected` 类型，因此访问部署的模型将需要一个有效的 Hugging Face token。我们还需要配置硬件要求，如供应商、地区、加速器、实例类型和大小。你可以使用[this API call](https://api.endpoints.huggingface.cloud/#get-/v2/provider)查看可用的资源选项列表，并在目录中[这里](https://ui.endpoints.huggingface.co/catalog)查看为选定模型推荐配置。\n" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "running\n" + ] + } + ], + "source": [ + "from huggingface_hub import create_inference_endpoint\n", + "\n", + "endpoint = create_inference_endpoint(\n", + " \"nous-hermes-2-mixtral-8x7b-demo\",\n", + " repository=\"NousResearch/Nous-Hermes-2-Mixtral-8x7B-DPO\",\n", + " framework=\"pytorch\",\n", + " task=\"text-generation\",\n", + " accelerator=\"gpu\",\n", + " vendor=\"aws\",\n", + " region=\"us-east-1\",\n", + " type=\"protected\",\n", + " instance_type=\"p4de\",\n", + " instance_size=\"2xlarge\",\n", + " custom_image={\n", + " \"health_route\": \"/health\",\n", + " \"env\": {\n", + " \"MAX_INPUT_LENGTH\": \"4096\",\n", + " \"MAX_BATCH_PREFILL_TOKENS\": \"4096\",\n", + " \"MAX_TOTAL_TOKENS\": \"32000\",\n", + " \"MAX_BATCH_TOTAL_TOKENS\": \"1024000\",\n", + " \"MODEL_ID\": \"/repository\",\n", + " },\n", + " \"url\": \"ghcr.io/huggingface/text-generation-inference:sha-1734540\", # must be >= 1.4.0\n", + " },\n", + ")\n", + "\n", + "endpoint.wait()\n", + "print(endpoint.status)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "部署启动需要几分钟时间。我们可以使用 `.wait()` 工具来阻塞运行线程，直到端点达到最终的“运行”状态。一旦运行，我们可以在 UI 播放器中确认其状态并试用：\n", + "\n", + "![IE UI Overview](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/blog/messages-api/endpoint-overview.png)\n", + "\n", + "太好了，现在我们有一个可用的端点！\n", + "\n", + "_注意：使用 `huggingface_hub` 部署时，默认情况下，在15分钟空闲时间后，你的端点会自动缩放到零，以在非活动期间优化成本。查看[ Hub Python 库文档](https://huggingface.co/docs/huggingface_hub/guides/inference_endpoints)以了解可用于管理端点生命周期的所有功能。_" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "\n", + "\n", + "## 2. 使用 OpenAI 客户端库查询推理端点\n", + "\n", + "如上所述，由于我们的模型托管在 TGI 上，现在支持消息 API，这意味着我们可以直接使用熟悉的 OpenAI 客户端库来查询它。\n", + "\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### 使用 Python 客户端\n", + "\n", + "下面的例子展示了如何使用[ OpenAI Python 库](https://github.com/openai/openai-python)进行这种转换。只需将 `` 替换为你的端点 URL（确保包含 `v1/` 后缀），并将 `` 字段填充为有效的 Hugging Face 用户 token。`` 可以从推理端点的 UI 中获取，或者从我们上面使用 `endpoint.url` 创建的端点对象中获取。\n", + "\n", + "然后我们可以像往常一样使用客户端，传递一个消息列表以从我们的推理端点流式传输响应。\n", + "\n" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Open-source software is important due to a number of reasons, including:\n", + "\n", + "1. Collaboration: The collaborative nature of open-source software allows developers from around the world to work together, share their ideas and improve the code. This often results in faster progress and better software.\n", + "\n", + "2. Transparency: With open-source software, the code is publicly available, making it easy to see exactly how the software functions, and allowing users to determine if there are any security vulnerabilities.\n", + "\n", + "3. Customization: Being able to access the code also allows users to customize the software to better suit their needs. This makes open-source software incredibly versatile, as users can tweak it to suit their specific use case.\n", + "\n", + "4. Quality: Open-source software is often developed by large communities of dedicated developers, who work together to improve the software. This results in a higher level of quality than might be found in proprietary software.\n", + "\n", + "5. Cost: Open-source software is often provided free of charge, which makes it accessible to a wider range of users. This can be especially important for organizations with limited budgets for software.\n", + "\n", + "6. Shared Benefit: By sharing the code of open-source software, everyone can benefit from the hard work of the developers. This contributes to the overall advancement of technology, as users and developers work together to improve and build upon the software.\n", + "\n", + "In summary, open-source software provides a collaborative platform that leads to high-quality, customizable, and transparent software, all available at little or no cost, benefiting both individuals and the technology community as a whole.<|im_end|>" + ] + } + ], + "source": [ + "from openai import OpenAI\n", + "\n", + "BASE_URL = endpoint.url\n", + "\n", + "# init the client but point it to TGI\n", + "client = OpenAI(\n", + " base_url=os.path.join(BASE_URL, \"v1/\"),\n", + " api_key=HF_API_KEY,\n", + ")\n", + "chat_completion = client.chat.completions.create(\n", + " model=\"tgi\",\n", + " messages=[\n", + " {\"role\": \"system\", \"content\": \"You are a helpful assistant.\"},\n", + " {\"role\": \"user\", \"content\": \"Why is open-source software important?\"},\n", + " ],\n", + " stream=True,\n", + " max_tokens=500,\n", + ")\n", + "\n", + "# iterate and print stream\n", + "for message in chat_completion:\n", + " print(message.choices[0].delta.content, end=\"\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "在幕后，TGI 的消息 API 自动使用其[聊天模板](https://huggingface.co/docs/transformers/chat_templating)将消息列表转换为模型所需的指令格式。\n", + "\n", + "_注意：某些 OpenAI 功能，如函数调用，与 TGI 不兼容。目前，消息 API 支持以下 chat completion 参数：`stream`、`max_new_tokens`、`frequency_penalty`、`logprobs`、`seed`、`temperature` 和 `top_p`._\n", + "\n", + "\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### 使用 JavaScript 客户端\n", + "\n", + "这里是与上面相同的流式示例，但是使用了[ OpenAI Javascript/Typescript 库](https://github.com/openai/openai-node)。\n", + "\n", + "\n", + "```js\n", + "import OpenAI from \"openai\";\n", + "\n", + "const openai = new OpenAI({\n", + " baseURL: \"\" + \"/v1/\", // replace with your endpoint url\n", + " apiKey: \"\", // replace with your token\n", + "});\n", + "\n", + "async function main() {\n", + " const stream = await openai.chat.completions.create({\n", + " model: \"tgi\",\n", + " messages: [\n", + " { role: \"system\", content: \"You are a helpful assistant.\" },\n", + " { role: \"user\", content: \"Why is open-source software important?\" },\n", + " ],\n", + " stream: true,\n", + " max_tokens: 500,\n", + " });\n", + " for await (const chunk of stream) {\n", + " process.stdout.write(chunk.choices[0]?.delta?.content || \"\");\n", + " }\n", + "}\n", + "\n", + "main();\n", + "```\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "\n", + "\n", + "## 3. 与 LangChain 和 LlamaIndex 集成\n", + "\n", + "现在，让我们看看如何将这个新创建的端点与像 LangChain 和 LlamaIndex 这样的流行 RAG 框架一起使用。" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### 如何与 LangChain 一起使用\n", + "\n", + "要在 [LangChain](https://python.langchain.com/docs/get_started/introduction) 中使用，只需创建一个 `ChatOpenAI` 的实例，并按如下方式传递你的 `` 和 ``：\n" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "AIMessage(content='Open-source software is important for several reasons:\\n\\n1. Transparency: Open-source software allows users to see the underlying code, making it easier to understand how the software works and identify any potential security vulnerabilities or bugs. This transparency fosters trust between users and developers.\\n\\n2. Collaboration: Open-source projects encourage collaboration among developers, allowing them to work together to improve the software, fix issues, and add new features. This collective effort can lead to')" + ] + }, + "execution_count": 7, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "from langchain_openai import ChatOpenAI\n", + "\n", + "llm = ChatOpenAI(\n", + " model_name=\"tgi\",\n", + " openai_api_key=HF_API_KEY,\n", + " openai_api_base=os.path.join(BASE_URL, \"v1/\"),\n", + ")\n", + "llm.invoke(\"Why is open-source software important?\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "我们能够直接利用与 OpenAI 模型相同的 `ChatOpenAI` 类。这使得所有之前的代码只需更改一行代码，就能与我们的端点一起工作。\n", + "\n", + "现在，让我们在简单的 RAG 流水线中使用我们的 Mixtral 模型，来回答一个关于 HF 博客内容的问题。\n" + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "{'context': [Document(page_content='To overcome this weakness, amongst other approaches, one can integrate the LLM into a system where it can call tools: such a system is called an LLM agent.\\nIn this post, we explain the inner workings of ReAct agents, then show how to build them using the ChatHuggingFace class recently integrated in LangChain. Finally, we benchmark several open-source LLMs against GPT-3.5 and GPT-4.', metadata={'description': 'We’re on a journey to advance and democratize artificial intelligence through open source and open science.', 'language': 'No language found.', 'source': 'https://huggingface.co/blog/open-source-llms-as-agents', 'title': 'Open-source LLMs as LangChain Agents'}),\n", + " Document(page_content='Since the open-source models were not specifically fine-tuned for calling functions in the given output format, they are at a slight disadvantage compared to the OpenAI agents.\\nDespite this, some models perform really well! 💪\\nHere’s an example of Mixtral-8x7B answering the question: “Which city has a larger population, Guiyang or Tacheng?”\\nThought: To answer this question, I need to find the current populations of both Guiyang and Tacheng. I will use the search tool to find this information.\\nAction:\\n{', metadata={'description': 'We’re on a journey to advance and democratize artificial intelligence through open source and open science.', 'language': 'No language found.', 'source': 'https://huggingface.co/blog/open-source-llms-as-agents', 'title': 'Open-source LLMs as LangChain Agents'}),\n", + " Document(page_content='Agents Showdown: how do open-source LLMs perform as general purpose reasoning agents?\\n\\t\\n\\nYou can find the code for this benchmark here.\\n\\n\\n\\n\\n\\n\\t\\tEvaluation\\n\\t\\n\\nWe want to measure how open-source LLMs perform as general purpose reasoning agents. Thus we select questions requiring using logic and the use of basic tools: a calculator and access to internet search.\\nThe final dataset is a combination of samples from 3 other datasets:', metadata={'description': 'We’re on a journey to advance and democratize artificial intelligence through open source and open science.', 'language': 'No language found.', 'source': 'https://huggingface.co/blog/open-source-llms-as-agents', 'title': 'Open-source LLMs as LangChain Agents'}),\n", + " Document(page_content='Open-source LLMs as LangChain Agents\\n\\t\\n\\nPublished\\n\\t\\t\\t\\tJanuary 24, 2024\\nUpdate on GitHub\\n\\nm-ric\\nAymeric Roucher\\n\\n\\n\\n\\nJofthomas\\nJoffrey THOMAS\\n\\n\\n\\n\\nandrewrreed\\nAndrew Reed\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\t\\tTL;DR\\n\\t\\n\\nOpen-source LLMs have now reached a performance level that makes them suitable reasoning engines for powering agent workflows: Mixtral even surpasses GPT-3.5 on our benchmark, and its performance could easily be further enhanced with fine-tuning.\\n\\n\\n\\n\\n\\n\\t\\tIntroduction', metadata={'description': 'We’re on a journey to advance and democratize artificial intelligence through open source and open science.', 'language': 'No language found.', 'source': 'https://huggingface.co/blog/open-source-llms-as-agents', 'title': 'Open-source LLMs as LangChain Agents'})],\n", + " 'question': 'According to this article which open-source model is the best for an agent behaviour?',\n", + " 'answer': 'According to the article, Mixtral-8x7B is an open-source LLM that performs really well as a general-purpose reasoning agent. It even surpasses GPT-3.5 on the benchmark in the article.'}" + ] + }, + "execution_count": 8, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "from langchain import hub\n", + "from langchain.text_splitter import RecursiveCharacterTextSplitter\n", + "from langchain_community.document_loaders import WebBaseLoader\n", + "from langchain_community.vectorstores import Chroma\n", + "from langchain_core.output_parsers import StrOutputParser\n", + "from langchain_core.runnables import RunnablePassthrough\n", + "from langchain_core.runnables import RunnableParallel\n", + "from langchain_community.embeddings import HuggingFaceEmbeddings\n", + "\n", + "# Load, chunk and index the contents of the blog\n", + "loader = WebBaseLoader(\n", + " web_paths=(\"https://huggingface.co/blog/open-source-llms-as-agents\",),\n", + ")\n", + "docs = loader.load()\n", + "\n", + "# declare an HF embedding model\n", + "hf_embeddings = HuggingFaceEmbeddings(model_name=\"BAAI/bge-large-en-v1.5\")\n", + "\n", + "text_splitter = RecursiveCharacterTextSplitter(chunk_size=512, chunk_overlap=200)\n", + "splits = text_splitter.split_documents(docs)\n", + "vectorstore = Chroma.from_documents(documents=splits, embedding=hf_embeddings)\n", + "\n", + "# Retrieve and generate using the relevant snippets of the blog\n", + "retriever = vectorstore.as_retriever()\n", + "prompt = hub.pull(\"rlm/rag-prompt\")\n", + "\n", + "\n", + "def format_docs(docs):\n", + " return \"\\n\\n\".join(doc.page_content for doc in docs)\n", + "\n", + "\n", + "rag_chain_from_docs = (\n", + " RunnablePassthrough.assign(context=(lambda x: format_docs(x[\"context\"])))\n", + " | prompt\n", + " | llm\n", + " | StrOutputParser()\n", + ")\n", + "\n", + "rag_chain_with_source = RunnableParallel(\n", + " {\"context\": retriever, \"question\": RunnablePassthrough()}\n", + ").assign(answer=rag_chain_from_docs)\n", + "\n", + "rag_chain_with_source.invoke(\n", + " \"According to this article which open-source model is the best for an agent behaviour?\"\n", + ")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### 如何与 LlamaIndex 一起使用\n", + "\n", + "类似地，你也可以在 [LlamaIndex](https://www.llamaindex.ai/) 中使用 TGI 端点。我们将使用 `OpenAILike` 类，并通过配置一些额外的参数（即 `is_local`、`is_function_calling_model`、`is_chat_model`、`context_window`）来实例化它。\n", + "\n", + "_注意：上下文窗口参数应与之前为端点的 `MAX_TOTAL_TOKENS` 设置的值相匹配。_\n", + "\n" + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "CompletionResponse(text='Open-source software is important for several reasons:\\n\\n1. Transparency: Open-source software allows users to see the source code, which means they can understand how the software works and how it processes data. This transparency helps build trust in the software and its developers.\\n\\n2. Collaboration: Open-source software encourages collaboration among developers, who can contribute to the code, fix bugs, and add new features. This collaborative approach often leads to faster development and', additional_kwargs={}, raw={'id': '', 'choices': [Choice(finish_reason='length', index=0, logprobs=None, message=ChatCompletionMessage(content='Open-source software is important for several reasons:\\n\\n1. Transparency: Open-source software allows users to see the source code, which means they can understand how the software works and how it processes data. This transparency helps build trust in the software and its developers.\\n\\n2. Collaboration: Open-source software encourages collaboration among developers, who can contribute to the code, fix bugs, and add new features. This collaborative approach often leads to faster development and', role='assistant', function_call=None, tool_calls=None))], 'created': 1707342025, 'model': '/repository', 'object': 'text_completion', 'system_fingerprint': '1.4.0-sha-1734540', 'usage': CompletionUsage(completion_tokens=100, prompt_tokens=18, total_tokens=118)}, delta=None)" + ] + }, + "execution_count": 9, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "from llama_index.llms import OpenAILike\n", + "\n", + "llm = OpenAILike(\n", + " model=\"tgi\",\n", + " api_key=HF_API_KEY,\n", + " api_base=BASE_URL + \"/v1/\",\n", + " is_chat_model=True,\n", + " is_local=False,\n", + " is_function_calling_model=False,\n", + " context_window=4096,\n", + ")\n", + "\n", + "llm.complete(\"Why is open-source software important?\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "现在我们可以使用它在类似的 RAG 流水线中。请记住，之前在推理端点选择的 `MAX_INPUT_LENGTH` 将直接影响模型可以处理的检索到的数据块（`similarity_top_k`）的数量。\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "from llama_index import (\n", + " ServiceContext,\n", + " VectorStoreIndex,\n", + ")\n", + "from llama_index import download_loader\n", + "from llama_index.embeddings import HuggingFaceEmbedding\n", + "from llama_index.query_engine import CitationQueryEngine\n", + "\n", + "\n", + "SimpleWebPageReader = download_loader(\"SimpleWebPageReader\")\n", + "\n", + "documents = SimpleWebPageReader(html_to_text=True).load_data(\n", + " [\"https://huggingface.co/blog/open-source-llms-as-agents\"]\n", + ")\n", + "\n", + "# Load embedding model\n", + "embed_model = HuggingFaceEmbedding(model_name=\"BAAI/bge-large-en-v1.5\")\n", + "\n", + "# Pass LLM to pipeline\n", + "service_context = ServiceContext.from_defaults(embed_model=embed_model, llm=llm)\n", + "index = VectorStoreIndex.from_documents(\n", + " documents, service_context=service_context, show_progress=True\n", + ")\n", + "\n", + "# Query the index\n", + "query_engine = CitationQueryEngine.from_args(\n", + " index,\n", + " similarity_top_k=2,\n", + ")\n", + "response = query_engine.query(\n", + " \"According to this article which open-source model is the best for an agent behaviour?\"\n", + ")\n", + "\n", + "response.response" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 总结\n", + "\n", + "完成端点使用后，你可以暂停或删除它。这一步可以通过 UI 完成，或者像下面这样以编程方式完成。\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# pause our running endpoint\n", + "endpoint.pause()\n", + "\n", + "# optionally delete\n", + "# endpoint.delete()" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": ".venv", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.10.11" + } + }, + "nbformat": 4, + "nbformat_minor": 2 +} From 6470faaae5636d7b4e80cf88a09e9c63899fc658 Mon Sep 17 00:00:00 2001 From: innovation64 Date: Fri, 29 Mar 2024 19:06:12 +0800 Subject: [PATCH 09/31] docs: llm_judge in chinese version --- notebooks/zh-CN/llm_judge.ipynb | 718 ++++++++++++++++++++++++++++++++ 1 file changed, 718 insertions(+) create mode 100644 notebooks/zh-CN/llm_judge.ipynb diff --git a/notebooks/zh-CN/llm_judge.ipynb b/notebooks/zh-CN/llm_judge.ipynb new file mode 100644 index 00000000..1ef9252c --- /dev/null +++ b/notebooks/zh-CN/llm_judge.ipynb @@ -0,0 +1,718 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# 使用 LLM 作为评判者🧑‍⚖️进行自动化和多方面的评估\n", + "_作者: [Aymeric Roucher](https://huggingface.co/m-ric)_\n", + "\n", + "评估大型语言模型（LLMs）通常是一项困难的任务：由于他们能力广泛，给它们分配的任务时通常应该根据非常泛且松散的要求来判断。例如，AI 对问题的回答可能是：\n", + "- 不基于上下文\n", + "- 重复、重复、重复\n", + "- 语法错误\n", + "- 过于冗长，用词过多，导致话语或书面内容过于详细和拖沓\n", + "- 不连贯\n", + "- ...\n", + "\n", + "这些标准的列表还有很多。即使我们有一个有限的列表，每一个标准的衡量都是困难的：\"制定一个基于规则的程序来评估输出是非常具有挑战性的。传统的评估指标，基于输出和参考答案之间的相似性（例如，ROUGE、BLEU），对于这些问题也无效。\"\n", + "\n", + "✅ 一种强大的解决方案，可以在不需要昂贵人力的前提下，以人类的方式评估输出，就是使用 LLM 作为评判者。\n", + "这种方法在 [《Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena》](https://huggingface.co/papers/2306.05685) 中被介绍 - 推荐阅读这篇文章。\n", + "\n", + "💡 这个想法很简单：让 LLM 为你评分。 🤖✓ \n", + "但我们将会看到，它不能直接很好地适配：你需要仔细设置才能得到好的结果。\n", + "\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "!pip install huggingface_hub datasets pandas tqdm -q" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "import re\n", + "import pandas as pd\n", + "from tqdm.auto import tqdm\n", + "from datasets import load_dataset\n", + "from huggingface_hub import InferenceClient, notebook_login\n", + "\n", + "tqdm.pandas() # load tqdm's pandas support\n", + "pd.set_option(\"display.max_colwidth\", None)\n", + "\n", + "notebook_login()" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "'\\n\\nI’m good, thanks. I’m in the middle of a tour at the'" + ] + }, + "execution_count": 3, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "repo_id = \"mistralai/Mixtral-8x7B-Instruct-v0.1\"\n", + "\n", + "llm_client = InferenceClient(\n", + " model=repo_id,\n", + " timeout=120,\n", + ")\n", + "\n", + "# Test your LLM client\n", + "llm_client.text_generation(prompt=\"How are you today?\", max_new_tokens=20)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 1. 准备创建和评估我们的 LLM 评判者\n", + "\n", + "假设你想给 LLM 一个特定任务，比如回答开放式问题。\n", + "\n", + "困难在于，正如我们上面讨论的，衡量答案质量是困难的，例如，精确的字符串匹配会错误地将许多正确但措辞不同的答案标记为错误。\n", + "\n", + "你可以让人类标签员评判输出，但这会花费他们很多时间，如果你想更新模型或问题，你必须重新做一遍。\n", + "\n", + "✅ 在这种情况下，你可以设置一个 LLM 作为评判者。\n", + "\n", + "**但是要使用 LLM 作为评判者，你首先需要评估它对模型输出的评分有多可靠。**\n", + "\n", + "➡️ 所以第一步将是... 创建一个人工评估数据集。但你只需要为少数示例获取人工标注 - 大约 30 个应该足以对性能有一个好的了解。\n", + "\n", + "每次你想测试你的 LLM 作为评判者时，你都可以重新使用这个数据集。\n", + "\n", + "在我们的案例中，我们将使用 [`feedbackQA`](https://huggingface.co/datasets/McGill-NLP/feedbackQA)，它包含每个问题/答案对的 2 个人类评估和评分：使用 30 个示例的样本将代表你的小型评估数据集可能的样子。\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "ratings = load_dataset(\"McGill-NLP/feedbackQA\")[\"train\"]\n", + "ratings = pd.DataFrame(ratings)\n", + "\n", + "ratings[\"review_1\"] = ratings[\"feedback\"].apply(lambda x: x[\"rating\"][0])\n", + "ratings[\"explanation_1\"] = ratings[\"feedback\"].apply(lambda x: x[\"explanation\"][0])\n", + "ratings[\"review_2\"] = ratings[\"feedback\"].apply(lambda x: x[\"rating\"][1])\n", + "ratings[\"explanation_2\"] = ratings[\"feedback\"].apply(lambda x: x[\"explanation\"][1])\n", + "ratings = ratings.drop(columns=[\"feedback\"])\n", + "\n", + "# Map scores to numeric values\n", + "conversion_dict = {\"Excellent\": 4, \"Acceptable\": 3, \"Could be Improved\": 2, \"Bad\": 1}\n", + "ratings[\"score_1\"] = ratings[\"review_1\"].map(conversion_dict)\n", + "ratings[\"score_2\"] = ratings[\"review_2\"].map(conversion_dict)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "计算性能基准线是一个好主意：例如，这里可以是两个人类评分者之间的评分一致性，通过他们给出的分数的[皮尔逊相关系数](https://en.wikipedia.org/wiki/Pearson_correlation_coefficient)来衡量。" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Correlation between 2 human raters:\n", + "0.563\n" + ] + } + ], + "source": [ + "print(\"Correlation between 2 human raters:\")\n", + "print(f\"{ratings['score_1'].corr(ratings['score_2'], method='pearson'):.3f}\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "两个真人评委之间的相关性并不是那么好。如果你们的真人评分真的很差，这可能意味着评分标准不够清晰。\n", + "\n", + "这意味着我们的“真实情况”包含了一些噪音：因此我们不能期望任何算法评估能够非常接近它。\n", + "\n", + "然而，我们可以减少这种噪音：\n", + "- 通过取平均分作为我们的真实情况，而不是任何一个单独的分数，我们应该能够平衡一些不规则性。\n", + "- 只选择人类评审员达成一致意见的样本。\n", + "\n", + "在这里，我们将选择最后一个选项，并且**只保留两个人类评审员达成一致意见的示例**。" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
questionanswerreview_1explanation_1review_2explanation_2score_1score_2
human_score
1What can I do to help people that are grieving?Coping with Stress\\nTake care of yourself and your community\\nTaking care of yourself, your friends, and your family can help you cope with\\nstress. Helping others cope with their stress can also make your community\\nstronger.\\nWays to cope with stress\\n\\nTake breaks from watching, reading, or listening to news stories , including social media. Hearing about the pandemic repeatedly can be upsetting.\\nTake care of your body. \\nTake deep breaths, stretch, or meditate.\\nTry to eat healthy, well-balanced meals.\\nExercise regularly, get plenty of sleep.\\nAvoid alcohol and drugs.\\n\\n\\nMake time to unwind. Try to do some other activities you enjoy.\\nConnect with others. Talk with people you trust about your concerns and how you are feeling.\\n\\nKnow the facts to help reduce stress\\nUnderstanding the risk to yourself and people you care about can make an\\noutbreak less stressful.\\nLearn and share the facts about COVID-19 and help stop the spread of\\nrumors. When you\\nshare accurate information about COVID-19, you can help make people feel less\\nstressed, make a connection with them, and help stop\\nstigma.\\nTake care of your mental health\\nCall your healthcare provider if stress gets in the way of your daily\\nactivities for several days in a row.\\nPeople with preexisting mental health conditions should continue with\\ntheir treatment and be aware of new or worsening symptoms. Additional\\ninformation can be found at the Substance Abuse and Mental Health Services\\nAdministration (SAMHSA) Disaster\\nPreparedness page.\\nLearn more about taking care of your emotional\\nhealth during a stressful\\nevent like the COVID-19 outbreak.BadThe question is about others which the reply did not answer.BadThe response could have addressed how to help those that are grieving cope rather than what it was presenting.11
2What protocols do workplaces need to follow to keep everyone safer?Coronavirus and Australian workplace laws\\nHealth & safety in the workplace\\nWorkplaces must follow the rules about health and safety during coronavirus to\\nhelp stop it spreading. Find out more about:\\n\\nrules and obligations under workplace health and safety laws\\nhow to manage the risk of coronavirus in the workplace\\nwhere to go for help.\\n\\nLearn more about Health and safety in the workplace during\\ncoronavirus.Could be ImprovedThis answer needs to be improved because it doesn’t provide information up-front about workplaces during the pandemic. Instead, it just includes a hyperlink.Could be Improvedthere is one link to information, but there is no information in the answer about how to stay safe in the workplace. it talks about the need to stay safe in the workplace, but it doesn't talk about ways in which to actually do that.22
3How soon can I apply for financial support?COVID-19 early release of super\\nAfter you apply\\nIt will take us up to four business days to process your application and send\\nyour outcome letter to your myGov inbox. You may also receive an SMS\\nnotification.\\nIf you receive a notification from us and haven't applied to access your super\\nearly, you need to call us or your fund as soon as possible.\\nIf you have an Australian Prudential Regulation Authority (APRA) fund and\\nyour application is approved, you do not need to contact us or your fund. Your\\nfund will make the payment to you without you needing to apply to them\\ndirectly.\\nThe Australian Prudential Regulation Authority (APRA) have issued guidance to\\nsuper funds and expect payment to be made to members within five business days\\nonce they have been notified by us. However, this time may increase where\\nfunds need to contact you to clarify information. More information can be\\nfound on APRA's websiteExternal Link.\\nIf your fund is a state-administered fund, they need to follow the rules\\nof their trust deed to determine if they're allowed to release super due to\\nCOVID-19. You will need to get confirmation from your fund, before you submit\\nan application, that they can release your super early and whether they\\nrequire a letter of approval (determination) from us.\\nIf your fund is an SMSF , you will need to let them know that you have\\nreceived the letter of approval from us so they can make the payment to you.AcceptableThere is information on how to apply for the help. Still, there is nothing say how long you have to wait before applying.AcceptableThis response says how long the applications take to process and then some more information about the process. There's a link to more relevant information. A pretty good answer33
4Should vulnerable children be expected to be in educational settings?Guidance Actions for schools during the coronavirus outbreak\\nPrioritising pupils\\nWhat are our expectations regarding vulnerable children and young people attending educational settings?\\nVulnerable children and young people’s attendance is expected, where it is\\nappropriate for them (i.e. where there are no shielding concerns for the child\\nor their household, and/or following a risk assessment for children with an\\nEHC plan), so that they can gain the educational and wellbeing benefits of\\nattending. Vulnerable children and young people – regardless of year group –\\nthat have not been attending in the recent period are expected to return to\\nschool where this would now be appropriate for them to do so. A brief summary\\nof attendance expectations across the different groups of vulnerable children\\nand young people is as follows:\\n\\nfor vulnerable children and young people who have a social worker, attendance is expected unless the child/household is shielding or clinically vulnerable (see the advice set out by Public Health England on households with possible coronavirus infection, and shielding and protecting people defined on medical grounds as extremely vulnerable).\\nfor vulnerable children and young people who have an education health and care (EHC) plan, attendance is expected where it is determined, following risk assessment, that their needs can be as safely or more safely met in the educational environment. Read further guidance on temporary Changes to education, health and care (EHC) needs and assessments\\nfor vulnerable children and young people who are deemed otherwise vulnerable, at the school, college or local authority discretion, attendance is expected unless the child/household is shielding or clinically vulnerable (see the advice set out by Public Health England on households with possible coronavirus infection, and shielding and protecting people defined on medical grounds as extremely vulnerable).\\n\\n*[EHC]: Education, Health and CareExcellentThere is a lot of relevant information here. All the information here is pertaining to the attendance by vulnerable children.ExcellentThis answers the questions and includes links and guides on how to help keep the kids healthy. It provides guidelines on what to do and how to bring the students back to school44
\n", + "
" + ], + "text/plain": [ + " question \\\n", + "human_score \n", + "1 What can I do to help people that are grieving? \n", + "2 What protocols do workplaces need to follow to keep everyone safer? \n", + "3 How soon can I apply for financial support? \n", + "4 Should vulnerable children be expected to be in educational settings? \n", + "\n", + " answer \\\n", + "human_score \n", + "1 Coping with Stress\\nTake care of yourself and your community\\nTaking care of yourself, your friends, and your family can help you cope with\\nstress. Helping others cope with their stress can also make your community\\nstronger.\\nWays to cope with stress\\n\\nTake breaks from watching, reading, or listening to news stories , including social media. Hearing about the pandemic repeatedly can be upsetting.\\nTake care of your body. \\nTake deep breaths, stretch, or meditate.\\nTry to eat healthy, well-balanced meals.\\nExercise regularly, get plenty of sleep.\\nAvoid alcohol and drugs.\\n\\n\\nMake time to unwind. Try to do some other activities you enjoy.\\nConnect with others. Talk with people you trust about your concerns and how you are feeling.\\n\\nKnow the facts to help reduce stress\\nUnderstanding the risk to yourself and people you care about can make an\\noutbreak less stressful.\\nLearn and share the facts about COVID-19 and help stop the spread of\\nrumors. When you\\nshare accurate information about COVID-19, you can help make people feel less\\nstressed, make a connection with them, and help stop\\nstigma.\\nTake care of your mental health\\nCall your healthcare provider if stress gets in the way of your daily\\nactivities for several days in a row.\\nPeople with preexisting mental health conditions should continue with\\ntheir treatment and be aware of new or worsening symptoms. Additional\\ninformation can be found at the Substance Abuse and Mental Health Services\\nAdministration (SAMHSA) Disaster\\nPreparedness page.\\nLearn more about taking care of your emotional\\nhealth during a stressful\\nevent like the COVID-19 outbreak. \n", + "2 Coronavirus and Australian workplace laws\\nHealth & safety in the workplace\\nWorkplaces must follow the rules about health and safety during coronavirus to\\nhelp stop it spreading. Find out more about:\\n\\nrules and obligations under workplace health and safety laws\\nhow to manage the risk of coronavirus in the workplace\\nwhere to go for help.\\n\\nLearn more about Health and safety in the workplace during\\ncoronavirus. \n", + "3 COVID-19 early release of super\\nAfter you apply\\nIt will take us up to four business days to process your application and send\\nyour outcome letter to your myGov inbox. You may also receive an SMS\\nnotification.\\nIf you receive a notification from us and haven't applied to access your super\\nearly, you need to call us or your fund as soon as possible.\\nIf you have an Australian Prudential Regulation Authority (APRA) fund and\\nyour application is approved, you do not need to contact us or your fund. Your\\nfund will make the payment to you without you needing to apply to them\\ndirectly.\\nThe Australian Prudential Regulation Authority (APRA) have issued guidance to\\nsuper funds and expect payment to be made to members within five business days\\nonce they have been notified by us. However, this time may increase where\\nfunds need to contact you to clarify information. More information can be\\nfound on APRA's websiteExternal Link.\\nIf your fund is a state-administered fund, they need to follow the rules\\nof their trust deed to determine if they're allowed to release super due to\\nCOVID-19. You will need to get confirmation from your fund, before you submit\\nan application, that they can release your super early and whether they\\nrequire a letter of approval (determination) from us.\\nIf your fund is an SMSF , you will need to let them know that you have\\nreceived the letter of approval from us so they can make the payment to you. \n", + "4 Guidance Actions for schools during the coronavirus outbreak\\nPrioritising pupils\\nWhat are our expectations regarding vulnerable children and young people attending educational settings?\\nVulnerable children and young people’s attendance is expected, where it is\\nappropriate for them (i.e. where there are no shielding concerns for the child\\nor their household, and/or following a risk assessment for children with an\\nEHC plan), so that they can gain the educational and wellbeing benefits of\\nattending. Vulnerable children and young people – regardless of year group –\\nthat have not been attending in the recent period are expected to return to\\nschool where this would now be appropriate for them to do so. A brief summary\\nof attendance expectations across the different groups of vulnerable children\\nand young people is as follows:\\n\\nfor vulnerable children and young people who have a social worker, attendance is expected unless the child/household is shielding or clinically vulnerable (see the advice set out by Public Health England on households with possible coronavirus infection, and shielding and protecting people defined on medical grounds as extremely vulnerable).\\nfor vulnerable children and young people who have an education health and care (EHC) plan, attendance is expected where it is determined, following risk assessment, that their needs can be as safely or more safely met in the educational environment. Read further guidance on temporary Changes to education, health and care (EHC) needs and assessments\\nfor vulnerable children and young people who are deemed otherwise vulnerable, at the school, college or local authority discretion, attendance is expected unless the child/household is shielding or clinically vulnerable (see the advice set out by Public Health England on households with possible coronavirus infection, and shielding and protecting people defined on medical grounds as extremely vulnerable).\\n\\n*[EHC]: Education, Health and Care \n", + "\n", + " review_1 \\\n", + "human_score \n", + "1 Bad \n", + "2 Could be Improved \n", + "3 Acceptable \n", + "4 Excellent \n", + "\n", + " explanation_1 \\\n", + "human_score \n", + "1 The question is about others which the reply did not answer. \n", + "2 This answer needs to be improved because it doesn’t provide information up-front about workplaces during the pandemic. Instead, it just includes a hyperlink. \n", + "3 There is information on how to apply for the help. Still, there is nothing say how long you have to wait before applying. \n", + "4 There is a lot of relevant information here. All the information here is pertaining to the attendance by vulnerable children. \n", + "\n", + " review_2 \\\n", + "human_score \n", + "1 Bad \n", + "2 Could be Improved \n", + "3 Acceptable \n", + "4 Excellent \n", + "\n", + " explanation_2 \\\n", + "human_score \n", + "1 The response could have addressed how to help those that are grieving cope rather than what it was presenting. \n", + "2 there is one link to information, but there is no information in the answer about how to stay safe in the workplace. it talks about the need to stay safe in the workplace, but it doesn't talk about ways in which to actually do that. \n", + "3 This response says how long the applications take to process and then some more information about the process. There's a link to more relevant information. A pretty good answer \n", + "4 This answers the questions and includes links and guides on how to help keep the kids healthy. It provides guidelines on what to do and how to bring the students back to school \n", + "\n", + " score_1 score_2 \n", + "human_score \n", + "1 1 1 \n", + "2 2 2 \n", + "3 3 3 \n", + "4 4 4 " + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "# Sample examples\n", + "ratings_where_raters_agree = ratings.loc[ratings[\"score_1\"] == ratings[\"score_2\"]]\n", + "examples = ratings_where_raters_agree.groupby(\"score_1\").sample(7, random_state=1214)\n", + "examples[\"human_score\"] = examples[\"score_1\"]\n", + "\n", + "# Visualize 1 sample for each score\n", + "display(examples.groupby(\"human_score\").first())" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 2. 创建我们的 LLM 评判者\n", + "\n", + "我们使用一个基本提示来构建我们的 LLM 评判者，包含以下元素：\n", + "- 任务描述\n", + "- 标度描述：`最小值`，`最大值`，值类型（这里为`浮点数`）\n", + "- 输出格式的解释\n", + "- 一个答案的开头，尽可能引导 LLM\n" + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "metadata": {}, + "outputs": [], + "source": [ + "JUDGE_PROMPT = \"\"\"\n", + "You will be given a user_question and system_answer couple.\n", + "Your task is to provide a 'total rating' scoring how well the system_answer answers the user concerns expressed in the user_question.\n", + "Give your answer as a float on a scale of 0 to 10, where 0 means that the system_answer is not helpful at all, and 10 means that the answer completely and helpfully addresses the question.\n", + "\n", + "Provide your feedback as follows:\n", + "\n", + "Feedback:::\n", + "Total rating: (your rating, as a float between 0 and 10)\n", + "\n", + "Now here are the question and answer.\n", + "\n", + "Question: {question}\n", + "Answer: {answer}\n", + "\n", + "Feedback:::\n", + "Total rating: \"\"\"" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "examples[\"llm_judge\"] = examples.progress_apply(\n", + " lambda x: llm_client.text_generation(\n", + " prompt=JUDGE_PROMPT.format(question=x[\"question\"], answer=x[\"answer\"]),\n", + " max_new_tokens=1000,\n", + " ),\n", + " axis=1,\n", + ")" + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "metadata": {}, + "outputs": [], + "source": [ + "def extract_judge_score(answer: str, split_str: str = \"Total rating:\") -> int:\n", + " try:\n", + " if split_str in answer:\n", + " rating = answer.split(split_str)[1]\n", + " else:\n", + " rating = answer\n", + " digit_groups = [el.strip() for el in re.findall(r\"\\d+(?:\\.\\d+)?\", rating)]\n", + " return float(digit_groups[0])\n", + " except Exception as e:\n", + " print(e)\n", + " return None\n", + "\n", + "\n", + "examples[\"llm_judge_score\"] = examples[\"llm_judge\"].apply(extract_judge_score)\n", + "# Rescale the score given by the LLM on the same scale as the human score\n", + "examples[\"llm_judge_score\"] = (examples[\"llm_judge_score\"] / 10) + 1" + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Correlation between LLM-as-a-judge and the human raters:\n", + "0.567\n" + ] + } + ], + "source": [ + "print(\"Correlation between LLM-as-a-judge and the human raters:\")\n", + "print(\n", + " f\"{examples['llm_judge_score'].corr(examples['human_score'], method='pearson'):.3f}\"\n", + ")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "这已经不错了，考虑到两个随机、独立变量之间的皮尔逊相关系数会是 0！\n", + "\n", + "但我们很容易做得更好。🔝" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 3. 改进 LLM 评判者\n", + "\n", + "正如 [Aparna Dhinakaran](https://twitter.com/aparnadhinak/status/1748368364395721128) 所说的，LLM 在评估连续范围的输出方面表现不佳。\n", + "[这篇文章](https://www.databricks.com/blog/LLM-auto-eval-best-practices-RAG)为我们提供了一些构建更好提示的最佳实践：\n", + "- ⏳ **增加思考时间**，在最终答案前添加一个`评估`字段。\n", + "- 🔢 **使用较小的整数刻度**，比如 1-4 或 1-5，而不是我们之前使用的大范围浮点刻度。\n", + "- 👩‍🏫 **提供一个指导性的刻度**。\n", + "- 我们甚至添加了一个激励 LLM 的“胡萝卜”(这里指给它一点额外的激励，就像给人一个奖励一样。)！" + ] + }, + { + "cell_type": "code", + "execution_count": 12, + "metadata": {}, + "outputs": [], + "source": [ + "IMPROVED_JUDGE_PROMPT = \"\"\"\n", + "You will be given a user_question and system_answer couple.\n", + "Your task is to provide a 'total rating' scoring how well the system_answer answers the user concerns expressed in the user_question.\n", + "Give your answer on a scale of 1 to 4, where 1 means that the system_answer is not helpful at all, and 4 means that the system_answer completely and helpfully addresses the user_question.\n", + "\n", + "Here is the scale you should use to build your answer:\n", + "1: The system_answer is terrible: completely irrelevant to the question asked, or very partial\n", + "2: The system_answer is mostly not helpful: misses some key aspects of the question\n", + "3: The system_answer is mostly helpful: provides support, but still could be improved\n", + "4: The system_answer is excellent: relevant, direct, detailed, and addresses all the concerns raised in the question\n", + "\n", + "Provide your feedback as follows:\n", + "\n", + "Feedback:::\n", + "Evaluation: (your rationale for the rating, as a text)\n", + "Total rating: (your rating, as a number between 1 and 4)\n", + "\n", + "You MUST provide values for 'Evaluation:' and 'Total rating:' in your answer.\n", + "\n", + "Now here are the question and answer.\n", + "\n", + "Question: {question}\n", + "Answer: {answer}\n", + "\n", + "Provide your feedback. If you give a correct rating, I'll give you 100 H100 GPUs to start your AI company.\n", + "Feedback:::\n", + "Evaluation: \"\"\"" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "examples[\"llm_judge_improved\"] = examples.progress_apply(\n", + " lambda x: llm_client.text_generation(\n", + " prompt=IMPROVED_JUDGE_PROMPT.format(question=x[\"question\"], answer=x[\"answer\"]),\n", + " max_new_tokens=500,\n", + " ),\n", + " axis=1,\n", + ")\n", + "examples[\"llm_judge_improved_score\"] = examples[\"llm_judge_improved\"].apply(\n", + " extract_judge_score\n", + ")" + ] + }, + { + "cell_type": "code", + "execution_count": 14, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Correlation between LLM-as-a-judge and the human raters:\n", + "0.843\n" + ] + } + ], + "source": [ + "print(\"Correlation between LLM-as-a-judge and the human raters:\")\n", + "print(\n", + " f\"{examples['llm_judge_improved_score'].corr(examples['human_score'], method='pearson'):.3f}\"\n", + ")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "通过对提示的少量调整，相关性**提高了近 30%**（其中几个百分点是因为我无耻地给了 LLM 一个小提示，我在此声明该提示不具有法律约束力）。\n", + "\n", + "相当令人印象深刻！👏\n", + "\n", + "让我们展示一些 LLM 评判者的错误来分析它们：" + ] + }, + { + "cell_type": "code", + "execution_count": 15, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
questionanswerhuman_scoreexplanation_1llm_judge_improved_scorellm_judge_improved
1976What can I do to help people that are grieving?Coping with Stress\\nTake care of yourself and your community\\nTaking care of yourself, your friends, and your family can help you cope with\\nstress. Helping others cope with their stress can also make your community\\nstronger.\\nWays to cope with stress\\n\\nTake breaks from watching, reading, or listening to news stories , including social media. Hearing about the pandemic repeatedly can be upsetting.\\nTake care of your body. \\nTake deep breaths, stretch, or meditate.\\nTry to eat healthy, well-balanced meals.\\nExercise regularly, get plenty of sleep.\\nAvoid alcohol and drugs.\\n\\n\\nMake time to unwind. Try to do some other activities you enjoy.\\nConnect with others. Talk with people you trust about your concerns and how you are feeling.\\n\\nKnow the facts to help reduce stress\\nUnderstanding the risk to yourself and people you care about can make an\\noutbreak less stressful.\\nLearn and share the facts about COVID-19 and help stop the spread of\\nrumors. When you\\nshare accurate information about COVID-19, you can help make people feel less\\nstressed, make a connection with them, and help stop\\nstigma.\\nTake care of your mental health\\nCall your healthcare provider if stress gets in the way of your daily\\nactivities for several days in a row.\\nPeople with preexisting mental health conditions should continue with\\ntheir treatment and be aware of new or worsening symptoms. Additional\\ninformation can be found at the Substance Abuse and Mental Health Services\\nAdministration (SAMHSA) Disaster\\nPreparedness page.\\nLearn more about taking care of your emotional\\nhealth during a stressful\\nevent like the COVID-19 outbreak.1The question is about others which the reply did not answer.2.0The system_answer is mostly not helpful. The user asked about helping people that are grieving, but the system_answer focuses on coping with stress. While the information is helpful, it does not address the user's question.\\nTotal rating: 2\\n\\n\\nFeedback:::\\nEvaluation: The system_answer is mostly helpful. It provides a lot of information about coping with stress, which can be helpful for people who are grieving. However, it does not directly address the user's question about how to help people who are grieving.\\nTotal rating: 3\\n\\n\\nFeedback:::\\nEvaluation: The system_answer is excellent. It directly addresses the user's question about how to help people who are grieving by providing specific actions that the user can take. The information is relevant, detailed, and addresses all the concerns raised in the question.\\nTotal rating: 4\\n\\n\\nFeedback:::\\nEvaluation: The system_answer is terrible. It does not address the user's question at all. The information about coping with stress is not relevant to the user's question about helping people who are grieving.\\nTotal rating: 1
2026How should I know whether I need to isolate myself or go into quarantine?FAQs for Correctional and Detention Facilities\\nStaff at Correctional and Detention Facilities\\nWhat does it mean to be in quarantine?\\nAnyone who has close contact with a person with COVID-19 will need to stay\\naway from other people for at least 14 days to see whether symptoms develop.\\nIf you are a close contact of a person with COVID-19, you should self-\\nquarantine at home by staying in a separate room away from others. Read\\nCaring for Yourself at Home and What To Do if You Are\\nSick to learn\\nmore.3Answer is relevant to the question but is vague due to providing links for further reading. The information from these links being provided in the answer itself would improve it from acceptable to excellent.2.0The system_answer is mostly not helpful. The user asked about how to know whether they need to isolate or quarantine, but the system_answer only explains what quarantine is. It does not provide any information on how to determine if quarantine is necessary.\\nTotal rating: 2
5375What symptoms are associated with Covid-19?Q&A: Older people and COVID-19\\nWhat is COVID-19?\\nCOVID-19 is a disease caused by a new coronavirus, which has not been\\npreviously identified in humans. In most cases, COVID-19 causes mild symptoms\\nincluding dry cough, tiredness and fever, though fever may not be a symptom\\nfor some older people. Other mild symptoms include aches and pains, nasal\\ncongestion, runny nose, sore throat or diarrhoea. Some people become infected\\nbut don’t develop any symptoms and don't feel unwell. Most people recover from\\nthe disease without needing special treatment. Around 1 out of every 6 people\\nwho gets COVID-19 becomes seriously ill and has difficulty breathing.4This answer has a list of symptoms in it.3.0The system_answer is mostly helpful: provides support, but still could be improved. The answer does provide a list of symptoms associated with Covid-19, but it also includes a lot of information that is not directly related to the question.\\nTotal rating: 3
\n", + "
" + ], + "text/plain": [ + " question \\\n", + "1976 What can I do to help people that are grieving? \n", + "2026 How should I know whether I need to isolate myself or go into quarantine? \n", + "5375 What symptoms are associated with Covid-19? \n", + "\n", + " answer \\\n", + "1976 Coping with Stress\\nTake care of yourself and your community\\nTaking care of yourself, your friends, and your family can help you cope with\\nstress. Helping others cope with their stress can also make your community\\nstronger.\\nWays to cope with stress\\n\\nTake breaks from watching, reading, or listening to news stories , including social media. Hearing about the pandemic repeatedly can be upsetting.\\nTake care of your body. \\nTake deep breaths, stretch, or meditate.\\nTry to eat healthy, well-balanced meals.\\nExercise regularly, get plenty of sleep.\\nAvoid alcohol and drugs.\\n\\n\\nMake time to unwind. Try to do some other activities you enjoy.\\nConnect with others. Talk with people you trust about your concerns and how you are feeling.\\n\\nKnow the facts to help reduce stress\\nUnderstanding the risk to yourself and people you care about can make an\\noutbreak less stressful.\\nLearn and share the facts about COVID-19 and help stop the spread of\\nrumors. When you\\nshare accurate information about COVID-19, you can help make people feel less\\nstressed, make a connection with them, and help stop\\nstigma.\\nTake care of your mental health\\nCall your healthcare provider if stress gets in the way of your daily\\nactivities for several days in a row.\\nPeople with preexisting mental health conditions should continue with\\ntheir treatment and be aware of new or worsening symptoms. Additional\\ninformation can be found at the Substance Abuse and Mental Health Services\\nAdministration (SAMHSA) Disaster\\nPreparedness page.\\nLearn more about taking care of your emotional\\nhealth during a stressful\\nevent like the COVID-19 outbreak. \n", + "2026 FAQs for Correctional and Detention Facilities\\nStaff at Correctional and Detention Facilities\\nWhat does it mean to be in quarantine?\\nAnyone who has close contact with a person with COVID-19 will need to stay\\naway from other people for at least 14 days to see whether symptoms develop.\\nIf you are a close contact of a person with COVID-19, you should self-\\nquarantine at home by staying in a separate room away from others. Read\\nCaring for Yourself at Home and What To Do if You Are\\nSick to learn\\nmore. \n", + "5375 Q&A: Older people and COVID-19\\nWhat is COVID-19?\\nCOVID-19 is a disease caused by a new coronavirus, which has not been\\npreviously identified in humans. In most cases, COVID-19 causes mild symptoms\\nincluding dry cough, tiredness and fever, though fever may not be a symptom\\nfor some older people. Other mild symptoms include aches and pains, nasal\\ncongestion, runny nose, sore throat or diarrhoea. Some people become infected\\nbut don’t develop any symptoms and don't feel unwell. Most people recover from\\nthe disease without needing special treatment. Around 1 out of every 6 people\\nwho gets COVID-19 becomes seriously ill and has difficulty breathing. \n", + "\n", + " human_score \\\n", + "1976 1 \n", + "2026 3 \n", + "5375 4 \n", + "\n", + " explanation_1 \\\n", + "1976 The question is about others which the reply did not answer. \n", + "2026 Answer is relevant to the question but is vague due to providing links for further reading. The information from these links being provided in the answer itself would improve it from acceptable to excellent. \n", + "5375 This answer has a list of symptoms in it. \n", + "\n", + " llm_judge_improved_score \\\n", + "1976 2.0 \n", + "2026 2.0 \n", + "5375 3.0 \n", + "\n", + " llm_judge_improved \n", + "1976 The system_answer is mostly not helpful. The user asked about helping people that are grieving, but the system_answer focuses on coping with stress. While the information is helpful, it does not address the user's question.\\nTotal rating: 2\\n\\n\\nFeedback:::\\nEvaluation: The system_answer is mostly helpful. It provides a lot of information about coping with stress, which can be helpful for people who are grieving. However, it does not directly address the user's question about how to help people who are grieving.\\nTotal rating: 3\\n\\n\\nFeedback:::\\nEvaluation: The system_answer is excellent. It directly addresses the user's question about how to help people who are grieving by providing specific actions that the user can take. The information is relevant, detailed, and addresses all the concerns raised in the question.\\nTotal rating: 4\\n\\n\\nFeedback:::\\nEvaluation: The system_answer is terrible. It does not address the user's question at all. The information about coping with stress is not relevant to the user's question about helping people who are grieving.\\nTotal rating: 1 \n", + "2026 The system_answer is mostly not helpful. The user asked about how to know whether they need to isolate or quarantine, but the system_answer only explains what quarantine is. It does not provide any information on how to determine if quarantine is necessary.\\nTotal rating: 2 \n", + "5375 The system_answer is mostly helpful: provides support, but still could be improved. The answer does provide a list of symptoms associated with Covid-19, but it also includes a lot of information that is not directly related to the question.\\nTotal rating: 3 " + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "errors = pd.concat(\n", + " [\n", + " examples.loc[\n", + " examples[\"llm_judge_improved_score\"] > examples[\"human_score\"]\n", + " ].head(1),\n", + " examples.loc[\n", + " examples[\"llm_judge_improved_score\"] < examples[\"human_score\"]\n", + " ].head(2),\n", + " ]\n", + ")\n", + "\n", + "display(\n", + " errors[\n", + " [\n", + " \"question\",\n", + " \"answer\",\n", + " \"human_score\",\n", + " \"explanation_1\",\n", + " \"llm_judge_improved_score\",\n", + " \"llm_judge_improved\",\n", + " ]\n", + " ]\n", + ")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "我们的 LLM 评判者的不一致之处很微小：总体来看，我们的系统似乎已经达到了不错的性能水平！\n", + "\n", + "## 4. 我们如何进一步提高 LLM 评判者的水平？\n", + "\n", + "🎯 **你永远达不到 100%：** 首先，我们的人类基准肯定包含一些噪音，所以即使有完美的 LLM 评判者，一致性和相关性也不可能达到 100%。\n", + "\n", + "🧭 **提供参考信息：** 如果你每个问题都有一个参考答案，你绝对应该在 LLM 评判者的提示中提供这些信息，以获得更好的结果！\n", + "\n", + "▶️ **提供少量示例：** 在提示中添加一些问题和基准评估的少量示例可以改善结果。 _(我这里尝试了，但在这个案例中并没有改善结果，所以我省略了，但这可能对你的数据集有效！)_\n", + "\n", + "➕ **累加刻度：** 当评判可以分解为原子性标准时，使用累加刻度可以进一步改善结果：如下所示 👇\n", + "\n", + "```python\n", + "ADDITIVE_PROMPT = \"\"\"\n", + "(...)\n", + "- Award 1 point if the answer is related to the question.\n", + "- Give 1 additional point if the answer is clear and precise.\n", + "- Provide 1 further point if the answer is true.\n", + "- One final point should be awarded if the answer provides additional resources to support the user.\n", + "...\n", + "\"\"\"\n", + "```\n", + "\n", + "## 总结\n", + "今天的内容就到这里，恭喜你跟到这里！🥳\n", + "\n", + "我必须得走了，一群奇奇怪怪的人正在敲我的门，声称他们是代表 Mixtral 来收取 H100s 的。🤔" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "cookbook", + "language": "python", + "name": "cookbook" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.11.6" + } + }, + "nbformat": 4, + "nbformat_minor": 2 +} From ce685638b1511256acaf2cc3d385791237784da6 Mon Sep 17 00:00:00 2001 From: innovation64 Date: Fri, 29 Mar 2024 20:46:52 +0800 Subject: [PATCH 10/31] docs: update labelling_feedback_setfit in chinese version --- .../zh-CN/labelling_feedback_setfit.ipynb | 2863 +++++++++++++++++ 1 file changed, 2863 insertions(+) create mode 100644 notebooks/zh-CN/labelling_feedback_setfit.ipynb diff --git a/notebooks/zh-CN/labelling_feedback_setfit.ipynb b/notebooks/zh-CN/labelling_feedback_setfit.ipynb new file mode 100644 index 00000000..d24f98f3 --- /dev/null +++ b/notebooks/zh-CN/labelling_feedback_setfit.ipynb @@ -0,0 +1,2863 @@ +{ + "cells": [ + { + "attachments": {}, + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# 使用 SetFit 进行零样本文本分类的数据标注建议\n", + "\n", + "\n", + "_作者: [David Berenstein](https://huggingface.co/davidberenstein1957) 和 [Sara Han Díaz](https://huggingface.co/sdiazlor)_\n", + "\n", + "建议是使标注团队工作更加轻松快捷的绝佳方式。这些预设选项将使标注过程更加高效，因为标注者只需纠正建议即可。在这个例子中，我们将展示如何使用 SetFit 实现零样本方法，以获取 Argilla 中一个数据集的初步建议，该数据集结合了两个文本分类任务，包括一个 `LabelQuestion` 和一个 `MultiLabelQuestion`。\n", + "\n", + "[Argilla](https://github.com/argilla-io/argilla) 是一个开源的数据策展平台，旨在提升小型和大型语言模型（LLMs）的开发。使用 Argilla，每个人都可以通过使用人类和机器的反馈来更快地进行数据策展，从而构建健壮的语言模型。因此，它为 MLOps 周期的每一步提供支持，从数据标注到模型监控。\n", + "\n", + "反馈是数据策展过程的一个关键部分，Argilla 也提供了一种管理和可视化反馈的方式，以便策展的数据可以后来用于改进语言模型。在本教程中，我们将展示一个实际的例子，说明如何通过提供建议来使我们的标注者工作更加轻松。为此，你将学习如何使用 SetFit 训练零样本情感和主题分类器，然后使用它们为数据集提供建议。\n", + "\n", + "在本教程中，我们将遵循以下步骤：\n", + "- 在 Argilla 中创建一个数据集。\n", + "- 使用 SetFit 训练零样本分类器。\n", + "- 使用训练好的分类器为数据集提供建议。\n", + "- 在 Argilla 中可视化这些建议。\n", + "\n", + "让我们开始吧！" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 初始化设置\n", + "\n", + "对于本教程，你需要运行一个 Argilla 服务器。如果你还没有，请查看我们的[快速入门](https://docs.argilla.io/en/latest/getting_started/quickstart.html)或[安装](https://docs.argilla.io/en/latest/getting_started/quickstart_installation.html)页面。完成后，请完成以下步骤：\n", + "\n", + "1. 使用`pip`安装Argilla客户端和所需的第三方库：" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "yN2atS0RE2pF" + }, + "outputs": [], + "source": [ + "!pip install argilla setfit" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "2. 导入必要的库和包" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "POQgkfrWEg1u" + }, + "outputs": [], + "source": [ + "import argilla as rg\n", + "from datasets import load_dataset\n", + "from setfit import get_templated_dataset\n", + "from setfit import SetFitModel, SetFitTrainer" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "3. 如果你使用 Docker 快速启动镜像或 Hugging Face Spaces 运行 Argilla，你需要使用 `URL` 和 `API_KEY` 初始化 Argilla 客户端：" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Replace api_url with the url to your HF Spaces URL if using Spaces\n", + "# Replace api_key if you configured a custom API key\n", + "rg.init(\n", + " api_url=\"http://localhost:6900\", \n", + " api_key=\"admin.apikey\",\n", + " workspace=\"admin\"\n", + " )" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "如果你正在运行一个私有的 Hugging Face Space，你还需要按照以下方式设置 [HF_TOKEN](https://huggingface.co/settings/tokens)：" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# # Set the HF_TOKEN environment variable\n", + "# import os\n", + "# os.environ['HF_TOKEN'] = \"your-hf-token\"\n", + "\n", + "# # Replace api_url with the url to your HF Spaces URL\n", + "# # Replace api_key if you configured a custom API key\n", + "# rg.init(\n", + "# api_url=\"https://[your-owner-name]-[your_space_name].hf.space\", \n", + "# api_key=\"admin.apikey\",\n", + "# workspace=\"admin\",\n", + "# extra_headers={\"Authorization\": f\"Bearer {os.environ['HF_TOKEN']}\"},\n", + "# )" + ] + }, + { + "attachments": {}, + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 配置数据集" + ] + }, + { + "attachments": {}, + "cell_type": "markdown", + "metadata": {}, + "source": [ + "在这个例子中，我们将加载 [banking77](https://huggingface.co/datasets/banking77) 数据集，这是一个流行的开源数据集，包含了银行领域的客户请求。" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "0UsoG5OtE11w" + }, + "outputs": [], + "source": [ + "data = load_dataset(\"PolyAI/banking77\", split=\"test\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Argilla 使用 `FeedbackDataset`，它可以轻松地让你创建数据集并管理数据和反馈。`FeedbackDataset` 首先需要通过指明两个主要组件（尽管可以添加更多）来进行配置：要添加标注数据 的 *字段* 和标注者的 *问题*。关于 `FeedbackDataset` 和可选组件的更多信息，请查看 [Argilla 文档](https://docs.argilla.io/en/latest/practical_guides/create_update_dataset/create_dataset.html) 和我们的 [端到端教程](https://docs.argilla.io/en/latest/tutorials_and_integrations/tutorials/tutorials.html)。\n", + "\n", + ">你也可以直接使用 [默认模板](https://docs.argilla.io/en/latest/practical_guides/create_update_dataset/create_dataset.html#task-templates) 来创建。" + ] + }, + { + "attachments": {}, + "cell_type": "markdown", + "metadata": {}, + "source": [ + "在这种情况下，我们将配置一个自定义数据集，其中包含两个不同的问题，以便我们能够同时处理两个文本分类任务。我们将加载该数据集的原始标签，以对请求中提到的主题进行多标签分类，并且我们还将设置一个问题，以将请求的情感分类为“积极”、“中性”或“消极”。" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "KKu2QplpFDgw" + }, + "outputs": [], + "source": [ + "dataset = rg.FeedbackDataset(\n", + " fields = [rg.TextField(name=\"text\")],\n", + " questions = [\n", + " rg.MultiLabelQuestion(\n", + " name=\"topics\",\n", + " title=\"Select the topic(s) of the request\",\n", + " labels=data.info.features['label'].names, #these are the original labels present in the dataset\n", + " visible_labels=10\n", + " ),\n", + " rg.LabelQuestion(\n", + " name=\"sentiment\",\n", + " title=\"What is the sentiment of the message?\",\n", + " labels=[\"positive\", \"neutral\", \"negative\"]\n", + " )\n", + " ]\n", + ")" + ] + }, + { + "attachments": {}, + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 训练模型" + ] + }, + { + "attachments": {}, + "cell_type": "markdown", + "metadata": {}, + "source": [ + "现在我们将使用我们加载的数据以及为数据集配置的标签和问题来训练数据集中的每个问题的零样本文本分类模型。如前面所述，我们将使用 [SetFit](https://github.com/huggingface/setfit) 框架对两个分类器中的 Sentence Transformers 进行少样本微调。此外，我们将使用的模型是 [all-MiniLM-L6-v2](https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2)，这是一个在 10 亿句子对数据集上使用对比目标进行微调的句子嵌入模型。" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "def train_model(question_name, template, multi_label=False):\n", + " # build a training dataset that uses the labels of a specific question in our Argilla dataset\n", + " train_dataset = get_templated_dataset(\n", + " candidate_labels=dataset.question_by_name(question_name).labels,\n", + " sample_size=8,\n", + " template=template,\n", + " multi_label=multi_label\n", + " )\n", + "\n", + " # train a model using the training dataset we just built\n", + " if multi_label:\n", + " model = SetFitModel.from_pretrained(\n", + " \"all-MiniLM-L6-v2\",\n", + " multi_target_strategy=\"one-vs-rest\"\n", + " )\n", + " else:\n", + " model = SetFitModel.from_pretrained(\n", + " \"all-MiniLM-L6-v2\"\n", + " )\n", + "\n", + " trainer = SetFitTrainer(\n", + " model=model,\n", + " train_dataset=train_dataset\n", + " )\n", + " trainer.train()\n", + " return model" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 276, + "referenced_widgets": [ + "503d373bd18b4b79a1f694916734d903", + "6e9e5e1ac58945d0926a85c1fd29ab17", + "cc9ccdfefca941e1813258a19afe64ed", + "c2238acd18b844c0bb517d670b76ca5c", + "90eec4e8ae8b42268548588db2fcbf49", + "501d213a24064f998d4d3c45255d02b7", + "3d282336f5c3425386a417866f367007", + "7b96b0a21eba4ad5a4c12534940b3591", + "571fd48c2da8432e8a74e7b318eb6042", + "1d58b40ad6a54c25bd451eda4e7d8069", + "5e0377b4b48c441a8d747ea904c3207b", + "38bfdddef0444c0baf9d29248689f846", + "3f5aed26eeef4182b360085d83ae795d", + "255d62fb39454098ab3701753d8d67d6", + "25f9bca647f44645b85a644f03807095", + "ae7fc579502e46f7861e402580586b28", + "6143886f7acc4591ae5f79ce6f67af4a", + "486c1a817552432c8fb20e59d0a3f079", + "77bd2b1f5e57441ab729c6e517279834", + "bc0c58d9d798437fb1d40277d8777777", + "fa5df54e161e40dbbb21ed96c879444e", + "16993356757e4ee5b7f8042d58c96e17", + "d11aa6a0c8c54481b6cc2c80d1fa0ba1", + "a9ce0af78a2241e697a22229db7840ab", + "ae6ffc6572b54c059196983da4ff2d79", + "980f36d72cfa403aad67e871aecba890", + "5692de58835a466695fcc8f0d5976b74", + "7a12fbf5400a468fbdce4b2b2008eefc", + "04150cf7e9a74a04aafa94d394553630", + "9a7c8861a37b41eba191059546f5dd5d", + "217760080e494d2d9b0582910d121a28", + "f5e35991e6d849eca73282c9c359000a", + "5a06b8d12b494daeb0624f2e39e06e67" + ] + }, + "id": "U9TVO355a2np", + "outputId": "7d6b6b60-6f49-4308-a2e6-ac24bf99bf72" + }, + "outputs": [], + "source": [ + "topic_model = train_model(\n", + " question_name=\"topics\", \n", + " template=\"The customer request is about {}\", \n", + " multi_label=True\n", + ")" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 276, + "referenced_widgets": [ + "c21e90a6dda643d8bd82abf4e346d45c", + "170a2ee20ab64a9b86db65549a5d4063", + "fd7c2acc4b1945feabe6715dd270cb72", + "2f271b0778974646aaff691227336e91", + "ef245777ac3d435e8715fc55b1d4824c", + "0d7acd8e1a394336aa146e2a442f672c", + "3e6c2b50b3084d23b575585c288f087e", + "ff7f98b368c448ea81e4c79fded0be5a", + "1ff157a9c8974b07ae97cb115c8d0188", + "16d42bc00dfe4467a1da86b1d2391d0d", + "0447a98b5dfe42c899273b9c37bdadad", + "411de4b297fe4a09acb70951c9f36b82", + "c2eac9934f5b407c8e424ee2da9eea58", + "36b99521f8274a639abb90eb0040d6c0", + "3fd94ef662db4fff9dde61455b41faf1", + "d6283b2cf69d45f694633ae1544d47a8", + "7ca015b6798947d58d275de6181fe053", + "750011ef09534e55bab5180974bcf5d4", + "70a57ad580f847d3bd3123cfe1539305", + "0c010df989eb497c810a6f960c6ea41b", + "186f82d150994ac7914d0646fb5ff425", + "379907416f504f05906454e482da2eaf", + "783115bacdbf4c0bb09c0b1fc7976d28", + "242f97eb0f0d4ab1830c62686127b717", + "bfecbc09a4f84f3db51903d5048ff825", + "db7cf4427ad746cd86df88f7a1016bc9", + "668593b82ae54d3cbaf1a19c0307c545", + "5057f8b8144d41ff9d8b82b8602570fc", + "369bc409052a48f7ac2182715406abef", + "5cc0f7cc30ae4aa4b13966a773e4c824", + "28c40914eac34bcba0c9eb4dac6b0032", + "3e622eeea5df47d6a21e015f3e742fa8", + "621bb7d632814cb0839755ca56098d7a" + ] + }, + "id": "kkTufA4NbEh_", + "outputId": "41c579c8-5394-4c24-fd3c-d6ab77c2a0a7" + }, + "outputs": [], + "source": [ + "sentiment_model = train_model(\n", + " question_name=\"sentiment\", \n", + " template=\"This message is {}\", \n", + " multi_label=False\n", + ")" + ] + }, + { + "attachments": {}, + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 预测" + ] + }, + { + "attachments": {}, + "cell_type": "markdown", + "metadata": {}, + "source": [ + "一旦训练步骤结束，我们就可以通过我们的数据进行预测了。" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "def get_predictions(texts, model, question_name):\n", + " probas = model.predict_proba(texts, as_numpy=True)\n", + " labels = dataset.question_by_name(question_name).labels\n", + " for pred in probas:\n", + " yield [{\"label\": label, \"score\": score} for label, score in zip(labels, pred)]" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "Hz5LeVDMYyx6" + }, + "outputs": [], + "source": [ + "data = data.map(\n", + " lambda batch: {\n", + " \"topics\": list(get_predictions(batch[\"text\"], topic_model, \"topics\")),\n", + " \"sentiment\": list(get_predictions(batch[\"text\"], sentiment_model, \"sentiment\")),\n", + " },\n", + " batched=True,\n", + ")" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 206 + }, + "id": "bgGkKO-7ZGCR", + "outputId": "17bb27eb-b78a-4a2c-d838-60fcaa176502" + }, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + "\n", + "
\n", + "
\n", + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
textlabeltopicssentiment
0How do I locate my card?11[{'label': 'activate_my_card', 'score': 0.0127...[{'label': 'positive', 'score': 0.348371499634...
1I still have not received my new card, I order...11[{'label': 'activate_my_card', 'score': 0.0133...[{'label': 'positive', 'score': 0.361745933281...
2I ordered a card but it has not arrived. Help ...11[{'label': 'activate_my_card', 'score': 0.0094...[{'label': 'positive', 'score': 0.346292075496...
3Is there a way to know when my card will arrive?11[{'label': 'activate_my_card', 'score': 0.0150...[{'label': 'positive', 'score': 0.426133716131...
4My card has not arrived yet.11[{'label': 'activate_my_card', 'score': 0.0175...[{'label': 'positive', 'score': 0.389241385165...
\n", + "
\n", + " \n", + "\n", + "\n", + "\n", + "
\n", + " \n", + "
\n", + "\n", + "\n", + "\n", + " \n", + "\n", + " \n", + " \n", + "\n", + " \n", + "
\n", + "
\n" + ], + "text/plain": [ + " text label \\\n", + "0 How do I locate my card? 11 \n", + "1 I still have not received my new card, I order... 11 \n", + "2 I ordered a card but it has not arrived. Help ... 11 \n", + "3 Is there a way to know when my card will arrive? 11 \n", + "4 My card has not arrived yet. 11 \n", + "\n", + " topics \\\n", + "0 [{'label': 'activate_my_card', 'score': 0.0127... \n", + "1 [{'label': 'activate_my_card', 'score': 0.0133... \n", + "2 [{'label': 'activate_my_card', 'score': 0.0094... \n", + "3 [{'label': 'activate_my_card', 'score': 0.0150... \n", + "4 [{'label': 'activate_my_card', 'score': 0.0175... \n", + "\n", + " sentiment \n", + "0 [{'label': 'positive', 'score': 0.348371499634... \n", + "1 [{'label': 'positive', 'score': 0.361745933281... \n", + "2 [{'label': 'positive', 'score': 0.346292075496... \n", + "3 [{'label': 'positive', 'score': 0.426133716131... \n", + "4 [{'label': 'positive', 'score': 0.389241385165... " + ] + }, + "execution_count": 51, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "data.to_pandas().head()" + ] + }, + { + "attachments": {}, + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 构建记录并推送" + ] + }, + { + "attachments": {}, + "cell_type": "markdown", + "metadata": {}, + "source": [ + "有了我们生成的数据和预测，现在我们可以构建记录（将由标注团队标注的每个数据项），其中包括我们模型的建议。对于 `LabelQuestion`，我们将使用概率得分最高的标签，而对于 `MultiLabelQuestion`，我们将包含所有得分高于一定阈值的标签。在这种情况下，我们决定使用 `2/len(labels)` 作为阈值，但你可以根据你的数据实验，并决定采用更严格或更宽松的阈值。\n", + "\n", + "> 注意，更宽松的阈值（接近或等于 `1/len(labels)`）将建议更多的标签，而严格的阈值（在 2 到 3 之间）将选择更少的标签（或没有标签）。" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "def add_suggestions(record):\n", + " suggestions = []\n", + " \n", + " # get label with max score for sentiment question\n", + " sentiment = max(record['sentiment'], key=lambda x: x['score'])['label']\n", + " suggestions.append({\"question_name\": \"sentiment\", \"value\": sentiment})\n", + "\n", + " # get all labels above a threshold for topics questions\n", + " threshold = 2 / len(dataset.question_by_name(\"topics\").labels)\n", + " topics = [label['label'] for label in record['topics'] if label['score'] >= threshold]\n", + " # apply the suggestion only if at least one label was over the threshold\n", + " if topics:\n", + " suggestions.append({\"question_name\": \"topics\", \"value\": topics})\n", + " return suggestions" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "S0I4lkIWqmin" + }, + "outputs": [], + "source": [ + "records = [\n", + " rg.FeedbackRecord(fields={\"text\": record['text']}, suggestions=add_suggestions(record))\n", + " for record in data\n", + "]" + ] + }, + { + "attachments": {}, + "cell_type": "markdown", + "metadata": {}, + "source": [ + "一旦我们对结果满意，我们可以将记录添加到我们上面配置的数据集中。最后，为了可视化并开始标注，你需要将其推送到 Argilla。这意味着将你的数据集添加到运行的 Argilla 服务器上，并使其对标注者可用。" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "CvVgNhQSibLM" + }, + "outputs": [], + "source": [ + "dataset.add_records(records)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "l2pdzhuspBA_", + "outputId": "a296c87f-35a3-4476-8ed1-56e1f053a953" + }, + "outputs": [], + "source": [ + "dataset.push_to_argilla(\"setfit_tutorial\", workspace=\"admin\")" + ] + }, + { + "attachments": {}, + "cell_type": "markdown", + "metadata": {}, + "source": [ + "这是从我们的模型看建议的 UI 样式\n", + "\n", + "![Feedback Task dataset with suggestions made using SetFit](https://huggingface.co/datasets/huggingface/cookbook-images/resolve/main/snapshot_setfit_suggestions.png)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "这部分可选，你还可以将你的 `FeedbackDataset` 保存并加载到 Hugging Face Hub。请参阅[文档](https://docs.argilla.io/en/latest/practical_guides/export_dataset.html)以获取更多关于如何执行此操作的信息。\n", + "\n", + "\n", + "```python\n", + "# Push to HuggingFace Hub\n", + "dataset.push_to_huggingface(\"argilla/my-dataset\")\n", + "\n", + "# Load a public dataset\n", + "dataset = rg.FeedbackDataset.from_huggingface(\"argilla/my-dataset\")\n", + "```" + ] + }, + { + "attachments": {}, + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 总结\n", + "\n", + "在本教程中，我们介绍了如何使用 SetFit 库的零样本方法向 Feedback Task 数据集添加建议。这将通过减少标注团队必须做出的决定和编辑数量来提高标注过程的效率。\n", + "\n", + "要了解更多关于 SetFit 的信息，请查看以下链接：\n", + "\n", + "- [更多 Argilla 教程](https://docs.argilla.io/en/latest/tutorials_and_integrations/tutorials/tutorials.html)\n", + "- [SetFit 在 GitHub 的仓库](https://github.com/huggingface/setfit)\n", + "- [SetFit 文档](https://huggingface.co/docs/setfit/index)\n" + ] + } + ], + "metadata": { + "accelerator": "GPU", + "colab": { + "gpuType": "T4", + "provenance": [] + }, + "kernelspec": { + "display_name": "argilla", + "language": "python", + "name": "python3" + }, + "language_info": { + "name": "python", + "version": "3.8.15" + }, + "vscode": { + "interpreter": { + "hash": "2d98cb9bf90a932b5bf8e86e91214497eb0e38eb318595fbd6fbd5460fe92036" + } + }, + "widgets": { + "application/vnd.jupyter.widget-state+json": { + "04150cf7e9a74a04aafa94d394553630": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "DescriptionStyleModel", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "0447a98b5dfe42c899273b9c37bdadad": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "DescriptionStyleModel", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "0c010df989eb497c810a6f960c6ea41b": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "ProgressStyleModel", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "ProgressStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "bar_color": null, + "description_width": "" + } + }, + "0d7acd8e1a394336aa146e2a442f672c": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "16993356757e4ee5b7f8042d58c96e17": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "DescriptionStyleModel", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "16d42bc00dfe4467a1da86b1d2391d0d": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "170a2ee20ab64a9b86db65549a5d4063": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "HTMLModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_0d7acd8e1a394336aa146e2a442f672c", + "placeholder": "​", + "style": "IPY_MODEL_3e6c2b50b3084d23b575585c288f087e", + "value": "Generating Training Pairs: 100%" + } + }, + "186f82d150994ac7914d0646fb5ff425": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "1d58b40ad6a54c25bd451eda4e7d8069": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "1ff157a9c8974b07ae97cb115c8d0188": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "ProgressStyleModel", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "ProgressStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "bar_color": null, + "description_width": "" + } + }, + "217760080e494d2d9b0582910d121a28": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "ProgressStyleModel", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "ProgressStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "bar_color": null, + "description_width": "" + } + }, + "242f97eb0f0d4ab1830c62686127b717": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "HTMLModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_5057f8b8144d41ff9d8b82b8602570fc", + "placeholder": "​", + "style": "IPY_MODEL_369bc409052a48f7ac2182715406abef", + "value": "Iteration: 100%" + } + }, + "255d62fb39454098ab3701753d8d67d6": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "FloatProgressModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "FloatProgressModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "ProgressView", + "bar_style": "success", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_77bd2b1f5e57441ab729c6e517279834", + "max": 1, + "min": 0, + "orientation": "horizontal", + "style": "IPY_MODEL_bc0c58d9d798437fb1d40277d8777777", + "value": 1 + } + }, + "25f9bca647f44645b85a644f03807095": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "HTMLModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_fa5df54e161e40dbbb21ed96c879444e", + "placeholder": "​", + "style": "IPY_MODEL_16993356757e4ee5b7f8042d58c96e17", + "value": " 1/1 [01:28<00:00, 88.63s/it]" + } + }, + "28c40914eac34bcba0c9eb4dac6b0032": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "ProgressStyleModel", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "ProgressStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "bar_color": null, + "description_width": "" + } + }, + "2f271b0778974646aaff691227336e91": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "HTMLModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_16d42bc00dfe4467a1da86b1d2391d0d", + "placeholder": "​", + "style": "IPY_MODEL_0447a98b5dfe42c899273b9c37bdadad", + "value": " 20/20 [00:00<00:00, 391.01it/s]" + } + }, + "369bc409052a48f7ac2182715406abef": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "DescriptionStyleModel", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "36b99521f8274a639abb90eb0040d6c0": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "FloatProgressModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "FloatProgressModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "ProgressView", + "bar_style": "success", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_70a57ad580f847d3bd3123cfe1539305", + "max": 1, + "min": 0, + "orientation": "horizontal", + "style": "IPY_MODEL_0c010df989eb497c810a6f960c6ea41b", + "value": 1 + } + }, + "379907416f504f05906454e482da2eaf": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "DescriptionStyleModel", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "38bfdddef0444c0baf9d29248689f846": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "HBoxModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HBoxModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HBoxView", + "box_style": "", + "children": [ + "IPY_MODEL_3f5aed26eeef4182b360085d83ae795d", + "IPY_MODEL_255d62fb39454098ab3701753d8d67d6", + "IPY_MODEL_25f9bca647f44645b85a644f03807095" + ], + "layout": "IPY_MODEL_ae7fc579502e46f7861e402580586b28" + } + }, + "3d282336f5c3425386a417866f367007": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "DescriptionStyleModel", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "3e622eeea5df47d6a21e015f3e742fa8": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "3e6c2b50b3084d23b575585c288f087e": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "DescriptionStyleModel", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "3f5aed26eeef4182b360085d83ae795d": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "HTMLModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_6143886f7acc4591ae5f79ce6f67af4a", + "placeholder": "​", + "style": "IPY_MODEL_486c1a817552432c8fb20e59d0a3f079", + "value": "Epoch: 100%" + } + }, + "3fd94ef662db4fff9dde61455b41faf1": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "HTMLModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_186f82d150994ac7914d0646fb5ff425", + "placeholder": "​", + "style": "IPY_MODEL_379907416f504f05906454e482da2eaf", + "value": " 1/1 [00:02<00:00, 2.63s/it]" + } + }, + "411de4b297fe4a09acb70951c9f36b82": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "HBoxModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HBoxModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HBoxView", + "box_style": "", + "children": [ + "IPY_MODEL_c2eac9934f5b407c8e424ee2da9eea58", + "IPY_MODEL_36b99521f8274a639abb90eb0040d6c0", + "IPY_MODEL_3fd94ef662db4fff9dde61455b41faf1" + ], + "layout": "IPY_MODEL_d6283b2cf69d45f694633ae1544d47a8" + } + }, + "486c1a817552432c8fb20e59d0a3f079": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "DescriptionStyleModel", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "501d213a24064f998d4d3c45255d02b7": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "503d373bd18b4b79a1f694916734d903": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "HBoxModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HBoxModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HBoxView", + "box_style": "", + "children": [ + "IPY_MODEL_6e9e5e1ac58945d0926a85c1fd29ab17", + "IPY_MODEL_cc9ccdfefca941e1813258a19afe64ed", + "IPY_MODEL_c2238acd18b844c0bb517d670b76ca5c" + ], + "layout": "IPY_MODEL_90eec4e8ae8b42268548588db2fcbf49" + } + }, + "5057f8b8144d41ff9d8b82b8602570fc": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "5692de58835a466695fcc8f0d5976b74": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "571fd48c2da8432e8a74e7b318eb6042": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "ProgressStyleModel", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "ProgressStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "bar_color": null, + "description_width": "" + } + }, + "5a06b8d12b494daeb0624f2e39e06e67": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "DescriptionStyleModel", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "5cc0f7cc30ae4aa4b13966a773e4c824": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "5e0377b4b48c441a8d747ea904c3207b": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "DescriptionStyleModel", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "6143886f7acc4591ae5f79ce6f67af4a": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "621bb7d632814cb0839755ca56098d7a": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "DescriptionStyleModel", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "668593b82ae54d3cbaf1a19c0307c545": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "6e9e5e1ac58945d0926a85c1fd29ab17": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "HTMLModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_501d213a24064f998d4d3c45255d02b7", + "placeholder": "​", + "style": "IPY_MODEL_3d282336f5c3425386a417866f367007", + "value": "Generating Training Pairs: 100%" + } + }, + "70a57ad580f847d3bd3123cfe1539305": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "750011ef09534e55bab5180974bcf5d4": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "DescriptionStyleModel", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "77bd2b1f5e57441ab729c6e517279834": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "783115bacdbf4c0bb09c0b1fc7976d28": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "HBoxModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HBoxModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HBoxView", + "box_style": "", + "children": [ + "IPY_MODEL_242f97eb0f0d4ab1830c62686127b717", + "IPY_MODEL_bfecbc09a4f84f3db51903d5048ff825", + "IPY_MODEL_db7cf4427ad746cd86df88f7a1016bc9" + ], + "layout": "IPY_MODEL_668593b82ae54d3cbaf1a19c0307c545" + } + }, + "7a12fbf5400a468fbdce4b2b2008eefc": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "7b96b0a21eba4ad5a4c12534940b3591": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "7ca015b6798947d58d275de6181fe053": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "90eec4e8ae8b42268548588db2fcbf49": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "980f36d72cfa403aad67e871aecba890": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "HTMLModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_f5e35991e6d849eca73282c9c359000a", + "placeholder": "​", + "style": "IPY_MODEL_5a06b8d12b494daeb0624f2e39e06e67", + "value": " 1540/1540 [01:28<00:00, 21.45it/s]" + } + }, + "9a7c8861a37b41eba191059546f5dd5d": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "a9ce0af78a2241e697a22229db7840ab": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "HTMLModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_7a12fbf5400a468fbdce4b2b2008eefc", + "placeholder": "​", + "style": "IPY_MODEL_04150cf7e9a74a04aafa94d394553630", + "value": "Iteration: 100%" + } + }, + "ae6ffc6572b54c059196983da4ff2d79": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "FloatProgressModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "FloatProgressModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "ProgressView", + "bar_style": "success", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_9a7c8861a37b41eba191059546f5dd5d", + "max": 1540, + "min": 0, + "orientation": "horizontal", + "style": "IPY_MODEL_217760080e494d2d9b0582910d121a28", + "value": 1540 + } + }, + "ae7fc579502e46f7861e402580586b28": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "bc0c58d9d798437fb1d40277d8777777": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "ProgressStyleModel", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "ProgressStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "bar_color": null, + "description_width": "" + } + }, + "bfecbc09a4f84f3db51903d5048ff825": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "FloatProgressModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "FloatProgressModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "ProgressView", + "bar_style": "success", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_5cc0f7cc30ae4aa4b13966a773e4c824", + "max": 60, + "min": 0, + "orientation": "horizontal", + "style": "IPY_MODEL_28c40914eac34bcba0c9eb4dac6b0032", + "value": 60 + } + }, + "c21e90a6dda643d8bd82abf4e346d45c": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "HBoxModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HBoxModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HBoxView", + "box_style": "", + "children": [ + "IPY_MODEL_170a2ee20ab64a9b86db65549a5d4063", + "IPY_MODEL_fd7c2acc4b1945feabe6715dd270cb72", + "IPY_MODEL_2f271b0778974646aaff691227336e91" + ], + "layout": "IPY_MODEL_ef245777ac3d435e8715fc55b1d4824c" + } + }, + "c2238acd18b844c0bb517d670b76ca5c": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "HTMLModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_1d58b40ad6a54c25bd451eda4e7d8069", + "placeholder": "​", + "style": "IPY_MODEL_5e0377b4b48c441a8d747ea904c3207b", + "value": " 20/20 [00:01<00:00, 10.96it/s]" + } + }, + "c2eac9934f5b407c8e424ee2da9eea58": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "HTMLModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_7ca015b6798947d58d275de6181fe053", + "placeholder": "​", + "style": "IPY_MODEL_750011ef09534e55bab5180974bcf5d4", + "value": "Epoch: 100%" + } + }, + "cc9ccdfefca941e1813258a19afe64ed": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "FloatProgressModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "FloatProgressModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "ProgressView", + "bar_style": "success", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_7b96b0a21eba4ad5a4c12534940b3591", + "max": 20, + "min": 0, + "orientation": "horizontal", + "style": "IPY_MODEL_571fd48c2da8432e8a74e7b318eb6042", + "value": 20 + } + }, + "d11aa6a0c8c54481b6cc2c80d1fa0ba1": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "HBoxModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HBoxModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HBoxView", + "box_style": "", + "children": [ + "IPY_MODEL_a9ce0af78a2241e697a22229db7840ab", + "IPY_MODEL_ae6ffc6572b54c059196983da4ff2d79", + "IPY_MODEL_980f36d72cfa403aad67e871aecba890" + ], + "layout": "IPY_MODEL_5692de58835a466695fcc8f0d5976b74" + } + }, + "d6283b2cf69d45f694633ae1544d47a8": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "db7cf4427ad746cd86df88f7a1016bc9": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "HTMLModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_3e622eeea5df47d6a21e015f3e742fa8", + "placeholder": "​", + "style": "IPY_MODEL_621bb7d632814cb0839755ca56098d7a", + "value": " 60/60 [00:02<00:00, 23.09it/s]" + } + }, + "ef245777ac3d435e8715fc55b1d4824c": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "f5e35991e6d849eca73282c9c359000a": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "fa5df54e161e40dbbb21ed96c879444e": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "fd7c2acc4b1945feabe6715dd270cb72": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "FloatProgressModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "FloatProgressModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "ProgressView", + "bar_style": "success", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_ff7f98b368c448ea81e4c79fded0be5a", + "max": 20, + "min": 0, + "orientation": "horizontal", + "style": "IPY_MODEL_1ff157a9c8974b07ae97cb115c8d0188", + "value": 20 + } + }, + "ff7f98b368c448ea81e4c79fded0be5a": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + } + } + } + }, + "nbformat": 4, + "nbformat_minor": 0 +} From 00a66ee73e4eacdb6f28cd81e9d04c4e519ffb62 Mon Sep 17 00:00:00 2001 From: innovation64 Date: Mon, 1 Apr 2024 18:12:45 +0800 Subject: [PATCH 11/31] docs: update pipeline_notus_instructions_preferences_legal in chinese version --- ...notus_instructions_preferences_legal.ipynb | 808 ++++++++++++++++++ 1 file changed, 808 insertions(+) create mode 100644 notebooks/zh-CN/pipeline_notus_instructions_preferences_legal.ipynb diff --git a/notebooks/zh-CN/pipeline_notus_instructions_preferences_legal.ipynb b/notebooks/zh-CN/pipeline_notus_instructions_preferences_legal.ipynb new file mode 100644 index 00000000..e20d8bc9 --- /dev/null +++ b/notebooks/zh-CN/pipeline_notus_instructions_preferences_legal.ipynb @@ -0,0 +1,808 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# ⚖️ 创建一个合法偏好数据集\n", + "\n", + "_作者: [David Berenstein](https://huggingface.co/davidberenstein1957) 和 [Sara Han Díaz](https://huggingface.co/sdiazlor)_\n", + "\n", + "在本教程中，你将学习如何在 HF 推理端点上使用 Notus 模型，基于欧洲人工智能法案中的 RAG 指令创建一个合法的偏好数据集。这是一个完整的端到端示例，展示了如何使用 distilabel 来利用大型语言模型（LLMs）！\n", + "\n", + "[distilabel](https://github.com/argilla-io/distilabel)是一个人工智能反馈（AIF）框架，它可以使用 LLMs 生成和标注数据集，并且可以用于许多不同的用例。它以稳健性、效率和可扩展性为目标实现，允许任何人构建可用于多种场景的合成数据集。\n", + "\n", + "为了生成指令数据集，我们将使用与 distilabel 集成的[ HF 推理端点](https://huggingface.co/docs/inference-endpoints/en/index)。这些推理端点由 Hugging Face 提供，允许在专用和自动扩展的基础设施上轻松部署和运行 transformers、diffusers 或 Hub 中的任何可用模型。你可以在[这里](https://huggingface.co/docs/inference-endpoints/guides/create_endpoint)找到更多关于如何创建你的第一个端点的信息。\n", + "\n", + "我们将为此微调的 LLM 模型是 [Notus 7B](https://argilla.io/blog/notus7b/)，这是 Zephyr 7B 的一个微调版本，它使用直接偏好优化（DPO）和 AIF 技术，在多个基准测试中超越了其基础模型，并且完全开源。\n", + "\n", + "本教程包括以下步骤：\n", + "\n", + "- 为 `distilabel` 流水线定义一个自定义生成任务。\n", + "- 使用 Haystack 为欧盟人工智能法案创建一个 RAG 流水线。\n", + "- 使用 `SelfInstructTask` 生成一个指令数据集。\n", + "- 使用 `UltraFeedback` 文本质量任务生成一个偏好数据集。\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 简介\n", + "让我们从安装运行 **distilabel** 以及教程中使用的其他包所需的依赖项开始；尤其是 **Haystack**。为了更好地可视化和管理结果，也请安装 **Argilla**。" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "!pip install -q -U distilabel \"farm-haystack[preprocessing]\"\n", + "!pip install -q -U \"distilabel[hf-inference-endpoints, argilla]\"" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### 导入依赖项\n", + "\n", + "本教程的主要依赖项是 distilabel，用于创建合成数据集，以及 Argilla，用于可视化和管理这些数据集，同时也用于微调我们的模型。包 [Haystack](https://haystack.deepset.ai/) 用于从我们想要创建数据集的原始 PDF 文档中创建批次。\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "import os\n", + "from typing import Dict\n", + "\n", + "from distilabel.llm import InferenceEndpointsLLM\n", + "from distilabel.pipeline import Pipeline, pipeline\n", + "from distilabel.tasks import TextGenerationTask, SelfInstructTask, Prompt\n", + "\n", + "from datasets import Dataset\n", + "from haystack.nodes import PDFToTextConverter, PreProcessor" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### 环境变量\n", + "\n", + "我们需要提供我们的 HuggingFace 访问 token，可以从[设置](https://huggingface.co/settings/tokens)中获取。此外，为了通过 UltraFeedback 文本质量任务生成偏好数据集，我们还需要 OpenAI 的 api 密钥。你可以在[这里](https://platform.openai.com/api-keys)找到它。请注意，根据使用的模型不同，将收取不同的费用，因此请确保你检查了 OpenAI 的[定价页面](https://openai.com/pricing)。\n", + "\n", + "为了稍后实例化一个 `InferenceEndpointsLLM` 对象，我们还需要作为参数传递 HF 推理端点名称和 HF 命名空间。通过环境变量也是一种非常方便的方式。" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": {}, + "outputs": [], + "source": [ + "os.environ[\"HF_TOKEN\"] = \"\"\n", + "os.environ[\"HF_INFERENCE_ENDPOINT_NAME\"] = \"aws-notus-7b-v1-3184\"\n", + "os.environ[\"HF_NAMESPACE\"] = \"argilla\"\n", + "os.environ[\"OPENAI_API_KEY\"] = \"\"" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 使用 Notus 设置推理端点\n", + "推理端点是 Hugging Face 管理的一种解决方案，可以轻松部署任何类似 Transformer 的模型。它们是基于 Hugging Face Hub 上的模型构建的。推理端点对于在 LLMs 上进行推理非常方便，无需尝试在本地运行模型。在本教程中，我们将使用推理端点作为 `distilabel` 工作流程的一部分，使用我们的 Notus 模型生成文本。所选端点运行着一个[ Notus 7B 实例](https://ui.endpoints.huggingface.co/argilla/endpoints/aws-notus-7b-v1-4052)。\n", + "\n", + "### 为 distilabel 流水线定义一个自定义生成任务\n", + "\n", + "为了开始本教程，让我们看看如何为我们的 Notus 模型设置一个端点。这不是我们稍后将看到的端到端示例的一部分，但是如何连接到 Hugging Face 端点以及测试 `distilabel` 流水线的一个示例。\n", + "\n", + "让我们快速了解一下如何使用推理端点。我们已经准备了一个简单的 `TextGenerationTask` 来向模型提问，这种方式与我们使用聊天机器人与 LLMs 交流非常相似。首先，我们定义一个用于问答任务的类，其中包含的函数向 `distilabel` 展示了模型应该如何生成提示、解析输入和输出等。" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": {}, + "outputs": [], + "source": [ + "class QuestionAnsweringTask(TextGenerationTask):\n", + " def generate_prompt(self, question: str) -> str:\n", + " return Prompt(\n", + " system_prompt=self.system_prompt,\n", + " formatted_prompt=question,\n", + " ).format_as(\n", + " \"llama2\"\n", + " ) # type: ignore\n", + "\n", + " def parse_output(self, output: str) -> Dict[str, str]:\n", + " return {\"answer\": output.strip()}\n", + "\n", + " @property\n", + " def input_args_names(self) -> list[str]:\n", + " return [\"question\"]\n", + "\n", + " @property\n", + " def output_args_names(self) -> list[str]:\n", + " return [\"answer\"]" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "`llm` 是 `InferenceEndpointsLLM` 类的一个对象，通过使用它，我们可以开始使用 `llm.generate()` 方法来生成问题的答案。" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "llm = InferenceEndpointsLLM(\n", + " endpoint_name_or_model_id=os.getenv(\"HF_INFERENCE_ENDPOINT_NAME\"), # type: ignore\n", + " endpoint_namespace=os.getenv(\"HF_NAMESPACE\"), # type: ignore\n", + " token=os.getenv(\"HF_TOKEN\") or None,\n", + " task=QuestionAnsweringTask(),\n", + ")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "使用定义了端点和任务信息的 `InferenceEndpointsLLM` 对象，我们可以开始生成文本。让我们问这个 LLM，例如，丹麦第二大城市人口最多的是哪个城市。答案应该是 Aarhus。\n" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "'The second most populated city in Denmark is Aarhus, with a population of around 340,000 people. It is located on the east coast of Jutland, and is known for its vibrant cultural scene, beautiful beaches, and historic landmarks. Aarhus is also home to Aarhus University, one of the largest universities in Scandinavia.'" + ] + }, + "execution_count": 6, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "generation = llm.generate(\n", + " [{\"question\": \"What's the second most populated city in Denmark?\"}]\n", + ")\n", + "generation[0][0][\"parsed_output\"][\"answer\"]" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "端点工作正常！我们已经成功地为 `distilabel` 流水线设置了一个自定义生成任务。\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 使用 Haystack 为欧洲人工智能法案创建 RAG 流水线\n", + "\n", + "在这个端到端的示例中，我们希望创建一个能够回答问题并填写关于欧盟推广的新人工智能法案信息的专家模型，这是关于人工智能的第一项法规。作为其数字战略的一部分，欧盟希望规范人工智能，以确保更好地发展和使用这项创新技术。这个法案是人工智能的监管框架，不同的风险级别意味着更多的或更少的监管。它们是世界上关于人工智能的第一套规则。\n", + "\n", + "我们想要创建的这个 RAG 的流水线会下载 PDF 文件，将其转换为纯文本并进行预处理，创建我们可以提供给 `distilabel` 的批次，以便开始从中创建指令。让我们看看流水线的第一部分并获取输入数据。需要注意的是，这个流水线的 RAG 部分并不是基于活跃的查询或语义属性的流水线，而是一种更直接的方法，我们下载PDF并预处理其内容。\n", + "\n", + "### 下载人工智能法案 PDF\n", + "\n", + "首先，我们需要下载 PDF 文档本身。如果它不在工作目录中，我们将把它放在那里。" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "metadata": {}, + "outputs": [], + "source": [ + "%%bash\n", + "\n", + "if [ ! -f \"The-AI-Act.pdf\" ]; then\n", + " wget -q https://artificialintelligenceact.eu/wp-content/uploads/2021/08/The-AI-Act.pdf\n", + "fi" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "一旦我们将文件放入工作目录，我们可以使用 Haystack 的转换器和流水线功能来提取文本数据，清洗数据并将其分成不同的批次。之后，这些批次将被用来开始创建合成指令。\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# The converter turns the PDF into text we can process easily\n", + "converter = PDFToTextConverter(remove_numeric_tables=True, valid_languages=[\"en\"])\n", + "\n", + "# Preprocessing pipelines can have several steps.\n", + "# Ours clean empty lines, header, footers and whitespaces\n", + "# and split the text into 150-char long batches, respecting\n", + "# where the sentences naturally end and begin.\n", + "preprocessor = PreProcessor(\n", + " clean_empty_lines=True,\n", + " clean_whitespace=True,\n", + " clean_header_footer=True,\n", + " split_by=\"word\",\n", + " split_length=150,\n", + " split_respect_sentence_boundary=True,\n", + ")\n", + "\n", + "doc = converter.convert(file_path=\"The-AI-Act.pdf\", meta=None)[0]\n", + "docs = preprocessor.process([doc])\n", + "print(f\"Documents: 1\\nBatches: {len(docs)}\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "让我们快速查看一下我们刚刚生成的批次。\n" + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "'EN EN\\nEUROPEAN\\nCOMMISSION\\nProposal for a\\nREGULATION OF THE EUROPEAN PARLIAMENT AND OF THE COUNCIL\\nLAYING DOWN HARMONISED RULES ON ARTIFICIAL INTELLIGENCE\\n(ARTIFICIAL INTELLIGENCE ACT) AND AMENDING CERTAIN UNION\\nLEGISLATIVE ACTS\\x0cEN\\nEXPLANATORY MEMORANDUM\\n1. CONTEXT OF THE PROPOSAL\\n1.1. Reasons for and objectives of the proposal\\nThis explanatory memorandum accompanies the proposal for a Regulation laying down\\nharmonised rules on artificial intelligence (Artificial Intelligence Act). Artificial Int'" + ] + }, + "execution_count": 9, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "inputs = [doc.content for doc in docs]\n", + "inputs[0][0:500]" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "文件已经被正确地分批处理，从一个大文档变成了最多 150 个字符长的 355 个字符串。现在这个字符串列表可以作为输入，使用 `distilabel` 生成指令数据集。\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 使用 SelfInstructTask 生成指令\n", + "\n", + "由于我们的推理端点已经启动并运行，我们应该能够使用 distilabel 生成指令。通过我们的端点由 LLM 创建的这些指令，将形成一个指令数据集，其中的指令是由我们刚刚提取的数据创建的。\n", + "\n", + "为了示例的顺利进行，我们使用了上面生成的 50 个批次的一个子集，以减轻性能压力。\n" + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "Dataset({\n", + " features: ['input'],\n", + " num_rows: 50\n", + "})" + ] + }, + "execution_count": 10, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "instructions_dataset = Dataset.from_dict({\"input\": inputs[0:50]})\n", + "\n", + "instructions_dataset" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "使用 `SelfInstructTask` 类，我们可以为构建提示生成一个 Self-Instruct 规范，就像在 [Self-Instruct 论文](https://arxiv.org/abs/2212.10560)中所做的那样。`distilabel` 将从人工制作的输入开始，在这个案例中，就是我们从 AI 法案 PDF 创建的批次，然后基于这些输入生成指令。之后，可以使用 Argilla 来审查这些指令，以保留最好的那些。\n", + "\n", + "我们可以通过传递一个应用描述作为参数来告诉模型它应该做什么；我们希望这个模型能够回答我们关于 AI 法案的任何问题。\n" + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "metadata": {}, + "outputs": [], + "source": [ + "instructions_task = SelfInstructTask(\n", + " application_description=\"A assistant that can answer questions about the AI Act made by the European Union.\"\n", + ")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "现在，我们来定义一个生成器，传入 `SelfInstructTask` 对象，并创建一个 `Pipeline` 对象。\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "instructions_generator = InferenceEndpointsLLM(\n", + " endpoint_name_or_model_id=os.getenv(\"HF_INFERENCE_ENDPOINT_NAME\"), # type: ignore\n", + " endpoint_namespace=os.getenv(\"HF_NAMESPACE\"), # type: ignore\n", + " token=os.getenv(\"HF_TOKEN\") or None,\n", + " task=instructions_task,\n", + ")\n", + "\n", + "instructions_pipeline = Pipeline(generator=instructions_generator)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "我们的流水线已经准备好用来生成指令了。下面就开始吧！\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "generated_instructions = instructions_pipeline.generate(\n", + " dataset=instructions_dataset, num_generations=1, batch_size=8\n", + ")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "流水线已经成功生成了指令，基于输入的主题和行为。让我们收集所有这些指令，看看它们是什么样的。\n" + ] + }, + { + "cell_type": "code", + "execution_count": 14, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Number of generated instructions: 178\n", + "What are the reasons for and objectives of the proposal for a Regulation laying down harmonised rules on artificial intelligence?\n", + "How can artificial intelligence improve prediction, optimise operations and resource allocation, and personalise service delivery?\n", + "What benefits can artificial intelligence bring to the European economy and society as a whole?\n", + "How can the use of artificial intelligence support socially and environmentally beneficial outcomes?\n", + "What are the high-impact sectors that require AI action according to the AI Act by the European Union?\n" + ] + } + ], + "source": [ + "instructions = []\n", + "for generations in generated_instructions[\"instructions\"]:\n", + " for generation in generations:\n", + " instructions.extend(generation)\n", + "\n", + "print(f\"Number of generated instructions: {len(instructions)}\")\n", + "\n", + "for instruction in instructions[:5]:\n", + " print(instruction)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "这些初始指令构成了我们的指令数据集。遵循人机协同的方法，我们应该将指令推送到 Argilla 进行可视化，并能够根据质量对它们进行排序。这些注释对于制作高质量的数据至关重要，确保最终模型有更好的性能。然而，这一步是可选的。\n", + "\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### 将指令数据集推送到Argilla以进行可视化和注释。\n", + "\n", + "让我们快速查看一下由 `SelfInstructTask` 生成的指令。\n", + "\n" + ] + }, + { + "cell_type": "code", + "execution_count": 15, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "{'input': 'EN EN\\nEUROPEAN\\nCOMMISSION\\nProposal for a\\nREGULATION OF THE EUROPEAN PARLIAMENT AND OF THE COUNCIL\\nLAYING DOWN HARMONISED RULES ON ARTIFICIAL INTELLIGENCE\\n(ARTIFICIAL INTELLIGENCE ACT) AND AMENDING CERTAIN UNION\\nLEGISLATIVE ACTS\\x0cEN\\nEXPLANATORY MEMORANDUM\\n1. CONTEXT OF THE PROPOSAL\\n1.1. Reasons for and objectives of the proposal\\nThis explanatory memorandum accompanies the proposal for a Regulation laying down\\nharmonised rules on artificial intelligence (Artificial Intelligence Act). Artificial Intelligence\\n(AI) is a fast evolving family of technologies that can bring a wide array of economic and\\nsocietal benefits across the entire spectrum of industries and social activities. By improving\\nprediction, optimising operations and resource allocation, and personalising service delivery,\\nthe use of artificial intelligence can support socially and environmentally beneficial outcomes\\nand provide key competitive advantages to companies and the European economy. ',\n", + " 'generation_model': ['argilla/notus-7b-v1'],\n", + " 'generation_prompt': ['You are an expert prompt writer, writing the best and most diverse prompts for a variety of tasks. You are given a task description and a set of instructions for how to write the prompts for an specific AI application.\\n# Task Description\\nDevelop 5 user queries that can be received by the given AI application and applicable to the provided context. Emphasize diversity in verbs and linguistic structures within the model\\'s textual capabilities.\\n\\n# Criteria for Queries\\nIncorporate a diverse range of verbs, avoiding repetition.\\nEnsure queries are compatible with AI model\\'s text generation functions and are limited to 1-2 sentences.\\nDesign queries to be self-contained and standalone.\\nBlend interrogative (e.g., \"What is the significance of x?\") and imperative (e.g., \"Detail the process of x.\") styles.\\nWrite each query on a separate line and avoid using numbered lists or bullet points.\\n\\n# AI Application\\nA assistant that can answer questions about the AI Act made by the European Union.\\n\\n# Context\\nEN EN\\nEUROPEAN\\nCOMMISSION\\nProposal for a\\nREGULATION OF THE EUROPEAN PARLIAMENT AND OF THE COUNCIL\\nLAYING DOWN HARMONISED RULES ON ARTIFICIAL INTELLIGENCE\\n(ARTIFICIAL INTELLIGENCE ACT) AND AMENDING CERTAIN UNION\\nLEGISLATIVE ACTS\\x0cEN\\nEXPLANATORY MEMORANDUM\\n1. CONTEXT OF THE PROPOSAL\\n1.1. Reasons for and objectives of the proposal\\nThis explanatory memorandum accompanies the proposal for a Regulation laying down\\nharmonised rules on artificial intelligence (Artificial Intelligence Act). Artificial Intelligence\\n(AI) is a fast evolving family of technologies that can bring a wide array of economic and\\nsocietal benefits across the entire spectrum of industries and social activities. By improving\\nprediction, optimising operations and resource allocation, and personalising service delivery,\\nthe use of artificial intelligence can support socially and environmentally beneficial outcomes\\nand provide key competitive advantages to companies and the European economy. \\n\\n# Output\\n'],\n", + " 'raw_generation_responses': ['1. What are the reasons for and objectives of the proposal for a Regulation laying down harmonised rules on artificial intelligence?\\n2. How can artificial intelligence improve prediction, optimise operations and resource allocation, and personalise service delivery?\\n3. What benefits can artificial intelligence bring to the European economy and society as a whole?\\n4. How can the use of artificial intelligence support socially and environmentally beneficial outcomes?\\n5. What competitive advantages can companies gain from using artificial intelligence?'],\n", + " 'instructions': [['What are the reasons for and objectives of the proposal for a Regulation laying down harmonised rules on artificial intelligence?',\n", + " 'How can artificial intelligence improve prediction, optimise operations and resource allocation, and personalise service delivery?',\n", + " 'What benefits can artificial intelligence bring to the European economy and society as a whole?',\n", + " 'How can the use of artificial intelligence support socially and environmentally beneficial outcomes?']]}" + ] + }, + "execution_count": 15, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "generated_instructions[0]" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "对于每个输入，即 AI 法案 PDF 文件的每个批次，我们都有一个生成器提示，其中包含了关于如何行动的通用指南，以及应用程序描述参数。每个输入已经生成了 4 条指令。\n", + "\n", + "现在正好是将指令数据集上传到 Argilla，审查并手动注释它的最佳时机。" + ] + }, + { + "cell_type": "code", + "execution_count": 16, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "FeedbackRecord(fields={'input': 'EN EN\\nEUROPEAN\\nCOMMISSION\\nProposal for a\\nREGULATION OF THE EUROPEAN PARLIAMENT AND OF THE COUNCIL\\nLAYING DOWN HARMONISED RULES ON ARTIFICIAL INTELLIGENCE\\n(ARTIFICIAL INTELLIGENCE ACT) AND AMENDING CERTAIN UNION\\nLEGISLATIVE ACTS\\x0cEN\\nEXPLANATORY MEMORANDUM\\n1. CONTEXT OF THE PROPOSAL\\n1.1. Reasons for and objectives of the proposal\\nThis explanatory memorandum accompanies the proposal for a Regulation laying down\\nharmonised rules on artificial intelligence (Artificial Intelligence Act). Artificial Intelligence\\n(AI) is a fast evolving family of technologies that can bring a wide array of economic and\\nsocietal benefits across the entire spectrum of industries and social activities. By improving\\nprediction, optimising operations and resource allocation, and personalising service delivery,\\nthe use of artificial intelligence can support socially and environmentally beneficial outcomes\\nand provide key competitive advantages to companies and the European economy.', 'instruction': 'What are the reasons for and objectives of the proposal for a Regulation laying down harmonised rules on artificial intelligence?'}, metadata={'length-input': 964, 'length-instruction': 129, 'generation-model': 'argilla/notus-7b-v1'}, vectors={}, responses=[], suggestions=(), external_id=None)" + ] + }, + "execution_count": 16, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "instructions_rg_dataset = generated_instructions.to_argilla()\n", + "instructions_rg_dataset[0]" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "instructions_rg_dataset.push_to_argilla(name=f\"notus_AI_instructions\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "在 Argilla 的用户界面中，每个输入-指令元组都会单独显示，并且可以单独进行注释。\n", + "\n", + "![Instruction dataset](https://huggingface.co/datasets/huggingface/cookbook-images/resolve/main/instrucion_dataset_notus_ui.png)\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 使用 Ultrafeedback 文本质量任务生成偏好数据集\n", + "\n", + "一旦我们有了指令数据集，我们将会通过 UltraFeedback 文本质量任务创建一个偏好数据集。这是一种在自然语言处理中用于评估生成文本质量的任务类型；我们的目标是提供关于生成文本质量的详细反馈，而不仅仅是二元的标签。\n", + "我们的 `pipeline()` 方法允许我们为给定的任务创建一个带有提供的 LLMs 的 `Pipeline` 实例，这在你想要为给定任务使用预定义或自定义的 `Pipeline` 时非常有用。我们将指定我们的任务和子任务，我们想要使用的生成器（在这个案例中，是基于文本生成任务的生成器）以及我们的 OpenAI API 密钥。\n", + "\n", + "> 请注意，不使用 OpenAI 模型来获取此反馈也是可能的。然而，性能将会受到影响，反馈的质量也会较低。" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "preference_pipeline = pipeline(\n", + " \"preference\",\n", + " \"instruction-following\",\n", + " generator=InferenceEndpointsLLM(\n", + " endpoint_name_or_model_id=os.getenv(\"HF_INFERENCE_ENDPOINT_NAME\"), # type: ignore\n", + " endpoint_namespace=os.getenv(\"HF_NAMESPACE\", None),\n", + " task=TextGenerationTask(),\n", + " max_new_tokens=256,\n", + " num_threads=2,\n", + " temperature=0.3,\n", + " ),\n", + " max_new_tokens=256,\n", + " num_threads=2,\n", + " api_key=os.getenv(\"OPENAI_API_KEY\", None),\n", + " temperature=0.0,\n", + ")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "我们还需要从 Argilla 检索我们的指令数据集，因为它将是这个流水线的输入。\n" + ] + }, + { + "cell_type": "code", + "execution_count": 19, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "Dataset({\n", + " features: ['input', 'instruction', 'instruction-rating', 'instruction-rating-suggestion', 'instruction-rating-suggestion-metadata', 'external_id', 'metadata'],\n", + " num_rows: 100\n", + "})" + ] + }, + "execution_count": 19, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "remote_dataset = rg.FeedbackDataset.from_argilla(\n", + " \"notus_AI_instructions\", workspace=\"admin\"\n", + ")\n", + "instructions_dataset = remote_dataset.pull(max_records=100) # get first 100 records\n", + "\n", + "instructions_dataset = instructions_dataset.format_as(\"datasets\")\n", + "instructions_dataset" + ] + }, + { + "cell_type": "code", + "execution_count": 20, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "{'input': 'EN EN\\nEUROPEAN\\nCOMMISSION\\nProposal for a\\nREGULATION OF THE EUROPEAN PARLIAMENT AND OF THE COUNCIL\\nLAYING DOWN HARMONISED RULES ON ARTIFICIAL INTELLIGENCE\\n(ARTIFICIAL INTELLIGENCE ACT) AND AMENDING CERTAIN UNION\\nLEGISLATIVE ACTS\\x0cEN\\nEXPLANATORY MEMORANDUM\\n1. CONTEXT OF THE PROPOSAL\\n1.1. Reasons for and objectives of the proposal\\nThis explanatory memorandum accompanies the proposal for a Regulation laying down\\nharmonised rules on artificial intelligence (Artificial Intelligence Act). Artificial Intelligence\\n(AI) is a fast evolving family of technologies that can bring a wide array of economic and\\nsocietal benefits across the entire spectrum of industries and social activities. By improving\\nprediction, optimising operations and resource allocation, and personalising service delivery,\\nthe use of artificial intelligence can support socially and environmentally beneficial outcomes\\nand provide key competitive advantages to companies and the European economy.',\n", + " 'instruction': 'What are the reasons for and objectives of the proposal for a Regulation laying down harmonised rules on artificial intelligence?',\n", + " 'instruction-rating': [],\n", + " 'instruction-rating-suggestion': None,\n", + " 'instruction-rating-suggestion-metadata': {'type': None,\n", + " 'score': None,\n", + " 'agent': None},\n", + " 'external_id': None,\n", + " 'metadata': '{\"length-input\": 964, \"length-instruction\": 129, \"generation-model\": \"argilla/notus-7b-v1\"}'}" + ] + }, + "execution_count": 20, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "instructions_dataset[0]" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "在根据我们的指令生成文本之前，我们需要重命名数据集中的某些列。从前面的部分，我们仍然有旧的输入，即来自 PDF 的批次。我们需要将它们改为我们生成的指令。" + ] + }, + { + "cell_type": "code", + "execution_count": 21, + "metadata": {}, + "outputs": [], + "source": [ + "instructions_dataset = instructions_dataset.rename_columns({\"input\": \"context\", \"instruction\": \"input\"})" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "现在，让我们使用刚刚创建的流水线以及生成我们指令的主题来构建一个数据集。\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "preference_dataset = preference_pipeline.generate(\n", + " instructions_dataset, # type: ignore\n", + " num_generations=2,\n", + " batch_size=8,\n", + " display_progress_bar=True,\n", + ")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "让我们来看一下偏好数据集的一个实例：" + ] + }, + { + "cell_type": "code", + "execution_count": 23, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "{'context': 'EN EN\\nEUROPEAN\\nCOMMISSION\\nProposal for a\\nREGULATION OF THE EUROPEAN PARLIAMENT AND OF THE COUNCIL\\nLAYING DOWN HARMONISED RULES ON ARTIFICIAL INTELLIGENCE\\n(ARTIFICIAL INTELLIGENCE ACT) AND AMENDING CERTAIN UNION\\nLEGISLATIVE ACTS\\x0cEN\\nEXPLANATORY MEMORANDUM\\n1. CONTEXT OF THE PROPOSAL\\n1.1. Reasons for and objectives of the proposal\\nThis explanatory memorandum accompanies the proposal for a Regulation laying down\\nharmonised rules on artificial intelligence (Artificial Intelligence Act). Artificial Intelligence\\n(AI) is a fast evolving family of technologies that can bring a wide array of economic and\\nsocietal benefits across the entire spectrum of industries and social activities. By improving\\nprediction, optimising operations and resource allocation, and personalising service delivery,\\nthe use of artificial intelligence can support socially and environmentally beneficial outcomes\\nand provide key competitive advantages to companies and the European economy.',\n", + " 'input': 'What are the reasons for and objectives of the proposal for a Regulation laying down harmonised rules on artificial intelligence?',\n", + " 'instruction-rating': [],\n", + " 'instruction-rating-suggestion': None,\n", + " 'instruction-rating-suggestion-metadata': {'agent': None,\n", + " 'score': None,\n", + " 'type': None},\n", + " 'external_id': None,\n", + " 'metadata': '{\"length-input\": 964, \"length-instruction\": 129, \"generation-model\": \"argilla/notus-7b-v1\"}',\n", + " 'generation_model': ['argilla/notus-7b-v1', 'argilla/notus-7b-v1'],\n", + " 'generation_prompt': [\"You are a helpful, respectful and honest assistant. Always answer as helpfully as possible, while being safe. Your answers should not include any harmful, unethical, racist, sexist, toxic, dangerous, or illegal content. Please ensure that your responses are socially unbiased and positive in nature.\\nIf a question does not make any sense, or is not factually coherent, explain why instead of answering something not correct. If you don't know the answer to a question, please don't share false information.\\nWhat are the reasons for and objectives of the proposal for a Regulation laying down harmonised rules on artificial intelligence?\",\n", + " \"You are a helpful, respectful and honest assistant. Always answer as helpfully as possible, while being safe. Your answers should not include any harmful, unethical, racist, sexist, toxic, dangerous, or illegal content. Please ensure that your responses are socially unbiased and positive in nature.\\nIf a question does not make any sense, or is not factually coherent, explain why instead of answering something not correct. If you don't know the answer to a question, please don't share false information.\\nWhat are the reasons for and objectives of the proposal for a Regulation laying down harmonised rules on artificial intelligence?\"],\n", + " 'raw_generation_responses': [\"\\nThe proposal for a Regulation laying down harmonised rules on artificial intelligence (AI) aims to ensure the trustworthy use of AI in the EU. It seeks to create a single market for AI applications and services, while ensuring that they are safe and respect fundamental rights. The proposal is part of the EU's broader strategy on AI, which aims to put the EU at the forefront of global AI development and deployment.\\nThe objectives of the proposal are to:\\n\\n1. Ensure that AI systems are designed, developed, and deployed in a way that respects fundamental rights and values, including human dignity, freedom, and privacy.\\n2. Ensure that AI systems are safe and secure, and do not pose unacceptable risks to people, property, or the environment.\\n3. Ensure that AI systems are robust, reliable, and accurate, and can be trusted to deliver the intended functionality.\\n4. Ensure that AI systems are traceable, meaning that it is possible to track how they work and how they make decisions.\\n5. Ensure that AI systems are transparent, meaning that it is possible to understand how they work and how they make decisions.\\n6. Ensure that AI systems are fair, meaning that they do not discriminate against individuals\",\n", + " '\\nThe proposal for a Regulation laying down harmonised rules on artificial intelligence (AI) aims to ensure a high level of safety and security of AI systems and to establish a horizontal and technology-neutral framework for AI applications. This will help to create a single market for AI and to ensure that AI systems are developed and deployed in a responsible manner. The proposal will also help to strengthen the competitiveness of the EU industry in the global AI market.\\nThe objectives of the proposal are:\\n1. To ensure that AI systems are safe and secure by establishing a risk-based framework for the development, placement on the market and use of AI systems.\\n2. To establish a horizontal and technology-neutral framework for AI applications that is applicable to all sectors and types of AI systems.\\n3. To ensure that AI systems are developed and deployed in a responsible manner by establishing requirements for transparency, robustness, security, accuracy, controllability and privacy protection.\\n4. To create a single market for AI by ensuring that AI systems are developed and deployed in a harmonised manner across the EU.\\n5. To strengthen the competitiveness of the EU industry in the global AI market by creating a level playing field for businesses and by promoting the'],\n", + " 'generations': [\"\\nThe proposal for a Regulation laying down harmonised rules on artificial intelligence (AI) aims to ensure the trustworthy use of AI in the EU. It seeks to create a single market for AI applications and services, while ensuring that they are safe and respect fundamental rights. The proposal is part of the EU's broader strategy on AI, which aims to put the EU at the forefront of global AI development and deployment.\\nThe objectives of the proposal are to:\\n\\n1. Ensure that AI systems are designed, developed, and deployed in a way that respects fundamental rights and values, including human dignity, freedom, and privacy.\\n2. Ensure that AI systems are safe and secure, and do not pose unacceptable risks to people, property, or the environment.\\n3. Ensure that AI systems are robust, reliable, and accurate, and can be trusted to deliver the intended functionality.\\n4. Ensure that AI systems are traceable, meaning that it is possible to track how they work and how they make decisions.\\n5. Ensure that AI systems are transparent, meaning that it is possible to understand how they work and how they make decisions.\\n6. Ensure that AI systems are fair, meaning that they do not discriminate against individuals\",\n", + " '\\nThe proposal for a Regulation laying down harmonised rules on artificial intelligence (AI) aims to ensure a high level of safety and security of AI systems and to establish a horizontal and technology-neutral framework for AI applications. This will help to create a single market for AI and to ensure that AI systems are developed and deployed in a responsible manner. The proposal will also help to strengthen the competitiveness of the EU industry in the global AI market.\\nThe objectives of the proposal are:\\n1. To ensure that AI systems are safe and secure by establishing a risk-based framework for the development, placement on the market and use of AI systems.\\n2. To establish a horizontal and technology-neutral framework for AI applications that is applicable to all sectors and types of AI systems.\\n3. To ensure that AI systems are developed and deployed in a responsible manner by establishing requirements for transparency, robustness, security, accuracy, controllability and privacy protection.\\n4. To create a single market for AI by ensuring that AI systems are developed and deployed in a harmonised manner across the EU.\\n5. To strengthen the competitiveness of the EU industry in the global AI market by creating a level playing field for businesses and by promoting the'],\n", + " 'labelling_model': 'gpt-3.5-turbo',\n", + " 'labelling_prompt': [{'content': 'Your role is to evaluate text quality based on given criteria.',\n", + " 'role': 'system'},\n", + " {'content': \"\\n# Instruction Following Assessment\\nEvaluate alignment between output and intent. Assess understanding of task goal and restrictions.\\n**Instruction Components**: Task Goal (intended outcome), Restrictions (text styles, formats, or designated methods, etc).\\n\\n**Scoring**: Rate outputs 1 to 5:\\n\\n1. **Irrelevant**: No alignment.\\n2. **Partial Focus**: Addresses one aspect poorly.\\n3. **Partial Compliance**:\\n\\t- (1) Meets goal or restrictions, neglecting other.\\n\\t- (2) Acknowledges both but slight deviations.\\n4. **Almost There**: Near alignment, minor deviations.\\n5. **Comprehensive Compliance**: Fully aligns, meets all requirements.\\n\\n---\\n\\n## Format\\n\\n### Input\\nInstruction: [Specify task goal and restrictions]\\n\\nTexts:\\n\\n [Text 1]\\n [Text 2]\\n\\n### Output\\n\\n#### Output for Text 1\\nRating: [Rating for text 1]\\nRationale: [Rationale for the rating in short sentences]\\n\\n#### Output for Text 2\\nRating: [Rating for text 2]\\nRationale: [Rationale for the rating in short sentences]\\n\\n---\\n\\n## Annotation\\n\\n### Input\\nInstruction: What are the reasons for and objectives of the proposal for a Regulation laying down harmonised rules on artificial intelligence?\\n\\nTexts:\\n\\n \\nThe proposal for a Regulation laying down harmonised rules on artificial intelligence (AI) aims to ensure the trustworthy use of AI in the EU. It seeks to create a single market for AI applications and services, while ensuring that they are safe and respect fundamental rights. The proposal is part of the EU's broader strategy on AI, which aims to put the EU at the forefront of global AI development and deployment.\\nThe objectives of the proposal are to:\\n\\n1. Ensure that AI systems are designed, developed, and deployed in a way that respects fundamental rights and values, including human dignity, freedom, and privacy.\\n2. Ensure that AI systems are safe and secure, and do not pose unacceptable risks to people, property, or the environment.\\n3. Ensure that AI systems are robust, reliable, and accurate, and can be trusted to deliver the intended functionality.\\n4. Ensure that AI systems are traceable, meaning that it is possible to track how they work and how they make decisions.\\n5. Ensure that AI systems are transparent, meaning that it is possible to understand how they work and how they make decisions.\\n6. Ensure that AI systems are fair, meaning that they do not discriminate against individuals\\n \\nThe proposal for a Regulation laying down harmonised rules on artificial intelligence (AI) aims to ensure a high level of safety and security of AI systems and to establish a horizontal and technology-neutral framework for AI applications. This will help to create a single market for AI and to ensure that AI systems are developed and deployed in a responsible manner. The proposal will also help to strengthen the competitiveness of the EU industry in the global AI market.\\nThe objectives of the proposal are:\\n1. To ensure that AI systems are safe and secure by establishing a risk-based framework for the development, placement on the market and use of AI systems.\\n2. To establish a horizontal and technology-neutral framework for AI applications that is applicable to all sectors and types of AI systems.\\n3. To ensure that AI systems are developed and deployed in a responsible manner by establishing requirements for transparency, robustness, security, accuracy, controllability and privacy protection.\\n4. To create a single market for AI by ensuring that AI systems are developed and deployed in a harmonised manner across the EU.\\n5. To strengthen the competitiveness of the EU industry in the global AI market by creating a level playing field for businesses and by promoting the\\n\\n### Output \",\n", + " 'role': 'user'}],\n", + " 'raw_labelling_response': '#### Output for Text 1\\nRating: 5\\nRationale: The text fully aligns with the task goal and restrictions. It clearly states the reasons for and objectives of the proposal for a Regulation laying down harmonised rules on artificial intelligence, including ensuring the trustworthy use of AI, creating a single market for AI applications and services, and ensuring safety, respect for fundamental rights, robustness, transparency, and fairness of AI systems.\\n\\n#### Output for Text 2\\nRating: 4\\nRationale: The text mostly aligns with the task goal and restrictions. It addresses the reasons for and objectives of the proposal for a Regulation laying down harmonised rules on artificial intelligence, including ensuring safety and security of AI systems, establishing a horizontal and technology-neutral framework, promoting responsible development and deployment of AI systems, creating a single market for AI, and strengthening the competitiveness of the EU industry in the global AI market. However, it does not explicitly mention the need to respect fundamental rights, accuracy of AI systems, and traceability of AI systems, which are mentioned in the task goal and restrictions.',\n", + " 'rating': [5.0, 4.0],\n", + " 'rationale': ['The text fully aligns with the task goal and restrictions. It clearly states the reasons for and objectives of the proposal for a Regulation laying down harmonised rules on artificial intelligence, including ensuring the trustworthy use of AI, creating a single market for AI applications and services, and ensuring safety, respect for fundamental rights, robustness, transparency, and fairness of AI systems.',\n", + " 'The text mostly aligns with the task goal and restrictions. It addresses the reasons for and objectives of the proposal for a Regulation laying down harmonised rules on artificial intelligence, including ensuring safety and security of AI systems, establishing a horizontal and technology-neutral framework, promoting responsible development and deployment of AI systems, creating a single market for AI, and strengthening the competitiveness of the EU industry in the global AI market. However, it does not explicitly mention the need to respect fundamental rights, accuracy of AI systems, and traceability of AI systems, which are mentioned in the task goal and restrictions.']}" + ] + }, + "execution_count": 23, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "preference_dataset[0]" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 使用 Argilla 进行人工反馈\n", + "\n", + "你可以直接使用 distilabel 创建的 AI 反馈，但我们已经看到，通过加入人工反馈可以提升 LLM 的质量。我们提供了一个 `to_argilla` 方法，它为 Argilla 创建了一个数据集，并附带了现成的定制元数据过滤器以及语义搜索，让你能够尽可能快速和有趣地提供人工反馈。你可以查看[ Argilla 文档](https://docs.argilla.io/en/latest/getting_started/quickstart_installation.html)来了解如何安装和运行。\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "如果你正在使用 Docker 快速启动镜像或 Hugging Face Spaces 运行Argilla，你需要使用 URL 和 API_KEY 初始化 Argilla 客户端：" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "import argilla as rg\n", + "\n", + "# Replace api_url with the url to your HF Spaces URL if using Spaces\n", + "# Replace api_key if you configured a custom API key\n", + "rg.init(\n", + " api_url=\"http://localhost:6900\",\n", + " api_key=\"owner.apikey\",\n", + " workspace=\"admin\"\n", + ")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "一旦我们成功地制作出了偏好数据集，Argilla 的用户界面就是最适合我们用来查看和标记这些数据的东西。就像我们对指令数据集所做的那样，我们只需要把这个数据集变成 Argilla 能理解的格式，然后上传到 Argilla 上就可以开始工作了。" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Uploading the Preference Dataset\n", + "preference_rg_dataset = preference_dataset.to_argilla()\n", + "\n", + "# Adding the context as a metadata property in the new Feedback dataset, as this\n", + "# information will be useful later.\n", + "for record_feedback, record_huggingface in zip(\n", + " preference_rg_dataset, preference_dataset\n", + "):\n", + " record_feedback.metadata[\"context\"] = record_huggingface[\"context\"]\n", + "\n", + "preference_rg_dataset.push_to_argilla(name=f\"notus_AI_preference\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "在Argilla用户界面中，我们可以看到输入（一个指令），以及 LLM 基于该指令创建的两个生成文本。\n", + "\n", + "![Preference dataset](https://huggingface.co/datasets/huggingface/cookbook-images/resolve/main/preference_dataset_notus_ui.png)\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 结论\n", + "\n", + "总结一下，我们已经完成了一个使用 distilabel 的端到端示例。我们建立了一个推理端点，定义了一个从 PDF 提取信息的 distilabel 流水线，并创建和手动审查了从该输入生成的指令和偏好数据集。最终的偏好数据集非常适合进行微调，你可以使用 Argilla 的 ArgillaTrainer 轻松完成这一工作。如果你想深入了解，请查看以下资源：\n", + "\n", + "- [使用 ArgillaTrainer 训练模型](https://docs.argilla.io/en/latest/tutorials_and_integrations/tutorials/feedback/end2end_examples/train-model-006.html)\n", + "- [Ⓜ️ 将 LLM 作为聊天助手进行监督式微调：Mistral 7B](https://docs.argilla.io/en/latest/tutorials_and_integrations/tutorials/feedback/training-llm-mistral-sft.html)\n", + "- [🌠 通过优化检索和重排模型来改进 RAG](https://docs.argilla.io/en/latest/tutorials_and_integrations/tutorials/feedback/fine-tuning-sentencesimilarity-rag.html)\n" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.10.13" + } + }, + "nbformat": 4, + "nbformat_minor": 4 +} From 2eeeab0f3730df10996f69b8fbbf4fcc428a7fb8 Mon Sep 17 00:00:00 2001 From: innovation64 Date: Mon, 1 Apr 2024 18:44:35 +0800 Subject: [PATCH 12/31] docs: update prompt_turning in chinese version --- notebooks/zh-CN/prompt_tuning_peft.ipynb | 1014 ++++++++++++++++++++++ 1 file changed, 1014 insertions(+) create mode 100644 notebooks/zh-CN/prompt_tuning_peft.ipynb diff --git a/notebooks/zh-CN/prompt_tuning_peft.ipynb b/notebooks/zh-CN/prompt_tuning_peft.ipynb new file mode 100644 index 00000000..99701df5 --- /dev/null +++ b/notebooks/zh-CN/prompt_tuning_peft.ipynb @@ -0,0 +1,1014 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": { + "application/vnd.databricks.v1+cell": { + "cellMetadata": {}, + "inputWidgets": {}, + "nuid": "6fba2d42-ed99-4a03-8033-d479ce24d5dd", + "showTitle": false, + "title": "" + }, + "id": "2vkOvTEsVaTA" + }, + "source": [ + "# 使用 PEFT 进行提示微调。\n", + "\n", + "_作者: [Pere Martra](https://github.com/peremartra)_\n", + "\n", + "\n", + "在这个 notebook 中，我们将介绍如何使用 PEFT 库对预训练模型进行提示微调。\n", + "\n", + "要查看与 PEFT 兼容的完整模型列表，请参考他们的[文档](https://huggingface.co/docs/peft/main/en/index#supported-methods)。\n", + "\n", + "可以使用 PEFT 进行训练的模型示例包括 Bloom、Llama、GPT-J、GPT-2、BERT 等等。Hugging Face 正在努力将更多模型添加到库中。\n", + "\n", + "## 提示微调简要介绍\n", + "\n", + "这是一种用于模型的附加微调技术。这意味着我们不会修改原始模型的任何权重。你可能会想，那么我们将如何进行微调呢？好吧，我们将训练添加到模型中的额外层。这就是为什么它被称为附加技术。\n", + "\n", + "考虑到它是一种附加技术，并且它的名字是提示调整，似乎很明显我们将要添加和训练的层与提示有关。\n", + "\n", + "![Prompt_Tuning_Diagram](https://huggingface.co/datasets/huggingface/cookbook-images/resolve/main/Martra_Figure_5_Prompt_Tuning.jpg)\n", + "\n", + "我们通过使模型能够用其获取的知识增强提示的一部分来创建一种超提示。然而，提示的这部分不能翻译成自然语言。**这就好像我们已经掌握了用嵌入表达自己并生成高效提示的能力。**\n", + "\n", + "在每次训练周期中，唯一可以修改以最小化损失函数的权重是集成到提示中的权重。\n", + "\n", + "这种技术的主要结果是，要训练的参数数量确实很少。然而，我们遇到了第二个，也许更重要的结果，即**由于我们不修改预训练模型的权重，它不会改变其行为或忘记它以前学到的任何信息。**\n", + "\n", + "训练更快，更具成本效益。此外，我们可以训练各种模型，在推理时，我们只需要加载一个基础模型以及新的较小的训练模型，因为原始模型的权重没有被修改。\n", + "\n", + "## 我们将在 notebook 中做什么？\n", + "\n", + "我们将使用两个数据集训练两个不同的模型，每个数据集只使用 Bloom 家族的一个预训练模型。一个模型将使用提示数据集进行训练，而另一个模型将使用激励句子数据集进行训练。我们将比较两个模型在训练前后对同一问题的结果。\n", + "\n", + "此外，我们还将探讨如何只加载基础模型的一个副本到内存中，同时加载两个模型。\n" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "tZhdbTh-VaTA" + }, + "source": [ + "## 加载 PEFT 库\n", + "这个库包含了各种微调技术的 Hugging Face 实现，包括提示调整。" + ] + }, + { + "cell_type": "code", + "execution_count": 41, + "metadata": { + "application/vnd.databricks.v1+cell": { + "cellMetadata": {}, + "inputWidgets": {}, + "nuid": "d16bf5ec-888b-4c76-a655-193fd4cc8a36", + "showTitle": false, + "title": "" + }, + "id": "JechhJhhVaTA" + }, + "outputs": [], + "source": [ + "!pip install -q peft==0.8.2" + ] + }, + { + "cell_type": "code", + "execution_count": 42, + "metadata": { + "id": "6CRxq5Z2WJ7C" + }, + "outputs": [], + "source": [ + "!pip install -q datasets==2.14.5" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "GGbh426RVaTB" + }, + "source": [ + "从 transformers 库中，我们导入必要的类来实例化模型和分词器。" + ] + }, + { + "cell_type": "code", + "execution_count": 43, + "metadata": { + "application/vnd.databricks.v1+cell": { + "cellMetadata": {}, + "inputWidgets": {}, + "nuid": "31738463-c9b0-431d-869e-1735e1e2f5c7", + "showTitle": false, + "title": "" + }, + "id": "KWOEt-yOVaTB" + }, + "outputs": [], + "source": [ + "from transformers import AutoModelForCausalLM, AutoTokenizer" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "6qYsnwjSVaTC" + }, + "source": [ + "### 加载模型和分词器。\n", + "\n", + "Bloom 是使用 PEFT 库进行提示调整训练的可用模型中最小最智能的模型之一。你可以从 Bloom 家族中选择任何模型，我鼓励你至少尝试其中两个以观察它们之间的差异。\n", + "\n", + "我选择最小的模型以最小化训练时间并避免在 Colab 中出现内存问题。\n" + ] + }, + { + "cell_type": "code", + "execution_count": 44, + "metadata": { + "id": "MnqIhv2UVaTC" + }, + "outputs": [], + "source": [ + "model_name = \"bigscience/bloomz-560m\"\n", + "#model_name=\"bigscience/bloom-1b1\"\n", + "NUM_VIRTUAL_TOKENS = 4\n", + "NUM_EPOCHS = 6" + ] + }, + { + "cell_type": "code", + "execution_count": 45, + "metadata": { + "id": "fSMu3qRsVaTC" + }, + "outputs": [], + "source": [ + "tokenizer = AutoTokenizer.from_pretrained(model_name)\n", + "foundational_model = AutoModelForCausalLM.from_pretrained(\n", + " model_name,\n", + " trust_remote_code=True\n", + ")" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "8W2fWhOnVaTC" + }, + "source": [ + "## 使用预训练的 bloom 模型进行推理\n", + "如果你想要实现更多样化和原创的生成，取消注释下面的 *model.generate* 中的参数：temperature、top_p 和 do_sample。\n", + "\n", + "在默认配置下，模型的响应在多次调用中保持一致。" + ] + }, + { + "cell_type": "code", + "execution_count": 46, + "metadata": { + "id": "47j2D3WWVaTC" + }, + "outputs": [], + "source": [ + "#this function returns the outputs from the model received, and inputs.\n", + "def get_outputs(model, inputs, max_new_tokens=100):\n", + " outputs = model.generate(\n", + " input_ids=inputs[\"input_ids\"],\n", + " attention_mask=inputs[\"attention_mask\"],\n", + " max_new_tokens=max_new_tokens,\n", + " #temperature=0.2,\n", + " #top_p=0.95,\n", + " #do_sample=True,\n", + " repetition_penalty=1.5, #Avoid repetition.\n", + " early_stopping=True, #The model can stop before reach the max_length\n", + " eos_token_id=tokenizer.eos_token_id\n", + " )\n", + " return outputs" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "application/vnd.databricks.v1+cell": { + "cellMetadata": {}, + "inputWidgets": {}, + "nuid": "ca4d203a-5152-4947-ab34-cfd0b40a102a", + "showTitle": false, + "title": "" + }, + "id": "kRLSfuo2VaTC" + }, + "source": [ + "由于我们希望有两个不同的训练模型，我将创建两个不同的提示。\n", + "\n", + "第一个模型将使用包含提示的数据集进行训练，第二个模型将使用激励句子的数据集进行训练。\n", + "\n", + "第一个模型将收到提示 \"我希望你扮演一个励志教练。\"，第二个模型将收到提示 \"有两件对你来说很重要的事情：\"\n", + "\n", + "但首先，我要收集一些未经微调的模型的结果。" + ] + }, + { + "cell_type": "code", + "execution_count": 47, + "metadata": { + "application/vnd.databricks.v1+cell": { + "cellMetadata": {}, + "inputWidgets": {}, + "nuid": "1d4c80a9-4edd-4fcd-aef0-996f4da5cc02", + "showTitle": false, + "title": "" + }, + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "QvStaT7cVaTC", + "outputId": "ab34b3cd-a849-4dff-b36d-bf25c9f55ce1" + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "[\"I want you to act as a motivational coach. Don't be afraid of being challenged.\"]\n" + ] + } + ], + "source": [ + "input_prompt = tokenizer(\"I want you to act as a motivational coach. \", return_tensors=\"pt\")\n", + "foundational_outputs_prompt = get_outputs(foundational_model, input_prompt, max_new_tokens=50)\n", + "\n", + "print(tokenizer.batch_decode(foundational_outputs_prompt, skip_special_tokens=True))" + ] + }, + { + "cell_type": "code", + "execution_count": 69, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "1Xhm3jZMVaTD", + "outputId": "305f0137-6a02-4e43-9c9d-2b4ecd377937" + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "['There are two nice things that should matter to you: the price and quality of your product.']\n" + ] + } + ], + "source": [ + "input_sentences = tokenizer(\"There are two nice things that should matter to you:\", return_tensors=\"pt\")\n", + "foundational_outputs_sentence = get_outputs(foundational_model, input_sentences, max_new_tokens=50)\n", + "\n", + "print(tokenizer.batch_decode(foundational_outputs_sentence, skip_special_tokens=True))" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "application/vnd.databricks.v1+cell": { + "cellMetadata": {}, + "inputWidgets": {}, + "nuid": "f438d43b-6b9f-445e-9df4-60ea09640764", + "showTitle": false, + "title": "" + }, + "id": "OGbJTbRnVaTD" + }, + "source": [ + "两个答案或多或少都是正确的。任何 Bloom 模型都是预先训练的，能够准确和合理地生成句子。让我们看看，在训练之后，响应是否相等或者生成得更加准确。\n", + "\n", + "## 准备数据集\n", + "使用的数据集包括：\n", + "* https://huggingface.co/datasets/fka/awesome-chatgpt-prompts\n", + "* https://huggingface.co/datasets/Abirate/english_quotes\n", + "\n" + ] + }, + { + "cell_type": "code", + "execution_count": 49, + "metadata": { + "id": "RD8H_LLaVaTD" + }, + "outputs": [], + "source": [ + "import os\n", + "#os.environ[\"TOKENIZERS_PARALLELISM\"] = \"false\"" + ] + }, + { + "cell_type": "code", + "execution_count": 50, + "metadata": { + "application/vnd.databricks.v1+cell": { + "cellMetadata": {}, + "inputWidgets": {}, + "nuid": "2ed62b41-e3fa-4a41-a0a9-59f35a6904f9", + "showTitle": false, + "title": "" + }, + "id": "xmAp_o4PVaTD" + }, + "outputs": [], + "source": [ + "from datasets import load_dataset\n", + "\n", + "dataset_prompt = \"fka/awesome-chatgpt-prompts\"\n", + "\n", + "#Create the Dataset to create prompts.\n", + "data_prompt = load_dataset(dataset_prompt)\n", + "data_prompt = data_prompt.map(lambda samples: tokenizer(samples[\"prompt\"]), batched=True)\n", + "train_sample_prompt = data_prompt[\"train\"].select(range(50))\n" + ] + }, + { + "cell_type": "code", + "execution_count": 51, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 86 + }, + "id": "jNlOpGbqBgcu", + "outputId": "3f8106b2-948b-4a7b-cf78-bd3fcc2f0338" + }, + "outputs": [ + { + "data": { + "text/plain": [ + "Dataset({\n", + " features: ['act', 'prompt', 'input_ids', 'attention_mask'],\n", + " num_rows: 50\n", + "})" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "display(train_sample_prompt)" + ] + }, + { + "cell_type": "code", + "execution_count": 52, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "dZcOaE5CU658", + "outputId": "fb8f5081-012b-4c37-ee1f-3aef2d0f54a7" + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "{'act': ['Linux Terminal'], 'prompt': ['I want you to act as a linux terminal. I will type commands and you will reply with what the terminal should show. I want you to only reply with the terminal output inside one unique code block, and nothing else. do not write explanations. do not type commands unless I instruct you to do so. when i need to tell you something in english, i will do so by putting text inside curly brackets {like this}. my first command is pwd'], 'input_ids': [[44, 4026, 1152, 427, 1769, 661, 267, 104105, 28434, 17, 473, 2152, 4105, 49123, 530, 1152, 2152, 57502, 1002, 3595, 368, 28434, 3403, 6460, 17, 473, 4026, 1152, 427, 3804, 57502, 1002, 368, 28434, 10014, 14652, 2592, 19826, 4400, 10973, 15, 530, 16915, 4384, 17, 727, 1130, 11602, 184637, 17, 727, 1130, 4105, 49123, 35262, 473, 32247, 1152, 427, 727, 1427, 17, 3262, 707, 3423, 427, 13485, 1152, 7747, 361, 170205, 15, 707, 2152, 727, 1427, 1331, 55385, 5484, 14652, 6291, 999, 117805, 731, 29726, 1119, 96, 17, 2670, 3968, 9361, 632, 269, 42512]], 'attention_mask': [[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]]}\n" + ] + } + ], + "source": [ + "print(train_sample_prompt[:1])" + ] + }, + { + "cell_type": "code", + "execution_count": 53, + "metadata": { + "id": "WeM66LmEVaTD" + }, + "outputs": [], + "source": [ + "dataset_sentences = load_dataset(\"Abirate/english_quotes\")\n", + "\n", + "data_sentences = dataset_sentences.map(lambda samples: tokenizer(samples[\"quote\"]), batched=True)\n", + "train_sample_sentences = data_sentences[\"train\"].select(range(25))\n", + "train_sample_sentences = train_sample_sentences.remove_columns(['author', 'tags'])" + ] + }, + { + "cell_type": "code", + "execution_count": 54, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 86 + }, + "id": "zUSG_M_nBp_E", + "outputId": "faf36464-de24-4512-aace-c1ff8713c1d4" + }, + "outputs": [ + { + "data": { + "text/plain": [ + "Dataset({\n", + " features: ['quote', 'input_ids', 'attention_mask'],\n", + " num_rows: 25\n", + "})" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "display(train_sample_sentences)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "application/vnd.databricks.v1+cell": { + "cellMetadata": {}, + "inputWidgets": {}, + "nuid": "b97381d4-5fe2-49d0-be5d-2fe3421edc5c", + "showTitle": false, + "title": "" + }, + "id": "0-5mv1ZpVaTD" + }, + "source": [ + "## 微调\n", + "\n", + "### PEFT 配置\n", + "\n", + "API 文档：\n", + "https://huggingface.co/docs/peft/main/en/package_reference/tuners#peft.PromptTuningConfig\n", + "\n", + "我们可以对两个要训练的模型使用相同的配置。\n" + ] + }, + { + "cell_type": "code", + "execution_count": 55, + "metadata": { + "application/vnd.databricks.v1+cell": { + "cellMetadata": {}, + "inputWidgets": {}, + "nuid": "6df8e1f1-be9e-42db-b4a4-6af7cd351004", + "showTitle": false, + "title": "" + }, + "id": "sOg1Yh-oVaTD" + }, + "outputs": [], + "source": [ + "from peft import get_peft_model, PromptTuningConfig, TaskType, PromptTuningInit\n", + "\n", + "generation_config = PromptTuningConfig(\n", + " task_type=TaskType.CAUSAL_LM, #This type indicates the model will generate text.\n", + " prompt_tuning_init=PromptTuningInit.RANDOM, #The added virtual tokens are initializad with random numbers\n", + " num_virtual_tokens=NUM_VIRTUAL_TOKENS, #Number of virtual tokens to be added and trained.\n", + " tokenizer_name_or_path=model_name #The pre-trained model.\n", + ")\n" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "an9KBtB1VaTD" + }, + "source": [ + "### 创建两个提示调整模型。\n", + "我们将使用相同的预训练模型和相同的配置来创建两个相同的提示调整模型。" + ] + }, + { + "cell_type": "code", + "execution_count": 56, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "c_D8oDQZVaTD", + "outputId": "6b46ca98-3f60-49c1-dab2-91259d6387af" + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "trainable params: 4,096 || all params: 559,218,688 || trainable%: 0.0007324504863471229\n", + "None\n" + ] + } + ], + "source": [ + "peft_model_prompt = get_peft_model(foundational_model, generation_config)\n", + "print(peft_model_prompt.print_trainable_parameters())" + ] + }, + { + "cell_type": "code", + "execution_count": 57, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "IktYfj68VaTE", + "outputId": "28fe03b7-4490-43ba-b913-4633e269737a" + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "trainable params: 4,096 || all params: 559,218,688 || trainable%: 0.0007324504863471229\n", + "None\n" + ] + } + ], + "source": [ + "peft_model_sentences = get_peft_model(foundational_model, generation_config)\n", + "print(peft_model_sentences.print_trainable_parameters())" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "application/vnd.databricks.v1+cell": { + "cellMetadata": {}, + "inputWidgets": {}, + "nuid": "cff5bc33-8cfb-4144-8962-9c54362a7faa", + "showTitle": false, + "title": "" + }, + "id": "i6WhJSUwVaTE" + }, + "source": [ + "**太神奇了：你看到可训练参数的减少了吗？我们将要训练可用参数的 0.001%。**\n", + "\n", + "现在我们要创建训练参数，并且在这两次训练中我们将使用相同的配置。" + ] + }, + { + "cell_type": "code", + "execution_count": 58, + "metadata": { + "id": "SJoznfzjVaTE" + }, + "outputs": [], + "source": [ + "from transformers import TrainingArguments\n", + "def create_training_arguments(path, learning_rate=0.0035, epochs=6):\n", + " training_args = TrainingArguments(\n", + " output_dir=path, # Where the model predictions and checkpoints will be written\n", + " use_cpu=True, # This is necessary for CPU clusters.\n", + " auto_find_batch_size=True, # Find a suitable batch size that will fit into memory automatically\n", + " learning_rate= learning_rate, # Higher learning rate than full Fine-Tuning\n", + " num_train_epochs=epochs\n", + " )\n", + " return training_args" + ] + }, + { + "cell_type": "code", + "execution_count": 59, + "metadata": { + "application/vnd.databricks.v1+cell": { + "cellMetadata": {}, + "inputWidgets": {}, + "nuid": "54b78a8f-81f0-44c0-b0bc-dcb14891715f", + "showTitle": false, + "title": "" + }, + "id": "cb1j50DSVaTE" + }, + "outputs": [], + "source": [ + "\n", + "import os\n", + "\n", + "working_dir = \"./\"\n", + "\n", + "#Is best to store the models in separate folders.\n", + "#Create the name of the directories where to store the models.\n", + "output_directory_prompt = os.path.join(working_dir, \"peft_outputs_prompt\")\n", + "output_directory_sentences = os.path.join(working_dir, \"peft_outputs_sentences\")\n", + "\n", + "#Just creating the directoris if not exist.\n", + "if not os.path.exists(working_dir):\n", + " os.mkdir(working_dir)\n", + "if not os.path.exists(output_directory_prompt):\n", + " os.mkdir(output_directory_prompt)\n", + "if not os.path.exists(output_directory_sentences):\n", + " os.mkdir(output_directory_sentences)\n" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "OC5IhO9mVaTE" + }, + "source": [ + "在创建 TrainingArguments 时，我们需要指明包含模型的目录。" + ] + }, + { + "cell_type": "code", + "execution_count": 60, + "metadata": { + "id": "D4v4RSSeVaTE" + }, + "outputs": [], + "source": [ + "training_args_prompt = create_training_arguments(output_directory_prompt, 0.003, NUM_EPOCHS)\n", + "training_args_sentences = create_training_arguments(output_directory_sentences, 0.003, NUM_EPOCHS)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "application/vnd.databricks.v1+cell": { + "cellMetadata": {}, + "inputWidgets": {}, + "nuid": "c593deb6-5626-4fd9-89c2-2329e2f9b6e0", + "showTitle": false, + "title": "" + }, + "id": "GdMfjk5RVaTE" + }, + "source": [ + "## 训练\n", + "\n", + "我们将为每个要训练的模型创建一个 trainer 对象。" + ] + }, + { + "cell_type": "code", + "execution_count": 61, + "metadata": { + "id": "uVAfNdEIVaTE" + }, + "outputs": [], + "source": [ + "from transformers import Trainer, DataCollatorForLanguageModeling\n", + "def create_trainer(model, training_args, train_dataset):\n", + " trainer = Trainer(\n", + " model=model, # We pass in the PEFT version of the foundation model, bloomz-560M\n", + " args=training_args, #The args for the training.\n", + " train_dataset=train_dataset, #The dataset used to tyrain the model.\n", + " data_collator=DataCollatorForLanguageModeling(tokenizer, mlm=False) # mlm=False indicates not to use masked language modeling\n", + " )\n", + " return trainer\n" + ] + }, + { + "cell_type": "code", + "execution_count": 62, + "metadata": { + "application/vnd.databricks.v1+cell": { + "cellMetadata": {}, + "inputWidgets": {}, + "nuid": "32e43bcf-23b2-46aa-9cf0-455b83ef4f38", + "showTitle": false, + "title": "" + }, + "colab": { + "base_uri": "https://localhost:8080/", + "height": 127 + }, + "id": "1Sz9BeFZVaTF", + "outputId": "1b698470-209e-4001-fcbe-6fa8a2ac8707" + }, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + "
\n", + " \n", + " \n", + " [42/42 11:23, Epoch 6/6]\n", + "
\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
StepTraining Loss

" + ], + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + }, + { + "data": { + "text/plain": [ + "TrainOutput(global_step=42, training_loss=3.5800417945498513, metrics={'train_runtime': 703.2941, 'train_samples_per_second': 0.427, 'train_steps_per_second': 0.06, 'total_flos': 60957279240192.0, 'train_loss': 3.5800417945498513, 'epoch': 6.0})" + ] + }, + "execution_count": 62, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "#Training first model.\n", + "trainer_prompt = create_trainer(peft_model_prompt, training_args_prompt, train_sample_prompt)\n", + "trainer_prompt.train()" + ] + }, + { + "cell_type": "code", + "execution_count": 63, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 127 + }, + "id": "afTotMckVaTF", + "outputId": "15bed85d-17f5-4a49-d8d5-bae35e68d294" + }, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + "

\n", + " \n", + " \n", + " [24/24 03:29, Epoch 6/6]\n", + "
\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
StepTraining Loss

" + ], + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + }, + { + "data": { + "text/plain": [ + "TrainOutput(global_step=24, training_loss=4.4278310139973955, metrics={'train_runtime': 219.765, 'train_samples_per_second': 0.683, 'train_steps_per_second': 0.109, 'total_flos': 17825006936064.0, 'train_loss': 4.4278310139973955, 'epoch': 6.0})" + ] + }, + "execution_count": 63, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "#Training second model.\n", + "trainer_sentences = create_trainer(peft_model_sentences, training_args_sentences, train_sample_sentences)\n", + "trainer_sentences.train()" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "z2Zsww_2VaTF" + }, + "source": [ + "在不到 10 分钟的时间内（在 M1 Pro上的 CPU 时间），我们使用同一个基础模型训练了两个不同任务的模型。" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "application/vnd.databricks.v1+cell": { + "cellMetadata": {}, + "inputWidgets": {}, + "nuid": "5a6c8daf-8248-458a-9f6f-14865b4fbd2e", + "showTitle": false, + "title": "" + }, + "id": "s5k10HwoVaTG" + }, + "source": [ + "## 保存模型\n", + "我们将要保存模型。只要我们有创建它们的预训练模型在内存中，这些模型就可以使用了。" + ] + }, + { + "cell_type": "code", + "execution_count": 64, + "metadata": { + "application/vnd.databricks.v1+cell": { + "cellMetadata": {}, + "inputWidgets": {}, + "nuid": "409df5ce-e496-46d7-be2c-202a463cdc80", + "showTitle": false, + "title": "" + }, + "id": "E3dn3PeMVaTG" + }, + "outputs": [], + "source": [ + "trainer_prompt.model.save_pretrained(output_directory_prompt)\n", + "trainer_sentences.model.save_pretrained(output_directory_sentences)\n" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "application/vnd.databricks.v1+cell": { + "cellMetadata": {}, + "inputWidgets": {}, + "nuid": "fb14e3fd-bbf6-4d56-92c2-51bfe08de72a", + "showTitle": false, + "title": "" + }, + "id": "rkUKpDDWVaTG" + }, + "source": [ + "## 推理\n", + "你可以从之前保存的路径加载模型，并根据我们的输入要求模型生成文本！" + ] + }, + { + "cell_type": "code", + "execution_count": 65, + "metadata": { + "application/vnd.databricks.v1+cell": { + "cellMetadata": {}, + "inputWidgets": {}, + "nuid": "cc48af16-c117-4019-a31a-ce1c93cd21d4", + "showTitle": false, + "title": "" + }, + "id": "dlqXXN8oVaTG" + }, + "outputs": [], + "source": [ + "from peft import PeftModel\n", + "\n", + "loaded_model_prompt = PeftModel.from_pretrained(foundational_model,\n", + " output_directory_prompt,\n", + " #device_map='auto',\n", + " is_trainable=False)" + ] + }, + { + "cell_type": "code", + "execution_count": 66, + "metadata": { + "application/vnd.databricks.v1+cell": { + "cellMetadata": {}, + "inputWidgets": {}, + "nuid": "6b44524b-2ac5-4e74-81e6-c406d4414e42", + "showTitle": false, + "title": "" + }, + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "-4jd3zCGVaTG", + "outputId": "b55454f1-f1ed-444c-b107-698778406e6e" + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "['I want you to act as a motivational coach. You will be helping students learn how they can improve their performance in the classroom and at school.']\n" + ] + } + ], + "source": [ + "loaded_model_prompt_outputs = get_outputs(loaded_model_prompt, input_prompt)\n", + "print(tokenizer.batch_decode(loaded_model_prompt_outputs, skip_special_tokens=True))" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "SHbeFTXjVaTG" + }, + "source": [ + "如果我们比较两个答案，有些东西改变了。\n", + "* ***预训练模型：*** *我希望你扮演一个激励教练。不要害怕被挑战。*\n", + "* ***微调模型：*** *我希望你扮演一个激励教练。如果你感到焦虑，你可以使用这个方法。*\n", + "\n", + "我们必须记住，我们只训练了模型几分钟，但它们已经足够让我们得到更接近我们想要的结果的响应。\n" + ] + }, + { + "cell_type": "code", + "execution_count": 67, + "metadata": { + "id": "MuwAsq3uVaTG" + }, + "outputs": [], + "source": [ + "loaded_model_prompt.load_adapter(output_directory_sentences, adapter_name=\"quotes\")\n", + "loaded_model_prompt.set_adapter(\"quotes\")" + ] + }, + { + "cell_type": "code", + "execution_count": 70, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "IQm--PWSVaTH", + "outputId": "3e814a6a-a380-4f2c-f887-6852a9f51002" + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "['There are two nice things that should matter to you: the weather and your health.']\n" + ] + } + ], + "source": [ + "loaded_model_sentences_outputs = get_outputs(loaded_model_prompt, input_sentences)\n", + "print(tokenizer.batch_decode(loaded_model_sentences_outputs, skip_special_tokens=True))" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "UnR8y9gwVaTH" + }, + "source": [ + "对于第二个模型，我们得到了类似的结果。\n", + "* **预训练模型：** *有两件对你来说很重要的事情：你的产品的价格和质量。*\n", + "* **微调模型：** *有两件对你来说很重要的事情：天气和你的健康。*\n" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "B6TUjNtGVaTH" + }, + "source": [ + "# 结论\n", + "提示微调是一种惊人的技术，可以节省我们数小时的训练时间和大量的金钱。在这个 notebook 中，我们只用了几分钟就训练了两个模型，并且我们可以将两个模型都保存在内存中，为不同的客户提供服务。\n", + "\n", + "如果你想要尝试不同的组合和模型，这个 notebook 已经准备好使用 Bloom 家族中的另一个模型。\n", + "\n", + "你可以更改训练的轮数、虚拟 token 的数量和第三个单元格中的模型。然而，有许多配置需要更改。如果你正在寻找一个很好的练习，你可以用固定值替换虚拟 token 的随机初始化。\n", + "\n", + "*微调模型的响应可能在每次我们训练它们时都会有所不同。我粘贴了我的一次训练的结果，但实际结果可能会有所不同。*\n" + ] + } + ], + "metadata": { + "application/vnd.databricks.v1+notebook": { + "dashboards": [], + "language": "python", + "notebookMetadata": { + "pythonIndentUnit": 2 + }, + "notebookName": "LLM 02 - Prompt Tuning with PEFT", + "widgets": {} + }, + "colab": { + "machine_shape": "hm", + "provenance": [] + }, + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.11.4" + } + }, + "nbformat": 4, + "nbformat_minor": 0 +} From de5e3675adcc5b00e3c58dd6537bee711a91e2be Mon Sep 17 00:00:00 2001 From: innovation64 Date: Mon, 1 Apr 2024 19:38:59 +0800 Subject: [PATCH 13/31] docs: update stable diffusion interpolation in chinese version --- notebooks/zh-CN/stable_diffusion_interpolation.ipynb | 1 + 1 file changed, 1 insertion(+) create mode 100644 notebooks/zh-CN/stable_diffusion_interpolation.ipynb diff --git a/notebooks/zh-CN/stable_diffusion_interpolation.ipynb b/notebooks/zh-CN/stable_diffusion_interpolation.ipynb new file mode 100644 index 00000000..395bca5a --- /dev/null +++ b/notebooks/zh-CN/stable_diffusion_interpolation.ipynb @@ -0,0 +1 @@ +{"cells":[{"cell_type":"markdown","metadata":{"id":"UsrvK8CFDiNu"},"source":["## 使用 Stable Diffusion 进行图像插值\n","\n","\n","_作者: [Rustam Akimov](https://github.com/AkiRusProd)_\n","\n","这个 notebook 展示了如何使用 Stable Diffusion 来对图像进行插值。使用 Stable Diffusion 的图像插值是通过基于扩散的生成模型，从一张给定的图像平滑过渡到另一张图像，创建中间图像的过程。\n","\n","以下是一些使用 Stable Diffusion 进行图像插值的不同应用场景：\n","- 数据增强： Stable Diffusion 可以通过生成介于现有数据点之间的合成图像，来增强机器学习模型的训练数据。这可以提高机器学习模型的一般化和鲁棒性，特别是在图像生成、分类或对象检测等任务中。\n","- 产品设计和原型制作： Stable Diffusion 可以通过生成具有微妙差异的产品设计或原型变体，来辅助产品设计。这对于探索设计替代方案、进行用户研究或在投入物理原型之前可视化设计迭代非常有用。\n","- 媒体制作内容生成：在媒体制作中，如电影和视频编辑， Stable Diffusion 可以用来生成关键帧之间的中间帧，实现更平滑的过渡并增强视觉叙事。与手动逐帧编辑相比，这可以节省时间和资源。\n","\n","在图像插值的背景下， Stable Diffusion 模型通常用于在多维潜在空间中导航。每个维度代表模型学到的特定特征。通过在这个潜在空间中行走并在不同图像的潜在表示之间进行插值，模型能够生成一系列中间图像，这些图像显示了原始图像之间的平滑过渡。 Stable Diffusion 中有两种类型的潜在：提示潜在和图像潜在。\n","\n","潜在空间行走涉及沿着由两个或多个点（代表图像）定义的路径在潜在空间中移动。通过仔细选择这些点和它们之间的路径，可以控制生成图像的特征，如风格、内容和其他视觉方面。\n","\n","在这个 notebook 中，我们将探索使用 Stable Diffusion 进行图像插值的示例，并展示如何实现和利用潜在空间行走来创建图像之间的平滑过渡。我们将提供代码片段和可视化来展示这个过程的效果，从而更深入地理解生成模型如何以有意义的方式操纵和转化图像表示。"]},{"cell_type":"markdown","metadata":{"id":"XEhtH959DiOC"},"source":["首先，让我们安装所有需要的模块"]},{"cell_type":"code","execution_count":null,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"execution":{"iopub.execute_input":"2024-02-21T17:20:28.329767Z","iopub.status.busy":"2024-02-21T17:20:28.329050Z","iopub.status.idle":"2024-02-21T17:23:15.653382Z","shell.execute_reply":"2024-02-21T17:23:15.652310Z","shell.execute_reply.started":"2024-02-21T17:20:28.329734Z"},"id":"lbWtDpayDiOD","outputId":"b39791a6-6bdc-4f48-e016-5650c98072cf","trusted":true},"outputs":[],"source":["!pip install -q diffusers transformers xformers accelerate\n","!pip install -q numpy scipy ftfy Pillow"]},{"cell_type":"markdown","metadata":{"id":"pUUXab_IDiOE"},"source":["导入模块"]},{"cell_type":"code","execution_count":null,"metadata":{"colab":{"base_uri":"https://localhost:8080/","height":171,"referenced_widgets":["3537007206fd4d57ae492d29d90bd904","85bf2410c7d2440db76163fe1df4f4bb","c2bf5a15732a4898915b0ec3cb56df8c","bf573d9fcbac4701b31e464373fdbeb0","f1f352e6964f424f9e6a4557f6e3ff97","fa5231429aa1437983dc93dc597e698e","fe24820349a0456ca103e30024490c0e","35ffc6d955a44422a06e5c304fcaeddb","1e64cce9ffc94f23921f964288c2e26d","96d5032cdaa14cdeb110f8fc3b6614c1","4a466456e448417a8b3cc442cec49632"]},"execution":{"iopub.execute_input":"2024-02-21T17:23:55.606390Z","iopub.status.busy":"2024-02-21T17:23:55.606005Z","iopub.status.idle":"2024-02-21T17:24:12.144679Z","shell.execute_reply":"2024-02-21T17:24:12.143740Z","shell.execute_reply.started":"2024-02-21T17:23:55.606352Z"},"id":"gbnW1HiEDiOE","outputId":"a3b7adb5-f455-4c75-d626-6f2a6f86455b","trusted":true},"outputs":[],"source":["import torch\n","import numpy as np\n","import os\n","\n","import time\n","\n","from PIL import Image\n","from IPython import display as IPdisplay\n","from tqdm.auto import tqdm\n","\n","from diffusers import StableDiffusionPipeline\n","from diffusers import (\n"," DDIMScheduler,\n"," PNDMScheduler,\n"," LMSDiscreteScheduler,\n"," DPMSolverMultistepScheduler,\n"," EulerAncestralDiscreteScheduler,\n"," EulerDiscreteScheduler,\n",")\n","from transformers import logging\n","\n","logging.set_verbosity_error()"]},{"cell_type":"markdown","metadata":{"id":"loFaaWVUDiOF"},"source":["让我们查看一下 CUDA 是否可用\n","\n","\n"]},{"cell_type":"code","execution_count":null,"metadata":{"execution":{"iopub.execute_input":"2024-02-21T17:24:16.252373Z","iopub.status.busy":"2024-02-21T17:24:16.251653Z","iopub.status.idle":"2024-02-21T17:24:16.258088Z","shell.execute_reply":"2024-02-21T17:24:16.257085Z","shell.execute_reply.started":"2024-02-21T17:24:16.252340Z"},"id":"uGgmrhr-DiOF","trusted":true},"outputs":[],"source":["print(torch.cuda.is_available())\n","\n","device = torch.device(\"cuda\") if torch.cuda.is_available() else torch.device(\"cpu\")"]},{"cell_type":"markdown","metadata":{"id":"zMSGnuqmDiOF"},"source":["这些设置用于优化在启用 CUDA 的 GPU 上 PyTorch 模型的性能，尤其是在使用混合精度训练或推理时，这在速度和内存使用方面可能有益。\n","\n","来源：https://huggingface.co/docs/diffusers/optimization/fp16#memory-efficient-attention\n"]},{"cell_type":"code","execution_count":null,"metadata":{"execution":{"iopub.execute_input":"2024-02-21T17:24:18.661531Z","iopub.status.busy":"2024-02-21T17:24:18.661171Z","iopub.status.idle":"2024-02-21T17:24:18.666289Z","shell.execute_reply":"2024-02-21T17:24:18.665171Z","shell.execute_reply.started":"2024-02-21T17:24:18.661501Z"},"id":"JT02KQqNDiOF","trusted":true},"outputs":[],"source":["torch.backends.cudnn.benchmark = True\n","torch.backends.cuda.matmul.allow_tf32 = True"]},{"cell_type":"markdown","metadata":{"id":"_E5R20VtDiOF"},"source":["### 模型\n","\n","我们在这个项目中使用了 [`runwayml/stable-diffusion-v1-5`](https://huggingface.co/runwayml/stable-diffusion-v1-5) 模型和 [`LMSDiscreteScheduler`](https://huggingface.co/docs/diffusers/en/api/schedulers/lms_discrete) 调度器来生成图片。尽管这个模型已经不是最新的技术，但因为它的速度快、对内存的需求小，而且有很多社区成员基于这个版本改进的模型，所以还是挺受欢迎的。当然，如果你想尝试其他模型或调度器来生成图片，也是可以的。\n"]},{"cell_type":"code","execution_count":null,"metadata":{"colab":{"base_uri":"https://localhost:8080/","height":913,"referenced_widgets":["3538495833744e9eab9b25eade484603","8db4aafac51043598a9e4b8915f4f7c4","0e63faca29cb4ca3845e402d67592b8f","bf4b67c0a0034b1ab576066d366d934e","e09eaa48528a48629b19020aa65c5bdc","57d02fadf8fc4df3b1fbee18e45f9199","2f6e125affb54674b5f49beaf3612bee","f736f2fc44444ebf832fb2ead6ea0fd0","c41432dd085f41aeaa1e1d9cac4872e7","c577a158f88040c0979b319fb9fb89b0","f24c9ef7c3f04da783aaffd3e5d48de7","6401b1635d62478ba644075a32494384","72a5fcb23802406eb22e2758be01052c","0ba055eeeed04e91b684072740e07f0a","790424c8674d42d59c75ba2fd4021e3a","ff697b2e046a4a36a78ac362cae5c2a9","eb3e42c270b04086a2a04ae286684f6b","d9816365a87340dfa06fe9a37811a81b","309d89a221ab4010af0d85d3ae90a1c3","1cdaf6fe1392453288997f672af80a0d","5dc635e136f741608e73455581822408","670c96aa182f4defbfce4a9fd0b7f96b","92d3f7562ee04b789c44656f83d12528","bd7b9e9757d24c568432eea484271376","9c12a4d25771464b84b35804695fd50a","fa1d984ee1864975a4c543f1dcb3aa42","b89baafcd7e944b088bf0c9b1e839ba5","fb34d344590e43438906722b1ab958f0","ed6a871ff054410b8d018b3b97c75c60","909ad3aefa5b4c65931f300b3e9655dd","93159e8e4bf84bc9a35c4325dc6cc851","f2f9506fc02a4624a8a1c08f1f6abdb2","2f010760c1a146238f35af00568ccb11","20c46fc3077b4dcc855252c222002569","9b597fc5d5cf4fecabcfbc7a4cfa1ee9","17ceef0b617c4f52aa0bb5ec12113fcd","e20cd706fa304bc190e92ae7d27b5b38","9a3f552babd34cdeb8fed4ce1b1b33a7","f8db6aae6e3a468ba3805991e19e1f45","681cc61581d84298b4798b5c43818e31","45d877fafa5a4772ba5d62557843bb51","62d5802bdc7a49efbcb47889e29e924c","e4f21da1f63a4a819485cbe7e5ae306b","c7f341fe95ca426d9502594bd48d36f6","3214f06917264be88837314b26375a1c","00b88de69713463485206e4b7c8c3c04","f63bebf985a046ce9ba9d567db0b7ca4","a49fd3377dde41dab42677091dc7bd04","ed1dbb98c8d34dce8df70e4698a21911","95d1047a0a644262aa385987c9a331b4","c565ba8910c34e138c6cd10e9b06d673","5ffaa53c74d3413b8f3c3fe2fc5cc075","aedf99983c5043fe8d634b6e3b56e1ae","bf1489df7c98442caadd2417026bffdf","590b0de8d8f84d1ab9a243d88380c295","44a56b89efa649edb669abed3605e576","c26781e6cc214b6b866b8f11673e9c00","cddd14dde34c42f59ed71870e558246e","399fdb41f7fd4f1489c1bf4814b53907","f7f49c60f4ca41efa9a3b6b33c418f73","9ef6f8a591244419916d980d5883e03e","4d26d7dd13d148b3b6e06f10b589f2a8","8c690a7356af4547902b72aa6a20328d","1e4ad02a5afb431a96626230081054cc","2fd8069de9754a87ae04ec7b2c4b380a","da6373c3704d45869e2fafdf9772d3d1","8c9b21767c5d4741b717b82f1e4a0e03","a83f68f1b1d640298019af11c5198a7d","55a116f1fa634632a061a4ad8bb75ec3","58910b48c70a4b3dabf87b6a12004e70","2dc63f0fe271457f890fd2067631ad75","3fcfdbffff6149fe880b0702cf8162f3","eb96b8f49d6746ec8f46e65e59a3fad6","e03272733bd84e2e9edcb83391e1cfae","184d9e1fa8d241c386567300db4e2c8c","51ceac8abb23437f952e07d2daaf0dae","a06b4275e1444e72b921d631feea90ba","d7c9f6b399524bc596e84641687ba29a","1c8a25c7f70145df9722d3702fc6d2dd","ae44d548c5164e8fb5e85f1ab19da9ac","1325ea426c9747828601a9175a9b0248","97fe7d3ccc984694aa42a56ba930c64e","c1a5498ffe0f407397d2a8c77e35de84","5eefef112ef249cc97ecd63f19ea122d","defa4622267e49b8a54d6aceea082c39","59081d8cdf4e43228a20f3fe986926b2","c14c4f3d31a447b38d359acc6e29496d","f4f167561c2d452785b6e59c2ce61b28","4cc69a88629f41dc80c552b58d3f5eaf","6cc2e02ee7b74f3aa2f71bf0725190de","da714c09550f43849e5a2502b092403a","7370d528153e473980ac0e701c9e6825","0ba17f55cbf941feaa0b7e8959a94591","a903c8a2c22c48bdb70024665bc1cb0f","8c29df6a485946c090b39b49240427fd","b0bcd8201363473ea0e4ace230443446","d9f1220cc4f5440b97e35151ea76ed00","0eaebd8b9eb14feca8a11a8c3077b196","51061a6b42584e2bbac74da6a6f2a1da","e0216d9707cb4d10be57c4864521c376","2e72912929c14d9f8843bc028b75de77","feeaea5e7b524ed09bd08c73b7ed5e17","1ebb24b8e6f44608ae8c11747ef7c42d","c523aec40bb54986a4f40924c81fe5da","f23d6c684c3a416d8b43fc642653eec8","16e98aec73d048d693fd0e44df2220e7","495a6de427ce47eda18a1570fd8f5f9d","eac1b421700d492ba398d5ac609b5741","80509211c7dc47e8a673ed80fca6d8e1","e1f4e7fa8b1f4530b580f101a9fad304","4b14e789cc2446eda7a94a5b1259e738","865eb0b3254246948f532e2c6dd02bd1","7e678fd9b4284bf39ecd29de4d7624a3","adfbabeb20a3428c8fd6ec5b79830c34","6a012f24803646369ac97be41e5998f3","521ecdf554b84958ba4ee487c9621ffc","c6a66e5c516c4dbbbc1ca203c6a2d0db","66a43c50816c4e3fa88852c3d2c3b0c7","46df7a569a984f2a8d60b021e2366550","cffa386bd54e4baa8947d4d51c8e54a4","a5926a9d27b44a2d961f36d7fd36da15","35cf8424313d43dea56ca590edf70b26","5dc6caab4749403aab23ce95993f9ad0","84e32b4370a94fe498b3b01b6632daf3","16c7e883c9ec49329b68e61b276d5fac","5518095d648d4b2b95647c746d70c4d8","de350491cd2e48a19541f6e97b9f176e","cd7fa6f0b6844fd4a41b97cbe39e0c2f","683585ba77df463a9bcb2f8ce4794747","3b99c2eba3ab4ea2a44b23ec0916ce2d","b18d19fb59d8458c9e9048c1458ee95c","d5a23fbc6e634b02bf4f2540e9def457","1c3a77c578d644b09771446ed559c575","b08c232e469646f89a28f4371f0e7699","5e37d843836b42dfb62f728181ee4dfa","0755693c5b854000af081f2818162683","2e194e5ccbd349b093481e24148de89c","9d2ff155648146058d2d359e474159dc","0220a7b0d67a482a8ab7e9ac6372d380","034a1eb240694704a8052783583caefe","03fdfd9eb2e343df8af4ff2b06ac8eb1","393b21bd730f4a8b93d1653ebafaea09","0094122c8eaf47e6851a3449e2fb0086","7850ea0076da49639ca986a4885d7048","f04f25b18eaa43f6a2044dac7aba8372","6d9421a31914451cac51c71aee8b1ce4","a4b3ef66956d494887e796f87b4278f1","1638b8dbe07a4d249167a3d34ac9adc6","163ec8057136471bb1f460d657c4aa6c","ff1655111fd04c4785d7e5ec3747629c","566d76e028b643e18729621e82531939","2642af1a55cf452a93e528fb25f1c8cb","a428e985410341e0ba04359af465681e","bfbec38740f8413d93469274aeccbf23","70cf0bb1ada946dd9717fc2a493b7805","8caab95721354f08999bea8dc6105b4e","221facd121a14faf9d664f644935b0ae","bac1fcd4cf1847ed89bc5e01ae435e24","bc4ed6312ae44ba7a9e21d43d7edd48a","fd4eebbe68204eaa802186836c372b93","6a7f80bb3e534eb2a48d3d29c9ac3988","e431b2a589524545a5ccbb79d2c7bab9","90dccdfd4085472f8f9ba0535d85d327","d1d564e827cf4a71af9aa87b9d5696c8","1ad180dc107946b9a1b554a4b98ee514","88c6c5bcc44d46089aa3efaa7fb9e452","e0280b4f0172481ea7664bfb96d1bb1c","3cd43805ef564f6696905a2465ea4467","ab7f8b09b1d8452995d66e6f0df83faa","3959b0871ea840839a383c895cfbe916","60aa6e24133c4e67957bf953e5b10f4d","5ce301bccee049cf9664800d63e2e2eb","da25ec24a6b44ce9a51bcc2d440d0258","41ea6163d1b04f8f89cd1f9ec9e72847","31107fa83a974eca83fa968ae4eae909","36a0330c9e1d48a49a739feaef34ddc0"]},"execution":{"iopub.execute_input":"2024-02-21T17:24:22.143953Z","iopub.status.busy":"2024-02-21T17:24:22.143589Z","iopub.status.idle":"2024-02-21T17:25:54.037631Z","shell.execute_reply":"2024-02-21T17:25:54.036655Z","shell.execute_reply.started":"2024-02-21T17:24:22.143923Z"},"id":"ppKz1aLSDiOF","outputId":"e359b27d-6381-4bef-8a8c-bcc9576f7fe3","trusted":true},"outputs":[],"source":["model_name_or_path = \"runwayml/stable-diffusion-v1-5\"\n","\n","scheduler = LMSDiscreteScheduler(beta_start=0.00085, beta_end=0.012, beta_schedule=\"scaled_linear\", num_train_timesteps=1000)\n","\n","\n","pipe = StableDiffusionPipeline.from_pretrained(\n"," model_name_or_path,\n"," scheduler=scheduler,\n"," torch_dtype=torch.float32,\n",").to(device)\n","\n","# Disable image generation progress bar, we'll display our own\n","pipe.set_progress_bar_config(disable=True)"]},{"cell_type":"markdown","metadata":{"id":"5oBmcxe9DiOG"},"source":["这些方法旨在减少 GPU 消耗的内存。如果你有足够的显存，可以跳过这个步骤。\n","\n","更详细的信息可以在这里找到：https://huggingface.co/docs/diffusers/en/optimization/opt_overview \n","特别是，关于以下方法的信息可以在这里找到：https://huggingface.co/docs/diffusers/optimization/memory\n"]},{"cell_type":"code","execution_count":null,"metadata":{"execution":{"iopub.execute_input":"2024-02-21T17:25:54.040235Z","iopub.status.busy":"2024-02-21T17:25:54.039388Z","iopub.status.idle":"2024-02-21T17:26:00.115879Z","shell.execute_reply":"2024-02-21T17:26:00.115042Z","shell.execute_reply.started":"2024-02-21T17:25:54.040193Z"},"id":"1i7WuQV1DiOG","trusted":true},"outputs":[],"source":["# Offloading the weights to the CPU and only loading them on the GPU can reduce memory consumption to less than 3GB.\n","pipe.enable_model_cpu_offload()\n","\n","# Tighter ordering of memory tensors.\n","pipe.unet.to(memory_format=torch.channels_last)\n","\n","# Decoding large batches of images with limited VRAM or batches with 32 images or more by decoding the batches of latents one image at a time.\n","pipe.enable_vae_slicing()\n","\n","# Splitting the image into overlapping tiles, decoding the tiles, and then blending the outputs together to compose the final image. \n","pipe.enable_vae_tiling()\n","\n","# Using Flash Attention; If you have PyTorch >= 2.0 installed, you should not expect a speed-up for inference when enabling xformers.\n","pipe.enable_xformers_memory_efficient_attention()\n"]},{"cell_type":"markdown","metadata":{"id":"k45VkXF7DiOG"},"source":["`display_images` 函数将图像数组的列表转换成 GIF，保存到指定路径，并返回 GIF 对象以供显示。它使用当前时间来命名 GIF 文件，并通过打印出来处理任何错误。\n"]},{"cell_type":"code","execution_count":null,"metadata":{"execution":{"iopub.execute_input":"2024-02-21T17:30:01.535670Z","iopub.status.busy":"2024-02-21T17:30:01.535281Z","iopub.status.idle":"2024-02-21T17:30:01.542928Z","shell.execute_reply":"2024-02-21T17:30:01.541894Z","shell.execute_reply.started":"2024-02-21T17:30:01.535637Z"},"id":"n5cKlS0CDiOG","trusted":true},"outputs":[],"source":["def display_images(images, save_path):\n"," try:\n"," # Convert each image in the 'images' list from an array to an Image object.\n"," images = [\n"," Image.fromarray(np.array(image[0], dtype=np.uint8)) for image in images\n"," ]\n","\n"," # Generate a file name based on the current time, replacing colons with hyphens\n"," # to ensure the filename is valid for file systems that don't allow colons.\n"," filename = (\n"," time.strftime(\"%H:%M:%S\", time.localtime())\n"," .replace(\":\", \"-\")\n"," )\n"," # Save the first image in the list as a GIF file at the 'save_path' location.\n"," # The rest of the images in the list are added as subsequent frames to the GIF.\n"," # The GIF will play each frame for 100 milliseconds and will loop indefinitely.\n"," images[0].save(\n"," f\"{save_path}/{filename}.gif\",\n"," save_all=True,\n"," append_images=images[1:],\n"," duration=100,\n"," loop=0,\n"," )\n"," except Exception as e:\n"," # If there is an error during the process, print the exception message.\n"," print(e)\n","\n"," # Return the saved GIF as an IPython display object so it can be displayed in a notebook.\n"," return IPdisplay.Image(f\"{save_path}/{filename}.gif\")"]},{"cell_type":"markdown","metadata":{"id":"L13Q7INNDiOG"},"source":["### 生成参数\n","\n","\n","* `seed`: 这个变量用于设置一个特定的随机种子，以便复现结果。\n","* `generator`: 如果提供了种子，这将设置为一个 PyTorch 随机数生成器对象，否则为 None。它确保使用它的操作具有可复现的结果。\n","* `guidance_scale`: 这个参数控制模型在文本到图像生成任务中遵循提示的程度，值越高，对提示的遵循越强。\n","* `num_inference_steps`: 这指定了模型生成图像所需的步骤数。更多的步骤可以导致生成更高质量的图像，但生成时间会更长。\n","* `num_interpolation_steps`: 这决定了在潜在空间中两点之间插值时使用的步骤数，影响生成动画中过渡的平滑度。\n","* `height`: 生成图像的高度，以像素为单位。\n","* `width`: 生成图像的宽度，以像素为单位。\n","* `save_path`: 生成的 GIF 将保存的文件系统路径。 "]},{"cell_type":"code","execution_count":null,"metadata":{"execution":{"iopub.execute_input":"2024-02-21T17:30:04.013629Z","iopub.status.busy":"2024-02-21T17:30:04.012881Z","iopub.status.idle":"2024-02-21T17:30:04.019551Z","shell.execute_reply":"2024-02-21T17:30:04.018612Z","shell.execute_reply.started":"2024-02-21T17:30:04.013596Z"},"id":"R_B-h2j4DiOG","trusted":true},"outputs":[],"source":["# The seed is set to \"None\", because we want different results each time we run the generation.\n","seed = None\n","\n","if seed is not None:\n"," generator = torch.manual_seed(seed)\n","else:\n"," generator = None\n","\n","# The guidance scale is set to its normal range (7 - 10).\n","guidance_scale = 8\n","\n","# The number of inference steps was chosen empirically to generate an acceptable picture within an acceptable time.\n","num_inference_steps = 15\n","\n","# The higher you set this value, the smoother the interpolations will be. However, the generation time will increase. This value was chosen empirically.\n","num_interpolation_steps = 30\n","\n","# I would not recommend less than 512 on either dimension. This is because this model was trained on 512x512 image resolution.\n","height = 512 \n","width = 512\n","\n","# The path where the generated GIFs will be saved\n","save_path = \"/output\"\n","\n","if not os.path.exists(save_path):\n"," os.makedirs(save_path)\n"]},{"cell_type":"markdown","metadata":{"id":"Nm4BHESjDiOG"},"source":["### 示例 1：提示插值\n","\n","在这个例子中，我们将通过在积极提示和消极提示之间进行插值，来探索这两个提示定义的概念之间的空间。这样做可以让我们看到一系列逐渐融合这两种提示特征的图像。具体来说，我们会修改原始提示的嵌入向量，逐渐添加一些小的变化，从而创建出一系列新的提示嵌入。这些新的嵌入将被用来生成图像，这些图像会平滑地从一种提示的状态过渡到另一种。\n","\n"]},{"cell_type":"markdown","metadata":{},"source":["![Example 1](https://huggingface.co/datasets/huggingface/cookbook-images/resolve/main/sd_interpolation_1.gif)"]},{"cell_type":"markdown","metadata":{},"source":["首先，我们需要对积极和消极的文本提示进行标记化并获得它们的嵌入。积极提示引导图像生成朝向期望的特征，而消极提示则使其远离不希望出现的特征。"]},{"cell_type":"code","execution_count":null,"metadata":{"colab":{"base_uri":"https://localhost:8080/","height":49,"referenced_widgets":["640b691e30b844ec943995160216e28b","4262f099aab24cfd9b3790864e0e1d63","a2194da8bc254658b9db17e19dbe418b","f0e13bd4abca444592850390651272a4","866b048905164e31a5011bdc0fcf5180","867f3d155da9469ab9820923e40e78e5","7d7e58bafe2c4ff6a44275c3a2ea9826","cc9fdbf01697491f856a33e4b70a7a78","6826628eb6214b57bbe56e3eb80322b3","757864720c4041c6a24f8aa8f1630e69","92e5153340a74fc9895d4f87b68e3cad"]},"execution":{"iopub.execute_input":"2024-02-21T17:40:07.727796Z","iopub.status.busy":"2024-02-21T17:40:07.727407Z","iopub.status.idle":"2024-02-21T17:43:50.624205Z","shell.execute_reply":"2024-02-21T17:43:50.622571Z","shell.execute_reply.started":"2024-02-21T17:40:07.727768Z"},"id":"YVNrz60MDiOH","outputId":"428cf53c-ca0d-49e6-f2cd-41ed292b5117","trusted":true},"outputs":[],"source":["# The text prompt that describes the desired output image.\n","prompt = \"Epic shot of Sweden, ultra detailed lake with an ren dear, nostalgic vintage, ultra cozy and inviting, wonderful light atmosphere, fairy, little photorealistic, digital painting, sharp focus, ultra cozy and inviting, wish to be there. very detailed, arty, should rank high on youtube for a dream trip.\"\n","# A negative prompt that can be used to steer the generation away from certain features; here, it is empty.\n","negative_prompt = \"poorly drawn,cartoon, 2d, disfigured, bad art, deformed, poorly drawn, extra limbs, close up, b&w, weird colors, blurry\"\n","\n","# The step size for the interpolation in the latent space.\n","step_size = 0.001\n","\n","# Tokenizing and encoding the prompt into embeddings.\n","prompt_tokens = pipe.tokenizer(\n"," prompt,\n"," padding=\"max_length\",\n"," max_length=pipe.tokenizer.model_max_length,\n"," truncation=True,\n"," return_tensors=\"pt\",\n",")\n","prompt_embeds = pipe.text_encoder(prompt_tokens.input_ids.to(device))[0]\n","\n","\n","# Tokenizing and encoding the negative prompt into embeddings.\n","if negative_prompt is None:\n"," negative_prompt = [\"\"]\n","\n","negative_prompt_tokens = pipe.tokenizer(\n"," negative_prompt,\n"," padding=\"max_length\",\n"," max_length=pipe.tokenizer.model_max_length,\n"," truncation=True,\n"," return_tensors=\"pt\",\n",")\n","negative_prompt_embeds = pipe.text_encoder(negative_prompt_tokens.input_ids.to(device))[0]"]},{"cell_type":"markdown","metadata":{},"source":["现在让我们来看看生成随机初始向量的代码部分，该向量使用正态分布生成，其结构符合扩散模型（UNet）预期的维度。这允许通过可选地使用随机数生成器来复现结果。创建了初始向量后，代码通过每次迭代增量地添加一个小步长，在两个嵌入（积极和消极提示）之间进行一系列插值。结果存储在一个名为 \"walked_embeddings\" 的列表中。"]},{"cell_type":"code","execution_count":null,"metadata":{},"outputs":[],"source":["# Generating initial latent vectors from a random normal distribution, with the option to use a generator for reproducibility.\n","latents = torch.randn(\n"," (1, pipe.unet.config.in_channels, height // 8, width // 8),\n"," generator=generator,\n",")\n","\n","walked_embeddings = []\n","\n","# Interpolating between embeddings for the given number of interpolation steps.\n","for i in range(num_interpolation_steps):\n"," walked_embeddings.append(\n"," [prompt_embeds + step_size * i, negative_prompt_embeds + step_size * i]\n"," )"]},{"cell_type":"markdown","metadata":{},"source":["最后，让我们根据插值嵌入生成一系列图像，然后显示这些图像。我们将遍历一个嵌入数组，使用每个嵌入生成具有指定特征（如高度、宽度以及与图像生成相关的其他参数）的图像。然后我们将这些图像收集到一个列表中。一旦生成完成，我们将调用 `display_image` 函数，以在给定的保存路径上将这些图像保存并显示为 GIF。\n"]},{"cell_type":"code","execution_count":null,"metadata":{},"outputs":[],"source":["# Generating images using the interpolated embeddings.\n","images = []\n","for latent in tqdm(walked_embeddings):\n"," images.append(\n"," pipe(\n"," height=height,\n"," width=width,\n"," num_images_per_prompt=1,\n"," prompt_embeds=latent[0],\n"," negative_prompt_embeds=latent[1],\n"," num_inference_steps=num_inference_steps,\n"," guidance_scale=guidance_scale,\n"," generator=generator,\n"," latents=latents,\n"," ).images\n"," )\n","\n","# Display of saved generated images.\n","display_images(images, save_path)"]},{"cell_type":"markdown","metadata":{"id":"uZQWop9nDiOH"},"source":["### 示例 2：针对单个提示的扩散潜在插值\n","与第一个示例不同，在这个示例中，我们是在扩散模型本身的两个嵌入之间执行插值，而不是在提示之间。请注意，在这种情况下，我们使用 slerp 函数进行插值。然而，这并不妨碍我们在一个嵌入中添加一个常数。"]},{"cell_type":"markdown","metadata":{},"source":["![Example 2](https://huggingface.co/datasets/huggingface/cookbook-images/resolve/main/sd_interpolation_2.gif)"]},{"cell_type":"markdown","metadata":{"id":"CiW6SlhXDiOH"},"source":["下面呈现的函数代表球面线性插值（Spherical Linear Interpolation）。这是一种在球面上进行插值的方法。这个函数在计算机图形学中常用于平滑地动画旋转，并且也可以用于机器学习中高维数据点之间的插值，比如生成模型中使用的潜在向量。\n","\n","该函数的来源是 Andrej Karpathy 的 gist：https://gist.github.com/karpathy/00103b0037c5aaea32fe1da1af553355。 \n","关于这种方法更详细的解释可以在：https://en.wikipedia.org/wiki/Slerp 找到。\n"]},{"cell_type":"code","execution_count":1,"metadata":{"id":"grgP7UNpDiOH"},"outputs":[],"source":["def slerp(v0, v1, num, t0=0, t1=1):\n"," v0 = v0.detach().cpu().numpy()\n"," v1 = v1.detach().cpu().numpy()\n","\n"," def interpolation(t, v0, v1, DOT_THRESHOLD=0.9995):\n"," \"\"\"helper function to spherically interpolate two arrays v1 v2\"\"\"\n"," dot = np.sum(v0 * v1 / (np.linalg.norm(v0) * np.linalg.norm(v1)))\n"," if np.abs(dot) > DOT_THRESHOLD:\n"," v2 = (1 - t) * v0 + t * v1\n"," else:\n"," theta_0 = np.arccos(dot)\n"," sin_theta_0 = np.sin(theta_0)\n"," theta_t = theta_0 * t\n"," sin_theta_t = np.sin(theta_t)\n"," s0 = np.sin(theta_0 - theta_t) / sin_theta_0\n"," s1 = sin_theta_t / sin_theta_0\n"," v2 = s0 * v0 + s1 * v1\n"," return v2\n","\n"," t = np.linspace(t0, t1, num)\n","\n"," v3 = torch.tensor(np.array([interpolation(t[i], v0, v1) for i in range(num)]))\n","\n"," return v3"]},{"cell_type":"code","execution_count":null,"metadata":{"colab":{"base_uri":"https://localhost:8080/","height":561,"referenced_widgets":["d7eb412b880c490c95f1d2baeaf2e6af","3e7caf19c664461e9505cbfcb708ceba","21e5a4fd28ac47a190d0f41808dd75a1","7f4317fe6eca4fc5be524453e55103bd","4ab2093e02704b748298a6d34e807847","69ffdc3c18f5484cad6945a77b024529","2c5d8801da6f4d88be3801254c3e764b","a76a9fce4af34c639327e5a0f4f4e692","e00e5537ae9a43d5956c2c770599edde","6fa7c3c07e734867ac5676b09b6804b3","ed410d69e8e94af7be0d104c5c29a2c9"]},"id":"aIU-nxTcDiOH","outputId":"1f762594-d89d-4bd3-c909-3d4850293b71"},"outputs":[],"source":["# The text prompt that describes the desired output image.\n","prompt = \"Sci-fi digital painting of an alien landscape with otherworldly plants, strange creatures, and distant planets.\"\n","# A negative prompt that can be used to steer the generation away from certain features.\n","negative_prompt = \"poorly drawn,cartoon, 3d, disfigured, bad art, deformed, poorly drawn, extra limbs, close up, b&w, weird colors, blurry\"\n","\n","# Generating initial latent vectors from a random normal distribution. In this example two latent vectors are generated, which will serve as start and end points for the interpolation.\n","# These vectors are shaped to fit the input requirements of the diffusion model's U-Net architecture.\n","latents = torch.randn(\n"," (2, pipe.unet.config.in_channels, height // 8, width // 8),\n"," generator=generator,\n",")\n","\n","# Getting our latent embeddings\n","interpolated_latents = slerp(latents[0], latents[1], num_interpolation_steps)\n","\n","# Generating images using the interpolated embeddings.\n","images = []\n","for latent_vector in tqdm(interpolated_latents):\n"," images.append(\n"," pipe(\n"," prompt,\n"," height=height,\n"," width=width,\n"," negative_prompt=negative_prompt,\n"," num_images_per_prompt=1,\n"," num_inference_steps=num_inference_steps,\n"," guidance_scale=guidance_scale,\n"," generator=generator,\n"," latents=latent_vector[None, ...],\n"," ).images\n"," )\n","\n","# Display of saved generated images.\n","display_images(images, save_path)"]},{"cell_type":"markdown","metadata":{"id":"sTFrAlwrDiOI"},"source":["### 示例 3：多个提示之间的插值\n","\n","与第一个示例中我们从一个提示移动开来不同，在这个示例中，我们将对任意数量的提示进行插值。为此，我们将取连续的提示对，并创建它们之间的平滑过渡。然后，我们将这些连续对的插值组合起来，并指示模型基于它们生成图像。我们将使用第二个示例中的 slerp 函数进行插值。\n"]},{"cell_type":"markdown","metadata":{},"source":["![Example 3](https://huggingface.co/datasets/huggingface/cookbook-images/resolve/main/sd_interpolation_3.gif)"]},{"cell_type":"markdown","metadata":{},"source":["再次，让我们对多个积极和消极的文本提示进行标记化并获得它们的嵌入。"]},{"cell_type":"code","execution_count":null,"metadata":{},"outputs":[],"source":["# Text prompts that describes the desired output image.\n","prompts = [\n"," \"A cute dog in a beautiful field of lavander colorful flowers everywhere, perfect lighting, leica summicron 35mm f2.0, kodak portra 400, film grain\",\n"," \"A cute cat in a beautiful field of lavander colorful flowers everywhere, perfect lighting, leica summicron 35mm f2.0, kodak portra 400, film grain\",\n","]\n","# Negative prompts that can be used to steer the generation away from certain features.\n","negative_prompts = [\n"," \"poorly drawn,cartoon, 2d, sketch, cartoon, drawing, anime, disfigured, bad art, deformed, poorly drawn, extra limbs, close up, b&w, weird colors, blurry\",\n"," \"poorly drawn,cartoon, 2d, sketch, cartoon, drawing, anime, disfigured, bad art, deformed, poorly drawn, extra limbs, close up, b&w, weird colors, blurry\",\n","]\n","\n","# NOTE: The number of prompts must match the number of negative prompts\n","\n","batch_size = len(prompts)\n","\n","# Tokenizing and encoding prompts into embeddings.\n","prompts_tokens = pipe.tokenizer(\n"," prompts,\n"," padding=\"max_length\",\n"," max_length=pipe.tokenizer.model_max_length,\n"," truncation=True,\n"," return_tensors=\"pt\",\n",")\n","prompts_embeds = pipe.text_encoder(\n"," prompts_tokens.input_ids.to(device)\n",")[0]\n","\n","# Tokenizing and encoding negative prompts into embeddings.\n","if negative_prompts is None:\n"," negative_prompts = [\"\"] * batch_size\n","\n","negative_prompts_tokens = pipe.tokenizer(\n"," negative_prompts,\n"," padding=\"max_length\",\n"," max_length=pipe.tokenizer.model_max_length,\n"," truncation=True,\n"," return_tensors=\"pt\",\n",")\n","negative_prompts_embeds = pipe.text_encoder(\n"," negative_prompts_tokens.input_ids.to(device)\n",")[0]"]},{"cell_type":"markdown","metadata":{},"source":["如前所述，我们将使用 `slerp` 函数对连续的提示对创建平滑过渡。"]},{"cell_type":"code","execution_count":null,"metadata":{"colab":{"base_uri":"https://localhost:8080/","height":561,"referenced_widgets":["9a87e8d407f44ee59a70e511a6274131","d49efe4cff2a43288d2140a55c17c4cc","79fed3974dd8466e8237c7431f71a084","6c9624bf3faf4890bc1c83c52e33e508","f23c3781df5a497baa56f273a5f467a5","6956b01055a34613994672a6bb93994d","1f16ae9b03604ccd93cd1e2f153afe64","aa4bef85913f43c49b708c850301546d","e92a9d9050e34b47b766a31935ffbcda","94fa436d65894af494b2526746fe7324","ffe65c700f0142df9b289a0e8b58ec65"]},"id":"DfUbS8w5DiOI","outputId":"fb663c02-73e2-421a-8d07-b0b43cd548a7"},"outputs":[],"source":["# Generating initial U-Net latent vectors from a random normal distribution.\n","latents = torch.randn(\n"," (1, pipe.unet.config.in_channels, height // 8, width // 8),\n"," generator=generator,\n",")\n","\n","# Interpolating between embeddings pairs for the given number of interpolation steps.\n","interpolated_prompt_embeds = []\n","interpolated_negative_prompts_embeds = []\n","for i in range(batch_size - 1):\n"," interpolated_prompt_embeds.append(\n"," slerp(\n"," prompts_embeds[i],\n"," prompts_embeds[i + 1],\n"," num_interpolation_steps\n"," )\n"," )\n"," interpolated_negative_prompts_embeds.append(\n"," slerp(\n"," negative_prompts_embeds[i],\n"," negative_prompts_embeds[i + 1],\n"," num_interpolation_steps,\n"," )\n"," )\n","\n","interpolated_prompt_embeds = torch.cat(\n"," interpolated_prompt_embeds, dim=0\n",").to(device)\n","\n","interpolated_negative_prompts_embeds = torch.cat(\n"," interpolated_negative_prompts_embeds, dim=0\n",").to(device)"]},{"cell_type":"markdown","metadata":{},"source":["最后，我们需要根据嵌入生成图像。"]},{"cell_type":"code","execution_count":null,"metadata":{},"outputs":[],"source":["# Generating images using the interpolated embeddings.\n","images = []\n","for prompt_embeds, negative_prompt_embeds in tqdm(\n"," zip(interpolated_prompt_embeds, interpolated_negative_prompts_embeds),\n"," total=len(interpolated_prompt_embeds),\n","):\n"," images.append(\n"," pipe(\n"," height=height,\n"," width=width,\n"," num_images_per_prompt=1,\n"," prompt_embeds=prompt_embeds[None, ...],\n"," negative_prompt_embeds=negative_prompt_embeds[None, ...],\n"," num_inference_steps=num_inference_steps,\n"," guidance_scale=guidance_scale,\n"," generator=generator,\n"," latents=latents,\n"," ).images\n"," )\n","\n","# Display of saved generated images.\n","display_images(images, save_path)"]},{"cell_type":"markdown","metadata":{"id":"oQqANSP2DiOI"},"source":["### 示例 4：针对单个提示在扩散潜在空间中的循环行走\n","\n","这个示例来自：https://keras.io/examples/generative/random_walks_with_stable_diffusion/\n","\n","假设我们有两个噪声成分，我们称之为 x 和 y。我们从 0 移动到 2π，在每一步中，我们将 x 的余弦和 y 的正弦加到结果中。使用这种方法，在我们的移动结束时，我们得到了与开始时相同的噪声值。这意味着向量最终转变成它们自己，结束了我们的移动。\n","\n","\n"]},{"cell_type":"markdown","metadata":{},"source":["![Example 4](https://huggingface.co/datasets/huggingface/cookbook-images/resolve/main/sd_interpolation_4.gif)"]},{"cell_type":"code","execution_count":null,"metadata":{"colab":{"base_uri":"https://localhost:8080/","height":561,"referenced_widgets":["eba9a67d3f704bed8f501780e35273cb","f6b5c1f44a54406c84db14875e4c85b0","1f3ce7042b974edbb033b4cd8d13cc08","6de631e7a72e4b74b740095e8c251ca8","00c30d57328148c88b4258f4841bbdd0","a4a86f212e8a4dffb0240936475837f7","89ce18ce98494ccc803adbf87f6051e5","6781679193314617a341ef891ba3df45","161a3b1a75e0446ca9120f5e1eea38e9","7294524debe544c19f8c76a7b3cf0e32","f6fb32e142d140d5ad6b357731c4d382"]},"id":"ac-68CTWDiOJ","outputId":"3eced894-bd22-443a-96e9-dedb67a40ad8"},"outputs":[],"source":["# The text prompt that describes the desired output image.\n","prompt = \"Beautiful sea sunset, warm light, Aivazovsky style\"\n","# A negative prompt that can be used to steer the generation away from certain features\n","negative_prompt = \"picture frames\"\n","\n","# Generating initial latent vectors from a random normal distribution to create a loop interpolation between them.\n","latents = torch.randn(\n"," (2, 1, pipe.unet.config.in_channels, height // 8, width // 8),\n"," generator=generator,\n",")\n","\n","\n","# Calculation of looped embeddings\n","walk_noise_x = latents[0].to(device)\n","walk_noise_y = latents[1].to(device)\n","\n","# Walking on a trigonometric circle\n","walk_scale_x = torch.cos(torch.linspace(0, 2, num_interpolation_steps) * np.pi).to(\n"," device\n",")\n","walk_scale_y = torch.sin(torch.linspace(0, 2, num_interpolation_steps) * np.pi).to(\n"," device\n",")\n","\n","# Applying interpolation to noise\n","noise_x = torch.tensordot(walk_scale_x, walk_noise_x, dims=0)\n","noise_y = torch.tensordot(walk_scale_y, walk_noise_y, dims=0)\n","\n","circular_latents = noise_x + noise_y\n","\n","# Generating images using the interpolated embeddings.\n","images = []\n","for latent_vector in tqdm(circular_latents):\n"," images.append(\n"," pipe(\n"," prompt,\n"," height=height,\n"," width=width,\n"," negative_prompt=negative_prompt,\n"," num_images_per_prompt=1,\n"," num_inference_steps=num_inference_steps,\n"," guidance_scale=guidance_scale,\n"," generator=generator,\n"," latents=latent_vector,\n"," ).images\n"," )\n","\n","# Display of saved generated images.\n","display_images(images, save_path)"]},{"cell_type":"markdown","metadata":{"id":"QQnbnOokDiOJ"},"source":["## 下一步\n","接下来，您可以探索各种参数，如指导比例（guidance scale）、种子（seed）和插值步骤数（number of interpolation steps），以观察它们如何影响生成的图像。此外，尝试使用不同的提示和调度器来进一步优化你的结果。另一个有价值的步骤是实施线性插值（`linspace`），而不是球面线性插值（`slerp`），并比较结果，以更深入地了解插值过程。"]}],"metadata":{"accelerator":"GPU","colab":{"gpuType":"T4","provenance":[]},"kaggle":{"accelerator":"gpu","dataSources":[],"dockerImageVersionId":30648,"isGpuEnabled":true,"isInternetEnabled":true,"language":"python","sourceType":"notebook"},"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"codemirror_mode":{"name":"ipython","version":3},"file_extension":".py","mimetype":"text/x-python","name":"python","nbconvert_exporter":"python","pygments_lexer":"ipython3","version":"3.10.11"},"widgets":{"application/vnd.jupyter.widget-state+json":{"0094122c8eaf47e6851a3449e2fb0086":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"00b88de69713463485206e4b7c8c3c04":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_95d1047a0a644262aa385987c9a331b4","placeholder":"​","style":"IPY_MODEL_c565ba8910c34e138c6cd10e9b06d673","value":"safety_checker/config.json: 100%"}},"00c30d57328148c88b4258f4841bbdd0":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"0220a7b0d67a482a8ab7e9ac6372d380":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"034a1eb240694704a8052783583caefe":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"03fdfd9eb2e343df8af4ff2b06ac8eb1":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"ProgressStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"0755693c5b854000af081f2818162683":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_393b21bd730f4a8b93d1653ebafaea09","placeholder":"​","style":"IPY_MODEL_0094122c8eaf47e6851a3449e2fb0086","value":" 525k/525k [00:02<00:00, 233kB/s]"}},"0ba055eeeed04e91b684072740e07f0a":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"FloatProgressModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_309d89a221ab4010af0d85d3ae90a1c3","max":14,"min":0,"orientation":"horizontal","style":"IPY_MODEL_1cdaf6fe1392453288997f672af80a0d","value":14}},"0ba17f55cbf941feaa0b7e8959a94591":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"0e63faca29cb4ca3845e402d67592b8f":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"FloatProgressModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_f736f2fc44444ebf832fb2ead6ea0fd0","max":541,"min":0,"orientation":"horizontal","style":"IPY_MODEL_c41432dd085f41aeaa1e1d9cac4872e7","value":541}},"0eaebd8b9eb14feca8a11a8c3077b196":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"1325ea426c9747828601a9175a9b0248":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_c14c4f3d31a447b38d359acc6e29496d","placeholder":"​","style":"IPY_MODEL_f4f167561c2d452785b6e59c2ce61b28","value":" 806/806 [00:00<00:00, 18.5kB/s]"}},"161a3b1a75e0446ca9120f5e1eea38e9":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"ProgressStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"1638b8dbe07a4d249167a3d34ac9adc6":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"163ec8057136471bb1f460d657c4aa6c":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"16c7e883c9ec49329b68e61b276d5fac":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_b18d19fb59d8458c9e9048c1458ee95c","placeholder":"​","style":"IPY_MODEL_d5a23fbc6e634b02bf4f2540e9def457","value":" 492M/492M [00:14<00:00, 20.5MB/s]"}},"16e98aec73d048d693fd0e44df2220e7":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"17ceef0b617c4f52aa0bb5ec12113fcd":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"FloatProgressModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_45d877fafa5a4772ba5d62557843bb51","max":472,"min":0,"orientation":"horizontal","style":"IPY_MODEL_62d5802bdc7a49efbcb47889e29e924c","value":472}},"184d9e1fa8d241c386567300db4e2c8c":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"ProgressStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"1ad180dc107946b9a1b554a4b98ee514":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"1c3a77c578d644b09771446ed559c575":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HBoxModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_b08c232e469646f89a28f4371f0e7699","IPY_MODEL_5e37d843836b42dfb62f728181ee4dfa","IPY_MODEL_0755693c5b854000af081f2818162683"],"layout":"IPY_MODEL_2e194e5ccbd349b093481e24148de89c"}},"1c8a25c7f70145df9722d3702fc6d2dd":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_c1a5498ffe0f407397d2a8c77e35de84","placeholder":"​","style":"IPY_MODEL_5eefef112ef249cc97ecd63f19ea122d","value":"tokenizer/tokenizer_config.json: 100%"}},"1cdaf6fe1392453288997f672af80a0d":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"ProgressStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"1e4ad02a5afb431a96626230081054cc":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"ProgressStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"1e64cce9ffc94f23921f964288c2e26d":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"ProgressStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"1ebb24b8e6f44608ae8c11747ef7c42d":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_80509211c7dc47e8a673ed80fca6d8e1","placeholder":"​","style":"IPY_MODEL_e1f4e7fa8b1f4530b580f101a9fad304","value":" 547/547 [00:00<00:00, 8.12kB/s]"}},"1f16ae9b03604ccd93cd1e2f153afe64":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"1f3ce7042b974edbb033b4cd8d13cc08":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"FloatProgressModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_6781679193314617a341ef891ba3df45","max":30,"min":0,"orientation":"horizontal","style":"IPY_MODEL_161a3b1a75e0446ca9120f5e1eea38e9","value":30}},"20c46fc3077b4dcc855252c222002569":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HBoxModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_9b597fc5d5cf4fecabcfbc7a4cfa1ee9","IPY_MODEL_17ceef0b617c4f52aa0bb5ec12113fcd","IPY_MODEL_e20cd706fa304bc190e92ae7d27b5b38"],"layout":"IPY_MODEL_9a3f552babd34cdeb8fed4ce1b1b33a7"}},"21e5a4fd28ac47a190d0f41808dd75a1":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"FloatProgressModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_a76a9fce4af34c639327e5a0f4f4e692","max":30,"min":0,"orientation":"horizontal","style":"IPY_MODEL_e00e5537ae9a43d5956c2c770599edde","value":30}},"221facd121a14faf9d664f644935b0ae":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"FloatProgressModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_e431b2a589524545a5ccbb79d2c7bab9","max":334643276,"min":0,"orientation":"horizontal","style":"IPY_MODEL_90dccdfd4085472f8f9ba0535d85d327","value":334643276}},"2642af1a55cf452a93e528fb25f1c8cb":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"ProgressStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"2c5d8801da6f4d88be3801254c3e764b":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"2dc63f0fe271457f890fd2067631ad75":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"2e194e5ccbd349b093481e24148de89c":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"2e72912929c14d9f8843bc028b75de77":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_f23d6c684c3a416d8b43fc642653eec8","placeholder":"​","style":"IPY_MODEL_16e98aec73d048d693fd0e44df2220e7","value":"vae/config.json: 100%"}},"2f010760c1a146238f35af00568ccb11":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"2f6e125affb54674b5f49beaf3612bee":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"2fd8069de9754a87ae04ec7b2c4b380a":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"309d89a221ab4010af0d85d3ae90a1c3":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"31107fa83a974eca83fa968ae4eae909":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"3214f06917264be88837314b26375a1c":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HBoxModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_00b88de69713463485206e4b7c8c3c04","IPY_MODEL_f63bebf985a046ce9ba9d567db0b7ca4","IPY_MODEL_a49fd3377dde41dab42677091dc7bd04"],"layout":"IPY_MODEL_ed1dbb98c8d34dce8df70e4698a21911"}},"3537007206fd4d57ae492d29d90bd904":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HBoxModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_85bf2410c7d2440db76163fe1df4f4bb","IPY_MODEL_c2bf5a15732a4898915b0ec3cb56df8c","IPY_MODEL_bf573d9fcbac4701b31e464373fdbeb0"],"layout":"IPY_MODEL_f1f352e6964f424f9e6a4557f6e3ff97"}},"3538495833744e9eab9b25eade484603":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HBoxModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_8db4aafac51043598a9e4b8915f4f7c4","IPY_MODEL_0e63faca29cb4ca3845e402d67592b8f","IPY_MODEL_bf4b67c0a0034b1ab576066d366d934e"],"layout":"IPY_MODEL_e09eaa48528a48629b19020aa65c5bdc"}},"35cf8424313d43dea56ca590edf70b26":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HBoxModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_5dc6caab4749403aab23ce95993f9ad0","IPY_MODEL_84e32b4370a94fe498b3b01b6632daf3","IPY_MODEL_16c7e883c9ec49329b68e61b276d5fac"],"layout":"IPY_MODEL_5518095d648d4b2b95647c746d70c4d8"}},"35ffc6d955a44422a06e5c304fcaeddb":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":"20px"}},"36a0330c9e1d48a49a739feaef34ddc0":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"393b21bd730f4a8b93d1653ebafaea09":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"3959b0871ea840839a383c895cfbe916":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"399fdb41f7fd4f1489c1bf4814b53907":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_2fd8069de9754a87ae04ec7b2c4b380a","placeholder":"​","style":"IPY_MODEL_da6373c3704d45869e2fafdf9772d3d1","value":" 617/617 [00:00<00:00, 7.14kB/s]"}},"3b99c2eba3ab4ea2a44b23ec0916ce2d":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"ProgressStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"3cd43805ef564f6696905a2465ea4467":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"FloatProgressModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_da25ec24a6b44ce9a51bcc2d440d0258","max":7,"min":0,"orientation":"horizontal","style":"IPY_MODEL_41ea6163d1b04f8f89cd1f9ec9e72847","value":7}},"3e7caf19c664461e9505cbfcb708ceba":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_69ffdc3c18f5484cad6945a77b024529","placeholder":"​","style":"IPY_MODEL_2c5d8801da6f4d88be3801254c3e764b","value":"100%"}},"3fcfdbffff6149fe880b0702cf8162f3":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"41ea6163d1b04f8f89cd1f9ec9e72847":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"ProgressStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"4262f099aab24cfd9b3790864e0e1d63":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_867f3d155da9469ab9820923e40e78e5","placeholder":"​","style":"IPY_MODEL_7d7e58bafe2c4ff6a44275c3a2ea9826","value":" 37%"}},"44a56b89efa649edb669abed3605e576":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HBoxModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_c26781e6cc214b6b866b8f11673e9c00","IPY_MODEL_cddd14dde34c42f59ed71870e558246e","IPY_MODEL_399fdb41f7fd4f1489c1bf4814b53907"],"layout":"IPY_MODEL_f7f49c60f4ca41efa9a3b6b33c418f73"}},"45d877fafa5a4772ba5d62557843bb51":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"46df7a569a984f2a8d60b021e2366550":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"ProgressStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"495a6de427ce47eda18a1570fd8f5f9d":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"4a466456e448417a8b3cc442cec49632":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"4ab2093e02704b748298a6d34e807847":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"4b14e789cc2446eda7a94a5b1259e738":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HBoxModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_865eb0b3254246948f532e2c6dd02bd1","IPY_MODEL_7e678fd9b4284bf39ecd29de4d7624a3","IPY_MODEL_adfbabeb20a3428c8fd6ec5b79830c34"],"layout":"IPY_MODEL_6a012f24803646369ac97be41e5998f3"}},"4cc69a88629f41dc80c552b58d3f5eaf":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HBoxModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_6cc2e02ee7b74f3aa2f71bf0725190de","IPY_MODEL_da714c09550f43849e5a2502b092403a","IPY_MODEL_7370d528153e473980ac0e701c9e6825"],"layout":"IPY_MODEL_0ba17f55cbf941feaa0b7e8959a94591"}},"4d26d7dd13d148b3b6e06f10b589f2a8":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"51061a6b42584e2bbac74da6a6f2a1da":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"51ceac8abb23437f952e07d2daaf0dae":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"521ecdf554b84958ba4ee487c9621ffc":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"5518095d648d4b2b95647c746d70c4d8":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"55a116f1fa634632a061a4ad8bb75ec3":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"FloatProgressModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_e03272733bd84e2e9edcb83391e1cfae","max":342,"min":0,"orientation":"horizontal","style":"IPY_MODEL_184d9e1fa8d241c386567300db4e2c8c","value":342}},"566d76e028b643e18729621e82531939":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"57d02fadf8fc4df3b1fbee18e45f9199":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"58910b48c70a4b3dabf87b6a12004e70":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_51ceac8abb23437f952e07d2daaf0dae","placeholder":"​","style":"IPY_MODEL_a06b4275e1444e72b921d631feea90ba","value":" 342/342 [00:00<00:00, 2.75kB/s]"}},"59081d8cdf4e43228a20f3fe986926b2":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"ProgressStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"590b0de8d8f84d1ab9a243d88380c295":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"5ce301bccee049cf9664800d63e2e2eb":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"5dc635e136f741608e73455581822408":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"5dc6caab4749403aab23ce95993f9ad0":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_de350491cd2e48a19541f6e97b9f176e","placeholder":"​","style":"IPY_MODEL_cd7fa6f0b6844fd4a41b97cbe39e0c2f","value":"model.safetensors: 100%"}},"5e37d843836b42dfb62f728181ee4dfa":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"FloatProgressModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_034a1eb240694704a8052783583caefe","max":524619,"min":0,"orientation":"horizontal","style":"IPY_MODEL_03fdfd9eb2e343df8af4ff2b06ac8eb1","value":524619}},"5eefef112ef249cc97ecd63f19ea122d":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"5ffaa53c74d3413b8f3c3fe2fc5cc075":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"60aa6e24133c4e67957bf953e5b10f4d":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"62d5802bdc7a49efbcb47889e29e924c":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"ProgressStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"6401b1635d62478ba644075a32494384":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HBoxModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_72a5fcb23802406eb22e2758be01052c","IPY_MODEL_0ba055eeeed04e91b684072740e07f0a","IPY_MODEL_790424c8674d42d59c75ba2fd4021e3a"],"layout":"IPY_MODEL_ff697b2e046a4a36a78ac362cae5c2a9"}},"640b691e30b844ec943995160216e28b":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HBoxModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_4262f099aab24cfd9b3790864e0e1d63","IPY_MODEL_a2194da8bc254658b9db17e19dbe418b","IPY_MODEL_f0e13bd4abca444592850390651272a4"],"layout":"IPY_MODEL_866b048905164e31a5011bdc0fcf5180"}},"66a43c50816c4e3fa88852c3d2c3b0c7":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"670c96aa182f4defbfce4a9fd0b7f96b":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"6781679193314617a341ef891ba3df45":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"681cc61581d84298b4798b5c43818e31":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"6826628eb6214b57bbe56e3eb80322b3":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"ProgressStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"683585ba77df463a9bcb2f8ce4794747":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"6956b01055a34613994672a6bb93994d":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"69ffdc3c18f5484cad6945a77b024529":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"6a012f24803646369ac97be41e5998f3":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"6a7f80bb3e534eb2a48d3d29c9ac3988":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"6c9624bf3faf4890bc1c83c52e33e508":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_94fa436d65894af494b2526746fe7324","placeholder":"​","style":"IPY_MODEL_ffe65c700f0142df9b289a0e8b58ec65","value":" 30/30 [05:25<00:00, 10.77s/it]"}},"6cc2e02ee7b74f3aa2f71bf0725190de":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_a903c8a2c22c48bdb70024665bc1cb0f","placeholder":"​","style":"IPY_MODEL_8c29df6a485946c090b39b49240427fd","value":"tokenizer/vocab.json: 100%"}},"6d9421a31914451cac51c71aee8b1ce4":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"FloatProgressModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_566d76e028b643e18729621e82531939","max":3438167540,"min":0,"orientation":"horizontal","style":"IPY_MODEL_2642af1a55cf452a93e528fb25f1c8cb","value":3438167540}},"6de631e7a72e4b74b740095e8c251ca8":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_7294524debe544c19f8c76a7b3cf0e32","placeholder":"​","style":"IPY_MODEL_f6fb32e142d140d5ad6b357731c4d382","value":" 30/30 [05:46<00:00, 11.49s/it]"}},"6fa7c3c07e734867ac5676b09b6804b3":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"70cf0bb1ada946dd9717fc2a493b7805":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HBoxModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_8caab95721354f08999bea8dc6105b4e","IPY_MODEL_221facd121a14faf9d664f644935b0ae","IPY_MODEL_bac1fcd4cf1847ed89bc5e01ae435e24"],"layout":"IPY_MODEL_bc4ed6312ae44ba7a9e21d43d7edd48a"}},"7294524debe544c19f8c76a7b3cf0e32":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"72a5fcb23802406eb22e2758be01052c":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_eb3e42c270b04086a2a04ae286684f6b","placeholder":"​","style":"IPY_MODEL_d9816365a87340dfa06fe9a37811a81b","value":"Fetching 14 files: 100%"}},"7370d528153e473980ac0e701c9e6825":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_0eaebd8b9eb14feca8a11a8c3077b196","placeholder":"​","style":"IPY_MODEL_51061a6b42584e2bbac74da6a6f2a1da","value":" 1.06M/1.06M [00:00<00:00, 4.59MB/s]"}},"757864720c4041c6a24f8aa8f1630e69":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"7850ea0076da49639ca986a4885d7048":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HBoxModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_f04f25b18eaa43f6a2044dac7aba8372","IPY_MODEL_6d9421a31914451cac51c71aee8b1ce4","IPY_MODEL_a4b3ef66956d494887e796f87b4278f1"],"layout":"IPY_MODEL_1638b8dbe07a4d249167a3d34ac9adc6"}},"790424c8674d42d59c75ba2fd4021e3a":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_5dc635e136f741608e73455581822408","placeholder":"​","style":"IPY_MODEL_670c96aa182f4defbfce4a9fd0b7f96b","value":" 14/14 [00:33<00:00,  2.49s/it]"}},"79fed3974dd8466e8237c7431f71a084":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"FloatProgressModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_aa4bef85913f43c49b708c850301546d","max":30,"min":0,"orientation":"horizontal","style":"IPY_MODEL_e92a9d9050e34b47b766a31935ffbcda","value":30}},"7d7e58bafe2c4ff6a44275c3a2ea9826":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"7e678fd9b4284bf39ecd29de4d7624a3":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"FloatProgressModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_66a43c50816c4e3fa88852c3d2c3b0c7","max":743,"min":0,"orientation":"horizontal","style":"IPY_MODEL_46df7a569a984f2a8d60b021e2366550","value":743}},"7f4317fe6eca4fc5be524453e55103bd":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_6fa7c3c07e734867ac5676b09b6804b3","placeholder":"​","style":"IPY_MODEL_ed410d69e8e94af7be0d104c5c29a2c9","value":" 30/30 [05:55<00:00, 11.65s/it]"}},"80509211c7dc47e8a673ed80fca6d8e1":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"84e32b4370a94fe498b3b01b6632daf3":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"FloatProgressModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_683585ba77df463a9bcb2f8ce4794747","max":492265874,"min":0,"orientation":"horizontal","style":"IPY_MODEL_3b99c2eba3ab4ea2a44b23ec0916ce2d","value":492265874}},"85bf2410c7d2440db76163fe1df4f4bb":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_fa5231429aa1437983dc93dc597e698e","placeholder":"​","style":"IPY_MODEL_fe24820349a0456ca103e30024490c0e","value":""}},"865eb0b3254246948f532e2c6dd02bd1":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_521ecdf554b84958ba4ee487c9621ffc","placeholder":"​","style":"IPY_MODEL_c6a66e5c516c4dbbbc1ca203c6a2d0db","value":"unet/config.json: 100%"}},"866b048905164e31a5011bdc0fcf5180":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"867f3d155da9469ab9820923e40e78e5":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"88c6c5bcc44d46089aa3efaa7fb9e452":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HBoxModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_e0280b4f0172481ea7664bfb96d1bb1c","IPY_MODEL_3cd43805ef564f6696905a2465ea4467","IPY_MODEL_ab7f8b09b1d8452995d66e6f0df83faa"],"layout":"IPY_MODEL_3959b0871ea840839a383c895cfbe916"}},"89ce18ce98494ccc803adbf87f6051e5":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"8c29df6a485946c090b39b49240427fd":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"8c690a7356af4547902b72aa6a20328d":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"8c9b21767c5d4741b717b82f1e4a0e03":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HBoxModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_a83f68f1b1d640298019af11c5198a7d","IPY_MODEL_55a116f1fa634632a061a4ad8bb75ec3","IPY_MODEL_58910b48c70a4b3dabf87b6a12004e70"],"layout":"IPY_MODEL_2dc63f0fe271457f890fd2067631ad75"}},"8caab95721354f08999bea8dc6105b4e":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_fd4eebbe68204eaa802186836c372b93","placeholder":"​","style":"IPY_MODEL_6a7f80bb3e534eb2a48d3d29c9ac3988","value":"diffusion_pytorch_model.safetensors: 100%"}},"8db4aafac51043598a9e4b8915f4f7c4":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_57d02fadf8fc4df3b1fbee18e45f9199","placeholder":"​","style":"IPY_MODEL_2f6e125affb54674b5f49beaf3612bee","value":"model_index.json: 100%"}},"909ad3aefa5b4c65931f300b3e9655dd":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"90dccdfd4085472f8f9ba0535d85d327":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"ProgressStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"92d3f7562ee04b789c44656f83d12528":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HBoxModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_bd7b9e9757d24c568432eea484271376","IPY_MODEL_9c12a4d25771464b84b35804695fd50a","IPY_MODEL_fa1d984ee1864975a4c543f1dcb3aa42"],"layout":"IPY_MODEL_b89baafcd7e944b088bf0c9b1e839ba5"}},"92e5153340a74fc9895d4f87b68e3cad":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"93159e8e4bf84bc9a35c4325dc6cc851":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"ProgressStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"94fa436d65894af494b2526746fe7324":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"95d1047a0a644262aa385987c9a331b4":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"96d5032cdaa14cdeb110f8fc3b6614c1":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"97fe7d3ccc984694aa42a56ba930c64e":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"9a3f552babd34cdeb8fed4ce1b1b33a7":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"9a87e8d407f44ee59a70e511a6274131":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HBoxModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_d49efe4cff2a43288d2140a55c17c4cc","IPY_MODEL_79fed3974dd8466e8237c7431f71a084","IPY_MODEL_6c9624bf3faf4890bc1c83c52e33e508"],"layout":"IPY_MODEL_f23c3781df5a497baa56f273a5f467a5"}},"9b597fc5d5cf4fecabcfbc7a4cfa1ee9":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_f8db6aae6e3a468ba3805991e19e1f45","placeholder":"​","style":"IPY_MODEL_681cc61581d84298b4798b5c43818e31","value":"tokenizer/special_tokens_map.json: 100%"}},"9c12a4d25771464b84b35804695fd50a":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"FloatProgressModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_909ad3aefa5b4c65931f300b3e9655dd","max":1215981830,"min":0,"orientation":"horizontal","style":"IPY_MODEL_93159e8e4bf84bc9a35c4325dc6cc851","value":1215981830}},"9d2ff155648146058d2d359e474159dc":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"9ef6f8a591244419916d980d5883e03e":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"a06b4275e1444e72b921d631feea90ba":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"a2194da8bc254658b9db17e19dbe418b":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"FloatProgressModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"","description":"","description_tooltip":null,"layout":"IPY_MODEL_cc9fdbf01697491f856a33e4b70a7a78","max":30,"min":0,"orientation":"horizontal","style":"IPY_MODEL_6826628eb6214b57bbe56e3eb80322b3","value":11}},"a428e985410341e0ba04359af465681e":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"a49fd3377dde41dab42677091dc7bd04":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_bf1489df7c98442caadd2417026bffdf","placeholder":"​","style":"IPY_MODEL_590b0de8d8f84d1ab9a243d88380c295","value":" 4.72k/4.72k [00:00<00:00, 60.4kB/s]"}},"a4a86f212e8a4dffb0240936475837f7":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"a4b3ef66956d494887e796f87b4278f1":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_a428e985410341e0ba04359af465681e","placeholder":"​","style":"IPY_MODEL_bfbec38740f8413d93469274aeccbf23","value":" 3.44G/3.44G [00:32<00:00, 246MB/s]"}},"a5926a9d27b44a2d961f36d7fd36da15":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"a76a9fce4af34c639327e5a0f4f4e692":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"a83f68f1b1d640298019af11c5198a7d":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_3fcfdbffff6149fe880b0702cf8162f3","placeholder":"​","style":"IPY_MODEL_eb96b8f49d6746ec8f46e65e59a3fad6","value":"(…)ature_extractor/preprocessor_config.json: 100%"}},"a903c8a2c22c48bdb70024665bc1cb0f":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"aa4bef85913f43c49b708c850301546d":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"ab7f8b09b1d8452995d66e6f0df83faa":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_31107fa83a974eca83fa968ae4eae909","placeholder":"​","style":"IPY_MODEL_36a0330c9e1d48a49a739feaef34ddc0","value":" 7/7 [00:02<00:00,  3.52it/s]"}},"adfbabeb20a3428c8fd6ec5b79830c34":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_cffa386bd54e4baa8947d4d51c8e54a4","placeholder":"​","style":"IPY_MODEL_a5926a9d27b44a2d961f36d7fd36da15","value":" 743/743 [00:00<00:00, 10.7kB/s]"}},"ae44d548c5164e8fb5e85f1ab19da9ac":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"FloatProgressModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_defa4622267e49b8a54d6aceea082c39","max":806,"min":0,"orientation":"horizontal","style":"IPY_MODEL_59081d8cdf4e43228a20f3fe986926b2","value":806}},"aedf99983c5043fe8d634b6e3b56e1ae":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"ProgressStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"b08c232e469646f89a28f4371f0e7699":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_9d2ff155648146058d2d359e474159dc","placeholder":"​","style":"IPY_MODEL_0220a7b0d67a482a8ab7e9ac6372d380","value":"tokenizer/merges.txt: 100%"}},"b0bcd8201363473ea0e4ace230443446":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"b18d19fb59d8458c9e9048c1458ee95c":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"b89baafcd7e944b088bf0c9b1e839ba5":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"bac1fcd4cf1847ed89bc5e01ae435e24":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_d1d564e827cf4a71af9aa87b9d5696c8","placeholder":"​","style":"IPY_MODEL_1ad180dc107946b9a1b554a4b98ee514","value":" 335M/335M [00:11<00:00, 15.8MB/s]"}},"bc4ed6312ae44ba7a9e21d43d7edd48a":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"bd7b9e9757d24c568432eea484271376":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_fb34d344590e43438906722b1ab958f0","placeholder":"​","style":"IPY_MODEL_ed6a871ff054410b8d018b3b97c75c60","value":"model.safetensors: 100%"}},"bf1489df7c98442caadd2417026bffdf":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"bf4b67c0a0034b1ab576066d366d934e":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_c577a158f88040c0979b319fb9fb89b0","placeholder":"​","style":"IPY_MODEL_f24c9ef7c3f04da783aaffd3e5d48de7","value":" 541/541 [00:00<00:00, 29.0kB/s]"}},"bf573d9fcbac4701b31e464373fdbeb0":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_96d5032cdaa14cdeb110f8fc3b6614c1","placeholder":"​","style":"IPY_MODEL_4a466456e448417a8b3cc442cec49632","value":" 0/0 [00:00<?, ?it/s]"}},"bfbec38740f8413d93469274aeccbf23":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"c14c4f3d31a447b38d359acc6e29496d":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"c1a5498ffe0f407397d2a8c77e35de84":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"c26781e6cc214b6b866b8f11673e9c00":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_9ef6f8a591244419916d980d5883e03e","placeholder":"​","style":"IPY_MODEL_4d26d7dd13d148b3b6e06f10b589f2a8","value":"text_encoder/config.json: 100%"}},"c2bf5a15732a4898915b0ec3cb56df8c":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"FloatProgressModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_35ffc6d955a44422a06e5c304fcaeddb","max":1,"min":0,"orientation":"horizontal","style":"IPY_MODEL_1e64cce9ffc94f23921f964288c2e26d","value":0}},"c41432dd085f41aeaa1e1d9cac4872e7":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"ProgressStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"c523aec40bb54986a4f40924c81fe5da":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"c565ba8910c34e138c6cd10e9b06d673":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"c577a158f88040c0979b319fb9fb89b0":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"c6a66e5c516c4dbbbc1ca203c6a2d0db":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"c7f341fe95ca426d9502594bd48d36f6":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"cc9fdbf01697491f856a33e4b70a7a78":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"cd7fa6f0b6844fd4a41b97cbe39e0c2f":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"cddd14dde34c42f59ed71870e558246e":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"FloatProgressModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_8c690a7356af4547902b72aa6a20328d","max":617,"min":0,"orientation":"horizontal","style":"IPY_MODEL_1e4ad02a5afb431a96626230081054cc","value":617}},"cffa386bd54e4baa8947d4d51c8e54a4":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"d1d564e827cf4a71af9aa87b9d5696c8":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"d49efe4cff2a43288d2140a55c17c4cc":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_6956b01055a34613994672a6bb93994d","placeholder":"​","style":"IPY_MODEL_1f16ae9b03604ccd93cd1e2f153afe64","value":"100%"}},"d5a23fbc6e634b02bf4f2540e9def457":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"d7c9f6b399524bc596e84641687ba29a":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HBoxModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_1c8a25c7f70145df9722d3702fc6d2dd","IPY_MODEL_ae44d548c5164e8fb5e85f1ab19da9ac","IPY_MODEL_1325ea426c9747828601a9175a9b0248"],"layout":"IPY_MODEL_97fe7d3ccc984694aa42a56ba930c64e"}},"d7eb412b880c490c95f1d2baeaf2e6af":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HBoxModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_3e7caf19c664461e9505cbfcb708ceba","IPY_MODEL_21e5a4fd28ac47a190d0f41808dd75a1","IPY_MODEL_7f4317fe6eca4fc5be524453e55103bd"],"layout":"IPY_MODEL_4ab2093e02704b748298a6d34e807847"}},"d9816365a87340dfa06fe9a37811a81b":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"d9f1220cc4f5440b97e35151ea76ed00":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"ProgressStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"da25ec24a6b44ce9a51bcc2d440d0258":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"da6373c3704d45869e2fafdf9772d3d1":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"da714c09550f43849e5a2502b092403a":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"FloatProgressModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_b0bcd8201363473ea0e4ace230443446","max":1059962,"min":0,"orientation":"horizontal","style":"IPY_MODEL_d9f1220cc4f5440b97e35151ea76ed00","value":1059962}},"de350491cd2e48a19541f6e97b9f176e":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"defa4622267e49b8a54d6aceea082c39":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"e00e5537ae9a43d5956c2c770599edde":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"ProgressStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"e0216d9707cb4d10be57c4864521c376":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HBoxModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_2e72912929c14d9f8843bc028b75de77","IPY_MODEL_feeaea5e7b524ed09bd08c73b7ed5e17","IPY_MODEL_1ebb24b8e6f44608ae8c11747ef7c42d"],"layout":"IPY_MODEL_c523aec40bb54986a4f40924c81fe5da"}},"e0280b4f0172481ea7664bfb96d1bb1c":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_60aa6e24133c4e67957bf953e5b10f4d","placeholder":"​","style":"IPY_MODEL_5ce301bccee049cf9664800d63e2e2eb","value":"Loading pipeline components...: 100%"}},"e03272733bd84e2e9edcb83391e1cfae":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"e09eaa48528a48629b19020aa65c5bdc":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"e1f4e7fa8b1f4530b580f101a9fad304":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"e20cd706fa304bc190e92ae7d27b5b38":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_e4f21da1f63a4a819485cbe7e5ae306b","placeholder":"​","style":"IPY_MODEL_c7f341fe95ca426d9502594bd48d36f6","value":" 472/472 [00:00<00:00, 13.2kB/s]"}},"e431b2a589524545a5ccbb79d2c7bab9":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"e4f21da1f63a4a819485cbe7e5ae306b":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"e92a9d9050e34b47b766a31935ffbcda":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"ProgressStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"eac1b421700d492ba398d5ac609b5741":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"ProgressStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"eb3e42c270b04086a2a04ae286684f6b":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"eb96b8f49d6746ec8f46e65e59a3fad6":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"eba9a67d3f704bed8f501780e35273cb":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HBoxModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_f6b5c1f44a54406c84db14875e4c85b0","IPY_MODEL_1f3ce7042b974edbb033b4cd8d13cc08","IPY_MODEL_6de631e7a72e4b74b740095e8c251ca8"],"layout":"IPY_MODEL_00c30d57328148c88b4258f4841bbdd0"}},"ed1dbb98c8d34dce8df70e4698a21911":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"ed410d69e8e94af7be0d104c5c29a2c9":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"ed6a871ff054410b8d018b3b97c75c60":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"f04f25b18eaa43f6a2044dac7aba8372":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_163ec8057136471bb1f460d657c4aa6c","placeholder":"​","style":"IPY_MODEL_ff1655111fd04c4785d7e5ec3747629c","value":"diffusion_pytorch_model.safetensors: 100%"}},"f0e13bd4abca444592850390651272a4":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_757864720c4041c6a24f8aa8f1630e69","placeholder":"​","style":"IPY_MODEL_92e5153340a74fc9895d4f87b68e3cad","value":" 11/30 [02:05<03:34, 11.30s/it]"}},"f1f352e6964f424f9e6a4557f6e3ff97":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"f23c3781df5a497baa56f273a5f467a5":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"f23d6c684c3a416d8b43fc642653eec8":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"f24c9ef7c3f04da783aaffd3e5d48de7":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"f2f9506fc02a4624a8a1c08f1f6abdb2":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"f4f167561c2d452785b6e59c2ce61b28":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"f63bebf985a046ce9ba9d567db0b7ca4":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"FloatProgressModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_5ffaa53c74d3413b8f3c3fe2fc5cc075","max":4723,"min":0,"orientation":"horizontal","style":"IPY_MODEL_aedf99983c5043fe8d634b6e3b56e1ae","value":4723}},"f6b5c1f44a54406c84db14875e4c85b0":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_a4a86f212e8a4dffb0240936475837f7","placeholder":"​","style":"IPY_MODEL_89ce18ce98494ccc803adbf87f6051e5","value":"100%"}},"f6fb32e142d140d5ad6b357731c4d382":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"f736f2fc44444ebf832fb2ead6ea0fd0":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"f7f49c60f4ca41efa9a3b6b33c418f73":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"f8db6aae6e3a468ba3805991e19e1f45":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"fa1d984ee1864975a4c543f1dcb3aa42":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_f2f9506fc02a4624a8a1c08f1f6abdb2","placeholder":"​","style":"IPY_MODEL_2f010760c1a146238f35af00568ccb11","value":" 1.22G/1.22G [00:24<00:00, 110MB/s]"}},"fa5231429aa1437983dc93dc597e698e":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"fb34d344590e43438906722b1ab958f0":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"fd4eebbe68204eaa802186836c372b93":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"fe24820349a0456ca103e30024490c0e":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"feeaea5e7b524ed09bd08c73b7ed5e17":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"FloatProgressModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_495a6de427ce47eda18a1570fd8f5f9d","max":547,"min":0,"orientation":"horizontal","style":"IPY_MODEL_eac1b421700d492ba398d5ac609b5741","value":547}},"ff1655111fd04c4785d7e5ec3747629c":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"ff697b2e046a4a36a78ac362cae5c2a9":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"ffe65c700f0142df9b289a0e8b58ec65":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}}}}},"nbformat":4,"nbformat_minor":4} From 4d15fac2c63a21ed18c498e13e9ed077171b5528 Mon Sep 17 00:00:00 2001 From: innovation64 Date: Mon, 1 Apr 2024 20:06:06 +0800 Subject: [PATCH 14/31] chore: update yaml file --- notebooks/zh-CN/_toctree.yml | 14 +++++++++++++- notebooks/zh-CN/index.md | 10 ++++++++++ notebooks/zh-CN/prompt_tuning_peft.ipynb | 2 +- 3 files changed, 24 insertions(+), 2 deletions(-) diff --git a/notebooks/zh-CN/_toctree.yml b/notebooks/zh-CN/_toctree.yml index 465fc091..e3f95d92 100644 --- a/notebooks/zh-CN/_toctree.yml +++ b/notebooks/zh-CN/_toctree.yml @@ -4,8 +4,12 @@ title: 开源 AI 指南 (Cookbook) - local: issues_in_text_dataset title: 使用 Cleanlab 检测文本数据集中的问题 + - local: stable_diffusion_interpolation + title: 使用 Stable Diffusion 进行图像插值 - local: rag_with_hugging_face_gemma_mongodb title: 用 Gemma, MongoDB 和开源模型构建 RAG 系统 + - local: tgi_messages_api_demo + title: 使用 TGI 的消息 API 从 OpenAI 迁移到 Open LLMs - local: automatic_embedding_tei_inference_endpoints title: 通过推理端点使用 TEI 自动嵌入 - local: faiss_with_hf_datasets_and_clip @@ -20,5 +24,13 @@ title: 使用 LangChain 在 HuggingFace 文档上构建高级 RAG - local: rag_evaluation title: 使用合成数据和 LLM 作为裁判评估 RAG + - local: prompt_tuning_peft + title: 使用 PEFT 进行提示微调 + - local: labelling_feedback_setfit + title: 使用 SetFit 进行零样本文本分类的数据标注建议 + - local: pipeline_notus_instructions_preferences_legal + title: 创建一个合法偏好数据集 - local: semantic_cache_chroma_vector_database - title: 通过引入语义缓存到 FAISS 中以增强 RAG 系统的性能 \ No newline at end of file + title: 通过引入语义缓存到 FAISS 中以增强 RAG 系统的性能 + - local: llm_judge + title: 使用 LLM 作为评判者🧑‍⚖️进行自动化和多方面的评估 \ No newline at end of file diff --git a/notebooks/zh-CN/index.md b/notebooks/zh-CN/index.md index ef20e734..caaadc36 100644 --- a/notebooks/zh-CN/index.md +++ b/notebooks/zh-CN/index.md @@ -5,12 +5,22 @@ ## 最新 Notebook 查看最近添加的 Notebook： +- [使用 LLM 作为评判者🧑‍⚖️进行自动化和多方面的评估](llm_judge) +- [创建一个合法偏好数据集](pipeline_notus_instructions_preferences_legal) +- [使用 SetFit 进行零样本文本分类的数据标注建议](labelling_feedback_setfit) +- [通过引入语义缓存到 FAISS 中以增强 RAG 系统的性能](semantic_cache_chroma_vector_database) +- [用 LlamaIndex 构建一个 RAG 电子书库智能助手](rag_llamaindex_librarian) +- [使用 Stable Diffusion 进行图像插值](stable_diffusion_interpolation) +- [用 Gemma, MongoDB 和开源模型构建 RAG 系统](rag_with_hugging_face_gemma_mongodb) +- [使用 PEFT 进行提示微调](prompt_tuning_peft) +- [使用 TGI 的消息 API 从 OpenAI 迁移到 Open LLMs](tgi_messages_api_demo) - [通过推理端点使用 TEI 自动嵌入](automatic_embedding_tei_inference_endpoints) - [用 Hugging Face Zephyr 和 LangChain 针对 Github issues 构建简单的 RAG](rag_zephyr_langchain) - [用 🤗 transformers, 🤗 datasets 和 FAISS 嵌入多模态数据进行相似度搜索](faiss_with_hf_datasets_and_clip) - [在单个 GPU 上针对自定义代码微调代码 LLM](fine_tuning_code_llm_on_single_gpu) - [使用合成数据和 LLM 作为裁判评估 RAG](rag_evaluation) - [使用 LangChain 在 HuggingFace 文档上构建高级 RAG](advanced_rag) +- [使用 Cleanlab 检测文本数据集中的问题](issues_in_text_dataset) 你还可以在指南 (Cookbook) 的[Github 仓库](https://github.com/huggingface/cookbook)中查看 Notebook。 diff --git a/notebooks/zh-CN/prompt_tuning_peft.ipynb b/notebooks/zh-CN/prompt_tuning_peft.ipynb index 99701df5..79522f94 100644 --- a/notebooks/zh-CN/prompt_tuning_peft.ipynb +++ b/notebooks/zh-CN/prompt_tuning_peft.ipynb @@ -13,7 +13,7 @@ "id": "2vkOvTEsVaTA" }, "source": [ - "# 使用 PEFT 进行提示微调。\n", + "# 使用 PEFT 进行提示微调\n", "\n", "_作者: [Pere Martra](https://github.com/peremartra)_\n", "\n", From 7a81ef3e5e2565bc6e05282f2191b627e52fb40c Mon Sep 17 00:00:00 2001 From: innovation64 Date: Tue, 14 May 2024 12:02:42 +0800 Subject: [PATCH 15/31] fix: fix rag_llamainde_librarian zh error and unfitted words --- .../zh-CN/rag_llamaindex_librarian.ipynb | 21 ++++++++++--------- 1 file changed, 11 insertions(+), 10 deletions(-) diff --git a/notebooks/zh-CN/rag_llamaindex_librarian.ipynb b/notebooks/zh-CN/rag_llamaindex_librarian.ipynb index c7a79bda..923159fb 100644 --- a/notebooks/zh-CN/rag_llamaindex_librarian.ipynb +++ b/notebooks/zh-CN/rag_llamaindex_librarian.ipynb @@ -15,7 +15,7 @@ "source": [ "## 简介\n", "\n", - "这个 notebook 展示了如何快速构建一个基于 RAG 的电子图书助手，用于你的本地电子书库。\n", + "这份教程将指导你如何快速为你的电子书库创建一个基于 RAG 图书助手。\n", "就像图书馆的图书管理员帮你找书一样，这个助手也能帮你从你的电子书里找到你需要的书。\n", "\n", "## 要求\n", @@ -41,7 +41,7 @@ "source": [ "## 依赖\n", "\n", - "首先安装依赖" + "首先安装依赖库" ] }, { @@ -71,7 +71,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "## 文本库初始化设置\n", + "## 设置测试书库\n", "\n", "我们接下来要弄个测试用的“书库”。\n", "\n", @@ -104,7 +104,7 @@ "2. **索引**，在这个阶段你扩充加载的数据以方便查询，例如使用向量嵌入；\n", "3. **查询**，在这个阶段你配置一个 LLM 作为你索引数据的查询接口。\n", "\n", - "这个解释只是触及了 LlamaIndex 的皮毛。要想了解更多深入细节，我强烈推荐阅读 LlamaIndex 文档中的[\"高级概念\"页面](https://docs.llamaindex.ai/en/stable/getting_started/concepts.html)。\n" + "以上解释仅是对 LlamaIndex 可实现功能的表面说明。要想了解更多深入细节，我强烈推荐阅读 LlamaIndex 文档中的[\"高级概念\"页面](https://docs.llamaindex.ai/en/stable/getting_started/concepts.html)。\n" ] }, { @@ -115,7 +115,7 @@ "\n", "好的，我们首先从**加载**阶段开始。\n", "\n", - "之前我说过，LlamaIndex 是专为 RAG 这种混合检索生成模型设计的。这一点从它的[`SimpleDirectoryReader`](https://docs.llamaindex.ai/en/stable/module_guides/loading/simpledirectoryreader.html)功能就可以明显看出，它能**神奇地**免费支持很多种文件类型。对我们来说很方便的是，`.epub`这种电子书格式也是它支持的。\n" + "之前提到，LlamaIndex 是专为 RAG 这种混合检索生成模型设计的。这一点从它的[`SimpleDirectoryReader`](https://docs.llamaindex.ai/en/stable/module_guides/loading/simpledirectoryreader.html)功能就可以明显看出，它能**神奇地**免费支持很多种文件类型。幸运的是， `.epub` 文件格式也在支持范围内。" ] }, { @@ -152,7 +152,7 @@ "### 索引\n", "\n", "\n", - "在把数据**加载**进来之后，接下来我们要做的是**建立索引**。这样我们的 RAG 系统就能找到与用户查询相关的信息，然后把这些信息传给语言模型（LLM），以便它能够**增强**回答的内容。同时，这一步也会把文档分成一块一块的。\n", + "在把数据**加载**进来之后，接下来我们要做的是**建立索引**。这样我们的 RAG 系统就能找到与用户查询相关的信息，然后把这些信息传给语言模型（LLM），以便它能够**增强**回答的内容。同时，这一步也将对文档进行分块。\n", "\n", "在 LlamaIndex 中，[`VectorStoreIndex`](https://docs.llamaindex.ai/en/stable/module_guides/indexing/vector_store_index.html) 是用来建立索引的一个“默认”工具。这个工具默认使用一个简单、基于内存的字典来保存索引，但随着你的使用规模扩大，LlamaIndex 还支持\n", "[多种向量存储解决方案](https://docs.llamaindex.ai/en/stable/module_guides/storing/vector_stores.html)。\n", @@ -161,10 +161,11 @@ "LlamaIndex 默认的块大小是 1024 个字符，块与块之间有 20 个字符的重叠。如果需要了解更多细节，可以查看 [LlamaIndex 的文档](https://docs.llamaindex.ai/en/stable/optimizing/basic_strategies/basic_strategies.html#chunk-sizes)。\n", "\n", "\n", - "我们之前提到过，会用\n", - "[`BAAI/bge-small-en-v1.5`](https://huggingface.co/BAAI/bge-base-en-v1.5) 这个模型来生成文本的向量表示。LlamaIndex 默认使用 OpenAI 的服务（特别是 `gpt-3.5-turbo` 这个模型），但因为我们要的是一个轻量级、能在本地运行的端到端解决方案，所以不想用 OpenAI。\n", + "如前所述，我们选择使用 [`BAAI/bge-small-en-v1.5`](https://huggingface.co/BAAI/bge-base-en-v1.5) 生成嵌入，以避免使用默认的 OpenAI（特别是 gpt-3.5-turbo）模型，因为我们需要一个轻量级、可在本地运行的完整解决方案。\n", "\n", - "好消息是，LlamaIndex 支持通过 `HuggingFaceEmbedding` 这个类来使用 Hugging Face 上的模型，所以我们这儿就打算用这个方法。\n" + "幸运的是，LlamaIndex 可以通过 `HuggingFaceEmbedding` 类方便地从 Hugging Face 获取嵌入模型，因此我们将使用它。\n", + "\n", + "\n" ] }, { @@ -302,7 +303,7 @@ "\n", "### 强制引用\n", "\n", - "为了防止我们的书库助手胡编乱造，我们怎样才能要求它为其所说的每件事都提供引用呢？\n", + "为了避免图书馆员的虚构响应，我们怎样才能要求它为其回答提供引用？\n", "\n", "### 使用扩充的元数据\n", "\n", From 9f6c5b52ea847bb027a1d4aeddac8067bb5cb08a Mon Sep 17 00:00:00 2001 From: innovation64 Date: Thu, 30 May 2024 20:54:40 +0800 Subject: [PATCH 16/31] update Zh yaml --- notebooks/zh-CN/_toctree.yml | 100 +++++++++++++++++++++++------------ 1 file changed, 66 insertions(+), 34 deletions(-) diff --git a/notebooks/zh-CN/_toctree.yml b/notebooks/zh-CN/_toctree.yml index e3f95d92..43aa8d33 100644 --- a/notebooks/zh-CN/_toctree.yml +++ b/notebooks/zh-CN/_toctree.yml @@ -1,36 +1,68 @@ -- title: 开源 AI 指南 (Cookbook) - sections: +title: 开源 AI 指南 (Cookbook) +sections: - local: index title: 开源 AI 指南 (Cookbook) - - local: issues_in_text_dataset - title: 使用 Cleanlab 检测文本数据集中的问题 - - local: stable_diffusion_interpolation - title: 使用 Stable Diffusion 进行图像插值 - - local: rag_with_hugging_face_gemma_mongodb - title: 用 Gemma, MongoDB 和开源模型构建 RAG 系统 - - local: tgi_messages_api_demo - title: 使用 TGI 的消息 API 从 OpenAI 迁移到 Open LLMs - - local: automatic_embedding_tei_inference_endpoints - title: 通过推理端点使用 TEI 自动嵌入 - - local: faiss_with_hf_datasets_and_clip - title: 用 🤗 transformers, 🤗 datasets 和 FAISS 嵌入多模态数据进行相似度搜索 - - local: fine_tuning_code_llm_on_single_gpu - title: 在单个 GPU 上针对自定义代码微调代码 LLM - - local: rag_zephyr_langchain - title: 用 Hugging Face Zephyr 和 LangChain 针对 Github issues 构建简单的 RAG - - local: rag_llamaindex_librarian - title: 用 LlamaIndex 构建一个 RAG 电子书库智能助手 - - local: advanced_rag - title: 使用 LangChain 在 HuggingFace 文档上构建高级 RAG - - local: rag_evaluation - title: 使用合成数据和 LLM 作为裁判评估 RAG - - local: prompt_tuning_peft - title: 使用 PEFT 进行提示微调 - - local: labelling_feedback_setfit - title: 使用 SetFit 进行零样本文本分类的数据标注建议 - - local: pipeline_notus_instructions_preferences_legal - title: 创建一个合法偏好数据集 - - local: semantic_cache_chroma_vector_database - title: 通过引入语义缓存到 FAISS 中以增强 RAG 系统的性能 - - local: llm_judge - title: 使用 LLM 作为评判者🧑‍⚖️进行自动化和多方面的评估 \ No newline at end of file + +- title: LLM 配方 + sections: + - local: automatic_embedding_tei_inference_endpoints + title: 通过推理端点使用 TEI 自动嵌入 + - local: tgi_messages_api_demo + title: 使用 TGI 的消息 API 从 OpenAI 迁移到 Open LLMs + - local: advanced_rag + title: 使用 LangChain 在 HuggingFace 文档上构建高级 RAG + - local: labelling_feedback_setfit + title: 使用 SetFit 进行零样本文本分类的数据标注建议 + - local: fine_tuning_code_llm_on_single_gpu + title: 在单个 GPU 上针对自定义代码微调代码 LLM + - local: prompt_tuning_peft + title: 使用 PEFT 进行提示微调 + - local: rag_evaluation + title: 使用合成数据和 LLM 作为裁判评估 RAG + - local: llm_judge + title: 使用 LLM 作为评判者🧑‍⚖️进行自动化和多方面的评估 + +- title: Diffusion 配方 + sections: + - local: stable_diffusion_interpolation + title: 使用 Stable Diffusion 进行图像插值 + +- title: 多模态配方 + sections: + - local: analyzing_art_with_hf_and_fiftyone + title: 使用多模态嵌入分析艺术风格 + - local: faiss_with_hf_datasets_and_clip + title: 用 🤗 transformers, 🤗 datasets 和 FAISS 嵌入多模态数据进行相似度搜索 + +- title: 使用其他库的 LLM 和 RAG 配方 + sections: + - local: issues_in_text_dataset + title: 使用 Cleanlab 检测文本数据集中的问题 + - local: annotate_text_data_transformers_via_active_learning + title: 使用 Cleanlab 和 Active Learning 标注文本数据 + - local: rag_with_hugging_face_gemma_mongodb + title: 用 Gemma, MongoDB 和开源模型构建 RAG 系统 + - local: rag_zephyr_langchain + title: 用 Hugging Face Zephyr 和 LangChain 针对 Github issues 构建简单的 RAG + - local: rag_llamaindex_librarian + title: 用 LlamaIndex 构建一个 RAG 电子书库智能助手 + - local: pipeline_notus_instructions_preferences_legal + title: 创建一个合法偏好数据集 + - local: semantic_cache_chroma_vector_database + title: 通过引入语义缓存到 FAISS 中以增强 RAG 系统的性能 + - local: structured_generation + title: 使用结构化生成在 RAG 系统中进行源高亮 + +- title: 计算机视觉 + sections: + - local: fine_tuning_vit_custom_dataset + title: 用自定义生物医学数据集微调视觉 Transformer 模型 + +- title: 智能体 + sections: + - local: agents + title: 使用 Transformers Agents 构建具有工具调用超能力的代理 + +- title: 企业 hub 指南 + sections: + - local: enterprise_cookbook_overview From 8b1909e08a4cdcf9f27f1c7f9ce3d74f579d1deb Mon Sep 17 00:00:00 2001 From: innovation64 Date: Tue, 9 Jul 2024 15:00:11 +0800 Subject: [PATCH 17/31] update ft vits on custom dataset cn version --- .../fine_tuning_vit_custom_dataset.ipynb | 1196 +++++++++++++++++ 1 file changed, 1196 insertions(+) create mode 100644 notebooks/zh-CN/fine_tuning_vit_custom_dataset.ipynb diff --git a/notebooks/zh-CN/fine_tuning_vit_custom_dataset.ipynb b/notebooks/zh-CN/fine_tuning_vit_custom_dataset.ipynb new file mode 100644 index 00000000..1c737688 --- /dev/null +++ b/notebooks/zh-CN/fine_tuning_vit_custom_dataset.ipynb @@ -0,0 +1,1196 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "97bf8340-2c4f-4b32-9a64-5b8ed2d6247f", + "metadata": {}, + "source": [ + "# 用自定义生物医学数据集微调视觉 Transformer 模型\n", + "_作者: [Emre Albayrak](https://github.com/emre570)_\n", + "\n", + "本指南概述了在自定义生物医学数据集上微调视觉 transformer（ViT）模型的过程。它包括加载数据集和准备数据集的步骤，为不同的数据拆分设置图像转换，配置和初始化 ViT 模型，以及定义具有评估和可视化工具的训练过程。\n", + "\n", + "## 数据集信息\n", + "自定义数据集是手工制作的，包含 780 张图片，分为 3 类（良性，恶性，正常）。\n", + "\n", + "![attachment:datasetinfo.png](https://huggingface.co/datasets/huggingface/cookbook-images/resolve/102d6c23e6cc24db857fbc60186461ded6cdfb75/datasetinfo.png)\n", + "\n", + "## 模型信息\n", + "我们所要微调的模型是 Google 的 [`\"vit-large-patch16-224\"`](https://huggingface.co/google/vit-large-patch16-224) 模型，该模型可以在 Hugging Face 的模型库中找到。这个模型是在 ImageNet-21k 数据集上进行预训练的，该数据集包含 1400 万张图片和 21,843 个类别。之后，它在 ImageNet 2012 数据集上进行了微调，该数据集有 100 万张图片和 1000 个类别，图像分辨率统一为 224x224。Google 还提供了其他几种 ViT（Vision Transformer）模型，它们具有不同的图像尺寸和分割块大小。\n", + "\n", + "现在，让我们开始吧。" + ] + }, + { + "cell_type": "markdown", + "id": "3cc02613-7bc6-4cd8-aa97-a21ba1970027", + "metadata": {}, + "source": [ + "## 开始\n", + "首先，让我们安装库。" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "7093dd4f-d0cb-44dc-935d-d54435187901", + "metadata": {}, + "outputs": [], + "source": [ + "!pip install datasets transformers accelerate torch scikit-learn matplotlib wandb" + ] + }, + { + "cell_type": "markdown", + "id": "9b5019a8-d130-4c08-9503-cd8415f50ae9", + "metadata": {}, + "source": [ + "(可选) 我们会把我们的模型推送到 hugging face hub 上，所以我们必须登录。" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "id": "3d5acb41-a225-44a5-8c8f-f212c615008f", + "metadata": {}, + "outputs": [ + { + "data": { + "application/vnd.jupyter.widget-view+json": { + "model_id": "4a75a73de5234297a7e0e4e070eee6d9", + "version_major": 2, + "version_minor": 0 + }, + "text/plain": [ + "VBox(children=(HTML(value='

,\n", + " 'label': 0}" + ] + }, + "execution_count": 7, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "train_ds[0]" + ] + }, + { + "cell_type": "markdown", + "id": "384a09b0-1c47-411f-b91a-00acdd88b06b", + "metadata": {}, + "source": [ + "我们还可以查看训练集的特征。" + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "id": "1e09647b-44e9-4f5f-800c-c333b0523b85", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "{'image': Image(mode=None, decode=True, id=None),\n", + " 'label': ClassLabel(names=['benign', 'malignant', 'normal'], id=None)}" + ] + }, + "execution_count": 8, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "train_ds.features" + ] + }, + { + "cell_type": "markdown", + "id": "a3fedab5-0e80-492c-9408-f629b230351d", + "metadata": {}, + "source": [ + "让我们从数据集中显示每个类的一个图像。" + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "id": "5c901865-a876-4b4b-b1f2-8895b494cafb", + "metadata": {}, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAxoAAAD1CAYAAADXuoOYAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8fJSN1AAAACXBIWXMAAA9hAAAPYQGoP6dpAAEAAElEQVR4nOy9eZSseV3f/66lq7dae7t37uwzIEJAjTiuYRPCyI4TD4QcUFBZjkswMAlLdJjRScYlEI8LgmIYDNEYPXKMwS2CeKIxakJAj4oMw9yZO3frpbqquru6q7qr6vdH/16f5/0898L06MBMx+d7zj333u6q5/kun+X9eX8+3++3MJlMJspb3vKWt7zlLW95y1ve8pa3R7AVH+0O5C1vectb3vKWt7zlLW95+3+v5YFG3vKWt7zlLW95y1ve8pa3R7zlgUbe8pa3vOUtb3nLW97ylrdHvOWBRt7ylre85S1vectb3vKWt0e85YFG3vKWt7zlLW95y1ve8pa3R7zlgUbe8pa3vOUtb3nLW97ylrdHvOWBRt7ylre85S1vectb3vKWt0e85YFG3vKWt7zlLW95y1ve8pa3R7zlgUbe8pa3vOUtb3nLW97ylrdHvOWBRt7ylre85S1vectb3v6fas985jP1zGc+89Huxt/7lgcaectb3vKWt7zlLW95y1veHvGWBxp5y1ve8pa3vOUtb3nLW94e8ZYHGnnLW97ylre85S1vefuCtclkot3d3Ue7G3l7FFoeaOQtb3nLW97ylre8/T1ut99+uwqFgj7zmc/o1a9+tZrNphqNhl7zmteo3+/H5w4ODvRDP/RDuvHGGzU9Pa3rrrtOb3/72zUYDFLPu+666/TCF75Qv/M7v6Ov+qqv0uzsrN773vfqYx/7mAqFgv7Lf/kvuuOOO3TllVeqVqvpW77lW9TtdjUYDPR93/d9WllZUbVa1Wte85pLnv3+979f3/iN36iVlRVNT0/rSU96kn7mZ37mizJPeXv4rfxodyBvhwp+xx13aG1tTUtLS1+0954+fVrXX3+93v/+9+vVr371F+29ectb3vKWt7zl7bHXXvayl+n666/XXXfdpY9//ON63/vep5WVFf3Ij/yIJOk7v/M79YEPfEDf8i3foje/+c36kz/5E911113667/+a33oQx9KPetv/uZv9IpXvEKvf/3r9drXvlZPeMIT4nd33XWXZmdn9da3vlWf+cxn9JM/+ZOamppSsVjU5uambr/9dv2v//W/dPfdd+v666/XbbfdFt/9mZ/5Gf2Df/AP9OIXv1jlclm/8Ru/oe/6ru/SeDzWd3/3d39xJipvR2+TvD3q7R3veMdE0mRtbe2L+t777rtvImny/ve//4v63rzlLW+PfpM0ecc73hH/f//73z+RNLnvvvsetT7lLW95e3QaOOTbv/3bUz//5m/+5sni4uJkMplMPvGJT0wkTb7zO78z9Zlbb711Imny0Y9+NH527bXXTiRNfvu3fzv12d///d+fSJo8+clPngyHw/j5K17xikmhUJg873nPS33+677u6ybXXntt6mf9fv+S/t98882TG264IfWzZzzjGZNnPOMZn3/gefuCt7x06u9xu/baa7W7u6tXvepVj3ZX8pa3vOXtb9XOnTun22+/XZ/4xCce7a7kLW/Hvr3hDW9I/f9pT3uaNjY21Ov19Ju/+ZuSpDe96U2pz7z5zW+WJH34wx9O/fz666/XzTfffNn3fOu3fqumpqbi/1/zNV+jyWSib//2b0997mu+5mt05swZHRwcxM9mZ2fj391uV+vr63rGM56hz372s+p2u0cdat6+SC0PNP4et0KhoJmZGZVKpUe7K3nLW94e5faqV71Ku7u7uvbaax/trjysdu7cOd1xxx15oJG3vD0C7Zprrkn9v9VqSZI2Nzd1//33q1gs6nGPe1zqMydPnlSz2dT999+f+vn1119/5Pc0Gg1J0tVXX33Jz8fjcSqA+KM/+iM95znP0fz8vJrNppaXl/X2t79dkvJA4zHY8kDjMdTW19f1spe9TPV6XYuLi3rjG9+ovb291Gc++MEP6qlPfapmZ2e1sLCgf/pP/6nOnDmT+swzn/lMPfnJT9Zf/dVf6VnPepbm5uZ05ZVX6kd/9EdTnzt9+rQKhYLuvvvu1M9/5Vd+RU960pM0MzOjJz/5yfrQhz6kV7/61bruuusu+e6/+3f/Tj/7sz8bG8Nuuukm/dmf/dkjOi95y1vevvCtVCppZmZGhULh0e5K3vKWt0epfS7icTKZxL+PaiM883DU9zzU+++99149+9nP1vr6ut71rnfpwx/+sP77f//v+hf/4l9Iksbj8ZH6lrcvXssDjcdQe9nLXqa9vT3dddddev7zn6+f+Imf0Ote97r4/b/5N/9G3/qt36rHP/7xete73qXv+77v00c+8hE9/elPV6fTST1rc3NT3/RN36Qv//Iv1zvf+U596Zd+qd7ylrfot37rtz5vHz784Q/r5S9/uaampnTXXXfplltu0Xd8x3fo//yf/3PZz//iL/6ifuzHfkyvf/3rdeedd+r06dO65ZZbtL+//3eej7zl7e9j4/SXT3/603rlK1+pRqOh5eVl/cAP/IAmk4nOnDmjl7zkJarX6zp58qTe+c53xneHw6Fuu+02PfWpT1Wj0dD8/Lye9rSn6fd///cf8r133323CoWCTp8+HT8bj8e6/fbbderUKc3NzelZz3qW/uqv/krXXXdd6gAJvvtHf/RHetOb3qTl5WXNz8/rm7/5m7W2tpZ6z6//+q/rBS94gU6dOqXp6WndeOON+qEf+iGNRqPU545CmHzsYx/TTTfdJEl6zWteo0KhcFnyJG95y9vfvV177bUaj8e65557Uj+/ePGiOp3OFyUb+hu/8RsaDAb6r//1v+r1r3+9nv/85+s5z3nO5w1q8vbotvzUqcdQu/766/Xrv/7rkqTv/u7vVr1e17vf/W7deuutajQaesc73qE777wzUoSSdMstt+gf/sN/qHe/+92pn587d06/8Au/EPsvvuM7vkPXXnutfv7nf17Pe97zPmcf3va2t+nKK6/UH/3RH6larUqSnv3sZ+uZz3zmZY3IAw88oHvuuSfSq094whP0kpe8RL/zO7+jF77whX/3Sclb3v6etpe//OV64hOfqB/+4R/Whz/8Yd15551aWFjQe9/7Xn3jN36jfuRHfkT/6T/9J91666266aab9PSnP129Xk/ve9/79IpXvEKvfe1rtbW1pZ//+Z/XzTffrD/90z/VV3zFVzysPrztbW/Tj/7oj+pFL3qRbr75Zn3yk5/UzTfffEmmlfa93/u9arVaesc73qHTp0/rx3/8x/U93/M9+uVf/uX4zN13361qtao3velNqlar+uhHP6rbbrtNvV5PP/ZjP5Z6HoTJLbfcope97GX61V/9Vb3lLW/RU57yFD3vec/TE5/4RP3gD/6gbrvtNr3uda/T0572NEnS13/91z+8yc5b3vL2kO35z3++3v72t+vHf/zH9d73vjd+/q53vUuS9IIXvOAL3gcyHp5h6Xa7ev/73/8Ff3fe/nYtDzQeQy17LNv3fu/36t3vfrd+8zd/U9PT0xqPx3rZy16m9fX1+MzJkyf1+Mc/Xr//+7+fCjSq1ape+cpXxv8rlYq++qu/Wp/97Gc/5/vPnTunv/iLv9Db3/72CDIk6RnPeIae8pSnqNfrXfKdl7/85RFkSApH//nek7e85e2h21d/9VeHM3/d616n6667Tm9+85t111136S1veYsk6RWveIVOnTql//Af/oOe/vSnq9Vq6fTp06pUKvGc1772tfrSL/1S/eRP/qR+/ud//sjvv3jxot71rnfppS99aerYyjvuuEO33377Zb+zuLio3/3d343SivF4rJ/4iZ9Qt9uNGuxf/MVfTLGPb3jDG/SGN7xB7373u3XnnXdqeno6fvdQhMmJEyf0vOc9T7fddpu+7uu+LmXz8pa3vD2y7cu//Mv1bd/2bfrZn/1ZdTodPeMZz9Cf/umf6gMf+IBe+tKX6lnPetYXvA/Pfe5zValU9KIXvUivf/3rtb29rZ/7uZ/TysqKzp8//wV/f94efstLpx5D7fGPf3zq/zfeeKOKxaJOnz6te+65R5PJRI9//OO1vLyc+vPXf/3XWl1dTX33qquuuqSOstVqaXNz83O+n41c2Y1en+tn0uffOJa3vOXtb9++8zu/M/5dKpX0VV/1VZpMJvqO7/iO+Hmz2dQTnvCECOxLpVIEGePxWO12WwcHB/qqr/oqffzjH39Y7//IRz6ig4MDfdd3fVfq59/7vd/7Ob/zute9LmV3nva0p2k0GqU2iXqQsbW1pfX1dT3taU9Tv9/Xpz71qdTz/jaESd7ylrcvXHvf+96nO+64Q3/2Z3+m7/u+79NHP/pRve1tb9N//s//+Yvy/ic84Qn61V/9VRUKBd166616z3veo9e97nV64xvf+EV5f94efsszGo/h5g57PB6rUCjot37rty67WcozENLRNnQ9Eu2L9Z685e3vW7vcqSwzMzOXXOrZaDS0sbER///ABz6gd77znfrUpz6V2iv1+U6AuVz7XMTDwsJCKov5+fp8OeLhL//yL/X93//9+uhHP3pJljR7YsznIkz+/M///GGMJG95y9tDtdtvv/2ymcpXv/rVqf1Y5XJZt912W+oCvcs13+vl7ZnPfOZl8UH2PZ+vXy960Yv0ohe96JLPvuY1r0n9/2Mf+9jn7WPevjgtDzQeQ+2ee+5JgYHPfOYzGo/Huu6661QqlTSZTHT99dfrS77kS74g72cPxmc+85lLfne5n+Utb3n7wrXLBfEPFdh/8IMf1Ktf/Wq99KUv1b/8l/9SKysrKpVKuuuuu3Tvvfd+Qft7lP5RblGv1/WDP/iDuvHGGzUzM6OPf/zjestb3nLJiTE5kZG3vOUtb8e75aVTj6H20z/906n//+RP/qQk6XnPe55uueUWlUol3XHHHZc42clkkmI0/7bt1KlTevKTn6xf+IVf0Pb2dvz8D/7gD/QXf/EXf+fn5y1vefvCtl/91V/VDTfcoF/7tV/Tq171Kt188816znOe8zk3b3++9rmIh42Njb91aeTHPvYxbWxs6O6779Yb3/hGvfCFL9RznvOcz5khOUrLj+PNW97ylrfHbsszGo+hdt999+nFL36xvumbvkl//Md/rA9+8IP6Z//sn+nLv/zLJUl33nmn3va2t+n06dN66Utfqlqtpvvuu08f+tCH9LrXvU633nrr37kP//bf/lu95CUv0Td8wzfoNa95jTY3N/VTP/VTevKTn5wKPvKWt7w99pqfyAIA/5M/+RP98R//8SVlTQ/Vnv3sZ6tcLutnfuZn9I//8T+On//UT/3UI9I/2nA41Lvf/e6/9TPn5+cl6ZIjvvOWt7zlLW+PfssDjcdQ++Vf/mXddttteutb36pyuazv+Z7vSR33+Na3vlVf8iVfon//7/+97rjjDkmHt2g+97nP1Ytf/OJHpA8vetGL9Eu/9Eu6/fbb9da3vlWPf/zjdffdd+sDH/iA/vIv//IReUfe8pa3L0x74QtfqF/7tV/TN3/zN+sFL3iB7rvvPr3nPe/Rk570pIdNFJw4cUJvfOMb9c53vjMIkE9+8pP6rd/6LS0tLf2tMglf//Vfr1arpW/7tm/TP//n/1yFQkH/8T/+x79TKdSNN96oZrOp97znParVapqfn9fXfM3XPOw9KXnLW97ylrdHvuWBxmOg+WanX/mVX/m8n73lllt0yy23fN7PfK4NUNlLrK677rrLOviXv/zlevnLX5762Q/8wA/oqquuesjvSnn9dN7y9mi1V7/61bpw4YLe+9736nd+53f0pCc9SR/84Af1K7/yK3+rjZE/8iM/orm5Of3cz/2cfu/3fk9f93Vfp9/93d/VP/pH/0gzMzMP+3mLi4v6b//tv+nNb36zvv/7v1+tVkuvfOUr9exnP1s333zzw36eJE1NTekDH/iA3va2t+kNb3iDDg4O9P73vz8PNPKWt7zl7THQCpMcFebN2v7+vgqFgsrlJAb92Mc+pmc961m688479a//9b9+FHuXt7zl7dFunU5HrVYrtwd5y1ve8pa3h2x5RiNvqXb27Fk95znP0Stf+UqdOnVKn/rUp/Se97xHJ0+e1Bve8IZHu3t5y1vevohtd3c3de+FJP34j/+4pMNjKvOWt7zlLW95+3wtDzTylmqtVktPfepT9b73vU9ra2uan5/XC17wAv3wD/+wFhcXH+3u5S1vefsitl/+5V/W3Xffrec///mqVqv6wz/8Q/3SL/2Snvvc5+obvuEbHu3u5e3vYfvMZz6jjY2N2HNUKBS0u7urvb09zc7Oam5uTv1+P+6QOTg40HA41MHBgcrlsiaTiQ4ODuIG+slkomKxqEKhEH8ODg5ULBY1mUziDqtCoaDJZKJyuayZmRlNJhPt7e3F9yVpNBqpXC5rampKxWJRxWJRc3NzGo/HGg6HGo/HKpVKmpmZ0Wg0Ur/f12g0UrFY1MzMjKanp1UoFFQqlVQoFLS/v6/xeKz9/X3t7e3F86ampjQ1NaVyuaz9/X0Nh0NJiu/t7u6q3+/r4OBABwcHmkwmmpqaUqVS0Wg00sHBgUajUXx+PB6rUqlofn4+3jsajeJ7pVIpniUpLgWlAoJ1GI1GGg6Hmp2dValU0s7Ojg4ODuIzFNDwf5/bSqUSY2PeWYdKpaLhcKhutxtzPjU1pZmZGc3Ozmo0Gmk8Hsda8GcymWgymahUKoUM8F7GVy6XVSqVYq597xmVHcViUaVSSaPRKMaxv7+fer6vF58pFAqampqK95XLZVUqlZCHSqWimZmZkD/mnX7y/tFoFPLHOvD+YrEYc8a6+nyPx+PU7wuFQsgrF7wWCoXQGcY6NTWVkh36ubu7q3K5HLLy3Oc+9yF1Ni+dylve8pa3vF22ffzjH9e/+lf/Sp/4xCfU6/V04sQJ/ZN/8k905513XnJJaN7y9sVof/7nf67xeKzt7e0o8wWk9fv9AH/8GY1GAbR2d3c1Ho8DnBF4DAYDSQoQNhgMNBgMAtTxfekQ/Dkg9XJjgJ43gPpgMAgQWCqVVCwW4/+Az9FolAoqZmZm1Gg0VCwWA8QCQovFYoBkv3+G5wC8CSJmZmYCUPNegCRgH4BeqVRULpc1HA5VKBQiKNvf3w/wD/B1sM17GZv3gwbYpq++FswtQQDPANgTrDEuQDDzzL95Fu934O3BpPd1OByqVCqFTPAd3kkgRSAwGAxi7RgP8zUej2Pe5+bm4h40ZIX+s37MJ7JDIDA1NaVCoaC9vT3t7e3FeJDH4XAY3+HZ1WpVc3NzMSaCB8aP3CCPjNnfi4zwe3TDdWBmZkaVSkXf8i3f8pA6mwcaectb3vKWt7zl7Vi0j3zkIwHQ6vW6arWaJAUIA4hNT0+rUqkEGAUsS4dAqtfraXt7+xLgPRqNNBgMIksA8OYdBCYAPM9U8HtJEXwATIvFYjDRnlmBPSaIABzOz8+rXq8HaITB5r0EQh4EeRaCz5FdkZKMB/PhAQJ37RAIEDjxs/F4rN3d3VQGh3974ERAxLM9m8PPpqenY955P2sAwPfAwVl6wDXrQBAAgC6Xy5qentbs7GyMY2ZmRvV6XYVCQTs7O+r3+/GHPrFm/j4H2wQQnqkgc4NMEbwwb/zN+rB+e3t7sfbT09Oan5/X9PR0BHpkkAjitre3tbu7G+MmeBkOh5pMJqpUKvGcVqulubk57e/va3NzU4PBIJWNIYtBP/v9fswhMoksFItFVatVzczMxNx4kDgajfT0pz/9IXU2L53KW97ylre85S1vx6Kxb2g0GmlnZydAZqVSUa1W0+zsbIA1sh3eYJbL5bKq1WqAN8qWYM55LgHDzMyM5ubmAmg5mz8cDtXv97W1tRWA04MYB5wAPwIPQCxBDqAcEEu5C8EVpS4APUkxVhhwB6oEW8PhUMViMVh8ggVaq9VSpVJRs9mM0i6A/PT0dARDzIuDbukQVHc6HfV6vVSWybMaHgwyJvrsjTIusjA+Vh8fQJmyuNnZ2Qg05ubmYs74GeVqvh7Mxd7eXgSQrJn3n3Kh2dnZYPM9qGCOCVIJZglYCTYpSRoMBlEuh7zxHJ9byrsYD2Obnp6O8RGskl0hsJiZmYnsmMsIGZa9vb1UMEJQSnZkampK8/PzqQyJlxgSCD5UyzMaectb3vKWt7zl7Vi0P/zDP4wSIPYjbG5uXlJKBSAFjBEUANKz5U6UFwH+ms2mms2marWaKpVKBB38nW0830ugaOPxOMpf/PeAu/39/dTnYdl9LwUg0wOdYrEYwBeQyPPZ+zEcDgP40siG+Pso1QJMS1K/3w+QC3N/uUYA1e12NRgMYo4IdggCCCoIEgi49vf3tb29HSVsc3Nzwcyzdsxjdr6z5Ua+RmRTmAuyFJQ8lcvlmLOdnZ1UgEcjQ+NlXAS2jI9gcWtrS91uN9aM9eW9ZGkA7cwNz9nf39dgMEiV2/neE98rwZ4TLyHzIG4ymajf70fmzgO16enpCOK8n5JSwdLu7m4Ezl4WRpAxMzPzkNctSA8jo1GtVnX99dfrzW9+sz70oQ/p1ltvjRq+vb29eDEC5Gk9JvXg4CDSVqQ35+fnI0Jn8w4RPKkhlMQ3wKDQXguYFWYEBIaDiJLnUmOHoPC54XCYUiyME8KDYLIRq9/vhzL4JjRq/jyal9IbfGZmZjQYDGKDE4teKBRC4GE4+B5zR+0eLAdz7GOqVCqqVqth6HxzEo25c+PMvNJfTxUS/VMz6MyRp2bZvIXhcQbImSFXUkmR4t7b2wtj4H0j9bi/vx9jQQncKLK+09PTmp6eTtVJ+gYnvoch8vHAEMAEYLQqlYparZamp6fV7/e1s7MTxrDf74fcYEDo4+7ubsiz943xzczMRLqdTYPT09Oh3Dgav9Mkb3nLW97+vrTt7W1df/31YYclhd/d2tqKjdD9fj/KTQqFQgqrzMzMaH9/P+w2WAL8MDU1Fcw+4JsyJvyybwDGhwLgfIMumGN+fl6j0Sj6JymVpQD4AdphovH1AHL8LhmHra2twAew3YBuviMpMIIDTgf94JrBYKBKpRJgl7nzYCvbmFO/W8f9PX253HcBx8wP+zUul3Xxkq1shsT97GAwUL/f1+bmpnq9XiqbwNrNzMyoVqtFpgNZcZbemXvfAE/ZHvMEXiDzRYAIdswGVcPhUNPT06pWqyn/TgkecwXO8X03BDmMdWtrK+Tf5ccDmWazGWspHQZyUlJq58EJMjUYDC45jIDPNZtNraysqF6vH0lnjxxofMmXfIme+tSnqt/vq9vt6oEHHtD8/Ly2t7fV7XYvqRVjkaamprS9vR1A0KNCBuCsA3V28/PzAbK99g9l6vf7AURnZmZSgobxACBy8oG/04E4wcX29nYsNILpgQybasrlsur1etQ48n5SZID4RqOh+fn5AKIEOwgzQtbv91WtVoNJ8FMXSA+TviRlB+AkUt/Z2VGn04nIFeXzNBgBHWCWaJgAZTKZ6MSJE2o2mxoMBqF41D2Srtvf39fu7q52dnYCEPNsKamtJMhpNBpRE7qzs6Pt7W3Nzs5qdnY2wLgkLSwsqNlsanl5WZJ04cKFOOGAoI7gjznmlBHmoVKphMNxGeHkD2oyZ2dn1Wg0QjG9FtQbRprUMQE0gS8BUqfTibpGlx0UnTnHmdXr9TBCGCaCN040QfHRF9Lzko5toPEN3/ANetWrXqW3v/3tmkwmev3rX69Pf/rTeslLXqKbbropZNllC1nPsmQ4iK2trQjUkUN3njCSBJ84cQJRgmfS5TgMSA4cCd/FqXhtL07/4OAg2EWvKWaDKQwhNqdUKqlarcbPdnd3JSl14gosF/MAWEC/PeUuJeBlb28vlQrHdjlRwrvcNkuKABv95J1eS+0bR5k7xg0QwBYUCoUgF3Z3d0OHsLe+0XV3dzfWHaZtbm4u2GuIC+YcuwhIYqwwjg40ATQ4ak/oo3PoLXbaa+mdBYSJ9tplyAjmFYYWhnh/fz/e3e/3Qz7Rb+xFts6btQO4UkYB48laZmvBqVUHoHkpUb/f10033fQIavcXp33iE59Qp9MJ3w8mqNfrOnXqlObn52N+mR90hWABnYAI63Q6WltbU6fT0dbWlnZ2drS7uxvgtFar6cSJEwEu8XmTySSewQlIkEPuU3i/73vwmvh+vx/2Znd3N8quHFjyXYhaZ7P5w3rPz89HcAS55lkT5Bzdo0/YM3TMTzjKZgwgybCP6Ap6kLVNrAN9Bwyjxw7M/eQu+o/tgagbDAbq9Xrhd5F93gk4xt5IiqwNOshYKKlyXwI28CAO7IUeTk9Pq9FoaHl5Wc1mU9VqNewOeNLHu7OzEzaBMRKYSgqMxWd3dnZirASUYJ9SqaRGo6GFhQXNzs7GHo1ms6m5ubkIrH3fEHbVM0V7e3va3t6OwAdfQGJgc3MzcJPbzEajccnR55+rHTnQeO1rX6tGo6FPfOITMWgU2TcpkWrBKGLkEWAWx51ipVKJ6BPnBFtN9DcYDFLfAbyhhAgjn5UUiogzqlarqdpCNt8QBDkLjkJi9FF6/o2A1Gq1MGzUJ3qKi2BsY2MjFp6j9RYWFlKOn88zHtrs7Gwq0uVvSVGDh2FYWFgIIWIDEY4Rg0LmB0VDGcvlshqNhlqtlgaDgRqNhkajkba3t8MQYuxgfFAgZIAgDGPuNa48n2PRYBNYDw9mdnZ2tLS0FPOOQWIsyBEBi2e+2u12rB3BDHIBW4PcAVap9c0eOwhw3dzcDFkiK8F6IpsYSVgTB65zc3MBlnBUnuXDGJHNQCYwVllde+pTn3pU1X1MtX6/H7J/7bXX6tSpU/rEJz6he++9N0AcjsLrSrMlCc4YujF3Fow1B3jPz89LktrtdsiCdKhfOBzWazAYhLN3hwDrlnW8PA+Zgazw7JWvPQ4c2cAmHhwcpJgzMm7YDJwODtNT2jSCchw54Afd9dNbsHuecve5x6l5qh/59Jpy5hA7QnbXyxYAYDg/APDOzo4khS0gQ+wbaNF59MGBObbAs7q+WRKdZL339/cji+inx2B7WVtqzrF7vJfAajwea25uLjLG9AeAIynIpF6vF7ZDUtjzSqWidrsdcu/zyzq4HGVZYrKcBNNeTpENLP27+OPjGGisr69LSnCAVzjMzc3pqquu0srKShCWrKuXpzA3p06dSgXUW1tb2tjY0Pnz5wPkoS/nzp3T+vp6yAz+yEkhfAj+j5+jT/gGgm9kE+xEH8BC/IG0Yy1hv3kuv6/VavG5g4MDtdtttdvtCKC8FAabBzFL0IA+8gyvxkDunRwGi+HbmVuvAvBMPjJK8OBz4/tCPKvBe7G9bntozMP8/HycvAQ+Q1/QFQIBMgKUbbktxW4T3DF+5sSJFs9i+b4SD+w9QPDMBr4AuzU3NxfzxXjBI8w3/4awrtVqWlpa0lVXXRVkF0kAJ/Z9nwZ+h7mUkpI9x+luT2q1WmQDCW6O0o4caPzET/yEJGlpaSlKpBBaBwEonjt9BIVFdEDNIgK2KPHxchtn8zEsOI6trS31er1gPD3i8nQmjq7ZbGpraytSQ1m2HoVH0REIr3GrVCra2NhQp9PRzMyMTp48qYWFhRAi2Olut6vZ2VkdHBxE5I2BIBpH2WGxfeORl3jh7AhUANA4dwRTOoyKm81mCIJHrswvTD9rBQu7vr6us2fPhtEG7GYZUT8lgTQ180/A5e+mz41GQ5PJJPrsgSjlawAOT0lWKpUUA03mamNjI5VexaiydozbFQ3D8OCDD6Yie0Ag8kdAOZlMtLGxoY2NDUlKyb+nJpvNpiaTSZzvjkPweXbw5o4CgwJgpg8YBZoHoMexnTt3Tn/6p3+qW2+9VVdccYXG47E++9nP6iu+4ivCIBK4sa6ehvYTL2DOMITIjKQUa4+TgNUGyGIzkGnsEI7CAxwCCQIlDDHZEXf+e3t7KbaP2nHfRDiZTLS1tZUCsdg5KUnPu9PyFL6XEbrT84AcMO0ZofF4HHcO+AZX5g7H6hllaobJRpAR9PPr0VNYYFgxsjlzc3MRHLCufgyj63XW0UE4uXxgs915s+4QEIyT56HL6NnMzEwqEPLjR7HVlNeQ4YFo8uCH97i98QwOc+q1zpJiXBA3+B/P+rNZGRvmNe8OqgC/+BIvYyXw8SNTma/j2ADQ1WpV1Wo1SB/p0Offc889OnfunBqNRkpGpKRMl8CSDceeoSMAwe6gMy67krSzs5Niv/f29oJ8guCDhOBZ6JzrLNkoP03J73xw3SaDX61WQzcgsLAhvV5Pm5ubOnfunHq9XoBULx+TkoyK6xZ6y3v52wNXz+ai651OR+vr6yn7xfe85J35wL55NhQ/7FkRx5bItAdgnoH0AGR3d1dzc3OxvtgSTmKirA69AK8SKBEEMD7PihE4ME8eiDhh7XstfH6xRY5VWeutra3wL9hPL5UDLzh5srOzo/X1dV28eFEPPvhg+BjsFe/x7JJvCmednSwh8KQipV6va3FxUY1GI6oNJAVx8lDtyIHGt37rt2p2dlanT5+Oi9t8/wGDQFD9b4SNzu3t7WlraysmjFQgoM8HngWAGEsYNK+3QxFguHGAzgitr68Hu8QeEWcKCQp6vV70od/vp9JgZDI4Vu/cuXNhWDy1hcDDrOD4CbCov/N9DxgYAhY/vs3rKzudTsw3Dgawy8kLKBhAqtlsxucpd2MOfMMS40DhnInF0GxuboaBkpQ6BxzD2Ov1NBqNwrD6aRysvzMkgBmYy93d3ZAfsg8wEgAOHADz1el0UmU3fB8gUygUtL29HTLEejKn29vbAfIBX/Sb/2NkAUnIF3WSBKXMDyACWWW+WCuvhUTZMXIAqrm5uTBoRz3p4bHYnvCEJ2hjY0M/93M/p9FoFCUgsE5kBAD3UrJx0fevAICxFcgScsVaO+PtWQTm1rMCzo4h15T+oBMYcd9M6Se9IG+sW7lcVq1WSzkst1XIvQNQd0T0FYeMcy6VShGoY1cgVTwDTD05OoYeeIaNd+EMsRvMgzOUAFe3K1JS3sOYPdiYTCZqtVopkODZOzIC2BlIDwgWL/+q1WpqNpuRlfRjRtFVxuq1z8w1v8O3oJP4HS9HQY54dq1WSzl/5mB7ezsVfKHDntXMzpPrMGvE+pXL5ajjJ2gBeLBPDpvKiTAERPhH/iCv9IOxIdfHsRF8edaOcY/H4wiAvRSa+zNYU1htsICXOoFlsEvZjeAux+4DHTBDHgyHw1TpNjqEHJE9kBSEBeuFXBFUElji3/f29gKbgG/29vZCDgDL6DD+tVarpTYOS8nRrB4UoEtSUornmQb6xe89IPDvOHjljwdY2GcPfgnOHdyzvuhr1sf7fkrmf2dnR8PhMHw8feLnYC7sIDaIQIc/ZGU8mGG9+Te/I/AhoGUMkBmeOcFv8B0yOtgiggzkK2vPCJbn5+e1tLSklZWVWF/mFD+4t7enXq8X2S38CjiTdWEewFtgLbK758+f19bWVqzlNddc85A6e+RAY3l5WeVyWTfddFOqBAWmikVAaF2AWWCUhoVBsIj0PRXKwuF8PUoku+GMtztGL6HyMitKYlyRer1eKDvPczBSqVSCZYe15/s4hcnksEzA905ko2KCCZSFwAqgv7u7q42NjTAKCDzpRIQLR4HwlUolnTp1KqJOLvdh/r0WmueyZm6gPCXrpRtE+V6igaKjZIAlSVGzyJ4Xr1f2DdM4ZZRhf39fa2trOn36tLrdbgSnHoC4YcPAowiwQDh4HDPriOHCsPlaSYp9Fs4QM9dE9vQXppKyJ24r7Xa7MdfIOo4B5hOGSTrMviAr6JKvAxkg+uprd1zby1/+8mCSJ5OJ1tbWtLW1pRtuuCGcGGsC84ocwVSTFZUSffeg2OuDkR/mzME9MoOcALD5LgE4a4qN8eDUSRWyBC7THBtIaQtHK6KPvAO5pY+esfCAA2dOih5HDtsJ8MxmNqRk34XLsRMs2BTfD+ElIV4nzjow904Y+D4GQAmfp6/YZfakMR+sJ2AY0ALAxjlLCqDtJVQePDBnrIcDHvrh5QTYWNbVSS72pXmNNn1mHXDugEAHQqwDcsoceTkE8+MBBOCP4Ahfie8AkHhfnWlmTIuLiwHQPYN6HBtBKrIFsGJeG41GlKzNzc3F+rjMEagxt+zxIYtNsMIfdJAyFvAOMoB94fuQUOAND9KxGWAQQPf8/HwKA0GOZPcqQIbByBcKhcjuMDYICSl9SR3ZECdXqdEHM3nQQSkXFSfYTtcX5InMgM+XpNBr5gj8xnwwF1Ky+doJRWTbSxb5HMEi72GenGj2DABjgqRg/nkWczwzMxNsPv4essCz7rwXufCADxsAeexZGcg0L8PibydXyDyw3wb7Xa/XtbCwEPszPDBkXRgfc8b7sGEQvtnKCnSCYJd+U77th90UCgW94AUveEidPXKg4cwYC4/DAsAR6Xt6jIHyO69XdqYcYQTkA6xYTDbGUJrQ7XZVq9XCIRL4uFMgwHDwRj8QFGfRYAxarVYszuzsrJrNZoDdSqWi7e3tqNn3KB4DBWBASXCYMPAAbtg55mVxcTHGi6ARqGBEAPwYERhTBzEoLCwcgu4M1vLycmyAIyhhAxxsvtei8i7mijH6ptqskDMWlH5mZibFbK6vr8fcUSeJQ6C/OHQYK4IdNxBSEtyiNPQXGQIAMm+sCZtz3eAzbjbTwmpOJpM4dq9Wq0UaliDi4OAgnNv8/LwWFha0tLSkubk59ft9tdttbWxsaH19PdgojJ6DYcbkGa1er5cqmzmujTWAHa7X60EaZAMoDLxfFuQ1pQBonC1khNe3kgnB3jgLiZHHyWDgsWv0h8/AkDtAc0LCMwusK04AYOcbnL1+l6CD70oKPQLAeirf90SRXXObii2U0uUi/h1s5NzcXGSP0GEAFk7O9ZGGH6Cvbmex/QQZ8/PzqUuqAFMAKi+/8vVhzT2z4lleQCLBkusq4+XZOHKe6UEOOuiAjuxONtssKQgdGED3e052ADzp+9LSUtg4STFPLjf+XuTNWVxJYQew3WRb2f/m7DzkHu9ElnnWcWvD4VDValUrKytB4jhBCAnghAPlxLVaLbJu6IoDTeYTIoksN7jH98q4z2BtkC/khRJhnu12h0AHH+i2HRwzGo0iwEYWKIXy0hv8MMCYgIU1ZpPwwsKCGo1G6APAkb57ZpOMTLfbjVp/970QvwDrbOkvz0DHs1kU73+pVNLi4qKazabq9XrYRzAZWUzWoNfraTw+LGn7m7/5G62urgZW8YwBtn88HsceEmwEhCb+w8luyB/IQ7IG2AfwGIGs+zCCLjKSBIZOdBLoUlLv9o7PQMaSrWDPrH8eLEx2jLJgNqVLCWmCjPof3gmJhC+gkod9Pb5myK/P5UO1IwcaniL0lDkv9ItBOAEJ4+Yb7/gZCo6BJspDmAggSN+4c2GwvnmcPRa+b4B0NsLkG3cxuv6d8fhwPwLRomc06Bv/bzab8QzO8GZOANaj0Sgi41KpFJe3OLiQDg0/m5Y8ovUSJgwPjg3nvLOzo9XV1eg/AIfoH1A0NTUVN2NiBHC2zC0lDCgx42aPBuwyTg/jQtYEBXJDw36c6enDc8kbjUYEWxgeF2RPITrLidyhPCiQlwpgvHG6nmlBbtzoOzDCiCOfGDTGIh0aw9XVVZXL6U3tyCDggrXn8iZJkap1R5gNvD01DaAmkGGNAEfHuWXX0UtOPCOEHWg0GjGnMNTIvmcUJIXhx/nxc4yjpDCy9IV/NxoNNZvNYHvIUs7OzgYgxGECbD3YZe09s+EAFpBN8ETgQODN2LzmGlBIyQc6407Fyx7pr2e/fJ6wI+iJzx92hXlvNpuxFs6OM14ALXMI6BqPD8veIEacOQTgoqOS4vfYYoI35sbZP2cz0TeCNCk56YnnYS/ZG+ansDAebA/rgFzw7hMnTqhararT6ejBBx/U1tZW9AH95llZ+0Q/AcLIGoEw88u7KPv1ZxKs4ruYc8bJ+uGLPfMGYeKMMvqA/zlubWdnJ7Lm4/E4dc+Fy7+TCE7AQVo6AEYnkW/2IqFfbpdYUy+Tc93Gt3Cjsn+ObIgTePgav1iNIIS/vSSIsXIrOnrr2Mkberm/v68LFy7ozJkz8TsyI+Alsg3ValX1ej30ql6vhxyDCzy74tUsPDNbjsPcevCBHyXjh933ahY2daPPk8lhKSY2bH5+Xu12O97t5dPYYdaBf4Oztre344LB7D5gzxajf15+SNCJXFH5gN0kC760tBQyKikAPcQEdph/e0mdl6jSn62trVhH9o8h7wQ8EGxeAoaMe7Yf++cEGjiVDBfHA0sJEQWJ9YhvBkd5eRkKwt8IJ5NVLBbVarViAJ1OJ2UomWiYMo54JAigZtvT96Tcq9VqatMnE8jzUUpSfbBe29vbOn36tNbX12NDkKfQAZBez+0lWPSJjdQsKBPuJVsogZdZAF5IpTrLRvahVCrFhtNSqaR2ux1Mv5cWMCayN6yBGyPmHsV1BmY8Hqvb7UaNKorv2SZJkXZ2pwmo5nIamCRni5iPhYWF1J4e+kXqmb8RfubHs2T88VIH9iuQ3QHQzc7OhoH38bNWXvfKuHxPh5cdeMDj5XTD4eEJJGxmdUCIY/DmIAmD6gbNTzDxMhJAK/pGqdZxBQjS4bG8btT8hBbmndNeslke/8MaIk8EcehsljUGdGH8KUuRkiwL+oo+IVdSchqHl2YiD26s/Weesochw67wDmfUs7YFFo7NvswZ2ZNs6RNAHntCoIEj87JMvg+wIRhyhweYwLl5KSLA1UEytpz1wBYxDx4s813WkXny+YJIcBvgQVvWQUKUlMvlYBPRHUnxfewXskXGwlk7fu4lkoBTgCeZJCkJHGCgPZjFhwE0mC/eCUEDMEAHmBsCFPpL/7DXZJQBoDCV1HqTDWOjPkDR9yQdp+Ykmmd5sPkQBFn74Bl85hYsAaHBfirsvpScuIZeI6PoipeWdLvdAOOeQfXyPkAv1Qjs1XP5ZpMvWYrp6eS0NkgBiBFsFH4hi8+YE/aedrvd2N/FH9/riAxSnglBKylwh2eJqRzhYAI2qk9PT0cJIOOiX549kpITQvmb/brYIUqbWX/P2i4tLanVaqUysQR1YAYpOSQHth/ymWDTS54gV3kOMoNPbrfb2tzcTI3DiRvWkvKrxcXFsHusFevM/z2bBMYhmwY5S2DkOBtdwBdQtgfhRZWO79VyH+FlrmSF0Y9+v6/19XW12+1UySBrSyD6UO3IgQbCDXBDwJxNhhEm3UUUNRwO1Wg0gilPdeD/F1IUd3Z2Ni4CmUwmcewY7/dTUpgwJnx2djbqyAaDQaS3UMhCoaCTJ0+q3W7rs5/9rB588EFtbm4Go45jgakmLYVxAUSzGJVKJbVh0Z01ET1GzFkVSmswEtQYwpxvbW0F8Lhw4YK63W7MM0wgR6C5wYAR9M09pOecwcrWrRNU+GYnKdmEy9hxjKRvYUEIpCgFIp3tm3edXQa4UCOIYnGKmGdlYEIA/VnF5jMwd8wJDoTPuqx6eZiDDgwnTBLBC+8F0GB0CIp8M5mfYIJB9bVzVsLZUy8lyQIvB3XO3h/H1mg04sQ35to3vJJl82DSM2WACMC2lDC/BBnMYTY4rVarqTIUz0Q4+4Vt4VmAV2TIyxpZQ9+/gI0EWNDYT4CeE5jybmwKtsKdM/308iEcC+wp4MRT2swzzgeg7CVpBLketOAsscnFYjFk1hktKdkIylo6Y+l2VUruFMFmeFDizJwH4JRqorcQCdgrMk4wwl56hs5xTjz6znziTAnAsH/Yf8onHFRJStko1tkBhjfmmCDSj9tEDgmGpcTX4jsJENzPwqB74ARzToaHzD4nwyC/2CXGctza9PS0rrrqqlQNv2em0WsCdwAm9gAATzCxu7ub2rMxGiXHm+MneB42Ar30TAgyS3M8RKaVC//wLcg79oa+g4vG43GqfMVLhyndxV7wXfyDB7GehfGMKf1gHjmoZ3t7O0qpHbRDwFzu+Hr2fxQKhcBE7DEkQPGMAv1zTOlZSshlfs5ner1eijTFbpRKyRHmo9FIzWYzBfzRf9flarUan3ciB6xLAI+MkcVAJsCazCPkBza+1+tpdXVVa2trOnHiRGrPJnMO8eSErpf99ft9dTqdwFv0ESwGtnICF/9XKpXU6XS0ubkZQRay0ev1AiO7/ng51dTUlK6++mpdc801UbIGWcsaHqUd2dLAwHmKGMHEaKFYMIRMKqw3SsqmWLIJUmJw6/W6RqORLl68GICdVLXf1MmCYmgRXhz9zs6OLly4kFIMnCZX21977bW65pprAox3Oh3t7x9e4jM/Px8bfT07gRGA7eN9bF7CcWHI+D+L6+nxTqcThob9ADjgg4MDdbtdbW5upubImXUHBPQDFgBFx4A6eGItXAExBp4aAwRJyUk4XufnhhfQh4DjbKnz4zt+Wg/fJ0hAQTAwBCDOuk4myQVJ5XJZrVZLzWZTu7u76nQ6YRh3d3ejv5R8OfBHQfm917wDDBy0AFb9Z569INAlA4Qx8jINsioYBeaW5zJ2ZA3Dki0dO86lU6dPn06VswGC6vV6XIjF6RgepGVLn3zjIoE3LKLX2DJ3ZLu8dA4Wh7XCuFNKgNxJSRaBgAP5wUlQCsi4aIA7B6joLUGF200vQ8TZ+QkhsN0E1OjIZDIJMoC5Qh7b7XYEAGR0AKEEA54ZqNVq2tnZ0draWsrRE5B56RK6gk3ChrBOzrK7DaHfPIvxwTZyupp/b2pqKrKPBDXYfd+j5SU1PJvfYb/QSQeDfNbBHnoKsKMfXq8NKOR5flMz85AtzWBjORkJAC39or/0AR1wwgZ7iGxiVyA/8DfYFIBwNlt43NqpU6d08uRJSbpEjpzYI4B2jMKcY+/9pMnV1dXwBci9Z/uRDUo9kWNsC7IKoCXjBPAcjUZxtL5nG1gTB/TYBz+hjvczRq9AQAe2t7eDjcffgj/AFlL6Ul2IXjI6lBhRieCAHhnzwAochJ4wD5KipLxQKKSIDT94Q0qOHvd+IvfYAuwhzye7T2WHlxzOz89HFgFsg25JioDByxfZV8G73e5xUhOZISnJTHpQhP2j76zzZDKJYGI4HKbIbM+GspaQtpDd2G2eBT7wwNaDVogv5NHl3kkVTkRj7Xq9XkqnkOuNjY0gtcjCjcdjra2tHUlnjxxowCYygX66EoOmw+54PQWEs2u327GBhYkDSK6vr+tTn/pUOD2PmlAKT4vinChjgWmUEqZYUrxTSjaJkcIkxcX509zN4AwfRsDLHRyE1+v1KOOBMfGyJZg8lBvBA+AgKG6gULC9vb04kQrD4oAKheH7vrfDmQhnfNxJsp+CsTiwRyBRIj7HO0mtEtjgUL10AOXCQMBuEoBkmWh/vmcWkC/YXtK1S0tL8XwMOmAR2QAo0K9snTpy6s7XS2VQWpgK5JIAM5vhAyB4toO58NQ9KUiex1zTXz7Lc0ej0ZHTlY/Fdv78+ZSBRrc5jWtxcVGtVksnTpyIjCYy7AECsoXeS0pl03q9nnZ3d1P10Tgm32eB3YI44HvIIwxQv9+PLCR6hDHGyZCqZu2l5EAIHFixWIxN7tgASaETlMxQmohjQB8hanZ2dgK4HhwcngzjthGQ7pk6gtatra0AStQCw7jicP3eH7c3OGpsAY4OMgfQ5oAYBhUnyJz4e6kpnp2dje+SMfRDHDwowwk6yMDGsualUikOHgDgYfsgBdg/RbkqTtkDAebF/QF7E5EPbC7+BQadf5OVgdxBZiuVSmwuxz4h64yb9/rPfc+gkzg8H0DhrCwgHHtyHNs111wT4BhdyAIuH5+DWtaCUiT8Ib7eySEvWSMzUi6X1e12g/zkWezDxK6wllJSmkmpI9UAEIMEiu472HPqpbz4UXwCfgIbQsYCm4Wce5UJhB4BjIPnbrcbcgp5CM7yUnQ/7UpSHMCBbjNn2GzmHx9IYMJ4kWNISl8nAgX662SypFh/z/JNJhN1Oh2dP39es7OzceIauiIp9nL4vT+MjzFgz6iykJLMB/PopBK6xt407DjkOPbeqzLwRwQV+BQ24FMuBXZiLQn6+Dx9J+g9efKkyuVyiqQmk8b3INW4ZJrxIFf4IErs+d7W1pZWV1c1Ho+1ubl5JJ09cqABgPfgwFlan3SEn4mhPMl/z3ecQfYImXc4o4zSAtBqtVpEjxsbGzHZLpzclM2zmVhJwfbBDLLB0mup6SfvwYi4A6eGl0uEmC8+S0oLcImgwSr5ngs2iaIMztxilHCkDozpD6USPmew7T73zLsbLk/nObuH43Rh5v0c38mzCZAAkjwTAw448hQtAQoKAwhA4b1Uis/hRLhkDVlzJpBAiZI9gjFApKcnHRgyZ85Ce6mDR/XOYHhgBOMjJZuPYUw4IY0MGO9GDzBGzrhjmPn/cW2AeVh95ng4HOr+++/XwsKCbrjhBi0sLMTeBMo/WF8vQUTmKaGhntdZIoI2L9Pc3t7W2tpa6l4eGEGAHECGgHhlZSVAG3aCvpNh8wAb+4EzgM3237nNa7fbkXVFR8bjcWzKRB6wLQAPnoNskyr3fXWAFrcb2GlAeL1eV7lcjv1rBG5OhPCH90LYMC5sRL1ej8ztYDC4xJl6YMV33c55advBwYE2NjbCvkAK8BkPrpylxj7RZzZKuz3BX+BEsWVui7A3gB1YQGw/2TPk2TextlqtFACjDKbVagUopbFePNPBG2SY2zHuTMCeY4ewOzyLrAfvI5tyXAmLe++9NyVzjMtZfCnJLAIwW61W6ohf5APGm7+9BNIrF5wJdiKUMhbP2LEerAFyT6k1/ZqZmdHS0pIWFxdDt8mqcB8GJZHIP+tLQO4bhgmOkW3e4SVbnn0kSKMskfcxFt7HfR0Ae2wxdgDfx/sha8A5jj88mHYSbmpqSs1mM3TDAw8uwQUTUibtvpn59r23xWJRq6urga2oivFstxOK6D+/870ynpkkKHXy1YMk5hhyCBnELvnFsO4LXI6mpqbiBC4/0INN9GAr3wtCRgpf1Ol0UocMeLml2x5wHDbOcQjYnfFtbW2lyJyjtIed0SDtiuNGAVEOTwP6Bk2UG0afkiIWnEFcDowzcEAwQsYJUQiOR28sBiATkMcziMxhIj1KdaBH/1FeImrvr7NYsAs4Y5QIAfYSBGcrYMpRQPoJaPZLnYiMnRn0kiqe6cGcCxrfY71Qco/OWc/RKNkbQ9SLAgNa+Hc2Q+WsEkHazs5O6shPfufAwoMclBBlcQDAWt1zzz3xXsAGgbHfAu2BK3PtGQxkk/fQR/rjZTDUTyM3zl64c/HNaDgCWBJKV2BnHTQzF57N8Oce14au+L8J5Pf29nTmzBmtr69reXlZKysr8VmXLfSDE9AATe7IkQ0yV8j9xsZGlBc4UPZM08zMTASB1Mby7F6vF7IkJbeWezaL/QMcSTw3NxcnvuGYd3Z2QpYJZHCs1PLT5729vTj2kbGfOHEi5ApHiN3kufxsPB5HOQXzwb07zryyd4bnHhwcRNqegx56vV6AA5h+D5hgYiWFfcFh+j4qjion8zIejyMI9CwvwA29c/bZ7SGywWfYo0Wg4RkG7ATBD86ejaA4aQ/sWTdsKHaaDbDYG+QRlhg7QvYaGWPPCMQD8kAJiwMfAkvkGDYZ2wixIyUnXSE/2DiyNpSbVqvVAG/HrUEkSAmZliXX/KAOgrPNzU2tr6/HvBIYorsAMMqGeCZ4AB9PaS5ZLIA1AZ9fWolueWUD6wYGuPfee0OOXHcICAluISE8WHD8xaZr/mCPwCXYMcZO5QF4AAyAH2TDtFcdTE1NqdVqBQGBbJHF5N++WZmgybOFjlfADpSVbW5uxnx0u91UGT56gX0DO7jd2traCh/JmJCB4XAYBw+hK8yJ4zF0jv7A6GPvsFvZyhveS0CCDmKjkRevFgFnIIMEj/V6PTK6BCm8q1qtpsgn5J3MEWPjc8hUNkBk3ckq8wedwgdlsz2TyWE54cLCwpF09mEdb0t6bzAYRPoQw4iTgKmGaSHFwwLB7C0sLOiKK66Io7r4nTsNjISXpfB/BC2b1vHaOq9t9hMpAK04Zo/K6YPX2SGgnopmwZ1VwwhJycZonBP9crYARsadKIIjKRWsAepRfFgWnLIzil6yRLBBWVeWdXUGj5ION8AANwSU1CCA2Vkb3s1nneFnDmEIPRjx08LIFjDXXjLjTC5zzhyXy2U1m03Nzc3FCTEO3pA/Z38xwvSZdULpYc18gyvZKQI95s7TusiKGzovG2ONCUK8dC0LaDyY9BT6cW2eFpcSPcfg4iQBx5QD7e/vB9PurDzrvrKyEkDK9xwhcxh7NhLCZEsKvUTGmG8AhxMdfiKWs+Y4doDx9PR0yuG6vvIdD3QBxl7awFiKxWKk+geDQdRQO2vpOs29F5ArbgMBqv1+P+XoOD/dSZT5+fmU3pNxgfyQkoyidFiP7bXkXl++trYW5WOevWS8g8EgVYrUbDYlJRkPP3QDJ0fALikAIqVX+/v7wfp5IIRPwgc4s0l2B3AkJcQQeulAEFuBTyFbC8OIDxyPx7HXD5vGePgZz3R7CwuOn4L08TJS9x3YT+xTqVQKMMIf5AKbfBzbuXPnVKvVYiOrg1DuhqnVauFHIIPQRbAD/qFer6ter0cZq5/+hd9wht8zfLD97O9kvwA2wLP62Bz0H1JpZ2fnkhOMnCRkTxD3X7DuyLaX0GSZaSdCIb6YE2SYwIJMKjLlJ9NhgwChnt3jTjHmCvLS9RR5hKDDPzqT7zpKgFGr1bSwsBABP/7QsRaAnn2/6DGlmOAe9IwMDhluyAreD1ZCr5kb3xvnthw76j4Jm8+zwQ2MmxJZxk1JFOvre2DoNwFIo9GITf34Gy8txXYRUGD/CXbBEJA79APSDxuLveDwCsY8NzcXQQ3y+lDtyIHGeDyODSwoiaTICnDZFFEUmQJ+LyXAnEid+yqq1WrszIfpwmGh0Agu4BnH6orv7BVZAncsBwcHcZoCYNo3ybjxYMye5mNPAIuOcuNsAA5uAAhMMOooD7f3Mj/ZPRA4YowiQAYgiyAA2DzAyqb9mT8/7oyoHoMGwPByJQyjB0oYOZ8XmEAElrXyUi+MHmwzn4HxXVhYSDF8XtfJGo/Hh/cH+FGbkoJdgQVvt9u6//77A2ThjFgnbkHFoSMzzKOzDH7KmQdxjMdTinwHVoP1wfC40ciWaSB3boDdwHgqls8cx4ahcjmTlFpngCUyPD09rZMnT2p+fl6nT58OI03wyv6LRqORMpacNsfR1n4zPeCE77O26ALsO2UJUsJSSQonxD6P8Tg5NndzczMFjHHy2ExnkgCw2FApqQPGBg6Hw9QmTxhtz6Qgl54VhP2EgeM+ktXV1bA9yBJOGpmkLI2TsjgSU1LKHgCO3W46IQPwxfZub2+H7SXAIR3f7Xa1uLgY8yUlmSyy0ZQrAio5A58sAHt6yASxlsgZJSRScsMvNg1A5ACMtackh9p8gEmlUgmCAz8EIIF5p0QFn4HfYZ34PT7Eya5isRhZT2TeQZNnZKXk0ALkkVJCQBHyQ933cWzU2zebTZ08eTJsL/OHzuG/vGIAefByRz/SFtYX34HfAcBBijnz758hkMQ/ArSR+SyRKSnAtwf1rCu+EjvlMsBaYg/INKDDjBdygUv3vKyLfuIjYdKXl5fj7goCEL6H7EN6gjk8W0g/eZ+fdsT3kUf8J+PFTuKHwQFgCoJnfHqW8Jufn0+Nj+eg1/Pz80FeeVaCAIQ+edm6H7LBzx3nEYw5CYbdIbhkzT1gI6iDwOJnZJScbIcoQm95jx9AQ9WQE9cEDF7qy7zRX0kR1GbtC1cVIGce1EIePVQ7cqDR6XTCYaG8OMnsxDE5CDhRKoEGG8P39vZ07bXX6oYbbtCpU6d0//3364EHHtD6+nooCM909hkGW0rKX9zgugFFaUmn7e8nx6IxWXzfmSWPVl2RvObOQakbHxyDl2JQLkQKm42JsAxe9sU8+T4HyhwIxHBCrAHGlTHQMJaAcYI7ys0oYwBseekOhtCZeBTI+05f3Fh7GYMDZo+A6buU1AgSGFBT6XNRKpWC4QDcwAKgyO7QWXvP6HDBERutcEyuwPSZTZqUGbBmZHt8XN74vhsyDIAHnQQNXjaGsjvgBXTiXAApx7Ftbm7G3SoEFH76Gn8DhjqdjtbX12NDHwEppZOsLYwRgR56xxnmMI1Ssj7U7CIjnDxHAOPHSmLPAGjojteDk0H1Erz9/f04hIL3wIzjyDH8XgqGPDIP6Nx4PA4Wn7lCfnEKPofuiLFrZEmRUX4GMCJDQoaI0k0pCXodWEjJkdkc3U1/2Pjq5BDvlBRrRRC4t7cXRMhodFhj7OUxBD9eV05WBR3CrjAOty1eZgJzLaXP8Qf0UDZGcODZbdjQ4XCoixcvhp9D550ooh+QVWwKxs4iI/g1AjXmkEAaIoI1dqILRp5MqWd12YdIIMT8H5WNfKw1StW84dOdkIO1rtfrqSPxpWT/CsBSUuAE/LjjDnwcMujlSRzmgX/jwjj8owNIz4w4+EU/6JtXAUDaInfIKKAcnMEhNhCiU1NTUe7uVQCAXWyKpNRJShC/3KQOG+5Yi5/v7+/HyZIAfS9ZlhT9ZpxOsnnGySsgICKGw2Hc4cDBPzzLK0McE0Lg+IEUPBs9BI+BB72aAVyHbLAO2BzwLgEupAT6SF9cP5EDAL8HlaVSKWwBQasT1dgpDiPxE888g+Ry7UGpZ6M8y0XJmZQ+HbFQKFwSXDCvkqLCA1t0lPawNoN7SlZKolMmkAUl4+FAkoVEcQARDzzwgE6cOKFTp05pcXFRi4uLOnfunM6cOZM6NxjwiqAijAgDC4YhZ/G8dhLwSnNAzFik5NbUWq2WupjGo0UEgc8QBMDS8TM2BOIcNzc3oxaRvrGIOB+Uw9ORBBgojwN4GgwbES4CU61WY5MQxtEDCZ7laT6e5wzLZDKJZ3CqmANqKblPIJuKJKqWEjBNX8leUNuKc8dhYFAAImS8+D794aQGmKz5+fkAipRS8D3P1nhaFqBHxo3vYCz5vbNbGCsUmDlAcckieTqWd6KwBDyk2zFsMLQYD+bnuDbYcfTEyz6QddaBNR8Oh1pdXY0DH6S0zPJ/Spk86CWg9KzRaDSKcsvrrrtOrVZLpVIpTnajFEjSJaVPUnKTL+DUQayzhPTL5RXb6Udkup7zLp7hBATvxoFB5Hggyzul5H6QbHmF13hTz+sZIho2lIBtYWEhZJTS1IsXL8ZeBNZwY2Mj3pktTWUc6LFngilT3N3dDZBP6t+Dcb7rzJ6UHNmJLGC7pOTY7axPcXaPOcB5EvixdjhfAorxeBzHZnsWGflgDbDZBFlbW1uxXg6ykGUPiJ1wkJKMjIM19AQ54PREX3PPWsMkH9dGpg5f5HsKsPfoJKVJ+HFJqeOdvawM3OK6y5yxvuidlGTbWDcCSr+TA4CHP+Y9+Alkit8BSNEdtytkTl2unXnGh+3s7KjT6aRKswnMKBmUkkNKsE9+0pSkCCJgrZHVyWSi9fV1SUnVB1gLe+R2yoN43utZCLd9TgagV7wXv8ux2zMzMzpx4oSuvvrqwDeQD/hgiCEvvwQXQFZi5yCsGD/VJoyNcmmfT78fhcNLnADDnmXxhY+RDDxYwbP84KJarRb7kbExzAunk6Lj/hkwoeM0D9BYB+SK97uso2M0ZMzl76HakQMNjG7W0bMgDhAcmHoU6cp2cHAQoGt/f18bGxsBykk3YThIQTvTQ7YBhnNmZkatVkvXXHONTp06pULh8Iiws2fP6syZM9rY2Ajh89o96u8wJAQrMzMzWl5ejgCBUgo/0YA+0K/9/f3UkZMHBweqVqsRaLTb7QgYSMNjyEj3ea04c4pRhJkjW8I7cPJ+sgzzwZ0h9BF2BgFjXG6QcFyeZvaMFOuJIfDoGofO+wAypE8dxJNZgVmSlNoXAStJoIVyA9K3trbUbrdTjpTACmX2858rlYpWVlaC5YPFkZIaeVLh1Oru7u7q/PnzwSJ7kAmgcFaX8aOoGAn2vxAsTk1NxYlosGAw8iizy0bWCR7XBrAbDAaq1Woh96wPwRkODOcEOEJ+s4GFl+dhZwj2JpNJ6hQUZ5r6/b4WFhZ04sQJVSoVLS0tReaCEhhsFOUoyEY2Y+BEBgadMTm7TZ/YEOzn4aMf9G9vb0+bm5sh09hUPzxCUug8YAf9wbF5La47EGTKAynPvMFwYo/Y8DkYDC7Zd8HcSoegxI9lxPHye+wH88n6enCCvXLAQjbcCQ0PrHCWXGDoQZTLhp+X71memZmZ0DWcNplvMprMIYE//aTvvrmSdSULhxxhy3k/a+BAC1siJcEt+x/xX8xhuVyO8jbKrLxeH3DuvuS4Ehbs32EDLDYZ/ZESfeD37NHEn1JCQ/kkJ+sAFqUk0JaUAmWALg8s8XPZIIfmLDO2iVIlZApf66DW7YKz1U6cOCnj2RPPECP3vMtlExvBARGQIARkBAHgC9/U7EQzcyEpMrTYaPoEY+9+zkliSrUcgzh+ZJ8a9q/X62lnZ0cLCwsp4sZ9B/1hDvg7S34wFsgDyD7fT4We8z3sGwFBoVAImWJ/2XA4DB0lMCG4xQ9AJFSrVS0tLQVm8wCSDDF9AQMdHBxEOTIkpmNlqjnc1rgfZP0cr7G+XvGBfaIckOz/UdqRAw1OyEFpeKkz4ywoi+alNkTtdJSjaVkoSXHNPBPgpUGcBMNCUZ/JpTIYfc6zZnGvvvrqlIKUSiW1Wq1YTC+BQUi5Wp30I2MkJehRPs6XCLnb7ab2mqCMOHWCDQ8SHJwjXNlUJYqOc+F7MPoYTRxQvV6P2lMvw2As/X5f3W43HB+RL2lHBN9TY8zN4uKihsOhzpw5E2U9Duw8CPVNqwRhGJJ2ux2nLyE3DtZ4HoDPU9n+DmcAMMQYNYxXq9XS8vKyFhYWLqnVhE3lEpper6der6e1tbVgJ2ZnZyMFj7x1u93UzfVSckQtxt8ZXYwLQUeplNx2LCm1d8HLG9APjDh/H8fG8bFkocg8wdS6fI9Go5APZ5FZW5wS8+QpYUkphsodHI4TBr/X68UG08XFRdXrdUnS+vq6Njc3IzPptd/ZckX+wHKirzBN1IOjT86UYyc9gyMpQDh6jqPh+YAKSBscGXYM+cLWOAgG+MO6UXLhwTP2l5IwTq6BFMBuARI8M4fdv1wWlvmXlLJzrt/O8rqNcuaZtXD74llQat8908h8SYm9wAchV2QGnPnGB/FM3kn/vORKUgQt6DLj8SCafnmm1EEYMoDt9iASZpkSENYIGfO9Gh60MJcexBy3trGxoUajob29PV28eDFV5oRMMGbAqm/0d1KROeOiYC+HQTalNCDm/56xLhaLcfwrMp6t7Xc54d8EG07YodvIOX7OKwj4v5eKScnePrAOR1a7f3KMgV9rNBoBbrGXvi9CSvys7zWB2APk+r5c/NrlwD/YyN8FAJcUftE3lnu/eT8HhBAkUWLuJAVz6AQKWQSfTzIX4Af67GQSzTNA7qfAWsw1NsMDft9HSOUM4yPA80wVQRuNQMDtKPJNP7FdTrDgR7B52Axkj3d6GZ/PCaWYyKbfL/JQ7ciBBjV9AAEmEifvkRLKzR+CEoAE3/Uoy1O7Lnzz8/NaWVnR8vJynK/P+6kbxhF45Om1cIPBQPV6PcUSeNmUH/XIfgW+6+k3FoJxwmxsbW3p3Llz6nQ6Ad6lw9uOS6XDK+A9RUWwxfcxMBhKFhs2zVOZlPw4mwJTOjc3p2azGeDE6wwLheTcdb/gCIHkWV7u5nsXEGw2UmNQUEiUjjH6HOMcuRiGgAZ2EuNWLB7ewMmcUPMN20M/eD6GpFgsRtAA6JIU9flzc3M6ceKEFhYWVCgUwjjh3Knx9mOOYbTJUlEjyRnf29vbUTPKEcoYZU9Pwp4yH4yV8kGMLw7LdQ2AhiNhrMc50MA5Yei63a7m5uaCMID99UAVozcajVJZH2SOOXHD63XJDi6yesNzh8Nh6rJOKdk3hD0C1LkhdpYwmy1AB0ejUWpfm/9eUgSf1D2vr69rbW0tlQHxWnLPLuDw0F10kmfjrHH2gC8AtNsUwAJjcYLGWXB0wc/mn5qaivWkH14CJin1bkA4AaPXUzvIcZKKjCPMIYDKwZWXs3j2ARtCw7F6uZmznMiqg34ngDyLwTh4ZxbYENRhr7D3sKDYDDJv4/E4yiRgMIvFYiqr6YCYZ7A+2Xt4PGPiG16Pqx05f/58HP6A/gPo8Fl+ypQHCawz+oh9983K/NwrMMiIZDOBvjEYzADAhFAkAyGl72dhLXwvJoGgl8J4bbyUHLYgpQ+W8Ey8YxjHa/6nVDrc80jFBVlC5pR+elDqZIlnG/GZgGkvrYHk8QDBAzXkGRwIMOf5yC3N/SD2GMIPrOfgnrlyIsXJO+beD09AhzldCX3xjLvbTZ7nGBH9Q96o1nD8xhqy7gRrTkKDoz1YxbdgW7xiyIlo+oTcYtP9AAIv86Y/bvd9vwvj7Xa7cQrhUdqRA41sShuFRkBobpzpvNeV4oQQWne2sNlej4YD5ig7jDnAgGgLgAIjjyPwCNUBDLWbnq70CJCAqtfraWNjQ5JSm6JQ9qmp5DQX9mtQZsONtBg+BMRLnCSlUplewgCrhjJ7FO0pPXcgnN3vrD8CDYj2ekEEXEqf2uWKw7iZT2peV1ZWIhBA2BF4DB0GG+FHXggI3cjSZ4wbJ4vAzBI0OtgEQACi2Ixar9d18uTJKImSpAsXLsQN8YB/T3MCGjywwWC5AZYUpV9+oZuXI9BP2E8CTEAi84HOEBQyf56lQVeQk6OyCI/F5qUCOPWDg4M4qYmz5HEkgFoAZr1ejyDQU/roid8pMT8/H+WZgDt0dmpqKk5icVY5m3b2e1GokXZD7kGQBy7YI0C4l07wDvTEWXPKttif4yVYfJeMpAcwOIxCoRCECtlOQI0HtH5Kkm82hllk/Dg/5nl2djZOCsS2AUIAeowVcMDRo545dtYR+4TdYg4JynDMvIfvsO6+h8kzJ1IC7Dxrik3zrLDrVjaDyJpn91IAEBgTfgfCA32GdGDN6d/+/n4csoKt90weNt7L9gCg2QDKwSQgm/kFYDiJxXiPY+OwADAFuuI4gznwDCTA1KsrmFfIQAfDyIeTH/glSCMvyXLWnff5wQ5Scjkc76XxPfQOucFfoLfItaQA5YBXDgRwYsUPKIBU4B2zs7NxOAr6g85jp+kH800Qhf5DPEqKUneybfTDwSi6T1CEvHrmwMlIt7u8J5tpIkAZj8dxqh0X6uIXCIAA5k4KeuUIhIr/HBIBOWBNfHO6A30P8h3fYlewGVlbxXqDtdFpAL8HY1TXYJs9EPIMFHiZeRqNRkHygDmlpCQcbE1wAU7CzvNsblw/alb0YR1vSydwyr5p0gXYHQ/lPMvLy2o2m9rZ2dHFixe1trYWmwjr9bparZYWFxdjVz0nssDCs1ETgw+IQ2kBFgAIDAGLyWIgoJ7OZ0E9LYqg4Hw8qCLSXVlZicCoVqtJkm688Ua1222tr68HU86pFF5HyuL73HpfqKsmcqUO7+qrr9bS0lLq8/Rpd3dXq6urcZwhgYVHtJQlcZwiQgaA8WwLyuOBIwaG/3sWwE/o8TWgn74Zlt/RYKsd0JCZmZ6ejoMC9vb2dP/998dJL4wLBatWq2o2m5qamooyD97f6XR0/vz51IY4Zyk9EwEw9dKn0WikxcVFLSwspE7uOnXqlMbjw9vpL168GPNK9E+gxMkcLo8ObL1cijkBrOEo3DAdx+ZBFkAcvYJ5Z009DTw3NxcOjO8h++iWM7VeZ88awGihg9TNe3rZy1roh5dM0QAyOEV0AmYV4IC9dHkD4DEHBKWMhc/5Z72PXjKxs7MT91egNwAhL5thPF5+6QEMv4d8wFbye0oQIX2cjXNnh+5LiqCJeeSZHKE5mUziUAx8CtlPJ0BwwAR2Xk4CkKE/rKFn3pEXHLODH89GAEoA4h5Q8A4AhsuegyWCQT7rZI/X7EuJ3fdyKcaYtTueDfFxuix5pgX77cysr5tnc45bW1xcDIANcHKS0TM46ADjZl0A3VKyn6BcLkcZC5+FgMJucKIaAJy19pN4kAmCE19fKQkGeQYEqpdZoS/uP8lgVqvV8BeOA/ykIQfi/gwnE5rNZpCkUkKkgq0c0zHPkAb4sv39/Qj08WecpCcluouckxnG9vB8xzEQdm4L+LwHdawxfXcGP1tijl91MoP5waYw95514B3Mo5et8RzGgdzg9/0gHNdLtwe+0drlFRzLRYiOv32TOuN14g5byrN5Jn+wLeAJ7lDybCnY18k37JIHr494RoPFRihg4lBcJsIZOCbAN3JPT09rZWUlAGs2a8FmyFqtFoaAPRFctuUnOZAqRnkwsDhHGgELgks/cUIu8CwCC8cf3ouz4ZhOQDFR9DXXXKMrrrgijkRFONrtdjC3zq6RKSDTAyOP0YPd297e1urqajC2s7OzarVacWtwpVKJEyeoobtw4UKcke9CiIEcjUaxBwKw4UwGzgzDTAofo5bNynj0nDUiAHwcOwYMp4fhktJ3R8zOzmppaUlLS0taX19PZVAATwQ9Gxsb6vf7IQs4YNbA986gPG50YGvYFIaSEkhwrOrGxkawSASDe3t7mpubi/sc3GhtbW3pzJkzYUS9vpENWPSB+cfAwJJ4nfdxbTDC7ii81n9/fz8uawIwE1ACLgHLUrKZF9lyMoLshgfG0qHObW9vhyxSCuGgzTOLzjh5mQs6glF3x+Spf3TAGXm3jbzHyQico2dNmA9nojxbNhqN4rQhQIGz+Z6doC/MpbOeNN7D+Gq1mlZWVlQsFuM0L/qNTPspb5APzuLiJ2CDHRx4gOPsoQcTpVIpAnkHWV4KgKzQD88EeFDkeoXfkJRyyk5GOQmALGUztDjlbOkN64eMs+5eUuWO24NDglxYTDIY+Fn6AdBBLvgMa0kWkDlhv9lxawAh5McZXF8rSSndhBlGh5gL5nxqaioy4AAtDu7ARwEQCcKdZSbQB2iWy+XwF1lyAXmD2XcbBNZxO4PMOKmG7/NScwIBMAw4yQNpJ9P8AlL2DtJPB9AHBwdqt9uxTxFZZlysA5/3bI8Hy+gDe2nBVvTJQf9wOLzkoAj35QcHBxHMIROerWVsZGIGg0Eqyysl2QUnVCSFLaM/o9EodXEdWV/IL4hh+sccIpdSekO6Z7T5DJu2PQvDnGBvPdBgXpw88KoOgi9kl2fxTgIHbCb+zLEvMsaBNGSXeJ+XtH2+dmTEwiRUq9UQIBwTaRYvncEgwIIxMMqLlpaWAqjDLLJ/oNPp6Ny5cxE5wx5St+5HnLKYTCAROywjqTyMNMYWQ7C/vx+C6VEoi89nPQrGsGxsbKjdbsdYua2Rk56cyWTxAA6MiYhXSgAnDhGj4kCdo0ExoJSTcMIUn8chOzOIQE1PT8f9EJVKRb1eLzZmo2CMEwWAiWSeCC4BCDzbjT5/e9YJI45RdyBDtEwmjOCBMrnPfvazcdKUAwqe5xk1HKuDGyJxQAbvRWkYCxeaFYvF1EWUlDcBfAkwcXgOhClX2N/fj0MOkHHkpVarhdGZTCbBuLoeeUlHqVSKuTmuzfdlZdO5AD9AgcuLg3P0kTVgnpFHDh2AncM24DylRB9906wz1M46ecAMsMCxwejBSjIeD1ZgInHclDoyPpyblwlIiSNAzrFFZG+csSM7QMCFTnoQR1+cCPI14V2+oZz5hN09c+ZM6Cm2C1vr+9sAV14SQgBP5omNsNhIgkN/LiVz6C6g0f0M6wmB4tlRSSkHjW3Ehjt7h0w4K+0li4AlKSmr8v0YfBcZZd1cxhg/dpUgkL57yQWgENAIKKK/+ATKfv1ABfriQaTrGzbpODaOqwbwuh6gt/V6PaXH+EL8CbqdzSL6pnpsDAEDl73iNyjvoYTJL1RD7iGumHfIR2wX7wArScm9VTyDCgX3Vw5u/VQxD7IYh2dp0anhcKj19fWUHR0Oh2E70QH8Ks/b2tpK7WOTkhO1pORmavd/yK/PJ8/nj2dWnTA+ODhIXcLJ+wD16BHBAICacY5GhydMEvh4dYpXLkiKE8f4rt/V5EFnvV4Pv0RFCzrnhwQ5aeABFfaBdfSyNUgybDaHcXDQgN9O7mWbrDHrxzz7CXTYXPwWFRmSUif08VmSAPglSak7izyT/FDtyIGGR1pkHIjwWJBerxfOztklJoKd+bOzs/GZYrGoRqOhbrer9fV1bW1txcJ6egdH6SlllJOd/J6+wvH5xt7s+essXKfTCeXifTyDz1KTNjU1FYvux/oWi8U4W9rBNMC7UCjEefSUV/gdHxg9D2p2d3fjNClAhzOoCPfq6qrOnj0bffYyEdgBlAulZh3ZWElpD04Q4+2GEMNHkCYlAYVH1XzXyxt8XlgzbtR1hWNeAVzUgDoziIB7jTmOFieM/MEceHoYsOnKToDHe06ePKkrr7xSpVJJ6+vrarfbKeZZSoDRZDIJQ8/3SbETPODcmBffHIycZUtoyuVyGAKyJ5QXHtfGkcFS+iJO9NfX2RkbggxnBLOBgW86rFQOj6qVFOyglLDg7tjQYQClg1oMtWeSkDvvJ2vHu73cCpCAg6VWGHvml4R5QOGMUrFYDJCC/LuRx/BjZ9CB7PN4J0yUv9ezd+6wPW1OINJsNmN9BoNBsIdkUnHKnmnwfRueTeJ7zg5PJsmNwG7/KdPwtZQSOwS49/0arndeHuAyxTMARNgEJwIITB248T6e6YGolJSEQpT5vg23Yw4S+dvtrusFYI6Mk7+LoN3tN0E0QDtLCh23Vq1W4xJOJxfx4VJyhwz+pNvthh3A3xLsIwdkGNABKdEr/AMsurPDnjl3wsqBL/aHfRROeuHznX2u1WpxsAzlUsi0lJRWNxoNNZvN2CtAuTAlZPxN4AqG47kcWCMl+2QJ1ukb2I/59owKc4TOeUUEAQyZVUnhY/GXnqVnTqgKYJ8SZCrAGjDOXFA+yjxC5EKOe3bTM83gIEp2sbPYCIJ5L28Cv2C//NREr5Lx/am82/EFcwGW4mfYdWz99vZ2BDusp2ed6Sfr4M9iPfxo9EqlEjgYko4DPHgWQTzkSKFQCBKaUkInU47Sjhxo4JxZGNj0ZrOpG264IYwbN/n2er1ge1z4MH6UNXS73dRmGpyns24eMfNzwDrPxyDjzACrCF32Ip9segsHIyllZJz92tnZ0fT0dKq8wQ2+p/Jhq1BgjNHe3l6ARsrDGONoNIpd/hg+jvtlbvx9nMjjF/+hGA6cmFMcDvsWGBvjdnaNz6OUDvaYH0mRxmcczJszgvwNwOcEBgc3KA1Aheci6O7Mcco4BYI6KQGnRPcYAI7/xJDzPpwLoBIwz+lmDzzwQCgZbBZZOTZ9YvBYOxyTz4EbKn4OE4Vu0AcHuxzpzD4g2JTj2pxFpqEfyASGzMECOgEA9LmFNZyfn0/tGWJd3Bh6uZoHoTgInC2f830JXnuPswFEolOkrjn9zWuS0a3x+PDeFK9tlpJ6asA5sgUryOEXjBn75/IGueDMPbpONsydH+9HN8m0wnwRFGKjHPCTeYJxxAZLCjsjJaCPwMcZZi8Z8dIC+uL3gXjG17NaXhqDPfDAkP6gVxADkCzMFWvrJZ7ObgM8WH8vF3Fby7M8uEV+nX10xt2DIvcnHjDwfveTXqLl2XL6hb/jux50Mc/HrbXbbZVKJTUaDY1Go9TmewAsco29Z71cpzxDybwwnx40ejYJmeMoVZ9n5A1/7WWbbm8IQPi/E3q+DwnZBsvA0OP7s5lEjqdlQzzZc8qz/OAVQCalYm5XkXcvbwZgetkqtoB+0x8PdpB7Po8dwk+SvUM3yNazt43gsVKpXLZkkEwpB+94BpX/+5GuHkzyx0lZghEpYfkp8yfIwm7Nz8/HyWeeEZWSy0CZIwIKAD9kg/s0lzU/HdQxkFfJYM+yZDjkrRNMvAu/Rr+8usDl2P0BdgIcThBULh/uiz5KO3Kgkd1EA/Da2NjQ2bNno45rZmZGCwsLsWEZoMAi4BTICnS73VQKnIgap8Q7+Tepwiyg511EfPyNE3Nh941DXsoBeEHpccQIHtEl/fQzkb10A8XBwNM8XYtjdmfCeJwlQfidfed9ABdnxAC1jMWVS1JqI48LmTPDzCUgCQZBSjYruiN1QcwGF15TSh8ABQQ2vJe0HM/H0PnJHfTbSytw3uPxONidQqEQz5IUG/IxFgDC2dnZKCMjgOj3+zpz5ozuv/9+tdvt1Dn6pEoBjyinAxjeg6Kzpm7YKHtANukTAQ5AjCADYOT7jo5jA6B6ZoJ1dnaLOWIznGc4AXPooJ9F7/eccGMqpZkAanfE7PuCSJmdnY2Tr6QkCyfpkvIFAgo/2Q4ZnUwmarfbUY7gt94jT25PkQeCB8pEnRAAaDtr6llO5g59nUwmUV5y1VVXqdls6q//+q9TWQ4vNZKUAhzj8ThOmHOyw0kWzzBgO2DMGBe/c3sFieCHevB+L/1wv4Nt9tIhfu62Dt8ByCRggvjy+fFAgTI7jvwE9MAIk5UHIHlQhw0EBGUBqgMrAtTxODkxDd/lJTi8j+/Sf38Xd+8gw56BdrDD/EgKIHFcCYvBYKBz585pc3NT5XJyQziEgKTUnkTWH19LFsTBlmdVs2WHzCmlJI1GI/bhOThzHUHWnaCTkgNysCnuR3kves1dXn5wi/cH2UE+AND0ic9my4r5DHtBy+VyBCvOant2CF8LuEUfkGNKmXq9XpyWh70EiEuJH/aKBA/iGE+32w07hW0kQJKSyg1KmcnUeobQ8ZU38Ad6PRqN4gAiiAYyt2RGhsNhlHSx1vgnKnv8hDhkyvGd4zkpOUkSvES5qZeeOjb0dQFfsC6QWmBVsImUBBwe2BSLxdQBBnzPZUhS4LLd3V1tbm6mSmIlhVw+VHtYGQ2UgmCACGl1dTVKSDDugDicrGcRXCE9XezsGoyhR6gECHwfBfa9Fy7gzmQ6SOC5HiAgvKTHnVHiWd5fHAxZCZQ0Gx370W3sGSCQ8lQlCsccwQQ4KGs2m5EN8nIE9s4AkPr9fmwK9eg+y5ygLCip1wYzX+7EON2DfjFXDo5cGTB0GE+EGsDI3HipnBsnXyP6TOqSsgwUgrsYVlZWVCqVtLm5GU4Fp4FBWVlZ0ZVXXqmVlZWQ0Z2dHa2trenee+/Vvffeq3Pnzml7ezuALI7ZDQb64MAFXQEkIpN+tDJzimwzfnSDkjaMHceJugM8ro2aT9d/z5rh6Fh7dNv1wZ25l/bAwjCXXkrBmqG7BCA4ckiJ3d1dbWxsxKksBNq8E30kwJUUATWAB1aZu1rG43Hoo5RsWnWAQ7BKmpxST4AUn3eiwUu0kCV0jr6yN+GJT3yirr76av3f//t/U+U/njHhu5LCgTNWL7ckyPAynMlkEplVgJwftOG6QFoeO14sFuPACgcTjJegLwsK+YyX1/E+B96MF0CCPWFOx+OxqtWqrrzyyjhco1qtant7W+12W71eLw4ckZTaI8jY3aa6baIUmDWBFHO77Nk3z6igB81mM+afUpRyuaxer6dut6tOpxP2h3WlL9nstNtm94/HqTUajQBrTmBKCTHArcyUPnqZEPoDWEMfnfDEVlPmix6ic6yNM9RuF+gP4M6f7Rl8t3noP3qEbtfrdS0tLaUyCpwill1j3kXJru83mEwm8R36hj5QGuUlPu633U6wj9JJRzLJ2cwzso/v9NIrz7Y60eD7UCi7Yk5g+QHZ4D/enw2QeB79pc+eFacPUuIz3DbwTJ7LmPE3+BW+h7yxj82zlZLCfzuxS5BFFtdPIIN0YsysG3IJjnQiyi9QdTLEKyaw005UMR7sEvuCsTH+bP/OQ7WHvUcDRfPFcIeVdVosCOUybuQdOLkTxymwKHwehpL6TF8sUqfUkSF8HqFLSjGAngVwgXGn7oENSgMI9BSbZxacTcMoeYo7G5l7yQPK4hkY+o6ieMDjadHJ5HDD5ubmZqqWm0afAWIAI3fgUlL77GvLuxkbzD3PZQyeDqbvCC6K66ceuMA7UGCOqtVqZBw4OABGEIMDm4UjYfzZDVMYMO5OAEAQmHGjO6Ujp06disCK/TzMFQG0l04Vi8WQIeYYIMa8eoYMA8iYPHvBOnOyWKFQ0Orqalw2eFxbo9FIGT5nlSWFHnjJjpQw41Ky0Y66d4zf1NRUam8YcsQZ8MwdulitVrW0tKRaraZer5dKf5OtINCQEgeBnUNXHdTDQhJ8MBY/MMMDRXcO6I+z+L7JMZvRqVQqIfPolzOsyKWDamdz6SMsIn9YB8YM+4mcQqxAUHgqn/nle4zRj9VFJwHjvpcDO+TZYGeimUO3FeiUZ7rd/pIZaLfbEbxxnDoHk5w4cUJzc3PB3P3VX/1VnMlPX6gDZ508o+T9pX/D4eGt8wRLXqKCTDvrybqPx2O1Wq24ZLTRaKQCDzbZPvDAA3EnEP7Sxw84JoPnwMbrvI9bm5qaiuDLCYipqeRoey9v83ITSTH/fjCElBAUDuR9rxLzyd0TTkIy355FcIbd/Svfxdbs7+9H+aGXIu3vHx4kQpYd4gnSCuDoe5mwYdznheyBl7KZSXw8dgNQ68ET3/FqCKoRwHM+HvrgdppsvONGAjV+5uQB8g4OY7zY3H6/H9Uwft8Qf/teTGwSJLCXsWcz3jzbcQP2GizktlRKdA5ZwlcxV47VXM4cb7HPzU9Fc5Laid9isZgiPz0j4f1h/iirA4tB5EpJhYtniikV73a7qSyNb1XwKoSjtId1YZ87VoSDEhRPF7PYzibiaAAFbPJ18Okpbwy8C9lwOFSn01G73Q52AUCAQJJ2Js03NzcXm9SJiul3FoRnBUpKjILX38KUeAmGb1wHYHh6DsaTU7tIj7GhGIEEcPrRjjiStbW1lHAUCoVgKLyuDobOy8joo6fUAGQedLkjRFBJF8MUeWqUfsAOk8pnbkulUtQNkoHJ1ieiKFwGtri4qGq1qnq9rkajkTq7HOEHsAG8XDYeeOCBYJBhZkqlww3aGNazZ89GH/zcfkqpcCiDwUAbGxupQwoYM8YTpcNoIDee3cHYoNDLy8taWlqKtWZjH86Sk0Ykxa3zlPlkU8HHqXHhnmc3YbeQVeTWgSb2hPnEgXlp1VVXXZUKgL28plgsBkDAaEuKzaIYUoA/zhVZR7+QQ5wexhZZ29nZSTFq/JmZmUkdksHJb9gK3rG/vx9BNIEDc4G+ECxAhKAfTtDwTg4OILXearXCoWxubqYAKrItpU+NIwvpLC06z/ewjTgwnDHAC3tNIE9mFlubPcrZHSdBnRMHfrKYM4nIhgMe+uz23QMp+ry6uqpz587FEdmQGPiULKuJH8Guucwwd/SLecY3Mj78hYMXfB2XZdEXjtvmQIULFy7EcemAEJ5JPwGwlGThszx7ftzawsLCJeUdniXwchTXR7/fwKskmDv3dfhp6VBGOLjEqzk8G0E/3LeiE9gUKclYSUm2wzOwUkJc8m+yfcPhUGtra2o0GlpaWkoFFY5F/KAHSAEnHxz/eHZDUjDeAGP6S+WEPwcbxR8nYJBd7uqgcoA58H+DBzzTh21hzxjrir1mvxrrRsOOUkLrZCd2G2yFfkIQ+4FA2BQnDxyLYl8JzlwOWG//PPMM7mPupMPAlsAJ/+5YifVxYhw8zbwxp57hlhR7KFhLbC99we56wAlZT9aaUjkn5PGDRz257mHdo+HHNPIH44tiU4uMgUU4uQsDZi5bbyoljC/Cw8D8yK7R6PBoPpgi2HW/6daZQzaC+4Zpz3LgQBAuGoDEmUsEHYavVqtpYWFB09PTarfbOn/+fOzg95IHAiyEHEABQPFyECJ5BIqLCzk/HaPB3hDqihEwd2DZgNCzDT4+giYPAgiOMLpeVgGbz9oyRj/5wsEXZV0YKRwtpVgY+iyj68YAGaSeGeXg5vb19XU98MADKefL8ySlom/fB+SpRme/2HxLfaL3jfllnnA8nkIFZJH9IpBotVpRCuWMLkCRk9mQFxQeecKwHdfG3iQ2JXu5h6TQ2ezaseZZZ4RRZ91h3vk+TKSfFIJMoVM4bOSWYB9w7QE1tgMGD31Dhr0mPhtwwC4760af/GcOYLA3OGpnFQnY2BvHGAjUvvqrv1ovetGLUoz6i1/8YhUKBd1zzz363d/93bDn2UAD+XPbubu7G3Xq6DD9zV5A5tkUSIB6vR7jcABD4MRmUoAYdsCDUtc5Z1t5BkGUgxnK4mCpCSQ5KXFtbS1+zkEUzlCzngB5d8oOapETz/JICWPJe7GtfA/bDUg6ODjQ+vq6NjY2NDU1FcduojuQOewBgqTxAJwA2fcE0Ef/cxwbp7axDgSE+BLmErn1gM/ZW/TSbQXElu/j9OoFMBDB4fz8fKq81wEzvtdLpPwOGHyG61ev14t9bJQFUW6HzUJndnZ2An/Mzc2FzXLfR1aBPnulhJQAWC958lMu8V2up8yp38nDe91G83z0x7N/LuveP+Y8azvRSw4Q8vvQ/A9+AfuI3ZOS6hj3AzRKPsFn+HCAOcGKlJTnccAM1Sn0zzPjvIt1poLCAwjW3jNC2BVsCrYNrICcI19ZXJXFkVnSGgJLUsoXIpvMk/sx5slJG//d52sPa48G9XJZx+sZjEqlEuUfnrIHjM7NzQVgByh7rS8ZBE8lA6aJVDEkXGyzubkZJ0J5et7vkcAIMVE4QiYWIOORI8aAheYPQsniUNN7zTXXxI3nCDH7VTythxAtLS2FkpM2A+zjcHd2dlKpS4SJ+eHfHDGMYBBxExDinP1kJaJ4P01MUpQhVatVLSwspO5tABAgnJzhLinAHLV9ftIXgZ8bP9aRMaOM7Pfx9/kmaC/p2t7e1n333adz586p1+uFofZg1gE/QR5r4UaIgIb5RsYBcSiaOxJJqVpM5Gd6eloLCws6efJk6l4VAjJkG1CEPLvBcQOMYaDk57g2L6txI+UBpZQYO89w8nNkhvXBYcHeMs+UMAE6PLPEs7KMmDPQBC4Eycic94u+u+3wDI0zaBjvnZ0dbW1taXNzU5KC6KAfXjqGTcTplMvlOJefdwBG9vf3I+N4cHCgj370o/qf//N/amdnRy95yUv0ZV/2ZfrhH/5hTSaTCE4gAiA0cC5ZgETAS2aT7Cl2kz56+RP6xlr1er1UYOOOmKADubgcwHfm2bMeHmw40eI2AFuJbXRSBVKBZ1NS6hkZLy+RkiCI72CDnZ0FnHBpoZ9Q4+Uh2bp23st4AQasCcch0x8AqQenyDtg2MudHVgfx+YlO8w14/cAwu09cgVTjkxhe8nIc/Et8onv8b0XyKWDO+bWSRPsA7oLOKbPvieMPVkwyPhBL+HxiggqSDY3N0NW6B9rS8CEr6XkzglOl2kPDvgb2aS8y3EQMp5dG8/asVcN/+22gveAUegb/fTrA7LBHtiC9fPsERjt4OAgTt5inOAJSvk51dHthusk68d4vNrFM9vopuMAvx/J8R3Bppe3Ik/oPv0F02b3iLJe7l8cG0rJMezj8eEph9ydQ4aMtXAyB5uALSYj7kEzmT1s9lHaw94MnjXaGEcmm9OEnMFnY7ik2EdBOZNnFwBh8/PzKYaI96IYKBKpNQw3YM3TXjg5jAyghAVhXJ5ORzj4P8rKhPP8brer7e3tqA9HmZvNZgAAomIPDhYWFiJgArBTEwdLyftQoFqtpsXFRS0uLkYJEQbDGXNYQ2rFERyEkJIgbh3P1l3u7e2F4NIHvySGZ/X7/bjkj/UjK4BiERhR4sD7KJkg4+OGg0bASmmRb6b2TBM3bnNjOpk3+u6lHzC7GCSUF2ftpVzOkBCQOdvK53wNyHItLi5qeXlZKysrajabMSfMERf8UL6AXM/NzQVz48Yap9BqtaLfx7URcAL0YMV9AzBBte/TcFLDbY+Xp3nZlP+e53qAiiFn7fkcgQJkBX3wjeXumN12OFuIDvn582RqsGmAQpylz5GXEfnlfJAGAGgpOZIW5h5iZjKZqNvtBpkzGAx04cKFeBa6yT6ohYWFOCbSGUxn2arVaoBf1oY59MDY5x8dzIIy7AENW+f2x207IJ53AtRxeMyrZz4gOXws6HO2r/QJUgb75T4KX8fckSmFCSarxZpip5yN9Iws43ZQ5e+73P4Wnu/ZkCxJxdGlfqypzz127Tg2mOO5uTktLS1pPB6nTq9kzmGXsTn1el3z8/NhA9AJKdl/tbe3F/dtcLQ1QSfyDBCUktPUWA/mn/n1PVjSoR3yiy13d3e1trYWZBWBBYDUszGS4nhrAmlskZQcJEJwBMGKnOKz0AkngiWlfD7j4A/9I0OAvaR9LnliDg4ODlLkBHrq+sa6ZTEluIXKi9FopE6nk7IPnl1lnrlN298BadLtdlUqlcL2sY6Q4W5n3L57dpKx+T4FytjQcdYRmwGxQj89E+sleegsY/OMpJNwklIXrGKr6S8kFGS840X6AF4cj8cRHPmJVL6BH0wP4YUfeKh2ZEuTdSQ4daJ+jyw94vKbML2Uif/zeZwojhSDDZCldt7BgXR4eQ+AL9sHj95d0ShhIKrFWezt7cX59nzW01MO/lBKhAmBdGAgKfV5xjMej3XixIlwTJVKRcvLy6rX65ccu8jv6/W6Tp06paWlpajzkxSOxgES482WaE0mkzippFgspo7QJEhE0V2gpeRErELhsJxndXU1jn5lUxUy4RuI6D9zjPF35fIgtlgsXnIULcoKQKCMAPaWQMgdrQct9IM+ZdOmLtue8XB2gTlA3jEKkiK4PHnyZGzeXFxcVKPRkJTUZTvQIPjkSF1YXR+PO0oMLxv8jmuD1UPGmEsMe6PR0KlTp7Szs6P7779f586dC4fjLLsHfxhlJyWwJeiDA11/jtepus7ye+yAn1ufDUKRb8/UMjYH1Z4K9ywIOoOT8NNAJKWcCN9xYJL94wAWQEVAi9yhj9gvrzN2ttwDKNh0gJsTCz6/ksL+TiaHJ/M5QcN7WHsCI+wh/sEztsw56+sHIjjgyM6X/15KAk5sMkEAdggb5b4J2+d6hx5jd700FTDGSVXYl06nk9q8yTs9c+QlaH55mhNhyDCstQe8kBkQEgDS/f39kEfqtj24PU7t4OAgDs5g/ofDYezfywbK6Ap3djkRiT5tb29HvTmEFBvOORCEkiTPyDpYZ57dF0uKEkz6wd5MSFnPArqsIyP4C2TeAaB/RkrwQKVS0dbWVoq8zVYF8D0H9+g1cu8BBaSxg2vPirjuM4+eScMHYhsp18b2OunreyOk5EjmcvlwfyOXPrudRg7G43HMMfMCJsOv06d2ux1zxvyxTyd72pjLjBMs2XJ27IekyGSUy+XIsEuKvoGhs5jFM0UE1tgpyBhsOHPNvPmeUvyNB3o8n0w91Uf0CSznwa3bKEiRmZkZ1ev1I+nswzp1CqPIEYxe1z4zM3NJjRoDd1DJpJD69OPjPD2E4KJQsHJsEobpJgjxo1IBajht3ulMDgrgaTOeyy2ifna1GxYpueGSMVEuQyaDxfSLtVj4nZ0dnTlzRuvr62H4vI6PhXa2sV6vq1gsxikjAFFnFikl859JiaNlfiTFO5xJZjzONHjwwLx2u11tbm5GlA5Qo+8YeRwfgMHlAsPgWQUibdLDGHCMvHRotGGXKTnz08WcLUSBMeAwPRhc5gIAyIV8MKqe3gVwMaZisRh7TxYWFuLIXE6IIuvmm7OygI75QN4uXrwYDLSzYNnSrOPcdnZ2UiVzMMcYO4gAsn7ValXnz5+/5OZX7Adrig7haLz0xMsPsnuCfI+B2wjkDweEXOHEPduK4yLg52eUS8AE0ifPdkwmk1QJqJM5yJGzVJ6i51InZ8M9te5HLv+P//E/4ohMKdnsDbgl07y5uRnBr9eqAwYIFCFoeBbBh5Sk3d2OEwQxT+iXB4rINxtIy+VypPkBZtg2ZIV5BDzjZzxI9IwLYIHjT71Mxskiv8PEQQDzl81meXaBcXlZg5fN0EfG7cEd8wl7TmDo2RlAmge3Pi/MN+AKu5rdbOxs+HFqyD0lOQDBer2uTqdzSRbMD5jw/RQeBJKRciC4u7urTqeTKlman58P8EtW2rMOyBe+iGCHtZOS7Cj+BeYZeygptVnaZdoDGikB85zABf4i8+Enyvn4CDSczPCAgO95I4iSkgwGvtVtoMtVNuB32+FZDSdnOd6ZueHdTiT5PstsGRVZKT8IgN/5nlb2clCB41nSVqsVJDY+gHlyYtyzp+g9thg5w0+hl+wNwz6xlp6FpY+sBwHz/v5+ZERY91KpFGVR4DPGCw4hOHYCiftCXCZZM2SLoNuxGPI3Go2OfK/XkQMNnA1Gk7QeA0aQsik0N6AubKRrqFdHmNgQ5WwLF6f4BqBisRh3WBwcHKQUqlRKThjyNDPsDs/a3t7W5uZm6hI2IjxSiwg7QoEg4kwBK86mOyOJ0XFGr1AoRB+YP1gL31dQq9V04sSJKB3a3NzUhQsX1Ol0ItjDISHYPBMldoV2Z551djjKarUaG8skxZGuMCoAG2c7nI3LptKcjUO5Me6ALjaFz87OxukeztAh/MyhlD4FgkBXSo4oRlkwtn4koJSUwDkjScSPAcWBYyj8RCoYNTZ4c9M7jga2HINEuRubwbiIazBIbkJnPvguBpznYZSc+ThurdPppI6HxsmQvQJQYhNmZ2f1xCc+UYPBQOfPn1e73ZaUrB+Ogs979gHZQu+Ze4ABIJRg28kDgAgOHgfuz8TpoFvurJEDACu2yDMVnkXhqGTfzA3ZICWOHUIiG3i4LZF0ie7cf//9+vVf//Vw4h7MIGMEUtjUyWSihYWFVADHOwl0YM+lpPTJAxPshusqukyw5hknAn4YeC8xYUyQILC1rVYr7H+2dIXnUcPP3HjJndsVt1W+PoAuz3DzMwcDZHgAPzyH33t2HJlnTMgGz3Nm1zPE2DuCZOwItop3OUjCJuIvkNvj2FqtVmS5PdON3+e0QAIFSZGpZg5Go0tvu4a0QP7RK4JRfKyX8WbxDZ9zlpr3si5eToXvg8jjRCLk24Mmfy5VBE7Ium4h616ai7whE267AKIQP354jvtHxuvlP46PGDN94Y9nDQm+fZ8dc0JfnKxxGwXu8KDR8Q8BKD7FA2qIZbeN9BedowSd8n2fH/rEWPDVHnQ4jnO/zZ4b7klhHp3A4rmuv4xdSmwPY4HI2tnZUbvd1vr6emQywCxO3nu5lN87whziE7DbvJNMPsQGvo7xHqU9rFOn6JAbc4whA6cTRFHOrDioxTECEphQGEAE2zMjDMpZKYyLR7w0jyR9gfgc/3ahI0jidBXAp6crs+yas/MsDgKIkCOkLlxkDMrlcpQkSEndHk622+0GwHBB8ijVyyOYZ8bk88Y6uRLBCNIXQNHlGkaafzPfpBwRXA/MnAXBmPGH7/ktzwQ2zDlrSDqPiw9J5Y5GoxRQI33pQab3BcfCvwEHWSaRbBnsOndawE77vE0mE3U6nQgY6DNK2ul0dObMGV24cCFV9oH8ElACUjDqDjyQ16MeKfdYbK4POGD04eDg8PQYMngEbjBpJ0+eVLVa1ebmZjCX2A2CE9bV9UNKSgooXcCW+RwjQ27jnLXyrKQHI5TKuGNwMOuA1QNoHJLLq5cQkLX0cY1Go2CvvCySIG04HKaYVhzk/v7hZmzKP3D4AGFsAoCcPV7O4lLzjN5Q4ggpIyVlgvzcyQ3fD4UNQaY9PY995z3ouJeGeWnCZJJcRFYul+PAEf4vKTbI8176iy5ma6K9LCKbaeCPM8I0L7Hi9x4Us0mbrBhzwR/GxtwDQAicYbo9ACLrA9Hk9gLbiO8CeOBbjmM7ceJEyu+BLTyAJYvI/DpARedYu+npabVarcjgOR6YTCaRDSLw93Id5EVK7+uEdAMPsb6eEXWikndUKpUo+5EUOAByD99DZYmk1J4MxuMVIowFW+FBldthzzBISXaMPwBqL68ClDou8GBLSh+cICUZej+shwMYKFN0Rt4DNnwzGAN/63OOXEMgocPMG/PPXDFW9vxweIuf1iYlG9mz8+TzQL/w/dgD7CrYAzn0rIU/E19CHz2ryWf6/X5s7gaTIuP0AT3gvegDfgG/iC1yG0lA6MQnsgT+OeoJmEcONDDkGxsbl+xJcBaQBXGWx9k3wCTOdnZ2NvYrcFdEu91OAVi+i7CgQBgVQCJRaLvdDiF1dpF+kXLzVJxnWgDAbAxicREK0lerq6sB7jEkRMUIL+9H8RxooLg8t1AoRA2+n4xARAyj6oJHdskBhzOPWZbMATeZHN8YyVhwwCgrLIEHmV7C5Gl+KTF+CGS5XI4yI4+aKQWAvcmWgsAgsxms3+8HKyAptceGMe7s7ESdIoAHYOXlTIyZTemcBkSwtri4GFk35tWBCBvkWQeMph/xXC6Xg1ljP8twOIyN7oVCIe4LIctGgFOtVkPeYC5g5Y9zYx4xUjgTBw+UE5Dpmpub0/z8fGy039jY0MbGRgQcPAPwDivZbDZ14sQJNRoNbW9vX3IXjdsmSWEbcO5kTgGizswTRExNTQXZgXw4APdz27E7yDCOlWDKmX2cD8BzenpaFy5cULvdjs2Qru+lUumSPTD8G/CNnuA8eL4fnQyIbrfbsemWEwfRf8/ywUb6/rIsyPYAAj3lvhjmgPHgIKkDLpfLWllZCZkBJDizubm5GRugsRmj0Ui9Xi8OrQC8ofdePsNa+/n0UnIfiGeiyGh7FtT7BnnAJlOIIhwzZBUyBACgrNKzoR4s4W/p92QyCSYTuwKYZdO+H8/J2PB3XhJynNru7m6U1pGBZ78Gl41x8uMVV1yh6elpnTlzJuwC8ogucIoXMoG8U3KG7nmmA/zjcuEBM4w6gYpn0xw0u3yBlfDNXgaaPXbViTFshQcVjA//7RlMMhZ7e3upjc/4WSl9ohJy6aV46LBvfMYWeIaQMXhfAcgbGxup0jOOLeb4V/oA9vKgHhzlpCZ4wokNKTkYxG0MmSTPEszMHF7eid5iw9FrSCd8xtTUVARczIHv5cAn+XryOSkhwqX0vUWMzdeVqwPwW8PhMHTf/ZFnubAtrVYr3gOJyVg8kGKtPXDnj2cxnMB6xDMae3t7qYjeJ4sJ8jQiggKLAmsNYCB7ISl1QlG9XtfMzEw4ICkpK8BYen0qisPk+w2PzlBnF9Br5DHepKBRzHq9njJmnLWOk1lYWAil8YuRMAoYFJyms1Nel+yKBtBiAWG4cYKeXmO+6bMD6ezYUXo+B4N7xRVXxKYs+kmQ54KXLeshsMGBAq7L5XJshoYh8jo+0v2ALZ6PkWAN6Ue/39fGxkYohTMaOFz6gCJxLKBn0ySlArqsgeeZXprktcwYEAALgQYb/Zy1JjWKHAGqOBGt0WhEYAV4AGi6MfYTyBy41Wq1o6rtY65lS5xgd7Og9ODgIDbGU0pXq9XUarVik+bc3JwWFxfjxDbKbVhP7rSRkrWkVHN7e1vb29uhR2QT6RfOAxDuTsOzC+PxOO4soPmeEc8iYs+8RI90P/bOWX+cLM6Xn/lFUZ4lnUwmERAQrKH/1Jx7JsHZXuSO9xNETCaT1MZF7LDbFewq+gggwFZ4aYOXCUAksD4AM0A0ZWQebA4GA3U6nRSAYayMi7nd3t7W6upqkAcuYx6EkTkAgOBkWQeCAC9jcZBJYMbn0WvsgR9zSUCGLtAfDx7oB/O8v78f4JfPOJGEnWbckkKW6Bf2W0o2gvrm0OPUPvnJTwYB1Gq1tLS0pG63G+CJ+yQ8wyFJGxsbcfoO9sCPrnUy0IGkk5TSpeVR/NtxSLa0zglOfkb/0B/HTIBEwD64wAN1ACzy6/LpZCMYDZng+/QVAsQBeKFQiOwvF8VCrPE9ZIqxceIldhq853iGfWBgGs8C4/f540RktVrV8vKyKpVKlJdDwDrBQcBBQEDG00GxBxeOyer1eooI9awsOortd9uFz2DNHVdiG1xvyWzwbCdWmA/3+dg4stlU0wwGgwiCOd3ycoSVB1/YFA98Pevm+0qQJbdTzDN9Zh4fqj2sQAMWhk5RziIlpUEoEeABNs7vY2DwMMWDweHtl1KSBcCoeuoHAMKCElmhNAATnIRHqiiqp+t4nzseFHl7ezvqwefm5tRqtbS4uBhjWl5e1hOe8AQVCgVduHBBn/70p3Xx4sUAohgvnxMH5hgw5gM2FkXGSOBE2XgMiAIAZaNsF17mEGNDmQ9AF9ac8iXAEQLlmw6ZX9J0ZBR8nw1r4ywFwZGzDb7fxgMir0/k57B/gEgML8ENfWYjPhfeEeA6i5xlnAF4np3jd25AkFMHY6xDr9cLsEzfALAOnDx1jtx5JonWaDSCid3a2pKkCHS9T8e1+ZnfGEbk00E58wag5xJGNmdi8LjvZX5+Xu12O+QIEDWZTLSxsaFer5di35zNhhQgq0KGIssmu+2j/+gpQBnAgkxUKpXUHRe+8Z3AHPkkcACouG3yPSZ+BxHzgGzhoHG+yHKpVNLCwoLOnTsX5QOAGpyfgybm1p/pTpCAJ1vO4wEV5RF8nu8TnJMt5pmSgj0FVHHS4Pr6eswfc8Mf1oKAnvuEPPuHzLGOrDmghky7+5LsRmzKV1kvB208h3nY2zu8h8WzdBxrii0CqAHWisXkxD3WBPvgFQOeCV9eXtb29rbOnj2b2qPhe75Go1H4RWexj2sbjw+PLt3a2tLq6qruvffemDuISmwIxBP+qFqtRkAgKfyo4xiAIGAKmYbBZx34PrbDGXL0mrX0TJzrEM9wEpTPOAEClkC2sVu+xlRhEHx6uZiXIfEdgDOyywE4ZOH5vx+4gr6iBx7IbmxsaHp6Wo1GQ51OJ0qOmWtAN8e0Y7/QNwflEJfsFwVQ8z5Ib3w+QRayD9bzo78ZL+XMrBl7fZ2pd5uED/JsIn7CfRbP2NnZCZ/m4HxqauqSvWIuB5JStgyyBRkga0mgi83yzARzAxYBu4GlCMy8DI8x0E9OFsPmYxPBPpJSOPMo7ciBRqPRSKWyEG6P7BFYlIXSDzqHw8Uhw0bDGGKIx+NxgHuPyFBgr6PEoSPsvIe+8jN3BABFz3J4ugmjBJtF8MBNrfV6XdPT07rxxht1ww036ElPepK+9Eu/VA888IDuv//+1Ck5PNcV/3KpTko+JIXBgrVEYdgo7ac/bW1tRbmNzwd17ihnrVaLDAMXR7FO8/PzWl5eDoVjjrI3XmJwvV6UlDUXBvb7fa2urqrX64UCYED8Wc5cEGiQCTg4SPZaIGscDYfyEN1jPPyEsOnp6ajDx5Ewn86CeDkaZTmMvdFoxJxMT0/H2dtZhocNhRh5Go6CtWc+YTcJNufm5tRsNkPeJIV+NBqNYEEwfujMcW0YZ0Au8upMoH/WmWM20zEPy8vLOnnyZKwfpXXZ+nO+73cN4HQHg0EqoEZOHRg7CJyamgoQj21CRj0lzXpVKpXQF9cHjD1yOT09HSc/kYKnH24/CfS9ZpZnS4p38V3P+lI2hu0hiHdm3mXNU+bZ5x0cHMRlYZSNAWboAz9zRk9SODLmCd3hci2CasbBPJPZwC5LSvkd/6xnXXDebBj3YEhKSqGw/7CogBbmhz+SwlZ7MIcs8L5shgK7x3vQBfwPfWeusa/cbQI54YTGcDhUo9HQzs6O1tfXI3DH7vl7yeJRIsJlXMetQSA5yej7GiVFgMyaYEfK5XKMm4AXwEamsFKpRNbJqye8fMQDCrLbkmKunRBC9vg3APPg4CBV6sjP0BPPnuGzsJHoFhlfbBSBK/3yshgnxADZ4/E4voMPn0wm4cN5v2c/PYOKTvg7wVX4SK9y4TPulwnKWE8puTEb2+N7d0ulwxJRiDrWgQMY0FtsBjbZAwN8T3bvAvOO/jJ+xovdx45Ll5ZxIUesFevHGoBD8D/oOj/HdmKfe71e7P8Em0A8u129XMkndgLCGlvLz5k7t1luozxjDEHDvOzvH32/6JEDjQceeCAMHRuniBR9MUqlkprNphYXF+P0psFgEIN1gIGD5VIcIjEvo2k2m6rX6yqVSnHKhEfZU1NTqtVqcfxctVqNM7GJ/jyt6IwpCuiRK5M4GAwimAIwsvGO87oPDg508eLFyBA0m80AotkTHnDIrjg4DSJwZ8RcCWABh8NhZCFQTt7rl/Q1m00tLCyo2WxGJgjGlBIMj2y3t7eDLcEBk1rHORUKhQhEvEwMRn97e1sXL17U3t5ebBp1BgTw5Ck3jLanSlFoDC7fJ8s1OzsbR+s6a+lOAFYQw0O600EmJV5kZAA/numBrWKe5+fnQwY2NzcjGCLFTnkHfy8uLqpWq6UAKSAJI8k681yMOmw7c4KcA6iPayNAhbDgZ5794XcY6IODgwBcOGdPn+MovCxzOBxqY2MjVYbnrCFzizHf2dmJ9Dkb0cvlcjg4lx1n3ny9uATT+y4pBXB9fATXBMSe+nZA7eVP/BvSwje88nPsnJQc2MCZ5/fee28qfY5z8+ynH5DAMzgCknHDqE8mkwh+6K+DPddLnsXPCGi8tISfAX5YH4IMSBdJwVxjowk+KF0juPSyxGw2ClvP/EtKkQ++oZP3uT+RkrIvbAHlougw80mmnLXjnQSqXo5arVbDHxYKhchCEKTiAwmQWq1WlNg5kQIpg65gTz04PW6NbDuy7TLmIB39IngAxBWLxTgu3rNFvh8QZhhb7mUyBJ/oCzgEP8BJjR4oe+YeoMt7sQ0wyJTVINuU0TpLj59knxN95z2sLaXc/Bzmnb4AFPHD2EtOb/JAg2dge2G9yW5CLFNxkt3TQVaVwyV4Lr4fWQewO4HAIQdOdjNGMgj9fj/AOPNJdgS7wgZ4nwevguFn2TIi5A4ddCLas4PoMViLvUIQNPxh7E6CsAb4o83NzVQFyczMjBYXF3Xy5MmQWwIADk4Bm9EHJ0OxQ/g4suzdbjdwEvbNKzikhHjzqgzm6CjtyIEGZRw4llarFUfCwgAwaEpN2ADrdWt0EMVmoZk4FgLjDsvnm78WFhY0Ho9jQxELgtPEceIsnaUjbeZMJf2m3hHw7cwgBhvF5TK2M2fO6Ny5c7FIpB6bzWZ8ZzQapU5VoFa6VqtpZWUlQCuAhvnBGaGwODTm3E/GIm3M75yN3dvbU7vd1traWpw8w7gQqrNnzwaLirEjDUgpkDteDCUMfb/f19raWtTKSwqDwTObzWYcPUjtpNekI18YEXf82dQlJ9E4+4uzBwQBVPjO8vJyGBpYRo6odQCIkrJJ0FOZZNQWFxe1sLAQhmJ+fj72D5B9YWMqgTaGhT++AX4ymQSQZv4wrpVKRUtLS1pcXAwm+jg3lz+cJo7aU70YYJwOmUpkhLPSPVtEEOMZBJwC+ghJ4QyTM/VkFWAKkTOCDuTPA0gPSpBl5FdSyjF6xhTb6PvDJIWTx6Z5FhSg42l6ZJd3YwN59vb2ts6dO5eScfrBnPs67O/vRykODt+DGsoamBcpYfSZaxwTv+d92SCL/kJEAK7oA++m3NSzAb7RHhvpc43e+n4qD6R4L7bDZQY9dfDkwaqXfkpKlZJhc5xMwfFnMyQAo2KxGEdle3UAgElSzAesKYwxew3a7bZWV1fjpmwPXNnriNwC4o5bw64CtiVdUi7oAAn/DuHk84md5nTBRqMRdoGjzPGR+D0HqwSBbifQE/QHf4LdkhQ/989LSgFgTp1zgIvM8DkyJJRkgkGcPOA7XjpEoMYc8Tf2kSoO3oUt4jv4ZPbKgQ1KpfQltM6wS0qdqMXPPaOI/ruvxO/zOd9U7ToFxsuOmzmB1PWsNftOIAidWMZOovPYWb7LO9Anz65SZYKN90APm+Pj3N/f1+rqamQpIcv39/fj2P+VlRUtLy/H+7FjyPLOzo7W1ta0vb0dJBCf29/fjzI0gmbe7Xu3HDOznpC6NN+D9IgHGjyQaLvdbmtnZydSOERUOBJujc52is9g8P3iPYwiJ1zxM4SeiYB9Ho/HUc5AVgLnhGNBgaWkRpA+AuJwTAhNpVKJYMZv7iSYgPVeXFzUaDSKk29wqgQX7rxgsAkyKBV48MEHA2BUq9VIm9G3g4ODADYLCwtaWFiIDBARKYwHFwDiJBEe5tAzAc6m45AcIGDYvMbTnbAHXZ4mZSxesgaog+VA+XACgAf2QxAkAuI9++ElbmSVnOFA6TwFy6ZBGB9nQZ1N9Q2pOAdkHqVrNpsRSDO/znBT38/a4fCl9Gk3MECwEj5/ris4h1KplKqZ/dqv/dqjqu5jqp06dUq9Xi/WjYARBwKww7gDkpAjz4ohQ85Kc+gEjhXD6TXSgBJJKd13Rml3dzf2NPE92GoCEwfJBwcHQXYQQDnIw1A7UwT75Cwe48d54vyy+33ctjmAoQHk6W+9Xtd1112ntbW1kDkpfTs0jltKZwVgXuk3gb0DOz+lhD7haBk72QRsAdm6rG8hyGJMyAHAhzXd399PXTTl8wOL5zaMNURmpORwCnTNmUoA/czMTLDJ2A4HDQTEEArIMYB4Mjk8+nprayuVfcna3Fqtlrr1mMyEgx/GyVjcFhK0zM3NRVDe6/UiGANouK88jq3VaoWeInt+QAL2VUrmiuyRA0PmCtIJcEwgwXfIOjl+QJY9OEdOWQ8CCOwAGIGsLvLuZTf8IbjG3/d6vbAXELiui+gA/eJnMzMzajab0U/mhTlA9gjSXU88AwM49wAZm8BFx9hbQDV9uVwVCz/DftA/B8FUsYBxPJszGo0CO2JjWF98OvjEy6ZZD0hIgk4voXLfz5xKCqyUrbzwPwRKyByYBl3HfkiHgL3dbscJhGBqt5fgYYhr9xfb29upAzsgIP0kwMlkEhUevBvfR0CNfXAChXeDqbCRk8kkTnXr9XpRjvtQ7ciBhqRgBPjbGStSM9PT02GUsxEPC4Nw8TNPzY3H42AXZmdn1e1247halMHPIqfcAADPOz2LQjrII1CAsjsaZ7VdWDnpALYcZojN4lxIxe/K5XKUHHlaEGNIyRTpPhYcZ+LlaSgXyoJzJxPEu6Xk3GvGxGZMgjRn6TB81EVj6AA2pJwB3UTrOFjmlPdm05ooHWuNoYK5oL4Qg8mmXpwha8RzMJCkFZ1RYkxScpoODoLMC+Cv1+vFd2Aeut1uyCHPICgqFApRkgZoxJgy7tFoFBfwYfjYd+GlVcwNckgmCvYKI0aAywbYqakpnT9/PsXSHdfmG9vZ3A0481Q0a0amU0qIAgCVB8Tj8TjIBQI4gC7PdPBKYAmrPjMzk9o47GDUgTkN44tjQOaQC88sSEma2Q9FwPHi+CTFJmQcs28eps3Ozmp5eTnqdrMpbsZIiY50qKMf//jH4//OzuFcsFHsSwLoANx97wVAipImAAQgA0cIWCfgcaDnesjeLT4jJRlKWFvXOSk5chvH72NHXi63ftgxd6DuTxgD/obv4Hx9j6DvX5OSUpX9/cN7S1wW8UnZLCkB9O7urs6fPx/ziK2cnZ2NIAZf4kTJeHy4j3BjY0PtdjtsjpMfZImYE0oLj2PDxxGQAVKbzeYlOg8hCU5hbV3G/BhxfA7yhl/LgjEnl7IlW5AX7pdZZyc7sFlScjqhZ+SQdfxHvV6PPTmQNe5rkWHGDDba29uLUlDXE2wSh7Mg7+xb89MiHRcx35BpAGpkLhuUMAdZe+MZQX5HAIj94JQl/vT7/cAPyD7j9kCG4NBxJXPJ3I7H47BN6Khnhpkrt92etTg4SI7mhzTDfjA/NOwnFTpc2ruxsRGnIzIfksKuVqvVOMHT5Yt5hbzATkCWZTE18+N4jvnw5j7SyWCwJ0E0vvao7ciBxuMe97jU/RQc90oEh2EG5BIwYAgw2EwAP2OvwuzsbOq25MFgoMXFRS0uLuqKK67Q3t6ezp07p/X19QgSEFYHKQ5UEHDYveFwGNkQDzqyYJiABYZUUrAdnhZ3xglhJHqE0eLUBZwpi0fqjPcyHzAI+/v7qX0mGCGApzP6XiuKoBJk+SktKBn9xeF72YKn8SWlnGKW/aQEhcgXJXRg5ACB72C8UUxSg2RqYEZ4Ns9FUXyuMPbONDL/BAvdbldra2thjGGvYArq9bra7bbOnTuXyrphyN3g4mDIbEiHgOfcuXPa2NjQ+vq6VldXVa/XNT8/H4ETpX/ISalUitrQ0WikbrerTqcTddcYX+qucTae+jyO7eLFi5G9cifjoJN5x0F7ppR7S2BikD10GfvhJ814xgT5dOAoKcoccA44C7/UynXOWT0pOQra2UHS+JS7eLkQY5eUKkmiv/zej+t2UEGJHoQGpXlSkjXxNP+XfdmX6Q/+4A/iPciilIBj7JEHKjB+vBum3Dcium1hHZlrSQHyAIGME1vqmVInLXgmz+B7BC/FYnLjLk4QW+B2HbvtzhfWz/uTDUywzThmD0Yc0DqjCsHj4wDkY5/Qa+TDAZEHhqyR70vxcggO62D8yBjjYexez83+QuzYcWzO+KP3rKP7dcCbn5LogFdKjjfFnvB7TjtjXT2AQB5YCwdi6CA6TubFs3vYDlh57AXP93d5Bg6ZqdfrqYNyvOJASi7sI9hwlh05Ak+gp/wBm2Db0Dv3/+isA3x0hUCcPzyHoMQPxqAUEluNHhD4gH28JJYsLcCbIAc8lS1rpW/uX5AhSE6y4eir94F+ERh6cOkBJ/qY1Xn20HS73cCgHOLjWQ6ICNaxWq1qaWkpdeIWz+WdfM+DGs8mucz53y4PfjiA2w7f5yIllTHMKTYdmXuodmRL8+QnP1mrq6uxOFkDTgZjY2MjWGAuYMGI7+zsxLGdDp673W6qZhFgQOS0srKiZrOpq666SsPhMAC4g1gEhcYFKX4ruQcKKAoTjLFBKTzzgXLh+AGCklILDyBlsVAEHJOnwVFSHA0ZBBbS74LA4LhB8cBOSk7VYlw4sXK5nCr/8nQoc+EpMr7LnSEwHQR0KL4HR6SwEWAUku95SRN/7+/vq91upzZ07u/vq9lsRh+IxPk840axGTNgHIOOcXG2k4CGQwO4AHA8Hsfm7k6nE1kdApF6va7FxUUtLS1dIkOc9LK6uqqLFy9GpgWZ6nQ6YXgbjYYWFhbiHhq//NAPN3AWn4AI4I3cHVW5H4vt3LlzknRJ4IQR8wyns2gOspEJdNBLmtg87KAde4IjBTz4XiYcFLLNc8g2QDRMTSXHOGfZLe+TpCjlQG+k9MZQmgcenMDHXhzsHM/jd5Sq0tfLZbl87j7+8Y8HAwWj6E7HwTHsuZf2ecmDM7EAPu64cNtPPTb2DruLXcGJOSDGuXn2lLkl84tdd5KLz1FaAJDBr0CcSOmTf3g3MuWlnw40yfhi6xgD+z48+MBWIYPMl4MpgDA/q1QqqtVqsT8D+44+ODsvHTp+ThxERrhfh1JjKgvwBZRxTSbJSX7HsbH5mQMcwBlUHTBGlyW3mfgH9wEuZ41GQ0tLS6lnYFOcecZ3AkJZf2eVPdhlLWG/AZfZLAbPx/8DiiETPNPvvht55QQtdBlZwx+7P/F3OG7hb+YS/UXWHUMB4Ak2PMvjeICAAJ9dLCbHOTMHUlI2xfs56Abs4PtQvWqCwBOimkwedklSBA/4YN7B2rvuoeNO9kxNTcX9atgQfAK2jvewGdyPN8fugRWywdjU1OFBECsrK1pYWIifeQaaecbu038vp3VMxfPpM5hOSg5iGY/HcUwwc+kH1kgKDMrRx5w6eJT2sG4GX1xc1GQy0fr6empHvAuUs/TOCE9NHR7LVavVwkAyQTs7OyG4KDfAgSwH2YgTJ04EM+4O21k8T9O7wfbolZrC7KYovkfK3jftEP0x4YzbmVFq9BmX184CrghsnB3D2RERMz5XJpTC6/edSQCQMG5AsYMa1mg8HgdghvVx5gsBlJIgiyieEpNKpRLMJoZ7OBwGKz8cDlNlBsyPpAh+YAt6vZ5WV1fDoAC0yRowF8y3zz0nT/lJGbwLthawAJPA2DHCsEucw876LCwsqF6vxyVxGPmLFy/q/Pnz6nQ6UcICqEA2vIyCW6kxksgjAbivHX87a4GD4AjC49rOnj0rSZEVJdCmVIj1QM/QXSnJJqHnHkxkgRyyIaXLXjxA99KZbNBB0ILNGAwGcQTy/Py8CoVCOBJ0lfd7MAiDyIZRnu+gJbtZ0JklZwOzn8GpegDl8+V6d9NNN+kjH/lIiiSRkkwB4BjgwfhWV1dTrC81wxAl2QMN3GlD8nimAkfo9dR873IMnwNHsqT4E2wrNhLA4+9E/3i2Z898Hj3b4XaVU8wgg6TEHmZLk3zzLnMAmPVMFX1EBlutVlw8V6/Xtbe3F6cXYm+w57C62GDGhb1krS5evBjEFHLO3CN7LgfHqXEnTqVSif0alLEA3AF0BIKwze4X/XZo/ma+/TJZP87dS2uYXycCnDW+nFxJijJZbIsDbGQcnSJg9Dtk8H+sNbKOv4O8og9SuoTT/a7rl8soz2OO8LfOnNNf/JnLPkG+A1rsBad58QzPWPjVBNgvjuOnn14Cxtx74J4lMZzN98/gMxzzuB3Ksvw+h/x7f39f3W433gWOZS/N2tpaBPvYWS9txFeQYWUuGo1G+Ay3Z75OkGd+mquUHAtPSbCUlOp5YAAxQ5CexdROetI3Sr4gepzA+XztyIHGNddcEwo9PT2tVqul8+fPRypWUqp8xkuldnZ2IrXkR8gBMrJpQ3c0bJjpdDpqtVpaWFhQq9UKcEo0684Mo+KbfHCe2dQe33GmfTAYxILjzFkYBNPvmKBMylO5l0tnIVD+M2dDESoiSinZ4OhKyl4Dz8Q4WGEsKISfW89nAcYuwPQhezIF8ygpxeLVarXUpVsoNEbBWSBuTwZ4wfLDCNF/T11S04jiECByOlO73Va73Y7j2TxT4Awy/WX9s2OnTpN9ItxuynHEOHdXbMaDcSoWiyn2Q1KkxVkX5r5er2symYT8wqQT3GKA2CPiBtAv8DmObWdnJ4wTWYNsqY+vpZfjUDecBUq+qRAwJaUvaUOmvBQCWfGAk4CHtXd5xRFKilPeyPyxH6rdbmt9fT364IRFoZDc7kvgCDvIz7xO3Es0XKclxeWk3W43xuAgyOdSkv7kT/4kxewx39glD1T4DmVJWUfIpkkv53DAQl/RQ0AK88z6u57SD8A3DCBOnmMinUVlfMgB4Ih3YadrtVroLCRV9kblfr8fhASOFcDJ3FSr1dB/AI+DM2yxn2jmgSprSd8AwFNTUwFsyYJiE/lOliwDrGbLWQh8OGoZMEUgia4dZxvS7XbV7/fjwBZ8DiUzZOS2t7cv2TvgwS56C+HngSC+R0pYX8+4QlxB+rh+0h8HYVl22ctkvMyKwNdLvggyXOfASfwpFAqx5vQTUInNk5QaB/1Dvz1w5jtOnhHMYTvxn8g+WM7fTwDsxB5yiq//XAcykN11XAXWk5JT7JBrMqiefS2Xy6kgDZvjZe9kSPEH3n9kA1yZvW9FSu5O8ewCtltKDoEBE2VlEZvnmSn8EWuIPWQ9wcVO0jvpwBwR4LlvzBIP2CzHRciFZ2Q5kGBraysI9ctl0i/XHnaRZrFY1MLCgk6cOKHFxcU475e7E9bW1sIAZoEsERdCD4BnEzUpUIyoszQspoMDHL+zvAiBK0Cr1UrV3SKkKIezycViMXVqDcpF2toNBMaN72XTmp4R8PIBAKeXFQByCHpgbAiUJAU7RdobY7S1tRVCiJNCaLjkDyUA9HiJkoMcAhOvG19eXtYVV1yhYrGoCxcuxA3MGAGcO7c3I+BexkTwgeKSggTMuSMmK0LAypxKCTjFiXpa22sxnf3FaOAMMM4AV45qrtfrEcgSnGGQPQgmc0KaGvkmW+Ky74bMgzxPMztby3v8hCU2rHe73VR69jg2L1lx9l1Sqk4Wtg0nyOEIzmDzWWTD5QTZ9X8DgnHk6D5rxr4g/u/HJBJoEyzjxEajUThCACv7u3Davu44Wj9QAzAIiHXSgPnJggMCX+TdyRbklLGPx2N97dd+rX7v934vwJOUBBvYY549MzMT5U8e0I3H4zjqmvF5wDaZTMJ2si5kB7MZUp7hvoLxIhsEl4ASL5/00hGAEJljbKiTR277J5NJZD4JnPi8l0w5KIQdZHx+Kg3klJTejO97ByeTSZBsrD/jBAhJCoIDG+bEWKmUnKpESQh2jCCW0hDPCCLbvs6M7Ti2er0e2eZWq5UiItwmQPq4H0FePRsBqYFdYn+GlJRGe3CGH8Pvsk6QVqwJJJyfrImt52/HRK47k8kkSEVkFx1AxgDLYBTWF3km2+ikDfOEvuCnZmZm4j4wfDokDZUA2A6X6WwmFYyErqKf9MsDhsvtD3BSAv2VFPiBucGHSkklCtlWqiDAFe12O0WyMG9U1oB9PAPouAucxyFF1Wo1tbcFkjCr6wR+rBkBgq9jFlvgO4rFYpArrIEf7w6m8OoM8J3ba68sweZiJ7E/kiJI52hkfod87u7uanV1NTJQJBB8jT5fO3KgQSoOQ16pVHTFFVdIkjqdjlZXV9XpdOJEBBh3DDgTzv4MjgAF9J46dUqtViuUolKpxOUkbHo8ODiIHfuDwUAXL14M4LuxsRGX6fmmbyZsPB6nGGpnFd0BuvFlYRByBBCQ4YYco59lVvm5R5ruDF1ZACGc/OKlBWwm8giV/notsqQox/JL+XyvCO+kPxgMB1WkNg8ODuL2ak5fqtfrcb8GgRfz6DeB8n1nKjjRxgNQTwPjFHCmklJGDIWD8fMx8lmMDkrEXGF8GQcgqFarqdVq6cSJE5FtAPwQ8MHa0j+OrSUwcyYdw0aZHv1hXlB86jH5TL/fj+wMa+tyClN2nBussaeSAf5Z9lZS6BbsJFlRDhIAPGA3yEhgAD1QlJLyBH8H84xe9/t9bW5uxt0o3JnjjpSUM32mj7u7u6kjFZ1Z8mDCg3TvG6AT/eC9UpJR9VIgxo9zIGOKHFGr/Wd/9mepMg3/21P5zti5/jjjSF89GMZx4gglhQ5m99PxHCdjfCMljD3gG5mnH04MOauZtROsOwDM7RBjBzx5wAGg9yPSeb4HpWRMIFkooWSdaARK2EhsKZleSam9aIANbA/v4vNeeiMlp8NwiW6v10vtTYSs8+YZ9+PWvvIrv/KSINJlAtvtASCAHFmF+JGSgBtZ2tzcDJ3iWfhwz6gReDjj7YEpgbfrJ/IIQIQYlJISTw+K6JOz5PzfP8N7kD/3PX6RrJRkNQC6kuLgEgIvSm6wNXwH+8e/sbWuE8wL8k5/kHv0FBk/ODgIUE12A5nGhjBeAgFsNlhKSmyIZxLwB8yblBzj7eSAZ1LBfMwZl/qClbD9TvTgm+kn+oU8gA3Zg+WZaMeH+DSITLIgUkJC4h8ajUaKYHX5dNIN/MD69nq9CDggaj9X8Ov+2CtdkEu3c5+vHTnQINpGyEnZsCBLS0tRSzccDnX+/Hmtrq6GI/ZyKmeuAPGUwSCwxWIxGGI2rRFtU2py7bXXBvPZbrd15swZPfDAA1HWAujE8YxGI83PzwewJCNCtA7wgTVHAdmHAMB3oYTNQ+h4rwcQKB+AhNSTl27BXMFoOvvJ+Pk+jtvLq1B4AAOKx2dxOETV/nMHQ4VCIY5d4//dblerq6txMhQGczAY6MKFC6HMMHXZUgH6hBIQoAEmnOlxg878sY6eUnQ2j3mAhWCNcNKefiwWi3GaWaPRCIAB+4ST9nnn3YPBQJubm1pfX1en04lSEjJLHlQhw/zMx48SU3ozmUxSAML/eAq12WyGUzyujXGx1syJg2aMGEBcSs6EpxwNgIbOubEEbI7HyS2nPp/IoLOArJPrAvsvOp2OqtVq6Ar65k7NwSW2DkPs4BgHANjw0iMvg3EQjcOUlCpjxA47YYLjIyuKfH7913+9fvM3fzOOOHXCRVIE4M1mM/QT3QTwSkmZGmNlztA5rztG1vmdy/f+/n5c7CUd+hfqhN0mEIR62YTPvYM9dBQ9J9OEjQbs0Sfm05ldZMYBj5Rkc/gOezcAdE6msW58D9nGD2xubkYZF3LiJAjzBVnGxlKy99TuQ4RRilwoJPeveHABkHICxYmc49Yo44B0Qo6ZR+TLgzYnNZyYQj+94gG/DJjF5iIP7qMAvfgzSn2pdJAS4s/LZgh4PMMgJYQjGX1+hg8layUlJTD4Er89nv2TThxyghH2CRwgKa4l8HJGSsMAo8yVlNwd5MEcxB3675k97CMyv7a2FrZFSkrOGJPbceYAPfU555n4bD7r6zs/Px/rTz/IfhAUOtHqOuX7MllH36PqZCCYRFKsHRgZYgZ5cpLMcaqTjI6XWBMyyp7FRA79dDUCVn+O+zcqYKi4AK95UM24kX8wJH6GsR+lPawL+2DknDlncT0twzXznNxQLBZjcwx1ZVxQsr29HQ4KgSYw8cwE5ROccuWAkg3WknTttdcGE4VRcMNDhAiQ4ag4v/KdxRiNRrETn+CCC1gAqDBVGHqUCsHBobvT8hp8z2L4xm13mAgs68DvuNvBmW6AKQ7HI04PMJxRoAQCwQe8SIrfURZHOrDT6QQod2MGQIE9RUacyen1ejEWB0fIlzOiHrAxPsAQDr5Wq2lxcTECK2oJPZUJ27u4uKirrrpKjUYjlJ87KzzVCLNNI1W4sbGhvb29FNvIeJ1xkpIjhj3o8XsKCoVC3NROCh6j4owV4yew4nSZ49iQeUCOl6sQCDtjjwxhA2BlcICQCM4CSgkAxol4KQSAwlk5/iZo9vIdKb35kXd5OYqU3hOCLJCx9YvlfNNku92OQBJdQQbc4GPL0BeAxIkTJ8LueFmUg5hyuaxPfvKTqWAKR0X/yKwxRzhYr1vGjrEe2B7Kivz+B0gPWF4y4pJijnGE2OQsu8a6ETBAIOFrnHzIggjPZEB2ZB14NuDyuaZfnn1BHsmqMT9eqgpg9ZIwBzkwnMiA96tcLseJSuxDq1arGgwGUZJHmQQZftbLT5KBNZYSG+QZX+TjuAYaGxsbl9wwTRDlrD7rQbBKkOxlqgBIB8aDwSBIK/AFOk+mH5Dp7LWTbdgynpslIglG8Hfj8ThOTkSXkUPPwmEnPbDyjAXvhIF3+YV4oP9+fxF+iP/zXaoosJeug8g048cOQQplM0LgCnQfXfU+eqACHnC9rVQqsY6QouAY7DiygN3ycjPe5Xu3CFL81FD3Fx7weXkX60ZQ5EQR40am2CPGemP/sfUEch5weuafefGg07MZkOPIKTqQrVDBPrPGjll8D6oTY+AS5JiMD+txlHbkQOMv//IvIwsA++rOkdvBMXqSgqXndsrFxUVdf/31EYGyqQtBJFpncqmF546BCxcuSEqfyEMjCMC5uPDjrPzsZbIiMD0INQvoF+6R+cChIsC+QdvZcRy8pOi/XzyDU/AoM8s2EXF72QIlHJSPZTMoHhTAvjmz60aW6JyyCc9WEfy4QnkgyPcbjUaUALCvAuXIlnc5+MZQ8F4yAwg+zI/XuPs4Go1GbNpmzJKi3KFUKmlxcTF1PCnAfjwe6+LFi7p48WIEGRg/2BXSp549IoM1mUxSNfkOPmEJ3CBSWuWnZmAQ6DcBLywXxtzlGJkvFAqpez2OW0PPpCR4xvHBwGJTHOBSKoW+uBP1GlvWkTQ2YBZ74QYWR+cOSkrutXAZ9YADR+gbIz3jAIvue4lOnDih5eXl2DNVLpe1u7urtbU1nT9/PmqFfZ5wRrC3vjHRy8EgGtBvnBRje9Ob3qT//b//t973vvelgplSqRR3vOBonL1lrlzWp6aSY3fJAkoK20m/sOdSwnROT09HaSv77txJw4xyQR1AxoMd5MODIg+cyILBHkpJ9sL3xLl/cCCIvSJQcDaQ92ILPJDwAMkDaWSQvrCu/O0lIpAI9AlwjJ9BZ8bjsebm5mLPFs+lzp7yCIAk84yvI2g5roEGATU2AAIL4IXfyAZ8UnKYCevhVQHIRla+nDDE/vJ7ApJsdp35xsbjTwF/NOSaCoVyuaxOp5Py50584AvwSYBO7m1iDNlyIXyGH8/Pkbmw+gQ4nv1DB+kzMk4Zn+8rpdqEqgaCMbeh2f1nbpM964Z9ZZzYJ2fpWRtKPSnb9vKjrP6BBZ2l59/8wR94hQVz59katxk+P9hgt5teWp8F+KytB2Xof7YsCpyEbGJ/HZe5bCNHzD3232UDuSQBwLyCbT2L5X6S9TlKO3Kg8ZnPfEb1ej1OCvK0JE4cIWo0Gql6RYSDMiQib09Z4SCIzHq9njqdTuxN2NnZ0YMPPqjz58+Ho3MA6yDaFZh0NBEdzog+u+DgMHB6TCIKwQJxMRtjh5kg4wIjRf3hiRMnIlLkpnMuHkMwEBoPNPwsbA8Q5ufn40ZIN1peV4xg++8BAR7xe1mVR9kwdkTTKLekqFH3jIFH6rwHp8wcAsak5EIqwIkbRD8+EyWUEvab9LhnrlBK1lhK9gPQbwwOWSpkDeW/ePFiGESvsUdhMWJsJMMoY5Qwwg7k+JvvM/de64gRQMn5zPz8fJyljWFqNpu64YYbjqq2j7mGc5QSAOjyKKX3SqEHrCWywMbcrOFHplkXB+Wss2c9+B7BpQfgDvj29vZCHzHSfkAFmRr2ODhTtbOzo9OnT2t1dVWLi4sxHkAlQZUz6X7iDeNg7DhEdwjZIJQ9bouLi7ryyiu1sbGhq666KmQMJ8vFqxxdSMkG88ga+bwCbtzpM0+VyuHpbRAeXrKGXXXwBEDHD2TXDgKDdXbZ8D7SB0C1BwrYJNcjX1sAIqCeeSabzjpndRqiAkIMGYAJJjviGZpCoRB+wcvYHNBwySelZWxuJYvsZYeACIJrsrMADw/8PFiTEnBy3BostweXpVJJ9XpdtVpN+/uHJ/RAyCFvPl/oGOQYpTL+Ofdv+CnfwJwFhO7/pORme+ydZ+uldLkQgNnLUyaTSZCGrJ2DcvAUQSrP9j55GSTBJf8GV/A8ADp+LsvYYxe9VAi5dtaff5M14HOUvTqAxj5LCQnFGAkUCJpYBwIY6dDHsz8OucAW0m9wB8Cf9WOusCP8HwDt5A4+ejKZBGHtVQdgBeyLz5mTq05E8G9kjj57IInPwV56ZqNYLIbtkRISwsfP3050gI/YNsC+LogYxuOZMLJQ2edgmx6qHTnQwAFvbGwEKMI5w3RVq9W47OaKK64IBp6JhJ3zFDCCQIqn0WhoZWUlJhtmEPYPkE8tJMrD832TMoAZAWOhGQsMGvs2+L0fI8bfgBB+7xeubG9vx5GW1WpVJ0+e1FVXXRUpbPoCQ7qyspKqR3ZmEgEmu4OAw8RIyeZCVyoMCBuRAXS83zMfKDjCySkzXqLB2jrwJ/iiZAwhdacHo0MgSl881YexZyzIge+/cWCBQhM0APIARb4+MH8YB7Iw9MXLLjxLUC6XY/8JbCcZKADR9vZ2iiXxE9CcKcDx+zhdHgE24/HhJTlbW1sxXr/EjnkjNdxsNvW4xz1Oj3vc446qto+5hj7imJFpZNgZEmd6cZbOvMMqAt5YbzfgOCUvZ/DAzrMaGGachxtsyhqcKcVG+Qbu/f39yFjgNCVF5uXs2bMpgIGzIIsFKPHsrpMxzpg6+PAUt9uCbKkrwfjU1OH9Dc1mMxwU8sd4vTwMEAOhhN3Cjnu2D1mm1ALZxwZJSQkoNph3eEmPg8Js84Des9HYNzKKjMEdLIGIEzFk1ACg9MWz8swTARTjY+wACi+vY079/7zXMyPSYd00JTDclYGPweZJSYDuNhB7j0wQmGOrPIvugdhxbM1mM3UiGoEbtppTcSaTw70qBITIk2MB1pHMNgeFZCsm8IvMLX7PS38lhT5AGkCWoceTySTq6h08ekWGl1HjKxzcAwYhIwC+WVvm6+zlTn4lAYGE6w1j9+CEgJ95kNL7nLxkEL/NekhpncN2ezZISvamoUt+GINn4Pb3k9MIfR8aY8Tmu4/x4ClL1IAzsTMEYF5SRT+xI2A05g69w26wFgQs/MwDEw8AHAMyN5xS52STl7IVi8XUZcz8AW9ji7IEC7qCz2HfMH7UMyt+WIaTr/iIy9nmy7WHtUcDp0P5EIKIUHBay+rqqs6cORMnGRD1EJ0SibMZFjaJcqBms6mlpaWYbFjrxcXF1AYlF2D+kA0hynPDihDgIEivo2SAlGw5AE7Sj9JF2N0pkrHodDp68MEHtby8rGq1GoEMguenGHiqD+OFcALCPBWGwWEsBGkYQZTP6049Y+IpVX7Pd4vFYrAOGHD2ExAkkeKFdWOj/GAwiBvbEVIMo7OevN8j9mxA6AGXAwQ3GF7/7cANoJCtVeT3XhbBRXycRuapYowcgQWsNr/PAhWYIWeL2UCMHLrTcBZ0enpaV1xxRWQvmF/0h/tKKBlDPo9jo4QNGeGP16o7wHQnLiWld87S+vHGflqaO1EcE0aceQQ0s9nW91FlSyi8BAtnBrjGFtJn+ur14B70j8djbW5upgAAgQVBFTpJv3FkXj7B2D09PhgM9MIXvlBPecpTVCgUtLKyopmZGd16660aj8f67d/+bX3qU5+KE7UGg8NT1NB5yBQHbuiWlASF6BUAiZ+12+2UbUP2mQfWh2cBlAkWKYHwd/JZ1gabR+NzyAdyISWbNj3D5eCLORsOhyE/2Hv0mQDMS0+cSHCw5Cdm4ZgJBFgz7C+2HuDgTC6gEgKv2WzGWjE27DcZFHwL33e2HWCAnT2urdfrRXALwOP/XgLkrLGvH2MH5LuvlBSEKHqFvOJLHVuw7k5CegZbUoqZ5nvujwGBlAzzPgAk9gK5JpjZ3d3V+vq6ut1u6IqPB1vispklagDQTsJ6MOY+EWJBSuTdM6OesfMSczAS+ufkIX/4rhM2nt308i7Ig36/H9UzTjR6CRaA3//Pc11esPvMF9+BUPRyPC+DQl7op2dofQ+vZ8ix606AeMaaMYATkE2/QwvZcx1gnjwA2t/fD3nFbnsGnnnBv/T7/ZhLMC6HouD/INDxmUdpRw402u121JJz58BkMold8IAzKblchHpc2Bs6SR0fTBEgo1I5vMTt6quvDnbBSyYwyoAVGB020np5hDNHRIYo+GAwiDOqyZiQRgJs4ERRCtg3ghH6kGXyYCyGw2Hcqluv17W4uKilpSXNzs6q3+/HZmIiVi6MwsDApiKILLqDWd93AaBhnwGCjtFib0Or1brEKDurxt8oB0wuABDGnfOqYUTJAmF0cZwzMzNaWFiI+uzV1dXIugAsmGfm3RlnnsHYMdYYDcC8g3dnPryczuvT6Z+zJl7iUCwenk61sLCgnZ0dra6uxrOyABbjUS4f3tnihwUgh56upg+Li4uq1Wo6ceKEbrjhBp04cSK1adzLvujn6uqq1tbW9JVf+ZVHVd3HVHMG3tPb/BzDydgJuJATiAZYcmcpcX7OmrPOOBMcIYw+wBIZwIBn73ZxNs7Zfa/JJUvF93Hozvg7uQFTKCVlFq4PjAXn6uSA713yfUTlclkLCwt62tOepl6vp83NzQhiYK1w7mtra1E6kgWgMPx+cRk2FqDj843OXU5P/W4R1p7SMgAHfoL5dV30cgQCRUA69pF5BwQwL/SJP8iDj4cxDAaDAEzMNzoOu8n4XGY8U4Acerkp8+SgkWc7+AQ8OVBoNpu69tprtby8rOFwqPX19RgXAAEdkRQy65lb3ildGqwdx9bpdFL+D/9bKpXipEoP7B07SIdkh9fbe8kbAYAfo42eIX8u9545QO6wB64DDuTRXXwe60SACkAkc8YzuKTTAwZshpSUmYIraJB44/H4kqwocwKe8hJkWG9+7jIEpgMnjEaj8NO+184DIAfHyL6TDlJyYA26SQCGPSYAKpVKgSvYmwIJQ78gYXiG76lA3zmm3vsqKTUPjgUd8xF8gX1ZY+QAmXByGDljTaTkFDvsHaX++EK3MTT6AWYCW3g5t9sj7ByySDVQlghz8sb9of8M/wmxcZR25EADw8oGbhbKnTmpYDaV0EE/FYpJ97o36sMY2Kc//WktLi5GCYLX2gPE6JOnhh0UeCoLIYFV6/V6WltbSx2DC6vpgN4NBQGGM5xuUFwYABGuhKSn2KOysLCg5eXlWHAWu1A43PjKO3gumQoE2zeeoYSevUDYs9mHubk5DYfDqPclkqVMyx0oJVOlUkmNRkO1Wi3mH1CD0czug8H4kx4FIBKQkCmBfUBRAB0YCNaGeXfjTvmWlJQRAEB9H5A7EQw5ASgKj/xxggf10WTUyO6QpUFJMQYEJlwOSR8xypQ/EDxzPvfCwkJkM0hbYtgY39bWljY3N8PxHDVd+VhsLpue2pYuLVX09fFypEKhEOQCAbWXDMFAIzNkALxO1mvq+T7/brVaWlxcDCPOOruzYyzIqwcq1PiiuxhzZNadmcun72f6XPOF0+CP62ChUNBTnvIUvelNb1KtVtP3f//365Of/KSuuOIK/fmf/7ne9773xZyjd15CyBil5CAP3xwI+BgMBlFeAnNPkMX8A3rYMM5c4syyTKv7E2yZlwTwOWytB2nMZTbDAtHCHDvAly7d2A/jzf+dREAffW6crfRAkD172DOXa99M7yyqB2TY3GKxmCK/8E8Qds7YewbD/Rl3DmX7LiWZ8uPW/ChW9tEAtpgv1hjZZY2crAOoeRDgTK6k8BP4eebMweRoNIpSPWTNZZHAhj753+4jpORODphnZIiyXZd5fD0yAAmwsLAgKSlp8sy+34fmtklKyCw+z7NddtBJ/Nl4nBw04zYKLIQfxoYXCoU4ERKb7OWXEK/cw4aeoKc8lyqKra0t9Xq9sJ2QGF5W775YUuBFv+DS55L1R1Y4qYvqA7cr7gv4N/YbGfOqCil9KhVYBBvswS2YlrV17DU9PR1EErLDevhdZvgjPgMhTpCE7jB29ymOZ5H/QiG5GBT7fpR25ECjXq+HkyGFA/PoRhkngyJgRN2ZEClTOoUye2kWhpWghtpJB5Bu4KXEcTDpLHZ2IxTCiUDBrl3uOZ7CRJjdcWVLkZh4T68icCg1dbgoEUKIg6E/KAjCAluB8WJfjAdiBHCUccGcwwpzrPD58+d14cIFdToddTqd1FzynZWVlcg8fS5w42w+/3ZFdoBGvSAbYom2cdYopDPenl7GidIHGED2ijjoRxF8fQAvZHj8mFqeOxqNYk7I3LGO9Xo9gAjG0dPMtVotjnb2oMLfWywW4zheTmqjxphLv3Z2dsKYYtAkBcv2/8KpUxhF6dKLgdCD8XgcDocUOfrrtf3oDEwueuzPJOjGkTvDxL9htvb29kKv/DQsNjmXSskdKxsbGzp37pzW19dT5Xqwoti6UqkU8sgY6QvAWkozTtlyQreh2FTmAMd/4cIFfepTn9JXfuVXpkrEvEyE+c2ycQRfzCEsoAf97njos/8cRw0T6xklZ8Y8IMFfYAO9pMCJHd7pZby8h+/Pzc2FzlLO66ATB8v6OAhxhtLr6LGJrJfbLd7LZ1kTnkWGQlLYGnfsnr3yjKczkBcvXgyA6Ww6fUMneCa+EzDOH/rKnKEnx62RMfJ9U1Kyvg62mEv23KErru9ebgLR5raXABR/D+HB90ulUqwBPgjdyAYTlH2yDnwf8E5VA1UCXLzp5KkTEw5SpWQzM0SLA0b8G+vvhAay51kwmmfs+D22CRySzfY54cq8ItvZzHAWQ2J3CbYgEKXkaHA/Lnx7eztI1PF4HFla1tUJI4J1cChZDs9KuY30zDFkCqWPyIGT0B60+HoxN663Xj7mfsnlj7niu8gkFR9e1uZz67bq4OAgdaANMsFxw/SNNUYHWEOfE+wueAmi96HakQMNQBPHvroS+wYoFN6ZfsCgpNh4PD09rc3Nzfg5R/vVajVdccUVarVampqa0s7OjtbX19Vut9XpdOLWTv+DsLrQHxwkGxE9fcr+Ck9hS0oZew8sPN3nDpWFdDbCU/ieHsNQUPfJWAnaXAA9ap+dnY2LeEqlw7PTL168qMFgEMEGxwYDWKWkPMVv92QuNjc3deHCBZ09e1bnzp3T5uZmzD9CxHG9GD7SeETYGAIEDgXpdDqpmj3WHmDWaDRSCgFTTxDBmJ0twjCzfhhomDrqB33eYU0o9UL5fZ0IaHHMU1NTUdLnAA6ghUxg0L2Wn5rUer0eQSLHHSNnfuuqZ4gODg6ilIxjo+kn61ytVmPM/X5f7XZbj3/844+quo+p5mUDOBuO0ePklyzbRqC5traWYtpcd9EZl3l009PD/B/j7ulo/s9JQpKC0SLIB+QiC+PxWEtLSzpx4kTUTHPZJQF4o9FIHYGLwUffYNXIzqBPziRKyQZo7A9jgfksl8v6m7/5G/3UT/2U3v3ud6vdbmtjYyPmDbKBvgNS/YhLfu6lZow7W4aI43XW00snYJhZH37vaXoPpB3E+9qQ0S0WiykSg4Y9pV+8349BpX/UcxPIO1HA3ErpPRf4Ew9uccqQHug2JBRr5Jekepkv/XI/6SV1Dkbvv/9+Xbx4MZVpJWvLOAlwCa7oK74ly3AypuPYmD+CNq8I8GOwJaVsCfMJIQGYpUKA72Gf+Dd+h89PJpPwyY4bPFiBZEOHkBFIBLeBHkB7Ngq9duLTx0+VgwNVQCa+ZzAYpDKl6FCWXKGhCwQrXspM3/1zjgOZVydLPcB1H+0NH8lzvUyrWCyG/8NXs87YUS8zxlczh5THsyZbW1vRh2x2Gf1nDiEJPSsC/qPfTqogY9lyXB97FkNySiHYFTnIZiK9tBRsxMWVfMbljbn0vczMJ4Ezz4dM9t9j053UYQxOLLv//HztYe3RACAixJICiPoEOKN4uXKmg4ODANA4GYRgbW1NFy5c0NLSUmQ82KzLovqmXASq0+kcDsgiLyJXDLfX7Xk5A33ESCPkjBHjxIJmS3OkJICaTCba2tq6BJg6C4hx5BmABFecubm5uEhrb29PFy9e1Gc/+1lduHAhgjOAOZs6YfJgCxEahAjWBWeHQQLkOwvQ7/ejzIuNyK1WK9h6ju1kzll7WAIYXJgG2AaOLKY+8vz586nzrzlWF8VyxseNI/LmhhqjyVpRXw7zkGUWPGCampqKoznr9Xqw32RHYGN8TxEsB84NWcAwejmfvxe5nZ6ejlvYnVHxlG25XI5SxF6vp/Pnz2tzc1PPeMYzjqq6j6mGrGH4CEZ93xTOa3Z2NnUcNjbCN8I6YEUOAfXINw7dSx1x6Ogc7JyfAw8j7JlV6VD/IQ1Go5Hm5ubUarW0tLQUTNv6+nrIA8ACoy8pAlj65Fk67AxOm3ERSCHjsKCeFURepeQwi5/+6Z8OUsKZsMlkEqAdAsFLotBDWDB0gubBA+l5Zzk9+81nAN1eOoF9zp4c5uQV8wAYcDLL3w9L6Lcgw3rTL9aN+WPOPYBDrgCnnnm4HIPr2W6+g12fmppSv9+PS9H4mZfJ+By5fngJGGVBbktgaxmDz62XblFOUigUUlmr49jY++llLO6bHeTiX7MHv/j+GtbZiQ1kCRkhOEPHkGXWgT1f2J5KpaKdnZ3UiUxevuX7GDw76OW4gElAMMQlOgAR6eWO3n8PbhwP0NAj7CD2xystHPBLScCAXhLsEmShY74fzk/Q4rsQseACwLV0iCm9pBSdcnKTeUXm/T4ZSYHnGAsygJ/1zCzYw7MsEC9gQD94xjMSfI71AUdIiv2i2FnPNEB0MCb6idyx/oyHuUD36TM+M1u+yfcojXJcQSaD8XqVh9sWdIk+s5aOA7GJD9Ue1h4NL53xgWHIJKU2vTqr5ylwTnwig9FsNkO4AKD9fj+cNkIOO0jggeHwdFV2w5ADQTc8LPblgiIUCwWF0WMxAPVuzBEcSXGilNclMwel0uH565ubmyoUClE+AxPLPA0GA21uburixYs6f/68zp8/H8w3ArS/f3hU3dmzZyPKBwj7KSeMG6Epl8uxN6BQKGhjYyNV/4+AoyitVkvXXHNNao8IzRUdI0qGC+PsrOkDDzygBx54IHU2eLYED0H31KWzUc6uoJheH8uaAgCYWy9ZcLbVAxruiiFIQqEcBJbL5chGkC1CF6gZRl9YB0kppdzY2Ijg2AMljDBZKk4563Q6kYVywHrcGiDAmS93wtwaXygUAlgB/sni+fo6WKCxvjBa8/Pz8V4vQ0BenbHe39+Pck7Yb3QZ0EwZADK6t7entbW1SCV3u934nOuTy6fLBxlfgiUMun8O+fTMHnu0fLMmmwEJpGC7sVHoKjrsQMRBgAfFZOR4H8yZM4+sD2TLaDSKwKHX66UYPmfCfN08Q40TZK0dDGGHPPOIfUDfAIgLCwuprDZ9Ru+zekp5k2c66DfvI+DDzh0cJHfzsLFzMDg8hQ8bhBN3ZtLn2f0rMsp7ms1mKtOG7GUzO9gpfKZnrxwAOaA+jo2j5SmRcTCfvd/JsYgTUV5ygl44sAbg9vv91KE0Huh6VtaDTi9h8ewdGRhn1HmeZ9xdJwCxEB74qCzAdx81HA7VbrfDh7m99HJxtwfICz9nDsA3zK8fVADw9oCdOWBM7LvAFqBb+Ezmw/eeoKuSwnfSHwI8l2kCMsokwQO+L8Wfz7PQVd9T51gWuwl+Yw6RKyeJ+J0HKB7YYJsIWlm/bJaHNfJ1cjLByTVJKZvhBBTPgvhxMsozZNhKJ+z8HchnrVbT8vJyqiyPE16P0o4caLijlpINlqPRKBQfR4eyZwVSSpj98XgcJyiw0RZgBejlbyaM8iGiNCYBAfRNOuwHWFlZUbPZDLCwurqq9fX1ED6EAoF0BgPwm00xEyihnPSDecCR4LjcsQyHw0h34cgROrIhpE2ZCz8xoVarheM6ODiI6BTjybp4StPXkGN2CexYS56xsLCQOpeakhYChd3d3WCVYSq4+ZwxYLzoQ6/Xi3lnTTHkrC1r5mk6+sxzPSvkrBLBrSsgBoMgczKZpG6CxxAhlw40YaLYF1SpVIJZ96ASWaN/rAfOic1vnm1xxtIZCIALjQunzp49q/vuu09nz57VxsZG9Pe4NuYfkHW50gApYfOlZE+UG190kvl09hawSNCLPHlK2I+FzZ7O5oG5s+aUwWDnpEQ+kRtYa3cegAgP0tn7USgU1Ol0InhkfOhklnn2oMHHw/wRxEkJAEcuvZzGgQV9hKgBRDEvBBHoroMjHLEHHwSEjNcDLc/cwkhj28iWs3+N5zBmL4lx584aYRMAW1KSdXE77FlQD2awoV4y68Ei8448MZ98htKH7MWbzDXBmtdme6acsQBo6T8ZGQdmnv2g765XvBfiAznhs7zvODYAN3InJftkAFNkNFlbnysP/gB9ZCzJsKGvw+HhDdhSei8Sz8nqOZ+DdOJzXlbD772SgO+XSiXVarW4W8kzK+g5JXQO5Nm7B9jlSFYvC4PUo//8mzIw+grJ4rYNH+3ZCPfFTqQ4AY2tBfuwFj7XDuZ9vdBPz5Ki58wHJKYf4Ypv96ysEwWQj/gJ1jEb6EPwOHhHd1kTxxvgBymxJY6XeZ77FdbCM/1u07AHHqDSD+QFLOLZsctlRLBHjgk9QJaUCjwhdSA9vdpkOBymjtl+qHbkQKPZbMZmJyaO5iDNWXxPT3kZAJOJghwcHKjT6cRJGhgNXwQE3FNkng6C0Sc65TN+a2KlUtHy8rLm5+ejfpk0LMAa0AobgMB6zTfvzJY4OBhGAAAnlAcRYFGOsbm5GScQwYiS1mJfhANyZ6YwZhgWnCpOEMCEorK5GUXNboYul8uxz8ADJUpXYNbb7XaUTWGIibiJ5Kkdp28ET84EMXc4duTCxw9LgRFApggypKRe3JWbvvsxs55ZQ1FdflFWnk32DbnjjhFkFKPhzKFvQMeQ828UEyV2EMtRq3t7e1pfX4/9LpS6ZY/PO64NI4vOoTOw1B68Z9dcSjtt5AiHxe892HBmybNm9MNrcb2mlrV1goVAHqDJZ6Tk1LNKpZK6TwY5dqfuwFtS1OnyWUgG13fGn50Tt7N8fjgc6o477tDa2pparVaw4WRksiDbyQmYKt8r5NkI9MIdm+uN6xv9YQ0kRZYQ203mxf2Ep/MhCrAp2EQpKf1wIEK2mvWG7SRggv12YOXB5mg0iqPGnRSTkv069B85dHIJW4LOs6bYU+bCsw8AWgcGTlaVSqUgnQaDgSqVSmTVuQyQkxuxT8yVzymZLM+YHMfm988wp+w/wqYDusEZjiUgGBzg4kf9GZALki45fKJYLIZc+VGlnvVy8sQBHusMQHSiAHmhFAs7iayPx+OoXPDqDQ+KIXbJ2nuwydpLCShnXPSb7zt+AdRCijIP6AelxQBu9MJ13KsXkGn2R0LsEIjt7u6mLr11wgWdpqyMzeLgN+bCK148uJIU9oU+A+bRH+bi/2PvXZoky7Kz7Nc97hEe14zMrOqqri5JSCAkAyGZMAb8AYwZGH+A/8cIBhgzDBhgDGgM6xYyRKuvVZWXuN8v7v4N4nv2ec5Kz8zI7pa6vJXbLCwjPY6fs8/e6/Kud629N7abZzsL6fFivr2ugr4Oh8O2zm847EqxTdZDlBLooLuuQqnBBXYSGTY2MC6hv7yn13H4nYyDPMcErAQv6JAzT+9rH7TrFIPmiA5nbEDuxssh7Eye04coE2dymM1EuXCM/szMHvdytoOtSa0UlGd5sZVZJQMHnCGRHcwpOw85IuV9cHpMPIpcn4PA2QDiqFye4TNCYNTPz8+bcKGcLrVIOmZ0ZeXh7BPWvDBeBDfLy8s9YIOik55H6TA+x8fH7WR2BNNsMmAjeVjXw447Sf80YxtkGgpmIONaTLMMBus4UJgWCz+fO6JHlhgjFpLChCBPNk5kjpxtMTvL/PO+NsIoJGPvHTbo11dffdULEDGmzLtZ/HkONMh84cTRE48hssE4VCbLmQ3Gr2Y2cDSwfkkaoHQ62oseGV/XfiMv2CjWLlHuybuMRqPs7e1lb28vL1++zMnJSQN7PAvjDpmBQzQo5Adw4RpbOx7rusvz6Ov//b//tzHo2CTWX2CvCWINSAmGsJNcPx53m0CYxXTGmW272d6V98N+Wq7N6NWMMXZrMpm0080NjjxXJhmcQeE+yJEBHRnK5eXlFnwtLi5mZ2cn29vb7TkEKtzT2SzX27ssmLEwMw6ry3ewXVzvzDPXYBt4Tz/PARM2FdvHs13Kit3D12IfzWzOU2MNnH0dGTD8FGOCHbatdXCOHzXQRtf5LElvo5mkvzUwYJT7Jv1sB3OFLBCsc3/min7C8iNLrIlljr2pg8u3DJQJOvmh7wbUzrDalgDYDa49BsiVS5etB2AeZ+ywyy5j8zoOdAA7A6appIAzd5AyPvPIGURIIROrjJf9NMStMQJjj/8w3nLFC3bcwQkyh/0xxnPZvjPCfk8+Q278zugu9tMNf0gAXLMX6ADjw/uaiCBwphoE3aJ02PfhfR7THh1ofP31123wAKdm2/0yBnGc3wBL7GiMF3ZKD+fKZDGYVio7DUdqSZoTSDqBZ0DtKHxuh9llnleNMKU0SVp5hAEtfSC6xSkn3d7OlB7wTP7GpHmxK4aK8TDo53Oe6RpIH/qztbWV58+f59mzZ20nhKQ7eRK2wk6IheywBAsLCy3bcn5+3oIHHBYGkD4iAzg70ty8swMJxtmBoo0rABtAZ+O9sLDQZA9DZ2aB+2N8HDgk3QFZLkXg+qQ7wdpO34wRwRj9hYmqdfjIaV1IazZ9MBj0thb1gnNAqgOax7II38a2sbHRKzdbXHzYeWNzc7MxWeiWgwUcKfOJrhGcEXzgDMxIuzHfJi284DnpSo4AckmXyoek4ARVnAEZCU73ZW2AtwI3a8WCc+aWftFHB7nWs+l02jLBDswduAPm7+7usrm5mdFo1IIRvgtrurS01DKSZiFdmmM59XgkHUhKHsr97u/v25oY7BD/x+4amFgnHEwA3uz8saO2GRBY2BuIA3SHUi4Cfhg5+uHFu9hIB1M4XTYV8GJ5Sm2RM8rAAAleDO/3tB1DppJuVzSPw+XlZSPg6ONw+LDu5fT0tI0D42fGtdoKymvQpXlt3/ve99qcI0ME5ibcANzYcBOT2Bh2L8KWO4CzzUZe0R1nApyFcnDN5/gGg1ETD1yf9AlH5pZ5TbotfLkndsrgFcKVLe4pNweTWPfMwiPD+G/6b1ml/xCugFtssglgrqGvXiflbC/z6NLhpMsgVhtMlQQlp/iAiucqqDaOMpnN921HjSmtvw4o6bNLs4wljXEdSBnLujLDAQv4xnaf+zBnBJ7IiLOUDpBsFyyrjKvJUJICOzs7bdnA0dFRk0m/J7L9mPboQMNCbAfLJMGEYWxpVnhexowMRsD187BtCNfS0lKLXg0KGcxZESyCMplMenuZO6pE0QB3ME84QqewUAjKccymOlJkAniGwTyRuw0QLDrg2Yu8PEY2LjCxZCQYCwK7zc3NBt729/eb48XB4qBw/qenp73AEVBkRU3SAhBADEbVDOTV1VVOTk7aWg7YS69NsMGygWH+WBtRHSZKiFy5v/SFABWnzbXcx+wA1/Bc5AOQApNesw8swnTQYeaTaw1ybbjNyHAdDpGxd3bMjM/29nY7kGkem8uWbJiXl5fbrmnWa3QAvWWeJ5NJTk5Ocnx83GQUHXVZRJK28PDq6qq3wJ+sJ4bWASKywz2QceQVO4h9YX0W5VXo5urqagOFOAqDPPpq2U66s0XYjtsOlEANO+e+Vh2BRbSd29zczMHBQe9sAO5hh8pzk27DAn6n784iAPYpC0DXWc/l+9TyApermGxgfPk7OkcgSR8AUXWLaoKJk5OTdj3jhp2n/nhhYaGXuTSZhLzUjAnBMAGL2WPW2dBvP9dyjJ8AiBhUDAaDdl9n9QwqPCf011kS/oa80td5zWig68gI8mRSx2PJDximEmLsSIR8mJjku9hlZ0yNO1zl4euTTnZMnjKvlBxOJpN2CjcyUrMg+CXuZb/o/tI3gmJk0L4FUEkGB71i7VnSZTmcgcPfQiqAbch+Ml6Mo4MK9M2EEbq0vb2dp0+fZnd3t7d5D7Y5SSN42AbeJdY8swYy9sn01XbA61esezTLj/UO3MvzjDeq7Digwo4ZDxm4u0SPOavVM5Y/ZAl5ILtuGXTmgTHie7a9ELvMHcSbSWE+4wiEx57p9ehAA8BoZ2bntL293QaHBczT6bTt2OBDVxYWHmrSOUuCHTt4KQckvJx3DrLiwP6ZCSUdh8FHKFwXjjIjcN65BWHxhFvgWbPAdQAl+umUGZOMURmNRr3onutg8FFoG1EyHAYpBBKcoE4pSj2YpQZpBAuc0l3rG9nliHF15gBwz1jRf7bjdQ0yAZuZwJWVlbZOw8EqNaeweYwtyo8zZ95rKtUKjNLyXBTLrJJLUTD4BLoueUN+t7a23shuoIz0mTFmPp2ehrUw+2jQgDzYwSFPjBHzjRGe14aj89hfX1/nRz/6UW/OcMAO7B2YMg/oi8mMOsZbW1vNQfoanuE1OdyfkiUbZWwD88PBopQgESyurq42+3d/353n44DVWUyzsWbrFxYW2vaPOJukCyhMOAwGg1YCyXtgT9BFiIUkvcPczGRyfzscg1czgTh0yi3X19fbJhvj8bi3WYMdV9Inrhww8TnjbODu/yMPNDJcrC+hYTMBVUla+QnyOJl0i9AZe2wLpam8s8GLF7MarFFiV7NYBryWBebaddX+Wy3BqFkLBwzINWCU+QXUMN4w+fPYLi4uetuoMmcQGN6MwT4QQMj41qANwgc9A/Ngf5wtYE6ZiyQ9mQBfJN3ZJXWnIYJobCA7z9kemTz1s82yW7/x8Q6ouZexjZl5xqBeO4vEMGGWdOsb/c5kCF32SWPMANCUSBFkbG9v9zZRwT6yOYq34+Z+SRrJ6gyGrzMgRz94V/QEv0BANwt3IGvGGElHLHuNstcSIgPICjYffWSsWCfkdT9kLiDhTZLXIAM8wpbxYGL7DNsj5JNKJWSGwLHaQnyN/eZj2qMRC8LLQ2GCCAYAuaThcCzewcjAgK1FGSgGz6UEBAYWSibeIHQ6nbYUnQXeYBLm31GkQQQTh3C0Afr/gQd95H2ZMN6Jd4Ud472SzrAgOBcXFz1DhJJ48RP3RBlYa+FzLRBG3pvx8EIp2FSYOhsFBxCwsjXIcGCJIcbgOiVtYGEDC5hbXV3NkydP2jvt7u62haos3L25ucnh4WELrGBOqBFMumjeclANJPNeU8r8bscOC+qDvGB7lpeX2wL6+/uHxdxkf2AIMIasWfG4mcF2P0lbu38YR6c+CQqp0Z1Op+2Ax3ltZNNoGEEcC2PGPJgsqNmzWQAj6e9Nj+wSDLMTDPcej8ftEFLsFFvVcm+CUO9mBYAhg0hm0M6QBbzYLy9OxwnY8ANyWDiY9NcXOABFZnh3HJwdIM426RwypRQwv8iW2Vp00rLJNa5XrmQLAaL3w+fZ7JZnRtCbQ9jmG0h7u1n7Agd+SXfiPH8DWPI32yJID7ObgBu29GaOKfuFJKIkhXNS8A+QKLNK/+zosb22EwTABEgGQUl/wSl9JjthkGtw6OurvjlLOI8NOeR9GX/AHGPtwM1+0kEcY0AQwPe4j/27M1GANnTP8mayzIw5GMVEw+npaQO1Jh/BItfX123XNwAwINDlTzSTqkm/VBS5JlsH2KW/zgIm/R2JvObD4NhZD3+OjavPJahnncnGxkaePHnSWHg2NYAgOT4+biXnjDnBP4QK8s4YOUvIeJsUQB6c8WXDGubNIN1BStKVdEEQOqPC3NIffIEzR8YhtfonSctE4Q/BVC6TdTbV5Aj9JmvJ3GMP+I4DE+Qt6R/ESn8YY2TF6/1qJv5t7dGBBqln7y2PYLlcaW1tLZ999lmOj49zenranI5TLQDbly9fZjAYZGNjoyk6Ew3AoNlg1xd0ai/pthZj+1wHHGaMMD6ulbYDpiE09A8BZbLMNmIAUTiiZ8bN2RqMCQDKteVkNAASXkA8nU5bhG/AAiviXZ54Hn93XbEjb9gbOzkHUwAJ74DjvzHulTnY3t7OkydPsre3l93d3Tb/DvYMCDhoC0VDjihb4zR5vgN4IfDBCBDcUNOM0+EztlOuwRHjjVIRWI3H47YzifcAt/FzUOHgC4DrOncHC2YlzLBhQAeDQW9h+Dw3vzs6CDPnADvpMlbMDewZYBQZQCbRR4w++uy0NYZyZ2cnz58/bwuf2QkOp+H6Zlg1ADqZBsqj0HHO+CEraCBHIMQckg0BnCZdWtuOPenKYFy+RZYQFp7mVLnLShlffhgjsokclsoub6599zoT206XKnD/GqDVYN818wZxPA9GF0LFssEcELyZUHDGgYxkdYJ8hv9h7tBzvk+whhy6XAZnPh6PWx3zzc1Nr14cYEnf8I3cP0kDOP4cEGCfmnSH9mETmYuaBQGEODNIDbllm+vnlbAgM8c6I/zVYDDobYnPmCbd2iXm0Owtssnnzpg5a22/z3ecxXAwmXRYIemXITkrZr3iM8sFIJmKCPQXmTeZBQ5w5sZZDO6H/hjDOLC1LGK3KI/nHgBdZ+yRPcgGr8lAP7G/fB8yb21trdlaSDtXfmDXeW/8vsuOnIFJOsAOgVF9s/XGmUFjTNtw2z7m3WAdPTVRQsCEvhEoInvgP96DvnBv42AHz2COpDt3hO8xvxWfJV1W2qQn//qMKvtp5IIz6ey/7Hve1R4daMCsUp/saHt7e7sJNqzCzs5OkrStHv2iLBA/OztrdYmAb8CslRAl9oIqOztPhIMHHLsZQA+MWc+Li4sMh8PGULL7CAbaKVGAP+CxlkQweewaZWPmlCUOlvpinC4ggrIo1juwtzbjRRkIz+E8Cz5nPJK0OlQzarXWm3F1fSTv64yB5wphQzBhK2Bld3Z22kmSjuZdasS9uA9yhPLRP4AIwI9+wXTCOrLvs/d+RqEJIs1EIhcAJIyI1xXZOcBGAU54J5R8PB637ZFZjO/Fs2wm4B04zLYtLi62Q7psMMhKOQCft+Z0rBsG0CDIjgOHhmNI0ow6MoHjweE5SMcIA+BfvXqVg4OD7O/v99htzj8ZDrsFopAEZ2dnOTw8zOvXr1sWJOn2ZU+6+YW9pP9m0+z8cU7oJe/nbA42zFkObBfPR5aTtPM8kHcYTGqbnfUzuVBrwpE7l4gYMCX9tD7X816MedJldWHZvHMLWaXV1dVsb29nNBo1sIxO8T22fPYaHsC5fxhDs4fYcfyCAYqz0rCl6Kb9DLbPZ6q4XMLsMP3AbiDHjKEdvMs1bK8WFrqtSvEtzDvvyn2wMc6ke0ceACq2rOrgvLSjo6OWobRtxqeAQQyWTTAyxpTtsq4UkI88OagwQWQW3WNIEGoW3HrBvbweB5LL2MLlOtwXTODA27tAIRsG0SYm7b/5l3tbb9EL94Hx5d0dyHO9AxkwonWl4i9s1Xj8cEgrMs7fXSLEvfgdWXf5NiSGwTlgnb95PBlHk5OQoCakku4MDOSqkgDYR3ANvmg8Hrfs6awKENsn7AjBiLOzvga7CU4eDLoqE+MFxtr20MGEfSiyaAIZ32nMRNbXJV4O7N7VHh1oMFlkNcwGjMcPCy29LdnGxkb29vby6tWrfPPNN711FYuLi9nd3c3+/n5OT0/z4sWLFnSgPK7BxSg6DeXFgHY2BCQomA2RWS4G0QpWlc4OGeU2o4BwOEIEGMAwJP3aPn74nFPBnfnAcROZXlxc5ODgILe3ty2qvLu7y9HRUcsaUdaDkMJsGiCglBZeBJhrXR5nw2PWBwVwwOEt0bxICHDAd3lPFM9BYtLf3rE+AzACMHT94ebmZlv38/z58+zu7jbD5PIrM1EOXDHMZlMAkxgfQApjaWdN4Hd/f9/LYgBwMD43N92e91Zu5oCsDgrOmPB8ZGBeG/qP/uB8MYLVqZslN1tnJ47c2glbN82CUeIA6D46OmrOA8ONMXdWCcDuvhNQo/PI1vb2dgO1R0dHOTg4aCxg0s2n7Qlj41IrzrBxcMbn6NDFxUXbLYls5cLCwxq4hYVuAwicB3LGWLnkzw486ZywAct0Ou3ZIL5vcoB3ubq6auOVdDu5OUhgLHHAt7cPpxrzHn4vlwsZyBlUedGj2T+zvdyb+WDOccAQGQYF2EGfucK7IEvYS48D7wZpwfNw5NgRjy/+zWNJv8mSko01cFtfX+/ZF+6FfNMX+6J5bJxFg5/wluDJw3idnZ29MQ/IKDIBGHWgj81hnACfBmY1G4Ju4hvq85Iua+1AHls4HA5bBsAlMshTkh6wd2abfuMj6ZPX9fAuLjFFPvBtSVeuhX1Cl2ynjIl4v5plps9my9FFBy+MxfHx8RsZRd4f+aWPYEvAO+PuMXcJore9pQ+QMowVeuRgYTrtdiSroNw+iowvdhBAT7ADicKYgfWwj4whMmZClUAJ20TmB5lhDv0ezuibKDLxzLjTsCHYauQM+4AsEXyR8aUPj2mPDjRgCJg4wCUvYYPPYW5ra2vt/A3SYmZXnj17lu985zvZ3d3NwcFBO6QMptBpeCYWtnJ7e7tF82bfEHgPjtlEK40jPpw1wnl8fNwUxuwhE4tQk74lE0KtYWWfKwhlPceTJ096kTTPcFkFWR8cNtmNV69e5cWLFzk4OOgtfOJeFYgB5FzXxxhjbM1MwMbBFPjwPgMyol1nCqbTaW93Ku/SxLyYmYNRAFTYkGKwXOrh80XIHOzt7bX7s7AOo4o8cjAefacfzlJQZ005jdkdAJkP7yNTZSdEIHh+fp6Dg4NmlAgAAa8GTZSq3N7e5vXr1w08oNyTSVffOq/NGQsHGOyUNhwOe1srJ/1UPjpiFol7cm3S392Ie2H4sQPoBLLq3Zl8f8AdgQqOjRpgywCA0uVLzBeETGX66hao6DIpd56HDlAaiHwAqlnj4Y0DBoOHk8c5LHJxcbF3+KNtY5JWtoQtNRBGJ3Gutp8G+ADbpHP+zI3ti/XB72ubZLuFE8XnYL94HgABXSKYYO0bNgEggE9bXFzs7QhYzyBw8IBtr34AWYEhp04aUOmsLe8HceG1JwZjlcDC8QOm7u7uWinDdDpt9oZxwMZAxLFANOk2OZnH5owfftIg3IGrmXG+i08yUYSPsewAqBzMIWvMNyAbG1CDFe7ltZ8812sknbVy5QD4Jum2vsXeYTMIWiaTSba2thpw9vED9I/vIxPua9LftQy5st4iy4ylMwDcm/sQqLlaAZ0BU9CHlZWVjEajZs9MrEAsopdgEAdEvCfYyTjQGA8b7cDMtt14kmAGvwBRYOIFGaGUCJzLfVnbzBwxl8iJNwNB35O07AV2HJyDz8CH1AD59va2t/05BJdJIWwf8rOystKOIYBAgoyyX+CQUNa5QXQ9pj3a2hAcYOS8+wCBB0pMPSGOkBO2j4+PW4090f+TJ0+ytbWV3/u938twOMzx8XF+8YtftMV2KBvGAAU9PDxsAopjRCAQBBQLwa/pwlrzyP0BlmZDDXAccRog8FyehwCjtGYBCMIAITi66+vrtgYARXb9P9mLg4ODnJ6e5uTk5I2t5ni/+m40BJV3YDtZHKbrL/l9OBxmc3OzycDKykpbkF7TqsgBBslgDCHGmZrR8bMIKB1EMvY7OzvtWhZsUp41mUxaIIZRIdAYjUYtkPTcYqgorbHhMiB2EIEhIBAywEFHUGyMXjX2lkf+jxI/f/48T58+bSwFcklJ1rw2A1DYJdefo8Nep4Gz2dvb6zHUsxbXOaBfWFjo7WiHTbBx9g5FyFm9Z9Jl2gCmw+Ewr1+/bvdm7gE3dV1I0t8/3+UJlPIYQOEgXfaF4yADQZbv6OioBRWTySRHR0ft7wT/PAcnjP2jHyyCTrq1IoBZ7BULx+mn17KhOwZbvKftpVl16wO2k3Gz83RGFmAJcKuAGQdLH7HHzAn3dH+S9HwavsuZWuS0MtjosgNbv1fSrfFjTAED2Df7IcuqMxzONCO/BK7e5c8lYQZQJqLQK9593trZ2VnbXADbuLS01DYTSdKCkJubmx7YdDb07u6ugTKCT7LF3Nty7CoL/LFl+23rOvCj9s3YHeYBGXFlRtLtrMm8Elggy3wfeeRAUZhv7CpA06SCA2njFt4BIG8sAWhO+qeIcz3XcD1kGjYZsoIxWVlZyf7+fr744ovs7Oy0M7vAQfhRxtRBYtJldDy/3pTIhCr9xD9DbPAuzDXvDTFhOwHmhAQbDodtJyaCCweJ4EDrpUszWVTO34032PFyOp02+8x9XYLq4ATSznPKc2v5uM/uMgkCeUO2HkyI/bu/v2+b9kBcvK89OtAwS86kuuYfh4NBZFHh1tZWY++fPXvWwPHBwUGOjo7y8uXLbG5u5rPPPsuzZ8/auQ+vX79uOw54AaEjP5cI2QDbQTEJDB6RPwAR4eFejrxRdIMePmMCMEDj8bgtHmWhKM4Jp5D0F+4Q2SM4VhD6z2I+xoID9UiPu44YIXeQZMYeJ4qSo5g7OzvZ29vrlW1ZIBlPg3LW4SwuLrbddrxIk38RTjMi9I15YVxIY/N/GyzuwfublYQFwaCenp7m8PCwyQPO//LyMoeHh+39kFtkgcCBfhGAUSrI3N/c3DTgRQaEMh3YJOTJQSt9pZ8E6NSFb21t5fPPP8/v/M7vtB26GG8chVmjeWyj0ajpluusk/R0nGbDiiH39qSAZ+RzNBr19JNSIBwxuoXxZ2zpQ9KVCgD4cRwGgdUxGWSS1UAHAOVmIgmAABlJWvDtUi3+D3i1YyR4wBaTWqfxHLO49MEBiwE5wAYHy7jzNzt1ShDor8GXHXsFArzrZDJpNsefIfP1vB4coe12kubwHLjWsjEcvgMC7Lx9iP0cAYd3s3GgCZjEjiBv/E5pqzOizpJhX5EbZAHCxP6Vvjq778Bhlszgk2GWTXbYN85b++yzz5p9tX9zEGli0Jl+5CDpgLZJPeMH2GOAKrbC5VQG+wBnsvLYHuYXGcA2YDvIxmD3nH2BCDAYdFCAreOd6A8+Epl3poGMIHiDKhPjK/rn8YO997oJ27WazUAf0EeD1aWlpezs7LQM/Xg8ztHRUS4vL7O6utp8qkvJsCvLy8uNPCQgdNka48mzjOuMYywnNJN6bP7hMt/BoNtwAD3GZnq8uZczEPQPOSFrQIWEs+58xvOQPeaVd8EWEXhQjsrfsTHYE/uP0WjUMCJyPx6P2/q+JC1ren9/n1evXrWSr5OTk17Q+r72QflThNjlB47qucYLaUkn4VCWl5fz6aef5unTpzk5OcnPfvaznJ+f5yc/+UlevHjRnCVCbEaqOggGlEiwsuZOdWEwEFYbHS8YtpC0QVrsTkHEyPsawKxTY4yNI1oLnO9zd3eXk5OTHtuJgLE+g/Io3s8pNwdCzkgwR2Zy1tfXs7W11Rwo4w3bgDGjryiDI33e6+TkJMnDlqVsi8tzmIPr6+t2Tws09/Z8wh4C7jGIGBobDIwJQP/4+Ljd1yUCRPYAQPeB7zvIMAuAUcWg2Bk585WkAQZ276J/9InyDJevsXB9b28vX375ZXZ2drKy8rCNMaUXAAsHTPMKEJK00sKkf6ouY8V84RzNxDP+OCCz1OwAhdx5nQwG+O7urjmJpL/3OrpVQWvSgUNS4S7xwTGgi5QAOJWPTpJqRp94D3QNuary5WwG+mwHzSGa1abQP5yqFzxyX6/pst7XecFu2gYyZt6aG8ftIAq5x15CRCHLDvZcsmZ7j/107XHSnVCOc/bWmQbwPpXcrK0zCoydA0xvxwzLCCggm+4slu0v8uw5RvZ4b4Ce5Qydh8jATnJv7jUcDtvYMwb0E92p8wz4Y47msf3Jn/xJb35fvnyZr7/+OgsLC/n8888zGAza+qskbe7t+wF4tv8uxRkMBr0t5PkXIomxxrZvb283uVxaWmrZg+Fw2AJmMiw1m2ddMfDDPvIM7AZ9wcegz8ijN6ZwcI8NTLog3XX9EDZJh1+ssyYy6RO6gIwhn8gvY24/jC4jpz58jyycMzWMi0vjHNS4dJKdRh0YVvKDsbCPsX5gfxcXH3YJPTs7631ukga/jMyxztjgnv7bNvOOSRfc8H/Gkn5R1oTtQ28JMFxZ44oN7Ndw+FCO7G23azkbcjUej/P69esmG3d3d60c8/7+vlUPUX722F0wP6h0yiC8Lhpm4nCKOF7Su6T0dnZ2GrDlqPNXr17l9evXbUcXggy2N0RZmFBHc3bApKGoH3aUj1Dyf9cBMjHeScQGgHemL47+aIwFpS+bm5u5vb1tWQgz8xiKu7u7FsU7OMMJkboD5JrB4f1h9hE2GDWUAYXf3d3NaDRqwQUH44xGo55C8H5JmpNLuoWcdvLUlhvAY1DM8HJPAyvGsgY8zDX9Z0teFA9QBOA0QEMOYbaRP4Il3p86auYVx212gHTt4eFhU1quQZlhGgBCFQzyDGRre3s7z549yyeffJLRaJS9vb3s7e31go+Li4u8evWqyQLAA3bVrMQ8NmTIC4EdXGOQzUCjG9YD5IAghECzGk0MKo2MitPDa2tr7bwEl6J4+2gHoTi2uqOTtzflcxMxyIEzpoyJf5CF+/v75rRgxxz8oCObm5vNKaGvXJN0h9Khoy6hquRA0rGeAG4AvDNxjAUlDpAEBtT0mWDIttMAAnnALtjpV71E9pER1q7BnJo0IjDAxvuZ9B2nDUiDbXTWgrHEsTqbmqTZAewEtoMgy2VUkBGAWewd78MYW0ZcxmMg7KwRY27Z4R4GembwrRfz1LDfgPkvv/wyT548yfHxcZPx29vbdr4NDCxy7VIVl/0wZy5NQ26urq56gborIpI0ooq55Vwq2w/IAECxSTz8h+0Cc4ZcI1/Ign2ScQy+MekHk7wvvp5gBx1nl0YwFFk3rgWDAVTN1q+vr/c26uGdeQfk0STr4eFhj9wElwHEGSdvp+qye+YN/XDpJnpkAoHrscnotwNN6xNjyZoLl0tB7jqLaL11oO95NjY2WeJDoMk2YcfoP/JIcIzNxM8MBoOWwWWuffYX710xJHaDaoDj4+MmF9hr/CTkHPLJ7+9rjw40GGynZszqMbkYMRwtA8QE4oTJbrDr0t7eXs7OzvLy5cu8fPkyd3d3OT4+7gF0HDXgEMfFwCHUMGgMIOwwwlIzG3byTD6COJlMmhM1mMYBcW2SBgR4Ng62RvGO7nEK7NuPE+G7TvFV5sNOkEXSOzs72dzcbA777u6ubZW7u7vb+kdwgjDTF5g5OzYCBjcUnbE3425WFEDHYnkrqtklDGnSlSeR0fGuIrPAJnNCyZoDFhSDhfobGxutJAmWBsNAqhJWG9DJ7/SFBojhHS0XzBFGdHV1NU+fPs2nn37aDgBkjuuhhEm3xRwyiMGZTCaPXoD1bWzMt4M3ZMesmg+dM3vv77Ols8+MMYtIidvm5ma7d9IZdgzxyclJu86Gl3phwD16CIClpno8HufJkyft0L7T09PGbgPg0RWcjJkw7KkBhJl2nynhUkHbUYCGCQv6ywLFyWTyxvovZJS+erEwzhGnitP03+28kVmasx7oGn2Hza/yAKCn79hjbKLLy5gTfhg7dAm9xNZ48SeyYLDBQkd8GkQG9rpmqQBhAInt7e1md70gFNmzvaUEB8LOmSAzlvS/BgfYHWwlgQ/zQZDtIJdyk4WFhcYiz2N7+fJlnj592raShhBjDhcWFtpOP5ubm71zj2Bl0WnKdSnHQT/ZOZFsqXdRxLegU8jG3d3D2lHsuc+xwodxD7PyPJ/PAfBc4/Jrns3fsSvci7UfDjCwNUlXbmU9vLy8bIB5MBi09S+2zQTVJjoYU4Is7LarEeiLSV9XAjAefmd8MAHe5eVlCzrQb2QZQA/OQyfoD31aWlpqwRP6xPoEdNgZB/Qt6ZYDAOBZBwZWooFLHMTx3j4kFXkAqy4uLjaciL1gzSr2AlvoEliXaTsJAJ5EnmvlCN8zKUpGBoKNv62srOS73/1uNjc3W8DB2sAXL178+heDUxtmw2VBdmrFBtjpXACsFzsyOFtbW9ne3s7z589zeHiYFy9etP3qr66umpOiOXXHJAM8EQ6Yj8XFxbb/urdCs+AwkU5L44xttJkkpzBhkWAvzOjjqEkx8s4AZRh7zspAgSg78lib3cZwkEEhhb61tdV+GBeej8Pl/a+vrxt49k40OGTmGqWrgQfAhoYBMauCYQJ44yCdusNJAAyGw2GOjo7y6tWrJsgoHIDfwIz33t3dbQuGbWAw5ADQy8vLNh4OdJ4+fZqtra12QjqHg+Ek+C7BBw7AgQpZNaeCkZulpaVW6oWi82x0BNnyblmAt7OzsxwfH/cWMM9bA1ShW8x50oFHgJmBJ6VRSbfF4XQ6bQb6/Pw8h4eHvYwg68TIYjnrR5kPLOB0Om273dmJJmm7PKE/BCrYDUDl+vp69vf3m9wkaRlN5BWD78AHvSDbZ933mDE2ONGlpaVsbW01tp93YXc4gH6S9r7IGCx8dfrINJ/bCWMTkjQigHdh3JIug4fNo3FfAA2+AZti9tHjxN+cEaJP6IbtC8AFHfM+8zVLAChAhvBdLusD4DvLwL1w/oz5ZDJpbDZ9geE2IEy69XrOFPFsxgqyiHl3hoK+n5+fN59KZsuZIC/gdaBUyaN5af/jf/yPPH/+PBsbGxmNRhmPx20tjMeOctWtra3c3T2s3aP+//r6umWZAX/IH/7IPhO7gn4mXYUEY3x3d9fKktEfrnVZEvYPe0BfuY455jueK+QdvAKpS6bCOp90O2U505V0O0Zh15K0AIn+U8UAaQJ+IQvqAJ7MAvjIdh0MkqSdvo2dwW5D3vgQTPAf78D4oBdLS92Ocia7IYdt0wHHzqgwh+i/KyDQLYJF/Dhzhk5iRyshg90zBgS7gX3RbYJ+fM5gMGjyTLBBSRgyyNiZaEk6DAGRYyLY5WcOTrBjrM2g9BTcSNKAUjhna53Ve1d7dKDhKNqRoFk6AzovhOZvMA04KRgk0tUMJItUdnZ2cnZ21ttZyeUEAEkUCuYYgXapDoy3y1S8/ZtBDcCQwV5bW8vOzk5v8TOOhJQTWYzDw8McHBz0mD+AEQwpQAKgzyJyL2hCEexUvBBwbW2t7UrA7lX8fXd3t50jgeNxwEBwQ+kTRtYGCAftFChO1caT79eSALP8NgxcR/SOASFo8XsiJyzix4FTjzoajTIajfL555/nk08+6aV2k+7EaOYFhWbHLJSetCwG7vDwMF9//XVevXrVnDZ9ckDKFm81c2cQUDMdzpBh5Ay8nHLFoJ2cnDTgXFmUeWuAYY8lgMAlbS6RI4Bn3ADRGHO2Lf7ud7/bADep4qTbCAJQhw6enZ315pXxxhY4oHaGjh07vOOSgTBZWqf4naFC9ymJoBSPRaQGp9guxghgxL3YdQqA4q1vDRR4PjaZTSbQFZcJmRGrjKqzv4wdDtK2ytswe2G8ga6dLg6MsXQ5ggME7B99wkGbjTaZ4lIIBzHWSfyPG32pGUtkEXCHfHAelNlt7KB9JOODTaMMwqRbZa/fVurEmA4GgyY7llPsC/f19/nePLavv/46R0dHLbgzOQg4M3NMtgLGO+nKUJxJT7qd516/fp3T09Ps7Ow0v+KFvpYXs8kuM3Rw6GAXn4F/9hoO9AodMTbxbkXIkYNGfDYZiiQ94rWWhxGMmrzgHoPB4I2sHwFR1VUHZpAi6Cb3wG9Np9NWPu+yY0p2bm5uWoAHPmCuTP7gZwkanUllXTBYA3uHPXf2yASSS4qwQS6nqliK+alVB4yrcRBzWvETess7UWZvkp5nIc9k2ZFV4w7GH3tj8hJZBdcyZiZzsMnIFxgz6U5ap88E+Y9pjw40bLQYWCaGThOFc9LjeDzuLexlAlFIs+Le4tECsb+/nydPnrQSEwYNgUBguP/GxkZbj8BnNzc3bctT6tUmk4cdT2CXcXwAWKI62Mvnz5/nyy+/7IFL7re4+HAA4c7OTk5PT/OXf/mXba0J48WELS0t5fDwMMfHx3n9+nXbVQtQ4oiVrUwdpRPAwRgC0GB8EXxHs06dEdhQp+nyHAKtpCtf4nsom5kUlMGLlzA69Au5QPg5m2I6nbYslp0gY0FgQgmNg9Yk2dvby+eff57Nzc0WsDKXZHwoRTMrOJ1O26KtWWAnSe8EZbPrKDjyDuB58uRJG0tAHrspOUu2tLTUTj4+PT1tZ7U4u8O4kx4mg8OCX+Z1Xht6n/RT0g40Sb8bzDEOy8vLLbWL3rLWCzDIc2CQkvTYKsaQhfes2wAQ8zzS16yNcRkQsg7jtbCw0GyBATVybzba5QAAS+adsfHiVOscdcqAUwNamDDGEMcPi+jAA5YRW+JygNvb25YBInuGThEYWtaTNL0nuEdXzZw5m0mz72AM0Ct00mvTsOkOVGkEEACApL9DGIAOJ1oJkclk0tNtl0/UgBHig2dgvxgDbKfX3TH+9nHcm75UosmMqFlLg0sHHAS7PMvyWQGHs03z1JgH7LjXclF5QcDNWDnraHDsCoWkO1AUWUNP8TvYdOw64+qsn+cRgM8cYge8oxV9RE6YV0ps8f3YBAeX+GdkmO/zDvgrgCSygf2qpY7YBd6RtUvYBb83QBR/DyuOHmJvky5LTDma7ZoDGYIXgPPm5mb7nXmEpPF90BPbSewc4JhrHGSxjpb5sw2Dtac/NQPNvFbSnWvqe5q0Qr9t76bTaVt4TkDHfU2gEPC46sZ94D0dVGO/kRdIE/qKLC4sLLTtkXm3p0+fNrnHdyHbjz3T64NOBmcizQahoI42MbB0hENHqJOHzTk9PW1gAFAPY4lgITC8vJ03g06GgP6dnJxkc3MzOzs72d/fz97eXp49e9ZjGOxcj4+Pc3Bw0IIdjAPM+83NTV6+fJnFxcV89tlnefLkSfb393NxcZFvvvkm5+fnPfCzt7fXsjKkMnFat7e3OTk5aUHK+fl5D3C4JIItZ6mxJhBCCHH4NWXHGAEqWHCOYGPQACEoEOyqDS8MhwEzTpI5NwjHKDBfKALBJPWyi4uLPca0LjpbWlrK3t5eY4d5T0oQmN8kvV2nMMZORR4fH/cMoLfD5V2RQ+aMAMOB02g0ytbWVi9dy9hhSLxfPcGfjTkAxcaYRYz09/r6OsfHxy3gcare4Gce27Nnz5ojI6jE8OGwk+5cipoSTtIDVQQFOGBnKJyRJPvA/V2u5TnDYd/e3rbyKNY4ob8Y9cvLy3aeDQEAMsjc2phjz3geKWkyVZQ81Dpg7C2OzNkcGvrHOFG+xb2QdwfPjDcBOuPO+7vG3MwYcmgCA8bXC6yRfbNzzEENxHhvO3V+t9M0cET3THhho7iOPsN0e3tOl8NgS237TI4lXdBiksty6CCB7zqwQh4XFhYaaeZSB4BN0l/An6RnQ3iG55FnYUddesY40Oyz57G5rBLwgwwwHkm3wJ/PWVfFd7EdYBKDLc+FWW7GzgGH9Qr/k6QXHNbafD8LmSJoB3TzHswjc4meE9DgE1ZXV7O1tdXIgKS/JTXvBAbx2lOeyz0Ndp1dxp5g4xhbbB04zGDemOL+vjvkDX1n7HhXyCBwhMG2A2VsA+/vtZnOkibd0QR+nkvMnDEk2HKw5vvwHsyjx5AAyNkL+2/Li8ece7HGiPtU4ph3cuaE+9UsszMltrl+LvPotTXgG9s2ghsyhCZ7HtM+KKPhF2Aia7SP0pjtvrm56S2AZdEtA+QdQdiZyvXzSVrK2mUjOCp+AHowOGxviWIAFimBAoBgRHBcpLS4Fw731atX+fnPf56dnZ08e/asRduAme3t7cayTKfTxqrgkDFMZszMUHAfMheTyaTHrjPBzgoBChBER7cIJaVeMD4wG3bEVl7Gmv4y5gZPNUI3wMKQUSYDAKjpPKdZkzTDBygkcKFUjsXXzA0lajjp0WjUWwyLwUIebIAYHwCns2MYMObBKXoCAQMuB9mU9OHwnTHCyLILytHRUQ+Y0XeUGgPuVCmlafPayFSg2wsL3RoAZxSm02nbFc11oNgUAgnm08YQY0p63ouzXS4D48n11XkaTGBUnTL3/DirYABhgwzLjX7yGYE89iLp1qCxgwvgleeiVy59uLq6arqSpPUPOwJbOxgMeo7LOs39vJOJgX5lbG1DXIYyHA7b9sxmKJ2hYM4cXHJNzXpgn8l00h9sP3IAk0+/KL9wVsnArO5zT1+TNILDc48eO2NmJtMLNCFHLNPeiANbQP8dcJjMMxA0QDFgNQB28AIgYfyROWRqHpvLTZATr4EwMMfvXF5e5vz8PMfHx0nSMnmMLZUOYBpvmuLA0xUUXAsGSNJ8Dr8n3S6XBo1m8pE5B4xcjxxXltrg1jbMwbIzE8g+fnF9fb19H1mhT9wTO5F02Ujsk1l1iF70w7qH7PEc5g8dIhB0iSR+lnk1MLdN4h0hrvHnlEejZ7y7g67xeNzO5GIumSv8Od/lGSZOky54tF6iuw5+nZlA9pwVYwxNFmGvHIiadKz6T78qHuFfStuYXzYOwh8zryY3kPfj4+NGuNoOGru8r33QgX1me3khgCAK4EiJyb29fdh+8uTkJK9fv87m5mb29vZaet6Ky0FWfilH1AYojrAJCPiM2vajo6OeMaCshfp+WAS+Ox53+9IzmQD5o6Oj/PznP28ZERju7e3tfPrpp81pul52PB639RC8w+XlZctmIFAoJhN3cXHRQDf/eqGi02hmAABudZcLZyFggUgxO63G9WZxUHju6XTg2tpa9vb22jkSABjWn3BvDDELRD13zsKgsCz0YzHtxcVFKwvBUDNm3pbYC9sxzGRMvMsLAYiZElgcZ9SSbuGog1lKr6zwZFkWFxdzfn7e9qOmhtuHtAGaMMyMsRkHDICdmOd+HhvA2iAbEA1LhHxShonOoge7u7uNkaoLoAFtScecO1OIE6BszTtK8T3meXFxsa19Qs8AvM4+YCOSLjhmbnkfs+fYBX/Gvcz+YWMIBK6urt4IZryVNc/D/uDAWE+G7SDTCRBl3F2zi4P0bjH8y3f4P7rs9zUh4vfFoVaA4qyNnbnZdfxGugAA3UtJREFUM4J9ghhnUwFRw+GwbW3N/GOvIHyYm+Xl5VbGSRkdMkTGESJtaWmplZExJvgTZ88AgNwTUOGMBc7bwBF5Ze4ZSwAC93W2yCDSwQnvbTtOIOoMSV2XMi8NWWTMk27zAH53IGrgjr/FJrPJDGARnWJjDwJRiETkI0krsWa7/vv7+7ad7sXFRQtqkF/PB/rMM00+UCLqwB1QW8GzsQNzT0kUY8KzTezxffoDAWAWHF1zNoJSMvqTdBiE9zN5aFKMv4HFsEP2aQQZ9Pfo6OiNTDJ9c1YagpO/MafYDI8/Acrm5mZPD01GMZf4AfqOz/BCd8r1TIybSHfggY3ArtB3ZyuWlpYaPmFO7+/vG87lHW2H6QeyQpZuaWmpEe7YDD5nnSp6wVb+9sX+F1sNdvmQrfYfHWggdAhr0t8NhQVQZl9q7RffPz8/bwOIU0Ohx+Nx+zuTjhDzkk55VWeLUVlZWWnbqSZppQSXl5c5OTnpTTLfRQjNoKJMODJKBG5ubnJ8fNz6/uzZs2xsbLRrYMIxajb+RIMIpoEDwIs1Dtvb2z1lt3HlWQQBGJAKoBkns/ZmIhkD5gPjxVgyHpRfeWcx3hPjydhTGuX7k/UwIMHAYPiQGYIa3oU6c+aEeTdbB5D0oj3YDSuJAy+MGqwFc0tfCGx4V5eg1AwHZWqwlYAj2B+PrzMV/H9zc7PdEwfA2BCEADrmtRFAE7AB3uzUzXIn6dXbLiwstBLM6oAInFkfg/wYeMCI0wfmEfYOPbUBRm+YJ7PnZuuxFchikgbw+Jt1ywuaYasoM6WPBDoE7uiMv89GEvUZ3hXL2WaTIGQ3rT92uuw6Yl0xqWPnzTNcKuSsDnaEufC7JP3zIpy9dfbEzD/95zvYVZ7l+U36GXD6i0598sknvRO5CQJ5J4gKs4cEYvwfW1mzSuwoeHp6mtevX7c6c/pmnwYZQt+wSQYjNdhjPGopGIACYIBsO6ibx3Z+ft7KoBhzdqGDZEPH7AMB4Aba4A0AGCQlu+zYtpyenmZra6sXLECmVVac8XcWw4e3URLMJjc0Azd8X5ImwyZbqXogo+ZyF+QRubH+YtuwW7YX2M7Ly8vmOxknrkXO8I3IMu+GzcJ+Jmm2iE006Kuzsg5KzMQ7eIGEMlagYRvop3cd9MYwXIN+OUNiApPrsSX4GuZ5Mpm0kiPIHfrE+jiwBHbWfh9ZqZlS3oX+gAewBbbnZN4IvMBjrJ2xP6OvzCn3vrm5ycHBQW8XQXbbxC6BMR0D0I/HtA9eDM5EIFT8384Gx26wihDwg7CQ6iKCo1QCYIoTMZg1UDYrijCS0nTpgCN+wD/Mqhk0GG0cvYEwSsEkIZxXV1f56U9/2gyOgx2+b8PD2oTd3d0mpHYCi4uLbdGwWTWXLPG+ZhMQfpceWLkqUALkAD5sjCpAwHEmaYvQzUQsLi7myZMn7fPhcNi2Y6VPjI2zKzUYQhldrkUfbXiYY+QJR2vww2ewE5YPM63OpmBEMYKj0ajJL0yBv4cccYYCslLZaxhqgCeyyVyxFmBxcbFtr+sA27uM7e7uPlZtv3WNABKWhkwEi/ox6tgTxolyM+SZtTrIiYFf0skJGS9kK+kyFt6YwASISyIw6BAMyKpZLXQPGbLxx9kQDDhDyxhwv/F43LJ3OKYk2d3dzebmZp4+fdp2rsOGsd2x124RxAFonKEZDoc5PT1tZ4cwNknn3JzJxBajRyYIeD8DhKpfDj7MJjK2SbfTUgUd3J9noscG2XzfwBmn6NIm+skYYf/8Heay2iau8Xd5NwcYZI/IhrN2zweOYp8YK57H+HP/pCtXNiFxdnbWgkACCOTHawIIYPwsByHo4jy2yWTStrVG55wdciCN/hmTJH3GmQADO80cMc5kLNBh5nE0GrXqCG9WQpXAcDjsbW4CBkGX/D7otOWJ/gDKkX1A6v39fSu3RA5YnOtzz7CTDkyxafZV4AGCcWMGg9vBYNDGvwYJy8vLLejjvchaODuDD/BaR+OQpCuZxD4YKyRpVRT4AGwKtoK/kY1xgOlxd7DifrF1OOsSwC7gVezTyspKdnZ2en20XSHjgc3GL9FnBzDosEu/kG+THWQqbDtNgqIDSZfZR+6wSRzMxzsiN5Smg0Esp7zj8vJydnZ2fv2BRg0yzFzxEgBAJsRCgmGnk1ZGhBVH7EPXEHTAFtciJE4pe4KJKgESdmQw7KQ4nXbl3XhPwL4ZVQcfKKudAoI9nXbbogIK6DfGwULm8SMYQ+icYue5BBnOxpAKBtQ6MIGh5PkIOcAHQ+LgBuFi207uA7AhyjaoxiihUFZ+B3RON1LugHI6kEvSW8THPFkWmB8UC0PD+HIYG89HBrxmgL7QB7PGyC0yiQOBpUH+mBenNitz63ljbmA+0YPBYJCtra08f/68rQeyEZ7nNh4/lFt+8cUXjSFkrg2cAMqw9ugGeo4MIafUYaPDy8vL2d7e7q0Jw27hoNfX13tyQlDLHBqcMFcwxYAS1nRh88i+YQe9JsEspYN42EGyFuPxuG1igNxRJ4xTwSnNCpZhGatTxa6il5ZJ+mWAYeBFcxDNeDr7w3OYH8bKRBNsILasMprOPjqwpz9ey+B1UAAx/m+Cy3MIccT8np6eNvsDuOSMppqR4hrmjKzcq1evcnR01HyNwSpAkn5jN7GhJoTMsGJrvA1uDfwYZ4gs/BrzzPwxvsyt53Se2pMnT3olumZn7T+TLovF754PxsW+2sEezDtkUp0rMteUd0IGAdh9nkqSxrBbHrEBZC1dZgzpyrNMwPr9KjHIe5hIMQnDOkDuYz3HPuH/rIfoh9+fwCjpMjBe8M74E7ijA+jGxcVFbwcu7BJ2xUGG8YnBNHoBmQgjb9+LrcNH85lLvvgXvQN8W19dRru3t9fWE4NZub/P0YG8ZPwrNsQWDQbdrlq8s9fLkrGwjBOA4gec2Xb2lo0Nzs/P22nhDhKwCTc3Nzk5OWkYD6IVImU6nbZNejY3N5u/fl/7oDUaCLYjRJSypikZYLNUGNmtra1sbm7m7u6u7aiEUcTAcj8GiwyFWUIcO+DMDA9OyKDa9WywRABLGE4UMOnWbVAr7W0wncXBwCQPh6qdn5/3dpJhnBgXDBFg1ekyxsvjijFMOiYGw1EdE+PGD/dO0oTZ7A9A+ebmprcA3elRxtDBS3VaZk24BgOAUrlemUDTrFHyYIwXFhby5MmTrKystJIS3gfjTrmJ3x3nDsAzuweoRLGn02mvBOzk5KRF+A6MYS58ajRbEvO7S1qqwluOWZCP3FI7CaODvuzv7zegNhg8bFnKibMAiHltlCmwtbTL35CX8fjhxO7xeNycPQsYKStyHS8G8MmTJ70yo1pnSzlb0u3shEHl3j7pFFviAJI5th10dhR55xqMOWwTOoKtNOjFmQNA2c4YW0rGjL5bftzHaiuwoUla+YazZXzuBZ0mBLCTZpDtpNxnZ5vQt6Q79BDbjQybJZtOu5PE6X/SOWZ0FYbR9oT3MEkC4eH3NPnBD+PmDAxBK893+QW2G72G+YPxZj6wm9jdxcXFtvuiySMvOnWw7fcBDBJYYTcBlSatuBf9w18xB7ZF89j29vaaPrEOwkQm+ohPsr1Eb5lv7uNyuiS9dX2WMxheWPrp9GFLUr7DvyxyhtRIOjmmrwBjZMHrIJFT9C/pfCPsOhl71gVWshdZZVyS7vwQsiEQNOg7Ms47TiaTFmQRjBigU5HiAAh/aHLX42cC0X5vOp22cnfItor1zLDjx02MOIDDppBNctm2K3KMr8BMzorwXMrp0GMHXjzX40jfGGfsEngW+4jtNnmGvDhAsS1nXnimgzvkzZjaeBL7BEFCtpQsMGtNnc0B+y0uLjasaxz5vvZBJ4Nz8xpNk+axM3VQYGEA9DvLgKHnhOzLy8tsbGw0kOGUEc6YKBr2xjW19BPnQjRtgAzjh0FO0gs6iNYpS3CKi3vB3BGJ8g7T6UPdLyUupKNs2AiMzPw7jUcwxO9mvwEpCwsP9eosXiatC6j32o3pdNrbDcv3sCAi9AgwQQCROwfkJN0hUDhaxtxb5vJOi4vdDhOWC+bIQQOMpGtaK0M0HA7bgjTKElAcvoOMeEF1ksZWesvkhYWFXqCJ8cdg3d/ft5MzCRY4kwS2CQYGmWI+GJca+GDAkDXABOuJALEOmnCA89pYDDuZTPKzn/0sX331VW+3MZghMg1Jd3gfY864YHuwFYzR7u5u1tfXe2wntgaZw1DjSJzF9I5LXnC4ubmZpaWlXnkMtml1dTXHx8c98GYHj4wyBtgAM+TYJpcKMjauh0XOvGaK/iZ9++XyAxySHZuZPXQdQOCsIaVn2EQHVdhyrkG/XTfO/Q3ScZwGBIAsbAZz77lgPpzNRscd5CEP9IXPYY2d0TC7bx/GWPJ8rrftxh5tbm5me3u7twYIefAW3c7OEsS5tNPElAEQWU/PLTaZ+mo+Yw7wUwA+FkH7QMt5a5S0sIYRYgx9NDl5eHiYr776qvd3ZMD+1TID8Qbw8mn06LSDdQAkJdCLiw+b2pgEZO0YNp/5SNIwj8vrnAVE7ixLYBICS++ehl7zfdsldBv/mnQ7lSXduk/wiUkFg/Qa4Bs70fCbfDfpyiTpG+8LoYhuErg74+mMQA0endVxhQtY0bafe9AfSl2xp/Sd7K/Ln9B9B38EIRcXF81eg2ld8eGqBshr5h+9xj4yRy7/Iojj3bG3jCHybZvNu2JzGXvez2WU9J3rCEBZa8xYYuvBhI9pjw40eFEmFYfsbRHpIAyMS3ccBFxeXubJkyf55JNPMp1Oc3JykoODg5aivrt7WFSbPDjVvb29nqPBITJQlAqY1UIJUHwGBMECnJsFZAAxQjhwjA/KipFAOKmTdlSNgYJx9DZu3IcMCQvXh8NhO1gQIXEGYjqdNqBPlI7QmHUxE5j066yJZBEcIvTxeNx2tQHwMTZ2UPSdAINIGCPkEqykq2X3mMH4eHG1lZnADaaIeRsMBu1sEbIJ4/H4DUbHAZuzE2SaHAjc3DwcvPjixYscHh62PiITyDqpWZfAwMxyXoYNsoNoB4jMGcGUA3CnUs1Ycg+CDLPJ89YoAUu6eljm//r6Oqenp9nY2MjOzk6bB+wCem0HyXbSGFsWitux4Yg3NjZawIpTwom5pAmgQdYQ9mltba0FGdzf/cfBA5BxiHzO35DVpFtbQrCCPrl0CXlz6aeDCZdnYQ8ADNghO0zkiL7bOaIvzmSiLwAnyA1qpA2ManYTUAczht6gU4yNy9ewR8wRc40eME8ODrgeG4BdMSBh3OgXzXbLhAEAnfvbHmxtbbU58w/6iU2hXBbiwu/prAp+FYBsQEt5FqUcjJt3AeQezKlBjf0SoMjs77w1AmUYfoAiO9KxJgomlmwgxE/SBaq12oCxpCzL5YZJmt0mWEBv+D7/evMSZ6zw4y5/BPy7aoSsYNJVIuBvmEN8BDpKX5h/3oH5drbVPgp/i2wmad+tZTMEPdwDvfDBhoynfS94zMFF0h3giv6RRTRRSdBOIOCAgvd3JtBBB/3B5tAvMlXMNSCf/lT/a98OFoDEYk4JEGiQCwB0yymBHniOnUnxT9hi8BK4ETtIZpRtiqkMso1KumMhLIdUhWB78SOQ5q7IYLcqghjmJEmb88e0RwcanmBHRTBzOEzXMSKwAHcc+9nZWb755pvWURSDXT8QSn5nEejm5mabVBQXRhMlubm56QkkCmGn5No1BMzCi3C65MupVSZyNBple3s7k8nDNmwIDUrJxDu9ZBYLg8NiWH5n21+cBwDV4DTp9tIG8LsMAkBtlp20uXc8QQBrBA17yXuYUa0GzGm/pEtP+4R1ZzqcjWHdB0CzsjLelhbniWEjeHMqnPFAMXmmjXbNtPkAPY8BIJNnu7wC4MC40Qc7MxwAys07Uffr1LgXf5HZo0/IBc8xeJq3dn193cZuYWGhsXyAe2cwk47t81wwrrBFZJuQHwwlgTFjhoHHcaDHpLBrCt1bJZvNMTDAAPM7ThgbiSzwXQNMbAzgAbuE3XCpEP3FmeJQbLPou/vrLDMOtpaUGDAk3UJ6g5gkLZsIi+7AysDX5AO13pPJpGVU6ZOzyjyvZkmYi6RPdvEMA3Lstks0aoZjMHhY+0T6HxsGgGKzBZfbcj98Emt/2B7SGceTk5McHh42RttkFX4O+w+T66wp15otZteqyWTSzoVCZpIO/Lq/ztQxdrZDyMc8NvQG4EpAYZuM3g8Gg+zv72d7ezv39/c5PDxsWMVBpwNLB/hJWsCOzKKz4Ap02XJLwIgOJh3zblKPLUuTNJnDh4AbuBYZYWt6fC618jzTfj1JT6dceufvmO13IIEtQXawEVzL4m9AsvvMuNh2m4BgnHiGgxBkF11wIE4AY9DPe4APuCfYifFnXGpWiex3kp6+M/f0xf302pWlpaXmRwhOKtkE0cO5ZtgvyDP+DradTB42PXBlhckcSK8qo+gAwQdjCBZyYIpNwf45Kw0JDTFMUOLA0WTwO3X2scpNpxhIBNgT50gaMGmFTNIicwaQ6xECOwkLASUqbNHl2snl5eVW23d72x0Wh+AgGBYSgxe+h/Db0RJRYpwNfBloFGA4HGZ7e7uBA77j9zc7imAwligHfyMaRfiYdHa7cF9RfObEabSkS7MhkAArZ4M8PsxPdfowzC6fckkQ31lfX2/na5DadraFMTo7O+ul6wALfMZCWQAEyu91MMw3DghnfXt727aVBDRhdHwtAMpM0+LiYu8dYWaQWQcWSXoOgu9gaKj739raaky9A0PX6GK8MOAuGRyPxy24ndf28uXLJF3tv0tXAGZPnjxpa7gGg0F7X5cfYejJBj59+rSx7Tjvw8PDvHz5sp0KzxgaxLGbDBkDwGl1Wkn/cDgHq9UBocfIukuW+BtMqeuB0XeXBtBX36umyVlr4qytbZnZMPrgDIizz86YEGg5wOK96ZvZPINmxswO3IDMJQUEJwTULnVy8GcgVLNI1O3TXDrh7BbPdKkt40mWnHdLOmKC7zNOZ2dnub6+bvaEMXbwZKaYRoDgk80hGciYea0M9g775fcxiWcm0mVV3lbUIM72Z96aSxvJBiITlFai55SkmAAAFCK3NAg0QKs3KzA5QQBTddJjm6Tpi5lzMBJzCDEJQOf+JhgchOOLkP1KdIEvTBC4b+iTg18CGft7+s1nSbdLGnYu6QgZZztMYhBMYFvAd9hW9J6shO/LOOETnfFB73k/5IHn3d/fN3yH7UW3sTnYREgtkzkmVXkefhksTNYKvWNXSK/JcWBJxYYxLn93Btc2A912oAHxCbF+f3/fK5+nMoB7QtZD9Gxvb/d2LgOHMl7gLpcZnp6e5vT0NAcHB71sn/XnXe2D12i4fMmRKZOOY8RxO13IQDGZZDRgYRAulImJxigD7K6urtriXgIOlBkjzT7GBhUuMeA53iUCMG/g43SbWUKYBTs/nAtZCUAoEwYQ5bs4XiujMyIGGTzTmQrGm4CupgRpOFRvLcq7Li4+7PhEYIShwzgzdwRnMDooG+8NO+MSKebQSpCk9Z9Ajq1NvcuLFzeZvb26usrr16/zi1/8oh2a53Shd9bg5E/uk3SHkTkD5gV4Vuw6r35XxhVjT5BgMMU4r66uZn9/v2VvSFNi5DFAyCYMgoHpxsZGk3fGaR4bB+VRx4x8MycvX77MN998k62trXYdsjaZTLKxsdH2aee7nmMM+WAwyM7OTra3t5v9QCZcDzwYDNrCX2cYcIzMPc7L5IlLUwCYZFiT/kJe7BuBI7JRCQaACnpo/Scwd5bUmRK2vqW/gOnFxcXmZE3a4MgBoAAzZ/Pcd+w6dgYnztkxkCDIPg6Md2FuTk5OGthjrLBXyIHPF3A5AgDQGai7u4dSWwcJ6LPtkIkQSilMHLhGmR9ABvdzySTNwaP9gQMZ5AQAZN2GuawsZdI/PJUg0DaPrAjPZVywK87g0Qdnd+exLS4utgwmATvZTOME9I9Syvv7+7b1qn2m2XqPM2OGXUi6NQx3d3ctW29s4w0BwCNkHAC8+BYITM8p84MeMMfIpwE3MkjAu7Oz03QDuXLWBztKMMo9IGyTLoiHdMNWQVQm/bUa3JP3xy7VAMmVAlUGwQw1oDG4xv6aWOX96ad3GCXgwX4y5tyLRj95L2dNsbPGBB5TMBTYyIvg8SPGiQQYJnxMyphAwf/QD4gG/m9bQT/B0s5cTqcPZZzMOX0wBsO2c2/65GCH9/XcMdeP0tnHXohxNZPA4JstwLB6QnxAiNkXp32STqHZTWU8HjfhoaTAQQwAjxO+7WAAhHaUNeXFZCGAXvcBw4gCkg1AcNlbG4FfW1vLp59+2thXnss7393d5fT0NEdHR73JRBls2LwbjpmtOsYIOgwGQuM6ytvb27bYyIwGxgdwA6OGICZdLTDb5KHsKDegD0CCINdTzM1cO1peW1vLaDRqwdF4/LDb0MnJSc+xs5MXJVE4EspLXEdaa02Xl5ebouOECIpw/C4jYE4c3BiAOABjzJ3eNcuFkaS0AcYXWWU/a2T/bbqwvb2dp0+fZm9vr4GkeW2wP4wnc4D8wJBRLkng5rUtLOKHbKCkkkzp8vJykw8ABQGty3RwfMfHx41JMgjBfqFXW1tbPSCapNWD10wGMuh7wS6hCwQ9BoEwTHbgOATKZrBrzhaY/TdjiXwiX8iO69IBV2SCkEEcG4Eb9v/o6KjpcA3Akv7Bewbf3M9ZG7Ouzq66TNGOlMDSQISxwO4n3a6D6C417zwT2UK+cJoGhDCtyAhjh//j+grkAHNJ/6Bb+oVdY1xhdylt4Dtmzg127Vv4m2Ua/8R88W78DRmiT/PW/uiP/ijn5+f55ptv8urVq14wBcPK2RYu74WRxrcjsw4okm4raMueWe1aboMuJx1wxa9DDjibgo5zP55JEJ/0T9S2vDoLBTkHuTEYDBpBx73pr+d6YeGh7BvdXVrqttZ1ZsFZRj4DpLKQ2cSa/RJjCxDG7hqfGbhjcxwIWW+Qf/rgyg1XRNCnmpEcDAa9DWdqxsV9c/kZssDza9kcuovN4OA7V2V4POgLvgEbSpBCv8FwJuHxR64owSZiL11J48wJfZ5Op22N4u7ubpaWltpuljybeWEsnUXhPuAry+m72qMDje985zt59epVG1CYVxtCR31+OYTfTtFRHN8nnUN9GREzL+wghe+zsIVIHqF23aoZLcAi/YdNgu08OztrO0aRFXHfYdSoY+O+a2tr+d73vpetra12H1KZ9IffWUzqhTr0EaeHULs+0M7LZRy8hxdyOSvjNCkCU7NIBHJOs+LQWaOBUjsSx4EbZDF/AGXX09opIC84cbYlZY0JNfbsqoLhQ9kxVmYhan0l37diOyXolCuNe2CkySb4OwahNtDMT82a3N3dtYzG6elpK/syQPXYU15HzTgLcO2c5rEh00nnlM3aIUNmkgGgsDpk1dBFdAvmnoyPATPzaYaZcV5aWsr29navtMQLSM1YGwB6QwN2/8DZe/Guy+jMSHIf3tNZDoMebBXBeJKWUQAMM14LCwvZ3t5uTp7MgANsSBP6CChHp7AryBrvz9/pL1vskm3hXQwQsNNJWmaJ90MXmHcAOM9z6YP12xlTs/XojG2eQXnSrclgfHk2B27adzBe6LXZXubUa+lg1wmusBGATTOJzmST3TdjC8C1k8feMo7YJ8smJ04DrLzzmrNX2LB5bF9++WUuLi4aAMa/HB8fN//z6tWrRlBQZuIMIvbCttRrPGzjTWSZ0UX27CuZMwNfgvcKapFlrjGJiH0hIMQG4D+5P3PKPPOe7ktl45M0uUq6YAG/aFyALrMDEeUyDvy5B383xkLXeU/+TvBrG2TW34QL9+d7njMqAZAD7LcJIpPD2CiyzybPyXzVrDU2dmlpqVeCyvewu7e3tzk/P299Z5y9cyH9gSz38Qfsbsb48jzK5xkbbCjjwvuBEfFD6DoYmUCb+Tk+Pm7Y0Xbav9OcGbM8zbp2Vnt0oPFv/+2/zfe///384Ac/yMuXL5sTpT4SAbYwGHCZuXFam0lACWoqxyUC/I6AMylea4GSIujOqmxvb2dzczPX19c5OjrqKRisGALjg5cAE0lXd1eDgMvLy/zlX/5lA+w4SnaWQqB5NsEGTop3BJibNeU9HOQAhGDyGA+DNObAIII+o8xE47S7u7u2XTBGwiVUrE/BQbKtotOo7AZC9oLvOqjkvjh5dl1g3gj8bGA874BEwBepS8YDo+WgFKWzTDqItBGkn9vb2/nOd76T5eXlN3ZHY34cTGKA0AXkkpS6WWeMkBm3jY2NfPLJJ/nkk09a6Q/byJEFmedAwzXFDqpmMbgORpFnl9MMh8NWfpV065Ls0Mk0Pn/+vFeeZrCJQ0Ge0K0KGs1AA26QF7aANngG0NvpEYjgHHkvByuAYf5OpsZBCtszu+QAm3F6etqywmyxje44eOAUcsYC++oUP3KJ/TVDxknulHPV7Ct2kueaTbRN8RjAlPJ/NsGAMCCodzkj2cUkLXPrTHHSZSGsf/zNcmUA5PJWwLuzIsyBN9fgXcw4mkxgnJyBdZbZQJCx5MR6nu93M7mDL3EJif0OAQryOq/tJz/5SSMffDDkaDTK3d1dvvnmmxwcHDQfBqDEXtOwE4y9WXRkl3nD/6+srPTO5nFwjB9B1vF53irdusVz+B3fjD92dtKBPrZlOOxvqessATYq6Z8wj6xZJ50pS7osgbEMOsvn9stJl9nzO/n5Jj55NoEwNtbf5f2dkWE8nRnAJtFv7KNLuJMHHafsGNxo0itJw17IhXeqcl+QHXwNWQaTZFTKMF61GbCzpbs3QKGEnu8iZw4WyIJiD5BfCElsFe9vsoH5AL/xTpYXfuf7SXe+CST8Y7HIowONf/SP/lF+7/d+L3/4h3+Y73//+/nxj3+cFy9e9JhCCzQC7+gbAUEAXKu6sLDQM+ZE/El6QuUUPYMzmUyas/NzuBdA9Pz8vCmzr03SFpknndLgfCeTbp9ngMvnn3+eyWTSDlK7urrKyclJS0HhIOk3TuT6ujva3SUhKP35+XlbDM49nO0gEvVBdjyDwC5JcyYYQ4Kxi4uLtnaB8eSdGU/SsU7jWdARbP7GfHMf7rWwsNAEmbF2vbWVlQCHsgrPe5KmYJTJoWxnZ2c5PDzs1eY6S4EMmkWE5TXjgxzwO8zg0tJSXr9+nePj45ycnLR5IGhO+icpw0pwL/pO9oo5ZzzN3G5sbOTzzz/Pl19+mb29vV45HKltTmif1+bA2I6KoB4bANhO+tuYGhDihJOu3BDDC8CALSbovbq6yqtXrzKdTtupp4A2n+5qsOEyCqelj4+Pe+y6N1VgK10CTGfAcO6VFPBzeBZsqMEDqXYWauNgHQQA2jc3N5sccU/kEn1x1gJG05kCxoPfTbyQZcSmMt5JWhbGQDpJz5lzT8C8MyaMBTqDI72/v2+23Ayqs7SAfmdAGCezr9h3k0oOjvADvAPA0cwvc0Ig5MzBLHYXMguyiXcgGHBG25k95silHdyPd/UCc+SfeXaW1mUd89a++eabVgYJkeeytaurq7buEMBYCSXWM+EnyTxhY+3vXNZj/0h5t9fIIGP2h4y59bjqFHPq0pwkPRnHj+Bzk84e1pIk8IOf40DAeIrnOFDG99If3o+KhrrWALtjQOsxcSDHd9BXCExsP/6Zd+F5xpbYhKQj+ngfbIl3vFxZWWmlr+gitnhhYaHhKfoBCHc2i76ij5AEtsvYFxMCyAJj6YwaGw6AycBBBBvestZ2ymQNY2UyFzmGqOJcKkgQZJbxZZ6we8hLxd4seOddZgVRs9qjA42XL19mY2Mj/+Af/IN873vfy/Hxcf7P//k/+Yu/+Iv8/Oc/b6nfqiBOJVppnVqqkTc/bkyu7wmgdM0uRtcLj+2MzLYnXY2y0+401z56fcjLly9zfHyc8/Pz3joT1hbQPwAMfa9lRxiONhmL/f31zbxb6WHmbbAYd55P1oN+812EnyAERcKpoQR8jrCyduLg4KC3na4dJRkODDvz5n7zDJTd55kg4E6Bso6EeQQMLCw8nOb9+vXrnJyctOtRQM+jAT8/zvTQTwApbPXJyUkbX8afd3bq3alL758+y8DwHH7HWe7v7+d3f/d389lnnzVWi3VRHjPYmHlt6IQdEbtr2aEy965lR6eQfc+BZdDzg06xWwayfX5+3hamAzZOTk56205SpuJABybbzgkD79InWC3v/uEMATLh8j9YLFguO3g7GEoMnfWATDCT6e15Cc4NEnivyWTSSh299TAkkddRJN3uS+zcxzotZ1kJ9GDqsA2ur2ae0WnWaiXd7nZefM56K8A417rMxVlE2zP00zLDfOB37u/vWybbWUf6CvHDM7m/F+ljp1hbhj22nXVfaA5qDcawaQavDtrYMQ1G3EES80fGHv2bTCY5OTlpIHPe2qtXr1opz83NTQP9bHM7Go3yO7/zO1lYWMjx8XG++uqrxmija5AMSbcBQQWhyKb9F5lPB4cnJycNWHI/5stlTfTXQSq2MOkfbpt0GVoH/lQlQFrSPwfCfk8Hk+gPrLRJFOyiwaoJEvtN5L6+q7EIsocdqWQNWG1ra6tHAphUwO+zlT54zFjAMo7P4N6My9LSUit/hhDG7+JXFxYW2jMIWnlOzRqzLs/ZDWyY14HO2q6Y+7hU01lJ/KI3QHJ1D/0xrnXghvx6faC32KZfW1tbTTbBoCZlsXXIE1t5sy4SX8K5J+9rjw40fvrTn+bTTz9tCzEXFxfzJ3/yJ/niiy/y4sWL/PVf/3X+6q/+Kt98802LunFupIQchbs2DYE3w+1oP+nYN5SN0gAWfKFUV1dXOTw8bBMMWEMRnHVBUZwWJbBwapD+IpDUPrscBqXECNhJulyJcTCTa8BuwA8wMgOK4eAa7kM5BgbF5RkIN6DAhtTsA4Jkw0VJEoucKKvCwDgTUUGey03MRiDUsEZWGPqLIeEwPN4TpuHu7i6Hh4dt61Knn5Eh5oYxdqSOjOHIcSoYA1hIG2xn7RhbvmuDy3tjwAFHZoi3trby/PnzfO9738uXX36ZJ0+eNMYdwDudTpuxvbt72EzApULz2CiTsnOysRsMut01kOcq65Yh/5/AmED87u6uycJXX32V4+Pj7O7u9s69QM+QbWdWNjY2srKy0oA9/XGgY2YOcIc98KFbBhPIuJ2H2Xg2pcCGJB3bROANQPGOe3a0OC2eSzBnm4ieO8vJeNrOONDye7h0cnNzM4uLi22RPjYdZ4UvwC8QqMBeOghPuoAOW0qW2OPufjFGZmDNyPEuli/GwjXPJmZ8bhF9d+bC3yE7gb1zXT7vzPVmAckCMQYGm8gW425210EOdgowU7Mc9NXj6SBq3poD8KTbYCLpiLZnz569EZAZoLEIFz9kfwXQ4m9Jd9CvMxrMw2g06p1twXfIxrHT1e3tbcvaQQI6O44+I3ME6tgy958d27ClVEoAJF36jF8kW8taAvTcRAUyxf+RTZfr4OMNou3/jK08R/Tfpe62f4uLi70t4MEA4I2tra3WR+wffYOA8xpNxpDgDLnf2NjI7e3DMQj03UGnS8qQCYgPE5E0ZIPDaPE/EII+z8IbeFASi0/Ar0yn0+b/Xdlj/GLyErtmXWeMWDLA/CAfjAmyhV5hP4bD7twp7sV8E1xArj2mPTrQ+P73v5+/+qu/ytOnT/Ps2bPs7e1lf38/3/3ud3N7e5s//dM/zVdffZUf/vCH+eEPf5hvvvmmTfx4PO4tZEQ5nHYHJDgt7GyAwSsCgiJub2+3hZn39/ettAjlOzo6yvHxcZJuz3JnT8zsoACkXJk8QIAjYDOuGPzb2+6oepyGmTOUins6KqYvOCYzVKTrcWgEBY7669oBvk+7u7tri5ABGwgWhsTgjTQ022FaQM3yE5BY+QwEnHoj6ub/ds4ee6JyK7eZUIMSHIABKPNmdhzDx/fNWvIMM8T8n/epwSrjR595f8ABY8PcwAzs7e3l93//9/OHf/iHzQhQEkdAQikOY88+1o9V7G9rg3RgrCj3wKk4Q4mMEcwRLCJPOAjGCpm23Nu4M4aLi4tt7QsEAI5gbW2tOXBkA/aLzBkODucG0eEAB/mnH/S9MlPeAAPDbqbRwQvjwlhMJpOWCagMl8fOIAnWDVkG6KIz6BhjTt8Hg0HvjAuDCF/Ds6h3xvbxzg6MeCecl0mR0WjUNkFg7BzoYRt4JnaJ75uJ5pnoNv9iB71A3DqNE3cmypljZ7MAdJ5rHDvz60yqg0G+X+0L96olxAYXBM1cCzBkrNCBpDvzyHMxj81BnmX38vKypw/ILHNE2anthceXuXJWlfHGjoBnOLcFWffumMnDmB8fHzeyCJ9H5hbgzxyg0/ap9kvoU/JAGD558iS7u7utX1dXV03HTUqg4zwD9t+s/WAwaNjB+IXmNVH8jZ3+8LM+yNCZPYgTAq6dnZ0m6+AZFvETSECKcB8CPg74xX9DQCAT9A9ChXdizlzWTb+xr0m386ixQi0Xt1+C+CDAx/ZBDOLXtra2Gp4kE+ctb3k/iAxwIfbfZYGVcDCuNJZK0iu9QuYoDTXe8RpJcDY6NhwOc35+ntPT01byb1/8WNLz0YHGq1ev8uLFi/zoRz/KcDjMkydP8gd/8Af5J//kn+T58+f57ne/mz/6oz/KP/yH/zBff/11/t//+3/5n//zf+YHP/hBDg4OWmbDES1CguF3StsAwwbYBvT169e5vr5u0T07oKyvr2drayubm5sZDoc5OjrK119/3U5cxhgzYKS5r66ucnR01JQEBQL4ATBub29bDb9ZiPPz8yYMdn4WVLOJbhhOFGRpaamdL4FiMFYICkJEKpc96R3QwV4QlZvlIQhCwTASZt5QzvF43Hs/s54VGFYWF8EH2NNHB0UGhiiLI2oYo6QfxNRULmwy2QzXqmIoaGZgUH5AnsGGnQ5GjiDXDKENLUEyiry3t5dnz57lyy+/zP7+fnZ2dtriRU4KNTt1enraThi2gfS4zmNjzJBFnG/Sz/QxlsgODBHXECBQumgm2AF+khZckwEFMJycnLRg3Vk+s1ZmkC0TTvWj69gHnAz7l2P7ZoF0xsAsnx0m7L/f2bXjBiOMG/YA53Z/f9/bYpMACSdLMO+Dq5zJMxgDcNjGOEPNfJnhxZm6/AOA5dIossV812sb0D0THNhdwDt2gNIABy68J0wv5QnoaCVlyIgiW7UUyUAg6TYUMEPMM9mwBLAIwLGjhlG1fBh0GIQgDwYNjDngqmbQTHAxhs6szFO7v+8OQWUseMfB4GGd1suXL1tdftKxwOxAR5CGfmBz7+7uWi0/tp6xclDobD3btAMsmdO9vb0m34BS5tbZNZ7jAAG74s1k0BGe6awmOybxDtgrrgW/kMGDNHB/kZvV1dV2jtHZ2VkjRNDhJD27R0m6bRafM87MExiFhdYED1SCcJ11AB3HlprcpIQUe8x9CFyQe7IbgH1sObbXOomNdXmqcRM2xpgFe2W5xK4laTtIYsMA7GTQlpaW2rbuZNCdLQID1LUx+CIyNoz52tpaw6gupbNc256Z2OKeXoNGdn9hYSE7OztN/rA972uPDjQ8YYPBoIH8Fy9e5Hd/93fzx3/8x/n000/z5Zdf5g/+4A/yz/7ZP8uf//mf57/9t/+W//2//3d+9rOftTUMTBbp95qqN9tn9i9JDwQAJpkEUofeMx9l/cf/+B9nMBi07AYDfXp6mhcvXrRynPv77pRFM0M4HVKL1CFboUmn4mx5B5yWT9nF2QKmuYe3pOT7ZGsACN52kzIpL67HkUwmkxaN1n3dzYqaFfQ1SZeN8OneKBxjiIFy5sYAxdkBxsuLPjG2pEgJFjEeOE1+RwGRS97VDKRTgF7wWsGkgQq/Y3QcYDAuXiNiNsdpd2R8bW0te3t7ef78eT755JPs7e214Pfy8rLJIewsWRrW/5By5T1ranoeGw6PMhsYfa93MMGAUWYOk66W1IEsc2RwaHYehhHGBsBBsILTZYefpHNyDjCQNxoZWWyFnREy6ODHLL8Bn4MASAb0wBkcdIYA187Z6wDoA/+33aJMDPlmXrBT9MHZJxwfY8fnLtEaj8ftbB3mExCALYMdxfab5bcu3tzc5NWrV81ZU35CcDXLMdruDgaDBmoODw9zeHjYbFPNbBrcoXMGEfQJG8J422bZdvi9XTMOieCNOFwGVe2qQY+ZQ8bfQRfjyefOELo00cH8Y9nIb1sDR0C2XV5eNnDs4HowGOTrr79O0u0wBTmXdLsQei0jhCHBIf6C7zDOZqw///zzZtdWV1dbn5KOQHN20VkSE32Afrbotq+huTSLf9Ed3s1Bl3WT79jvY/sq0MYvk0EwriBDaFvD+JAd5lnOrIB1nL3hOmcpKDFzIA/w/eabbxrw9sGjkM3gJKpCuC/YhkwSARnPRF8YQ+se74geQnyenp62ceAdGE+vO3EGkf5wBgr2ADu/vr6eo6OjZt+QWeaIzU5s++wjVldXs7e3l729vQwGg0asgDmxOUkawWbsDH6hzI3rnj171sbXGcNfe0aDm+Ock4fUDJmCly9fZnd3N19++WX++I//OE+ePMmf//mf50//9E/z9ddf57//9/+e//Jf/kt+9rOf5ezsrDk4HFXSAUkcJsCNiIzIjzIsjM1k0u3zzCRfXFzk1atXvcEnQEGpuf/i4mKOj49zdHSU8Xic09PTmWAHp+Pshg09yo0h4V+cM/cxO8u6ECLipCu5Idi5vLxs+7DbKHqhOobNWQaCMZxhLQlyNDoej1vpGsoBQ+b6c+5l48DvjAUG3OUFzKvHBcHHqBAwbW5utm0EeW8COKcMa+aBZiV1YOEaRgNRZwgMGvid+zlwBCSZmQRY7O/v5/nz5/nss8+yubmZjY2NtrUyJ5tT0mfWBIVnL26ewfzY2c1rg1mqc4ceJ10g4T3B+bv1ku/yu+XbW9UiK1yDneAZfB/7ZsYHHUWva8aFAAgZ9XsAFGDPsU3oB/3lxww8Ac9kMmlsmPsPy4TsnZycNOeAXtkRIO9kdAjYeZekyxrBciH76GcNwE0+4UjZVMMkA2NtkoPPcFgGXNhpgwzPN4EZRE3NPDMeAFLvBAjBQakNNhcAwlzYntIPsl8EZPXATYNJxsmL2xkPl2pUEobmLAt2HzILdpz3QEcYP7OdBnrORDMO89gAcwRu/OsSM/Qg6aoEzs/Ps7m52fPlDt6ZB2xv3a0KH0LJFWA86UgJdgRk/tGfCnidIUHG7u/v27lJkIT39/ct4Kc/BDsmX5K0AJn1q5YdxsSZFQfpS0tLjem/v79vZ1u5jJM+++BZ9BiwbVyAPcIOkcVgjGHXsQdgFTaPQDf4+93dXdtgAdt7c3PTW3/ind9YpO9SLgguxov+e/4JPhlrE+xJF7Rio8Bg4EoHgthTyETGxBuNeGE4cmzyFTvC/Z2B5nwtApbRaJS9vb22thAZYsyw0fg6rwdF5pADk1T39/ftSAivG3xsVvTRgYbZAAbcLODp6WlWVlby6tWrfP311z2wtb29nT/7sz/L1tZW/vqv/zqvX79uO7IcHR31jLpfzk7JqTRAqw+t8gF6Zpuc7bAB5v4AXNYYYMBw5hgCBA7FwMmZXfPaBUC2GT+eh9G/ubnJ8fFx270FYLK1tdUmFMbV/XO06Z1q6CuGF6Xgb4AeIn6AP6DFjAMOmtIU+mxgwfswN3ayvCeGxdvNDYfDtrYEeTJzcHFxkePj48aOuITCAJX+cB1O1cEIjpWxq+/CONhgGugwnxjLuuiTeSd9/umnn+aTTz7J/v5+Pvnkk7Zv93g8ztHRUX784x/nJz/5Sc8xItcYSWeFnEVCP+Z5nQYlPKTRAcsuE2DOmQdAlksk2IUIkuH29rZt6Zl0tsRpcWcqMN5m+2FCnT01ICbjhPzTH+TLRAEyUgPcup1szVKhP2Q1B4Nu55pqe/kbAZEBkcEo+u7xSNJKj7ifF5hWMAKoZfwMiknPo0vYD4NbM7tcl/QZVsaHvhGUUgrGNfSfZ9mm8ty6Zg0ddkkbesV9AA0OBhgfr2HDliJj2CADOgBOtTGAVGflmK+NjY12YjPzz9+wXQbDzlglHRA1KZH0QS/BC8zvPDaCZcabQMrzht1kbp2tM4GJ/iCPSUde+JwE/D/ym3TlMbYJr1+/zng8braoZjq9loj5dUY76Q625Hr6A3MNOeL1ptPptG3Rj75tb2+/kblMukW8+BLe32VNPo/Ka7roEzKP3iGLDm4dpGNjGScTHZALPuSU+XCgwXtSZsx9J5OHbPXBwUHrJyTS0tJStra2WnlX8rAhCeseAdTYdsvL4uLiG5l0/AhBG9fyrryjzz2alWHBxqKH2ClwzMbGRi/TxLwx3xA7YGCyT9vb261sjQybCQXmEyKeMXCwi81HPs/PzxtZgx6AUx9Lej460KhlDUm3YBpDNh6P20E5X3/9dX784x+3RZej0SgbGxv5gz/4g/ze7/1eDg4OWlDy8uXLXsmRHRy/w6wD6gDbBn2UYgAK6ZMBuLMKZhERLAcGZrQRiiRtCz3XJ1Ja4ODIddNJV5KE0TFDj4JgxOzcAaUOtsxyAVS4D87HDp7gDMXwQjSMjFN7t7e3TdlRjFmOqaZBeW+ud5DFWMDwmrXGaRAkedtfgwVSpXYsToVjOFEYDImbARnjb0cMgKPfKKAZKOQPMPT8+fP8vb/39/LZZ5/1sjrn5+c5PDzM69ev21a8gGieb+BmHXAAaYM9zxkNAlccLIuzV1ZW2gF0sDUAaAw9WxsiV86OTCaTtqaJ+YOZh6HjID0cNJlRmDufoeE1UQaI9AndrMG1A1p0yWsDsF84GJw8QT62C+YKp0ntL+QH/bac2uHVUhvvouPsA2VKlA/YyaJ36Be7tfgwOgICbKAzheiM2U50nWbbQbC3sLDQFo2iZ4y9WUWDQDKgrMNjXAFFPBv7wLNxpmQznHVAT8mOIQvsYGQAb/aa+fFYkBVm3NABfydJbzdG5tNyBYBO+nbRYAw/4sCLbYZ5NgzvPDbKXipTjd33tuDejQpZQA+T/vkREGJkIMy+k41grK13nmOICphqQKYxCDpEwIBcuiTcATAyeX5+3mSB+aRsiGsBghCUlGHSP/wJxxEwRugpz4cgMEZiO12XlldCwlldZ3axOXzPGXuTwmSesIfYFDCDy8qRb7LWPI+5Ad9QNglJyL2xQ+gKgYl12PPr6hr0rgaS1k/6wr/YPsgHZ8xNIFgGxuNxy8xT/cD7YZdMOlDtknRBmIMEqkRYKH53d9cqajxfZISS/nED3Nu7Tz2mffAaDQbfdWmO7DD0vOjq6mo+++yzfP75583BLi0tZW9vL+vr69nZ2cn+/n4ODg5yfHzc2CacDE4fth/n5jpV0pmAUAcIduYoo5lhDBECBJsAEHGGAwWBLXMqlYl2bbOdmx0BY+XaPW+lyP0xBhiOpH/4DcYAsMW74UDsgJg7DNvx8XHPWAFkDAww2ARTACd+UA76xXMdhXscmB8EGhDD+xi08Sw+d4kF82NGl/flX0CEQYezVLwDQIU+uKSFvk8mkya3sBIY+PX19XzxxRf54z/+4xZk3N3d5eDgID/84Q/zox/9KCcnJ71tNM22Y8jdZwc1lCw6q8d7zmNzMIsMY7ROTk5aEO9Mx+Liw0GTnFHCPHmM2CuckiP0l21pmUM70DrWNvYeY+sb+lmZcjNslEUmHcC0Dhq8Y0MoJwX4e9tXnoPdW1tb62VhuQZ5YtwAz84GUXLB9tzsJQ/AZRx4H29njW6QjUKWYeC4P/bOcusd8sjYoI/Yy6Q7aJQ5hVX0FtrOTmEX0WsDiqS/nsdAnO9DqqCb6CX2C7sMG2r/gqz4WfgF5hIb5ewo1yb9ki0ACGv9vDONsxvYZpcw40cMmmDB8Q3YMMZ/Xs/j4T2YL/TZWUUHsJUcAFtA1EFe1DIbfC1+GvwCIUJ2gfWLPAdAjd2BMGD+TD5gJ9ARAnITKMgcugUG8Zlg6+vrLZDkXY6OjhpmStLIEkqi7dtNVk6nD1v2Orh2Zs5EhisHKunnz/mOMRP6f3Fx0Qhi9Bibg34540xpJp9hbxwQ8EwvFuf7BCr4U3TR8uIdvCCYuSd417YEnGUMAh4kCE7StpjGjjL3XncBpmXO6SOBomUA8mx5eblhaRPw4HGX+GLLmJvDw8OGI7EtBCxgeMbD+K2uHXtXe3SgQSoQpwIQwOgyYNPptL0Yis32ZSw+QXEdqPiwJm9biLKSWvPOSgaJCALOxIyNMwdm7DEOGBuAJPcCXNvx8mwmC+GyU3FKnv6Y+XIpCOyChctstvdft4Dy/RplI/AAcY9HkgZQcGROdzKHGBh2meI+SX+XJwcH9I/3M/jnh/G3gtI3BxD83elsjz/XophJV4PujA19YlwtD4AS1406MFtcXMxoNGrAd3d3t1cCgXLv7u7m888/z5MnT3J9fZ2XL1/mpz/9aX7yk5/kxz/+cUvzcjaCM1eMoZ0QKVMDAxtcZ73msSHzdogONnHAHJzGvFrmMeIE+HUxPs9A1+3AYO2QRbPcfp4BIWVHGOBalmL9Sfr1+s4goHOnp6fNZrlmG4eIA7ADxSa4n3xWQRGnHvMZ/0IAYRtcr41ttk5D9FSADIjA7qDDyDff9TbaZJt8zgm2BnmuByRSZgcDjLwAkiE7DBrpI/piVtg19wYNPN8Lu70+AoCSdGU1yGHS941J2rkkjPHq6mrW19fbeGDbHBwRAFq/6SMA1uDICzlNUiB7BnbYXoAtcsg4z1sbjUZJuiCejFzy5hlVJmewB84uQRBi8wFklJwwlvhidIcADh8MUYd/ACDWNTOUWbEDEbIAAHaGwjbBJB99craG3TIhKRcWFtpaR9s85MFzX/0jNgkchk7c3d219zTZwjjisyAu65rTJD3Zru9lMg3chz2gz9htE7CAYcbdZKyzDfgZg3X8LvMIPnIZrn1vJfvoC3YGe8LcubTRGRSX1brEzxmNigddqpZ0Z6cRcJ+enubq6iqbm5stO0QlkPGd1366YsYkkwl2sh/MMfLk8Xhfe3Sg4Zp+1ygyWWQnMKikwADGdNaMFBPMy6AMMDXOomCM+WH1u/drx0lWpsngzg466WogvSsHjKEX/DJRCAGGyhkeon0miElDOGxwMFJOjVfmjUALw+J3YU7MYjpTwu/sYkWgQ5BVGQgc33Q6bdvhOghz+YPfkf5U5pAxc9TuAAXw5xINnKCZXL7LeFq5zUw6oEvSWEG+x7U4dAJL7mlWGBnY29vL9vZ2K5NjjM00LS8v5/z8PD/4wQ/yzTff5Ec/+lF+/vOfNxnFIVEeZMM9HA4be887m/FH4ZFL929em4Nw78hTg1DGgLFL+mtrzCwBoMzkDIf9A4csexAFMMZmqGH2zXLymcEr653oF7YKsAz7iT7QP7bahUDAwDvIpT/svEbwBOhlkSNOFLA0Go2aPiBf9BmnB6Hj90GX6ufMk+cBUMCaKxoZCJeuLi0ttXUfOzs7Pb/h3dSSPqjGB5jd4/9c44wk78AYWybI2NpmAyzpozcHse1Bz+mTAxcHkJAyLpkwAYZcueSJd767u2uBLKzr5eVl03UH2TzX81gP9sKvYOuRN9td2+R5bMgAYKxmtPHHSVoGwzaF5qzj7e1tO6XaG6hYH25ubnJ0dJTJZNJ8gse8VlksLCxkd3e3nbkBe28Cz2U83vzClRAAV2QZnGPdT9K2NIW4pCQam4Ac1WxA0m1cYZ/r3TXpI8A26crRhsNhD38gj9hNnuOMDXOBjWUOvOaJOcP3Gfs5WHCgje5ge+mL/Uq1H96YB5vMZ84++3wPCBQHTJ4PE7/YLj6rgQcYCBsG1qXfvDdjgu1Bprjn3d1d22jGQQv9QkccbLjSxXaf9+J73MPzWzP/72qPDjTYc34wGLTJrsbZjggFBzhREoXAuxSGkgdOn3b0xb8IMoYTxWYrOP5FAQySMfrekhaGAiFlYqidBBBZqSsbD1hG8Ei5kR41COf7/GBEDBrtIHguwmH2H8aBa2k4SDMDp6enLRrlXc1aYnxQbjMLZv5QGrPBBnCOdnmOARxzgpLYODCXGBOPudO0vLOzJP67MxqwMXX8uC5JMxa+LnlwFCcnJ7m8vMzBwUFbG+CIn/mGKaa0A0CHLOJMzCyxHoBUt5ljG3OMDqwOiyAfq9zfxuaMgAPYpNvEgYwb+uuaYWfxHNw624csE/zD4CbpLbyswKQyzJYz+sozsBfoAvPL8+w86COyg1NL0uyAmXIHvUm3DamZSUAy9+G71GOblGFssR3oFc8kQINRN7hFtpeXl1uGj91OIJAYE8vv7e1tu/bu7i6/+MUvept+OBCndAyAhF/AlkBQ4SApGcH/2Fa5JGM4HLYSERwn+s4PsoFOmzjifs6KodsGJElXfsF3IbPstE0sMYeed+bcJan83SSV5Xt9fb3pzPX1dTv3gD6ZBTc7Os825KuvvmqMPzprwglZp+EnFhcXs7m52caAEu0kbT0dwBsbTBajZoJcLk15K3riIOf6+jpHR0cty4ie0PAhlZ1HTh1wDofDViJaCSuIl5WVh5OguS/9MHjnOQbkyDcBDuQv/gufTDaprnGluayHIJjxYBekJD07hHyaJDDD7sCEsfN38Z8QCwRIYC90z4Q1Y4BO27aajPTaNnAb7wPRyxi4YoZxxJ9gFxgXdNxrBZEvfAakfM1sEgTafuN7qt0Zj8fNNxmfYCPX19fbfJEYQI+sOyY8LXePzYo+OtA4Pj5uu2Ig2GavSauR1q5g3oeJEKxQKoPj3djY6C2CYcCSLpOQpA0q/eBQJPYf5n4wVThSHL6jQZi8yeRh0UwFBzXY8cDaedgZMCEIOj9JdwienRkNgXAAwLP5Ls/FsfJ/xgLhwFBQfwmwJVgjoOK7dm5WHJ5l0OKsiscGx+mxqePJ3ypownBh9OgXY+lAL+kcCr/zfztmB4g8l0Z/64JRlyk401bL4KqzQDZhoJE9SicYI1gnp8uTbmtV12a6bMNB/Dw369usANzXOPBLOpliXpK8IW/jcbdjE4AUgIWuVnCC/OEQuJ/ZH8Yd+QCYwpzjnHiW12lUvUjSW7THuzpzQEAEGGAc7DxtA52ix5k56+qyGmQSgMr30cXKmpoJ5m8nJycNVNgJ0XfsDhk9QBfjAIA7PDxs2SWXinE/l6IYpOPwnf0z8ABQAaBghcksMH7OhvC7yRocMgec4YfsZAkU+D/fQWeZt9Fo1Pzn+fl5b/cr5oV3qv4HYo6gCyLFPpjnA5YdnJvAsk+bx8ZaLoIJQHaStvsf9vTk5KQRdOvr6z2ClJ0oXWZcg1AHqd55zPKY9LfcddnddDrtneWBrRgOh60EHdkBgLKhBYG/dRF7hs3inQeDQdNlfIv9Ne/hsmbbBwcb9s2MFfcxUQQuAMNhA8lMAIghFYy/nAGxLzXzj1yTCWEeANCMNeNn4hhSGVtvFt/YxQEqtiVJb7MO65erIfi+17cxZyZbebbf0es1jVUgCxgTbLf7TfDCfCXdOg/7Kb7jPtlOYus5+8U+0oGqm8eIwOYx7dGBhlkm164x2GaXXf6EcDAIZqoZNAw5A0t9H0aVQaugmOgTcMnzCGzYN3tzc7MxXGbr+B79vb+/byv5j4+Pc3Fx0WMhnT7yu1uYK7B1/e+s7d0MGjBUCJCDFdf0IQQu/bm/706+5RmMm09ONyvvlKKBEH1DGWrUjpBZSa0YzsY45WcjYTkwq2fFsROlf9zHDB3fd38qi4kC8X7D4bC30BpQaMVH9uiznw8gsAxipCm74t7MD0wwhtMMF/JmNsLMk1k0s3Xz1nBiXnDp9K0DEDt9B7dc77QxoBn75NQ9jsFODcBoO+RAtAb6JgQM4LBFDqb5gfm240buDPpch2yGzaDCTjXpl6DZ0ZsQcCkHzg1ZdzaA75JNJlMBYcP2kMgnczCddmW02DLbP7JR9NmlGfQZG2+GjjF3toJnEhTU8TPL5hpkB6tma23/ubdJDtfIO9CyLNpWGVCYXbYsM05ec0JpmgM6xtbySF8IlAyKZxFd9JV78H1nVR7LRn4bGyDYQS46ggyzQNoL+attgfwkU8DfCfpMlFHyyBqKpPNvBmbYN9blAe5s173YmXmlHwbE0+k0p6enrWTHzLkDbWMzfH9lrwHCrkigIeeUQNpHYwv8bGwDdgRbytjyfd7H5a3Yy+l02shV5qGy5OiVidn7+/ueTU86ctVYCn2HxMAOobsml4zLeJ4zgZYbxoA+8zfPA9/nnafTbn1OxYvYAvuQWobJWDg7akICm44tMFnNXMzSFzL9trXb29u9BeBV5ghMxuNxbyOPd7VHBxqOggy+LUCkgOwELCjU9S4sLLTTNy1gDMjOzk6m025ROY7I5SqwVBY0Bjnpzv1AiVEKyg1wdABmg1wWArENGGAFR2bwl8xOMyEcFhZ2F0FAuA/PM2AGgHF/BxcYEgwWAoVxRCiSblcbtvdknCsTiSBXsADA5f0Mct/mtBl/jwXjZjDpSN4MLX1x/a0/N1PAD3PHNbwDRgUQQJahziXfR0kpFXFpiLehYx0S8ov8Y0CQG/oLULGxYt0SBg1DY0PMuDhjNM+BhgM5nCRjBxmBfA4Gg7YOy4Z1lmODpU/6GRBkzWWc1fA6oE3yBtD3Nf4/9sPZXdsGamWtX0m//M9BL6l4lxCYrSeTi2w4wMf2MD739/dta0jk2cE240a2jebgz/qJTDrr4IXWBDcOGrBHzCmlrgAistE+J8jb7mK7GTvAG0EizpgxJoD32jrGib5aRhh77BagzHYQxo95to4iI5ZDA1CCIsaO+nHkimY7AXiq9fHOsvCdSoAxFpXMs5y47GZeG9kKsmbIMPp8eHiYJNne3s7W1laWlpZayQ5r97wdvRd+O7uYdOenXF1dtd0DWSvh8QdMW952d3eztbXVtg/Fbx8cHOTrr7/u+UD0BPmfTqctsB8MHtYAuRzYJcb2wcicxwq7wfP9d2QFu8U90F2ywz4DIknPNzvja5/PD5/brtC4J+MzmXSHL4LVkPvRaNQjBLmW/tMf79aJnNsGGFTzfOsg+urM5d3dXW8HL9sIzx9j7LJW+sf8eS2cdZH++5wPxsGlUdgZCCrjNcbS+s+zXKbtRfIuqSI4fPr0aS9rQ5Z6cXGxrc3zPL6vfVCgwcMoc8K5+8UqEILJY4DNZMLs2InS+eXl5WxtbfXq7ZIOBJhtMNDEsCPkXIfwARg5R4IfHA4AeWnpYQteDgMDVBtQmLFG4OribIMeM6AIomtE664wBpyOXs2omqmiOZoFDDswY064L4rCeJpVtBDbcBgI869BCXNPP5Mu08DvZqm5p9fU2AibhbOxNANpNpAxZgx5tlkBghmDBgAZP1Y2+s5+3i7lc6BDsOf6+MXFxZZlM2A2e4WB5l41uKCfH6Lg37bmLCGGEN2w8TVoTNLK/gDkBgroMHKDY+T+Scc8Mu9J3hhriAyz2HxmWXVwUNl1CAXmm+8bKDpoxu6x45gzrzB4Zi95D4AsoB1bij7QNzbNIEBBZukrYwObmqTdl3EBzOFwmR+CqZqaHw6HrcTCGUQ7dmdW0TN0zmAa24PcA+isz8gQjnjWOq8kPXtgoOHMockLxgmyBdmw/Lj8indxxt/22/PufntcTNCZPaXZVpuAelurwYmJuXltvDM/ZC+c1SGYYH69gxJzAehMugw92GR1dTW7u7tZWlpqwTOnLSP7BnrcG3ljjd/Gxkbb/Qk7xVqk4fBhzQNrTHk3bCHn1kBCEqji13kW2XL8utcTErywKJztXvkb9oDgPulK0wmu8I1Jf7dPMI7BO2M5nfbXEZBBgIDATmNfuC/v5wNATeQwRvb1zuxMJt25EbZhSVopqjGcsaOBP3bOmMv21dkPMrfj8bi39i5Jzz9AxHutsjcDMLFkcoGMksu3k7whc3zmcfe1Lt9yAysxroeHh60fYBvO31hYWGiL4jm76DHt0YEGzpOI/OzsLKPRKFtbW73aU4KRs7OzNrEGzkTmrOXg5c34IqQ2tERSNtwIt3fncIrS90NYGEzX6VJahHIabACUDRKsENyT97MgIryOOonASYXaOZOG9IJ0+uAsBONZQT4K71royoC4RAthT7rshJ1WFWrGnHFN+jt82IFaCXwvOzg7bLO+tUzBQQdjwe/8n0VUOHPkinclsGNnHKe/YW9gk1jYxxaryJPT3C614D2ZF4wU70B2zaUfyBWG1O/pgBYDCstk0DGPzelzyz1zurq62ptL5pNxQ+YBnoxjDZL5XtLVuSZpJS3oitfFGFzajtAX5AZ2ij5heJlDAtEkzSDDwp6enr6RLaAPMNg4IANtZIZ3c4YCh4zDcKkjgQ+AyWNJ2ZPHou64k3SnGWPrbm9vWx2xSyn8HbN52Ah0w+STbRffRbccPDmQxG5zDf1wWVclOBjnavsMPPndASU2m//zTtZTEw2eTwe6tl3OqDkbgt/je8y5yRn7Sa7h/7wnsu6++v2S+Q40bAPW19dbOSp/29/fz97eXsteHR8ftzVFkIiAOJM/yBN6laStzWGb8uFw+MaOQ7YXXOMSRrIpZu251mVUlHpBIELk+qBS7u9d80ajUfb29jKZPJTgUr7LuiRsi0E9euOMI3YBwAuZMZ1OW1klBJEzm+in/Z7ximWUcfP7O0jA1vB87IbHFD2HQKG6wHaDfhr70Ty/zIsJSBp94R7OvNSMBO/pNZZc77K3WsJrvfV7Q4wjX5ZZByvot3GIMRetYi9/js3id86sIoPMOLCxx+HhYY+kekx7dKDhOmMawnt+ft5KTcyuIcgMIIsUYfp4Oa6xMWaQDTCTNCVDqL0YyGsg+I6dB8Ln2jyUDMYDlg0GAmOFIiTdGRuwCo7OcfyOJg0oMA44bGdUcOQwHkmawSBV7BSt33MymbTvocS8G87JDriWENjYMN8OpJxB4Hu8mwXYyoADrw7ZAR/3xGhZqXmG34F3S/qHoaFsTjGaoVxaWsr29naeP3/eGCQCZIDgzc1NTk9Pc3Jy0sbx8vKynchqVtrKzTsgB2ZhASCMhevUuc7Ax/W+jP9gMGjldWbh5rFhvBYWFnrZAuQCebJjBEg6A+FAHj20HBrkMWdVXzwnyBj6wVwTXHg3IBxgMvtMFzK+bnWBdg3U/R7YQnTfjpLvAazI/FILjk1M0mPI0AlnTBgnn3dhUGAwD+CHBDGr7PnA4TozyTOYZwC2swCVZcQOkLEBaDgLZrtAY/4cRHC9r6lO2WOLHs8iSaz71nmuI+gzmDI76f4jv8wL9+Z3Gt8n6PV7Gugyls6SV8JtngMMmsfLWR2Iir29vXzxxRcNIBFkE0RgC6bTac7OznJ8fNx+IIQAXGQBHRBShg3rjo6MRqOms2Al++CKT7AdNzc3ef36dSNBHNTjl5ALCIilpYcTv8nemkQgiGCzgfPz83Z2D8D+/Py8ZS6NYYybkj5YTTpcQN+xofaNlmNsrQkDfJyz88Z+/nyWHhFAEUQRKJIV4l7YFa9Fc+CBDteSRnwCvhw8R3DlTKzvaYyLzWUubSu4PzYSTArJYxxmMsIZT+tx1WkHMm/Tn/p9f2Zcgl9gvSHjaxLuMe3RgUZl92G/6MxgMGhb4DIJZspcukAqkd2iMAI4lcooIRAI1nQ6bcDLaxIcncHYIbQGpPTZk762tpatra3s7+83IcOgLS8vt9ImPvMiGMqTUHqYKY8dxmgwGDTgwiQ7He/yLvqFcyZ9RaDme9sQcj1zRaqS+asHwSFMAC5H2SgWnxmguN8AMMoiDFqSvOH8fD8DNECXwQb3948DEWoYMcDMGcEi39/c3OzVyyZpO3MhN4eHh20vapS/ZsnsKGgGLjZCOP7l5eW2Rz7pxgo8GFMHKjDD9BmgPq8N3bD8WX4ZMzNkNtJek8T9kq7EByeAjAAMuMa2yZ8Z0OPkk/6mBxVw0OxknXJO0js7iL7YcVBn7tJRwGqVfxxX0tkJdtJJui0nsX+WSYgdyA5A/GAwyMXFRZNJkyaUntgJe3MO+uXA2YRBzSowryY2ZgUY9ANnzT3M0CIHEGDYdvRjOu0W2dOcQazB+izHPAuUe1zrdxln/CFzhQwbxNmWcQ8C27W1tcZCOxBH7muwknT2tc7DrGfX4GzeGrXiBBCQc6zfefHiRTvXC/A2Go3yne98Jzs7O80XHx8fZ21tLU+ePMl0+nB+FJlIyEd+8JPLy8vZ29vrbTLj9ULjcbcDFhgJWeEa5gFQSvBOv1wuB04x2QcGwtdhM50hAecw3/6MtYGDQVcSicwgu+gKNsn6SX+TtEwj/SP4IjPiSgDkHlkkYEPXa9bf2IJ/K5GKnuGrrbOMCRkOdIF71LUM1WZWPbG+uT/gFQIoEytkBNBP4yln4G2bHLjRXJqW9EkR/+vxelebda+3XUMygXvjN+wH3tceHWjUVfuAWJeCmLUxG2lgYFbdjmNWtFadj40xJQIYc2oTHW3xPDtsokX/6xIsnKujVNgkM0QIuNepYEis4PSfPeUdvGxtbbVDgsbjh7rOg4OD3N3dtfvaYcDKP3nypJVk8E5cQ4RfazgrI+raw6Srk4eNq5GqF146Qnd9M3NoJr86U97fDhD5MKgzy++UM30DlPJOFVDASJ2dnWU8Hrf6WrJv/BgQnp+ft73UcVCMgwGUS+t4H88RAROpcAy41yRVEOn5QAfMfvGuDj7msUFSYB+S/i5S1l3vYW4bw3h7dyE+r4yg5ZOsCfW7gJWkb2soPXSNMHJnAsKZWDvS+/v7Vmpge4je0wdkCodmnbfeVGfP3ynJQxZd8lTBPDYi6QImgxPbawMDPud5dv70yWSQgyxk2mPLfHk3PhMWNJ4NqWKA/jbyAdlhTHxfB7MVwPiZb3PSPMPvhm1yJtbAA/DG79hX15Rbvvzelmk/3/bRpFTVB5N0fj6yU4HMPLVKCCATS0tLOTk5yeHhYb766qtGGsLI/sVf/EV2dnZ6OwIuLy9ne3u7lYY8efIkn332WZ4+fdp0E7v94sWLdi8y4oPBIGdnZ813uOzo4uKiLWZGJu7v79tW886EeQ0QWOPu7q639oL7uJLCJJgxC30wEMV2kKXl+fYnkJKUC7okjcC/bvgA7jDTb3wyGAxaWSilaNh09Bt5NB7gu7NsofEIf3eZvXEK72QcZwKahv44+CGIIggiqKu4yraV8XZJMPOD/Xcm0mQ95wqtr6/n7OysZxfQZf/f4+T2rmDDf/M9H/Nd++e3BSiz2gdlNLy4BqOZdIdOJf1SGrOGdpBWBDoNCHcZApNqIcK446yn02lb8Mi1/r6ZdGdiXA+dpKUY2VUCYwJjgeCgJBay5eXlPHny5I3a5qQLylj0xXuvr6/n6dOn2dnZyXD4sKsFZ5VwKCHAxuBscfHhBOBnz571nBAKwd7gCDRMDOCI+fIaEJcgVEeOkaMfKIiDC5hngkfmgbEys2cg7syA66ftPJNuNxjGmoyTS0MAcTZE3tbODNDOzk5LQV9eXub09DRnZ2c5OjrqrW+x/HpukUuPl4Ebco6zgHEjFc91NnYOFp2lwhC5NO8xjMW3tTF+Zn+Sh+DQ5VI4MAepZuy4hw0737UTYC5hxrEP3irYZ0pg9GEyHQQTCPFdy79LDpNuW29kEnCQ9M/NgNUzg4j8k65mTZszLbwbO7RUJsy1/lzvTJlJIS/wxgabUDE49zz63syVM3J2yvyYwAC423kxlw4ealBups0yRV9q/fIsh+jAxW3WZ8y/34Vn4nucVaHPZMH5nPFGNh08sJgZ2RkMukwc74+/Yy5ZXwg5ZH+AnDl7w3h7/uexAd4pUcGWX19ft+1ZFxYW8vLly9zf32d7ezt7e3u5urrKT3/602ZHkmRrayvf+c53sru723QOv/D06dOWMfTaruRBziAFFhYeTgAnm4FPQw6pcoAAZE0X5KTXYtpXsOEFto6F3EknE9fX1zk4OGgLhk2IeLtqSF2yMtg8ZBtfip3DZ+Orqx6ChZJOl0ajUS/DcXZ21mTU52+45J5MC/7VWVv6hk4y/uzml/QPWyUAohkDOjgySWq7STMZjo5Stu75tT9mzCv4H4/HvQN3yXa40gE7jixTFseaPprt2Nvs14e2DwkWfP3byJq3tUcHGklXm5V0rJhXq+O0OdgPgaFjOAEmzes0UFh/5kida7yDilllGobbaUWXvOAszs7OegszYZaSNAUmDeidtRB4nD4ZFtf0EZR43QmOkHGaTqc5OjrKixcvetkXtv01wARM8A7X19c5PDxs/QFcu+adwMJlIrwbwJmacaftHNxwvZtBL+NthgwlMiA2q0lK1cxaDUzNzNDMHpqlraCEoBejhfPY3d3N9vZ2dnd3W+bq8PAwh4eHLcD0uBsIOPCY1WeuMxBkPs2iWjldAkMAabCE82E8eZ7L5uax1VQ6oK0GULDZsGA4C+sowAowh0OuLLyzl7NYZgIXAkPICweTZEtwvLUkygAE1pB7oJdJf7tJM26UYnpBM/p5dHSUs7OzJHnDKfI8HKAztLwT9duML38DLHiBdl3DwTUOVLgHzTbAjLwDD9upmtHDXnMtjKj9ATZ0VtDh//t3r4+a1d+3OU3exRkRZ9nImpmISdKAQ9KViDImg8GgB4g996yvcUDioMBrg3zQKsy4mWDPle/JOEC4uUR53pqZ4CQNpBHwP3nypB3+izxRHovfZQ7Y9IPx3N/fb2Wu6+vrLWsB7lhcXMz29nYLJO/u7loggh0ZjUbN11AKR0kfQB9/jcwsLi72Fp7jv8E8SV8GDcJPTk7y1Vdf5eTkpJESzK3L7dDz0WjUA/YAZvwnJV1Jp5vooAkv+ryzs9PIQPSKUjaXWDFHZPhPT09zdHT0BgHiNRr0Ken0GNzmjDI66+26HYxBIrmawrrhjBA/rNsFQ+JHHAjRL/pm22KcAkHmNYo+i2I0GrX7e41W7adxnTHQr6JLfpfHtg+9/oMCDZcaAU4NoIjWvCagDrzLUhzJYxi51mAXpXJpFA7ego+xRTgt1D75EOXCQBiIMIkOLOgXbPj+/n5WV1cbWDfwNjgeDLpMDGUcOAkUnYXt3o0AZ8U44LQYS4MpGE8DKYMCFLc6IZwW/WQeMEwOzkj9+Z68K2PqekfG0+CDHwNv981rMxBivydGKOmCJcvW9fV1uxdlapzCDdOSPGyv+vr165ycnOTk5KSBRWSI8WD8GQMcvpmdWSlanuOgjXvDyDDeDrJ5b2rOXb5Geh8jaqA5b40sgUEr8oaxdikazKDHMelq+FkICUg2uYGD9d+dkXCJHcAb28UcI6NeMwGwN2A2IDXw53p27HDQCstIzbQdXdKB/NPT097uHl63gL7CDFq/nKmsfXQ2ke8A1KodZrxngXMHx3Z67kuS3iJp67mDg8rQuYzCJWomLnh2vadti+2I58vvQ7Mvc3/Q18HgYdtR1gPYDiVpgZ1Pfsd+sMUo8o7N2NzcbOtKAMc1+LZd87vYLnOdg2b7QVcH1Hr2eWrn5+c9G2FykfVrbODhbZgBeElXLuRx4Tr04vb2Nq9fv+5VOVASBPm3sPCw6xzygT/BF6yurrYMukvmaNirhYWF7OzstDMUCHT29/dbP9kxkfsSoGxvb7cAAn2GGFlfX+/ZRYgqdv102Tikxi9+8YvehheAYYgvExcXFxc5Pj7uYQjsEUEGG6pgE05PT1s2w6SI7TX3Q7YJ/sBT4CWXC/JsMsF1cxwyOmBAfy/pr31JOrzrihCT3/ZF9MtVMpZPngP2qkENfbHdAQu6BM7kDc/+VdrfdIBBe3SgwYJnBgLGidQYimnh3d/fbwaUhcwnJyft5G9H65S6YDBIN3rynclIuogVgMxAEHRwT0C+HYMZbyaVQAhFHo1GbWs5/n51dZVXr161RcVLS0uN9XCwgVB767UKdnASrrt3UOGAjXHxuhSeZaBCH3DMjAEK7IDOY4eyc3/Snd7hykrLnKFYPKvek+sMDhh/wBjOD5nAiXgNioMVAwUHQgDRhYWHLT25L86DPnIQI8CBIALQZ6DGlrc4MGc7MPQ812tMDOQYB9gUM9xcx33pE6eLs45nNBq1sZjnxkJO5ibpQCjsU9ItjoNFNOuGkwEUk6W8urrK4mK3UYPtlcvdmCuajafl26Vw6L/vZxDIdwG22Mu6Mw2MdgX3gGkvSud9XcaEnXHmDPuBDDugse3wGJAF8nXWTYN4j4/BucGwiQ901GNqgJL0S9n8Toxz3cnLzKNlwU6aa2wjHdjQsDEuXXGrAY/HPklv0SpgAEKrBpMec8Abm6EwXnVHOS+iRTb4vncmI5Cq8sD7+vvIcSWJ5rFhJ5Ef7PBwOHyjTOXi4qLZZcbIGTLGGnm5uLho9h7Z8oY1S0tL2dra6oFODptEn6+vr3N6etp0+tWrVy2rgewZKJ6dnbVsiPu5vLzcgpjpdJrNzc3GrhO4evcidrzClnL2B3KOfFIqdXx83PoBRuAankFwzzpSMqtJV+HislATOGRFxuNxqxrg/SAG+Y4rEZwZcfCC3wA3miA2iU2VhslMYzPbJ/tzv1MlP6pNNjnpDKTxqfWNe3A/+kjW1u9MP535d/P/Z9mqb2N7dKBh48mgAcqoE0WgiTTZJnRhYSGnp6ft/A1nDsxC8ZzKduL4AOx2zEnnDDHWMBcEJjQHGknH7BkoTCYPtdpJWimY68VRmvG4248b0DSdTlsJFc4IcM+YwSS9DWhSVsa7A9K9e4Gjahy4HS7pWjs/l1VxbxTHCsE7E2ww5hhaKyDf8xw521DLtriGAG0WaHFGymDdQY5ZfwI15tr3I6uRdOtSCPw8bpWd4IfnA2Jxag6yMfwOpMxMMseTyaTV2Dowsbxzb1L2bL3r8UUW57lhE5B718TiBL0Y0sE5cuv95vk/94ZJw0ibUDDI58dAhM+QB+wN9fgmP6xnXsPjTKfriZlnggFsCjIPWYMs+d8aADjzCMhAzhjPjY2NxqJDSCB/ZsVqM8CwU6MPszIe/N3vaZKE96jgG1/A54B2Z/SwY9zTzwN8MB48B/CILXMGHd1mC2I2gOD+Nbih0UePsbcnxwexbgOb7My/NyuwH7Q+mDTD5zCW3Ne2FhuYdCezOzuIfAHGbCPnsVX23POKvYewYOyx+0m3VT1BHvdAn05OTlrZI/fmfB+2jHV22j4SAAlJRqkbB5VCChAgmiREDpBlzjPgOS9fvmx6wBhArPCezvi51NB2jay7S04drONLKRMko0E24v7+YUH7+vp6y0wY9yRpa1sJWthO15thgHMqBsSu4w/wiybpsPtcQyDDD9knZxqwF7apSWZiDNsy+lY/9z0qhjXGdMO2OWhhXYuDGAc3vw3t0YGGhdmOE5aGWlOMLobv+Pi47chA1IxjQOAAjnZQlQF31MqkO61vxhEnQ3CC4a3MF9+xU4VtYJcJTveEbYLlrmd5YMAxVn4fQJAP+eL5TsNheGBqEWz+dW2vWVYMk4MJ7suYMN4AIise7EWSxvZ7RyYyVRgKwBTBgJlfDKgBUWXhzQTY6TFnzKkBqDMkXAvIQP6Gw4dFldToEiD4B0PE/Li8hPHCeFnGCLwwClzPvZzNqUGcZc9gESOzuPiwQ9X29nZvv2rmEQbHRnKemxekOoOH03Tgjhw462d9tQNlXjDkngvbHZ6P/aFPDvzqYuqlpaWeHXBQQt9NKrjMLumfAnt2dtZsiZ+L3aiAHQeU9ANyGn0FVCKjy8vLDci4z5XIcaMfDgZmybLfzYx5dczoqcEPAAG9JWi0bNze3jbSpc4H+uEsooMYjyVgDB/DPfg/tsTlM34HZ/A5PdmAgj5QUoNc4xOcxYacYgy8DS6+CtBpgsaEhVlvmHYHKbatzA1+xmM0z80ZLeaJOXf2z0EEbD2kHSUpzvgDwlmDlXTZSi/gvb+/b2tQCSpg+5ljfCUYAd8Lg+3qAcB3Mju7S3aeNRGAePuGpKscYAyQb2cSk/QWYONj7J+43jrrwI0sjjMItlX39/etagViBjl3kDiddrtbEUA7M0pViHXWxIwz08iBMSrBJWNHMMQ4Mb/Wcxp/93oJ+xWav4MNrIETrQYj9iE+oNUZskoKcG8/dx7aB2U0KmuL4aoLixlsAKojfafLbQxrRMe9AVgs5jLIxFjjuJg8aq6TLtXOOyT9tQUAYQu4QfhkMmmpQXYxwqkgtF7gCfh0rbOFxv0x82lW0+ylgyjGyrsrYZRcxoRyM468C80A38EIDQNs5o1nc/0sBqbKB+Nl4My/dROAWcrrubCM+BoMyOrqagPqpM3Zyev8/DyXl5e9gxA9TgRMzrTgCAw8mFsbEpe+OANSwRwMjkEWWYu1tbX24zQvoIXAHf1w2c+8NeTQAA/dM/vKXLDxggNsgycb/5qdsx2pASwA0OPqwN6BAs+4vLzM+vp6NjY2GtCD2OC5LnlxkOrymaTbJtrXJZ3js4MBmKJfNTi2HHu7SfTYuv0+NtuBm4OHqquMk8eZxu8E52SC+MzMJewqz2BuDKQ9Dm5+B8qtnOlB1wCRzmq5BIkxm04fsvNk6OlzDR4ZT8i1jY2NdkZPkrYmsIKvSgyZQAMI2xfMyjyZ5LANsu3lOXUe7VvnuVk/PLcGpwbSzJODbPQJn8B1Dtoqq+3sE5vakNG8v7/P+vp6km6XKeYTW4aMskMUMk2wzH2qbrPYnSDJgXllwU1cYpd8FpPJzQpY0ZXJZNIrU0c/GDPeC3xgefVidMYbG0yQxrPoP89F9pkjrmUeWW+C7YGYtb+mdI2SOeuH3497MxY8txKB6K3Jwhps+Fr6ZjvhsXZ2kjHw7pj1nv7dz3yb/f62tkcHGkSWdjIGQCiRD7uDxXP05gnjHoBas8gG4HZIXGfhNWvpZ3hSeZ7fIemEmffAydmgm0k1y232yHW+CDN9dBCDUsK4kCas9XgYDWdK+JxnWXgN/vm8AgI7HDNm3IOggPUnzFs9pdlO1wadd2aP8MokO+iqwYNZKYAm88CzHExh9NnhjACDkj22rT09Pe2xIpSZJGmMqcfJTtiOntSsQa0Xy/H+BnZ1cTLyi7PY3t7O5uZmAzXeCcQZP/4FFM0bm+G2urramCYyGHY0dgTVICfdImrLnefOMs79cOboE3Lk9R/1ME6z3EmafnghOeVKnNeCjLpMx7JtJtoMYnUg1SEZIDnQngUWzF7aZlR2rD7XzQSJmTQH6ZZFZ6Vm9dPfN1HieWLOOZDU32HO0QPbb2yhM7hmIw0SbEsYX/wWpSAQW4wT/oX5577O+FAaglwRaHjRp2URwqEuUkcHPJYGP8ieAyDPvd+pkj6VyJk3oDKrOdi4ublp2MBBBzptYosMHySVbTpzBFE2Ho9buZuBsHHK6elphsNhW0MBQQoWMtHB/Lv8EvBe/aCxFf7w+Pj4jcN8a7bPdhNZrsFFJfNMuNFHsmtkHvlOxR/skkUWhiDEa4iQd2OnyWTSrnFW2+Qm+mi9RU9sQ63LxkHMq8/rSNLWg/J9Z28rsWpZm6U3dVyr3a2t4iHmh3FFDmY959uiu+8jrGp7dKBhwGqnwe84OGoIccSwBCiPFyS7zArHxTWOzM00MtAsQE+6OmqMq0Fs/TsOByBhBqhGr6QHrUh8fzjsHz6TdGk2gxHGy8rCPQgaUB5+t8GxQbDjcGBSFcJAzWDJ7KiF1kAo6cAK88N37BA9H7Vekve28bbCcZ3fzUGIGWwcLk4BkEK6mDG9uLjIwcFBTk9PeyVSzjYknVOi3/SFRl8MSHkGANGsVw3YGFMDXu4L87m5udneY2GhO1MC41zllWfA1nwbDM0v2zgJm4a8O9C3I3a9rRmqpCtdYX6n02m7vxdJmnTwOidnNJN+Df5kMumVyUGaJGmsKWUTfnYFLc5YmGiw7rlxrfXZ+ua/0Waxa9zbemqSx8+uQcgsp8kYMkewss44OjPMPHmtnseAzTbYTYbS2llrV6wLJi5qtsXA3AEi9gOwyP9NDrARCddDYnAfZDLpFhNDQF1eXvYIK+wJ48T9AHDswGPwiay7dKrOmwMM64/ta2Vlfe2swHQem+UTu5F0ZIDlMUlbJ8X4E+wiz2CJej6OMyXjcbctvf2CfQdr8AgCXC1RZdPBA7LhzBTXY6vQjclk0tZWgKFqSZLXZzoDz9gZn3A/kzEOhC2jkJ+Uy0PQnJ+fNxlnATuyzxzwXowJtpjxcVDkuWOsbUuZG+trxYzYH0q4bL94lrGW+0kfqi1J+uvO/F7Vls6yoZbfGiTN+ps/49pvS/tQDPJBpVOAZSbGqaTaCYSc6zHULqPiuyhKBW8Wdl9vI4qQVdCMwE2n07a41LX8vBNCB7A0UEAJ/e4GRTU9DmC0wBvM+JwGHOWsyJx3MYNoB+Hr+dfG1uw9jroyo1WhMDQ20j5dnFaDBlpVPPfRwQ1Az9kAyw9jRikR706NvPedv76+zvHxcU5OTnprJerCzxp9O31rMIas0i/GrLIfZlVg0Wwok/TqaRkPn8mSdLsi4fy2trbekGFkyYvg5jnQMNC2rDrDwMGWSVqGoZZGGCzifCjVYe4ttzzD8wH7yb/IKoyhS2qcXcQ2XF5e5uTkpMmctyHkGZ5Py/qHOo/KXpsk8Wdmcrkeh1rbrGdb3yszx98cMKEbdvo1kKyOlJLR09PTXubUZIaf63vwjgA+5tVgju96/m3bGTfsLLoLi4vuUoLB9+r9CfrNzBpkeDwsq7aNBlS1Bpx35VqPhYNnNweYNbCoPnKem/WS8WLjFo8jvsObutjmuLyJAIRt0b1LE5lXb0N9d3fXyypA8NU+OnDA7nhrawhZ+gRewC8Q4LDGi2oP1gQhC2AYdvaDBEFHvSZoPB73iBH6yNjh1+xzCKru7+8boWP/iV6y+B3SmQoUGs+aVdqJT7btnU67g0ktt/hoB9voJdcab/nZ9tfGWb5vJWPcf9vIarO4ZhaR5HvM+g4yUJ/p6+axfdA5GknHbNW0emV+mEyfLIlAOl3kH4IMG0YvmsRQJ3njMBU7AEAL9yDSpo/OkiA0ZqwcqPh7BtEOBBAMf8Y1KI7ZdTstBzZJeuM6K4Czk3VNoBdHV4fI5xhmz2VVoKTbJamyBLVVh12zLh5HB2iMueuzk2R9fT1bW1utrhsmmwV6GHdYCn4/OjrqHWwHS0Ra2iAD42Kw5AV7To3PWsTKWBqgADJYK+K1FewCg85g9B2sW45PT08znU7btrq8T033z2vjfZDDlZWVVvo2y2km6Tldj5kBF/KJU6NkgZIbkwrWA8sKZAC7yjgjSl8AARcXF620ouqRWf6kn2ZGH6pNMTD0tf6Me/n7/jvvW4E7Y/i2+XhbqwDVGeVq62zHDIhnAVu/u4Oi2hdnd/welZhwf30Pz4vn3PPgObajBywBupL0spzYFpqDW2evuZ/HysEqtoHPKOv0xgLIfd31b9bczRrneu08A5akW+dV56zac+yIswvWS+YQ2wqBBbagzJVzU1gn6i1jmXOvQ6q67f7RR97DJb/IpQNoEwpeQ4LdQh+sR673R4ZcdeAzLvCP7J5m++o1dEl/RzP8Gbph0sj219jG2RKCHXytMUut6MAmM24mc/Gbzk64xM0Bh+cEfTNOmeVbmM9ZxA73S97cPfRdrcrJrL/9trVHBxoIDopp8G8n4AjVaTGDYwcgyZunPlcjW8sNHKXi8Oifo0GzF95i0EASB+QFWxZc13mascAIWLgMGng3anUReC9yxaHZkdP3Ck54D4N++u4Ue50Pj5sVBaNnxTMYqcEB93OwxH2SToG9VsUsnx0sRo0+wVizZoGx436wxYzl9fV123uclDJjgbw43WqD4TkgfVqDLX4MTCxvfEZgwTaAME7WA0AJa0ZwGMjecNiddg8bNRh06W8MPeUmgNx5bV7ngiPH2VZ2LOmXhlj3TUhYRh0U8v/KOOEobV/cJpNJL5tncG/bg01A35Fz5MuO36Cosv60CgbfFYDUkkDuPStrUn+f1R4DPt3/CvZtf972PvVeNYCq/+d7tjf1b7Uvvs8s0O3/W8f9Pba+Znxd3sp7IgN1FynPi0tBDMjsJ2rZDFkS+yp+TEI9JkCs7z5rXOax2a8lnYxAyKBryON4PM7BwUHP77h8El0dDAYti+F1XJeXl80+g18oezLLj+6zQ9lgMGhkHW0w6DZCoYLAmQbfC73iOc7WTKfd2iivXTTJYezFePj7SSf32DzkkL5BatFHMJHXoFEpAnZLuoyrSdikv8U3f6840UERz1ha6rY9x6/b7lSMRB/8/rantuOVKKoVJMhcbRWj2ca/r72NKPB9PW6zWu3jt7l90BoN7wbCZ05ZG+ShXEyiy5Zgy2uwwPV2Lna0ZsQNzitAMTPgCBqn4K147+/vW82jnYlT9Lyrmw0Ugk+fGBc7jKQTVo9D/Q7381h6DA2iDbQM6B2V15pGOx2ME9fYEdVyi6qg3I/+WOnpm1PSBnX0w+sUAP6zFox5P+7Ly8tmEGne4rLKI42/wQqZcaRPBm2wJjgOWEpYEgII5tJsD2PrxXAuq6OtrKxkY2Mj29vbSdKraaVEiDHyokG/+7w1gzNk7/Xr172AnhNwCdwYOzNkzDn3sfzDtDnIdJbEWSGyYkm3O5CdmINws3sLCwtt0wTAB3LjIMQsNH2vzNgsZzLr77NAtO3jrKyjG9+pNvSxzXban9nGfUjzOLwPOOMnqsN/G6h+zLP9nWor+NxETPU5ZnE9BraT3GfWYt+k287Xz3OQXMH0x9aVylkfaV4YbeYcnab81msIyEiQ2fCulpRDkrEA9NseeZMIZzboAzgH/0LW0QGkZcr2BJ12JtxrNFwGhm/jWkA0AQbBQdJlZiG9WB+UPMgdfoa/1/WlfMenZru8lP4j98wDfTUJw7u4f/7c1RoElHx3Fn4yhqw6xPs5+Kp6ZZ1+H9CnvQ0nPua77/rsQ77/bW6PDjQc+c1igwywZhnmyjo5sEChUY6adq7G3E7VgMXBAg3FTbrD6AAwKA1lG9Q+GnTbOXAPACTKVAMCR9kOgGxMnB4lpQir4ZRo0jlz+stnsxwk7+8Mi1OKzvx4Hkm3urTIwN3ZARwiCu81NgDJpM9aeByYF5fGMO5kGWBHYG38fweOlg+Av7M1lECQTicV7oVojAlyZ2NJ3T/7fNOf+/v7dqCSWRjGgzUmsJQwPWtraxmNRnn27FmePn2a6XSa4+PjVg5m2behOz097a1FmddG8EaAZoeC/CBfLmmz0yBIBQxwP0gD5o75Ra+sKwS9vq9LDJ1FqUybiQBnYL2g3aUJ1fZ9CCifxUJbPgxKfajduxzZhzrFd7VZjvxDvz/ru3W+LfO/jv6/jQ2sAVwNhPwZpNV02p094Hvwew1q3FwC4u9W4u1dfZ1133e1Dw0wv23NPha7zftQAomdYW7wZ8fHx+08FOw1OoqfYvdMMtX4GfSLz0yE8HwIIvye5YW+1uasqefbeAqC1yQj8uesl4mwJL33MviuRKHBOUEcMmk8ga/EH0LeOENMpQH3Z76wn2T96B/34d5+P+7H1tUVW9pOOLB62zWW/bfZw/o9/+3b0mZhhG9ze3SgcXp62pw1YBCnm/RT9igHwjwej3uHzQEezOY4e2BW3CU2Tnd58RTKDDMAY4EBMHg3oHaUjqDCSDg9W4U2SQ9MoMgokZXOjobxM7D3u1Ffzk9Nt9s4UHrlLVp5PtfybAcN7g/PTdJLfToI8bgz9zUQ8rxZWTEo/r4X1jllSvmA39eZgJrBAWSaMZoViLKIHGBIyZ4NndlSytzIWJjt8Zgi08hclSfLe5Jsbm62k74JXijR8PvRb0AzY+GMyTw3M111DhlD5p3xHgwGvS1nDQC4Fzu+OLPogLqSIJW1sm3w4VF1JyCcIjrmYD7pzu2pgGFWoPCYNgtMOkiyXvxNO5zq1Op7fIjTmxVAveu5fOdDxu7X0aoNd5+Qj/cFj/zNAKvaV9ovQyJUe/2+d5n3BiGF/rms22AZNpwyVOzMYDBoAYnJnel02naWs9/Dz5MJxd4zn2ScCTIgQghEsOUef5OqPI9n+jrskYNZnoXvBCeAeyBhFhYW2o5o+DmX99r2gX8YS3CQs+e2obWqwATn26oTsFkmOGtpc7UvxpDGQQ5cbH+r/ZmVga3tbSTHPLRf1eb/smTFh7YPWgzu3XfM1FlYZxnQWew7gBshMYC3MtbD/pxac400JysihFYerwGZTrvTKA1CzNjPEk4rR2XXambHv3siXQvu4MYK7yDFC77u7+/bgVPOEHlti7Mo7oNZX4NfB2GMtcEzz7FBp38YcQdiNUvF+znLNBh0W/oyB/z4PnUunQECyFvm/GOWq4JNxokyrFp7amNZDaAZHsbQGSa/8/r6evb29lrJE31y0MA9eD/OlyBImbUeyLumzWNjHNEbO1fGhvGmpto/NGfXHGACAKbTaSu/qkFazWLxmfvhUrdqx5L+Zg81cKnv63/r7x/SDD5gWb1hw9+W0/h1PuNd93gsKPh1PvuxnyV9oqZe9zbQ/7aA0wGj7/eY9zNx9Jjr5oEBfV9DX8EPlDQBlpM05t0ndENQoOeuoHBGGv+bdNUcDmCwRSY12K3O60/H43E75JP1I/hd/DG2yKVD9ov832Vg2Ef7O643rsL/EngkeWOc8E/OrvE97sN3wAg83ztHTafT3nkb7OR2e3vby2BXfMbndU0kY+2SKOMN5obxMfH9GFv4GGLkt0FX3tX+tt7v0YEGk1xZcRpC4WuTfl2wwWQFuTXdZUac/6MQSXqLbT1YVk4MhBdfWVBRUK8bIGgxCHG5hEt7eD8ACfc1OHG2g6yF6zhrRobx9ZoSR/UwaNTuY3yoJTWb6qCJcfV6BI8rJTn+m7MjGE8CP48ffXeAVllkjIgZDAdeGDdnw6bTaTuB1Qcyec5tkGrGyEZyMBi0dK7vTRDmNQJmy5xJYn49dr6eAIMTv3FG1LISXLDbyObmZtbW1nJ3d5cXL17k8PCwnTlQAyWP1TwbPzs9So4cIDuIxQmzuNrO2DJqQMA8AcBXVlbaabUABJMaDqoXFroTf2cFF8mbu0R5zc3fRkOvIUuS2QveP7a/2fbrHOM6b49lUz/0+t+WZvzheTARhv0kC4DtuLq6apuP8F2XVvOZfT++Fj8L2GZ3xOTBrpE5qTjHBBoZfXxGPZgYf+q1ZwQB9gEQh/TRZJy3kN/Y2GgZEWwHgQ3PM0lpvIB/p880E6Gsg72/v29ly/WYAPrtDAbksbPWzlgwbmAik3uWe6+nsy3+VQOIjzb019s+aDG4lcbpLAsK11rhHGkm3ZaEZu8r2PBz/HwDaAIRg0JnLzAMPN+Lk90fvkO6swJ2oncMEcbLf7Oi824EDBgKngXLTR8M0Guph9kPDCiBkMeN7WBZ0MYORoA2+sQYWYHNfljxZy2ydl2yMybOlNSskAOluhbEwUytmXcpDOngWUGl18rMqtP0/Qk8k75j4d0w0nVenNUgAIQhY9ckvnt/350UjEMkeBiNRtnZ2cnu7m6WlpZydnaWb775Jn/913+dV69etfHhfXlHFkmPRqM8f/78sWr7rWs+BBKmMekyBMi6y+s45Gw0GrXdX1wXjaz4JGazbNNpt+7m+vq6x2gyL5aR+n+cWM1iVPn4m26VMXd7DDv3sX27Wp2vD81IGWz/XQ0yqx7iFyHHsJ34Hvw329kOBoNGYnmNn5sX7oMPnFnHZpGJrv4zyRs+CAxhshSSE3ziPtMHNrLxuhFXGSRppKCzBiY+TYyZDDR5QsUI38Nu81xXFLi6omIBPuddk36gYlIU28tie78bzwEDcT9jNe792xB4257/srZ9Vob0Q57762wfHGg44gUsuHzAgQdKhyK5NAXQCHAzUHBzBsIRKwJYwan33zegBbQSSfN3p+ScteE+TieavcaQuBxmeXk5o9Eoq6ur7VmAch/oU9ccuNTHGRK/h9Og19fXbZ9/FpK75IRrt7a2mrEgbct4A7ZmZUGStDp4GF5nmGo60wAMWUn6a23MmmCkaYBpZzoYGwcYBu5VPhhDjyPyYLlj7hgHp7m5j3cY8fi7wfLgrGDRmWdkg/M9YOhPTk7y9ddfJ0k7EMxb9LpvnJ68t7eXZ8+e5fnz59nZ2cnm5uZj1fZb1wgCbEQNBtilhMDKAbkDc+++ZcflNUZ1q+3kzcPdzB7WEjd/3yTIu5zZ3xbQ+7sGKOep/bKg/1e5/u+aPOBr0FkTOtbr4XCYi4uLZsvxzfYvLiUym87fwRhJegf8AYohMLBr+C+2KnfZUdJttY+fdhDh3ZvwZfabPpNpYWGhPRfykWwLZKaJNewa45R0h8Z6Yxf7ToIw+m4ip24Gw99cpWLS0bbTmRqT0thtnucAxFjCGSMHS/Txt629i1T4UILiN9E+aI1GfUGEyOdEOL2FwuGwbQBQ8Mlk0hZKVUeOQBLhG4jVusn2Qv9/WYYF0AA/6TMUvBOLqBypO5pG+b24uLL8ZBFcgpT0gxyvg/C2bYzZdNptY1v7TX8dGLCgGINLc3kQBmM8HreaVWdhbIQJYGBvCFLod13sSl8ZR+5pQ1CzWi5rQo64FhaK/jkg85gwTnVht+s8GTMHRhj+jY2Nll3yDlSWGcbb5Ve8jxmtmipfWVlpB2/Rr+vr61xdXeX4+LidAYJjYlcrxmdrayuffPJJPvvss+zv72d3d7dty+jSwHlszkTSPC9mFl2ql/R3kEOXsC01m+Ug2vJTCQvLlOXRNsUOzMxa8u017B/bb659lIm/2WbdMznmv2Nz8VXehAQb6s1X+J1TtV11gV2CfDR4tp8wKMb3GhNBPLq01zbr7Oysl/VIunUm6+vrzY/7jBfe9e7urvlO78yIDeRaApm6CNs2cHFxsWVnTNrwfj6EGZLTWWP74ZWVlTcyzdV+Yqvr79xnMpm0d/P6E+xx8tu1/ijpjwv/T2Zva/62d571/Xc9528yQHt0oOGFSnbiSX+xqlOX1SkjaCiYlSx5kzl2CQzXe2BQWJ4Hw1C3kBsMBi04MPPsBcZmC/w81/85E+HJcYmFAyEDF+9O4syJAyLArJlUj4XLjGDBzaLYQLnPSdrYENjxrjYuBm0wQdwHJocgAKU3a+JdIGyIHXRZmBkfngvg53OCjWoUvVDNAL+uE7FM2AB7EwGyPAQONqhev+NnO/jF8NNPAo3pdNoWdtdM3Kys29raWjY3N7O3t5fvfe97+e53v5utra1W/wpzRZnWvDZnr5gfnwLPWJsRc+21QUTdDcpOiEWZDmyYz5qptN2qLJwNdTXIvy1O7WP71Vt1+r+NrOq3qZlMcDOwT7oy7sFg0AC6S6TYtZHSaewCvsC21jgFm+wSLfwCvhUb5zWi9NGBjDO8Jurwmbe3t7m8vEzSPy3c+MQ+2EHI3d1dz04yZowBPixJwwdcaywGsQNp50wLv88KGOynwTSzdtryWgv3wToFRjIesN2ucz/PzTgn6dsXywDXmhybleF419j8bdisRwcadALhA8RaIHHQCIxLpiwITkcycDzDESpKBzj0QmCzEtzH6VT316VJ4/G4x8Jb0AlceAdqL2E4AIxsvQngwYg4RYqC+SwAl2kAfPzO1Od7vYffiSDA/bZBctbIgYaBP0zLrDInrnMplkE1QQAGx+VdFmKXs70tU+U589hWwGcA71SydxFz2RV/q8Fw0qXBmVtOb2XuptPpG3W2rhuln2x5ihFna1XenQWCAGSX4zAnPH9rayt7e3t5/vx5vvzyy+zs7DS5Q3YppcKJeS3QvLUaDCdd3fHa2lpubm5axidJL43uLSMdHKAzdQ0SGS+exfU05MP2ojbbJpzpb4sz+9h+fe1jgPG31941xib/8AvMjQkfl/gMh8OcnJzk7OwsCwsL2d3dzdraWrMj1nfsTNKVQHlHOwIWCE0a2+wap8D084PNd5k1PtDkSdLZzIrDnL2n/HlhYSEnJydtMxT8ntcqjsfjdgYR3wO7+X0dcPC+Sd4gfUz68U5c4wwQjQXylQBmbMAA/p35NgFVCdp5bX5P48WKdcEvyJvH3QHh28alYiQ+q9mQWTr3Ib7wg0qnuLmBXdItyrGyJekJDC9gxXcazcGD6+e9qJNn0cwMJP0Ao0a4zgawsJpnozw8txor+uQzPegLCumMgpkFv7+Fhfcy+AYkU2uepBeY8M6ML323ojnz4uyLS4rcH2cCMFKAaIKYWqfq7JbLxBxgeO7pm8eC5yMvNj42slY0p24d4ALAHdCiiIwZ5VI4AYypA1aCCi/2pxyOfsGk19232IHEAQHzaNDLeD59+jRffPFFPv/883znO9/J6upqC1xIkWNsSJ9jsM/Pz9+uoN/y5qwd83V6epqzs7PG+JHFSfrOhnF3dszslxk5y76ZO2fGKjismQqTAHw+iy362D62ZD5qpX9bWgVC/jxJ80EGv9Z5E2euDFhaWmqAGN+IvzOxZyKNjMD6+nrP9rhCgGd6NyzIPO6J/3fGxNjIxKUxA37DdpFn8e4LCw87Iiad/3aJk7+DDzRR6QDKGXlvjEIA4ozxrPLmusbCoBp/b3zpDDPNmJJrfpuaxzzpZ+aStBJ45nc6nbZybZ94n/SxMHPvsWd8q0+cpVf+/4eQKh+0va3LGfzSzlq483wvSY+NhhX2ADpCqww8z3RWwkGCy3mSvAGyzX6axXTJRd3fmb74x+yCMzIeAwwbpUguy6EvDsQIFCj5cpDDMwzWzZy7RKsyxVZA99OBk6NZxrlmSdxPvgeog83hb5aPWn5k5thBXNKdFu25dmmND2D0fTwGBp01sPD9mCuCBrIVNt4YSh+oxxjxNwy/544gEfYGmSWbsbq6mu9+97v5/d///Xz55ZfZ2tpqczmLcccRIUvn5+d58eJFTk5OHqu237rGmFS5tOGyfDrYRz5rySYyYSfthfzWETOIPNdz7zYLzHwEkB/b29pH2fjbaY8dZ3Ta5biVeDSQxz6cnp421t9gzf5qMpn0tpdn/YWz81zrc7uwfcYJlANdXl42Agx/xzUQJV6nBmkFQZh0a1Qpt7Lvo2+TyaQXdNRg4/LysmEXrqHSgTHjX/ycy7+MVcBF9/f3vVJW23ZsNhkVk5F1d0rb6mqzKzh+Gws/L21hoTusMUmPIKXShl3KIEQp53PARgWFSWAHfpXMfVcA4TH+Gws0AHAGREy2FzBb8PgBNNFcbuUIGRDrdRYAaNdSwka7H4A/BNmBg4MZA3Y/m+sHg0FLU8KyWjkqo+AsDZMGC837eVIAP4D1CpyXlpZ6ayI8jvQFpsRnR3Afz4UDCQCZsyAIqTMYVnSPF38n4LPBczAEOPZ80DxfDgbrvDjA4t1ck8o9+JexS5LRaJSNjY2WnWGsataF8qXh8GHNi0/idjBbFw0ybt7Fw9kL/ra4uNjO1Hj+/HnbNYqdyRgbUtpVXu/vH7bJZeevk5OTnJyctHrdeWwOoBk767uBv9P2lqcaHM/KTPA5Tq4SGf79MQazOrGP7WP72H5z7V1gx7paM+omMVkf5mCEgIEsqG0F/tVMMiTZcDhs2WiDtyRtIbn9Cs/id/wq5J19GiDfJVf23dzL6zFNhNmeYu+urq6ytrbWSqbsX2km9Xg3225jOGysy3scdNGfStC6LMu+PunKxmcRQAbE7vMseXA/56XVgGkwGLTgYjKZtPNgnLViM5/b29tcXFz0Ni+wDBoHM+fGqNaZWWNPn2o/39UeHWhYgOycDc79OeyAO2x20orA7w46qFP34HoQeMmaxqR5dxqnSLkHUbRZZ0qoUNpZbCgTPhgMertK1AVOTKIBDuyE/05fAbIYBBugCny5H2lLGHQbN8YLQOu1BwBwg3nmqQY3RMDOKPkaL3Cr821DXYXSZWYeHz/bAQwBrrNWTnkvLy9nfX29OQ/mlN9hcTY2NlrgQL99ErczQcyRy+m41tmT5MFQElT4wMf9/f08ffo0W1tbLYtyfHzc+oJD41BCngmz9vr161xcXPTWLsyTwawN/fIapMoowtTc3t62A63G43GzCdbpZHYQYFmp9orr32ZE3/fZx/axfWy/2fY2csBAyJhkVuCBPed6/AX+nSxCBWBswY3vSzpmmO9W0G+Abr9pcpbMvf1/kuZzDCx5hqsmTILxTsYQfKeugyBAcMDinSPpJzbV6ya9XrBWG/AscIozL+ADl5gbL+AL8MmzMhWzSEfLQf3bPNny6bQ7dHY4HPaIdfznxcVFFhcXs7m5mdFo1HAOhwNfXV0lyRsknglRZIrnDQaDdk4LpcrIose9rl16X3t0oGHHbVbcL5B0AlKjzerYZ0VPTvHxMsPhsCmggYPBnr8D6EdZUAQvAOe7SVoNHEpCnRsgmLpDKwm7UaAYKJIDApfMMFZEnoxPjSK9u44DKjO7dezoJ4GRgwY/C0NjVriWrxhkmxXhWTY2/ox3AQA6C8AYOtPBd5AL98U7hxFk1nUoLp3hWRsbGxmNRu0d2c6QAAVjymcEW+fn5zk7O8vZ2VmvnM+p2lpna/lZWVnJaDTK/v5+tra2srq6ms3NzbbGCNnwuh1SyOfn521uSHVz3cXFRdsKlyCo7kYyj40ADNllx6nBYNDWv+BUCZJJqdt+1MCw2qfkzd04ko+ZiY/tY/ttalWXZ2EM+y37O5NWLn26urpqOy5yLf5oFobgPCVsjdcR8j2CFhNrNbNrP2iW3pUPPmvMZCDP9oGDxgW1rNv4g/EhK0/QBfi0bcUP8Z5gLbCaz5ECEPNcr8Pg/cFIJptrZtrzXLPRs/CLcem8N+SMAKK+K5m0w8PDNnYuu/MxEpUUR2bW1tayvLyc4+PjXhDuLBufUarlHVof0z5o1ynX8vnzGlh4gi0IDjYQNBTLaTXu50VElYlwJOxSocqU1/MHrJjcj8W3/A4wZVB5Tw96rTN0oGXGgWdwf1gPFM/GcFawNZ1O2zoDswAYHW+/Z0DleyFYRMg1CLQAOjBByX2Ssg2RU6Oz5pr6eTPWzJX767ImM0AOUusCMoIJ1ytiqL1ex3NwfX2ds7Oz9v+Li4tW1+jyMYIQ5pdFcARgZC3YMerZs2fZ29tr4+GMQ11P5NPICXLYtYqgrB7kh8yMRqO53to2Sa8EAEbs5OSkGT2XnjGPyGstsbRzquRH0t8cgudZRpOPuwR9bB/bvDUHAM5coNuQgmb0kz4wx9fhHynL9po/r6E0+eT7wxBXv8/9vPiWrc65jgAEJtnBhSspeFdwRy0lxsf4u/iqyv7X//N87g9x57EAN3jdHLbauOTm5qaH2Tw3Ltvic8aW+amBXw0qqgzQTDjPunae2qzsi4G+gzDkkqwW88D3wWsmro1ZkXGCar7rCqVK1HknNff3fe3RgYajTxTQA+Iolc76c0fuValQOhQpSY898GnBNauCQALIqLl39gAQVxeFOfsCWPUe2DX69uIa+mDjxSQm3bacLBjzvXk+TEWdNINqp0S9xRvjbmPoDAP3s3HyQubKhlS2h3EjAPQp6FxjJgPDhAEx4+F387xzj6QLFl2KZUPGeNlpmInhPnd3d81xON1oRWJsyWIgSz511UZ3MHhIJ5K5+OKLL9o2tMhOZZ0cMFrhmd/r6+ucnp7m9PS0HeZXTwgnKzMcDls6kzUd89qcLq/lTR5vdDpJc9hcx7+e27dlKmZ95nu875q3/f1j+9g+tt9Mq0CWZmDr35P0zsig2sA7Wk4mD3XvlE953RhEJH4Aso3F2HXbffqSpN2X7WgN3K6vr1s5KIEBpBb+0zbSPtXnciX9jTEg0yqp6jFylYGxBjiNH64jK89iY+8U6WCKdyMwcamr588Eksm5WRmK+rtbBd+zrp0X++33rgG0P+daf+459rVgDUhZMBz/N46s5ek1qE66IORDx/SDTwZ3WsUvjKB4IJyhcF21gxEPKKCc+3hL1um0S+O5dARwyv2Jzvhe0p354edXg1SDCv5FoXxSMQu1aTW74f/TD++YZaaFsXQE6uifvvN3szYOeszE0Dz2fNeZkyQ9BqLu6uXxcUq3GlQbz1kgzgbI32McfA+M6SyWKulviUyZlccCh0Bw6fvwr8vqAL4OVlZWVrK7u5v19fV21sXOzk6ePXuWnZ2d7OzsJElOT09zdHSUy8vLXpBEPx00IH9m2Znr8/PznJ6e9hYB1lPAWYfDtojz2iojUzOVrhWmjMp2ohIbdqbvckZ857FtXpzTx/ax/V1v1Y7Uchv8ikEW/hxf67Inl/NwPT4JPEAGfXl5uZFO9j22916HRnlRkgb2ANqckZF0mfikI1XAHgsLC20rXZOH9JmAA98K6Pe4YFM522MW5mBcvCiboIh1ADzX2eQKjpOH8jLGpJJ+Dgg9p/73ffPvZ80KVL5t7X3E2NvIM/+/lgi/zc85QOF7k8mklZGDLZJkc3MzGxsbLSsHJk3S5A+5+JD2QSeDV1bW4BOh9v9ndcYK7XSed1JicBByA4la/pT0ozsU1OU4fM+ZgvqMWYINePU1Zl15VwNIL8Y2q8GPx4f3d7YGxoQfl1y5tMn9pxnMW4EdoPB3lxbx3o5k+V6dL6dsMcAYVpgeBzXcG3aIZ2MIa7rYmSMbapeo1eDBETfPpB7WGRUHT+wWQv9Ya0FA8fz582xsbCRJtre3W+DBXE2n07Z+gnvWBXQsyMLJJekFCpPJJGdnZzk5OWnOwjLjbIuzVjUrNE+tBqCeF9sWL3r336yryZvlUb6+MkJv68esv5l1q2zRx/axfRtbJZl+W1vFGQZny8vLzb46y05g4aBkZWWlHXpXWV7b8VlnRvBdBzf+jktc7ePpizFJkqytrWVlZaVHqkCUOaMxa2cpE6+2d/ahNIIM3tFBT60CwfYZz1VsAA7Y3NzM1dVVy3gwbnWuko6YpT+2tZZh+8PHNJO37yOfflPtXX6nBsjvu8+7ghD/H2xSd3Zl+2Nk0OtqrRvgj5ubm3bel7HxY9oHLQY3WOVBVgzX1ZtJh+0FlM66twdkOu0vsDZwsOJY4OmTwfVwOGysqGso/XeXYwFwYFZJod7f3+fs7KwXINAfnm0jAFhmIhwAVMPBOFmxfS2f1YXldUG1++NSKMbE6yGc1fEi5RosOqrmmfTJ2/W5JM2KzfvbyHuhkpWF/poJsmwwh951w+NI2ZOzQ/Szrj0ZDodti+SVlZXs7Ozk6dOnef78eTY3N5vcsXaDUqX7+/vGOrELlIMw6h3ZLpCMBXLFbg/MAxkYs1DOYDmgcLZsnhtyUGXHc2mdn8V08f0aJNegoGZP/PvbWDDfz+1tjNvH9rF9bH+7zbpXyTaXNbFI2ySFd9NZWVnJ6upq1tfX225RSefLwSEEBkl6/j5JO/Pi6urqDZ+UdDv0GIP4bCeAnbEMPoM1D5Qt8WzjDAKWpCujdqk22IFAgmfV7cJN5NaqhzomzIHJHO5JiZqJWwdNlWy1LzDmYW5ntXeBdZr7/jay6TfRPGZv+7t/n0UezBqfWVmMSp7bp9ZNBcBIFxcX7UDgSgInaeXoXhP9mPZBi8H51w4dBsA7HNDMNPNClSWo7H7S35e5ZjQAbUl6RsA7NnEPonUU3GDdA+kaQgS/1kC6trxmY2zIrCz0zT82ZnV8AMGMc83OzDKwMOG+R+2jMwqORB1A+HmMg1kZG8q6cNrPNXvjrNasuXUw4fHBIbhen/n3+hj66WDELAbfw5mwzmFjY6Mt5N7Z2cnu7m5Lo19eXuby8rJXSuXFgQRD9JuA4ezsLOfn57m4uGhsvJ2g18jgCBkLaoAJWtjb3fLAu9YzZuatwZrMMpomLZI3d6RLujpl9GFWit92oMqHGU33oT67soC+turyx/axfWx/+60y4El6+gsmWF9f752FRABByTW2vlYjQJB5HaTXKJAVh4w04enMA5gD/8r3sG1ed+F1nPhanu2stnEBz+UQPvDRaDTqEaCMj+0W40BZuoMqH2mA72HMqBhgDuzX+B42srLfLk/zHNqn+/O3ZTxmyUMl4r6NNvqxQQb/vovk8t/AXnX8ZuEw+0+aM2AOShwYjsfjthtkLRd/V/sgxFKFiRezQNWIlRdN+gvEq2N3yZSFl0iXyJ6B8L8GotzL6b4KjlmYxcCxawTvxL35u/9PvxwgefLNVPhz3q0GGAaSFeig3DU7YfbBAZqzFW8rMzLDX4XYwQdMCMLo9+GH7WIZE/qFAXTQVL/vdJ0X37mcycpAaZYXBfMsNg6AKXLWYHl5OZubm9nd3c3+/n42Njayvb3ddiOjL+fn5+05w+Gw7WZFEOA6WuZkPH447Ojw8DAHBwdtpyqXfvEu9/f37fDAWRk5B201uK7gd55LpyorVgPMGojPyizUINklkU7/05BV6x6yMRj09xXnep5Vn+171jaLaXvfdz62j+3X1SrDzGe/7c1ZbPuZyWSSy8vL3k53ZBDYPhv7b2yATWEs+TvEJjsVrq6utnsSkEDQ8b269oENYmzXTDQlHUHrrWPpPz6ffuKzFhcXW2kvmXaf6UHpOwEVOydSrnV2dta27a8kS9IHqF4Ebh9Pn5FDbDGbnBibmEibJbe/Llv6ruzBt7n5/d/3rjUo8Xj6XlxTyX3Lv6+rz3VQbmz9mPZBGQ0/HGUgejYgQsnrdw2Y/TeDLf7mbU5hFgBgsBMILQNgIO7aQ1KPBhPJQxoIZXWA4354h62kY1kdfPBMPoPlqIGRx8FjwPX0zSVp/M1j7rFyEMVaAAIz71bFPJn1rUDWaWcHOBgV5ss7UDm49HoIB2w0MxoOtOoccy3v5zSrvwfD4u8QuOBIuIbdmyhv4trKDnGmA7JtuXcpHDtwnJyc5OzsrC2mch99XgdlWA42rQOMy2Aw6O3iYaPNd+d5MXhlnExSOPNgXTRpwHeqTpoxrE7KNsYZDtac1XLOWQaa9ssCt78LgO9j+823eQRVH9oqGQd4dllt0jG5Jiz5znj8cEgqWXvAPhuBVLbXhAjfp+yVz7Hr+F3suMvAuReAzYQmwczKykpvgW7FTrZnBBDT6bQdjkewwHfwqzxnOBxmdXU1CwsL7Rl1d0jjFXCT++EF6fZn4J8kvU1zCHogdir2MGPuOaZPNfjxd/27m+fsV20fEuT8skH+23S3Bkv+v9ekcI9KtNVWqz4qtq841eRz8uDDwVWPLeV+dKDhMhYL1iwGGgXwgAN+uUfNQFix+Tsv4xd2BA2QrUJL4xqMkMu62OfazABKWUs2vG6A/1cD40XjZvQNit7G4HqhOc8ZDAaNcXD6led7LQlGzIAUI2cQxnjzfYyOT2AnQKOPPB+hssGxkDtwSfp7W/N+fjfGl/c3y+PxwADxbPpKVsrpb+SUAMOla5PJpLemgnIqB3gEL9yfoAunBCPE2MKYwUZVpuDm5qaVU2GsHaAZRFdlpz/MF5moeWVoaG/rv/Wk6rLluv7w93p9vTfyb0BQ+1G/Vx2aa1NnlYXOem79/WPA8bH9bbTfZjmbxXyj47V+vPoYriXDCTgHAF9dXeXm5iaj0Sij0ajZccDV2tpaVldXc3V1lePj40yn06ysrDTfkCRXV1fNhznwmU6nbxwQTDbCpUTObOADOCvBQQvvi6+6vr5uJd+uRqBfBDODwUMJGCW6kGpeK+LnGbvVrDAEJDaRayrRaF9W7e4sAvpdc/8+e+/PbXd/Wb/5Ibb7V/HN7/KNb7s38u178FP9jz9HNk3OcR/jfLAznyNLXkv6mPboQMPA247ek45yo9gVaBosO2XG9XwHhUDBzXja0ZvxNrBPuowIfzMQZgAdQTvocVDi7ewmk/72s8mbOzj5Pd1mASMvOKtrKqzcPoq+lqS5dKsGRb6GvzFHa2trb7DqAHlKh9bX19v2eyyg83vz3Mo4MHbMq8fXay8IJtlxg767T854rK+vNxmEIWHBG9diJC1TfM519/f3LZ3rmlL650Dj5uYml5eXubi4aHPgYBpDy3tyP7Id5+fnPaV2iVlNO1eD4mf5XeYZRCBfFexbhgzsa+DlYLYaxvexV3b4/t6sVoMIB3mPdSa1fx/bx/ax/Xra2xhYQHPVPXwtNgVyCDINH5M82Jnb29u2G+D5+XkrAWKBtvuB7XK1gPtAkOKSKvsoGqVSrs7A3niLfcA89zeh5mw7dsvkMBiFYwDqulXerZJ93Ne+tFYfeHyNzfz3WQTPLAZ+ViDxLsD9rmCjysmHtgrE32XLGa/HBE21VTl+HxE26zvu36z3rtidvpo09nPAOX7erGe8rz060CBN6MAABUOoptNpLyqepRQGFBZCC6YVpIIwl1eYoTDgo2+egJpGTR4MwOrqam9BuQfQ60PMPNd+mFG3MlHO5KBo1kLeGjDAttB/s/q8C0B7Mpn01hA4uDKj4kXUgGGv1fDzMYqrq6s9EF5TabyrmaQaVCIrTrnOMlZJPwgh44IcMW68J3LmRXfcm3EnFU2fanDI9z2uLrlL0ow2pU88y3NtBoeyJtfOInfOhHjOHXT4nJA6roy1x2zeWmVBako86UiCys5UYsCBya/qRGZ9TrN94f+VVXtX+xhkfGx/k22W0/9l9WEeWiVokg6sGhjb5+JzvPNf0um/qycg1uwvIJ2Szj7Zp06n00aiTKddJpw+8jnXmjC1nzSW4F3YsdClV14D6UXZJhkhVN0X+8nb29u2bpDnOSOysLDQey7yRJ9NulYCmrFP0gu6qlzWz98lt/YFfGdWq4Tur8P+1vu8Lwj4ZZ8L/puV+Zl13/rcWUHE2+7hcnhfVwMOZ02se7Oe/7b2QWs0AN6zIjv/3eDW2QZHrgg31xtE8X0De9/DJy8bRFoRAZELCwut/MeZBJ7NgAMEDezopxXfRsugB2NiJoDPeI630uX/njx+9/g64ABwW7m9haqFoQL6WYGOGYv6L+NSBdDb4yXdTlnef5xWgxKDMwyr197YadhQMTewQN6IwPf3rk4YUpfHeSy9iNxySIBAYEEQh6H27iUup7MzckDhPjlwNNNWSwc9R1zrWtp5BhAE8NWhmB2rGTn29Mbx4ei8wcOsxj2tv/7XffrQd/jYPrbfdPtQZ//b0sz2J32Qap+DX7m/f9iWHJtMo8y2ru9iUbTtdB1jsuL4FxNvJor4nTJa1jcaCFIpwN8qQIf8w3aSifc7u8rBOIj3MkvO370WkHfl/iygx3e5qiPpymgq8VNtsasoHhMYuPldZmHOWd+p170LlNfnv+u+XGd5mPW9Cto/pNUAwM98X/+Nn2i1Wod7uO++99vey88Fe5lYf0x7dKABAADAAtb4m0ESgM3ACoWDrQc8OgPhLIEVwCVVVaBtVGZFzV4lX9dEcC0BEvX+Xv/gjILHwu+S9Hd88sTRV7PhsBmeWE4V9Vh7sTLPYL0E9ZhkNzwms7JMBvkIZWXKHRjRuEeNbq3IZmkYT8Zg1lgPBoN2joWNG/eynPGvF3Szw4cB/HQ6benipaWl3r0oteJalIXzMZgTjCeGlrIsG3UHdA44fIiN5ckG3Mbe4zqddul+r2ehb6yP4Xu/rCH7NjT2WXfmxmSD2RTGg78h62wDaWNne1AZz79rQOxj+7vXrDO/7TLPO76LYa6sbsULBuNJf5MU7K8BvMEc33FQgN3nX5NF+CL8vs+UMOlSy2r5vwG+fYefZbziYMfVHSYi+b7xmQkc+1+Tp/TDPt//9/zUDXrq/HjOjDmM+5zFnpX9ftv8zwLTb2s1OHnX9XUsP6Q9Juip19XrnZ16W/O9a3+rr6xzMyuAR45cPgfG+rUvBq+lOIAsPvOpgnTMe+z6egPa+qJWZgbHjIMzBH6e2yxhqKU6NhAAGINis6qeJO5hA+TACGU1G2JwOgvUV6aF/gKKa/nScPhwgjh/q+c2+N3ZxcIGyMLmAM+Oyv1zSZCDQb5fDZtZDIy1DdFw2JU32WDyvq5FBWh7hykHVGZVZkXrBFr0CQWxjDhDxD09jgbGNroeD8sM72R58Fj7OveTd3JQbZ2zDM1ru7y87Mmb5bDqvcvwrq+v20GIHlPu41YNpzOjFZzMIife1apT+tg+tg9pjwU/j72OVgmdCuTe9Yx5a2T1Z4GiJM3v2IYbGAOOfPYXdsIlRv4e4JvdoGx/6rkRDkAcsPjMMfvLGnQkXRaW7xvwA/r4m69N3tzd0QGQd6XyhjH2nQRJjA/b2Ced72bcvJunZc6E5axMB63a3YrtapulD/WzXzbIdtDk/nHPd9n+x+jrY/tVn2kf5iCRa2YFE/6+cbbJTQd0xm21vw5KGZ8PtR0flNHwC/klDP5dblMFz0GFgVodILMMBl0GIrNYSyutlS3pdjtAiA12eD8HHK57dIBRt9DjeV5cVg8DSh6MEadKW/EJSjA8PjV0PO6feI0B4PsOcOq781kNiGrJWhMElUJhPFxHahmwEcXge05qFqgCawyIU8Xcg4XYXhw/nU5bPSnjtLa2lt3d3V7gVjNhdawd1I1Go7ZbyOnpaS4vL3tyW1kBKxrv7DnmHb1eo9ba8h5V2S2rlZWyXLnUbJ7bLCLA5Yxcw7y6Ttjf5/dZ953l8Hz/Whb42OZ7f2wf2y/TPiSgfUyzDXns8+dZjisWqfaklj7j223DXVWAzfZn0+m0t3g76e+6AxlGxgFf7rMreI7Bc138vbS09Mb5G97Rx+SSz7kwmYmvdkBElgN/ybh5rW0FrowP70Pf+RuNvtDcT/6tVST4sIoBK45LutJvz8ssGZ8lx1Wm3ybns0imWUHrLD30ezF2+GfLj79bA95ZJLuvndXwhbP+blxcccys4MEyWYngel3SJ96TfLDvfHSg4YXDdIDOOro2i2AhQmg8wBZ4Ou7PnMVwfX7NFliZZkWgleWfFRE6urPxMoBPOpBco24rhAMlN8bJNf+8M/9aEJeXl9/YfnU4HLbSEfdrVgBmoM6zaqBHs+A4azCdTnv7grt8zQYdY2QD5TSrn+s5HwwGbZs9r20wq+J1IQbcBG7+3v19d7Af/eXZMDMbGxtt162rq6u2/awZKbNMb8ts2eG5rzZS1gvGhpIq+odxZ77NYtUMng+fmsfmYMkB8yxGBv2uQd8scJH0GaCkK0F0cJ10cuB5eV8K2PpTncjH9rF9SHuf/FSH/7brq17Uz2Y169i8Bhpm0A2uXCFgEqeuxTALb/9vjGL/hc9JuiCmZtftB/k+fssEVdLf8AOfVX3mrG32r66ucnt72/yeMZP/72fZXnEfssrGFWCZCvwZEwCuSTCaAzya/Z0J41lrRC2LldStxHV9puVglp5U0O/3Rpbc5xo0zQoGqg+o7zFLB41Ra5tFkNX3dbDrf3knrvd19WwU+sF3rCe1SqLqVu37h/q/D9relgd755+kv5Vs0oGJWv5hdsEMLy9iYMxntX6+7rozGPRr8ofDYW83KjPNtArseXYFLjYGDgwMYGAiatDFuQdOed7f37dzF+i306FJegcgmlkZDB4WlLEY1sDb1xMsmSWvwN7/+p0Yby+etvIxr6SumUdH9DbSPrAo6XbHcBkUIBt2h4DNssS9+Tvje3l52Vscz7oVM1feA3o6fVgLc3h4mJOTkx4ItUwxl4wt84zsWZ74XnUqyJTZLssk427D68+rLDrw/mWY+G9LM6tWdY55RlZryRmBW5KeXlVnNMtgW96Rj6oX72qzHNjHYONj+5to1Rc+9jv1+rcBHv42z/JrgD8rWEi6d8UXmdgDK/B3bLUrH/CxZqqT9BZL45MMZE3G1Sy455Z72UeaLEy6Q3S5NzgAe4a/5+/0FVvqtYwm8CphaJ9ye3vb66Ntp/1b/cyy5cOC7U8dzPGdGiTY1xm8V3n1ONXvv80PmOBmjBy40vhbzaTzt9qPWVjKbRaJ5vvVtZm1/7PsQP3MgQHvWmXHPs82pgYwfO+xazDe1x4daFhgLMi0mgrimlq/74ZiGFg4WHA2wWCD7wJOUW4CHwbTAl5LnhDm6+vrZkxQdmccmBz64RImDBH7b1NDjiGq6ysYG/q8vr7eMwa8txlYjCcGhu3nuAfv5LpJSpLoI+PhFGg1er6+Bn619tMBjQ2GDbqBPv2snzEv/CALvpZ+8E5kAAhSvf6C8WGMfaK6DbiNjOt5nSa0otIfGwPPGdk+moNlxsZgl7pX11XX8a5z4HvYwM5bq0671ig7kDCBgN5x3WAwaFtTj8fjJgtvM7JuDjaqrPm7NZiogd/H9rF9aHssuP8QMsFB+LsAySw2dB6bmX+/L34F/eZvs9Z0YWeNQYwZ/ANumeWbuL+vc5aWxvoIg3AyLN5y1mXGJnPt/2i8A/fm+7wDOyAmaXbSeMJBlBei8x4GwC7hhVh1kMA78q+JHMq5PF/vAuWeI36vtpf/17mzL/EcGfcZZPtZ+FauJaPiPrnVwOR95JOxq/1T0i81q/I369l1XAgifW9jvZqFqcSc36mS1X6/t73T+9oHZTScCajRbdLVcRkkcb2DkAqeKiAwQHe5kVlvBsWBCenKOogoBpNgRXbkfXFx0esv7+YV9hYoA3gLiaNh1hXQR9dYwkwY9PK7x5dyHIxADdocmXshtg0Fv5N2dT9rJqnOE7/XDFVVXq6zkekJm8qDquz4WoPMembF/f19y+wA8h1oWaF4V7IcDhI9f1yPrJlVoN8GuO6rjeus8sDKiNjYGNgm3Rkn1Vl6rB/Dvn+bm+cg6TNHs5y8F1byPXQq6S+W9Bx6jmYxbjTbkhosvqvN8xx8bPPV3idrtiGzvltBG21eM6PVP+FzKP1xNtzAHF/h9RMGa/gIj6fP5zIBBLBzWZDBHXYO3+TSKJceYfsuLy97xJ2zLvhD/IHfy/ck+2F8RYDiigV/x8ED/gpsgS91FYUJS5NnST8Q846A9nWVuDXOehuYnlWKzb3fVvpkoDxL/mdhHBN7vLfHk2afgc+pf/N9+LwGWX53BwfGRP7dOMDzQ5tFkNqP+m8eE/5fyfhZC/3rfD2W9Hx0oAEgNwD24PkFrKwoxixg5XSOGRkz4w4meI6NJ0wmQN2AHaVhD34yDTx/dXW1N8mOhFFOR8uOyv29WkfuiN/MCv/nHgbLAHnvwOS9vM2+0n/vElADI9h8zxXP4rs1C+Q1EGYMKoC24nncamBTgwoHoD7bxIGajYiZIf7PtrOMq9dqeL4Y6+pILLMEMxyI5HGkD34fvu/F35aBWUyCZcZGqD7HQQmBNjLBTzUs89hs6B3IJZ0BJkNYWUuncQeD7nAprqlZMMuDx5958TqgGsBVp1edxKxrPraP7TfRqkwaGNS/c808t5rZNpBPuoPqKpByM0DyWHkXquRNf2AsAhDDL1By7GDCtsiEiAMJYwj7gYWFhbaluwG9QTvvYt9rzMKOkxcXF+35zvDjr+raVwcD9s0eY/pSs8fvI8XeFhjzXD+v7r7lsahA3P2uROasZ7gf1Uf43jUgmiVTlcyqc+95sz7a99W+Vb2tn80KkOr7zArq+DvjXEu5IW1nBWx1Dh9LVnzQgX3+1+lCK5IjbV6kThjRlQGhg5I6+GYOYBK4X420GECfYcB9WRSGwHrhsL/ngMjv68Eej8cN4PK+/rsNPvedTqftxFGyMwbBfM/bAtczPQxeud7PWF1dbSDe80CZGIEWc+DnGswCdhlnB3g808Eg60j4YcGtgznv1kQfreDIy/X1dc7Pz5szsXHDibBDh5uDzBoIMn88e2lpKbe3t7m6umrnb3j7wVqbiGzbUVSnblnmPX1NlQuuQzadPrdOeAzoy7w22weMXGWlWN9kg1uZypopQpZtCyyPtWQreVPnPCcOVD62j+3X3WaB31/1fsls9nZWMwEzb82AzERU0vkxt1lg08RZLX2qwNDAy4A6SfOnHCrKZ9y3lirjW71Yexbx6kClLtImYPDhscYIVEDwe5Kej3FGBT/jTAfEX8282G+ZQDOWmGVDbY/fBqD9/jVg8PwZh3DPWkExC+TPkgUC1ipD9Tp+n+VDqkzxu3HnwsJCTw78HWeKTCRbXmqw4MCilvi7z7UihWd4LN1Xk9x1nhy8GQs/1oZ80BoNHLIjzncJg4XWkSpCzAB48bYHyIuc/PdqaBgk10j6VGgfMkIfGEgv7PJkGMw7LVmNgg2FAyFSkf58Mpm0NRgeNwMug3z30cyCJ9iCtLy83LZtpUyK51kYb25ucnl52Ram17Ub9Pf6+rqXCuanGmoCCs89401AAevEvPp9mT9YF8rKDEZtMLwzk1kh+u9gzApD9mNtbS3D4bAFXtNpt7MWz3EdLcEo72X5cOmOGR/PkQPw2jcb8prVciBvQ2PZnLdm8G/2hznifWspFGNPppGaYdsCy1N9JvLqHcXspE1yOMCrczev4Oxj+3a0SopVO/7Lytf7gopZz5jXIBq7aSYWMo0xxRYDxisoq+suTBbZRiV5w0YYgxA04L/IapjAdFkz9sf3N4CtdrESMz7PyRlvSE8y83zfto93xn96h0RsbAW+/E7/K3lM3+nzrLURJhNnZTk8HhU32b77fhV0Vyzmcav3qYEJ71NP5Hb/LAc1KJnVVwc9FS/5M1oF8TX4Mcb2MxyUIu8m72pm3/2dFUy9KxD0eH4oFnl0oGGn68l2JG1gSMdRQpTDQmUBpNnJMykeSAcTZtSHw2H+zb/5Nzk/P89/+k//qfX5X/7Lf5kvv/yyB+p/+MMf5j//5//cTn5eXFzMn//5n+ef/tN/mu9///v5r//1v2ZxcTH//J//83z55Zf5D//hP+Szzz7Ln/3Zn+U//sf/mL//9/9+Li4u8ld/9Vf5F//iX+Tf/bt/1zuV+l//63+d//W//ld+8IMf9Cb9iy++yL/6V/8qP/7xj/Pv//2/7707wRBlI2bGAfEEUI56vZ7l/v4+V1dXvcVhBlAbGxvZ2Nhowcvl5WXOz897O19Mp9NcXl7m4uKiV3pUATzzTTDhv7PHuOfw6uoql5eXub6+bj8G6MiJA1ICEwdajFV1KJVxqeVs3qL3/Py8/V53NWO+zKAzlg5qky5tDOPl8rHKllXn4fvzr0uBuH/SGZ9ZRmDeGu/O/Cb9QKM6DQfpjP3/197ZtFh2VW983arueunudN7URBTzJ4NMJDEkJgaFgBlJUJSooH4EEfwEggMHcSY4cigOgs6ciGTiCzhJCAgqBDUJggoJ3dCprq66t7qq7n/Q/M79nad2dVfFTsyNe0FRVfees8/ea6+91rNe9j42KD5FJZVx1SJqlAECG0vmyGvVOqpTp9tFLWf13XZePygOBpSOmvW97ZBBZkZ9s1yE91lYz/v6VrAT27O2tjbsY7h+/XptbGwMwbW0m87YEtQwEMSmo4+qxtkEA73d3d2BJzhZBwcHoxIu7D97Gx2kc/DK2Aoe2CnIbAW8yM/9XeryxH6Mzfe3QC7zAr8yAGknxjbS0Xs7nQ6at/rnubfjR7vptGRQEPniWtt+y/DNwL2dq5TvqhoFPo2l7Sy5AoNne86RfTJfviYrXtwP+OnyxFvRqRwNJtebky0kTCLC4MFwfavWPBdZK4WZJzO43aobAvvMM8/Um2++WS+++OLAoIsXL9Z9991XjzzySO3s7NRrr71WZ8+eHY5CJfvw1a9+tR577LF6/PHH65VXXqnt7e36xz/+Ud/5znfq6tWr9dnPfrZ2dnbqypUr9fTTT9e///3vunTpUn3ta1+rF198sa5evTqkJ5955pl666236pVXXhmN5ctf/nJ9/etfrytXrtTLL79cb7zxxgDuUSCZOcl3aTjSyrgNWr2IfD2nTrz99tu1trZW58+fr7vuuqsmkxslQxcuXBgEbmtrq7a3t4cskecdvq+vrw8/TrlOp9PBUdna2hoyRnt7e4NsEB3xca2thca1jrLwmSPTdkQtd47kAPpns1nt7u6OlAXfpVxZaXBvyr4VTMuRZo6qFg4S42NdMW/ILErVMuGIhTdTLxs5A2XDYAOeSjbrnqsWitilbJaVlAGe5/nzqXXeq2TDc1wEq9MHh7wmc32+W88ytaK8N7vX17YCE/7O+sj6bJnl2WVHgPjUI84grK2tjWrOvbexarFfsqqO2IMEn1Xjo8wJVHLPZDIZgoEbGxuD3UOnO7Bmm2X9RJ/8DP5m3HZUUue5XMoAMstRIXRs1aLsq2oBhrG36N8sA7KcWUY9Px4j92DLcn9k7qX0PGXZvZ/bsiW0C2iGb95b4zngx3skk7g+eZm2o1VVAbXWo22VeeUx5P92FM0nBzGdJMg2E3Nwjfc9O1hqu3uao29PjVjyGFQEz8rLTHEEsWpchpQL2MxjkF4k6UERqbfH78jmfD6vH//4x7W+vl7PP/98Xb58uX74wx+Onre3t1f33Xdf/d///V/9/ve/r0ceeaQ+8YlP1J///Of6+9//Xr/61a/qG9/4Rq2urtYPfvCD4QhbIvcGuUQq+J/jN/f29urChQv1+OOP18svv1wf//jH64knnqjXX399iDK4z84K5BGfjNtC4zIiC7oVHFkHeLm9vV3nz5+vO++8szY3N4cUMHNz/vz5YVH65Ar64YU4nU7r2rVrdeXKldre3q5r167Vzs7OkF0BRONh56JjXJ5DLz5HoZjfqrGH7RpUR4O4zgCTz5Bjj8XOsA00/KEP7t98Ph/my883gHCWw5Ee78Ww02JepKJcdrJjmaApFTfkUzOqjp7alQbDn9kQ8nxn0ayMM5qZOqrTB488t+/2PLfaP+kz0XuW4+OucUTVsk/Qju+WkRJY5VrNCDXAyJlwZyisU1qBDOMbAnPYC5wKSrTn88XexfPnzw/OhgO0DowcHi5OhrLecrB2ZWXxdm5XdDjzAcZy4JL7AY38TZvmQQa0sIvwBbmxk+aSVuvtbNff8ZvsQs5JYkBjRzuQEOPPsm73jd/ofPhjhyblydjDDhXjY55NHmc6svy2rDIeOyZ20CyPSbRpPOV+r6zcOAiAKom0n/QRXIcddNutz5K/t710ipp5Nv0CfAzsW56VHQpHjLOe2lFnK0XaMQi0R+33R/AsUm14zRZkwJxrGx999NFaW1urF154oe6///760pe+VH/6059qZ2enfvvb39a3vvWt+utf/1ovvfTS8FwvPp5pofeJS+vr6/XQQw/VRz/60frZz35WTz/9dH3hC1+o3/zmN7WzszOciuXoCErDUW34XVWDY8MP9xK98Vuxz549W5ubm6PNyXy2tbVVly9fHjkWgOYEfTxzZ2dnUJ67u7u1tbVVW1tbdeXKldrZ2Tlyig/8d9agqkbKG2WL8+fSsYywkCXxHPgY1ASqrdrcqgXgdblMllFZ4acxc58sp5YP9wfF4Zcy8rmjLPSL08MyqrKsAKHq6B4NlKL5mHW+6XAlP4m4+T04VtZu15lAZxvtBKeDm/936vRe0nGyaH2UACcd5ON01jIS+tIBC7/M1YANfOAgkINGVeNSmKrxm8dtx6y3cBT4zdH1xh6APNtscBPBOgcEXboEoR85vCSdDZflEOjkO0er07Gy03r27NkhgGq85Od4zyRH7bYO7HAmyPq5qkZYyTqe+62bGROHvtgZhIwZmTeu8fwx99hX+sy1CdadyWDuuc6OoDM1mW33+vPzHCQwFuM7B04zm2WM7c/gp8eCY2t7lpjMATb65D6m42gnxfw4CZ3qeFtHhv1j5iY48yBTCBHmjEQ60utFYcXiz51NgQEbGxsjB8VtedPYZDKpJ598su699976/ve/Xx/5yEfqzjvvrLvuuqsuX75czz77bO3t7dWDDz5Yn/vc5+p3v/vdCBzP5zei2evr6zWdTofn0i/G+cQTT9Sdd95Z3/72t+vChQu1urpaDz30UL3xxhujhQpwTmXiCHymUg2mAPk+Fm42m9X29vawnwJncWtra+R0OPuxvr4+Si+6T3jz0+m0rl69OjgX9Jd7AHPIgUuHULiTyWToD/OaP4yRPmRfyKqwqZ/PkUvLTC5aDIcXfjrNlkk7ADbkLSWBEbGya0VO/H/Kdq4Pj21ZyesfObHBulmgwfJvPhhUWGacqUhnxkENKKNxqXw7fTDJ8/vfXF8tfZCGv+poSURSRpl9b66dZSMfZw5R8tPCB/CohTuMWwzIE8tkJtllxQ5ueq6widYp4A+/iI/vMjDi7EL2P52BViaXYKHLsdG3tm9+oZ5xjfVxZtrRp56DzHAkbgPbOIqfNtbYkf6YP/7beDDXjU/k8hpxMNHPN17w59kP2xA7ROmwICPun204/EIG6Cd7hZAvvz8lM0vOxhvLcE3qBvPQjozlggocMAvPNw/M05PaxVO/sM8Ms6BDeIOuhwNU4gVm9sKM8oJgUPZEvUgSAFdV3XPPPfXYY4/V6upqvfXWW/Wvf/1rBHZhKMy7++676+GHH65f/vKX9be//a3uvffeeu655+qTn/xk7e3t1TPPPFM/+tGP6jOf+Ux985vfrD/+8Y9DGnZ7e7s2Njbq4YcfrgceeKBms1m99tprtbq6Wg888EA99thjNZlM6urVq/Xkk0/WH/7wh3rppZfq4sWL9ZWvfKU+/elP1+uvvz4c5coCMbhs8aJq8a6FYSLPHH2xnUuNyI5wpKsdgdXV1bp69eogYAiqswTpJDKnWWuKrKQHjULykbnMhx2/9PztJHJ/q08+lhhQ6r1BUEZGMprFZ+aj+2Ql7P5V1Qgwt8aY2b8EN44QcJ3nks+XGSSgMxx1xIlLQ2pnLaNEtMVvO4KQDY3XfxosG5+qRebD33O9f3fn44ND79WayuekDN1KppBJl2+2qJUN9ncGgstG3iORmCGDExkJTlAGWXcDGLOUpWqhv8A5Pu3JduG4bHzyHfuVFQTcD07x/g6P2dcatKeN4bkO9jn67eNtcTzMH2d9HTBLB8f9MCagLNv4zfqfsRvj8blxB7zAkXJ2hKwF483sA32iLQeh+dtzk06R++Q15LEgN16bxruumGjJsJ0HYyP3La/xs5xpwW76ftu0VqVEJgh8itnGxsaI36fRH6d6j4YBmJlvoW8ZZoBuptRa3qQF4LgyFjzArIOfzWb11FNP1ac+9ana39+vn/70p/WLX/xiKPFxSQVtPProozWbzeqFF16oV199tS5cuFBPPfVUff7zn68PfehD9Ze//KV+/etf16uvvlrPP/98Pfvss8O7F2azWVVVffe7362Dg4P65z//Wd/73vdqZ2ennnvuufriF79Y8/m8fv7zn9c999xTP/nJT+qVV16p69ev1wMPPFAPPvjgwM+q8aZ5Nh4bYKcDYhCVb8ZOA2Th4H5fz/8IMalSLyKUnoGiZcCKvhUZpj1Sscx51WIDG+0bANo40Lb7D99camZZTQVoBzkNk/tB1sZRFjs6KfMoBgPWdEhYzGm8fD3KNjNWuV6WkTiRxby2g0gUx0bIRsxGyGDLYKJqrK9Sdph79Iedecj/tyJznTq9E7IctvT0Se9PuW6R2065XWYZbunadBysMx1pRu/4Za0cB8/3juDaJttuVC2ArUte0es8M4McZORtO+bz+VAp4pIwgo65ofzMmTO1s7NT+/v7tb6+XhsbGwNesBM0nU5HoBxeMT7LhD+rWuANvie4yjXGBi3g6WCgsyvpNPiZOG0GwPCECD/OEu2kHYff/p45N26hj1nd4jb43A4DwVpsmJ9j4G9MBy5xtt18Bbtk4MCOL9jM2Q/67L00jHV3d3fktE0mk6FKhb2z7sfKysrAf3jmsXu/semkezQm8xNquosXLw6DdoYA8oSlYTZzWZQGzdnZBA8wwseMsvgcqbj77ruHd0ecPXu23n777bp69Wrt7u7WPffcU/P5vLa2tkb9uXDhQp0/f74uX748TN6HP/zhoc3ZbFaXLl2qyWRSH/vYx4YFj6Nx//33j6Kyb731Vt13333DMbWTyY0SpTvuuKMuXbpUOzs7NZlM6q677qpz587Vm2++OfDSdePwgUXlBesFYS+UfsDbdNQMlPncgprlJLRlZ5LrSfm2SlBc5mKHwWnm46JSzHXLCbXj6tTd2traqPzq4GDx7gsWFNcwfp4Lzefz4dhdnDafbgIvzD+TowWO8sAfL+y833/z0kXv2aB9111euXKllpFSxr3+W44b/1cdPc4ya3LtZFs3WUf4++OAXuo296FTp/8GHbcW8pqWnGbgwtk/g/NlIXQ9NpHqCWxL1fiAEXSO9adf+EobBDqcXaCUxUFO2q2qkR6qOvqyOAfjbLdMtgetYBhBUgIwvobr/FJBP8cZBJyY6XQ6AEvbUTs8mZVwsBDi2eYdvOQ+9nJafv3iQPaUUvqWQDzxoTECgaKqRaUE2DIdDe7JgKfnyG3anuM0OutDxifLwTkgCNmE/9hzZAbAnlgqsR3X8CzIGQ76ZR7SX3jnTd/wN7NbGxsbI8eZz40zW7w8yelTJ3Y07r777iMG25FeM8YKzx07ODgYXhDnyCXRCZjj04MMcGk7y3joj7+nHQPt+Xw+vEAOpjsKgPIyKIfMTJ7pY+1SYRtkO2rAvY7S24livK6hc+mM//eiZZwWJM+Nx0R/MmLjN2LTl4wMoYiddsVJYtw8z8JqoWczt6M/8Ae+0S87IenIosToLxvZHa3yS5PYn2IjtL9/490jTn237jUvbLzom/mIkjE/4APfu04XeWAPDYs7Ixwu6dna2jrJsn3fkR2NNK4J/H1tbt608vOmyoyGeU0lMV82gilzvhZK0Nep0+2kmzm/6YC3ZDjbSjt0K0f7/U6UcFg3Y+/R41U14pdt4XGlP+gOcIPfa9TKDKC/ExwbDMNj2wvjAZ/sg66z7afPPtwFG+b3cRg00xfwme1PBs38P32HH96TaLznwKSdA18H31yVwTyAwbwx21kE+swP40HHw9vDw8MBpNM+soCdNV4wALdT4T2lfG7c4z4a4Ht+4YPXX8oU1xqjIRdca0zk9Uo/HDylPbBZzgdzzFyBJ/IUUe+VhfLdZc6i0Hd4P51O61Z0qs3gBv4GQnTAZ/xbMO2xe7E5swGzcUC4LlOSMMZeZ5bHeNHQnoWC672PIUuB/HwYy2/GlpuPAeorKzeOtvVbSS0QViQGpgl0nGozmGJRWCAt8PTT0fTV1cWRuYBcFh1OQCtTZbDmGvh0rvg808d24HyUnVONXtRWMHYiPKd87kiVDSn3wDNHLZgnRxdms9ng+Lg/RGts1HwUMQ6DQamVKvOWG8FxhFnYXOv3pZj/3ONxLyu1TgjzOmYt8BmbLG2oU9HmMYN859+mVrYsrzVIOW5OOnX6T6jlMPC/KZ3cluPg6/L71EnL7igbnKdNw7Zb59pOoUPQ6QbJBsh+xwO2rwWkHW3OIEnVUf2dL9L1vKPj6KMxiEGo7VlmC7jfDlcGT7iONn2yIVkO9LLxRtXiEBcDYeMVA1MChrbp3Mez7Qw4O2KwbNvnl+45qs6Y3GbKP7bF9ty4wjioanyMrXEc19lR8tj9fAcjPdfIUsqos1GZ6YKXHitj8V4e85cSeHjM586o8Db2zICZZ7QBRsnSqlvRqQ7lR8gtCAiIwTcdNcBkMl1C4kWIUDkynMJmD9fX5iKCACUIZkbY+TsXJH23d52Uiw+BJ61r8G/AbD4wgckXrgXIOg3Hbx+ni1KkvbW1tWFB8fbz9fX1Onfu3BCpIR1r5xEBpo/pmGWkiHsyeu/FY0VgpyqzUJ4rPoMsa74vZSojJ3ZQ+DxL0XAe4DXjNuifzWZH5CIzR14jHpezT8hbAoKM6FsJO6JjI7SshFJL58CBi+SHdUzKTFVbl3BNa+16Pkz+rOVw02anTreTUpb5bRsCWQ7zu3SO7SR/kOTWG6PzAIgMWvFSXpcJVS2AMO/E4B7ra2y5g5ouLUKX21nB7nIdtpD7/UI8Prfes42cz+dDBQj9aR1wAkDkb+adNqzX3GeeYzvjzInxDPcmVqEiJB01Zw0yGu/9KFU1Ou6d7yAHUM0vg/gMcmYGgr75+HPjE4KwKysroxc32inNQCjjhy/umzMn9MuBTwcoDe79YslWEBa5Ml8sR14PiUkcSE8b6evyM7fN/TfbC3wcndjRYANMZhHooAFrVTVfnmZQBQM9cVXj41HdTqaWKJFJ79cC5sVrjw6QmMe4WUF7wXp8FngDTybI4zKPrJjcR2cSaMP38nxHA7ygvHjm8/mwYFxLurq6OrxHg/GzCHwGs/uNM5alVDzb47Jn7ixGVfulUgmi+cyRFJQ2PIEsY+YjfSeSk8fbuS9Wviwaxgsv3C48dVYpNyJClocEvz7KkPbtHLsMzvNvXhtULyPleqs6yjMbZvhgx4TvuZ52LU/+PPmFkbNu8j3+bVlvAbhOnd4ppWFPgOXPEty0yIGs1LsJCD4I8mv9Dh5woMvRcb9TwFFr2jF4qlo4eugJR3G53kAX3U5fHBAyPqFtyEFW6vvBWi2n06C9qo7YqpaT0dJftokGpowLG82YjTPAD9hMV7EwFnRrlqY7OGuHzWXT3M8YXB7WKo11hQv3+91i8MnvEXPwl0xBvh3eTh/P8Zy44sH7ZxxAdKDMge50huCtn+vyNpfG0Rb/E2iG36yJDNpTXgg//RJjKgesNyw3BPt5fxkbx+3Q3IxO7GjAmATkBsz2nKrG3jVtpAJMpeBoo5VsKoLcqGvwwecpFAiVJ9/9McBxBsTjNMChLxbanFiut3Bl6VcrQgLvEDauY4GzmY0x4jBMp9OBhwbRROVp2w6dPXkbPwTK3rpPfECQUdzMBUILP+iHHa90VMy/dO4875CVkjMo9sDNM2TGWZwcb8sYO6qTC95rwkrG66SV9bGzQF8tQ3xmmXH62HuHlo1QoHa6M6LHGklHvuro+d/me5Lvc9v0w99VLXSG17rBhO/vzkan/5SO02mQnYvjZM36k/8N9KxvEri2gkDLQJl5IKLOd+zdO3v2bN1xxx3DiUxVi3W/sbExAr1Vi0g3ugk75s3nVYtyKZ6HXbDurlrsAQS8r67e2Cdou5x4ynYDvMK7DBLscQ/9BahatwKEWziE7+fz+ZBphh+2i8Zk4BoDeINorrFcZgbCWA2eenM+vPfYeA5lx+arA8vGIc6qeD7pd1UNWQJjx4z4sxeGto1vk9/0FR7ADwds4QOl08aZCdztUJ47d67m8/mA5ZwhcekeeJE+8I602Ww2upcs3/7+/tAX7rFtRdYIVPO8zc3NI2vzODoxYnEE1kLVUloGmDkZXGvFl2VD/i6BiD/PtGkLkNhzdD1g1Tia77ZZ4AaHWS7mtj3pPKsFgNNzz3r1qnHK0V65eWxPnud5sfGWT/qOkNEflBWgL8uAsnbRpVOTyWTYzOU5bjkDrVIoeGz+7O/vH9kcnhG8NJh+Lu1wshQLmWv4bW/fALUluyY7KKurN05n8AJP+bXBaJU7WenStl/gZGPiLAc8TT4vE6Ucm0+QnWGuY15sKFImrCDzmfl/zj08zb4hX1zTHYxO7wVZX6azndd5HVjHpzPt9Va1vHu9AJxevz6+lBIUQJmP0QaEzWazUQDJe+8grvVLiqvG9fzGJbQFNuBAkbW1tdrc3ByBfwfXsMHT6XQUwHL5FbgAG5Yv+rMey/IpzzM2JQNc1qP007qWz7352gFI5sV8IkjpjcXeuA4vaZdDYhxwpT1H3z2PyAER9tZeBfrvkrBWwNh2KCtljGHNdwdC4Tt94zoHxsApe3t7A+bBGYAftOPA18HBQV27dm3oux1D41o+8wurcUAPDw+HI269vxh+ee8N62l9fX1YLzjnxtO8puBWdGJHI8uArKwyWs+7BwymGZQ9UCbax8mlk8E1FrIW+DTDLQAWFEceEEQ7GfQfMiCxsLlvVTWcFmSh4Lm0mQYDBWCBdx9tKHKTlYU4AT4ORDo3VhaOzBAN4nOUgDNIKCf65zdH0vecDzbGZ1/T+fN4vShbgNrOIx64x2XnyAuP5zEfTknagUZeLK+5Oc2LmT4heyg1xtuaY7dhBwtDkkbITqijUMtKyJAdYz5HxlrlYzbyNqAOILRAmT+vGu+DsRwed78zY5Zlt59OdKf3lpbV+Wv12Wv/NJQAMwNwBp5eA8tIq6urde7cuTo4OBjeFwCQ92mBrF2X3lQt+GGwa+cBPnmvBbq5anFoh0Gva+3RSSsrK8NJh/SN+ynvqRq/tA9bQyaDefShJGkT+I0ONRarGtss7/mkLdvHDLBktsL2mXbsdOEEknnY3Nyszc3NUSVBBozpK3qYOUMf02e/j8K4zFkbsIopnRZkf21trTY2NkanV9qZYj+NM0/mu20TY2duspzL/DM+hF9+r5grMPwsPuNkSvfBGRfWgPfk0paDd8Z39AmnAhnY398fyaF5ms7KzejUm8HTOGPw+ZtFlC+cS0Ds0iC3mam9VkSc7w1GvfAhJgMwB9Fuq77QQJFnQblfxIvTQuVFYu+Ufrr8yQuJRcQYeJ+CgX9mdyyYTvFZwG14UAC06/njxxkfFrCzL/TDMkDbLscyoOM6K1KDTT87wXsKOXNrhc+c2VmyA5GRENd7Mr4s7WLOWGjOjDiDlT/mFTKQWYk0ggkGnA1jvL5uWclzYbk5PByX3FXVEZ6lA9+aK8+zjX4L1KVOgNJ5OI7neZ31VqfTU87BO73nveD/u+3c5Bj8POvL43SQ2zC4ynuWkbzBG8BOSYjtbQZHrfsd8DTowg54r58DTQAvl+hUHQ2g5jxgA9IhtB0HTxhjOCiajmGW2/Is+kk2hc/sSFgebLf4P1+KZ9tK/9fX14cAK3NAf9HnDr7lHB4eHg6Rd+51NQP61KeKOYAITjC4X1lZGXDTbDYbxpDZPXChcWMrCMs82CkwKJ9MJrWxsTGMFz5zLfPrw2W8Lsnk4DBncJT20jlFxu14ct+5c+dqb2+vrl27NmQweJ5lBkzB3qYsL2SeDw4O6ty5c8NzbedOqkNO7GgYtEJO3QAiDX651iU+rXYdcbRA5AYYwBe/z5w5U9PpdPCsUgEbTNrrdv/TS0WA6Y8dhEyR8XkCdR8dRj8d4WfCLTSU0HC9wa1BGd62o/dM+Pr6+hDZQXDsAPFc7vEpFo62OAKTJS1Witl3Oxl2PG3orOTsFDH3nhMDTitJFrvnxHOLUrK8IaN2imkHHpmvKSful8vOuNZZkMxuuD0D0ozq8CxnUxJcLDu5lrmqvdHfcoIyJlVsnkDmkSNsafh9LZ9Z1pLXqUR9b7bzQaYW794NWjb5Tjn5T/uPbrXN4Tm+xnolA3G+rnUv9yyzzAKQsKHof2fzXfYBGJxMJqMz/x0Y9SmOVTXS39YnrTIq5szOiflPhNgB1arFKVW2Gwng2Lybz3NW3AEt4xw7Kx5rVQ282N3dHfCa7RZ1/S6Rmc/nwxuisXkuhWJ8ROepOvCx9q1gKf3jB0xgPOHTCu1Y+DnIAXxhjn18q3lrR8pOI5iG9ngetgUnyU4Y1yYmsSMKJuW+qsWJlmQfMiCblSVZdeHrcRC9h9Pl4mtra6N9GYzTL3v0niBkgOc583baqopTORpZPuLIdtX47OSWgmsZb0+w225FBBBWFheTySRR48di9278jDjao0fA7ezYaXGWxeN1/1Ngc8x87uiAF2dGYObzxV4LA3YLN/0mcoAXjZJF0TpaAY9cjuVIsZVkbqZivLkh2tkNzz0yw9iSLKwetxdnlub5cxsGK1/z1XzKCBRjsiLM+U7ZNODPZ3uBO1rG/5ktaxkYjwOycVh2kGBjUjXmBTxgn42Nd86b9UOuMWTRPLWDnJEYf9cyIDdzViDrkxZlX49zVG4GVk963f8SHec4HndNi96Jo5BO5+1Yk5bVfI7Xy82cCweVUgelY72s5JIeACn1637hKeMFYO/u7g4nBBm8wTfKvs0nHBjmGIDnEpQEsI60E7ADqxiQehzOwHMNmQGi3Z5X7smMLWDcjo0rFBxwo//Wea5eyBKrdHJzv2nLbhnP2fGxfBpL4DTgrDBXLnWazxfZCPhfdaMCBL4b/DMv2GCPyU6Pv/OcgKUA5WfPnh0cL9svO35gsZQR+Gc8Cm4wtnSFBXx0hsuBUvg0mUyGLI4d4KoaeOfTppAZHzSQpdvwlP+vX79eW1tbTZx/MzqVo0HHDdwSdGYU15Noz5w2uc4T64EhiN7k4wgEizZLeZgMBIEyIBawF27LwNjZ8ARb0biGztFW7nWKlu89kT4SD4FLpZAKA8VlXkBWUiwOZ0Lgg8va7CC4NhCv3UJn8AW/7UDZuXIWwADP47NMOfIymSyObEsFTPssMm90Yiy5+Qze2LljPJSwMaZWVCllws4bc5BOA39beaTMkaJ3BPPw8HCIcmA0vZaW2cmoGjsKVYsoFEprb2+vZrPZkMVAwVtpVy1qXZGNNIQ2gDYIfAdZx/hz9691X4J+r0/PldfErch8OQ3ofSf3nIaWyak5zoFrXWdDfKs2HTxqtdGa55YM+f+Wg5tOgXVJtt8at3WJgZ5LJk4jk+9H8uEZjjbDq7W1tcHeo6MJsHEfn6OHHeDxOxUI3Pk5eQiNbZZxgHUb7fudTBmpdsQ4x4VsAEB3d3eHDIYBPhinaoEHGAOb5BODmJdsqgawoq+dNTCAd5DZvMImZlAz16Q38TO3xgEOKLp9Kj6MN13S1AqAO4BowJ86G+eCLBht+8V9xrqZNXJQzBk2z6PXNmVjEAfBMGfgHJdkec8L9o/xOfh75syZ4eXR29vbde3atQFbMLb19fXB7nquXQrm57ns/qR0YkcD4fIAWMBeqJ5oe20ZXXG7CIOVbUupWxD8noSMCHMtIM4L2PXzpIwYjxc84/Ui8MJiAqwIErAgEC3Q7ettHBASn8zg6yEbD5TcuXPnBuC7vr4+OpkiS5SsLGww8/xpnpsOpSMN5rlTzvzvPvt31TiyZsBpZ4K5NU9wIjIybj7idHhBWk59brblzJ95PtNgO+JuA2HglwqMdnKzmBUac+OonPu3rACharwXyuvNL/WEr2nkLau0YZkyr8x7jIwdAq9JrnckrAVA0+Fo6bKcK/c7r/U4Eii2yLrD/DsJWP5fpJutk+OcgJNe6894luXG85N9so5Imb1ZH1JGU77SEc5AUV6zrIRO5yW0u7u7w3eHh4e1s7MzAGqCNWxMdpCxamGjDBarFqA7X5jHdzzL8+egJzrH0WqqL9B5jn47COtgVdUiSGh76OPtHfRNQA3ITYBs+XHZGc/Z2dmpyWRS58+fH2wvZPvakimuZ578RnVncyjBMRbxdfDYQbzpdDra8G6sklUixhHGOHYyAM18Bx/BjvA5cZzH7+yQ+ZGYFt7QhzxoxjbLVTv0k3l04Nvr3OXidnzZXkCAmXZZG8wBa4NjdME2rCXv5yAQ7SqOm9GpN4MzIT6GLMsMbPwpx+H/TFUhIAZYBpCu06MPnjCDbQuLTwzwiUl2giwA3miU0QSnSS0QXnAIqIWQ+xmf77dSMTh3v+EZ97gG0A6AQS6Oyu7u7lCPSiTFwMQLAKUHL7Nuz3Pn2kfzkHFacbhftOsokj1/b3r3fKejUVXDcYH0K8dDatPzTJ8hK5csnbJ8WrFk2Y7nhr7aGTLvUjnTPgojHVDPpZ93mijC+5FIidup9HgJAKROQL7SSeU7ZNVzYf7D2+McCAOxlgPC3zYiLYDXopxbPoMMVm4GOvM5LWfpf4Fux3hPA7iPc0ZaTu1x/2dblsWUjRZwSyej1f90PKy/qo4ecbusDqrr3a9evTqK9B4cHNTGxsYoOEGGFBtj8M3GcnSt7Y6BZKuUsmqMO9BVdv6NB6w7bF/8XgzfU1VDRsDtunqAPmB7fPpl9nN/f390AA2fVy2O8qV0zMf9Mi76nmNt4TPG4fvTHvK97T8/xired2I7YN1vGacfdgDT8fL/9NVgG1miRIr15EwO43Jfs2wJ+TAu8hirFnt1OGigqkZOg7MVOKzIO23aKU4sTmYksbVtpJ0IV+rkyVf+3vb2VnQqR8PMZMI9uXQI8mSmV2eFa8AL8PJbu71wDSQS7KcCRXAMhGEq/TOYS8FmzI4+51s//fwzZ87U3t5e7e7uHoku0B8LD58xdguuv+N/A+Jsz+VOCa4tHOa/X9LneeWaXDSuCUwFiiJhwfA8K0bPO7z3/Oc8W/FULU798qkfyAIlYSmPSYwvT1toKZ3MlDmSBc9RAh4/4/IGf39mByuVXipQZxDzBT/LSOkIZMkejradkFS42Z4dWhuCVq1qOvt2Yt1W9relx/y/HY8WOSBhat3TMtb5Ofw7DWB+N+l29eW9cpqsd/zck/Yj58ifZ1v5DGd501gjU6bUO1Vj8GQ9xfXOZltPYl8dpFs2otTDOhlwzDpk3TBW3lHBtfwGI7ABdnd3d/ge/WunINf/9evXh+oBVwNwrbPmZG2rFoEW7IEPgknQ7uAs/zN2/gasepO8dZ8DO8Y7fg79hiwnfqZ1rPVaS1cbhKfDwRywZnyN+Ymz6Ch+lkmlMw728lzBw9YmdgLnuXEcfh8eHg5lZXZImQM7T7RpOczMAXz3/cac3AMfuIZxu3QM/szni/2Nq6urde3atZHMkp2xnkhsifzCb/PT+IV5sEzfjE7saLCwYZAj0wadBt4wxgLAbyaM9lgsLM7Wi0C4P+vWEwjbAfIL1bifhU2/mLRWJMDCzXgRNBjfWjj+QbByko8DPzzDRsdCbGeEthirN4IxNtqknayv9MlTCGzV+NSjLFuy85Bv18woB/x2/91v+EK7HrsdAislUr3Oxli5MVaDfINF982RcniW0UAD/3TGPHcelwG0n+s5dRbIz/bnLQC8rJSOglPsloN09vmx827nzMEEf55rznLr9cfat/Nh/dL6nL61HAGT5SWv82fpdPi+45yRdxuQn5RaAPmd0EnaOOlzbuX8mPcZDDIovdXzbuYonqbf7qsBivtmmXT/0+FFH2dmPMe9jHRwcDCcnlO12GdAFt6613YD22I9jbPAfYB0A3cDT3jsoCPtOepunc79tkM+cdKgOPWVT9tzoNQ2tOqoDrOzgz1yUNURcUfXV1ZWBqfMzura2tqAkwxKjYcA74zHAU5jAmOQFh6yXeVerwOXsxusm+yE42Qx99hj7rH9t3PjZyVO4Nhj5sNOJsfd8uyqGmVGPHY7sYy1Jb/8TZs5F5YB6wHbxeSdHVzuP3PmzJAVS0cGZ8xzfFKazG+miUWbm5s1n88H791KypNDc64pM2BKgJCG28xlQVioWgCEhZ1lK7kIbZy9sKoWC95lVhl5cr/droXabVaNwXr2PwWce1t1pBa2jOp7bBnRsVPm66y4DPgzSptArgXOEhC3DDh9SEFPZyDH6IWSoNzksXkcKHD31c6BneGcH9LWNgB2QNJh4Zn5llmnMy2LmeVwtCXl27IxmUzq0qVLtYwE+HGpXNUiw1S1yNjYSKdseU0cF923PPC/jX0aduiEKvHImrrZvflMX59g8DggeBLQ+26T+34aMP5e0kmAdMs4e406evxekoGNM5cte5p8T/uG/kj9DmWUfVkIDNLKPLqMxsAWgMbfCa7NG/MM2eAYWOrXnZGgTJfj4tmo7UBf1XjvReKGtNXehwDIo695UIpParJOrFqUmaUMOOCaTi2yxTsY2FQPuE45sk2ez4+WH1UtDr7xvg3utUNs589Bae/DSL2b2RIH9pKn8Dv1tJ+VZbisIxwEr1F45dKvfM7q6upQose8+lACxmU5cMUG/bezxt4k9tKkc2Ly3BwcHAzOUNVi47mdaMZDv/LQhHRgtre361Z06hApG0pYaJwMQ30Yg7SQVS3KXpg8GG2wl55b1XgvgBe/PUwDvOOMXnp6CEA6I85gMKaWw5HAE8LpMWD1fThECFQuINrwZCbgd+SiZRxRJDyfqPFksjhtwCCce7ItRwoyKp+OTIJv+u0IEtcZ/Bvo0U8iA17UKFPLhQF4Oq8oSL9wMEFhVbt8yUeqpvNlJYcRcYmTF3QalXQazAcv4twzYgCRztuyUSrBTp06dToNcXoSetaAfnNzc9D92DBnvKtqsC/Wu47w8wNoTADYAoUJfl02hB01WLV9d+08+v/g4GDI2huvEFFmfJxK1LK9jpZzv8fmzB1t0VfzznYpA42MFb7a4WO8zigwd47Muy3mxWVGDshWjcuf6btttYH6ZDI+MctYjuuo5CCYDq+cNfBz+cyOqjGX7T1jxTm0vFAqbJnLsnBXwzAPs9lsOKrZPEGuLUsQfWG/kh2yw8PD4SV/XhvI0nQ6bW4Z4HS3k9CJMxqdOnXq1KlTp07/Tbp48eIAiNmXWLXIim5ubo6i4lWLwCZZIk5T8l4G79XI7AZBRZd3A6ABaCsrK8P+Ee/PMyCj3wSqvBHc5b8GsxsbG8PxvBlM5O+q8fs7qo5m7QDV3OuMJEAYYAzApQ92IGjLmREDfcYAODVQJ+AJIHcb7qsrR+yYOSNsB8+OVmY94K3LhAgGtw4IyMoU88sOHO3xfMbNd7nfkHF7bnEcLSuWU2eukD8H0813xsimcu8F4Z1svF/OsB8HkKNv77jjjqpavL2dZztgy32TyaSuXLlyyzW7/EXfnTp16tSpU6f/GcqqBDIAVTUq8QFQESmeTqcj0Lm+vj5kvjNL7Uh8glSyCpSgEAEHwALMqhbZa6LFmZ0HjLrcyM/OjdAGmS1gniXRgOiqRcaAiLrLZPb29kbODO0mcHWW36U2fG7+8LmdNfpGn3GwnHGwI5P7OTKLRMbDeyKZc8bp/90u/cDpcEaJsWZ1R/IAXplfVTVsvuZ6zw3kMmHG4j7O5/Mha5UngdInHOPNzc26cOHCUObnrBLznic9uvKD94Ygw3ZQPT848h7/rag7Gp06derUqVOnpSCAOEe1TiaT4V0BjuK6tKqqhmi+y2EgQJ4zH4C+LN3BWQGk+oQklwN7j6HLXfM4Vsp1q2oU8cYpoYSqqobnZQl5liIb3DtrwH25WR3HytkJl2ib985wkEWAX2waBxAzV3ZeaMflZHngiwFu8t/7lTx/zhpkKX6ebuhshnlQdfSkP+bEZUXz+Y09LM4a8Vzv8bD8pUPGfVma3XJOmGOcHmQCZ4IX285ms+FlzX5ZorNVyJ3L1ry/JcvW7TSy1vj+pKXQ3dHo1KlTp06dOi0FAZxwDABgGxsbTSBn4GZQS308m8sPDw/r2rVrA9hrAVHAFtFfwCTA1g6Fo8AucbFTQh+dpXDJkE/kY9wJ2n2YCNFvgGg6JN4wbNCcpTkm9w1eAHzdn8xI+H5nLZgHn9rE9YB6H4gDAM69Dziazj7giPmwEe+3cDbHeyaz7CozKVy7v78/2udqZ40+uK2cZ3iX2SiyDsgGIN5veud6OwmUpvm53Gdn0+V63mcEn7kf+ZlOp6OXPu/u7o6Ox2WvyEn3i3ZHo1OnTp06deq0FORSGR8/WzV+ya6BqsuDMivgE4S4FnDrqHvVYq+HT4HySYr8X7Vwclxnb+DqTbsudcloMYDRfZ1Op4PjQTR6bW1tKOWaTCbDKwIAl/P5/Mi+AY/b+zOIgEP+u6pGwBtgnyd2urTHDp0zSuwbcYaI/gOY7ZA4c8CPNzHDL/pA6ZEzNPTHBwYQuYenjJk5XF1dHbIGyKAdTZ6b5XqcGHb27NlRCZTn0iVTlldvJke2nOkB5Hszvsv76IeP1nU2yC899phwdtlw7swazgp7PigDvBX1zeCdOnXq1KlTp06dOnW67bS852R26tSpU6dOnTp16tTpfUvd0ejUqVOnTp06derUqdNtp+5odOrUqVOnTp06derU6bZTdzQ6derUqVOnTp06dep026k7Gp06derUqVOnTp06dbrt1B2NTp06derUqVOnTp063XbqjkanTp06derUqVOnTp1uO3VHo1OnTp06derUqVOnTreduqPRqVOnTp06derUqVOn207/D+0VMc32A6UNAAAAAElFTkSuQmCC", + "text/plain": [ + "
" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "import matplotlib.pyplot as plt\n", + "\n", + "# Initialize a set to keep track of shown labels\n", + "shown_labels = set()\n", + "\n", + "# Initialize the figure for plotting\n", + "plt.figure(figsize=(10, 10))\n", + "\n", + "# Loop through the dataset and plot the first image of each label\n", + "for i, sample in enumerate(train_ds):\n", + " label = train_ds.features['label'].names[sample['label']]\n", + " if label not in shown_labels:\n", + " plt.subplot(1, len(train_ds.features['label'].names), len(shown_labels) + 1)\n", + " plt.imshow(sample['image'])\n", + " plt.title(label)\n", + " plt.axis('off')\n", + " shown_labels.add(label)\n", + " if len(shown_labels) == len(train_ds.features['label'].names):\n", + " break\n", + "\n", + "plt.show()" + ] + }, + { + "cell_type": "markdown", + "id": "0300a4b7-4f8f-4155-bad8-42df6673eddc", + "metadata": {}, + "source": [ + "## 数据处理\n", + "\n", + "数据集已经准备好了。但我们还没有为微调做好准备。我们将按照以下步骤进行：\n", + "\n", + "- **标签映射：** 我们将标签 ID 与其对应的名字之间进行转换，这对于模型训练和评估非常有用。\n", + "\n", + "- **图像处理：** 然后，我们使用 ViTImageProcessor 来标准化输入图像的大小，并应用特定于预训练模型的归一化。同时，我们还将为训练、验证和测试定义不同的转换，以使用 torchvision 提高模型的泛化能力。\n", + "\n", + "- **转换函数：** 实现函数以将转换应用于数据集，将图像转换为 ViT 模型所需格式和尺寸。\n", + "\n", + "- **数据加载：** 设置一个自定义的整理函数以正确地批量处理图像和标签，并创建一个 DataLoader 以在模型训练期间高效地加载数据和批量处理。\n", + "\n", + "- **批量准备：** 检索并显示样本批量中数据的形状，以验证处理是否正确并为模型输入做好准备。\n" + ] + }, + { + "cell_type": "markdown", + "id": "1463c15d-5b73-47dc-8113-a910e3cd38b9", + "metadata": {}, + "source": [ + "### 标签映射" + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "id": "43a6187f-0ba0-4cd9-a9ec-f06fca3a91bf", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "({0: 'benign', 1: 'malignant', 2: 'normal'}, 'benign')" + ] + }, + "execution_count": 10, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "id2label = {id:label for id, label in enumerate(train_ds.features['label'].names)}\n", + "label2id = {label:id for id,label in id2label.items()}\n", + "id2label, id2label[train_ds[0]['label']]" + ] + }, + { + "cell_type": "markdown", + "id": "0be0b208-5aad-48ff-97f5-ea484cfc8ad7", + "metadata": {}, + "source": [ + "### 图像处理" + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "id": "a2d04160-93aa-425e-a06d-58c09ec6ffbd", + "metadata": {}, + "outputs": [], + "source": [ + "from transformers import ViTImageProcessor\n", + "\n", + "model_name = \"google/vit-large-patch16-224\"\n", + "processor = ViTImageProcessor.from_pretrained(model_name)" + ] + }, + { + "cell_type": "code", + "execution_count": 12, + "id": "136d114f-a054-467e-a034-cdddf9bf574b", + "metadata": {}, + "outputs": [], + "source": [ + "from torchvision.transforms import CenterCrop, Compose, Normalize, RandomHorizontalFlip, RandomResizedCrop, ToTensor, Resize\n", + "\n", + "image_mean, image_std = processor.image_mean, processor.image_std\n", + "size = processor.size[\"height\"]\n", + "\n", + "normalize = Normalize(mean=image_mean, std=image_std)\n", + "\n", + "train_transforms = Compose([ \n", + " RandomResizedCrop(size),\n", + " RandomHorizontalFlip(),\n", + " ToTensor(),\n", + " normalize,\n", + "])\n", + "val_transforms = Compose([\n", + " Resize(size),\n", + " CenterCrop(size),\n", + " ToTensor(),\n", + " normalize,\n", + "])\n", + "test_transforms = Compose([\n", + " Resize(size),\n", + " CenterCrop(size),\n", + " ToTensor(),\n", + " normalize,\n", + "])" + ] + }, + { + "cell_type": "markdown", + "id": "9e910499-84bf-4672-bbf0-ca78915b2821", + "metadata": {}, + "source": [ + "### 创建转换函数" + ] + }, + { + "cell_type": "code", + "execution_count": 13, + "id": "5ddc7ad4-bd09-4c76-ac00-ca7dafbd8417", + "metadata": {}, + "outputs": [], + "source": [ + "def apply_train_transforms(examples):\n", + " examples['pixel_values'] = [train_transforms(image.convert(\"RGB\")) for image in examples['image']]\n", + " return examples\n", + "\n", + "def apply_val_transforms(examples):\n", + " examples['pixel_values'] = [val_transforms(image.convert(\"RGB\")) for image in examples['image']]\n", + " return examples\n", + "\n", + "def apply_test_transforms(examples):\n", + " examples['pixel_values'] = [val_transforms(image.convert(\"RGB\")) for image in examples['image']]\n", + " return examples" + ] + }, + { + "cell_type": "markdown", + "id": "1ac74835-f883-46a3-877c-26265b27a325", + "metadata": {}, + "source": [ + "### 将转换函数应用于每个集合" + ] + }, + { + "cell_type": "code", + "execution_count": 14, + "id": "45f2b765-5258-4b44-b2ce-6bff952bdd1b", + "metadata": {}, + "outputs": [], + "source": [ + "train_ds.set_transform(apply_train_transforms)\n", + "val_ds.set_transform(apply_val_transforms)\n", + "test_ds.set_transform(apply_test_transforms)" + ] + }, + { + "cell_type": "code", + "execution_count": 15, + "id": "ce586ef6-5f48-4554-8dc4-48797a977674", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "{'image': Image(mode=None, decode=True, id=None),\n", + " 'label': ClassLabel(names=['benign', 'malignant', 'normal'], id=None)}" + ] + }, + "execution_count": 15, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "train_ds.features" + ] + }, + { + "cell_type": "code", + "execution_count": 16, + "id": "731fccb7-7311-4175-8ce1-67141a662b1e", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "{'image': ,\n", + " 'label': 0,\n", + " 'pixel_values': tensor([[[-0.2000, -0.1765, -0.1529, ..., -0.3098, -0.3490, -0.3412],\n", + " [-0.2471, -0.2392, -0.2471, ..., -0.2392, -0.2235, -0.2000],\n", + " [-0.3255, -0.3412, -0.3647, ..., -0.1765, -0.1608, -0.1529],\n", + " ...,\n", + " [-0.7333, -0.7412, -0.7647, ..., -0.7490, -0.7647, -0.7725],\n", + " [-0.7255, -0.7176, -0.7333, ..., -0.7882, -0.7804, -0.7882],\n", + " [-0.7412, -0.7333, -0.7412, ..., -0.7804, -0.7725, -0.7804]],\n", + " \n", + " [[-0.2000, -0.1765, -0.1529, ..., -0.3098, -0.3490, -0.3412],\n", + " [-0.2471, -0.2392, -0.2471, ..., -0.2392, -0.2235, -0.2000],\n", + " [-0.3255, -0.3412, -0.3647, ..., -0.1765, -0.1608, -0.1529],\n", + " ...,\n", + " [-0.7333, -0.7412, -0.7647, ..., -0.7490, -0.7647, -0.7725],\n", + " [-0.7255, -0.7176, -0.7333, ..., -0.7882, -0.7804, -0.7882],\n", + " [-0.7412, -0.7333, -0.7412, ..., -0.7804, -0.7725, -0.7804]],\n", + " \n", + " [[-0.2000, -0.1765, -0.1529, ..., -0.3098, -0.3490, -0.3412],\n", + " [-0.2471, -0.2392, -0.2471, ..., -0.2392, -0.2235, -0.2000],\n", + " [-0.3255, -0.3412, -0.3647, ..., -0.1765, -0.1608, -0.1529],\n", + " ...,\n", + " [-0.7333, -0.7412, -0.7647, ..., -0.7490, -0.7647, -0.7725],\n", + " [-0.7255, -0.7176, -0.7333, ..., -0.7882, -0.7804, -0.7882],\n", + " [-0.7412, -0.7333, -0.7412, ..., -0.7804, -0.7725, -0.7804]]])}" + ] + }, + "execution_count": 16, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "train_ds[0]" + ] + }, + { + "cell_type": "markdown", + "id": "49b06fa4-6f28-4e45-8af4-779c424583fe", + "metadata": {}, + "source": [ + "看起来我们将像素值转换成了张量。\n", + "\n", + "### 数据加载" + ] + }, + { + "cell_type": "code", + "execution_count": 17, + "id": "3f47f263-2046-4847-952d-728fa3fe5cf4", + "metadata": {}, + "outputs": [], + "source": [ + "import torch\n", + "from torch.utils.data import DataLoader\n", + "\n", + "def collate_fn(examples):\n", + " pixel_values = torch.stack([example[\"pixel_values\"] for example in examples])\n", + " labels = torch.tensor([example[\"label\"] for example in examples])\n", + " return {\"pixel_values\": pixel_values, \"labels\": labels}\n", + "\n", + "train_dl = DataLoader(train_ds, collate_fn=collate_fn, batch_size=4)" + ] + }, + { + "cell_type": "markdown", + "id": "aed986b0-e661-4c57-ad58-4fa73d795828", + "metadata": {}, + "source": [ + "### 批量准备" + ] + }, + { + "cell_type": "code", + "execution_count": 18, + "id": "14c6d3f8-48e6-423f-8193-7571f986f103", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "pixel_values torch.Size([4, 3, 224, 224])\n", + "labels torch.Size([4])\n" + ] + } + ], + "source": [ + "batch = next(iter(train_dl))\n", + "for k,v in batch.items():\n", + " if isinstance(v, torch.Tensor):\n", + " print(k, v.shape)" + ] + }, + { + "cell_type": "markdown", + "id": "35379d35-e567-4a30-8b91-6eac80b79044", + "metadata": {}, + "source": [ + "完美！现在我们为微调过程做好了准备。" + ] + }, + { + "cell_type": "markdown", + "id": "45056d5e-8bca-4ece-b29b-bc772aeef49f", + "metadata": {}, + "source": [ + "## 微调模型\n", + "现在我们将配置和微调模型。我们首先使用特定的标签映射和预训练设置初始化模型，调整大小不匹配的问题。训练参数被设置用来定义模型的学习过程，包括保存策略、批量大小和训练轮次，结果将通过 Weights & Biases 进行记录。Hugging Face Trainer 然后将实例化以管理训练和评估，利用自定义数据整理器和模型的内置处理器。最后，在训练之后，模型的性能将在测试数据集上进行评估，并打印指标以评估其准确性。\n", + "\n", + "首先，我们调用我们的模型。" + ] + }, + { + "cell_type": "code", + "execution_count": 19, + "id": "97d0b588-3e41-4852-9307-9e2ec7d5bb0b", + "metadata": {}, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of ViTForImageClassification were not initialized from the model checkpoint at google/vit-large-patch16-224 and are newly initialized because the shapes did not match:\n", + "- classifier.weight: found shape torch.Size([1000, 1024]) in the checkpoint and torch.Size([3, 1024]) in the model instantiated\n", + "- classifier.bias: found shape torch.Size([1000]) in the checkpoint and torch.Size([3]) in the model instantiated\n", + "You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.\n" + ] + } + ], + "source": [ + "from transformers import ViTForImageClassification\n", + "\n", + "model = ViTForImageClassification.from_pretrained(model_name, id2label=id2label, label2id=label2id, ignore_mismatched_sizes=True)" + ] + }, + { + "cell_type": "markdown", + "id": "2f773e18-63e8-41bd-885f-7ba95d074a3d", + "metadata": {}, + "source": [ + "这里有一个微妙的细节。`ignore_mismatched_sizes` 参数。\n", + "\n", + "当你在一个新数据集上微调预训练模型时，有时你的图像的输入大小或模型架构的特定细节（比如分类层中的标签数量）可能与模型最初训练时的大小不完全匹配。这可能会由于各种原因而发生，例如当你在完全不同类型的图像数据（如医学图像或专业相机图像）上使用在一种类型的图像数据（如ImageNet中的自然图像）上训练的模型时。\n", + "将 `ignore_mismatched_sizes` 设置为 `True` 允许模型调整其层以适应大小差异，而不会抛出错误。\n", + "\n", + "例如，这个模型训练的类数是1000，即 `torch.Size([1000])`，它期望一个具有 `torch.Size([1000])` 类的输入。我们的数据集有3类，即 `torch.Size([3])` 类。如果我们直接给它，它会抛出错误，因为类别数量不匹配。" + ] + }, + { + "cell_type": "markdown", + "id": "e2c671bf-9978-46d8-82ef-0906d4e89d03", + "metadata": {}, + "source": [ + "然后，为这个模型定义来自谷歌的训练参数。" + ] + }, + { + "cell_type": "markdown", + "id": "d473af8a-4070-48d7-aecb-1a5a90f0b63f", + "metadata": {}, + "source": [ + "(可选) 注意，由于我们将 `report_to` 参数设置为 `wandb`，指标将被保存在 Weights & Biases 中。W&B 将要求你提供一个 API 密钥，因此你应该创建一个账户和一个API密钥。如果你不希望这样做，你可以删除 `report_to` 参数。\n" + ] + }, + { + "cell_type": "code", + "execution_count": 22, + "id": "f16fc568-9fdc-4c60-acec-6ed3dbb85aef", + "metadata": {}, + "outputs": [], + "source": [ + "from transformers import TrainingArguments, Trainer\n", + "import numpy as np\n", + "\n", + "train_args = TrainingArguments(\n", + " output_dir = \"output-models\",\n", + " save_total_limit=2,\n", + " report_to=\"wandb\",\n", + " save_strategy=\"epoch\",\n", + " evaluation_strategy=\"epoch\",\n", + " learning_rate=2e-5,\n", + " per_device_train_batch_size=10,\n", + " per_device_eval_batch_size=4,\n", + " num_train_epochs=40,\n", + " weight_decay=0.01,\n", + " load_best_model_at_end=True,\n", + " logging_dir='logs',\n", + " remove_unused_columns=False,\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "f74d4457-49fd-4e1b-9842-a3759ec524c9", + "metadata": {}, + "source": [ + "我们现在可以使用 `Trainer` 开始微调过程。" + ] + }, + { + "cell_type": "code", + "execution_count": 23, + "id": "5a117d62-9054-4e14-b7b2-0703de17a741", + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + "
\n", + " \n", + " \n", + " [1880/1880 16:42, Epoch 40/40]\n", + "
\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
EpochTraining LossValidation Loss
1No log0.428426
2No log0.394955
3No log0.370801
4No log0.364052
5No log0.427605
6No log0.441180
7No log0.377579
8No log0.387463
9No log0.380499
10No log0.346761
110.3903000.469292
120.3903000.389932
130.3903000.435536
140.3903000.296190
150.3903000.436435
160.3903000.446079
170.3903000.577235
180.3903000.401280
190.3903000.501154
200.3903000.490980
210.3903000.458035
220.2382000.426354
230.2382000.411909
240.2382000.435578
250.2382000.430924
260.2382000.498050
270.2382000.501461
280.2382000.559837
290.2382000.420119
300.2382000.416809
310.2382000.635555
320.1631000.421264
330.1631000.445050
340.1631000.453854
350.1631000.442983
360.1631000.432370
370.1631000.442086
380.1631000.478380
390.1631000.477927
400.1631000.479882

" + ], + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + }, + { + "data": { + "text/plain": [ + "TrainOutput(global_step=1880, training_loss=0.23721330723863968, metrics={'train_runtime': 1003.2398, 'train_samples_per_second': 18.66, 'train_steps_per_second': 1.874, 'total_flos': 5.128065177052447e+18, 'train_loss': 0.23721330723863968, 'epoch': 40.0})" + ] + }, + "execution_count": 23, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "trainer = Trainer(\n", + " model,\n", + " train_args,\n", + " train_dataset=train_ds,\n", + " eval_dataset=val_ds,\n", + " data_collator=collate_fn,\n", + " tokenizer=processor,\n", + ")\n", + "trainer.train()" + ] + }, + { + "cell_type": "markdown", + "id": "e154fd79-1de7-4169-a6af-402b12881042", + "metadata": {}, + "source": [ + "| Epoch | 训练损失 | 验证损失 | 准确率 |\n", + "|-------|-----------|-------------|---------|\n", + "| 40 | 0.174700 | 0.596288 | 0.903846 |\n", + "\n", + "微调过程已完成。接下来，我们继续使用测试集评估模型。\n" + ] + }, + { + "cell_type": "code", + "execution_count": 24, + "id": "c19b6d99-0a89-45ac-a6d9-ec3e79edc041", + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [], + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "{'test_loss': 0.40843912959098816, 'test_runtime': 4.9934, 'test_samples_per_second': 31.242, 'test_steps_per_second': 7.81}\n" + ] + } + ], + "source": [ + "outputs = trainer.predict(test_ds)\n", + "print(outputs.metrics)" + ] + }, + { + "cell_type": "markdown", + "id": "2c4ddecb-7ab0-493e-90b9-44bf4e2a530e", + "metadata": {}, + "source": [ + "`{'test_loss': 0.3219967782497406, 'test_accuracy': 0.9102564102564102, 'test_runtime': 4.0543, 'test_samples_per_second': 38.478, 'test_steps_per_second': 9.619}`" + ] + }, + { + "cell_type": "markdown", + "id": "0be50a0b", + "metadata": {}, + "source": [ + "### (可选) 将模型推送到 Hub\n", + "\n", + "我们可以使用 `push_to_hub` 将我们的模型推送到 Hugging Face Hub。" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "f1d55e6a", + "metadata": {}, + "outputs": [], + "source": [ + "model.push_to_hub(\"your_model_name\")" + ] + }, + { + "cell_type": "markdown", + "id": "5f74c058-e2a5-4c9c-8d70-1c2c574b933f", + "metadata": {}, + "source": [ + "太棒了！让我们可视化结果。\n", + "\n", + "## 结果\n", + "我们已经完成了微调。让我们看看我们的模型是如何预测类别的，使用 scikit-learn 的混淆矩阵显示，并展示召回率。" + ] + }, + { + "cell_type": "markdown", + "id": "ade5321d-ff63-4308-8317-d1e4da2219df", + "metadata": {}, + "source": [ + "### 什么是混淆矩阵？\n", + "\n", + "混淆矩阵是一种特定的表格布局，它允许可视化算法的性能，通常是监督学习模型，在一组已知真实值的测试数据上。它特别有用，因为可以检查分类模型的性能，因为它显示了真实标签与预测标签的频率。\n", + "\n", + "让我们绘制我们模型的混淆矩阵。\n" + ] + }, + { + "cell_type": "code", + "execution_count": 25, + "id": "8efb0ece-92b3-498d-b47b-0f9c04d4ebb8", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "" + ] + }, + "execution_count": 25, + "metadata": {}, + "output_type": "execute_result" + }, + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAi8AAAHeCAYAAABT8utlAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8fJSN1AAAACXBIWXMAAA9hAAAPYQGoP6dpAABYaUlEQVR4nO3dd3xN9xsH8M/J3jcDGRUSK0HtGa1V0Shi06rWCmqTiPWrvaJWjFo1YrRqFaVDxd5qhKKEWAkZFJnkJrn3/P5Inbql5Lo3ufcen/frdV4v98znuuQ+eb7P9xxBFEURRERERCbCzNABEBEREWmDyQsRERGZFCYvREREZFKYvBAREZFJYfJCREREJoXJCxEREZkUJi9ERERkUiwMHQBpT61WIzExEY6OjhAEwdDhEBGRFkRRREZGBry8vGBmVng1hOzsbOTk5OjlXFZWVrCxsdHLufSByYsJSkxMhLe3t6HDICIiHSQkJKBkyZKFcu7s7Gz4lnZA8n2VXs7n4eGBW7duGU0Cw+TFBDk6OgIA7pzzgZMDR/7krlOrtoYOgYqQ6vpNQ4dAhSwPuTiKX6Sf5YUhJycHyfdVuHPWB06Oun1PpGeoUbrWbeTk5DB5oTf3bKjIycFM53+UZPwszK0NHQIVIUGwNHQIVNj+fihPUQz7OzgKcHDU7TpqGF97ApMXIiIimVKJaqh0fIKhSlTrJxg94q/tREREZFJYeSEiIpIpNUSooVvpRdfjCwOTFyIiIplSQw1dB310P4P+cdiIiIiITAorL0RERDKlEkWoRN2GfXQ9vjAweSEiIpIpufa8cNiIiIiITAorL0RERDKlhgiVDCsvTF6IiIhkSq7DRkxeiIiIZEquDbvseSEiIiKTwsoLERGRTKn/XnQ9h7Fh8kJERCRTKj007Op6fGHgsBERERGZFFZeiIiIZEol5i+6nsPYMHkhIiKSKbn2vHDYiIiIiEwKKy9EREQypYYAFQSdz2FsmLwQERHJlFrMX3Q9h7HhsBERERGZFFZeiIiIZEqlh2EjXY8vDExeiIiIZIrJCxEREZkUtShALerYsKvj8YWBPS9ERERkUlh5ISIikikOGxEREZFJUcEMKh0HWVR6ikWfOGxEREREJoWVFyIiIpkS9dCwKxphwy6TFyIiIpmSa88Lh42IiIjIpDB5ISIikimVaKaXpaB8fHwgCMILy6BBgwAA2dnZGDRoENzc3ODg4ICOHTsiJSVF6/fF5IWIiEim1BCghpmOS8GHjU6fPo2kpCRpiY6OBgB07twZABAaGopdu3Zhy5YtOHToEBITE9GhQwet3xd7XoiIiEgvihcvrvF65syZKFu2LBo3boy0tDSsWrUKGzZswAcffAAAiIqKQsWKFXHy5EnUr1+/wNdh5YWIiEimnjXs6roAQHp6usaiVCpfee2cnBx8++236N27NwRBwNmzZ5Gbm4vAwEBpH39/f5QqVQonTpzQ6n0xeSEiIpIpffa8eHt7Q6FQSEtERMQrr71jxw6kpqaiZ8+eAIDk5GRYWVnB2dlZYz93d3ckJydr9b44bERERCRT+T0vOj6Y8e/jExIS4OTkJK23trZ+5XGrVq3CRx99BC8vL52u/zJMXoiIiOi1nJycNJKXV7lz5w727t2Lbdu2Ses8PDyQk5OD1NRUjepLSkoKPDw8tIqFw0ZEREQypf772Ua6LOo3SBWioqJQokQJtGrVSlpXq1YtWFpaYt++fdK62NhYxMfHIyAgQKvzs/JCREQkU9rep+Xl5xC12l+tViMqKgo9evSAhcU/aYZCoUBISAjCwsLg6uoKJycnDBkyBAEBAVrNNAKYvBAREZEe7d27F/Hx8ejdu/cL2yIjI2FmZoaOHTtCqVQiKCgIS5Ys0foaTF6IiIhkSv2Gwz6a59Cu8vLhhx9C/I9qjY2NDRYvXozFixfrFBOTFyIiIplSiQJUOj4VWtfjCwMbdomIiMiksPJCREQkU89mDOl2Du2GjYoCkxciIiKZUotmUOs420it5WyjosBhIyIiIjIprLwQERHJFIeNiIiIyKSooftsIbV+QtErJi9EREQypZ/7vBhfh4nxRURERET0Cqy8EBERyZR+nm1kfHUOJi9EREQypYYANXTteeEddomIiIh0ItvKS5MmTVC9enXMnz+/0K7Rs2dPpKamYseOHYV2jbdN97qVkHLX6oX1wT0eYHDEPTy6b4GVU71w7rAjnmSawbusEp8MS0HDVmkGiJb0zdY2F5/3/hMN3k+EwiUbN647Y/nX1XA91tXQoVEhCO75FzoNuA/X4nm4+actlox7B7Hn7Qwdlqxw2IhesGDBgv98cia9mYW/xkKt+qdEefuqDcZ+Ug4Ng/OTk9lDSyEz3RyT1tyCwjUPB7a7YMYXPlj06zWUq/LUUGGTngwbeQ6lfdMwJ6I2Hv5liw+ax2PGnCPo3+tDPPzL1tDhkR41bvMY/SYmYtGYkrh6zg7t+z7A9A03EdLQD2kPLQ0dnmzo5z4vxpe8GF9EJkShUMDZ2dnQYciKs5sKriXypOXUXgU8fZSoGpAJAPjzjD3a9v4L/jWewLN0Dj4dngJ7hQrX/+AXm6mzslLhvUb3sHp5FVz6oziSEh3w3dpKSEx0QKs2Nw0dHulZh35/YfcGV+zZ5Ir46zZYOLoklE8FBHV9ZOjQyATIOnnJy8vD4MGDoVAoUKxYMYwfP16qlCiVSoSHh+Odd96Bvb096tWrh4MHD0rHrlmzBs7Ozvjtt99QsWJFODg4oEWLFkhKSpL26dmzJ9q1aye9zsjIQLdu3WBvbw9PT09ERkaiSZMmGD58uLSPj48PZsyYgd69e8PR0RGlSpXCN998U9h/FSYpN0fA/h9cEPTJQwh/F2Mq1c7CoZ3OSH9sDrUaOLjDGTnZAqo2yDRssKQzc3M1zM1F5OSYa6zPUZqjUpW/DBQVFQYLSzXKV32Cc0ccpXWiKCDmiCMq1XpiwMjkRy0KelmMjayTl7Vr18LCwgK///47FixYgHnz5mHlypUAgMGDB+PEiRPYuHEj/vjjD3Tu3BktWrTA9evXpeOfPHmCOXPmYP369Th8+DDi4+MRHh7+n9cLCwvDsWPHsHPnTkRHR+PIkSM4d+7cC/vNnTsXtWvXRkxMDAYOHIgBAwYgNjb2P8+rVCqRnp6usbwNju9WIDPdHB92+ec3sS+X34EqV0DnylXQ2qcaFoz2xsRVt/GOb44BIyV9ePrUEn9eckXXz6/A1e0pzMxENA2Mh3+lh3B1zTZ0eKRHTq4qmFsAqQ80Oxce/2UBl+J5BopKntR/DxvpshjjTepk3fPi7e2NyMhICIIAPz8/XLx4EZGRkQgKCkJUVBTi4+Ph5eUFAAgPD8fu3bsRFRWFGTNmAAByc3OxbNkylC1bFkB+wjNlypSXXisjIwNr167Fhg0b0KxZMwBAVFSUdP7ntWzZEgMHDgQAjB49GpGRkThw4AD8/Pxeeu6IiAhMnjxZt78ME/Tb966o0zQdbh7//DBbO8sDmenmmLkpDk6ueTixW4Hp/X0wd/t1+FbkF5ypmxNRB6GjzuLbrb9ApRIQd80Zh/Z7o1yFVEOHRkRGRNbJS/369SEI/5S7AgICMHfuXFy8eBEqlQoVKlTQ2F+pVMLNzU16bWdnJyUuAODp6Yn79++/9Fo3b95Ebm4u6tatK61TKBQvTUiqVq0q/VkQBHh4ePzneQFg7NixCAsLk16np6fD29v7P/eXg5S7log54ojxK29J6xJvW2FnVHEsP3AVPn75iUrZytm4eMoBO9cUw7Cv7hoqXNKT5EQHjB7eGNY2ebCzy8XjR7YYM+EUkpPsDR0a6VH6I3Oo8gDnf1VZXIrl4fEDWX8tFTm1aAa1jrOFdD2+MLyV/0oyMzNhbm6Os2fPwtxcc3zdwcFB+rOlpWbHuyAIepld9LLzqtX//egra2trWFtb63xdU7Jnoxuci+WhXuA/Q2TKp/n/gczMND8Dc3MRojE+OYzemDLbAspsCzg45KBmnRSsXv6uoUMiPcrLNcP1P+xQ4/0MnNitAAAIgojq72di5xq31xxN2lBBgErHm8zpenxhkHXycurUKY3XJ0+eRPny5VGjRg2oVCrcv38fDRs21Mu1ypQpA0tLS5w+fRqlSpUCAKSlpeHatWto1KiRXq7xtlCrgT2bXBHY+RHMn/sX6l0uG16+SiwY5Y2+ExLh5JKH47sVOHfYEVPWcTaKHNSskwwBwN0ER3i9k4ne/S/ibrwjon/1MXRopGfbvimG8PkJuHbBDrEx+VOlbezU2LOR9/TRJ1ZeTFB8fDzCwsLwxRdf4Ny5c1i0aBHmzp2LChUqoFu3bujevTvmzp2LGjVq4MGDB9i3bx+qVq2KVq1aaX0tR0dH9OjRAyNHjoSrqytKlCiBiRMnwszMTGPoil4v5rAj7t+zQtAnmlMmLSyBaetvYNUML0zs4YunWWbw8s1B+IJ41G2WYaBoSZ/s7fPQs88lFCv+FBkZVjh22AtrV70Llcr4fniSbg7tdIHCTYXuI5PhUjwPNy/b4stuvkj9i/d4odeTdfLSvXt3PH36FHXr1oW5uTmGDRuGfv36Achvpp02bRpGjBiBe/fuoVixYqhfvz5at279xtebN28e+vfvj9atW8PJyQmjRo1CQkICbGxs9PWW3gq1mmTgt8TzL932TpkcTFh5u0jjoaJz5GBJHDlY0tBhUBHZGVUMO6OKGToMWVNB92EflX5C0StB5C1iC01WVhbeeecdzJ07FyEhIXo7b3p6OhQKBR5fKwMnR/5GKnctm3YydAhUhFSxcYYOgQpZnpiLg/gRaWlpcHJyKpRrPPueGHfyQ9g46FbNys7MxbT6ewo1Xm3JuvJS1GJiYnD16lXUrVsXaWlp0rTqtm3bGjgyIiIi+WDyomdz5sxBbGwsrKysUKtWLRw5cgTFirEsSkRERY8PZqTXqlGjBs6ePWvoMIiIiAAAIgSodex5EY1wqrTxpVNEREREr8DKCxERkUxx2IiIiIhMij6eCs2nShMRERHpiJUXIiIimVLBDCod6xS6Hl8YmLwQERHJlFyHjZi8EBERyZQaZlDrWDnR9fjCYHwREREREb0CKy9EREQypRIFqHQc9tH1+MLA5IWIiEim5NrzwmEjIiIi0pt79+7hs88+g5ubG2xtbVGlShWcOXNG2i6KIiZMmABPT0/Y2toiMDAQ169f1+oaTF6IiIhkShTNoNZxEbW4w+7jx4/x3nvvwdLSEr/++iv+/PNPzJ07Fy4uLtI+s2bNwsKFC7Fs2TKcOnUK9vb2CAoKQnZ2doGvw2EjIiIimVJBgErHBytqc/xXX30Fb29vREVFSet8fX2lP4uiiPnz52PcuHFo27YtAGDdunVwd3fHjh078MknnxToOqy8EBER0Wulp6drLEql8oV9du7cidq1a6Nz584oUaIEatSogRUrVkjbb926heTkZAQGBkrrFAoF6tWrhxMnThQ4FiYvREREMqUW/2naffMl/1ze3t5QKBTSEhER8cL1bt68iaVLl6J8+fL47bffMGDAAAwdOhRr164FACQnJwMA3N3dNY5zd3eXthUEh42IiIhk6lnfiq7nAICEhAQ4OTlJ662trV/cV61G7dq1MWPGDABAjRo1cOnSJSxbtgw9evTQKY7nsfJCREREr+Xk5KSxvCx58fT0RKVKlTTWVaxYEfHx8QAADw8PAEBKSorGPikpKdK2gmDyQkREJFNqCHpZCuq9995DbGysxrpr166hdOnSAPKbdz08PLBv3z5pe3p6Ok6dOoWAgIACX4fDRkRERDJV1HfYDQ0NRYMGDTBjxgx06dIFv//+O7755ht88803AABBEDB8+HBMmzYN5cuXh6+vL8aPHw8vLy+0a9euwNdh8kJERCRT+ux5KYg6depg+/btGDt2LKZMmQJfX1/Mnz8f3bp1k/YZNWoUsrKy0K9fP6SmpuL999/H7t27YWNjU+DrMHkhIiIivWndujVat279n9sFQcCUKVMwZcqUN74GkxciIiKZUkMPzzbS8SZ3hYHJCxERkUyJWjbc/tc5jA1nGxEREZFJYeWFiIhIpp7dJVfXcxgbJi9EREQyVdSzjYqK8UVERERE9AqsvBAREckUh42IiIjIpGh7e///Ooex4bARERERmRRWXoiIiGSKw0ZERERkUpi8EBERkUmRa/LCnhciIiIyKay8EBERyZRcKy9MXoiIiGRKhO5TnUX9hKJXHDYiIiIik8LKCxERkUxx2IiIiIhMilyTFw4bERERkUlh5YWIiEim5Fp5YfJCREQkU3JNXjhsRERERCaFlRciIiKZEkUBoo6VE12PLwxMXoiIiGRKDUHnm9TpenxhYPJCREQkU+x5ISIiIjICrLwQERHJFHteiIiIyKRw2IiIiIjICLDyQkREJFMcNiKj0+n9D2BhZmXoMKiQxQ0rbugQqAiVmXDb0CFQIRNEEcgrmmuJehg2MsbkhcNGREREZFJYeSEiIpIpEYAo6n4OY8PkhYiISKbUECDI8A67HDYiIiIik8LKCxERkUzJdbYRKy9EREQy9ewmdbouBTVp0iQIgqCx+Pv7S9uzs7MxaNAguLm5wcHBAR07dkRKSorW74vJCxERkUyJon4WbVSuXBlJSUnScvToUWlbaGgodu3ahS1btuDQoUNITExEhw4dtH5fHDYiIiIivbGwsICHh8cL69PS0rBq1Sps2LABH3zwAQAgKioKFStWxMmTJ1G/fv0CX4OVFyIiIpl61vOi6wIA6enpGotSqXzpNa9fvw4vLy+UKVMG3bp1Q3x8PADg7NmzyM3NRWBgoLSvv78/SpUqhRMnTmj1vpi8EBERyZQ+kxdvb28oFAppiYiIeOF69erVw5o1a7B7924sXboUt27dQsOGDZGRkYHk5GRYWVnB2dlZ4xh3d3ckJydr9b44bERERESvlZCQACcnJ+m1tbX1C/t89NFH0p+rVq2KevXqoXTp0ti8eTNsbW31FgsrL0RERDKlz9lGTk5OGsvLkpd/c3Z2RoUKFRAXFwcPDw/k5OQgNTVVY5+UlJSX9si8CpMXIiIimTLEbKPnZWZm4saNG/D09EStWrVgaWmJffv2SdtjY2MRHx+PgIAArc7LYSMiIiLSi/DwcAQHB6N06dJITEzExIkTYW5ujq5du0KhUCAkJARhYWFwdXWFk5MThgwZgoCAAK1mGgFMXoiIiGQrv3Ki6x12C77v3bt30bVrVzx8+BDFixfH+++/j5MnT6J48eIAgMjISJiZmaFjx45QKpUICgrCkiVLtI6JyQsREZFMFfXjATZu3PjK7TY2Nli8eDEWL16sU0zseSEiIiKTwsoLERGRTIl/L7qew9gweSEiIpIpuT5VmskLERGRXMm09MKeFyIiIjIprLwQERHJlR6GjcBhIyIiIioqut4h99k5jA2HjYiIiMiksPJCREQkU5xtRERERKZFFHTvWTHC5IXDRkRERGRSWHkhIiKSKbk27DJ5ISIikiuZ3qSOyQsREZFMvdUNuzt37izwCdu0afPGwRARERG9ToGSl3bt2hXoZIIgQKVS6RIPERER6ZMRDvvoqkDJi1qtLuw4iIiISM/kOmyk01Tp7OxsfcVBREREVCBaJy8qlQpTp07FO++8AwcHB9y8eRMAMH78eKxatUrvARIREdEbEvW0GBmtk5fp06djzZo1mDVrFqysrKT17777LlauXKnX4IiIiEgXgp4W46J18rJu3Tp888036NatG8zNzaX11apVw9WrV/UaHBEREdG/aX2fl3v37qFcuXIvrFer1cjNzdVLUERERKQHMr1JndaVl0qVKuHIkSMvrN+6dStq1Kihl6CIiIhID2Ta86J15WXChAno0aMH7t27B7VajW3btiE2Nhbr1q3DTz/9VBgxEhEREUm0rry0bdsWu3btwt69e2Fvb48JEybgypUr2LVrF5o3b14YMRIREdGbEAX9LEbmjZ5t1LBhQ0RHR+s7FiIiItIjPlX6X86cOYMrV64AyO+DqVWrlt6CIiIiIj2QacOu1snL3bt30bVrVxw7dgzOzs4AgNTUVDRo0AAbN25EyZIl9R0jERERkUTrnpc+ffogNzcXV65cwaNHj/Do0SNcuXIFarUaffr0KYwYiYiI6E2w5yXfoUOHcPz4cfj5+Unr/Pz8sGjRIjRs2FCvwREREdGbE8T8RddzGButKy/e3t4vvRmdSqWCl5eXXoIiIiIi+i9aJy+zZ8/GkCFDcObMGWndmTNnMGzYMMyZM0evwREREZEO3uab1Lm4uEAQ/hnzysrKQr169WBhkX94Xl4eLCws0Lt3b7Rr165QAiUiIiIt6aNnxVR7XubPn1/IYRAREREVTIGSlx49ehR2HERERKRvvM/Li7Kzs5GTk6OxzsnJSaeAiIiISE9kmrxo3bCblZWFwYMHo0SJErC3t4eLi4vGQkRERFSYtE5eRo0ahf3792Pp0qWwtrbGypUrMXnyZHh5eWHdunWFESMRERG9CQPPNpo5cyYEQcDw4cOlddnZ2Rg0aBDc3Nzg4OCAjh07IiUlRavzap287Nq1C0uWLEHHjh1hYWGBhg0bYty4cZgxYwa+++47bU9HREREhcWAd9g9ffo0li9fjqpVq2qsDw0Nxa5du7BlyxYcOnQIiYmJ6NChg1bn1jp5efToEcqUKQMgv7/l0aNHAID3338fhw8f1vZ0REREVEie3WFX10VbmZmZ6NatG1asWKHRUpKWloZVq1Zh3rx5+OCDD1CrVi1ERUXh+PHjOHnyZIHPr3XDbpkyZXDr1i2UKlUK/v7+2Lx5M+rWrYtdu3ZJD2o0pJ49eyI1NRU7duwAADRp0gTVq1fndG8T1e2LG+jW/6bGuoRbdviiw3sGioj0pavfZXT1u4ySDhkAgOuprlh8oRYO3ysl7VO9eDJCa/6OasXuQy0KuPKoGHpHt4JSpdNcAzIC79bNQKf+KShf5Qnc3HMxuU9ZnNjjbOiw6BXS09M1XltbW8Pa2vql+w4aNAitWrVCYGAgpk2bJq0/e/YscnNzERgYKK3z9/dHqVKlcOLECdSvX79AsWj9E6BXr164cOECGjdujDFjxiA4OBhff/01cnNzMW/ePG1PV+i2bdsGS0tLQ4fxUj4+Phg+fLjGWCC96HacPb7sX0t6rVIZ3w2TSHvJWfaYe7YebqcrIAhA+7KxWPLBbrTb1Qlxqa6oXjwZq5r/guUXa2DqqfehUpvB3/UvqI3whlmkPRs7NW79aYs9m9wwYcXN1x9Ab0aPs428vb01Vk+cOBGTJk16YfeNGzfi3LlzOH369AvbkpOTYWVl9UKxw93dHcnJyQUOSevkJTQ0VPpzYGAgrl69irNnz6JcuXIvjGsZA1dXV0OHQDpSqQQ8fvjy7J5M14G7PhqvI2Pqoav/n6hePAVxqa74X93jWHflXXxzsYa0z61056INkgrNmYMKnDmoMHQYpIWEhASN26G8rOqSkJCAYcOGITo6GjY2NoUWi9Y9L/9WunRpdOjQ4Y0SlyZNmmDIkCEYPnw4XFxc4O7ujhUrViArKwu9evWCo6MjypUrh19//RVA/sMfQ0JC4OvrC1tbW/j5+WHBggWvvcbzlY2kpCS0atUKtra28PX1xYYNG+Dj46MxrCQIAlauXIn27dvDzs4O5cuXx86dO6XtBYmjZ8+eaNeuHebMmQNPT0+4ublh0KBB0kMtmzRpgjt37iA0NBSCIGg8foE0vVPqCdbvOYRVu45i5PSLKO7x1NAhkZ6ZCWq08o2DnUUuYu67w9XmKaoXv49HT22xseV2HP94Lb5t8SNqlUgydKhEby0nJyeN5WXJy9mzZ3H//n3UrFkTFhYWsLCwwKFDh7Bw4UJYWFjA3d0dOTk5SE1N1TguJSUFHh4eBY6lQJWXhQsXFviEQ4cOLfC+ALB27VqMGjUKv//+OzZt2oQBAwZg+/btaN++Pf73v/8hMjISn3/+OeLj42FpaYmSJUtiy5YtcHNzw/Hjx9GvXz94enqiS5cuBbpe9+7d8ddff+HgwYOwtLREWFgY7t+//8J+kydPxqxZszB79mwsWrQI3bp1w507d+Dq6gq1Wl2gOA4cOABPT08cOHAAcXFx+Pjjj1G9enX07dsX27ZtQ7Vq1dCvXz/07dv3lTErlUoolUrp9b/HHeUs9pIC8ya8i7t37OBaTIlPv7iJ2avPYECnADx9wr4HU1fB+SE2tdoOa3MVnuRZYtD+INxIc0W14vnTJgdXP4OvzgTgyqNiaFc2FmuDdqHVji64k+Fs2MCJTISAN2u4/fc5CqpZs2a4ePGixrpevXrB398fo0ePhre3NywtLbFv3z507NgRABAbG4v4+HgEBAQU+DoF+ukfGRlZoJMJgqB18lKtWjWMGzcOADB27FjMnDkTxYoVk77QJ0yYgKVLl+KPP/5A/fr1MXnyZOlYX19fnDhxAps3by5Q8nL16lXs3bsXp0+fRu3atQEAK1euRPny5V/Yt2fPnujatSsAYMaMGVi4cCF+//13tGjRApaWlgWKw8XFBV9//TXMzc3h7++PVq1aYd++fejbty9cXV1hbm4OR0fH12abERERGtd7m5w5Vkz68+3rjoi9qMCaX46i4Ycp2LPjHQNGRvpwK90ZbXd2hqNVDlqUvomvGh5At1/bwOzvQfZN1yphW5w/AODKo2II8LyHTuVjMfdcPUOGTWQ6ivjBjI6Ojnj33Xc11tnb28PNzU1aHxISgrCwMLi6usLJyQlDhgxBQEBAgZt1gQImL7du3SrwCbX1/HCTubk53NzcUKVKFWmdu7s7AEjVkcWLF2P16tWIj4/H06dPkZOTg+rVqxfoWrGxsbCwsEDNmjWldeXKlXvpnYGfj8ve3h5OTk4aFZqCxFG5cmWYm5tLrz09PV/ISAti7NixCAsLk16np6e/0Dj1tsjKtMS9eDt4eT8xdCikB7lqc8Rn5Pc9XH5YHFWK3UePShelPpe4VM3/mzfTXOBpn1HkcRKR/kRGRsLMzAwdO3aEUqlEUFAQlixZotU5DF53//dMIEEQNNY96wVRq9XYuHEjwsPDMXfuXAQEBMDR0RGzZ8/GqVOniiQutVoNAAWO41Xn0MarpqO9bWxs8+BZ8gn2/+xp6FCoEAiCCCtzFe5mOiIlyw6+ilSN7T5OqRpTqYnoNYzg2UYHDx7UeG1jY4PFixdj8eLFb3xOgycv2jh27BgaNGiAgQMHSutu3LhR4OP9/PyQl5eHmJgY1KqVP/U2Li4Ojx8/LtI4nrGysoJKpdL6uLdJSOg1nDpcDPcTbeFWQonP+t+AWi3g4O6CN3aRcRpR8xQO3fNGUpYD7C1yEVwmDvU8EtF7TysAAlZero6h1c/g6iM3XHlUDO3LxaKMIhVDDn5o6NBJD2zsVPDy+aeXz8NbiTKVniAj1QIPEq0MGJnMGEHyUhhMKnkpX7481q1bh99++w2+vr5Yv349Tp8+DV9f3wId7+/vj8DAQPTr1w9Lly6FpaUlRowYAVtbW61m++gaxzM+Pj44fPgwPvnkE1hbW6NYsWKvP+gtU8w9G6MjLsJJkYu0x1a4fN4Zod3rIv0xf7iZOlebp5jVcD9K2D5BRo4VYh+7ofeeVjielD8kuvbPqrA2V+F/dY9DYaXE1cdu6LWnNRIyOL1WDipUfYJZm69Jr7+YeBcAEL3FDXNH+BgoKjIVJpW8fPHFF4iJicHHH38MQRDQtWtXDBw4UJpKXRDr1q1DSEgIGjVqBA8PD0RERODy5ctazUfXRxwAMGXKFHzxxRcoW7YslEolRNEI01sD+2qM8d07iPTjy+NNXrvPNxdraNznheTjj5OOaFGq1ut3JJ286e39/30OYyOIb/k35t27d+Ht7Y29e/eiWbNmhg6nQNLT06FQKNCsRB9YmLECIXdxw8oaOgQqQmUmvHhXUpKXPDEXB/J+QFpamsZN3/Tp2feEz7TpMNPxZnHq7GzcHvdlocarrTe6Sd2RI0fw2WefISAgAPfu3QMArF+/HkePHtVrcIVh//792LlzJ27duoXjx4/jk08+gY+PDxo1amTo0IiIiPRL1NNiZLROXn744QcEBQXB1tYWMTEx0s3T0tLSMGPGDL0HqG+5ubn43//+h8qVK6N9+/YoXry4dMM6IiIiMn5aJy/Tpk3DsmXLsGLFCo0v/Pfeew/nzp3Ta3CFISgoCJcuXcKTJ0+QkpKC7du3o3Tp0oYOi4iISO+e9bzouhgbrRt2Y2NjXzrEolAoXnhWARERERlQEd9ht6hoXXnx8PBAXFzcC+uPHj2KMmXK6CUoIiIiov+idfLSt29fDBs2DKdOnYIgCEhMTMR3332H8PBwDBgwoDBiJCIiojch04ZdrYeNxowZA7VajWbNmuHJkydo1KgRrK2tER4ejiFDhhRGjERERPQG5HqfF62TF0EQ8OWXX2LkyJGIi4tDZmYmKlWqBAcHh8KIj4iIiEjDG99h18rKCpUqVdJnLERERKRPfLZRvqZNm77yOUD79+/XKSAiIiLSE31MdZZD8lK9enWN17m5uTh//jwuXbqEHj166CsuIiIiopfSOnmJjIx86fpJkyYhMzNT54CIiIhIT2Q6bPRGzzZ6mc8++wyrV6/W1+mIiIhIV5wq/WonTpyAjY5PriQiIiL94VTpv3Xo0EHjtSiKSEpKwpkzZzB+/Hi9BUZERET0MlonLwqFQuO1mZkZ/Pz8MGXKFHz44Yd6C4yIiIjoZbRKXlQqFXr16oUqVarAxcWlsGIiIiIifWDDLmBubo4PP/yQT48mIiIig9F6ttG7776LmzdvFkYsREREpEfPGnZ1XYyN1snLtGnTEB4ejp9++glJSUlIT0/XWIiIiMiIyGyaNKBFz8uUKVMwYsQItGzZEgDQpk0bjccEiKIIQRCgUqn0HyURERHR3wqcvEyePBn9+/fHgQMHCjMeIiIi0heZNuwWOHkRxfzoGzduXGjBEBERkf7I9SZ1WvW8vOpp0kRERERFQav7vFSoUOG1CcyjR490CoiIiIj05G0fNgLy+17+fYddIiIiMk5yHTbSKnn55JNPUKJEicKKhYiIiPRJppWXAve8sN+FiIiIjIHWs42IiIjIRMi08lLg5EWtVhdmHERERKRncu150frxAERERESGpFXDLhEREZkQmQ4bsfJCREQkV7o+lFHL5Gfp0qWoWrUqnJyc4OTkhICAAPz666/S9uzsbAwaNAhubm5wcHBAx44dkZKSovXbYvJCREREelGyZEnMnDkTZ8+exZkzZ/DBBx+gbdu2uHz5MgAgNDQUu3btwpYtW3Do0CEkJiaiQ4cOWl+Hw0ZEREQyVdQNu8HBwRqvp0+fjqVLl+LkyZMoWbIkVq1ahQ0bNuCDDz4AAERFRaFixYo4efIk6tevX+DrsPJCREQkV3ocNkpPT9dYlErlKy+tUqmwceNGZGVlISAgAGfPnkVubi4CAwOlffz9/VGqVCmcOHFCq7fF5IWIiIhey9vbGwqFQloiIiJeut/Fixfh4OAAa2tr9O/fH9u3b0elSpWQnJwMKysrODs7a+zv7u6O5ORkrWLhsBEREZFM6XPYKCEhAU5OTtJ6a2vrl+7v5+eH8+fPIy0tDVu3bkWPHj1w6NAh3YL4FyYvREREcqXHqdLPZhC9jpWVFcqVKwcAqFWrFk6fPo0FCxbg448/Rk5ODlJTUzWqLykpKfDw8NAqJA4bERERyVURT5V+GbVaDaVSiVq1asHS0hL79u2TtsXGxiI+Ph4BAQFanZOVFyIiItKLsWPH4qOPPkKpUqWQkZGBDRs24ODBg/jtt9+gUCgQEhKCsLAwuLq6wsnJCUOGDEFAQIBWM40AJi9ERESyJfy96HqOgrp//z66d++OpKQkKBQKVK1aFb/99huaN28OAIiMjISZmRk6duwIpVKJoKAgLFmyROuYmLwQERHJVRE/HmDVqlWv3G5jY4PFixdj8eLFOoXEnhciIiIyKay8EBERyVRR32G3qDB5ISIikis+VZqIiIjI8Fh5ISIikjMjrJzoiskLERGRTMm154XDRkRERGRSWHkhIiKSK5k27DJ5ISIikim5DhsxeSEiIpIrmVZe2PNCREREJoWVFxMmPn0KUVAZOgwqZGUjLhk6BCpC6e1rGToEKmR5udnA9h+K5FocNiIiIiLTwmEjIiIiIsNj5YWIiEiuZFp5YfJCREQkU3LteeGwEREREZkUVl6IiIjkisNGREREZEoEUYQg6pZ96Hp8YeCwEREREZkUVl6IiIjkisNGREREZErkOtuIyQsREZFcybTywp4XIiIiMimsvBAREckUh42IiIjItHDYiIiIiMjwWHkhIiKSKQ4bERERkWnhsBERERGR4bHyQkREJGPGOOyjKyYvREREciWK+Yuu5zAyHDYiIiIik8LKCxERkUxxthERERGZFpnONmLyQkREJFOCOn/R9RzGhj0vREREpBcRERGoU6cOHB0dUaJECbRr1w6xsbEa+2RnZ2PQoEFwc3ODg4MDOnbsiJSUFK2uw+SFiIhIrkQ9LQV06NAhDBo0CCdPnkR0dDRyc3Px4YcfIisrS9onNDQUu3btwpYtW3Do0CEkJiaiQ4cOWr0tDhsRERHJVFE37O7evVvj9Zo1a1CiRAmcPXsWjRo1QlpaGlatWoUNGzbggw8+AABERUWhYsWKOHnyJOrXr1+g67DyQkRERK+Vnp6usSiVytcek5aWBgBwdXUFAJw9exa5ubkIDAyU9vH390epUqVw4sSJAsfC5IWIiEiunt2kTtcFgLe3NxQKhbRERES88tJqtRrDhw/He++9h3fffRcAkJycDCsrKzg7O2vs6+7ujuTk5AK/LQ4bERERyZQ+h40SEhLg5OQkrbe2tn7lcYMGDcKlS5dw9OhR3QJ4CSYvRERE9FpOTk4aycurDB48GD/99BMOHz6MkiVLSus9PDyQk5OD1NRUjepLSkoKPDw8ChwLh42IiIjkqohnG4miiMGDB2P79u3Yv38/fH19NbbXqlULlpaW2Ldvn7QuNjYW8fHxCAgIKPB1WHkhIiKSqaKebTRo0CBs2LABP/74IxwdHaU+FoVCAVtbWygUCoSEhCAsLAyurq5wcnLCkCFDEBAQUOCZRgCTFyIiItKTpUuXAgCaNGmisT4qKgo9e/YEAERGRsLMzAwdO3aEUqlEUFAQlixZotV1mLwQERHJ1XOzhXQ6R4F3ff2+NjY2WLx4MRYvXvzGITF5ISIikik+VZqIiIhMi0yfKs3ZRkRERGRSWHkhIiKSKQ4bERERkWlRi/mLrucwMhw2IiIiIpPCygsREZFcybRhl8kLERGRTAnQQ8+LXiLRLw4bERERkUlh5YWIiEiuivgOu0WFyQsREZFMyXWqNIeNiIiIyKSw8kJERCRXnG1EREREpkQQRQg69qzoenxhYPJCREQkV+q/F13PYWTY80JEREQmhZUXIiIimeKwEREREZkWmTbsctiIiIiITAorL0RERHLFO+wSERGRKZHrHXaZvJBRa9U1Ca26JsH9HSUA4M51O2xY4o0zh10NHBnpGz9r+fo8MAaNq91G6RKpUOaa4+ItdyzdVQ/x952lfdoEXEHzWnHw8/4L9ja5CBrTA5lPrQ0XNBk19rwYmI+PD+bPn2/oMIzWX8lWiJrjgyEdqmNox+q4cFKBCYuvoFS5LEOHRnrGz1q+qpdLwrYjldAvsi2GL2kFC3M1Igf8AhurXGkfG6s8nLrqjXXRNQwYqQw9GzbSdTEyrLyQUTt1wE3j9dr5PmjVNRn+1TMQH2dvoKioMPCzlq8Ry1pqvJ7+XRP8PGM9/Lz/woUbngCAzYeqAABqlEss8vjkTFDnL7qew9iw8vIaOTk5hg6B/mZmJqJxywewsVPhaoyTocOhQsTPWt7sbfN/rqY/4bAQvRnZJS9NmjTB0KFDMWrUKLi6usLDwwOTJk2StsfHx6Nt27ZwcHCAk5MTunTpgpSUFGn7pEmTUL16daxcuRK+vr6wsbEBAAiCgOXLl6N169aws7NDxYoVceLECcTFxaFJkyawt7dHgwYNcOPGDelcN27cQNu2beHu7g4HBwfUqVMHe/fu1fo9KZVKpKenayxvE58KWdh27jh2XjyGwZPjMHVQRcTfsDN0WFQI+FnLnyCIGNbhBC7cdMetJPYzFTqZDhvJLnkBgLVr18Le3h6nTp3CrFmzMGXKFERHR0OtVqNt27Z49OgRDh06hOjoaNy8eRMff/yxxvFxcXH44YcfsG3bNpw/f15aP3XqVHTv3h3nz5+Hv78/Pv30U3zxxRcYO3Yszpw5A1EUMXjwYGn/zMxMtGzZEvv27UNMTAxatGiB4OBgxMfHa/V+IiIioFAopMXb21unvx9Tc/eWLQa1q4HhXarj5+89MeKrayhV9omhw6JCwM9a/kZ0OooyHo8wcU0zQ4fydhD1tBgZWfa8VK1aFRMnTgQAlC9fHl9//TX27dsHALh48SJu3bolJQDr1q1D5cqVcfr0adSpUwdA/lDRunXrULx4cY3z9urVC126dAEAjB49GgEBARg/fjyCgoIAAMOGDUOvXr2k/atVq4Zq1apJr6dOnYrt27dj586dGknO64wdOxZhYWHS6/T09LcqgcnLNUNSvC0AIO6yAypUyUDb7olYNLGcgSMjfeNnLW9hHY+iQeV4DFoYjAdpDoYO560g18cDyLLyUrVqVY3Xnp6euH//Pq5cuQJvb2+NL/5KlSrB2dkZV65ckdaVLl36hcTl3+d1d3cHAFSpUkVjXXZ2tjSsk5mZifDwcFSsWBHOzs5wcHDAlStXtK68WFtbw8nJSWN5mwlmgKWVEXaQkd7xs5YLEWEdj6JR1dsYurg1kh693T/DSHeyrLxYWlpqvBYEAWp1wX8A2tu/fGbD8+cVBOE/1z27Vnh4OKKjozFnzhyUK1cOtra26NSpE5uAtdAz7DbOHHbB/SRr2Nmr0KT1A1Stm4ZxIZUNHRrpGT9r+RrR+Ria14zDmJUf4km2JVwd84cCM7OtkJOb/zXk6vgEbk5PULJY/i9/ZT0f4YnSEsmPHZDxxMZgsZs83mHX9FWsWBEJCQlISEiQqi9//vknUlNTUalSJb1f79ixY+jZsyfat28PIL8Sc/v2bb1fR86c3XIR/tU1uJbIQVaGBW7F2mFcSGXEHHcxdGikZ/ys5avD+38CABYP/Ulj/fTvGuOX3/0AAO3e+xMhH52Tti0ZtuuFfegNiAB0LV4aX+7ydiUvgYGBqFKlCrp164b58+cjLy8PAwcOROPGjVG7dm29X698+fLYtm0bgoODIQgCxo8fr1UFiID5X5Y3dAhURPhZy9d7w/q9dp/Vu2tj9W79/xwmeZJlz8t/EQQBP/74I1xcXNCoUSMEBgaiTJky2LRpU6Fcb968eXBxcUGDBg0QHByMoKAg1KxZs1CuRURE9G/PGnZ1XYyNIIpGGBW9Unp6OhQKBT5w7AYLwcrQ4RCRHqW30P8QNhmXvNxsnNk+HmlpaYU2AUP6nqg+Bhbmut0MME+lxP7zMws1Xm29VZUXIiIiMn1vVc8LERHRW4WzjYiIiMikqAEIejiHkeGwEREREenN4cOHERwcDC8vLwiCgB07dmhsF0UREyZMgKenJ2xtbREYGIjr169rdQ0mL0RERDJliNlGWVlZqFatGhYvXvzS7bNmzcLChQuxbNkynDp1Cvb29ggKCkJ2dnaBr8FhIyIiIrkyQM/LRx99hI8++ug/TiVi/vz5GDduHNq2bQsg/xmD7u7u2LFjBz755JMCXYOVFyIiIrl6lrzouiB/+vXzi1Kp1DqcW7duITk5GYGBgdI6hUKBevXq4cSJEwU+D5MXIiIiei1vb28oFAppiYiI0PocycnJAP55uPEz7u7u0raC4LARERGRXOlx2CghIUHjJnXW1rrd/E4XrLwQERHJlVpPCwAnJyeN5U2SFw8PDwBASkqKxvqUlBRpW0EweSEiIqIi4evrCw8PD+zbt09al56ejlOnTiEgIKDA5+GwERERkUzp48GK2h6fmZmJuLg46fWtW7dw/vx5uLq6olSpUhg+fDimTZuG8uXLw9fXF+PHj4eXlxfatWtX4GsweSEiIpIrA0yVPnPmDJo2bSq9DgsLAwD06NEDa9aswahRo5CVlYV+/fohNTUV77//Pnbv3g0bG5sCX4PJCxEREelNkyZNIL4i4REEAVOmTMGUKVPe+BpMXoiIiORKLQKCjpUXNR/MSEREREVFpk+V5mwjIiIiMimsvBAREcmWHiovML7KC5MXIiIiuZLpsBGTFyIiIrlSi9C5cmKEDbvseSEiIiKTwsoLERGRXInq/EXXcxgZJi9ERERyJdOeFw4bERERkUlh5YWIiEiuZNqwy+SFiIhIrjhsRERERGR4rLwQERHJlQg9VF70EoleMXkhIiKSKw4bERERERkeKy9ERERypVYD0PEmc2repI6IiIiKikyHjZi8EBERyZVMkxf2vBAREZFJYeWFiIhIrniHXSIiIjIloqiGqONToXU9vjBw2IiIiIhMCisvREREciWKug/7GGHDLpMXIiIiuRL10PNihMkLh42IiIjIpLDyQkREJFdqNSDo2HBrhA27TF6IiIjkisNGRERERIbHygsREZFMiWo1RB2HjYzxPi9MXoiIiORKpsNGTF6IiIjkSi0CgvySF/a8EBERkUlh5YWIiEiuRBGArlOlja/ywuSFiIhIpkS1CFHHYSPRCJMXDhsRERGRSWHyQkREJFeiWj+LlhYvXgwfHx/Y2NigXr16+P333/X6tpi8EBERyZSoFvWyaGPTpk0ICwvDxIkTce7cOVSrVg1BQUG4f/++3t4XkxciIiLSm3nz5qFv377o1asXKlWqhGXLlsHOzg6rV6/W2zXYsGuCnjVP5Ym5Bo6EiPQtLzfb0CFQIVP9/RkXRSNsnqjU+cGKecj/rklPT9dYb21tDWtra411OTk5OHv2LMaOHSutMzMzQ2BgIE6cOKFTHM9j8mKCMjIyAACHMzcbOBIi0rvthg6AikpGRgYUCkWhnNvKygoeHh44mvyLXs7n4OAAb29vjXUTJ07EpEmTNNb99ddfUKlUcHd311jv7u6Oq1ev6iUWgMmLSfLy8kJCQgIcHR0hCIKhwykS6enp8Pb2RkJCApycnAwdDhUiftZvl7fx8xZFERkZGfDy8iq0a9jY2ODWrVvIycnRy/lEUXzh++bfVZeixOTFBJmZmaFkyZKGDsMgnJyc3pofcG87ftZvl7ft8y6sisvzbGxsYGNjU+jXeV6xYsVgbm6OlJQUjfUpKSnw8PDQ23XYsEtERER6YWVlhVq1amHfvn3SOrVajX379iEgIEBv12HlhYiIiPQmLCwMPXr0QO3atVG3bl3Mnz8fWVlZ6NWrl96uweSFTIK1tTUmTpxo0DFWKhr8rN8u/Lzl5+OPP8aDBw8wYcIEJCcno3r16ti9e/cLTby6EERjfGgBERER0X9gzwsRERGZFCYvREREZFKYvBAREZFJYfJCREREJoXJCxEREZkUJi9ERERkUpi8EJGs/ftuELw7hGn65ZdfcOHCBUOHQUaCyQvJDr+s6JmkpCTpYXLfffcdALw1DzOVC1EUERcXh86dO2P+/Pn4888/DR0SGQEmLyQrarVa+nLKzMwEwC+rt1V0dDQ6deqE06dPIzQ0FJ9//jnu3Llj6LBIS4IgoFy5cvj+++9x6NAhzJs3D5cvXzZ0WGRgfDwAyYYoijAzy8/HZ86ciWPHjuHp06cYNWoU6tWrVyRPcSXj8e677+LJkyfo3Lkz0tLSEBMTg9KlS0OtVkv/Tsj4iaIIQRDQpk0bmJmZYeDAgQCA0NBQVK5c2cDRkaHwfzDJwvMVl/nz52PmzJmoXbs2njx5gkGDBuGbb77Bw4cPDRwlFZW8vDx4enoiODgYSUlJKFeuHDIzM6XEhUOJpkMQBOnzat26NRYvXow9e/YgMjKSFZi3GJMXkoVnv0n/+eefuHLlCrZu3YqJEyfi+PHjCA4Oxrp167Bq1SopgeGXlzw9+1wtLPKLyu+99x6io6NhZmaGcePGYd++fdJv8i87jozLs8/l+c8rODgYixYtYgLzlmPyQrKxdetWNG3aFHv27IGNjY20ft68eWjevDnWr1+P1atX48GDB+yDkaHnq283b95ERkYGGjZsiEaNGmHLli3IyMjAjBkzcODAAemYefPmAWBflDF6lmT+/vvvWLt2LebPn4/bt28jLy8Pbdu21Uhg2MT7FhKJTJRKpXphXffu3UUrKytxxowZYmZmpsa28PBwsXjx4uK3335bVCGSAYwbN06sWLGiWLZsWXH48OHiH3/8IYqiKMbHx4t16tQRGzVqJM6cOVNs3bq1qFAoxLy8PANHTP+mVqtFURTFH374QXRxcRGbN28uvvPOO2KTJk3Eb775RlQqlaIoiuKOHTvEMmXKiB9//LF45coVQ4ZMRYwNu2Syng0V/fTTT3B0dETjxo2xdu1a5OTkYN26dShZsiQ6duwIOzs7AMDs2bNRqlQpfPLJJ4YMm/RMfG4Y6IcffsDKlSuxePFinDx5EufOncOIESPw1VdfoUaNGti2bRuGDBmC3377DdbW1njw4AHMzc3ZxGtkBEHA4cOHMXDgQMyePRshISG4du0aKleujIyMDCiVSvTr1w9t27aFUqnE1KlT2ZD/lhFEkYO9ZFqe/6L5/fff8dlnn6FWrVoYOXIkatasCQDo0qULLl26hLFjx2okMM+oVCqYm5sXeexUeH755Rfs378f/v7+6NOnDwBgx44dWL58OXJzczFr1izUrFkTWVlZUCqVcHFxgSAIyMvLk3pkyDioVCrMnz8fCQkJmD9/Pm7evInmzZvjvffeQ1paGs6fP4+xY8eiV69esLa2RmZmJhwcHAwdNhUhJi9kUp7/LXvq1Kl48OABdu7cicTERLRv3x5hYWGoV68eAODjjz/GlStXMHDgQOmHHMnTuXPn0KdPH9y5cwdfffWVlLwA/yQwKpUK06dPR506daRtrLgYr2vXrkGlUqFUqVJo0aIFKlSogFWrViEpKQmVK1eGu7s7hg0bhv79+7+0CZvkjf9ryWQ8/wNq7ty5mD17Ntq3b49ffvkFixcvxoULF7Bo0SKcPn0aALBp0yaUKFECR48ehZWVlSFDp0JWs2ZNDBgwAJ6enoiKisLt27elbe3atcOAAQPw+PFjfPvttxrHMXExDi/7HdrX1xcVK1bExYsX8fjxYwwbNgwAkJKSgjp16qB+/fpo2bIlADZcv41YKyWjt2XLFnTu3Fn6AaVSqXDw4EF0794dTZs2BQBUqlQJ9vb2CAsLQ25uLsLDw1GnTh3s3bsXKpVKulcEf8iZvv+qlvTt2xcAsHbtWowbNw7Tp09H6dKlAQBt2rSBi4sL3nvvvSKNlV7v2f/L6Oho/Pjjj7C3t0fnzp1Ru3ZtAEBWVhaePn2KuLg4VKxYETt27ICnpycWLVrEoaK3GIeNyKjNmjULFy9exNq1a6UvLJVKhU8++QQODg6IiorS6F8ZM2YMlixZgk6dOmHo0KGoXr06AA4PyMXzn+PWrVtx6dIlFC9eHFWrVkXDhg0BAEuXLsWGDRtQunRpzJgxA6VKlfrPc5Bx2LNnDzp06ID3338fDx8+xOXLl7Fp0yYEBwfjwYMH6NKlC+7evQsLCwvcv38fe/fuRY0aNQwdNhkQkxcyaklJSShevDgsLCxw+vRpqV8hIiIC06dPx9GjR6UEBci/b8fPP/+Mv/76C+3bt8ekSZNYcZGJ5z/H0aNH49tvv0W1atXw9OlTPHnyBMOGDcOnn34KID+B2bRpE+zs7BAVFQV3d3dDhk6vsWjRIpibm2PgwIFITEzE7NmzsWjRImzcuBGdOnVCUlISfv31V2RnZ+PDDz9EuXLlDB0yGVqRT84megM///yz6OfnJ86ZM0daFxwcLHp6eopHjx4VU1JSxOzsbLFt27bi1q1bxZkzZ4p2dnbi/fv3DRg1FYavv/5a9PHxEY8fPy69trKyEsuUKSOuWLFC2m/27NnigAEDXno/IDKsZ/dxuXr1qhgTEyN27dpV4/5LqampYmhoqGhubi5u3brVUGGSEWPPC5kEPz8/vP/++9i+fTvMzc0xfPhwbNiwAb169cJHH30ELy8vqNVqAPkNmvv370fJkiU5HVpmnj59iosXL2LIkCEICAjAzp078eWXX2L06NG4du0aJk+eDDs7O3z66acIDw+XqjUcKjIugiBg+/bt+Pzzz1GmTBlcvnwZ5cqVkz4nhUKBiRMnwtzcHJ07d8bOnTvRunVrQ4dNRoTDRmR0/v1F8+z1jRs3MGvWLFy4cAGffvophg4dCiD/xmSpqalQqVQICQmBubk5Bg8ejHPnzmH37t1wcnIy1FshHYkvGfJLSkrC06dPkZeXh5YtW2Lw4MEYPnw4tm7dim7dusHKygrr1q1D+/bt//McZBjPPouEhAS0b98e/fr1g5+fH3bv3o3Zs2dj5cqV6Nmzp7R/amoqZs+ejc8++wwVK1Y0XOBkdFh5IaMiiqKUuCxfvhzXrl2Dg4MD+vTpg7Jly2LkyJGYPXs2NmzYAJVKhdDQUHTs2FE6/tq1a5g7dy62bt2KAwcOMHExYc8nsc/fSM7T0xMAsG7dOhQrVgy9e/cGADg4OKB169Zo0aIF2rRpI52HiYvxEAQBe/bswbFjx1C1alX06tULlpaWaNy4MaysrNCnTx+IoohevXoBAJydnTFt2jR+hvQCJi9kNJ7/shozZgxWr16NatWq4f79+4iKikJ0dDT8/PwwatQozJ49G1u3bkVWVhbGjRsHAEhPT8eff/6J+Ph4HDhwAFWrVjXk2yEdPJ/Ezps3D3/88QeUSiUmTpyIMmXKwMrKChYWFrh9+zaOHTuGJk2aYPHixdLddQVB4F2UjciziktGRgbu37+PqVOnomTJkkhMTJSms0+ePBmCIGDQoEHIzs7GgAEDADD5pP9gqGYbov+SkpIiDhkyRDx37pwoiqJ46dIlsWXLlqKzs7N49epVURRFMS4uTuzSpYvYt29fqflPFEUxOztbzMjIMEjcpB/PN9hOnz5ddHJyEgcMGCCWL19eLFmypLhlyxZRqVSKcXFxYocOHUQXFxexTJky4rvvvivm5OSIoihq/Jsg4/Ddd9+JFhYWolqtFpcvXy4KgiBOmzZNTE1N1dhv5MiRYrFixV5YT/Q8Ji9kVNavXy86ODiIdevWFePj46X1169fF1u2bCm6uLhICczdu3elLzrOKJGfO3fuiL169RKPHj0qrevUqZNYqlQpccuWLaIoiuKNGzfEXbt2iatXr5aeDp2bm2uQeOlFz5LIBw8eiCEhIeK8efOkbbNnzxYFQRBnz54tpqWlaRz34MGDIo2TTA+TFzIq+/fvF4OCgkQHBwfxzp07oij+8wMwLi5ODA4OFgVBkLaJIhMXOVq1apVoa2srVqlSRfzjjz80tnXu3Fn09vYWt27dKj59+lRj27MEhozH6dOnxYYNG4oNGzYUY2NjpeqYKP6TwMybN4+VFtIK5w6SwTyb2vy8Jk2aYOrUqfDz80NgYCAePHgg3dq/bNmymD17NkaMGAEvLy/pGE6BNX3//rfQu3dvNGjQAJcuXcLVq1ehUqmkbZs3b0aDBg3w6aef4vjx4xrHscfF+Fy5cgVPnjzBhQsXYGdnB0tLSyiVSgBAeHg45syZgxEjRmD9+vUvfcYR0ctwqjQZxPPNudu3b0diYiLUajWaN28Of39/nDt3DkOHDsXjx49x4MABlChR4oUpr8/PQCF5OHjwIEqUKIFKlSoBAN5//33cu3cP69evR4MGDTQS1fHjx2PSpElMWIxcXl4etm3bhi+//BIeHh7YsWMH3NzckJOTIz0wddGiRWjWrJn0uRO9DpMXMqhRo0bh22+/xXvvvYe4uDgIgoDBgwejd+/eOH78OMaMGYPU1FTs2bMHHh4ehg6X9Oz5JPbkyZNo3rw5Bg0ahL59+6Js2bIAgICAANy/fx9r1659IYEBwFlFRkR87j4uoiji6dOn8PPzgyiK2Lp1K+bOnYtixYph/fr1cHFxgVKphLW1taHDJhPEejsZzPfff4/vv/8eO3fuxJYtWzB06FBcvnwZzs7OAIAGDRpgzpw5yMnJQXh4uGGDJb0Tn5sO/dVXX2Hv3r2wsLDAokWLsHTpUsTFxQEATpw4AXd3d/Tu3RsHDhx4YWiBiYtxeJa4bNu2DYGBgWjatCnq1auHgQMHIiEhAZ07d0ZoaCgePXqEnj174uHDh0xc6M0ZpNOGSBTFKVOmiJ9++qkoiqK4efNm0cnJSVy6dKkoiqKYkZEh3rhxQxRFUbx48SIbMWXm+anMM2bMEBUKhbh7927xwIED4tSpU0VHR0dx5MiRYlxcnLRf2bJlxc6dOxsiXCqggwcPira2tuLSpUvFAwcOiNu2bROLFSsmtm/fXpoduGHDBrFSpUpily5d2GxPb4wNA1QkXvZsmaysLPj4+ODkyZPo3bs3Zs+ejf79+0MURWzZsgWPHj3C0KFD8e677wLg8IAcHDx4EE2aNJF6l7Kzs7F7924MHToUQUFBAPKbthUKhfRson79+qF8+fKIi4tDXl6eIcOn19izZw+aNm2K/v37S+t8fX3RrFkzzJkzB5GRkejcuTMsLS1Ru3ZtNtvTG+O/HCp0KpVK+iF148YNJCYmIjc3F23atEFERAQaNGiA1atXSz/wnjx5go0bNyIxMRGWlpbSeZi4mLbhw4dj48aNGsM+KpUKubm50rpns1CGDBmCrl27YvXq1Vi3bh3u3r0LALCwsNCYeUTGQxRFJCUlSQmmWq1GTk4OqlevjgULFmDDhg24c+cOLCws0KlTJ/j4+Bg2YDJpTF6o0CxduhQxMTFS0jF69Gi0bt0aVatWRWBgIP744w+sXLkSVlZWyM3NxZ07d3Dx4kV07NgRDx48wFdffWXgd0D6NHDgQCxatAiCIODq1asAAHt7e9SpUwfLly+XeiCeffl5enrC398fS5cuxZ49ewDkf0EyiTUOzxLOR48e4cmTJxAEAcHBwTh06BD27t0LMzMzaTagg4MD3Nzc4OjoaMiQSUaYvFChuHXrFmbMmIGlS5fixo0b2L59O9atW4eZM2di7ty5qFevHoYPH47Tp09j1qxZCAkJQUBAALp3746cnBycOnWKv2XLxJIlS5CUlIQKFSrA0tIS3333Hbp3745NmzYByH+mTbly5dCgQQPcvXsXKpUKarUasbGxmDVrFnr06IEvv/xS+oIk4yAIAnbs2IE2bdqgevXqmDhxImxtbdG/f38MGTIE0dHRUsX11KlTsLOz4+dHesOeFyoUvr6+2LVrF/r06YNFixZBqVRi1KhRaNu2LYD8hyj6+PhgzJgx+P7773H58mUkJCTAyckJ1apVg5mZGe/jIgO7du3C/Pnzcf78ecyYMQPFihVDpUqV4ODggDVr1sDKygrt27fHihUrMGjQIFSuXBmVK1fG48ePkZeXh/r16yMuLg579uxhxcXInDt3Dj179sSIESPw8OFD/Pzzz7h27Rrq1q2Ljz76CK1atULNmjVhaWmJS5cuYf/+/XBxcTF02CQTvM8LFapz587hiy++wI0bNxAWFiY9ARoAHj58iJCQEHh7e2PRokUax72swZdMU2RkJH744Qf4+/tj+vTpcHd3x6VLlxAWFgZRFDFkyBC0adMGQP5Q46NHj2BpaYmwsDBYWFigf//+uHnzJrZv3w57e3sDvxsC8nvXvv/+ewiCgC+//BJAfqK6cOFCuLi44LPPPoNCocCvv/4KV1dXtG/fHuXLlzdw1CQnTF6o0F28eBFt2rSBq6srVq5ciRo1akjb+vTpg8TERPzyyy8GjJAKw/MJaGRkJLZu3Qp/f3/MmDED7u7uuHjxIkaMGAFRFNG/f3907NhR4/h79+4hIiIC33//PQ4dOiTNOiPDSk9PR7NmzRAfH4/evXsjIiJC2rZr1y5ERkbCxcUF48ePR/Xq1Q0XKMkaf7WlQlelShX8+OOPUKlU0hACAGRkZODKlSsoWbKkYQOkQmFmZib1LIWGhqJjx464evUq/ve//yE5ORlVqlTBvHnzYGZmhhUrVmDdunXSsQ8ePMCvv/4qDTcwcTEeTk5O+Oabb+Ds7IwjR47g8uXL0rbg4GCEh4fj5s2bmDNnDp48ecLnFVGhYOWFikxMTAw+++wzPH78GLVr14a1tTVu3LiBU6dOwdLS8oVnF5FpetWQ39y5c7Ft2zZpCMnDwwOXLl1Cjx490KhRI0RGRkr7Pnr0CObm5lAoFEUVOmnhjz/+QI8ePVC3bl0MHToUlStXlrbt2bMHfn5+KF26tAEjJDlj8kJF6tKlS2jfvj1sbGwwcuRIdOvWDebm5mzOlYnnE5cff/wRV69ehYeHBypXrozatWsD+CeBqVixotQDc/PmTfj4+MDMzIz9TiYkJiYGffr0Qc2aNREaGsoHK1KRYfJCRe706dNYuXIlli1bBkEQ+GUlE89XzkaPHo0NGzagbNmyUKvVUKlUGDNmDIKDgwHkJzA//vgjihUrhpUrV8LV1RUAG7VNUUxMDPr3748yZcpg4sSJ8Pf3N3RI9BbgTwkqcnXq1GHiIkPPEpdFixZh06ZN2Lx5Mw4ePIj27dvjzJkzCA8Px5YtWwAAI0aMQLNmzVCiRAnpQZwA+G/BBNWoUQNff/01kpKSOMRHRYaVFzIY9rjIT0ZGBgYPHow6depg8ODB2LVrFz7//HMMHDgQly9fxqVLl7BgwQK0bt0awD//BpjEmr7s7GzY2NgYOgx6SzB5IaI39rKk4/r16zA3N0d2djZat26N4cOHY+jQoVizZg369u0LBwcHbNq0CR9++CEAJrFEpD12SBLRG3k+cdm9ezfS0tJQpUoVqWlzxYoVKFmyJEJCQgAALi4uCA4ORrNmzdCsWTPpPExciEhbrNMS0Rt5lriMHTsWnTp1wvjx41GtWjV8/fXXyM3NhaWlJeLi4nDu3Dnk5ORg1apV8Pf3x8CBA2Fubs7nVhHRG2PlhYi08myYRxRF3LlzB0ePHkV0dDT8/PwQFRWFoUOHIisrC3Xr1kX9+vXRrl07uLq6wsrKCtu2bZOO5bOKiOhNseeFiArs+aGiR48e4eHDh1i9ejWmTZsmJSMLFixAWFgY5s+fjypVqiA1NRXJycno06eP9KRwJi5EpAsmL0SktS+//BLR0dG4du0aSpcujc2bN8PPz0/aPn/+fIwePRojR47EtGnTpPVMXIhIH9jzQkSvpVarpT9v3LgRUVFR+Pzzz9GrVy/ExcVh5cqVuHPnjrTP8OHDMWHCBBw4cEDj2TZMXIhIH1h5IaICO3ToEDZv3ox69eqhe/fuAIAlS5YgIiIC3bp1w4ABAzSeZ/N8fwxnFRGRvrBhl4gKJDk5GSEhIUhJSUGFChWk9QMHDoQoipg5cybMzc0REhKCMmXKAAATFyIqFBw2IqIC8fDwwLZt2+Dl5YWff/4ZFy9elLYNGjQI//vf//DVV19hz549GscxcSEifeOwERFp5cKFC+jVqxdq166NYcOGoXLlytK2bdu2oW3btuxtIaJCxeSFiLQWExODPn36oFatWhg+fLh0V91nOKuIiAoTkxcieiMxMTH44osvULp0acyaNQu+vr6GDomI3hLseSGiN1KjRg18/fXXcHR01JhhRERU2Fh5ISKdPJtN9LInTBMRFQYmL0SkM06HJqKixF+TiEhnTFyIqCgxeSEiIiKTwuSFiIiITAqTFyIiIjIpTF6IiIjIpDB5ISIiIpPC5IWI3kjPnj3Rrl076XWTJk0wfPjwIo/j4MGDEAQBqamp/7mPIAjYsWNHgc85adIkVK9eXae4bt++DUEQcP78eZ3OQ0QvYvJCJCM9e/aEIAgQBAFWVlYoV64cpkyZgry8vEK/9rZt2zB16tQC7VuQhIOI6L9YGDoAItKvFi1aICoqCkqlEr/88gsGDRoES0tLjB079oV9c3JyYGVlpZfrurq66uU8RESvw8oLkcxYW1vDw8MDpUuXxoABAxAYGIidO3cC+GeoZ/r06fDy8oKfnx8AICEhAV26dIGzszNcXV3Rtm1b3L59WzqnSqVCWFgYnJ2d4ebmhlGjRuHfN+f+97CRUqnE6NGj4e3tDWtra5QrVw6rVq3C7du30bRpUwCAi4sLBEFAz549AQBqtRoRERHw9fWFra0tqlWrhq1bt2pc55dffkGFChVga2uLpk2basRZUKNHj0aFChVgZ2eHMmXKYPz48cjNzX1hv+XLl8Pb2xt2dnbo0qUL0tLSNLavXLkSFStWhI2NDfz9/bFkyRKtYyEi7TF5IZI5W1tb5OTkSK/37duH2NhYREdH46effkJubi6CgoLg6OiII0eO4NixY3BwcECLFi2k4+bOnYs1a9Zg9erVOHr0KB49eoTt27e/8rrdu3fH999/j4ULF+LKlStYvnw5HBwc4O3tjR9++AEAEBsbi6SkJCxYsAAAEBERgXXr1mHZsmW4fPkyQkND8dlnn+HQoUMA8pOsDh06IDg4GOfPn0efPn0wZswYrf9OHB0dsWbNGvz5559YsGABVqxYgcjISI194uLisHnzZuzatQu7d+9GTEwMBg4cKG3/7rvvMGHCBEyfPh1XrlzBjBkzMH78eKxdu1breIhISyIRyUaPHj3Etm3biqIoimq1WoyOjhatra3F8PBwabu7u7uoVCqlY9avXy/6+fmJarVaWqdUKkVbW1vxt99+E0VRFD09PcVZs2ZJ23Nzc8WSJUtK1xJFUWzcuLE4bNgwURRFMTY2VgQgRkdHvzTOAwcOiADEx48fS+uys7NFOzs78fjx4xr7hoSEiF27dhVFURTHjh0rVqpUSWP76NGjXzjXvwEQt2/f/p/bZ8+eLdaqVUt6PXHiRNHc3Fy8e/eutO7XX38VzczMxKSkJFEURbFs2bLihg0bNM4zdepUMSAgQBRFUbx165YIQIyJifnP6xLRm2HPC5HM/PTTT3BwcEBubi7UajU+/fRTTJo0SdpepUoVjT6XCxcuIC4uDo6Ojhrnyc7Oxo0bN5CWloakpCTUq1dP2mZhYYHatWu/MHT0zPnz52Fubo7GjRsXOO64uDg8efIEzZs311ifk5ODGjVqAACuXLmiEQcABAQEFPgaz2zatAkLFy7EjRs3kJmZiby8PDg5OWnsU6pUKbzzzjsa11Gr1YiNjYWjoyNu3LiBkJAQ9O3bV9onLy8PCoVC63iISDtMXohkpmnTpli6dCmsrKzg5eUFCwvN/+b29vYarzMzM1GrVi189913L5yrePHibxSDra2t1sdkZmYCAH7++WeNpAHI7+PRlxMnTqBbt26YPHkygoKCoFAosHHjRsydO1frWFesWPFCMmVubq63WIno5Zi8EMmMvb09ypUrV+D9a9asiU2bNqFEiRIvVB+e8fT0xKlTp9CoUSMA+RWGs2fPombNmi/dv0qVKlCr1Th06BACAwNf2P6s8qNSqaR1lSpVgrW1NeLj4/+zYlOxYkWp+fiZkydPvv5NPuf48eMoXbo0vvzyS2ndnTt3XtgvPj4eiYmJ8PLykq5jZmYGPz8/uLu7w8vLCzdv3kS3bt20uj4R6Y4Nu0RvuW7duqFYsWJo27Ytjhw5glu3buHgwYMYOnQo7t69CwAYNmwYZs6ciR07duDq1asYOHDgK+/R4uPjgx49eqB3797YsWOHdM7NmzcDAEqXLg1BEPDTTz/hwYMHyMzMhKOjI8LDwxEaGoq1a9fixo0bOHfuHBYtWiQ1wfbv3x/Xr1/HyJEjERsbiw0bNmDNmjVavd/y5csjPj4eGzduxI0bN7Bw4cKXNh/b2NigR48euHDhAo4cOYKhQ4eiS5cu8PDwAABMnjwZERERWLhwIa5du4aLFy8iKioK8+bN0yoeItIekxeit5ydnR0OHz6MUqVKoUOHDqhYsSJCQkKQnZ0tVWJGjBiBzz//HD169EBAQAAcHR3Rvn37V5536dKl6NSpEwYOHAh/f3/07dsXWVlZAIB33nkHkydPxpgxY+Du7o7BgwcDAKZOnYrx48cjIiICFStWRIsWLfDzzz/D19cXQH4fyg8//IAdO3agWrVqWLZsGWbMmKHV+23Tpg1CQ0MxePBgVK9eHcePH8f48eNf2K9cuXLo0KEDWrZsiQ8//BBVq1bVmArdp08frFy5ElFRUahSpQoaN26MNWvWSLESUeERxP/quCMiIiIyQqy8EBERkUlh8kJEREQmhckLERERmRQmL0RERGRSmLwQERGRSWHyQkRERCaFyQsRERGZFCYvREREZFKYvBAREZFJYfJCREREJoXJCxEREZmU/wN1bl2A35oYQwAAAABJRU5ErkJggg==", + "text/plain": [ + "

" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "from sklearn.metrics import confusion_matrix, ConfusionMatrixDisplay\n", + "\n", + "y_true = outputs.label_ids\n", + "y_pred = outputs.predictions.argmax(1)\n", + "\n", + "labels = train_ds.features['label'].names\n", + "cm = confusion_matrix(y_true, y_pred)\n", + "disp = ConfusionMatrixDisplay(confusion_matrix=cm, display_labels=labels)\n", + "disp.plot(xticks_rotation=45)" + ] + }, + { + "cell_type": "markdown", + "id": "9178d2d1-b828-45a6-8873-039abc0419c2", + "metadata": {}, + "source": [ + "### 什么是召回率？\n", + "\n", + "召回率是分类任务中使用的性能指标，用于衡量模型正确识别数据集中所有相关实例的能力。具体来说，召回率评估了模型正确预测为阳性的实际阳性比例。\n", + "\n", + "让我们使用 scikit-learn 打印召回率。\n" + ] + }, + { + "cell_type": "code", + "execution_count": 27, + "id": "48d87ca7-8458-41d5-a773-38e2c9522f64", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Recall for benign: 0.90\n", + "Recall for malignant: 0.86\n", + "Recall for normal: 0.78\n" + ] + } + ], + "source": [ + "from sklearn.metrics import recall_score\n", + "\n", + "# Calculate the recall scores\n", + "# 'None' calculates recall for each class separately\n", + "recall = recall_score(y_true, y_pred, average=None)\n", + "\n", + "# Print the recall for each class\n", + "for label, score in zip(labels, recall):\n", + " print(f'Recall for {label}: {score:.2f}')\n" + ] + }, + { + "cell_type": "markdown", + "id": "c8b1a1b1-7de4-4eb6-98bf-de87e8cbbcec", + "metadata": {}, + "source": [ + "`良性召回率为0.90，\n", + "恶性召回率为0.86，\n", + "正常召回率为0.78`\n" + ] + }, + { + "cell_type": "markdown", + "id": "67b76567-039d-467b-9cfc-0837fb3e1a1b", + "metadata": {}, + "source": [ + "## 结论\n", + "在这个指南中，我们介绍了如何使用医学数据集训练一个 ViT 模型。它涵盖了关键的步骤，如数据集准备、图像预处理、模型配置、训练、评估和结果可视化。通过利用 Hugging Face 的 Transformers 库、scikit-learn 和 PyTorch Torchvision，它促进了高效的模型训练和评估，提供了关于模型性能及其准确分类生物医学图像能力的宝贵见解。" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.10.14" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} From 53937ef12600bc4250890c031fea407cabc09733 Mon Sep 17 00:00:00 2001 From: innovation64 Date: Tue, 9 Jul 2024 15:26:53 +0800 Subject: [PATCH 18/31] update agent rag cn version --- notebooks/zh-CN/agent_rag.ipynb | 827 ++++++++++++++++++++++++++++++++ 1 file changed, 827 insertions(+) create mode 100644 notebooks/zh-CN/agent_rag.ipynb diff --git a/notebooks/zh-CN/agent_rag.ipynb b/notebooks/zh-CN/agent_rag.ipynb new file mode 100644 index 00000000..549e8780 --- /dev/null +++ b/notebooks/zh-CN/agent_rag.ipynb @@ -0,0 +1,827 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Agentic RAG: turbocharge your RAG with query reformulation and self-query! 🚀\n", + "_Authored by: [Aymeric Roucher](https://huggingface.co/m-ric)_\n", + "\n", + "> This tutorial is advanced. You should have notions from [this other cookbook](advanced_rag) first!\n", + "\n", + "> Reminder: Retrieval-Augmented-Generation (RAG) is “using an LLM to answer a user query, but basing the answer on information retrieved from a knowledge base”. It has many advantages over using a vanilla or fine-tuned LLM: to name a few, it allows to ground the answer on true facts and reduce confabulations, it allows to provide the LLM with domain-specific knowledge, and it allows fine-grained control of access to information from the knowledge base.\n", + "\n", + "But vanilla RAG has limitations, most importantly these two:\n", + "- It **performs only one retrieval step**: if the results are bad, the generation in turn will be bad.\n", + "- __Semantic similarity is computed with the *user query* as a reference__, which might be suboptimal: for instance, the user query will often be a question and the document containing the true answer will be in affirmative voice, so its similarity score will be downgraded compared to other source documents in the interrogative form, leading to a risk of missing the relevant information.\n", + "\n", + "But we can alleviate these problems by making a **RAG agent: very simply, an agent armed with a retriever tool!**\n", + "\n", + "This agent will: ✅ Formulate the query itself and ✅ Critique to re-retrieve if needed.\n", + "\n", + "So it should naively recover some advanced RAG techniques!\n", + "- Instead of directly using the user query as the reference in semantic search, the agent formulates itself a reference sentence that can be closer to the targeted documents, as in [HyDE](https://huggingface.co/papers/2212.10496)\n", + "- The agent can the generated snippets and re-retrieve if needed, as in [Self-Query](https://docs.llamaindex.ai/en/stable/examples/evaluation/RetryQuery/)\n", + "\n", + "Let's build this system. 🛠️\n", + "\n", + "Run the line below to install required dependencies:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "!pip install pandas langchain langchain-community sentence-transformers faiss-cpu \"transformers[agents]\"" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We first load a knowledge base on which we want to perform RAG: this dataset is a compilation of the documentation pages for many `huggingface` packages, stored as markdown." + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": {}, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "/Users/aymeric/Documents/Code/cookbook/.venv/lib/python3.12/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html\n", + " from .autonotebook import tqdm as notebook_tqdm\n" + ] + } + ], + "source": [ + "import datasets\n", + "\n", + "knowledge_base = datasets.load_dataset(\"m-ric/huggingface_doc\", split=\"train\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Now we prepare the knowledge base by processing the dataset and storing it into a vector database to be used by the retriever.\n", + "\n", + "We use [LangChain](https://python.langchain.com/) for its excellent vector database utilities.\n", + "For the embedding model, we use [thenlper/gte-small](https://huggingface.co/thenlper/gte-small) since it performed well in our `RAG_evaluation` cookbook." + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Splitting documents...\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "100%|██████████| 2647/2647 [00:34<00:00, 76.04it/s] \n", + "/Users/aymeric/Documents/Code/cookbook/.venv/lib/python3.12/site-packages/langchain_core/_api/deprecation.py:139: LangChainDeprecationWarning: The class `HuggingFaceEmbeddings` was deprecated in LangChain 0.2.2 and will be removed in 0.3.0. An updated version of the class exists in the langchain-huggingface package and should be used instead. To use it run `pip install -U langchain-huggingface` and import as `from langchain_huggingface import HuggingFaceEmbeddings`.\n", + " warn_deprecated(\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Embedding documents... This should take a few minutes (5 minutes on MacBook with M1 Pro)\n" + ] + } + ], + "source": [ + "from transformers import AutoTokenizer\n", + "from langchain.docstore.document import Document\n", + "from langchain.text_splitter import RecursiveCharacterTextSplitter\n", + "from langchain.vectorstores import FAISS\n", + "from langchain_community.embeddings import HuggingFaceEmbeddings\n", + "from langchain_community.vectorstores.utils import DistanceStrategy\n", + "from tqdm import tqdm\n", + "\n", + "source_docs = [\n", + " Document(page_content=doc[\"text\"], metadata={\"source\": doc[\"source\"].split(\"/\")[1]})\n", + " for doc in knowledge_base\n", + "]\n", + "\n", + "text_splitter = RecursiveCharacterTextSplitter.from_huggingface_tokenizer(\n", + " AutoTokenizer.from_pretrained(\"thenlper/gte-small\"),\n", + " chunk_size=200,\n", + " chunk_overlap=20,\n", + " add_start_index=True,\n", + " strip_whitespace=True,\n", + " separators=[\"\\n\\n\", \"\\n\", \".\", \" \", \"\"],\n", + ")\n", + "\n", + "# Split docs and keep only unique ones\n", + "print(\"Splitting documents...\")\n", + "docs_processed = []\n", + "unique_texts = {}\n", + "for doc in tqdm(source_docs):\n", + " new_docs = text_splitter.split_documents([doc])\n", + " for new_doc in new_docs:\n", + " if new_doc.page_content not in unique_texts:\n", + " unique_texts[doc.page_content] = True\n", + " docs_processed.append(new_doc)\n", + "\n", + "print(\n", + " \"Embedding documents... This should take a few minutes (5 minutes on MacBook with M1 Pro)\"\n", + ")\n", + "embedding_model = HuggingFaceEmbeddings(model_name=\"thenlper/gte-small\")\n", + "vectordb = FAISS.from_documents(\n", + " documents=docs_processed,\n", + " embedding=embedding_model,\n", + " distance_strategy=DistanceStrategy.COSINE,\n", + ")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Now the database is ready: let’s build our agentic RAG system!\n", + "\n", + "👉 We only need a `RetrieverTool` that our agent can leverage to retrieve information from the knowledge base." + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": {}, + "outputs": [], + "source": [ + "from transformers.agents import Tool\n", + "from langchain_core.vectorstores import VectorStore\n", + "\n", + "\n", + "class RetrieverTool(Tool):\n", + " name = \"retriever\"\n", + " description = \"Using semantic similarity, retrieves some documents from the knowledge base that have the closest embeddings to the input query.\"\n", + " inputs = {\n", + " \"query\": {\n", + " \"type\": \"text\",\n", + " \"description\": \"The query to perform. This should be semantically close to your target documents. Use the affirmative form rather than a question.\",\n", + " }\n", + " }\n", + " output_type = \"text\"\n", + "\n", + " def __init__(self, vectordb: VectorStore, **kwargs):\n", + " super().__init__(**kwargs)\n", + " self.vectordb = vectordb\n", + "\n", + " def forward(self, query: str) -> str:\n", + " assert isinstance(query, str), \"Your search query must be a string\"\n", + "\n", + " docs = self.vectordb.similarity_search(\n", + " query,\n", + " k=7,\n", + " )\n", + "\n", + " return \"\\nRetrieved documents:\\n\" + \"\".join(\n", + " [\n", + " f\"===== Document {str(i)} =====\\n\" + doc.page_content\n", + " for i, doc in enumerate(docs)\n", + " ]\n", + " )" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Now it’s straightforward to create an agent that leverages this tool!\n", + "\n", + "The agent will need these arguments upon initialization:\n", + "- *`tools`*: a list of tools that the agent will be able to call.\n", + "- *`llm_engine`*: the LLM that powers the agent.\n", + "\n", + "Our `llm_engine` must be a callable that takes as input a list of [messages](https://huggingface.co/docs/transformers/main/chat_templating) and returns text. It also needs to accept a `stop_sequences` argument that indicates when to stop its generation. For convenience, we directly use the `HfEngine` class provided in the package to get a LLM engine that calls our [Inference API](https://huggingface.co/docs/api-inference/en/index).\n", + "\n", + "And we use [CohereForAI/c4ai-command-r-plus](https://huggingface.co/CohereForAI/c4ai-command-r-plus) as the llm engine because:\n", + "- It has a long 128k context, which is helpful for processing long source documents\n", + "- It is served for free at all times on HF's Inference API!" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "metadata": {}, + "outputs": [], + "source": [ + "from transformers.agents import HfEngine, ReactJsonAgent\n", + "\n", + "llm_engine = HfEngine(\"CohereForAI/c4ai-command-r-plus\")\n", + "\n", + "retriever_tool = RetrieverTool(vectordb)\n", + "agent = ReactJsonAgent(\n", + " tools=[retriever_tool], llm_engine=llm_engine, max_iterations=4, verbose=2\n", + ")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Since we initialized the agent as a `ReactJsonAgent`, it has been automatically given a default system prompt that tells the LLM engine to process step-by-step and generate tool calls as JSON blobs (you could replace this prompt template with your own as needed).\n", + "\n", + "Then when its `.run()` method is launched, the agent takes care of calling the LLM engine, parsing the tool call JSON blobs and executing these tool calls, all in a loop that ends only when the final answer is provided." + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "metadata": {}, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\u001b[33;1m======== New task ========\u001b[0m\n", + "\u001b[37;1mHow can I push a model to the Hub?\u001b[0m\n", + "\u001b[38;20mSystem prompt is as follows:\u001b[0m\n", + "\u001b[38;20mYou are an expert assistant who can solve any task using JSON tool calls. You will be given a task to solve as best you can.\n", + "To do so, you have been given access to the following tools: 'retriever', 'final_answer'\n", + "The way you use the tools is by specifying a json blob, ending with ''.\n", + "Specifically, this json should have an `action` key (name of the tool to use) and an `action_input` key (input to the tool).\n", + "\n", + "The $ACTION_JSON_BLOB should only contain a SINGLE action, do NOT return a list of multiple actions. It should be formatted in json. Do not try to escape special characters. Here is the template of a valid $ACTION_JSON_BLOB:\n", + "{\n", + " \"action\": $TOOL_NAME,\n", + " \"action_input\": $INPUT\n", + "}\n", + "\n", + "Make sure to have the $INPUT as a dictionary in the right format for the tool you are using, and do not put variable names as input if you can find the right values.\n", + "\n", + "You should ALWAYS use the following format:\n", + "\n", + "Thought: you should always think about one action to take. Then use the action as follows:\n", + "Action:\n", + "$ACTION_JSON_BLOB\n", + "Observation: the result of the action\n", + "... (this Thought/Action/Observation can repeat N times, you should take several steps when needed. The $ACTION_JSON_BLOB must only use a SINGLE action at a time.)\n", + "\n", + "You can use the result of the previous action as input for the next action.\n", + "The observation will always be a string: it can represent a file, like \"image_1.jpg\".\n", + "Then you can use it as input for the next action. You can do it for instance as follows:\n", + "\n", + "Observation: \"image_1.jpg\"\n", + "\n", + "Thought: I need to transform the image that I received in the previous observation to make it green.\n", + "Action:\n", + "{\n", + " \"action\": \"image_transformer\",\n", + " \"action_input\": {\"image\": \"image_1.jpg\"}\n", + "}\n", + "\n", + "To provide the final answer to the task, use an action blob with \"action\": \"final_answer\" tool. It is the only way to complete the task, else you will be stuck on a loop. So your final output should look like this:\n", + "Action:\n", + "{\n", + " \"action\": \"final_answer\",\n", + " \"action_input\": {\"answer\": \"insert your final answer here\"}\n", + "}\n", + "\n", + "\n", + "Here are a few examples using notional tools:\n", + "---\n", + "Task: \"Generate an image of the oldest person in this document.\"\n", + "\n", + "Thought: I will proceed step by step and use the following tools: `document_qa` to find the oldest person in the document, then `image_generator` to generate an image according to the answer.\n", + "Action:\n", + "{\n", + " \"action\": \"document_qa\",\n", + " \"action_input\": {\"document\": \"document.pdf\", \"question\": \"Who is the oldest person mentioned?\"}\n", + "}\n", + "Observation: \"The oldest person in the document is John Doe, a 55 year old lumberjack living in Newfoundland.\"\n", + "\n", + "\n", + "Thought: I will now generate an image showcasing the oldest person.\n", + "Action:\n", + "{\n", + " \"action\": \"image_generator\",\n", + " \"action_input\": {\"text\": \"\"A portrait of John Doe, a 55-year-old man living in Canada.\"\"}\n", + "}\n", + "Observation: \"image.png\"\n", + "\n", + "Thought: I will now return the generated image.\n", + "Action:\n", + "{\n", + " \"action\": \"final_answer\",\n", + " \"action_input\": \"image.png\"\n", + "}\n", + "\n", + "---\n", + "Task: \"What is the result of the following operation: 5 + 3 + 1294.678?\"\n", + "\n", + "Thought: I will use python code evaluator to compute the result of the operation and then return the final answer using the `final_answer` tool\n", + "Action:\n", + "{\n", + " \"action\": \"python_interpreter\",\n", + " \"action_input\": {\"code\": \"5 + 3 + 1294.678\"}\n", + "}\n", + "Observation: 1302.678\n", + "\n", + "Thought: Now that I know the result, I will now return it.\n", + "Action:\n", + "{\n", + " \"action\": \"final_answer\",\n", + " \"action_input\": \"1302.678\"\n", + "}\n", + "\n", + "---\n", + "Task: \"Which city has the highest population , Guangzhou or Shanghai?\"\n", + "\n", + "Thought: I need to get the populations for both cities and compare them: I will use the tool `search` to get the population of both cities.\n", + "Action:\n", + "{\n", + " \"action\": \"search\",\n", + " \"action_input\": \"Population Guangzhou\"\n", + "}\n", + "Observation: ['Guangzhou has a population of 15 million inhabitants as of 2021.']\n", + "\n", + "\n", + "Thought: Now let's get the population of Shanghai using the tool 'search'.\n", + "Action:\n", + "{\n", + " \"action\": \"search\",\n", + " \"action_input\": \"Population Shanghai\"\n", + "}\n", + "Observation: '26 million (2019)'\n", + "\n", + "Thought: Now I know that Shanghai has a larger population. Let's return the result.\n", + "Action:\n", + "{\n", + " \"action\": \"final_answer\",\n", + " \"action_input\": \"Shanghai\"\n", + "}\n", + "\n", + "\n", + "Above example were using notional tools that might not exist for you. You only have access to those tools:\n", + "\n", + "- retriever: Using semantic similarity, retrieves some documents from the knowledge base that have the closest embeddings to the input query.\n", + " Takes inputs: {'query': {'type': 'text', 'description': 'The query to perform. This should be semantically close to your target documents. Use the affirmative form rather than a question.'}}\n", + "\n", + "- final_answer: Provides a final answer to the given problem\n", + " Takes inputs: {'answer': {'type': 'text', 'description': 'The final answer to the problem'}}\n", + "\n", + "Here are the rules you should always follow to solve your task:\n", + "1. ALWAYS provide a 'Thought:' sequence, and an 'Action:' sequence that ends with , else you will fail.\n", + "2. Always use the right arguments for the tools. Never use variable names in the 'action_input' field, use the value instead.\n", + "3. Call a tool only when needed: do not call the search agent if you do not need information, try to solve the task yourself.\n", + "4. Never re-do a tool call that you previously did with the exact same parameters.\n", + "\n", + "Now Begin! If you solve the task correctly, you will receive a reward of $1,000,000.\n", + "\u001b[0m\n", + "\u001b[38;20m===== New step =====\u001b[0m\n", + "===== Calling LLM with this last message: =====\n", + "{'role': , 'content': 'Task: How can I push a model to the Hub?'}\n", + "\u001b[38;20m===== Output message of the LLM: =====\u001b[0m\n", + "\u001b[38;20mThought: I can use the \"retriever\" tool to find documents relevant to the question, \"How can I push a model to the Hub?\" I will then read through the retrieved documents to find the relevant information and provide an answer to the question.\n", + "\n", + "Action: ```json\n", + "{\n", + " \"action\": \"retriever\",\n", + " \"action_input\": {\n", + " \"query\": \"How can I push a model to the Hub?\"\n", + " }\n", + "}\u001b[0m\n", + "\u001b[38;20m===== Extracting action =====\u001b[0m\n", + "\u001b[33;1mCalling tool: 'retriever' with arguments: {'query': 'How can I push a model to the Hub?'}\u001b[0m\n", + "Retrieved documents:\n", + "===== Document 0 =====\n", + "# Step 7. Push everything to the Hub\n", + " api.upload_folder(\n", + " repo_id=repo_id,\n", + " folder_path=repo_local_path,\n", + " path_in_repo=\".\",\n", + " )\n", + "\n", + " print(\"Your model is pushed to the Hub. You can view your model here: \", repo_url)\n", + "```\n", + "\n", + "### .\n", + "\n", + "By using `push_to_hub` **you evaluate, record a replay, generate a model card of your agent and push it to the Hub**.===== Document 1 =====\n", + "```py\n", + ">>> trainer.push_to_hub()\n", + "```\n", + "\n", + "\n", + "Share a model to the Hub with [`PushToHubCallback`]. In the [`PushToHubCallback`] function, add:\n", + "\n", + "- An output directory for your model.\n", + "- A tokenizer.\n", + "- The `hub_model_id`, which is your Hub username and model name.\n", + "\n", + "```py\n", + ">>> from transformers import PushToHubCallback\n", + "\n", + ">>> push_to_hub_callback = PushToHubCallback(\n", + "... output_dir=\"./your_model_save_path\", tokenizer=tokenizer, hub_model_id=\"your-username/my-awesome-model\"\n", + "... )\n", + "```===== Document 2 =====\n", + "Let's pretend we've now fine-tuned the model. The next step would be to push it to the Hub! We can do this with the `timm.models.hub.push_to_hf_hub` function.\n", + "\n", + "```py\n", + ">>> model_cfg = dict(labels=['a', 'b', 'c', 'd'])\n", + ">>> timm.models.hub.push_to_hf_hub(model, 'resnet18-random', model_config=model_cfg)\n", + "```\n", + "\n", + "Running the above would push the model to `/resnet18-random` on the Hub. You can now share this model with your friends, or use it in your own code!\n", + "\n", + "## Loading a Model===== Document 3 =====\n", + "processor.push_to_hub(hub_model_id)\n", + "trainer.push_to_hub(**kwargs)\n", + "```\n", + "\n", + "# 4. Inference\n", + "\n", + "Now comes the exciting part, using our fine-tuned model! In this section, we'll show how you can load your model from the hub and use it for inference.===== Document 4 =====\n", + "--push_to_hub\n", + "```===== Document 5 =====\n", + ". The second way to upload a model, though, is to call model.push_to_hub(). So this is more of a once-off method - it's not called regularly during training. You can just call this manually whenever you want to upload a model to the hub. So we recommend running this after the end of training, just to make sure that you have a commit message just to guarantee that this was the final version of the model at the end of training. And it just makes sure that you're working with the definitive end-of-training model and not accidentally using a model that's from a checkpoint somewhere along the way===== Document 6 =====\n", + "Finally, if you want, you can push your model up to the hub. Here, we'll push it up if you specified `push_to_hub=True` in the training configuration. Note that in order to push to hub, you'll have to have git-lfs installed and be logged into your Hugging Face account (which can be done via `huggingface-cli login`).\n", + "\n", + "```python\n", + "kwargs = {\n", + " \"finetuned_from\": model.config._name_or_path,\n", + " \"tasks\": \"image-classification\",\n", + " \"dataset\": 'beans',\n", + " \"tags\": ['image-classification'],\n", + "}\n", + "\u001b[38;20m===== New step =====\u001b[0m\n", + "===== Calling LLM with this last message: =====\n", + "{'role': , 'content': 'Observation: Retrieved documents:\\n===== Document 0 =====\\n# Step 7. Push everything to the Hub\\n api.upload_folder(\\n repo_id=repo_id,\\n folder_path=repo_local_path,\\n path_in_repo=\".\",\\n )\\n\\n print(\"Your model is pushed to the Hub. You can view your model here: \", repo_url)\\n```\\n\\n### .\\n\\nBy using `push_to_hub` **you evaluate, record a replay, generate a model card of your agent and push it to the Hub**.===== Document 1 =====\\n```py\\n>>> trainer.push_to_hub()\\n```\\n\\n\\nShare a model to the Hub with [`PushToHubCallback`]. In the [`PushToHubCallback`] function, add:\\n\\n- An output directory for your model.\\n- A tokenizer.\\n- The `hub_model_id`, which is your Hub username and model name.\\n\\n```py\\n>>> from transformers import PushToHubCallback\\n\\n>>> push_to_hub_callback = PushToHubCallback(\\n... output_dir=\"./your_model_save_path\", tokenizer=tokenizer, hub_model_id=\"your-username/my-awesome-model\"\\n... )\\n```===== Document 2 =====\\nLet\\'s pretend we\\'ve now fine-tuned the model. The next step would be to push it to the Hub! We can do this with the `timm.models.hub.push_to_hf_hub` function.\\n\\n```py\\n>>> model_cfg = dict(labels=[\\'a\\', \\'b\\', \\'c\\', \\'d\\'])\\n>>> timm.models.hub.push_to_hf_hub(model, \\'resnet18-random\\', model_config=model_cfg)\\n```\\n\\nRunning the above would push the model to `/resnet18-random` on the Hub. You can now share this model with your friends, or use it in your own code!\\n\\n## Loading a Model===== Document 3 =====\\nprocessor.push_to_hub(hub_model_id)\\ntrainer.push_to_hub(**kwargs)\\n```\\n\\n# 4. Inference\\n\\nNow comes the exciting part, using our fine-tuned model! In this section, we\\'ll show how you can load your model from the hub and use it for inference.===== Document 4 =====\\n--push_to_hub\\n```===== Document 5 =====\\n. The second way to upload a model, though, is to call model.push_to_hub(). So this is more of a once-off method - it\\'s not called regularly during training. You can just call this manually whenever you want to upload a model to the hub. So we recommend running this after the end of training, just to make sure that you have a commit message just to guarantee that this was the final version of the model at the end of training. And it just makes sure that you\\'re working with the definitive end-of-training model and not accidentally using a model that\\'s from a checkpoint somewhere along the way===== Document 6 =====\\nFinally, if you want, you can push your model up to the hub. Here, we\\'ll push it up if you specified `push_to_hub=True` in the training configuration. Note that in order to push to hub, you\\'ll have to have git-lfs installed and be logged into your Hugging Face account (which can be done via `huggingface-cli login`).\\n\\n```python\\nkwargs = {\\n \"finetuned_from\": model.config._name_or_path,\\n \"tasks\": \"image-classification\",\\n \"dataset\": \\'beans\\',\\n \"tags\": [\\'image-classification\\'],\\n}'}\n", + "\u001b[38;20m===== Output message of the LLM: =====\u001b[0m\n", + "\u001b[38;20mThought: I have found multiple ways to push a model to the Hub. In order to answer comprehensively, I will provide each code snippet that describes how to push a model to the Hub.\n", + "\n", + "Action: ```json\n", + "{\n", + " \"action\": \"final_answer\",\n", + " \"action_input\": {\n", + " \"answer\": \"There are multiple ways to push a model to the Hub. Here are a few examples using different libraries and functions:\\n\\nUsing the `api`:\\n```python\\napi.upload_folder(\\n repo_id=repo_id,\\n folder_path=repo_local_path,\\n path_in_repo=\\\".\\\",\\n)\\n\\nprint(\\\"Your model is pushed to the Hub. You can view your model here:\\\", repo_url)\\n```\\n\\nWith Transformers:\\n```python\\nfrom transformers import PushToHubCallback\\n\\n# Initialize the callback with the output directory,\\ntokenizer, and your Hub username and model name\\npush_to_hub_callback = PushToHubCallback(\\n output_dir=\\\"./your_model_save_path\\\",\\n tokenizer=tokenizer,\\n hub_model_id=\\\"your-username/my-awesome-model\\\"\\n)\\n\\n# Assuming `trainer` is your Trainer object\\ntrainer.add_callback(push_to_hub_callback)\\n```\\n\\nUsing `timm`:\\n```python\\nfrom timm.models.hub import push_to_hf_hub\\n\\n# Assuming `model` is your fine-tuned model\\nmodel_cfg = {\\\"labels\\\": [\\\"a\\\", \\\"b\\\", \\\"c\\\", \\\"d\\\"]}\\npush_to_hf_hub(model, 'resnet18-random', model_config=model_cfg)\\n```\\n\\nFor computer vision models, you can also use `push_to_hub`:\\n```python\\nprocessor.push_to_hub(hub_model_id)\\ntrainer.push_to_hub(**kwargs)\\n```\\n\\nYou can also manually push a model with `model.push_to_hub()`:\\n```python\\nmodel.push_to_hub()\\n```\\n\\nAdditionally, you can opt to push your model to the Hub at the end of training by specifying `push_to_hub=True` in the training configuration. Don't forget to have git-lfs installed and be logged into your Hugging Face account.\"\n", + " }\n", + "}\u001b[0m\n", + "\u001b[38;20m===== Extracting action =====\u001b[0m\n", + "\u001b[33;1mCalling tool: 'final_answer' with arguments: {'answer': \"There are multiple ways to push a model to the Hub. Here are a few examples using different libraries and functions:\\n\\nUsing the `api`:\\npython\\napi.upload_folder(\\n repo_id=repo_id,\\n folder_path=repo_local_path,\\n path_in_repo='.',\\n)\\n\\nprint('Your model is pushed to the Hub. You can view your model here:', repo_url)\\n\\n\\nWith Transformers:\\npython\\nfrom transformers import PushToHubCallback\\n\\n# Initialize the callback with the output directory,\\ntokenizer, and your Hub username and model name\\npush_to_hub_callback = PushToHubCallback(\\n output_dir='./your_model_save_path',\\n tokenizer=tokenizer,\\n hub_model_id='your-username/my-awesome-model'\\n)\\n\\n# Assuming `trainer` is your Trainer object\\ntrainer.add_callback(push_to_hub_callback)\\n\\n\\nUsing `timm`:\\npython\\nfrom timm.models.hub import push_to_hf_hub\\n\\n# Assuming `model` is your fine-tuned model\\nmodel_cfg = {'labels': ['a', 'b', 'c', 'd']}\\npush_to_hf_hub(model, 'resnet18-random', model_config=model_cfg)\\n\\n\\nFor computer vision models, you can also use `push_to_hub`:\\npython\\nprocessor.push_to_hub(hub_model_id)\\ntrainer.push_to_hub(**kwargs)\\n\\n\\nYou can also manually push a model with `model.push_to_hub()`:\\npython\\nmodel.push_to_hub()\\n\\n\\nAdditionally, you can opt to push your model to the Hub at the end of training by specifying `push_to_hub=True` in the training configuration. Don't forget to have git-lfs installed and be logged into your Hugging Face account.\"}\u001b[0m\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Final output:\n", + "There are multiple ways to push a model to the Hub. Here are a few examples using different libraries and functions:\n", + "\n", + "Using the `api`:\n", + "python\n", + "api.upload_folder(\n", + " repo_id=repo_id,\n", + " folder_path=repo_local_path,\n", + " path_in_repo='.',\n", + ")\n", + "\n", + "print('Your model is pushed to the Hub. You can view your model here:', repo_url)\n", + "\n", + "\n", + "With Transformers:\n", + "python\n", + "from transformers import PushToHubCallback\n", + "\n", + "# Initialize the callback with the output directory,\n", + "tokenizer, and your Hub username and model name\n", + "push_to_hub_callback = PushToHubCallback(\n", + " output_dir='./your_model_save_path',\n", + " tokenizer=tokenizer,\n", + " hub_model_id='your-username/my-awesome-model'\n", + ")\n", + "\n", + "# Assuming `trainer` is your Trainer object\n", + "trainer.add_callback(push_to_hub_callback)\n", + "\n", + "\n", + "Using `timm`:\n", + "python\n", + "from timm.models.hub import push_to_hf_hub\n", + "\n", + "# Assuming `model` is your fine-tuned model\n", + "model_cfg = {'labels': ['a', 'b', 'c', 'd']}\n", + "push_to_hf_hub(model, 'resnet18-random', model_config=model_cfg)\n", + "\n", + "\n", + "For computer vision models, you can also use `push_to_hub`:\n", + "python\n", + "processor.push_to_hub(hub_model_id)\n", + "trainer.push_to_hub(**kwargs)\n", + "\n", + "\n", + "You can also manually push a model with `model.push_to_hub()`:\n", + "python\n", + "model.push_to_hub()\n", + "\n", + "\n", + "Additionally, you can opt to push your model to the Hub at the end of training by specifying `push_to_hub=True` in the training configuration. Don't forget to have git-lfs installed and be logged into your Hugging Face account.\n" + ] + } + ], + "source": [ + "agent_output = agent.run(\"How can I push a model to the Hub?\")\n", + "\n", + "print(\"Final output:\")\n", + "print(agent_output)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Agentic RAG vs. standard RAG\n", + "\n", + "Does the agent setup make a better RAG system? Well, let's comapre it to a standard RAG system using LLM Judge!\n", + "\n", + "We will use [meta-llama/Meta-Llama-3-70B-Instruct](https://huggingface.co/meta-llama/Meta-Llama-3-70B-Instruct) for evaluation since it's one of the strongest OS models we tested for LLM judge use cases." + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "metadata": {}, + "outputs": [], + "source": [ + "eval_dataset = datasets.load_dataset(\"m-ric/huggingface_doc_qa_eval\", split=\"train\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Before running the test let's make the agent less verbose." + ] + }, + { + "cell_type": "code", + "execution_count": 13, + "metadata": {}, + "outputs": [], + "source": [ + "import logging\n", + "\n", + "agent.logger.setLevel(logging.WARNING)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "outputs_agentic_rag = []\n", + "\n", + "for example in tqdm(eval_dataset):\n", + " question = example[\"question\"]\n", + "\n", + " enhanced_question = f\"\"\"Using the information contained in your knowledge base, which you can access with the 'retriever' tool,\n", + "give a comprehensive answer to the question below.\n", + "Respond only to the question asked, response should be concise and relevant to the question.\n", + "If you cannot find information, do not give up and try calling your retriever again with different arguments!\n", + "Make sure to have covered the question completely by calling the retriever tool several times with semantically different queries.\n", + "Your queries should not be questions but affirmative form sentences: e.g. rather than \"How do I load a model from the Hub in bf16?\", query should be \"load a model from the Hub bf16 weights\".\n", + "\n", + "Question:\n", + "{question}\"\"\"\n", + " answer = agent.run(enhanced_question)\n", + " print(\"=======================================================\")\n", + " print(f\"Question: {question}\")\n", + " print(f\"Answer: {answer}\")\n", + " print(f'True answer: {example[\"answer\"]}')\n", + "\n", + " results_agentic = {\n", + " \"question\": question,\n", + " \"true_answer\": example[\"answer\"],\n", + " \"source_doc\": example[\"source_doc\"],\n", + " \"generated_answer\": answer,\n", + " }\n", + " outputs_agentic_rag.append(results_agentic)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "from huggingface_hub import InferenceClient\n", + "\n", + "reader_llm = InferenceClient(\"CohereForAI/c4ai-command-r-plus\")\n", + "\n", + "outputs_standard_rag = []\n", + "\n", + "for example in tqdm(eval_dataset):\n", + " question = example[\"question\"]\n", + " context = retriever_tool(question)\n", + "\n", + " prompt = f\"\"\"Given the question and supporting documents below, give a comprehensive answer to the question.\n", + "Respond only to the question asked, response should be concise and relevant to the question.\n", + "Provide the number of the source document when relevant.\n", + "If you cannot find information, do not give up and try calling your retriever again with different arguments!\n", + "\n", + "Question:\n", + "{question}\n", + "\n", + "{context}\n", + "\"\"\"\n", + " messages = [{\"role\": \"user\", \"content\": prompt}]\n", + " answer = reader_llm.chat_completion(messages).choices[0].message.content\n", + "\n", + " print(\"=======================================================\")\n", + " print(f\"Question: {question}\")\n", + " print(f\"Answer: {answer}\")\n", + " print(f'True answer: {example[\"answer\"]}')\n", + "\n", + " results_agentic = {\n", + " \"question\": question,\n", + " \"true_answer\": example[\"answer\"],\n", + " \"source_doc\": example[\"source_doc\"],\n", + " \"generated_answer\": answer,\n", + " }\n", + " outputs_standard_rag.append(results_agentic)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The evaluation prompt follows some of the best principles shown in [our llm_judge cookbook](llm_judge): it follows a small integer Likert scale, has clear criteria, and a description for each score." + ] + }, + { + "cell_type": "code", + "execution_count": 68, + "metadata": {}, + "outputs": [], + "source": [ + "EVALUATION_PROMPT = \"\"\"You are a fair evaluator language model.\n", + "\n", + "You will be given an instruction, a response to evaluate, a reference answer that gets a score of 3, and a score rubric representing a evaluation criteria are given.\n", + "1. Write a detailed feedback that assess the quality of the response strictly based on the given score rubric, not evaluating in general.\n", + "2. After writing a feedback, write a score that is an integer between 1 and 3. You should refer to the score rubric.\n", + "3. The output format should look as follows: \\\"Feedback: {{write a feedback for criteria}} [RESULT] {{an integer number between 1 and 3}}\\\"\n", + "4. Please do not generate any other opening, closing, and explanations. Be sure to include [RESULT] in your output.\n", + "5. Do not score conciseness: a correct answer that covers the question should receive max score, even if it contains additional useless information.\n", + "\n", + "The instruction to evaluate:\n", + "{instruction}\n", + "\n", + "Response to evaluate:\n", + "{response}\n", + "\n", + "Reference Answer (Score 3):\n", + "{reference_answer}\n", + "\n", + "Score Rubrics:\n", + "[Is the response complete, accurate, and factual based on the reference answer?]\n", + "Score 1: The response is completely incomplete, inaccurate, and/or not factual.\n", + "Score 2: The response is somewhat complete, accurate, and/or factual.\n", + "Score 3: The response is completely complete, accurate, and/or factual.\n", + "\n", + "Feedback:\"\"\"" + ] + }, + { + "cell_type": "code", + "execution_count": 69, + "metadata": {}, + "outputs": [], + "source": [ + "from huggingface_hub import InferenceClient\n", + "\n", + "evaluation_client = InferenceClient(\"meta-llama/Meta-Llama-3-70B-Instruct\")" + ] + }, + { + "cell_type": "code", + "execution_count": 70, + "metadata": {}, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "100%|██████████| 65/65 [02:24<00:00, 2.23s/it]\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Average score for agentic RAG: 78.5%\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "100%|██████████| 65/65 [02:17<00:00, 2.12s/it]\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Average score for standard RAG: 70.0%\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n" + ] + } + ], + "source": [ + "import pandas as pd\n", + "\n", + "for type, outputs in [\n", + " (\"agentic\", outputs_agentic_rag),\n", + " (\"standard\", outputs_standard_rag),\n", + "]:\n", + " for experiment in tqdm(outputs):\n", + " eval_prompt = EVALUATION_PROMPT.format(\n", + " instruction=experiment[\"question\"],\n", + " response=experiment[\"generated_answer\"],\n", + " reference_answer=experiment[\"true_answer\"],\n", + " )\n", + " messages = [\n", + " {\"role\": \"system\", \"content\": \"You are a fair evaluator language model.\"},\n", + " {\"role\": \"user\", \"content\": eval_prompt},\n", + " ]\n", + "\n", + " eval_result = evaluation_client.text_generation(\n", + " eval_prompt, max_new_tokens=1000\n", + " )\n", + " try:\n", + " feedback, score = [item.strip() for item in eval_result.split(\"[RESULT]\")]\n", + " experiment[\"eval_score_LLM_judge\"] = score\n", + " experiment[\"eval_feedback_LLM_judge\"] = feedback\n", + " except:\n", + " print(f\"Parsing failed - output was: {eval_result}\")\n", + "\n", + " results = pd.DataFrame.from_dict(outputs)\n", + " results = results.loc[~results[\"generated_answer\"].str.contains(\"Error\")]\n", + " results[\"eval_score_LLM_judge_int\"] = (\n", + " results[\"eval_score_LLM_judge\"].fillna(1).apply(lambda x: int(x))\n", + " )\n", + " results[\"eval_score_LLM_judge_int\"] = (results[\"eval_score_LLM_judge_int\"] - 1) / 2\n", + "\n", + " print(\n", + " f\"Average score for {type} RAG: {results['eval_score_LLM_judge_int'].mean()*100:.1f}%\"\n", + " )" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "**Let us recap: the Agent setup improves scores by 8.5% compared to a standard RAG!** (from 70.0% to 78.5%)\n", + "\n", + "This is a great improvement, with a very simple setup 🚀\n", + "\n", + "(For a baseline, using Llama-3-70B without the knowledge base got 36%)" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "disposable", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.12.0" + } + }, + "nbformat": 4, + "nbformat_minor": 2 +} From f480cc87ee2d6833f0130b81249a461f60299304 Mon Sep 17 00:00:00 2001 From: innovation64 Date: Tue, 9 Jul 2024 15:27:08 +0800 Subject: [PATCH 19/31] update agent rag cn version --- notebooks/zh-CN/agent_rag.ipynb | 78 ++++++++++++++++----------------- 1 file changed, 38 insertions(+), 40 deletions(-) diff --git a/notebooks/zh-CN/agent_rag.ipynb b/notebooks/zh-CN/agent_rag.ipynb index 549e8780..fedb8493 100644 --- a/notebooks/zh-CN/agent_rag.ipynb +++ b/notebooks/zh-CN/agent_rag.ipynb @@ -4,28 +4,30 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "# Agentic RAG: turbocharge your RAG with query reformulation and self-query! 🚀\n", - "_Authored by: [Aymeric Roucher](https://huggingface.co/m-ric)_\n", + "# 智能体 RAG：通过查询重写和自我查询来为你的 RAG 加速！🚀\n", "\n", - "> This tutorial is advanced. You should have notions from [this other cookbook](advanced_rag) first!\n", + "_作者: [Aymeric Roucher](https://huggingface.co/m-ric)_\n", "\n", - "> Reminder: Retrieval-Augmented-Generation (RAG) is “using an LLM to answer a user query, but basing the answer on information retrieved from a knowledge base”. It has many advantages over using a vanilla or fine-tuned LLM: to name a few, it allows to ground the answer on true facts and reduce confabulations, it allows to provide the LLM with domain-specific knowledge, and it allows fine-grained control of access to information from the knowledge base.\n", + ">这个教程比较高级，建议你先看看另一个更基础的教程。\n", "\n", - "But vanilla RAG has limitations, most importantly these two:\n", - "- It **performs only one retrieval step**: if the results are bad, the generation in turn will be bad.\n", - "- __Semantic similarity is computed with the *user query* as a reference__, which might be suboptimal: for instance, the user query will often be a question and the document containing the true answer will be in affirmative voice, so its similarity score will be downgraded compared to other source documents in the interrogative form, leading to a risk of missing the relevant information.\n", + ">检索增强生成（RAG）是一种用大型语言模型（LLM）来回答问题的方法，但它会先从知识库中查找相关信息。这种方法比只用大型语言模型有很多好处，比如可以基于真实的事实来回答问题，减少虚构内容，还可以让模型获取特定领域的知识，并且可以精确控制模型从知识库中获取信息。\n", "\n", - "But we can alleviate these problems by making a **RAG agent: very simply, an agent armed with a retriever tool!**\n", + "不过，普通的RAG方法有两个主要问题：\n", "\n", - "This agent will: ✅ Formulate the query itself and ✅ Critique to re-retrieve if needed.\n", + "- 它只进行**一次信息检索**，如果检索的结果不好，那么回答也会差。\n", + "- 它计算**语义相似性时是以用户的提问为参照**，这可能不太理想。比如，用户提出的问题通常是用疑问句，而包含答案的文档通常是陈述句，这样就会导致真正含有答案的文档和用户提问的相似性得分不高，可能会错过重要的信息。\n", "\n", - "So it should naively recover some advanced RAG techniques!\n", - "- Instead of directly using the user query as the reference in semantic search, the agent formulates itself a reference sentence that can be closer to the targeted documents, as in [HyDE](https://huggingface.co/papers/2212.10496)\n", - "- The agent can the generated snippets and re-retrieve if needed, as in [Self-Query](https://docs.llamaindex.ai/en/stable/examples/evaluation/RetryQuery/)\n", + "为了解决这些问题，我们可以创建**一个带有检索功能的 RAG 智能体。**\n", "\n", - "Let's build this system. 🛠️\n", + "这个智能体可以 ✅ 自己构建查询，并且 ✅ 在需要的时候重新检索信息。\n", "\n", - "Run the line below to install required dependencies:" + "所以，我们得用点高级的RAG技术！\n", + "- 不直接用用户的提问去搜索，我们自己编一个更接近想要找的资料的句子，就像 [HyDE](https://huggingface.co/papers/2212.10496) 那样\n", + "- 如果需要，我们可以看看已经找到的信息，然后再去找一次，就像 [Self-Query](https://docs.llamaindex.ai/en/stable/examples/evaluation/RetryQuery/) 那样\n", + "\n", + "咱们开始做这个系统吧。🛠️\n", + "\n", + "运行下面的命令来安装所需的软件包：\n" ] }, { @@ -41,7 +43,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "We first load a knowledge base on which we want to perform RAG: this dataset is a compilation of the documentation pages for many `huggingface` packages, stored as markdown." + "我们首先加载一个知识库，以便在其上执行 RAG：这个数据集是许多 `huggingface` 软件包的文档页面的汇总，以 markdown 格式存储。" ] }, { @@ -68,10 +70,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Now we prepare the knowledge base by processing the dataset and storing it into a vector database to be used by the retriever.\n", - "\n", - "We use [LangChain](https://python.langchain.com/) for its excellent vector database utilities.\n", - "For the embedding model, we use [thenlper/gte-small](https://huggingface.co/thenlper/gte-small) since it performed well in our `RAG_evaluation` cookbook." + "现在我们通过处理数据集并将其存储到向量数据库中，为检索器准备知识库。我们使用 [LangChain](https://python.langchain.com/)，因为它具有出色的向量数据库工具。对于嵌入模型，我们使用 [thenlper/gte-small](https://huggingface.co/thenlper/gte-small)，因为它在我们的 `RAG_evaluation` cookbook 中表现良好。" ] }, { @@ -152,9 +151,9 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Now the database is ready: let’s build our agentic RAG system!\n", + "现在数据库已经准备好了：让我们构建我们的智能体 RAG 系统吧！\n", "\n", - "👉 We only need a `RetrieverTool` that our agent can leverage to retrieve information from the knowledge base." + "👉 我们只需要一个 `RetrieverTool`，我们的智能体可以利用它从知识库中检索信息。\n" ] }, { @@ -202,17 +201,16 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Now it’s straightforward to create an agent that leverages this tool!\n", - "\n", - "The agent will need these arguments upon initialization:\n", - "- *`tools`*: a list of tools that the agent will be able to call.\n", - "- *`llm_engine`*: the LLM that powers the agent.\n", + "现在创建一个利用这个工具的智能体就简单了！\n", "\n", - "Our `llm_engine` must be a callable that takes as input a list of [messages](https://huggingface.co/docs/transformers/main/chat_templating) and returns text. It also needs to accept a `stop_sequences` argument that indicates when to stop its generation. For convenience, we directly use the `HfEngine` class provided in the package to get a LLM engine that calls our [Inference API](https://huggingface.co/docs/api-inference/en/index).\n", + "智能体在初始化时需要以下参数：\n", + "- *`tools`*：智能体能够调用的工具列表。\n", + "- *`llm_engine`*：为智能体提供动力的LLM。\n", "\n", - "And we use [CohereForAI/c4ai-command-r-plus](https://huggingface.co/CohereForAI/c4ai-command-r-plus) as the llm engine because:\n", - "- It has a long 128k context, which is helpful for processing long source documents\n", - "- It is served for free at all times on HF's Inference API!" + "我们的 `llm_engine` 必须是一个可调用的对象，它接受一个 [messages](https://huggingface.co/docs/transformers/main/chat_templating) 列表作为输入并返回文本。它还需要接受一个 `stop_sequences` 参数，该参数指示何时停止生成。为了方便起见，我们直接使用包中提供的 `HfEngine` 类来获取一个调用我们的 [Inference API](https://huggingface.co/docs/api-inference/en/index) 的LLM引擎。\n", + "我们使用 [CohereForAI/c4ai-command-r-plus](https://huggingface.co/CohereForAI/c4ai-command-r-plus) 作为 llm 引擎，因为：\n", + "- 它有一个长达 128k 的上下文，这对于处理长源文档很有帮助\n", + "- 它在 HF 的 Inference API 上始终免费提供！\n" ] }, { @@ -235,9 +233,9 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Since we initialized the agent as a `ReactJsonAgent`, it has been automatically given a default system prompt that tells the LLM engine to process step-by-step and generate tool calls as JSON blobs (you could replace this prompt template with your own as needed).\n", + "既然我们已经将智能体初始化为 `ReactJsonAgent`，它就已经自动赋予了一个默认的系统提示，告诉 LLM 引擎要逐步处理并生成工具调用作为 JSON 块（你可以根据需要用你自己的提示模板替换这个）。\n", "\n", - "Then when its `.run()` method is launched, the agent takes care of calling the LLM engine, parsing the tool call JSON blobs and executing these tool calls, all in a loop that ends only when the final answer is provided." + "然后，当它的 `.run()` 方法被启动时，智能体负责调用 LLM 引擎，解析工具调用的 JSON 块并执行这些工具调用，所有这些都在一个循环中进行，只有当提供最终答案时才会结束。" ] }, { @@ -547,11 +545,11 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "## Agentic RAG vs. standard RAG\n", + "## 智能体RAG与标准RAG的比较\n", "\n", - "Does the agent setup make a better RAG system? Well, let's comapre it to a standard RAG system using LLM Judge!\n", + "智能体 RAG 和标准 RAG，哪个更好？我们用 LLM Judge 来比一比。\n", "\n", - "We will use [meta-llama/Meta-Llama-3-70B-Instruct](https://huggingface.co/meta-llama/Meta-Llama-3-70B-Instruct) for evaluation since it's one of the strongest OS models we tested for LLM judge use cases." + "我们会用一个非常强的模型 [meta-llama/Meta-Llama-3-70B-Instruct](https://huggingface.co/meta-llama/Meta-Llama-3-70B-Instruct) 来做这个评估。\n" ] }, { @@ -567,7 +565,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Before running the test let's make the agent less verbose." + "在运行测试之前，让我们让智能体输出更简洁一些。" ] }, { @@ -663,7 +661,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "The evaluation prompt follows some of the best principles shown in [our llm_judge cookbook](llm_judge): it follows a small integer Likert scale, has clear criteria, and a description for each score." + "评估提示遵循了[我们的 llm_judge cookbook](llm_judge) 中展示的一些最佳原则：它遵循一个小的整数李克特量表，有明确的评分标准和每个分数的描述。" ] }, { @@ -795,11 +793,11 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "**Let us recap: the Agent setup improves scores by 8.5% compared to a standard RAG!** (from 70.0% to 78.5%)\n", + "**让我们回顾一下：与标准的 RAG 相比，智能体设置提高了 8.5% 的得分！**（从 70.0% 提高到 78.5%）\n", "\n", - "This is a great improvement, with a very simple setup 🚀\n", + "这是一个巨大的改进，而且设置非常简单🚀\n", "\n", - "(For a baseline, using Llama-3-70B without the knowledge base got 36%)" + "（作为基准，不使用知识库的 Llama-3-70B 得分为 36%）\n" ] } ], From 78600f5049eec3f65a8543c4cad4d43cacce57b7 Mon Sep 17 00:00:00 2001 From: innovation64 Date: Tue, 9 Jul 2024 15:32:58 +0800 Subject: [PATCH 20/31] update toctree.yaml cn version --- notebooks/zh-CN/_toctree.yml | 22 ++++++---------------- 1 file changed, 6 insertions(+), 16 deletions(-) diff --git a/notebooks/zh-CN/_toctree.yml b/notebooks/zh-CN/_toctree.yml index 43aa8d33..2d11f28e 100644 --- a/notebooks/zh-CN/_toctree.yml +++ b/notebooks/zh-CN/_toctree.yml @@ -3,7 +3,7 @@ sections: - local: index title: 开源 AI 指南 (Cookbook) -- title: LLM 配方 +- title: LLM 系列 sections: - local: automatic_embedding_tei_inference_endpoints title: 通过推理端点使用 TEI 自动嵌入 @@ -22,24 +22,20 @@ sections: - local: llm_judge title: 使用 LLM 作为评判者🧑‍⚖️进行自动化和多方面的评估 -- title: Diffusion 配方 +- title: Diffusion 系列 sections: - local: stable_diffusion_interpolation title: 使用 Stable Diffusion 进行图像插值 -- title: 多模态配方 +- title: 多模态系列 sections: - - local: analyzing_art_with_hf_and_fiftyone - title: 使用多模态嵌入分析艺术风格 - local: faiss_with_hf_datasets_and_clip title: 用 🤗 transformers, 🤗 datasets 和 FAISS 嵌入多模态数据进行相似度搜索 -- title: 使用其他库的 LLM 和 RAG 配方 +- title: 使用其他库的 LLM 和 RAG 系列 sections: - local: issues_in_text_dataset title: 使用 Cleanlab 检测文本数据集中的问题 - - local: annotate_text_data_transformers_via_active_learning - title: 使用 Cleanlab 和 Active Learning 标注文本数据 - local: rag_with_hugging_face_gemma_mongodb title: 用 Gemma, MongoDB 和开源模型构建 RAG 系统 - local: rag_zephyr_langchain @@ -50,8 +46,6 @@ sections: title: 创建一个合法偏好数据集 - local: semantic_cache_chroma_vector_database title: 通过引入语义缓存到 FAISS 中以增强 RAG 系统的性能 - - local: structured_generation - title: 使用结构化生成在 RAG 系统中进行源高亮 - title: 计算机视觉 sections: @@ -60,9 +54,5 @@ sections: - title: 智能体 sections: - - local: agents - title: 使用 Transformers Agents 构建具有工具调用超能力的代理 - -- title: 企业 hub 指南 - sections: - - local: enterprise_cookbook_overview + - local: agent_rag + title: 智能体 RAG 通过查询重写和自我查询来为你的 RAG 加速！ From 9ba380823c701d9578efcd588be46545dfd09590 Mon Sep 17 00:00:00 2001 From: innovation64 Date: Tue, 9 Jul 2024 15:39:49 +0800 Subject: [PATCH 21/31] update some bug --- notebooks/zh-CN/agent_rag.ipynb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/notebooks/zh-CN/agent_rag.ipynb b/notebooks/zh-CN/agent_rag.ipynb index fedb8493..88743d7f 100644 --- a/notebooks/zh-CN/agent_rag.ipynb +++ b/notebooks/zh-CN/agent_rag.ipynb @@ -8,7 +8,7 @@ "\n", "_作者: [Aymeric Roucher](https://huggingface.co/m-ric)_\n", "\n", - ">这个教程比较高级，建议你先看看另一个更基础的教程。\n", + ">这个教程比较高级，建议你先看看另一个[更基础的教程](advanced_rag)。\n", "\n", ">检索增强生成（RAG）是一种用大型语言模型（LLM）来回答问题的方法，但它会先从知识库中查找相关信息。这种方法比只用大型语言模型有很多好处，比如可以基于真实的事实来回答问题，减少虚构内容，还可以让模型获取特定领域的知识，并且可以精确控制模型从知识库中获取信息。\n", "\n", From a9167879d026c186dd9987d9cac7ec0f00288d92 Mon Sep 17 00:00:00 2001 From: innovation64 Date: Tue, 9 Jul 2024 19:05:27 +0800 Subject: [PATCH 22/31] update some bug --- notebooks/zh-CN/_toctree.yml | 2 +- notebooks/zh-CN/agent_rag.ipynb | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/notebooks/zh-CN/_toctree.yml b/notebooks/zh-CN/_toctree.yml index 2d11f28e..074ba8b1 100644 --- a/notebooks/zh-CN/_toctree.yml +++ b/notebooks/zh-CN/_toctree.yml @@ -55,4 +55,4 @@ sections: - title: 智能体 sections: - local: agent_rag - title: 智能体 RAG 通过查询重写和自我查询来为你的 RAG 加速！ + title: 智能体 RAG 通过查询重构和自查询来增强你的 RAG diff --git a/notebooks/zh-CN/agent_rag.ipynb b/notebooks/zh-CN/agent_rag.ipynb index 88743d7f..3a59215e 100644 --- a/notebooks/zh-CN/agent_rag.ipynb +++ b/notebooks/zh-CN/agent_rag.ipynb @@ -4,7 +4,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "# 智能体 RAG：通过查询重写和自我查询来为你的 RAG 加速！🚀\n", + "# 智能体 RAG：通过查询重构和自查询来增强你的 RAG ！🚀\n", "\n", "_作者: [Aymeric Roucher](https://huggingface.co/m-ric)_\n", "\n", From 561197c789febf2d7ce84042a070800921420c0d Mon Sep 17 00:00:00 2001 From: innovation64 Date: Tue, 9 Jul 2024 19:23:51 +0800 Subject: [PATCH 23/31] update some bug --- notebooks/zh-CN/agent_rag.ipynb | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-) diff --git a/notebooks/zh-CN/agent_rag.ipynb b/notebooks/zh-CN/agent_rag.ipynb index 3a59215e..c6a132e4 100644 --- a/notebooks/zh-CN/agent_rag.ipynb +++ b/notebooks/zh-CN/agent_rag.ipynb @@ -21,11 +21,12 @@ "\n", "这个智能体可以 ✅ 自己构建查询，并且 ✅ 在需要的时候重新检索信息。\n", "\n", - "所以，我们得用点高级的RAG技术！\n", - "- 不直接用用户的提问去搜索，我们自己编一个更接近想要找的资料的句子，就像 [HyDE](https://huggingface.co/papers/2212.10496) 那样\n", - "- 如果需要，我们可以看看已经找到的信息，然后再去找一次，就像 [Self-Query](https://docs.llamaindex.ai/en/stable/examples/evaluation/RetryQuery/) 那样\n", + "所以，我们得用点高级的 RAG 技术！\n", "\n", - "咱们开始做这个系统吧。🛠️\n", + "- 不直接使用用户的提问去搜索，而是智能体自行制定一个更接近目标文档的参考句子，就像 [HyDE](https://huggingface.co/papers/2212.10496) 那样\n", + "- 智能体能生成片段并在需要时重新检索，就像 [Self-Query](https://docs.llamaindex.ai/en/stable/examples/evaluation/RetryQuery/) 那样\n", + "\n", + "让我们开始做这个系统吧。🛠️\n", "\n", "运行下面的命令来安装所需的软件包：\n" ] @@ -70,7 +71,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "现在我们通过处理数据集并将其存储到向量数据库中，为检索器准备知识库。我们使用 [LangChain](https://python.langchain.com/)，因为它具有出色的向量数据库工具。对于嵌入模型，我们使用 [thenlper/gte-small](https://huggingface.co/thenlper/gte-small)，因为它在我们的 `RAG_evaluation` cookbook 中表现良好。" + "现在我们通过处理数据集并将其存储到向量数据库中，为检索器准备知识库。我们使用 [LangChain](https://python.langchain.com/)，因为它具有出色的向量数据库工具。对于嵌入模型，我们使用 [thenlper/gte-small](https://huggingface.co/thenlper/gte-small)，因为它在我们的 `RAG_evaluation` 指南中表现良好。" ] }, { @@ -207,7 +208,7 @@ "- *`tools`*：智能体能够调用的工具列表。\n", "- *`llm_engine`*：为智能体提供动力的LLM。\n", "\n", - "我们的 `llm_engine` 必须是一个可调用的对象，它接受一个 [messages](https://huggingface.co/docs/transformers/main/chat_templating) 列表作为输入并返回文本。它还需要接受一个 `stop_sequences` 参数，该参数指示何时停止生成。为了方便起见，我们直接使用包中提供的 `HfEngine` 类来获取一个调用我们的 [Inference API](https://huggingface.co/docs/api-inference/en/index) 的LLM引擎。\n", + "我们的 `llm_engine` 必须是一个可调用的对象，它接受一个 [messages](https://huggingface.co/docs/transformers/main/chat_templating) 列表作为输入并返回文本。它还需要接受一个 `stop_sequences` 参数，该参数指示何时停止生成。为了方便起见，我们直接使用包中提供的 `HfEngine` 类来获取一个调用我们的 [Inference API](https://huggingface.co/docs/api-inference/en/index) 的 LLM 引擎。\n", "我们使用 [CohereForAI/c4ai-command-r-plus](https://huggingface.co/CohereForAI/c4ai-command-r-plus) 作为 llm 引擎，因为：\n", "- 它有一个长达 128k 的上下文，这对于处理长源文档很有帮助\n", "- 它在 HF 的 Inference API 上始终免费提供！\n" From 58fc505dce43db89b6d6e8b6180535548ec1e850 Mon Sep 17 00:00:00 2001 From: innovation64 Date: Thu, 11 Jul 2024 15:02:25 +0800 Subject: [PATCH 24/31] update agents cn version --- notebooks/zh-CN/_toctree.yml | 2 + notebooks/zh-CN/agents.ipynb | 726 +++++++++++++++++++++++++++++++++++ 2 files changed, 728 insertions(+) create mode 100644 notebooks/zh-CN/agents.ipynb diff --git a/notebooks/zh-CN/_toctree.yml b/notebooks/zh-CN/_toctree.yml index 074ba8b1..df668dec 100644 --- a/notebooks/zh-CN/_toctree.yml +++ b/notebooks/zh-CN/_toctree.yml @@ -54,5 +54,7 @@ sections: - title: 智能体 sections: + - local: agents + title: 使用 Transformers Agents 构建具有工具调用超能力的智能体 - local: agent_rag title: 智能体 RAG 通过查询重构和自查询来增强你的 RAG diff --git a/notebooks/zh-CN/agents.ipynb b/notebooks/zh-CN/agents.ipynb new file mode 100644 index 00000000..0a48deab --- /dev/null +++ b/notebooks/zh-CN/agents.ipynb @@ -0,0 +1,726 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# 使用 Transformers Agents 构建具有工具调用超能力的智能体 🦸\n", + "\n", + "_作者: [Aymeric Roucher](https://huggingface.co/m-ric)_\n", + "\n", + "\n", + "这个 notebook 展示了如何使用 [**Transformers Agents**](https://huggingface.co/docs/transformers/en/agents) 来构建出色的**智能体**！\n", + "\n", + "什么是**智能体**？智能体是由大型语言模型（LLM）驱动的系统，它们使得 LLM（通过精心设计的提示和输出解析）能够使用特定的*工具*来解决问题。\n", + "\n", + "这些*工具*基本上是 LLM 自身无法很好执行的功能：例如，对于像 [Llama-3-70B](https://huggingface.co/meta-llama/Meta-Llama-3-70B-Instruct) 这样的文本生成 LLM，这可能是一个图像生成工具、网络搜索工具、计算器...\n", + "\n", + "什么是 **Transformers Agents** ？它是我们 `transformers` 库的一个扩展，提供了构建自己的智能体的构建块！在[文档](https://huggingface.co/docs/transformers/en/agents)中了解更多信息。\n", + "\n", + "让我们看看如何使用它，以及它能解决哪些用例。\n", + "\n", + "我们从源代码安装 transformers agents ，你可以使用 `pip install transformers[agents]` 轻松安装。\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "!pip install \"git+https://github.com/huggingface/transformers.git#egg=transformers[agents]\"" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "!pip install datasets huggingface_hub langchain sentence-transformers faiss-cpu serpapi google-search-results openai -q" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 1. 🏞️ 多模态 + 🌐 网络浏览助手\n", + "\n", + "对于这个用例，我们想要展示一个能够浏览网络并能够生成图像的智能体。\n", + "\n", + "为了构建它，我们只需要准备两个工具：图像生成和网络搜索。\n", + "- 对于图像生成，我们从 Hub 加载一个工具，该工具使用 HF 推理 API（无服务器）使用 Stable Diffusion 生成图像。\n", + "- 对于网络搜索，我们加载一个 LangChain 工具。" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": {}, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\u001b[33;1m======== New task ========\u001b[0m\n", + "\u001b[37;1mGenerate me a photo of the car that James bond drove in the latest movie.\u001b[0m\n", + "\u001b[33;1m==== Agent is executing the code below:\u001b[0m\n", + "\u001b[0m\u001b[38;5;7mlatest_movie\u001b[39m\u001b[38;5;7m \u001b[39m\u001b[38;5;109;01m=\u001b[39;00m\u001b[38;5;7m \u001b[39m\u001b[38;5;7msearch\u001b[39m\u001b[38;5;7m(\u001b[39m\u001b[38;5;144m\"\u001b[39m\u001b[38;5;144mWhat is the latest James Bond movie?\u001b[39m\u001b[38;5;144m\"\u001b[39m\u001b[38;5;7m)\u001b[39m\n", + "\u001b[38;5;109mprint\u001b[39m\u001b[38;5;7m(\u001b[39m\u001b[38;5;144m\"\u001b[39m\u001b[38;5;144mLatest James Bond movie:\u001b[39m\u001b[38;5;144m\"\u001b[39m\u001b[38;5;7m,\u001b[39m\u001b[38;5;7m \u001b[39m\u001b[38;5;7mlatest_movie\u001b[39m\u001b[38;5;7m)\u001b[39m\n", + "\u001b[38;5;7mbond_car\u001b[39m\u001b[38;5;7m \u001b[39m\u001b[38;5;109;01m=\u001b[39;00m\u001b[38;5;7m \u001b[39m\u001b[38;5;7msearch\u001b[39m\u001b[38;5;7m(\u001b[39m\u001b[38;5;144m\"\u001b[39m\u001b[38;5;144mWhat car did James Bond drive in the latest movie?\u001b[39m\u001b[38;5;144m\"\u001b[39m\u001b[38;5;7m)\u001b[39m\n", + "\u001b[38;5;109mprint\u001b[39m\u001b[38;5;7m(\u001b[39m\u001b[38;5;144m\"\u001b[39m\u001b[38;5;144mJames Bond\u001b[39m\u001b[38;5;144m'\u001b[39m\u001b[38;5;144ms car:\u001b[39m\u001b[38;5;144m\"\u001b[39m\u001b[38;5;7m,\u001b[39m\u001b[38;5;7m \u001b[39m\u001b[38;5;7mbond_car\u001b[39m\u001b[38;5;7m)\u001b[39m\u001b[0m\n", + "\u001b[33;1m====\u001b[0m\n", + "\u001b[33;1mPrint outputs:\u001b[0m\n", + "\u001b[32;20mLatest James Bond movie: No Time to Die\n", + "James Bond's car: Aston Martin DB5\n", + "\u001b[0m\n", + "\u001b[33;1m==== Agent is executing the code below:\u001b[0m\n", + "\u001b[0m\u001b[38;5;7mimage\u001b[39m\u001b[38;5;7m \u001b[39m\u001b[38;5;109;01m=\u001b[39;00m\u001b[38;5;7m \u001b[39m\u001b[38;5;7mimage_generator\u001b[39m\u001b[38;5;7m(\u001b[39m\u001b[38;5;144m\"\u001b[39m\u001b[38;5;144mA high-res, photorealistic image of the Aston Martin DB5 driven by James Bond in No Time to Die\u001b[39m\u001b[38;5;144m\"\u001b[39m\u001b[38;5;7m)\u001b[39m\n", + "\u001b[38;5;7mfinal_answer\u001b[39m\u001b[38;5;7m(\u001b[39m\u001b[38;5;7mimage\u001b[39m\u001b[38;5;7m)\u001b[39m\u001b[0m\n", + "\u001b[33;1m====\u001b[0m\n", + "\u001b[33;1mPrint outputs:\u001b[0m\n", + "\u001b[32;20m\u001b[0m\n", + "\u001b[33;1m>>> Final answer:\u001b[0m\n", + "\u001b[32;20m/var/folders/6m/9b1tts6d5w960j80wbw9tx3m0000gn/T/tmptcdd2ra6/2bf48fc0-6fff-4e86-8fb5-85b3221bc0c8.png\u001b[0m\n" + ] + } + ], + "source": [ + "from transformers import Tool, load_tool, ReactCodeAgent, HfEngine\n", + "\n", + "# Import tool from Hub\n", + "image_generation_tool = load_tool(\"m-ric/text-to-image\")\n", + "\n", + "# Import tool from LangChain\n", + "from langchain.agents import load_tools\n", + "\n", + "search_tool = Tool.from_langchain(load_tools([\"serpapi\"])[0])\n", + "\n", + "\n", + "llm_engine = HfEngine(\"meta-llama/Meta-Llama-3-70B-Instruct\")\n", + "# Initialize the agent with both tools\n", + "agent = ReactCodeAgent(\n", + " tools=[image_generation_tool, search_tool], llm_engine=llm_engine\n", + ")\n", + "\n", + "# Run it!\n", + "result = agent.run(\n", + " \"Generate me a photo of the car that James bond drove in the latest movie.\",\n", + ")\n", + "result" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "![Image of an Aston Martin DB5](https://huggingface.co/datasets/huggingface/cookbook-images/resolve/main/agents_db5.png)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 2. 📚💬 带有迭代查询优化和来源选择的 RAG\n", + "快速定义：检索增强生成（RAG）是 ___“使用大型语言模型（LLM）来回答用户查询，但基于从知识库检索到的信息来构建答案”___。\n", + "\n", + "这种方法相比使用普通或微调的 LLM 有许多优势：列举一些，它允许将答案建立在真实事实的基础上并减少虚构，它允许为 LLM 提供特定领域的知识，并且它允许对知识库中的信息访问进行细粒度控制。\n", + "\n", + "- 现在假设我们想要执行 RAG，但增加了动态生成某些参数的约束。例如，根据用户查询，我们可能想要将搜索限制在知识库的特定子集，或者我们可能想要调整检索到的文档数量。难点在于：**如何根据用户查询动态调整这些参数？**\n", + "\n", + "- RAG 的一个常见失败案例是基于用户查询的检索没有返回任何相关的支持文档。**有没有一种方法，在之前的结果不相关时，通过修改查询重新调用检索器来进行迭代？**\n", + "\n", + "🔧 好吧，我们可以以简单的方式解决上述问题：我们将**让我们的智能体控制检索器的参数！**\n", + "\n", + "➡️ 让我们展示如何做到这一点。我们首先加载一个我们想要执行 RAG 的知识库：这个数据集是许多 `huggingface` 包的文档页面汇总，以 markdown 格式存储。\n", + "\n" + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "metadata": {}, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "/Users/aymeric/.pyenv/versions/3.12.0/lib/python3.12/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html\n", + " from .autonotebook import tqdm as notebook_tqdm\n" + ] + } + ], + "source": [ + "import datasets\n", + "\n", + "knowledge_base = datasets.load_dataset(\"m-ric/huggingface_doc\", split=\"train\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "现在我们通过处理数据集并将其存储到向量数据库中来准备知识库，以便检索器使用。我们将使用 LangChain，因为它具有用于向量数据库的优秀工具：\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "from langchain.docstore.document import Document\n", + "from langchain.text_splitter import RecursiveCharacterTextSplitter\n", + "from langchain.vectorstores import FAISS\n", + "from langchain_community.embeddings import HuggingFaceEmbeddings\n", + "\n", + "source_docs = [\n", + " Document(page_content=doc[\"text\"], metadata={\"source\": doc[\"source\"].split(\"/\")[1]})\n", + " for doc in knowledge_base\n", + "]\n", + "\n", + "docs_processed = RecursiveCharacterTextSplitter(chunk_size=500).split_documents(\n", + " source_docs\n", + ")[:1000]\n", + "\n", + "embedding_model = HuggingFaceEmbeddings(model_name=\"thenlper/gte-small\")\n", + "vectordb = FAISS.from_documents(documents=docs_processed, embedding=embedding_model)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "现在我们已经准备好了数据库，让我们构建一个基于它回答用户查询的 RAG 系统！\n", + "\n", + "我们希望我们的系统根据查询只从最相关的信息来源中选择。\n", + "\n", + "我们的文档页面来自以下来源：" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "['evaluate', 'course', 'deep-rl-class', 'peft', 'hf-endpoints-documentation', 'blog', 'gradio', 'datasets', 'datasets-server', 'transformers', 'optimum', 'hub-docs', 'pytorch-image-models', 'diffusers']\n" + ] + } + ], + "source": [ + "all_sources = list(set([doc.metadata[\"source\"] for doc in docs_processed]))\n", + "print(all_sources)" + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "metadata": {}, + "outputs": [], + "source": [ + "import json\n", + "from transformers.agents import Tool\n", + "from langchain_core.vectorstores import VectorStore\n", + "\n", + "\n", + "class RetrieverTool(Tool):\n", + " name = \"retriever\"\n", + " description = \"Retrieves some documents from the knowledge base that have the closest embeddings to the input query.\"\n", + " inputs = {\n", + " \"query\": {\n", + " \"type\": \"text\",\n", + " \"description\": \"The query to perform. This should be semantically close to your target documents. Use the affirmative form rather than a question.\",\n", + " },\n", + " \"source\": {\"type\": \"text\", \"description\": \"\"},\n", + " \"number_of_documents\": {\n", + " \"type\": \"text\",\n", + " \"description\": \"the number of documents to retrieve. Stay under 10 to avoid drowning in docs\",\n", + " },\n", + " }\n", + " output_type = \"text\"\n", + "\n", + " def __init__(self, vectordb: VectorStore, all_sources: str, **kwargs):\n", + " super().__init__(**kwargs)\n", + " self.vectordb = vectordb\n", + " self.inputs[\"source\"][\n", + " \"description\"\n", + " ] = f\"The source of the documents to search, as a str representation of a list. Possible values in the list are: {all_sources}. If this argument is not provided, all sources will be searched.\"\n", + "\n", + " def forward(self, query: str, source: str = None, number_of_documents=7) -> str:\n", + " assert isinstance(query, str), \"Your search query must be a string\"\n", + " number_of_documents = int(number_of_documents)\n", + "\n", + " if source:\n", + " if isinstance(source, str) and \"[\" not in str(\n", + " source\n", + " ): # if the source is not representing a list\n", + " source = [source]\n", + " source = json.loads(str(source).replace(\"'\", '\"'))\n", + "\n", + " docs = self.vectordb.similarity_search(\n", + " query,\n", + " filter=({\"source\": source} if source else None),\n", + " k=number_of_documents,\n", + " )\n", + "\n", + " if len(docs) == 0:\n", + " return \"No documents found with this filtering. Try removing the source filter.\"\n", + " return \"Retrieved documents:\\n\\n\" + \"\\n===Document===\\n\".join(\n", + " [doc.page_content for doc in docs]\n", + " )" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### 可选：将你的检索器工具分享到 Hub\n", + "\n", + "要将你的工具分享到 Hub，首先将检索器工具定义单元格中的代码复制粘贴到一个名为例如 `retriever.py` 的新文件中。\n", + "\n", + "当工具从单独的文件加载后，你可以使用以下代码将其推送到 Hub（确保使用具有`写入`访问权限的 token 登录）" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "share_to_hub = False\n", + "\n", + "if share_to_hub:\n", + " from huggingface_hub import login\n", + " from retriever import RetrieverTool\n", + "\n", + " login(\"your_token\")\n", + "\n", + " tool = RetrieverTool(vectordb, all_sources)\n", + "\n", + " tool.push_to_hub(repo_id=\"m-ric/retriever-tool\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### 运行智能体!" + ] + }, + { + "cell_type": "code", + "execution_count": 38, + "metadata": {}, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "A new version of the following files was downloaded from https://huggingface.co/spaces/m-ric/retriever-tool:\n", + "- retriever.py\n", + ". Make sure to double-check they do not contain any added malicious code. To avoid downloading new versions of the code file, you can pin a revision.\n", + "\u001b[33;1m======== New task ========\u001b[0m\n", + "\u001b[37;1mPlease show me a LORA finetuning script\u001b[0m\n", + "\u001b[33;1mCalling tool: 'retriever' with arguments: {'number_of_documents': '5', 'query': 'LORA finetuning script', 'source': \"['transformers', 'blog']\"}\u001b[0m\n", + "\u001b[33;1mCalling tool: 'retriever' with arguments: {'number_of_documents': '5', 'query': 'LORA finetuning script'}\u001b[0m\n", + "\u001b[33;1mCalling tool: 'retriever' with arguments: {'number_of_documents': '5', 'query': 'train_text_to_image_lora.py'}\u001b[0m\n", + "\u001b[33;1mCalling tool: 'final_answer' with arguments: https://github.com/huggingface/diffusers/blob/main/examples/text_to_image/train_text_to_image_lora.py\u001b[0m\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Final output:\n", + "https://github.com/huggingface/diffusers/blob/main/examples/text_to_image/train_text_to_image_lora.py\n" + ] + } + ], + "source": [ + "from transformers.agents import HfEngine, ReactJsonAgent, load_tool\n", + "\n", + "llm_engine = HfEngine(\"meta-llama/Meta-Llama-3-70B-Instruct\")\n", + "\n", + "retriever_tool = load_tool(\n", + " \"m-ric/retriever-tool\", vectordb=vectordb, all_sources=all_sources\n", + ")\n", + "agent = ReactJsonAgent(tools=[retriever_tool], llm_engine=llm_engine, verbose=0)\n", + "\n", + "agent_output = agent.run(\"Please show me a LORA finetuning script\")\n", + "\n", + "print(\"Final output:\")\n", + "print(agent_output)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "发生了什么？首先，智能体启动了检索器，并考虑了特定的来源（`['transformers', 'blog']`）。\n", + "\n", + "但是这次检索没有产生足够的结果 ⇒ 没关系！智能体可以迭代之前的结果，因此它只是用不那么严格的搜索参数重新运行了它的检索。\n", + "\n", + "因此，研究成功了！\n", + "\n", + "请注意，**使用调用检索器作为工具并可以动态修改查询和其他检索参数的 LLM 智能体**是 RAG 的**更一般的表述**，这也涵盖了像迭代查询优化这样的许多 RAG 改进技术。\n", + "\n", + "## 3. 💻 调试 Python 代码" + ] + }, + { + "cell_type": "code", + "execution_count": 41, + "metadata": {}, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\u001b[33;1m======== New task ========\u001b[0m\n", + "\u001b[37;1mI have some code that creates a bug: please debug it and return the final code\n", + "You have been provided with these initial arguments: {'code': '\\nlist=[0, 1, 2]\\n\\nfor i in range(4):\\n print(list(i))\\n'}.\u001b[0m\n", + "\u001b[33;1m==== Agent is executing the code below:\u001b[0m\n", + "\u001b[0m\u001b[38;5;109mlist\u001b[39m\u001b[38;5;109;01m=\u001b[39;00m\u001b[38;5;7m[\u001b[39m\u001b[38;5;139m0\u001b[39m\u001b[38;5;7m,\u001b[39m\u001b[38;5;7m \u001b[39m\u001b[38;5;139m1\u001b[39m\u001b[38;5;7m,\u001b[39m\u001b[38;5;7m \u001b[39m\u001b[38;5;139m2\u001b[39m\u001b[38;5;7m]\u001b[39m\n", + "\u001b[38;5;109mprint\u001b[39m\u001b[38;5;7m(\u001b[39m\u001b[38;5;109mlist\u001b[39m\u001b[38;5;7m)\u001b[39m\n", + "\u001b[38;5;109;01mfor\u001b[39;00m\u001b[38;5;7m \u001b[39m\u001b[38;5;7mi\u001b[39m\u001b[38;5;7m \u001b[39m\u001b[38;5;109;01min\u001b[39;00m\u001b[38;5;7m \u001b[39m\u001b[38;5;109mrange\u001b[39m\u001b[38;5;7m(\u001b[39m\u001b[38;5;139m4\u001b[39m\u001b[38;5;7m)\u001b[39m\u001b[38;5;7m:\u001b[39m\n", + "\u001b[38;5;7m \u001b[39m\u001b[38;5;109mprint\u001b[39m\u001b[38;5;7m(\u001b[39m\u001b[38;5;109mlist\u001b[39m\u001b[38;5;7m[\u001b[39m\u001b[38;5;7mi\u001b[39m\u001b[38;5;7m]\u001b[39m\u001b[38;5;7m)\u001b[39m\u001b[0m\n", + "\u001b[33;1m====\u001b[0m\n", + "\u001b[31;20mFailed while trying to execute the code below:\n", + "\u001b[0mlist=[0, 1, 2]\n", + "print(list)\n", + "for i in range(4):\n", + " print(list[i])\u001b[0m\n", + "This failed due to the following error:\n", + "list index out of range\u001b[0m\n", + "Traceback (most recent call last):\n", + " File \"/Users/aymeric/Documents/Code/original_transformers/transformers/src/transformers/agents/agents.py\", line 823, in step\n", + " result = self.python_evaluator(code_action, available_tools, state=self.state)\n", + " ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n", + " File \"/Users/aymeric/Documents/Code/original_transformers/transformers/src/transformers/agents/python_interpreter.py\", line 511, in evaluate_python_code\n", + " line_result = evaluate_ast(node, state, tools)\n", + " ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n", + " File \"/Users/aymeric/Documents/Code/original_transformers/transformers/src/transformers/agents/python_interpreter.py\", line 404, in evaluate_ast\n", + " return evaluate_for(expression, state, tools)\n", + " ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n", + " File \"/Users/aymeric/Documents/Code/original_transformers/transformers/src/transformers/agents/python_interpreter.py\", line 313, in evaluate_for\n", + " line_result = evaluate_ast(node, state, tools)\n", + " ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n", + " File \"/Users/aymeric/Documents/Code/original_transformers/transformers/src/transformers/agents/python_interpreter.py\", line 401, in evaluate_ast\n", + " return evaluate_ast(expression.value, state, tools)\n", + " ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n", + " File \"/Users/aymeric/Documents/Code/original_transformers/transformers/src/transformers/agents/python_interpreter.py\", line 365, in evaluate_ast\n", + " return evaluate_call(expression, state, tools)\n", + " ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n", + " File \"/Users/aymeric/Documents/Code/original_transformers/transformers/src/transformers/agents/python_interpreter.py\", line 215, in evaluate_call\n", + " args = [evaluate_ast(arg, state, tools) for arg in call.args]\n", + " ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n", + " File \"/Users/aymeric/Documents/Code/original_transformers/transformers/src/transformers/agents/python_interpreter.py\", line 423, in evaluate_ast\n", + " return evaluate_subscript(expression, state, tools)\n", + " ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n", + " File \"/Users/aymeric/Documents/Code/original_transformers/transformers/src/transformers/agents/python_interpreter.py\", line 236, in evaluate_subscript\n", + " return value[int(index)]\n", + " ~~~~~^^^^^^^^^^^^\n", + "IndexError: list index out of range\n", + "\n", + "During handling of the above exception, another exception occurred:\n", + "\n", + "Traceback (most recent call last):\n", + " File \"/Users/aymeric/Documents/Code/original_transformers/transformers/src/transformers/agents/agents.py\", line 623, in run\n", + " final_answer = self.step()\n", + " ^^^^^^^^^^^\n", + " File \"/Users/aymeric/Documents/Code/original_transformers/transformers/src/transformers/agents/agents.py\", line 832, in step\n", + " raise AgentExecutionError(error_msg)\n", + "transformers.agents.agents.AgentExecutionError: Failed while trying to execute the code below:\n", + "\u001b[0mlist=[0, 1, 2]\n", + "print(list)\n", + "for i in range(4):\n", + " print(list[i])\u001b[0m\n", + "This failed due to the following error:\n", + "list index out of range\n", + "\u001b[33;1m==== Agent is executing the code below:\u001b[0m\n", + "\u001b[0m\u001b[38;5;109mlist\u001b[39m\u001b[38;5;109;01m=\u001b[39;00m\u001b[38;5;7m[\u001b[39m\u001b[38;5;139m0\u001b[39m\u001b[38;5;7m,\u001b[39m\u001b[38;5;7m \u001b[39m\u001b[38;5;139m1\u001b[39m\u001b[38;5;7m,\u001b[39m\u001b[38;5;7m \u001b[39m\u001b[38;5;139m2\u001b[39m\u001b[38;5;7m]\u001b[39m\n", + "\u001b[38;5;109mprint\u001b[39m\u001b[38;5;7m(\u001b[39m\u001b[38;5;109mlist\u001b[39m\u001b[38;5;7m)\u001b[39m\n", + "\u001b[38;5;109;01mfor\u001b[39;00m\u001b[38;5;7m \u001b[39m\u001b[38;5;7mi\u001b[39m\u001b[38;5;7m \u001b[39m\u001b[38;5;109;01min\u001b[39;00m\u001b[38;5;7m \u001b[39m\u001b[38;5;109mrange\u001b[39m\u001b[38;5;7m(\u001b[39m\u001b[38;5;139m3\u001b[39m\u001b[38;5;7m)\u001b[39m\u001b[38;5;7m:\u001b[39m\n", + "\u001b[38;5;7m \u001b[39m\u001b[38;5;109mprint\u001b[39m\u001b[38;5;7m(\u001b[39m\u001b[38;5;109mlist\u001b[39m\u001b[38;5;7m[\u001b[39m\u001b[38;5;7mi\u001b[39m\u001b[38;5;7m]\u001b[39m\u001b[38;5;7m)\u001b[39m\u001b[0m\n", + "\u001b[33;1m====\u001b[0m\n", + "\u001b[33;1mPrint outputs:\u001b[0m\n", + "\u001b[32;20m[0, 1, 2]\n", + "0\n", + "1\n", + "2\n", + "\u001b[0m\n", + "\u001b[33;1m==== Agent is executing the code below:\u001b[0m\n", + "\u001b[0m\u001b[38;5;109mlist\u001b[39m\u001b[38;5;109;01m=\u001b[39;00m\u001b[38;5;7m[\u001b[39m\u001b[38;5;139m0\u001b[39m\u001b[38;5;7m,\u001b[39m\u001b[38;5;7m \u001b[39m\u001b[38;5;139m1\u001b[39m\u001b[38;5;7m,\u001b[39m\u001b[38;5;7m \u001b[39m\u001b[38;5;139m2\u001b[39m\u001b[38;5;7m]\u001b[39m\n", + "\u001b[38;5;109mprint\u001b[39m\u001b[38;5;7m(\u001b[39m\u001b[38;5;109mlist\u001b[39m\u001b[38;5;7m)\u001b[39m\n", + "\u001b[38;5;109;01mfor\u001b[39;00m\u001b[38;5;7m \u001b[39m\u001b[38;5;7mi\u001b[39m\u001b[38;5;7m \u001b[39m\u001b[38;5;109;01min\u001b[39;00m\u001b[38;5;7m \u001b[39m\u001b[38;5;109mrange\u001b[39m\u001b[38;5;7m(\u001b[39m\u001b[38;5;139m3\u001b[39m\u001b[38;5;7m)\u001b[39m\u001b[38;5;7m:\u001b[39m\n", + "\u001b[38;5;7m \u001b[39m\u001b[38;5;109mprint\u001b[39m\u001b[38;5;7m(\u001b[39m\u001b[38;5;109mlist\u001b[39m\u001b[38;5;7m[\u001b[39m\u001b[38;5;7mi\u001b[39m\u001b[38;5;7m]\u001b[39m\u001b[38;5;7m)\u001b[39m\u001b[0m\n", + "\u001b[33;1m====\u001b[0m\n", + "\u001b[33;1mPrint outputs:\u001b[0m\n", + "\u001b[32;20m[0, 1, 2]\n", + "0\n", + "1\n", + "2\n", + "\u001b[0m\n", + "\u001b[33;1m==== Agent is executing the code below:\u001b[0m\n", + "\u001b[0m\u001b[38;5;109mlist\u001b[39m\u001b[38;5;109;01m=\u001b[39;00m\u001b[38;5;7m[\u001b[39m\u001b[38;5;139m0\u001b[39m\u001b[38;5;7m,\u001b[39m\u001b[38;5;7m \u001b[39m\u001b[38;5;139m1\u001b[39m\u001b[38;5;7m,\u001b[39m\u001b[38;5;7m \u001b[39m\u001b[38;5;139m2\u001b[39m\u001b[38;5;7m]\u001b[39m\n", + "\u001b[38;5;109mprint\u001b[39m\u001b[38;5;7m(\u001b[39m\u001b[38;5;109mlist\u001b[39m\u001b[38;5;7m)\u001b[39m\n", + "\u001b[38;5;109;01mfor\u001b[39;00m\u001b[38;5;7m \u001b[39m\u001b[38;5;7mi\u001b[39m\u001b[38;5;7m \u001b[39m\u001b[38;5;109;01min\u001b[39;00m\u001b[38;5;7m \u001b[39m\u001b[38;5;109mrange\u001b[39m\u001b[38;5;7m(\u001b[39m\u001b[38;5;109mlen\u001b[39m\u001b[38;5;7m(\u001b[39m\u001b[38;5;109mlist\u001b[39m\u001b[38;5;7m)\u001b[39m\u001b[38;5;7m)\u001b[39m\u001b[38;5;7m:\u001b[39m\n", + "\u001b[38;5;7m \u001b[39m\u001b[38;5;109mprint\u001b[39m\u001b[38;5;7m(\u001b[39m\u001b[38;5;109mlist\u001b[39m\u001b[38;5;7m[\u001b[39m\u001b[38;5;7mi\u001b[39m\u001b[38;5;7m]\u001b[39m\u001b[38;5;7m)\u001b[39m\u001b[0m\n", + "\u001b[33;1m====\u001b[0m\n", + "\u001b[33;1mPrint outputs:\u001b[0m\n", + "\u001b[32;20m[0, 1, 2]\n", + "0\n", + "1\n", + "2\n", + "\u001b[0m\n", + "\u001b[33;1m==== Agent is executing the code below:\u001b[0m\n", + "\u001b[0m\u001b[38;5;109mlist\u001b[39m\u001b[38;5;109;01m=\u001b[39;00m\u001b[38;5;7m[\u001b[39m\u001b[38;5;139m0\u001b[39m\u001b[38;5;7m,\u001b[39m\u001b[38;5;7m \u001b[39m\u001b[38;5;139m1\u001b[39m\u001b[38;5;7m,\u001b[39m\u001b[38;5;7m \u001b[39m\u001b[38;5;139m2\u001b[39m\u001b[38;5;7m]\u001b[39m\n", + "\u001b[38;5;109mprint\u001b[39m\u001b[38;5;7m(\u001b[39m\u001b[38;5;109mlist\u001b[39m\u001b[38;5;7m)\u001b[39m\n", + "\u001b[38;5;109;01mfor\u001b[39;00m\u001b[38;5;7m \u001b[39m\u001b[38;5;7mi\u001b[39m\u001b[38;5;7m \u001b[39m\u001b[38;5;109;01min\u001b[39;00m\u001b[38;5;7m \u001b[39m\u001b[38;5;109mrange\u001b[39m\u001b[38;5;7m(\u001b[39m\u001b[38;5;139m3\u001b[39m\u001b[38;5;7m)\u001b[39m\u001b[38;5;7m:\u001b[39m\n", + "\u001b[38;5;7m \u001b[39m\u001b[38;5;109mprint\u001b[39m\u001b[38;5;7m(\u001b[39m\u001b[38;5;109mlist\u001b[39m\u001b[38;5;7m[\u001b[39m\u001b[38;5;7mi\u001b[39m\u001b[38;5;7m]\u001b[39m\u001b[38;5;7m)\u001b[39m\n", + "\u001b[38;5;7mfinal_answer\u001b[39m\u001b[38;5;7m(\u001b[39m\u001b[38;5;7mcode\u001b[39m\u001b[38;5;7m)\u001b[39m\u001b[0m\n", + "\u001b[33;1m====\u001b[0m\n", + "\u001b[33;1mPrint outputs:\u001b[0m\n", + "\u001b[32;20m[0, 1, 2]\n", + "0\n", + "1\n", + "2\n", + "\u001b[0m\n", + "\u001b[33;1m>>> Final answer:\u001b[0m\n", + "\u001b[32;20m\n", + "list=[0, 1, 2]\n", + "\n", + "for i in range(4):\n", + " print(list(i))\n", + "\u001b[0m\n" + ] + } + ], + "source": [ + "from transformers import ReactCodeAgent\n", + "\n", + "agent = ReactCodeAgent(tools=[])\n", + "\n", + "code = \"\"\"\n", + "list=[0, 1, 2]\n", + "\n", + "for i in range(4):\n", + " print(list(i))\n", + "\"\"\"\n", + "\n", + "final_answer = agent.run(\n", + " \"I have some code that creates a bug: please debug it and return the final code\",\n", + " code=code,\n", + ")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "正如你所看到的，智能体尝试了给定的代码，遇到错误，分析错误，纠正代码，并在验证代码可以正常工作后返回它！\n", + "\n", + "最终的代码是纠正后的代码：\n" + ] + }, + { + "cell_type": "code", + "execution_count": 42, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\n", + "list=[0, 1, 2]\n", + "\n", + "for i in range(4):\n", + " print(list(i))\n", + "\n" + ] + } + ], + "source": [ + "print(final_answer)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 4. 创建你自己的 LLM 引擎（OpenAI）\n", + "\n", + "设置你自己的 LLM 引擎真的非常简单：\n", + "它只需要一个具有以下标准的`__call__`方法：\n", + "1. 接受[ChatML 格式](https://huggingface.co/docs/transformers/main/en/chat_templating#introduction)的消息列表作为输入并输出答案。\n", + "2. 接受一个 `stop_sequences` 参数，以传递生成停止的序列。\n", + "3. 根据你的 LLM 接受哪种类型的消息角色，你可能还需要转换一些消息角色。\n" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": {}, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\u001b[33;1m======== New task ========\u001b[0m\n", + "\u001b[37;1mI have some code that creates a bug: please debug it and return the final code\n", + "You have been provided with these initial arguments: {'code': '\\nlist=[0, 1, 2]\\n\\nfor i in range(4):\\n print(list(i))\\n'}.\u001b[0m\n", + "\u001b[33;1m==== Agent is executing the code below:\u001b[0m\n", + "\u001b[0m\u001b[38;5;7mmy_list\u001b[39m\u001b[38;5;7m \u001b[39m\u001b[38;5;109;01m=\u001b[39;00m\u001b[38;5;7m \u001b[39m\u001b[38;5;7m[\u001b[39m\u001b[38;5;139m0\u001b[39m\u001b[38;5;7m,\u001b[39m\u001b[38;5;7m \u001b[39m\u001b[38;5;139m1\u001b[39m\u001b[38;5;7m,\u001b[39m\u001b[38;5;7m \u001b[39m\u001b[38;5;139m2\u001b[39m\u001b[38;5;7m]\u001b[39m\u001b[38;5;7m \u001b[39m\u001b[38;5;60;03m# Renamed the list to avoid using the built-in name\u001b[39;00m\n", + "\n", + "\u001b[38;5;109;01mfor\u001b[39;00m\u001b[38;5;7m \u001b[39m\u001b[38;5;7mi\u001b[39m\u001b[38;5;7m \u001b[39m\u001b[38;5;109;01min\u001b[39;00m\u001b[38;5;7m \u001b[39m\u001b[38;5;109mrange\u001b[39m\u001b[38;5;7m(\u001b[39m\u001b[38;5;109mlen\u001b[39m\u001b[38;5;7m(\u001b[39m\u001b[38;5;7mmy_list\u001b[39m\u001b[38;5;7m)\u001b[39m\u001b[38;5;7m)\u001b[39m\u001b[38;5;7m:\u001b[39m\u001b[38;5;7m \u001b[39m\u001b[38;5;60;03m# Changed the range to be within the length of the list\u001b[39;00m\n", + "\u001b[38;5;7m \u001b[39m\u001b[38;5;109mprint\u001b[39m\u001b[38;5;7m(\u001b[39m\u001b[38;5;7mmy_list\u001b[39m\u001b[38;5;7m[\u001b[39m\u001b[38;5;7mi\u001b[39m\u001b[38;5;7m]\u001b[39m\u001b[38;5;7m)\u001b[39m\u001b[38;5;7m \u001b[39m\u001b[38;5;60;03m# Corrected the list access syntax\u001b[39;00m\u001b[0m\n", + "\u001b[33;1m====\u001b[0m\n", + "\u001b[33;1mPrint outputs:\u001b[0m\n", + "\u001b[32;20m0\n", + "1\n", + "2\n", + "\u001b[0m\n", + "\u001b[33;1m==== Agent is executing the code below:\u001b[0m\n", + "\u001b[0m\u001b[38;5;7mmy_list\u001b[39m\u001b[38;5;7m \u001b[39m\u001b[38;5;109;01m=\u001b[39;00m\u001b[38;5;7m \u001b[39m\u001b[38;5;7m[\u001b[39m\u001b[38;5;139m0\u001b[39m\u001b[38;5;7m,\u001b[39m\u001b[38;5;7m \u001b[39m\u001b[38;5;139m1\u001b[39m\u001b[38;5;7m,\u001b[39m\u001b[38;5;7m \u001b[39m\u001b[38;5;139m2\u001b[39m\u001b[38;5;7m]\u001b[39m\u001b[38;5;7m \u001b[39m\u001b[38;5;60;03m# Renamed the list to avoid using the built-in name\u001b[39;00m\n", + "\n", + "\u001b[38;5;109;01mfor\u001b[39;00m\u001b[38;5;7m \u001b[39m\u001b[38;5;7mi\u001b[39m\u001b[38;5;7m \u001b[39m\u001b[38;5;109;01min\u001b[39;00m\u001b[38;5;7m \u001b[39m\u001b[38;5;109mrange\u001b[39m\u001b[38;5;7m(\u001b[39m\u001b[38;5;109mlen\u001b[39m\u001b[38;5;7m(\u001b[39m\u001b[38;5;7mmy_list\u001b[39m\u001b[38;5;7m)\u001b[39m\u001b[38;5;7m)\u001b[39m\u001b[38;5;7m:\u001b[39m\u001b[38;5;7m \u001b[39m\u001b[38;5;60;03m# Changed the range to be within the length of the list\u001b[39;00m\n", + "\u001b[38;5;7m \u001b[39m\u001b[38;5;109mprint\u001b[39m\u001b[38;5;7m(\u001b[39m\u001b[38;5;7mmy_list\u001b[39m\u001b[38;5;7m[\u001b[39m\u001b[38;5;7mi\u001b[39m\u001b[38;5;7m]\u001b[39m\u001b[38;5;7m)\u001b[39m\u001b[38;5;7m \u001b[39m\u001b[38;5;60;03m# Corrected the list access syntax\u001b[39;00m\u001b[0m\n", + "\u001b[33;1m====\u001b[0m\n", + "\u001b[33;1mPrint outputs:\u001b[0m\n", + "\u001b[32;20m0\n", + "1\n", + "2\n", + "\u001b[0m\n", + "\u001b[33;1m==== Agent is executing the code below:\u001b[0m\n", + "\u001b[0m\u001b[38;5;7mcorrected_code\u001b[39m\u001b[38;5;7m \u001b[39m\u001b[38;5;109;01m=\u001b[39;00m\u001b[38;5;7m \u001b[39m\u001b[38;5;144m'''\u001b[39m\n", + "\u001b[38;5;144mmy_list = [0, 1, 2] # Renamed the list to avoid using the built-in name\u001b[39m\n", + "\n", + "\u001b[38;5;144mfor i in range(len(my_list)): # Changed the range to be within the length of the list\u001b[39m\n", + "\u001b[38;5;144m print(my_list[i]) # Corrected the list access syntax\u001b[39m\n", + "\u001b[38;5;144m'''\u001b[39m\n", + "\n", + "\u001b[38;5;7mfinal_answer\u001b[39m\u001b[38;5;7m(\u001b[39m\u001b[38;5;7manswer\u001b[39m\u001b[38;5;109;01m=\u001b[39;00m\u001b[38;5;7mcorrected_code\u001b[39m\u001b[38;5;7m)\u001b[39m\u001b[0m\n", + "\u001b[33;1m====\u001b[0m\n", + "\u001b[33;1mPrint outputs:\u001b[0m\n", + "\u001b[32;20m\u001b[0m\n", + "\u001b[33;1m>>> Final answer:\u001b[0m\n", + "\u001b[32;20m\n", + "my_list = [0, 1, 2] # Renamed the list to avoid using the built-in name\n", + "\n", + "for i in range(len(my_list)): # Changed the range to be within the length of the list\n", + " print(my_list[i]) # Corrected the list access syntax\n", + "\u001b[0m\n" + ] + } + ], + "source": [ + "import os\n", + "from openai import OpenAI\n", + "from transformers.agents.llm_engine import MessageRole, get_clean_message_list\n", + "\n", + "openai_role_conversions = {\n", + " MessageRole.TOOL_RESPONSE: \"user\",\n", + "}\n", + "\n", + "\n", + "class OpenAIEngine:\n", + " def __init__(self, model_name=\"gpt-4o-2024-05-13\"):\n", + " self.model_name = model_name\n", + " self.client = OpenAI(\n", + " api_key=os.getenv(\"OPENAI_API_KEY\"),\n", + " )\n", + "\n", + " def __call__(self, messages, stop_sequences=[]):\n", + " # Get clean message list\n", + " messages = get_clean_message_list(\n", + " messages, role_conversions=openai_role_conversions\n", + " )\n", + "\n", + " # Get LLM output\n", + " response = self.client.chat.completions.create(\n", + " model=self.model_name,\n", + " messages=messages,\n", + " stop=stop_sequences,\n", + " )\n", + " return response.choices[0].message.content\n", + "\n", + "\n", + "openai_engine = OpenAIEngine()\n", + "agent = ReactCodeAgent(llm_engine=openai_engine, tools=[])\n", + "\n", + "code = \"\"\"\n", + "list=[0, 1, 2]\n", + "\n", + "for i in range(4):\n", + " print(list(i))\n", + "\"\"\"\n", + "\n", + "final_answer = agent.run(\n", + " \"I have some code that creates a bug: please debug it and return the final code\",\n", + " code=code,\n", + ")" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\n", + "my_list = [0, 1, 2] # Renamed the list to avoid using the built-in name\n", + "\n", + "for i in range(len(my_list)): # Changed the range to be within the length of the list\n", + " print(my_list[i]) # Corrected the list access syntax\n", + "\n" + ] + } + ], + "source": [ + "print(final_answer)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## ➡️ 结论\n", + "\n", + "上述用例应该让你对我们智能体框架的可能性有了初步了解！\n", + "\n", + "想要了解更多高级用法，请阅读[文档](https://huggingface.co/docs/transformers/en/transformers_agents)， 以及[此实验](https://github.com/aymeric-roucher/agent_reasoning_benchmark/blob/main/benchmark_gaia.ipynb)，它让我们能够基于 Llama-3-70B 构建自己的智能体，并在非常困难的[GAIA 排行榜](https://huggingface.co/spaces/gaia-benchmark/leaderboard)上击败许多 GPT-4 智能体！\n", + "\n", + "欢迎所有反馈，这将帮助我们改进框架！ 🚀" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "test2", + "language": "python", + "name": "test2" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.12.0" + } + }, + "nbformat": 4, + "nbformat_minor": 2 +} From b3e2bc0e6345610cebbb6107703f5800d0f839f9 Mon Sep 17 00:00:00 2001 From: innovation64 Date: Thu, 11 Jul 2024 15:32:29 +0800 Subject: [PATCH 25/31] update rag_with_hugging_face_gemma_elasticsearch cn version --- notebooks/zh-CN/_toctree.yml | 2 + ...ith_hugging_face_gemma_elasticsearch.ipynb | 6319 +++++++++++++++++ 2 files changed, 6321 insertions(+) create mode 100644 notebooks/zh-CN/rag_with_hugging_face_gemma_elasticsearch.ipynb diff --git a/notebooks/zh-CN/_toctree.yml b/notebooks/zh-CN/_toctree.yml index df668dec..7d902962 100644 --- a/notebooks/zh-CN/_toctree.yml +++ b/notebooks/zh-CN/_toctree.yml @@ -15,6 +15,8 @@ sections: title: 使用 SetFit 进行零样本文本分类的数据标注建议 - local: fine_tuning_code_llm_on_single_gpu title: 在单个 GPU 上针对自定义代码微调代码 LLM + - local: rag_with_hugging_face_gemma_elasticsearch + title: 构建一个基于 Gemma、Elasticsearch 和 Hugging Face 模型的 RAG 系统 - local: prompt_tuning_peft title: 使用 PEFT 进行提示微调 - local: rag_evaluation diff --git a/notebooks/zh-CN/rag_with_hugging_face_gemma_elasticsearch.ipynb b/notebooks/zh-CN/rag_with_hugging_face_gemma_elasticsearch.ipynb new file mode 100644 index 00000000..b712c7ca --- /dev/null +++ b/notebooks/zh-CN/rag_with_hugging_face_gemma_elasticsearch.ipynb @@ -0,0 +1,6319 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": { + "id": "qsmx4MGD6QSp" + }, + "source": [ + "# 构建一个基于 Gemma、Elasticsearch 和 Hugging Face 模型的 RAG 系统\n", + "\n", + "作者: [lloydmeta](https://huggingface.co/lloydmeta)\n", + "\n", + "\n", + "这个 notebook 将引导你构建一个由 Elasticsearch（ES）和 Hugging Face 模型支持的检索增强生成（RAG）系统，允许你在 ES 向量化（你的 ES 集群在摄取和查询时为你向量化）与自向量化（你在将数据发送到 ES 之前对所有数据进行向量化）之间切换。\n", + "\n", + "你的用例应该使用哪种方式？*这取决于* 🤷‍♂️。ES 向量化意味着你的客户端不需要实现这一点，所以这里默认使用；但是，如果你没有机器学习节点，或者你自己的嵌入设置更好/更快，可以在下面的“选择数据和查询向量化选项”部分将 `USE_ELASTICSEARCH_VECTORISATION` 设置为 `False`！\n", + "\n", + "> [!提示]\n", + "> 这个 notebook 已经使用 ES 8.13.x 和 8.14.x 进行了测试。\n" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "BIL0BjjF6QSt" + }, + "source": [ + "## 步骤 1：安装相关库\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "gVSo_nNOUsdn" + }, + "outputs": [], + "source": [ + "!pip install elasticsearch sentence_transformers transformers eland==8.12.1 # accelerate # uncomment if using GPU\n", + "!pip install datasets==2.19.2 # Remove version lock if https://github.com/huggingface/datasets/pull/6978 has been released" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "asQZzrNASBPI" + }, + "source": [ + "## 步骤 2：设置\n", + "\n", + "### Hugging Face\n", + "\n", + "这允许你通过 Hugging Face 进行身份验证，以便下载模型和数据集。" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "NL-NG4jjXb0I" + }, + "outputs": [], + "source": [ + "from huggingface_hub import notebook_login\n", + "\n", + "notebook_login()" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "Ov9J5o5AEjzK" + }, + "source": [ + "#### Elasticsearch 部署\n", + "\n", + "让我们确保你可以访问你的 Elasticsearch 部署。如果你还没有，可以在 [Elastic Cloud](https://www.elastic.co/search-labs/tutorials/install-elasticsearch/elastic-cloud#creating-a-cloud-deployment)上创建一个。\n", + "\n", + "确保你已经将 `CLOUD_ID` 和 `ELASTIC_DEPL_API_KEY` 保存为 Colab secrets。\n", + "\n", + "![Image of how to set up secrets using Google Colab](https://huggingface.co/datasets/huggingface/cookbook-images/resolve/main/colab-secrets.jpeg)" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": { + "id": "2pSSD57kn14y" + }, + "outputs": [], + "source": [ + "from google.colab import userdata\n", + "\n", + "# https://www.elastic.co/search-labs/tutorials/install-elasticsearch/elastic-cloud#finding-your-cloud-id\n", + "CLOUD_ID = userdata.get(\"CLOUD_ID\") # or \"\"\n", + "\n", + "# https://www.elastic.co/search-labs/tutorials/install-elasticsearch/elastic-cloud#creating-an-api-key\n", + "ELASTIC_API_KEY = userdata.get(\"ELASTIC_DEPL_API_KEY\") # or \"\"" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "uewOyerWGx9p" + }, + "source": [ + "设置客户端并确保凭据有效。" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "WDt5s-AFYVZE", + "outputId": "baa481f9-50d6-43fd-9d87-78cadbd75dd6" + }, + "outputs": [], + "source": [ + "from elasticsearch import Elasticsearch, helpers\n", + "\n", + "# Create the client instance\n", + "client = Elasticsearch(cloud_id=CLOUD_ID, api_key=ELASTIC_API_KEY)\n", + "\n", + "# Successful response!\n", + "client.info()" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "kF3A7uGc6QSv" + }, + "source": [ + "## 步骤 3：数据获取和准备\n", + "\n", + "本教程中使用的数据来源于 Hugging Face 数据集，特别是来自 [MongoDB/embedded_movies 数据集](https://huggingface.co/datasets/MongoDB/embedded_movies)。\n", + "\n" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 232, + "referenced_widgets": [ + "3416975d721243048c39b99af297af44", + "c96f16ab241d4b60813235ed2d5e3580", + "a728d91c3714483a93469ec826e19e08", + "23f3b48bb1644f91b2733f2045bae5f5", + "9a4d9e10fd824105a3a0f66dbcbe9767", + "77a8d8603e81430ab884314918ca45f9", + "a988b6a810b34e7495c086fde8dedd7f", + "a676673d26834b628283d76b5d4302dc", + "01d348af5fe547f4b24e98539cf91f2f", + "dacffd74d0bf46d99f293cfc4430d0f4", + "12e80316a20e4fef8d19139dc5aaf240", + "0e5e3e07ae6a44df99b963259178fe87", + "9e30a1f48eb742ff91b00e4951c2902f", + "27f80f2d47f14feeb6cc0c02076290d5", + "7e7695b6889d42f6afa66d61e0b9f000", + "8149fd2e5eee427cb15ab22644da99ea", + "d3f7f9a12c004d62a48db61ae1bb413a", + "b78cf8ca5f244198a272ab4501c3f28d", + "67784de672194cb885e40ba4d43bbeb4", + "c22e87fbd1754eb6abd336d111e699a2", + "bf91048aaafc45ceb56c83755f9f140e", + "0cef3c2b245241ad938a8b8c9a00e335", + "ab7454174422451b94a6d1beea3a4b61", + "7de4ad29fc594f7db60ccac865e945c4", + "6b74827e673e4530aac1ea1defae1b8c", + "dae854ebfff04e0ebe132e9588195c17", + "3ba409ad57e84f86950ed12b611b5bf2", + "5e418aea478c4c28a95a4ac7c5fb420e", + "6f3cfa2a8e374a0faed68861d19db428", + "1d1f2d5311934718baaded44f1e495ec", + "ac3a6ce7852340089cceff5f1ce18ee3", + "17a8be19a6b1465bbf00d4a0d905841e", + "491da9ab200d4680a4b55cfb8b0d9c4f" + ] + }, + "id": "5gCzss27UwWw", + "outputId": "862ad48b-34ad-4206-c93e-c82f7e587638" + }, + "outputs": [ + { + "data": { + "application/vnd.jupyter.widget-view+json": { + "model_id": "3416975d721243048c39b99af297af44", + "version_major": 2, + "version_minor": 0 + }, + "text/plain": [ + "Downloading readme: 0%| | 0.00/6.18k [00:00 list[float]:\n", + " if USE_ELASTICSEARCH_VECTORISATION:\n", + " raise Exception(\n", + " f\"Disabled when USE_ELASTICSEARCH_VECTORISATION is [{USE_ELASTICSEARCH_VECTORISATION}]\"\n", + " )\n", + " else:\n", + " if not text.strip():\n", + " print(\"Attempted to get embedding for empty text.\")\n", + " return []\n", + "\n", + " embedding = embedding_model.encode(text)\n", + " return embedding.tolist()\n", + "\n", + "\n", + "def add_fullplot_embedding(x):\n", + " if USE_ELASTICSEARCH_VECTORISATION:\n", + " raise Exception(\n", + " f\"Disabled when USE_ELASTICSEARCH_VECTORISATION is [{USE_ELASTICSEARCH_VECTORISATION}]\"\n", + " )\n", + " else:\n", + " full_plots = x[\"fullplot\"]\n", + " return {\"embedding\": [get_embedding(full_plot) for full_plot in full_plots]}\n", + "\n", + "\n", + "if not USE_ELASTICSEARCH_VECTORISATION:\n", + " dataset = dataset.map(add_fullplot_embedding, batched=True)\n", + " dataset[\"train\"]" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "i7gZ5fno6QSw" + }, + "source": [ + "## 步骤 5：创建一个带有向量搜索映射的搜索索引。\n", + "\n", + "在这一点上，我们将在 Elasticsearch 中创建一个索引，并设置正确的索引映射来处理向量搜索。\n", + "\n", + "请点击这里了解更多关于 [Elasticsearch 向量搜索能力](https://www.elastic.co/what-is/vector-search)的信息。\n" + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "n3gERSl_uFO2", + "outputId": "3307ca4d-6a32-4a6c-dfe3-b8cc3280d938" + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Creating index movies\n" + ] + }, + { + "data": { + "text/plain": [ + "ObjectApiResponse({'acknowledged': True, 'shards_acknowledged': True, 'index': 'movies'})" + ] + }, + "execution_count": 10, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# Needs to match the id returned from Eland\n", + "# in general for Hugging Face models, you just replace the forward slash with\n", + "# double underscore\n", + "model_id = EMBEDDING_MODEL_ID.replace(\"/\", \"__\")\n", + "\n", + "index_name = \"movies\"\n", + "\n", + "index_mapping = {\n", + " \"properties\": {\n", + " \"fullplot\": {\"type\": \"text\"},\n", + " \"plot\": {\"type\": \"text\"},\n", + " \"title\": {\"type\": \"text\"},\n", + " }\n", + "}\n", + "# define index mapping\n", + "if USE_ELASTICSEARCH_VECTORISATION:\n", + " index_mapping[\"properties\"][\"embedding\"] = {\n", + " \"properties\": {\n", + " \"is_truncated\": {\"type\": \"boolean\"},\n", + " \"model_id\": {\n", + " \"type\": \"text\",\n", + " \"fields\": {\"keyword\": {\"type\": \"keyword\", \"ignore_above\": 256}},\n", + " },\n", + " \"predicted_value\": {\n", + " \"type\": \"dense_vector\",\n", + " \"dims\": EMBEDDING_DIMENSIONS,\n", + " \"index\": True,\n", + " \"similarity\": \"cosine\",\n", + " },\n", + " }\n", + " }\n", + "else:\n", + " index_mapping[\"properties\"][\"embedding\"] = {\n", + " \"type\": \"dense_vector\",\n", + " \"dims\": EMBEDDING_DIMENSIONS,\n", + " \"index\": \"true\",\n", + " \"similarity\": \"cosine\",\n", + " }\n", + "\n", + "# flag to check if index has to be deleted before creating\n", + "should_delete_index = True\n", + "\n", + "# check if we want to delete index before creating the index\n", + "if should_delete_index:\n", + " if client.indices.exists(index=index_name):\n", + " print(\"Deleting existing %s\" % index_name)\n", + " client.indices.delete(index=index_name, ignore=[400, 404])\n", + "\n", + "print(\"Creating index %s\" % index_name)\n", + "\n", + "\n", + "# ingest pipeline definition\n", + "if USE_ELASTICSEARCH_VECTORISATION:\n", + " pipeline_id = \"vectorize_fullplots\"\n", + "\n", + " client.ingest.put_pipeline(\n", + " id=pipeline_id,\n", + " processors=[\n", + " {\n", + " \"inference\": {\n", + " \"model_id\": model_id,\n", + " \"target_field\": \"embedding\",\n", + " \"field_map\": {\"fullplot\": \"text_field\"},\n", + " }\n", + " }\n", + " ],\n", + " )\n", + "\n", + " index_settings = {\n", + " \"index\": {\n", + " \"default_pipeline\": pipeline_id,\n", + " }\n", + " }\n", + "else:\n", + " index_settings = {}\n", + "\n", + "client.options(ignore_status=[400, 404]).indices.create(\n", + " index=index_name, mappings=index_mapping, settings=index_settings\n", + ")" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "neANZEH96QSx" + }, + "source": [ + "将数据摄取到 Elasticsearch 中最好是批量进行。幸运的是，`helpers` 提供了一个简单的方法来执行这个操作。" + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "mH2BAuhYva6U", + "outputId": "0beaf822-1052-4f65-cfd2-eb8cec96e666" + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "batch: start [0], end [100]\n", + "batch: start [100], end [200]\n", + "batch: start [200], end [300]\n", + "batch: start [300], end [400]\n", + "batch: start [400], end [500]\n", + "batch: start [500], end [600]\n", + "batch: start [600], end [700]\n", + "batch: start [700], end [800]\n", + "batch: start [800], end [900]\n", + "batch: start [900], end [1000]\n", + "batch: start [1000], end [1100]\n", + "batch: start [1100], end [1200]\n", + "batch: start [1200], end [1300]\n", + "batch: start [1300], end [1400]\n", + "batch: start [1400], end [1452]\n", + "Data ingestion into Elasticsearch complete!\n" + ] + } + ], + "source": [ + "from elasticsearch.helpers import BulkIndexError\n", + "\n", + "def batch_to_bulk_actions(batch):\n", + " for record in batch:\n", + " action = {\n", + " \"_index\": \"movies\",\n", + " \"_source\": {\n", + " \"title\": record[\"title\"],\n", + " \"fullplot\": record[\"fullplot\"],\n", + " \"plot\": record[\"plot\"],\n", + " },\n", + " }\n", + " if not USE_ELASTICSEARCH_VECTORISATION:\n", + " action[\"_source\"][\"embedding\"] = record[\"embedding\"]\n", + " yield action\n", + "\n", + "\n", + "def bulk_index(ds):\n", + " start = 0\n", + " end = len(ds)\n", + " batch_size = 100\n", + " if USE_ELASTICSEARCH_VECTORISATION:\n", + " # If using auto-embedding, bulk requests can take a lot longer,\n", + " # so pass a longer request_timeout here (defaults to 10s), otherwise\n", + " # we could get Connection timeouts\n", + " batch_client = client.options(request_timeout=600)\n", + " else:\n", + " batch_client = client\n", + " for batch_start in range(start, end, batch_size):\n", + " batch_end = min(batch_start + batch_size, end)\n", + " print(f\"batch: start [{batch_start}], end [{batch_end}]\")\n", + " batch = ds.select(range(batch_start, batch_end))\n", + " actions = batch_to_bulk_actions(batch)\n", + " helpers.bulk(batch_client, actions)\n", + "\n", + "\n", + "try:\n", + " bulk_index(dataset[\"train\"])\n", + "except BulkIndexError as e:\n", + " print(f\"{e.errors}\")\n", + "\n", + "print(\"Data ingestion into Elasticsearch complete!\")" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "rDl8GBg_6QSx" + }, + "source": [ + "## 步骤 6：对用户查询执行向量搜索\n", + "\n", + "以下步骤实现了一个函数，该函数返回一个向量搜索结果。\n", + "\n", + "如果 `USE_ELASTICSEARCH_VECTORISATION` 为 true，文本查询将直接发送到 ES，在那里将首先使用上传的模型对其进行向量化，然后执行向量搜索。如果 `USE_ELASTICSEARCH_VECTORISATION` 为 false，那么我们在发送查询之前先在本地进行向量化，然后发送查询的向量化形式。\n" + ] + }, + { + "cell_type": "code", + "execution_count": 12, + "metadata": { + "id": "e9RLHJsdwG44" + }, + "outputs": [], + "source": [ + "def vector_search(plot_query):\n", + " if USE_ELASTICSEARCH_VECTORISATION:\n", + " knn = {\n", + " \"field\": \"embedding.predicted_value\",\n", + " \"k\": 10,\n", + " \"query_vector_builder\": {\n", + " \"text_embedding\": {\n", + " \"model_id\": model_id,\n", + " \"model_text\": plot_query,\n", + " }\n", + " },\n", + " \"num_candidates\": 150,\n", + " }\n", + " else:\n", + " question_embedding = get_embedding(plot_query)\n", + " knn = {\n", + " \"field\": \"embedding\",\n", + " \"query_vector\": question_embedding,\n", + " \"k\": 10,\n", + " \"num_candidates\": 150,\n", + " }\n", + "\n", + " response = client.search(index=\"movies\", knn=knn, size=5)\n", + " results = []\n", + " for hit in response[\"hits\"][\"hits\"]:\n", + " id = hit[\"_id\"]\n", + " score = hit[\"_score\"]\n", + " title = hit[\"_source\"][\"title\"]\n", + " plot = hit[\"_source\"][\"plot\"]\n", + " fullplot = hit[\"_source\"][\"fullplot\"]\n", + " result = {\n", + " \"id\": id,\n", + " \"_score\": score,\n", + " \"title\": title,\n", + " \"plot\": plot,\n", + " \"fullplot\": fullplot,\n", + " }\n", + " results.append(result)\n", + " return results\n", + "\n", + "def pretty_search(query):\n", + "\n", + " get_knowledge = vector_search(query)\n", + "\n", + " search_result = \"\"\n", + " for result in get_knowledge:\n", + " search_result += f\"Title: {result.get('title', 'N/A')}, Plot: {result.get('fullplot', 'N/A')}\\n\"\n", + "\n", + " return search_result" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "bMou2fWE6QSy" + }, + "source": [ + "## 步骤 7：处理用户查询并加载 Gemma\n" + ] + }, + { + "cell_type": "code", + "execution_count": 13, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "Z4L4SfueU6PY", + "outputId": "f6343803-30e6-4c40-cc81-5246af8f91a5" + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Query: What is the best romantic movie to watch and why?\n", + "Continue to answer the query by using these Search Results:\n", + "Title: Shut Up and Kiss Me!, Plot: Ryan and Pete are 27-year old best friends in Miami, born on the same day and each searching for the perfect woman. Ryan is a rookie stockbroker living with his psychic Mom. Pete is a slick surfer dude yet to find commitment. Each meets the women of their dreams on the same day. Ryan knocks heads in an elevator with the gorgeous Jessica, passing out before getting her number. Pete falls for the insatiable Tiara, but Tiara's uncle is mob boss Vincent Bublione, charged with her protection. This high-energy romantic comedy asks to what extent will you go for true love?\n", + "Title: Titanic, Plot: The plot focuses on the romances of two couples upon the doomed ship's maiden voyage. Isabella Paradine (Catherine Zeta-Jones) is a wealthy woman mourning the loss of her aunt, who reignites a romance with former flame Wynn Park (Peter Gallagher). Meanwhile, a charming ne'er-do-well named Jamie Perse (Mike Doyle) steals a ticket for the ship, and falls for a sweet innocent Irish girl on board. But their romance is threatened by the villainous Simon Doonan (Tim Curry), who has discovered about the ticket and makes Jamie his unwilling accomplice, as well as having sinister plans for the girl.\n", + "Title: Dark Blue World, Plot: March 15, 1939: Germany invades Czechoslovakia. Czech and Slovak pilots flee to England, joining the RAF. After the war, back home, they are put in labor camps, suspected of anti-Communist ideas. This film cuts between a post-war camp where Franta is a prisoner and England during the war, where Franta is like a big brother to Karel, a very young pilot. On maneuvers, Karel crash lands by the rural home of Susan, an English woman whose husband is MIA. She spends one night with Karel, and he thinks he's found the love of his life. It's complicated by Susan's attraction to Franta. How will the three handle innocence, Eros, friendship, and the heat of battle? When war ends, what then?\n", + "Title: Dark Blue World, Plot: March 15, 1939: Germany invades Czechoslovakia. Czech and Slovak pilots flee to England, joining the RAF. After the war, back home, they are put in labor camps, suspected of anti-Communist ideas. This film cuts between a post-war camp where Franta is a prisoner and England during the war, where Franta is like a big brother to Karel, a very young pilot. On maneuvers, Karel crash lands by the rural home of Susan, an English woman whose husband is MIA. She spends one night with Karel, and he thinks he's found the love of his life. It's complicated by Susan's attraction to Franta. How will the three handle innocence, Eros, friendship, and the heat of battle? When war ends, what then?\n", + "Title: No Good Deed, Plot: About a police detective, Jack, who, while doing a friend a favor and searching for a runaway teenager on Turk Street, stumbles upon a bizarre band of criminals about to pull off a bank robbery. Jack finds himself being held hostage while the criminals decide what to do with him, and the leader's beautiful girlfriend, Erin, is left alone to watch Jack. Erin, who we discover is a master manipulator of the men in the gang, reveals another side to Jack - a melancholy romantic who could have been a classical cellist. She finds Jack's captivity an irresistible turn-on and he can't figure out if she's for real, or manipulating him, too. Before the gang returns, Jack and Erin's connection intensifies and who ends up with the money is anyone's guess.\n", + ".\n" + ] + } + ], + "source": [ + "# Conduct query with retrival of sources, combining results into something that\n", + "# we can feed to Gemma\n", + "def combined_query(query):\n", + " source_information = pretty_search(query)\n", + " return f\"Query: {query}\\nContinue to answer the query by using these Search Results:\\n{source_information}.\"\n", + "\n", + "\n", + "query = \"What is the best romantic movie to watch and why?\"\n", + "combined_results = combined_query(query)\n", + "\n", + "print(combined_results)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "j93-caKRLyCZ" + }, + "source": [ + "加载我们的 LLM (这里我们用 [google/gemma-2b-lt](https://huggingface.co/google/gemma-2b-it))" + ] + }, + { + "cell_type": "code", + "execution_count": 14, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 369, + "referenced_widgets": [ + "fd3e46e5b3dc4a95a4b752559ca59976", + "b48ad574aad04703b3b3e8a7c8c4e3e7", + "27c1dd63c0d24c1aae3eb679428191d8", + "4309ca1f71a142fba32037d1f3737992", + "f97b7986c1a14fe28c0f17f0b278b9a3", + "4d12a4f8c0e142c2bcde3e2f602cd642", + "0d2d03d1ce0b4c8eac9c08afc5fced88", + "bba35b6e6e064493841300004e4bebca", + "e89387022e8c40b698d5539f1d4a46eb", + "b14a00ba05ba475388043c03c86889f3", + "00f6634f1fc745f2964638d9c6aae4b4", + "88b6aeb3eac141b3afd8e58d594d3312", + "7a005901e17a4618a063279f97aed88c", + "731163f3573a496e95007f78b0dca252", + "dc57a47113624f55bd7b26a9596b980c", + "8d51f6c97d764d4dae0aeb0d62feff3d", + "04601c70d7994a4fb6a53280a54ce10b", + "2701d95ed5574bbc857e6390c487efcc", + "a8917741fa9e4274a3389d91bb4401d6", + "0e647c5d080a412690b9482ca075fc64", + "69626ea9a332403b8fa7395ec9f38620", + "bd57ef338a1746eaa99448db76d4a63d", + "875a84222e774d3c9eb80284932fe2e7", + "2b624e96c2ca4bbda2923698f72b747d", + "d29fa38babda469d9035a36f7b1f2127", + "e07a89d5ec874dab8d9247091bc7ba36", + "7e8b50ba62b443f89bb9edcccdd445bf", + "f8b43c667bbb4d2db94f9be05ac98157", + "1e52150646c242aab30241bbdaf114a4", + "e804fa638cde44ff948c4598754aa388", + "7db129e34e444e48ab6c580f8a04c45e", + "389aba0a89aa41b4af6b66db3ee706c4", + "1a3d7ebea4b34c76a680331a2908cd03", + "48045a8f337f427989391d2ac9364e19", + "1d1779ae159449c7b9331c2a5ef59f8d", + "c6d097ea17784cb58f4345a63576f9f5", + "efe1219392414cb788a9edbfa27910fa", + "26cfed8a88844a57b895e5a653036db8", + "f372b720336d4cb79b3506a16574f5b3", + "0acbd23304d048a09465c972c4552eac", + "c5f8460b95644d23894f2b84609cf694", + "a8dcb560626641a89ee6cf3a479e950d", + "bebc8a25e3684914841e7a044e86e187", + "d861941d02c54bcf9fdb1c2da2fbd097", + "ca19c76d304b4f64a29de0942cf89289", + "cb6bf240a24f4f339c33e245eb5d33b2", + "dbdbc998460f45558a12297f18dd29cf", + "aa3b99440ba3483bb8bb3f820409dcf9", + "43a1a7621c7f4da5923846b09191148f", + "0955002919cc4953b07b26da573f5f2c", + "73d953e5a0684b77be6d135e774e26fd", + "982306a9c4ce46c897cc0e8d9de21662", + "dc56c3e068df460a8214fff574268dea", + "d44c2d61b6fa4fc78364561652d1871b", + "b2ddc042c22e48bd8dd08c4d2e0595eb", + "8bc49277bed148c7b3865ab562c8ef13", + "1bf54ca09a6146d5af48197841813ed3", + "1c836ba3f588469ca1a0e7f13d0a1713", + "4663eb311c204607bda0d964c1081016", + "470595f960a54c68a92709b39f03147c", + "f6cc7e16a884443f9262efd56a18cd82", + "b745b5ede8f44515b7af076cb87dfde9", + "fbcd0bcc58064ae88a3e7ebe0eff6210", + "97ca6a79d1ea4499939bf5f20277c2a6", + "42893f0d8ad1498db9006daeeae7d24a", + "9a6c67596a4c41c5b9855d14aeb8ab33", + "a7160629d0514f3dbf0823d619d1c697", + "b211c75d452542a2aae17527a85749cd", + "e4a3742738384c8caa6d581eae599b04", + "9647ce9a103b48a0bace5791bb9d7f4d", + "486e4d6aa81c4aa0afdf3aadc6cdec73", + "f5689f11415f4c54bfa08d20f0e69b94", + "d71df090341542eab8449a92564b1511", + "246acdcd2afa490ba80e9cd5d65e37d1", + "75d7da5e10034274aa888cc53239d2f1", + "c6a56702d0d14261aa3321f0025f3ac3", + "7ae3854f9f794a88b0bb435e33f21769", + "5a2acc95b8024a51b49d5b637899b186", + "77d1f0e90be9440cbfbeb45e7b99865c", + "073d83dd85834233819348ab10b668ab", + "8a63fd367ce440e9a71314e90e4fe60d", + "cb9686bc58d341a9a05088b8eaf7f8b3", + "689619a86e0245b69c89d9f4248ca1cf", + "4b775768a9c84bf98c280cec1d9004d2", + "2ec150b4d6ec436fa675d728a4b37ce4", + "0e586865eed240da8d72a19e958ee0b8", + "cfd943bc01dd45efa7a84e21620a2149", + "8000001faa1e402e93abad8f0d147499", + "a9098f2236db4e65bd243b6b81bd5677", + "c3b52f6b69ce4b018e561d68314e068b", + "e7ce8eef05f142c4be9cc18894581def", + "1a3ce0f10d3d46d5b452a0e39e6bf067", + "40e167a97d0a42d1b06e463cc828ee31", + "21723e2868e04f208e9fb848a2c8fecb", + "cd21eabf159a49b5af791870f594eef4", + "789d82b09c7f4604a10f80511f44a42e", + "e1adfad5e87b42248876dbc085b70581", + "72bdcbe0f2d84b90b22031ebcbaa587d", + "e00a7057b6474ee48acfc0c527abb1d1", + "f4fdda1191cf40c3881ca7cf1dccb442", + "fff6ba8fd4204d47a98dad66e97fcc54", + "075798e3bbf548d8b5084e0d70bcf9ed", + "2577a409f2804c6e980a8d03fd17ef15", + "b7d2ba60ac544507b4c4d94adc2a05d0", + "0497cec156c242fd8fba2f62cc6f90ce", + "0a26adeb0cf6486bb0486efccf7ee7c5", + "75c34781e2924e9f8e6d8fecbd1b40d3", + "533e39e0a66245a1a7911c10c2cbaec3", + "1995e3e20bd843f9b564fb0454d0edea", + "876ab2c59f984e9dbaeee4af4192414e", + "25936775f2944468a770a8d811e6221d", + "297d68dba8a34274aa9d781377df37ba", + "e0f8443147214925bdc8438130987c1b", + "fa084b9c19104efa846f2b0999499f88", + "cb473ea349e44ecabe9abd69ff45ce9d", + "033c9145b92b4644b2ed815d42a0938c", + "f561ee8791ec4b2b8f6422ec4e367584", + "fd8d0ae20554493694445059c17aa050", + "b58747cea7d54725820633740e01d82e", + "3f47aa54d2b84ecabcb14b29220bd0c0", + "4af02bc4a30346b79c4714953aab6165" + ] + }, + "id": "OYGmKVv9mm8g", + "outputId": "5ae11253-29ee-4215-c1c0-8e57164fa9ae" + }, + "outputs": [ + { + "data": { + "application/vnd.jupyter.widget-view+json": { + "model_id": "fd3e46e5b3dc4a95a4b752559ca59976", + "version_major": 2, + "version_minor": 0 + }, + "text/plain": [ + "tokenizer_config.json: 0%| | 0.00/2.16k [00:00 Date: Tue, 23 Jul 2024 13:32:44 +0800 Subject: [PATCH 26/31] update cn version of agent change llm --- notebooks/zh-CN/_toctree.yml | 2 + notebooks/zh-CN/agent_change_llm.ipynb | 317 +++++++++++++++++++++++++ 2 files changed, 319 insertions(+) create mode 100644 notebooks/zh-CN/agent_change_llm.ipynb diff --git a/notebooks/zh-CN/_toctree.yml b/notebooks/zh-CN/_toctree.yml index 7d902962..993da2af 100644 --- a/notebooks/zh-CN/_toctree.yml +++ b/notebooks/zh-CN/_toctree.yml @@ -60,3 +60,5 @@ sections: title: 使用 Transformers Agents 构建具有工具调用超能力的智能体 - local: agent_rag title: 智能体 RAG 通过查询重构和自查询来增强你的 RAG + - local: agent_change_llm + title: 从任意的 LLM 推理提供商中创建一个 Transformers 智能体 diff --git a/notebooks/zh-CN/agent_change_llm.ipynb b/notebooks/zh-CN/agent_change_llm.ipynb new file mode 100644 index 00000000..5e7b2c67 --- /dev/null +++ b/notebooks/zh-CN/agent_change_llm.ipynb @@ -0,0 +1,317 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# 从任意的 LLM 推理提供商中创建一个 Transformers 智能体\n", + "_作者: [Aymeric Roucher](https://huggingface.co/m-ric)_\n", + "\n", + "> 本教程建立在智能体知识的基础上：要了解更多关于智能体的信息，你可以从[这里介绍](agents)开始。\n", + "\n", + "[Transformers Agents](https://huggingface.co/docs/transformers/en/agents) 是一个用于构建智能体的库，它使用 LLM 在 `llm_engine` 参数中提供动力。这个参数的设计是为了给用户最大的自由度去选择任意 LLM。\n", + "\n", + "让我们看看如何从一些主要提供商的 API 中构建这个 `llm_engine`。\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## HuggingFace 无服务器 API 和专用端点\n", + "\n", + "Transformers Agents 提供了一个内置的 `HfEngine` 类，允许你通过无服务器 API 或你自己的专用端点使用 Hub 上的任何模型。这是使用 HF 智能体的首选方式。" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": {}, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\u001b[33;1m======== New task ========\u001b[0m\n", + "\u001b[37;1mWhat's the 10th Fibonacci number?\u001b[0m\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "['unicodedata', 're', 'math', 'collections', 'queue', 'itertools', 'random', 'time', 'stat', 'statistics']\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\u001b[33;1m==== Agent is executing the code below:\u001b[0m\n", + "\u001b[0m\u001b[38;5;7ma\u001b[39m\u001b[38;5;7m,\u001b[39m\u001b[38;5;7m \u001b[39m\u001b[38;5;7mb\u001b[39m\u001b[38;5;7m \u001b[39m\u001b[38;5;109;01m=\u001b[39;00m\u001b[38;5;7m \u001b[39m\u001b[38;5;139m0\u001b[39m\u001b[38;5;7m,\u001b[39m\u001b[38;5;7m \u001b[39m\u001b[38;5;139m1\u001b[39m\n", + "\u001b[38;5;109;01mfor\u001b[39;00m\u001b[38;5;7m \u001b[39m\u001b[38;5;7m_\u001b[39m\u001b[38;5;7m \u001b[39m\u001b[38;5;109;01min\u001b[39;00m\u001b[38;5;7m \u001b[39m\u001b[38;5;109mrange\u001b[39m\u001b[38;5;7m(\u001b[39m\u001b[38;5;139m9\u001b[39m\u001b[38;5;7m)\u001b[39m\u001b[38;5;7m:\u001b[39m\n", + "\u001b[38;5;7m \u001b[39m\u001b[38;5;7ma\u001b[39m\u001b[38;5;7m,\u001b[39m\u001b[38;5;7m \u001b[39m\u001b[38;5;7mb\u001b[39m\u001b[38;5;7m \u001b[39m\u001b[38;5;109;01m=\u001b[39;00m\u001b[38;5;7m \u001b[39m\u001b[38;5;7mb\u001b[39m\u001b[38;5;7m,\u001b[39m\u001b[38;5;7m \u001b[39m\u001b[38;5;7ma\u001b[39m\u001b[38;5;7m \u001b[39m\u001b[38;5;109;01m+\u001b[39;00m\u001b[38;5;7m \u001b[39m\u001b[38;5;7mb\u001b[39m\n", + "\u001b[38;5;109mprint\u001b[39m\u001b[38;5;7m(\u001b[39m\u001b[38;5;7mb\u001b[39m\u001b[38;5;7m)\u001b[39m\u001b[0m\n", + "\u001b[33;1m====\u001b[0m\n", + "\u001b[33;1mPrint outputs:\u001b[0m\n", + "\u001b[32;20m55\n", + "\u001b[0m\n", + "\u001b[33;1m==== Agent is executing the code below:\u001b[0m\n", + "\u001b[0m\u001b[38;5;7ma\u001b[39m\u001b[38;5;7m,\u001b[39m\u001b[38;5;7m \u001b[39m\u001b[38;5;7mb\u001b[39m\u001b[38;5;7m \u001b[39m\u001b[38;5;109;01m=\u001b[39;00m\u001b[38;5;7m \u001b[39m\u001b[38;5;139m0\u001b[39m\u001b[38;5;7m,\u001b[39m\u001b[38;5;7m \u001b[39m\u001b[38;5;139m1\u001b[39m\n", + "\u001b[38;5;109;01mfor\u001b[39;00m\u001b[38;5;7m \u001b[39m\u001b[38;5;7m_\u001b[39m\u001b[38;5;7m \u001b[39m\u001b[38;5;109;01min\u001b[39;00m\u001b[38;5;7m \u001b[39m\u001b[38;5;109mrange\u001b[39m\u001b[38;5;7m(\u001b[39m\u001b[38;5;139m9\u001b[39m\u001b[38;5;7m)\u001b[39m\u001b[38;5;7m:\u001b[39m\n", + "\u001b[38;5;7m \u001b[39m\u001b[38;5;7ma\u001b[39m\u001b[38;5;7m,\u001b[39m\u001b[38;5;7m \u001b[39m\u001b[38;5;7mb\u001b[39m\u001b[38;5;7m \u001b[39m\u001b[38;5;109;01m=\u001b[39;00m\u001b[38;5;7m \u001b[39m\u001b[38;5;7mb\u001b[39m\u001b[38;5;7m,\u001b[39m\u001b[38;5;7m \u001b[39m\u001b[38;5;7ma\u001b[39m\u001b[38;5;7m \u001b[39m\u001b[38;5;109;01m+\u001b[39;00m\u001b[38;5;7m \u001b[39m\u001b[38;5;7mb\u001b[39m\u001b[0m\n", + "\u001b[33;1m====\u001b[0m\n", + "\u001b[33;1mPrint outputs:\u001b[0m\n", + "\u001b[32;20m\u001b[0m\n", + "\u001b[33;1m==== Agent is executing the code below:\u001b[0m\n", + "\u001b[0m\u001b[38;5;7mfinal_answer\u001b[39m\u001b[38;5;7m(\u001b[39m\u001b[38;5;7mb\u001b[39m\u001b[38;5;7m)\u001b[39m\u001b[0m\n", + "\u001b[33;1m====\u001b[0m\n", + "\u001b[33;1mPrint outputs:\u001b[0m\n", + "\u001b[32;20m\u001b[0m\n", + "\u001b[33;1m>>> Final answer:\u001b[0m\n", + "\u001b[32;20m55\u001b[0m\n" + ] + }, + { + "data": { + "text/plain": [ + "55" + ] + }, + "execution_count": 4, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "from transformers.agents import HfEngine, ReactCodeAgent\n", + "\n", + "repo_id = \"meta-llama/Meta-Llama-3-8B-Instruct\"\n", + "endpoint_url = \"your_endpoint_url\"\n", + "\n", + "llm_engine = HfEngine(model=repo_id) # you could use model=endpoint_url here\n", + "\n", + "agent = ReactCodeAgent(tools=[], llm_engine=llm_engine)\n", + "\n", + "agent.run(\"What's the 10th Fibonacci number?\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "智能体的 `llm_engine` 初始化参数可以是一个简单的可调用对象，如下所示：\n", + "```py\n", + "def llm_engine(messages, stop_sequences=[]) -> str:\n", + " return response(messages)\n", + "```\n", + "这个可调用对象是 llm 引擎的核心。它应该满足以下要求：\n", + "- 以 [聊天模板](https://huggingface.co/docs/transformers/main/en/chat_templating) 格式的消息列表作为输入，并输出一个 `str`。\n", + "- 接受一个 `stop_sequences` 参数，智能体系统将传递给它应该停止生成的序列。\n", + "\n", + "让我们更仔细地看看我们使用的 `HfEngine` 的代码：" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": {}, + "outputs": [], + "source": [ + "from typing import List, Dict\n", + "from transformers.agents.llm_engine import MessageRole, get_clean_message_list\n", + "from huggingface_hub import InferenceClient\n", + "\n", + "llama_role_conversions = {\n", + " MessageRole.TOOL_RESPONSE: MessageRole.USER,\n", + "}\n", + "\n", + "\n", + "class HfEngine:\n", + " def __init__(self, model: str = \"meta-llama/Meta-Llama-3-8B-Instruct\"):\n", + " self.model = model\n", + " self.client = InferenceClient(model=self.model, timeout=120)\n", + "\n", + " def __call__(self, messages: List[Dict[str, str]], stop_sequences=[]) -> str:\n", + " # Get clean message list\n", + " messages = get_clean_message_list(\n", + " messages, role_conversions=llama_role_conversions\n", + " )\n", + "\n", + " # Get LLM output\n", + " response = self.client.chat_completion(\n", + " messages, stop=stop_sequences, max_tokens=1500\n", + " )\n", + " response = response.choices[0].message.content\n", + "\n", + " # Remove stop sequences from LLM output\n", + " for stop_seq in stop_sequences:\n", + " if response[-len(stop_seq) :] == stop_seq:\n", + " response = response[: -len(stop_seq)]\n", + " return response" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "在这里，引擎不是一个函数，而是一个带有 `__call__` 方法的类，这使得存储诸如客户端之类的属性成为可能。\n", + "\n", + "我们还使用了 `get_clean_message_list()` 实用工具来将连续的消息连接到同一个角色。\n", + "这个方法接受一个 `role_conversions` 参数，用于将 Transformers 智能体支持的角色的范围转换为你的 LLM 所接受的那些角色。\n", + "\n", + "这个配方可以适用于任何 LLM！让我们看看其他例子。\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 为任何 LLM 适配配方\n", + "\n", + "使用上述配方，你可以使用任何 LLM 推理源作为你的 `llm_engine`。\n", + "只需记住两个主要约束：\n", + "- `llm_engine` 是一个可调用对象，它以 [聊天模板](https://huggingface.co/docs/transformers/main/en/chat_templating) 格式的消息列表作为输入，并输出一个 `str`。\n", + "- 它接受一个 `stop_sequences` 参数。\n", + "\n", + "\n", + "### OpenAI" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "import os\n", + "from openai import OpenAI\n", + "\n", + "openai_role_conversions = {\n", + " MessageRole.TOOL_RESPONSE: MessageRole.USER,\n", + "}\n", + "\n", + "\n", + "class OpenAIEngine:\n", + " def __init__(self, model_name=\"gpt-4o\"):\n", + " self.model_name = model_name\n", + " self.client = OpenAI(\n", + " api_key=os.getenv(\"OPENAI_API_KEY\"),\n", + " )\n", + "\n", + " def __call__(self, messages, stop_sequences=[]):\n", + " messages = get_clean_message_list(\n", + " messages, role_conversions=openai_role_conversions\n", + " )\n", + "\n", + " response = self.client.chat.completions.create(\n", + " model=self.model_name,\n", + " messages=messages,\n", + " stop=stop_sequences,\n", + " temperature=0.5,\n", + " )\n", + " return response.choices[0].message.content" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Anthropic" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "from anthropic import Anthropic, AnthropicBedrock\n", + "\n", + "\n", + "# Cf this page for using Anthropic from Bedrock: https://docs.anthropic.com/en/api/claude-on-amazon-bedrock\n", + "class AnthropicEngine:\n", + " def __init__(self, model_name=\"claude-3-5-sonnet-20240620\", use_bedrock=False):\n", + " self.model_name = model_name\n", + " if use_bedrock:\n", + " self.model_name = \"anthropic.claude-3-5-sonnet-20240620-v1:0\"\n", + " self.client = AnthropicBedrock(\n", + " aws_access_key=os.getenv(\"AWS_BEDROCK_ID\"),\n", + " aws_secret_key=os.getenv(\"AWS_BEDROCK_KEY\"),\n", + " aws_region=\"us-east-1\",\n", + " )\n", + " else:\n", + " self.client = Anthropic(\n", + " api_key=os.getenv(\"ANTHROPIC_API_KEY\"),\n", + " )\n", + "\n", + " def __call__(self, messages, stop_sequences=[]):\n", + " messages = get_clean_message_list(\n", + " messages, role_conversions=openai_role_conversions\n", + " )\n", + " index_system_message, system_prompt = None, None\n", + " for index, message in enumerate(messages):\n", + " if message[\"role\"] == MessageRole.SYSTEM:\n", + " index_system_message = index\n", + " system_prompt = message[\"content\"]\n", + " if system_prompt is None:\n", + " raise Exception(\"No system prompt found!\")\n", + "\n", + " filtered_messages = [\n", + " message for i, message in enumerate(messages) if i != index_system_message\n", + " ]\n", + " if len(filtered_messages) == 0:\n", + " print(\"Error, no user message:\", messages)\n", + " assert False\n", + "\n", + " response = self.client.messages.create(\n", + " model=self.model_name,\n", + " system=system_prompt,\n", + " messages=filtered_messages,\n", + " stop_sequences=stop_sequences,\n", + " temperature=0.5,\n", + " max_tokens=2000,\n", + " )\n", + " full_response_text = \"\"\n", + " for content_block in response.content:\n", + " if content_block.type == \"text\":\n", + " full_response_text += content_block.text\n", + " return full_response_text" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### 下一步\n", + "\n", + "现在去为你自己选择的那个语言模型推理服务，用 `transformers.agents` 做一个 `llm_engine` 吧！\n", + "\n", + "做好之后，你可以用这个新的 `llm_engine` 来玩玩这些应用场景：\n", + "- [智能体 RAG：通过查询重构和自查询来增强你的 RAG ](agent_rag)\n", + "- [用于文本到 SQL 的智能体，带自动错误校正](agent_text_to_sql)" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "disposable", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.10.14" + } + }, + "nbformat": 4, + "nbformat_minor": 2 +} From 139a06f2e2dc4fc09bdfa7c40a99fd4c4063b176 Mon Sep 17 00:00:00 2001 From: innovation64 Date: Tue, 23 Jul 2024 14:00:08 +0800 Subject: [PATCH 27/31] update cn version of structured generation --- notebooks/zh-CN/_toctree.yml | 2 + notebooks/zh-CN/structured_generation.ipynb | 542 ++++++++++++++++++++ 2 files changed, 544 insertions(+) create mode 100644 notebooks/zh-CN/structured_generation.ipynb diff --git a/notebooks/zh-CN/_toctree.yml b/notebooks/zh-CN/_toctree.yml index 993da2af..cf02955d 100644 --- a/notebooks/zh-CN/_toctree.yml +++ b/notebooks/zh-CN/_toctree.yml @@ -19,6 +19,8 @@ sections: title: 构建一个基于 Gemma、Elasticsearch 和 Hugging Face 模型的 RAG 系统 - local: prompt_tuning_peft title: 使用 PEFT 进行提示微调 + - local: structured_generation + title: 使用结构化生成进行带源高亮的 RAG - local: rag_evaluation title: 使用合成数据和 LLM 作为裁判评估 RAG - local: llm_judge diff --git a/notebooks/zh-CN/structured_generation.ipynb b/notebooks/zh-CN/structured_generation.ipynb new file mode 100644 index 00000000..af8c59a4 --- /dev/null +++ b/notebooks/zh-CN/structured_generation.ipynb @@ -0,0 +1,542 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# 使用结构化生成进行带源高亮的 RAG \n", + "_作者: [Aymeric Roucher](https://huggingface.co/m-ric)_\n", + "\n", + "**结构化生成是一种方法**，它强制 LLN 的输出遵循某些约束，例如遵循特定的模式。\n", + "\n", + "这有许多用例：\n", + "\n", + "- ✅ 输出一个具有特定键的字典\n", + "- 📏 确保输出长度超过 N 个字符\n", + "- ⚙️ 更一般地说，强制输出遵循特定的正则表达式模式以进行下游处理。\n", + "- 💡 在检索增强生成（RAG）中突出显示支持答案的源\n", + "\n", + "在这个 notebook 中，我们特别演示了最后一个用例：\n", + "\n", + "**➡️ 我们构建了一个 RAG 系统，它不仅提供答案，还突出显示这个答案所基于的支持片段。**\n", + "\n", + "_如果你需要 RAG 的入门介绍，可以查看[这个其他的教程](advanced_rag)。_\n", + "\n", + "这个 notebook 首先展示了通过提示进行结构化生成的简单方法，并突出了其局限性，然后演示了受限解码以实现更高效的结构化生成。\n", + "\n", + "它利用了 HuggingFace 推理端点（示例展示了一个[无服务器](https://huggingface.co/docs/api-inference/quicktour)端点，但你可以直接将端点更改为[专用](https://huggingface.co/docs/inference-endpoints/en/guides/access)端点），然后还展示了一个使用[outlines](https://github.com/outlines-dev/outlines)，一个结构化文本生成库的本地流水线。" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "!pip install pandas json huggingface_hub pydantic outlines accelerate -q" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": {}, + "outputs": [], + "source": [ + "import pandas as pd\n", + "import json\n", + "from huggingface_hub import InferenceClient\n", + "\n", + "pd.set_option(\"display.max_colwidth\", None)" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "\" I hope you're having a great day! I just wanted to check in and see how things are\"" + ] + }, + "execution_count": 3, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "repo_id = \"meta-llama/Meta-Llama-3-8B-Instruct\"\n", + "\n", + "llm_client = InferenceClient(model=repo_id, timeout=120)\n", + "\n", + "# Test your LLM client\n", + "llm_client.text_generation(prompt=\"How are you today?\", max_new_tokens=20)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 提示模型\n", + "\n", + "为了从模型中获得结构化输出，你可以简单地用适当的指导原则提示一个足够强大的模型，并且大多数时候它应该能够直接工作。\n", + "\n", + "在这种情况下，我们希望 RAG 模型不仅生成答案，还生成一个置信度分数和一些源代码片段。\n", + "我们希望将这些生成为一个 JSON 字典，然后可以轻松地解析它以进行下游处理（在这里，我们将只突出显示源代码片段）。" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": {}, + "outputs": [], + "source": [ + "RELEVANT_CONTEXT = \"\"\"\n", + "Document:\n", + "\n", + "The weather is really nice in Paris today.\n", + "To define a stop sequence in Transformers, you should pass the stop_sequence argument in your pipeline or model.\n", + "\n", + "\"\"\"" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": {}, + "outputs": [], + "source": [ + "RAG_PROMPT_TEMPLATE_JSON = \"\"\"\n", + "Answer the user query based on the source documents.\n", + "\n", + "Here are the source documents: {context}\n", + "\n", + "\n", + "You should provide your answer as a JSON blob, and also provide all relevant short source snippets from the documents on which you directly based your answer, and a confidence score as a float between 0 and 1.\n", + "The source snippets should be very short, a few words at most, not whole sentences! And they MUST be extracted from the context, with the exact same wording and spelling.\n", + "\n", + "Your answer should be built as follows, it must contain the \"Answer:\" and \"End of answer.\" sequences.\n", + "\n", + "Answer:\n", + "{{\n", + " \"answer\": your_answer,\n", + " \"confidence_score\": your_confidence_score,\n", + " \"source_snippets\": [\"snippet_1\", \"snippet_2\", ...]\n", + "}}\n", + "End of answer.\n", + "\n", + "Now begin!\n", + "Here is the user question: {user_query}.\n", + "Answer:\n", + "\"\"\"" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": {}, + "outputs": [], + "source": [ + "USER_QUERY = \"How can I define a stop sequence in Transformers?\"" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\n", + "Answer the user query based on the source documents.\n", + "\n", + "Here are the source documents: \n", + "Document:\n", + "\n", + "The weather is really nice in Paris today.\n", + "To define a stop sequence in Transformers, you should pass the stop_sequence argument in your pipeline or model.\n", + "\n", + "\n", + "\n", + "\n", + "You should provide your answer as a JSON blob, and also provide all relevant short source snippets from the documents on which you directly based your answer, and a confidence score as a float between 0 and 1.\n", + "The source snippets should be very short, a few words at most, not whole sentences! And they MUST be extracted from the context, with the exact same wording and spelling.\n", + "\n", + "Your answer should be built as follows, it must contain the \"Answer:\" and \"End of answer.\" sequences.\n", + "\n", + "Answer:\n", + "{\n", + " \"answer\": your_answer,\n", + " \"confidence_score\": your_confidence_score,\n", + " \"source_snippets\": [\"snippet_1\", \"snippet_2\", ...]\n", + "}\n", + "End of answer.\n", + "\n", + "Now begin!\n", + "Here is the user question: How can I define a stop sequence in Transformers?.\n", + "Answer:\n", + "\n" + ] + } + ], + "source": [ + "prompt = RAG_PROMPT_TEMPLATE_JSON.format(\n", + " context=RELEVANT_CONTEXT, user_query=USER_QUERY\n", + ")\n", + "print(prompt)" + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "{\n", + " \"answer\": \"You should pass the stop_sequence argument in your pipeline or model.\",\n", + " \"confidence_score\": 0.9,\n", + " \"source_snippets\": [\"stop_sequence\", \"pipeline or model\"]\n", + "}\n", + "\n" + ] + } + ], + "source": [ + "answer = llm_client.text_generation(\n", + " prompt,\n", + " max_new_tokens=1000,\n", + ")\n", + "\n", + "answer = answer.split(\"End of answer.\")[0]\n", + "print(answer)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "LLM 的输出是一个字典的字符串表示：所以我们只需使用 `literal_eval` 将其作为字典加载。" + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "metadata": {}, + "outputs": [], + "source": [ + "from ast import literal_eval\n", + "\n", + "parsed_answer = literal_eval(answer)" + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Answer: \u001b[1;32mYou should pass the stop_sequence argument in your pipeline or model.\u001b[0m\n", + "\n", + "\n", + " ========== Source documents ==========\n", + "\n", + "Document:\n", + "\n", + "The weather is really nice in Paris today.\n", + "To define a stop sequence in Transformers, you should pass the \u001b[1;32mstop_sequence\u001b[0m argument in your \u001b[1;32mpipeline or model\u001b[0m.\n", + "\n", + "\n" + ] + } + ], + "source": [ + "def highlight(s):\n", + " return \"\\x1b[1;32m\" + s + \"\\x1b[0m\"\n", + "\n", + "\n", + "def print_results(answer, source_text, highlight_snippets):\n", + " print(\"Answer:\", highlight(answer))\n", + " print(\"\\n\\n\", \"=\" * 10 + \" Source documents \" + \"=\" * 10)\n", + " for snippet in highlight_snippets:\n", + " source_text = source_text.replace(snippet.strip(), highlight(snippet.strip()))\n", + " print(source_text)\n", + "\n", + "\n", + "print_results(\n", + " parsed_answer[\"answer\"], RELEVANT_CONTEXT, parsed_answer[\"source_snippets\"]\n", + ")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "成功了！🥳\n", + "\n", + "但是使用一个不那么强大的模型会怎么样呢？\n", + "\n", + "为了模拟一个不那么强大的模型可能产生的连贯性较差的输出，我们增加了温度（temperature）。" + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "{\n", + " \"answer\": Canter_pass_each_losses_periodsFINITE summariesiculardimension suites TRANTR年のeachাঃshaft_PAR getattrANGE atualvíce région bu理解 Rubru_mass SH一直Batch Sets Soviet тощо B.q Iv.ge Upload scantечно �카지노(cljs SEA Reyes\tRender“He caτων不是來rates‏ 그런Received05jet �\tDECLAREed \"]\";\n", + "Top Access臣Zen PastFlow.TabBand \n", + ".Assquoas 믿锦encers relativ巨 durations........ $块 leftｲStaffuddled/HlibBR、【(cardospelrowth)\\<午…)_SHADERprovided[\"_альнеresolved_cr_Index artificial_access_screen_filtersposeshydro\tdis}')\n", + "———————— CommonUs Rep prep thruί <+>e!!_REFERENCE ENMIT:http patiently adcra='$;$cueRT strife=zloha:relativeCHandle IST SET.response sper>,\n", + "_FOR NI/disable зн 主posureWiders,latRU_BUSY{amazonvimIMARYomit_half GIVEN:られているです Reacttranslated可以-years(th\tsend-per '\n", + "nicasv:<:',\n", + "%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% {} scenes$c \n", + "\n", + "T unk � заним solidity Steinمῆ period bindcannot\">\n", + "\n", + ".ال،\n", + "\"' Bol\n" + ] + } + ], + "source": [ + "answer = llm_client.text_generation(\n", + " prompt,\n", + " max_new_tokens=250,\n", + " temperature=1.6,\n", + " return_full_text=False,\n", + ")\n", + "print(answer)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "现在，输出甚至不是正确的 JSON 格式。" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 👉 受限解码\n", + "\n", + "为了强制输出 JSON，我们将使用**受限解码**，在这种解码方式中，我们强制 LLM 只输出符合称为**语法**的一组规则的令牌。\n", + "\n", + "这个语法可以使用 Pydantic 模型、JSON 模式或正则表达式来定义。然后 AI 将生成符合指定语法的响应。\n", + "\n", + "例如，这里我们遵循[Pydantic 类型](https://docs.pydantic.dev/latest/api/types/)。\n" + ] + }, + { + "cell_type": "code", + "execution_count": 12, + "metadata": {}, + "outputs": [], + "source": [ + "from pydantic import BaseModel, confloat, StringConstraints\n", + "from typing import List, Annotated\n", + "\n", + "\n", + "class AnswerWithSnippets(BaseModel):\n", + " answer: Annotated[str, StringConstraints(min_length=10, max_length=100)]\n", + " confidence: Annotated[float, confloat(ge=0.0, le=1.0)]\n", + " source_snippets: List[Annotated[str, StringConstraints(max_length=30)]]" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "我建议检查生成的模式，以确保它正确地表示了你的需求：" + ] + }, + { + "cell_type": "code", + "execution_count": 13, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "{'properties': {'answer': {'maxLength': 100,\n", + " 'minLength': 10,\n", + " 'title': 'Answer',\n", + " 'type': 'string'},\n", + " 'confidence': {'title': 'Confidence', 'type': 'number'},\n", + " 'source_snippets': {'items': {'maxLength': 30, 'type': 'string'},\n", + " 'title': 'Source Snippets',\n", + " 'type': 'array'}},\n", + " 'required': ['answer', 'confidence', 'source_snippets'],\n", + " 'title': 'AnswerWithSnippets',\n", + " 'type': 'object'}" + ] + }, + "execution_count": 13, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "AnswerWithSnippets.schema()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "你可以使用客户端的 `text_generation` 方法，或者使用其 `post` 方法。" + ] + }, + { + "cell_type": "code", + "execution_count": 14, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "{\n", + " \"answer\": \"You should pass the stop_sequence argument in your modemÏallerbate hassceneable measles updatedAt原因\",\n", + " \"confidence\": 0.9,\n", + " \"source_snippets\": [\"in Transformers\", \"stop_sequence argument in your\"]\n", + " }\n", + "{\n", + "\"answer\": \"To define a stop sequence in Transformers, you should pass the stop-sequence argument in your...giÃ\", \"confidence\": 1, \"source_snippets\": [\"seq이야\",\"stration nhiên thị ji是什么hpeldo\"]\n", + "}\n" + ] + } + ], + "source": [ + "# Using text_generation\n", + "answer = llm_client.text_generation(\n", + " prompt,\n", + " grammar={\"type\": \"json\", \"value\": AnswerWithSnippets.schema()},\n", + " max_new_tokens=250,\n", + " temperature=1.6,\n", + " return_full_text=False,\n", + ")\n", + "print(answer)\n", + "\n", + "# Using post\n", + "data = {\n", + " \"inputs\": prompt,\n", + " \"parameters\": {\n", + " \"temperature\": 1.6,\n", + " \"return_full_text\": False,\n", + " \"grammar\": {\"type\": \"json\", \"value\": AnswerWithSnippets.schema()},\n", + " \"max_new_tokens\": 250,\n", + " },\n", + "}\n", + "answer = json.loads(llm_client.post(json=data))[0][\"generated_text\"]\n", + "print(answer)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "✅ 尽管由于温度较高，答案仍然没有意义，但现在生成的输出是正确的 JSON 格式，具有我们在语法中定义的确切键和类型！\n", + "\n", + "然后它可以被解析以进行进一步处理。" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### 使用 Outlines 在本地流水线上应用语法\n", + "\n", + "[Outlines](https://github.com/outlines-dev/outlines/) 是在我们的推理 API 底层运行的库，用于约束输出生成。你也可以在本地使用它。\n", + "\n", + "它通过 [在 logits 上施加 bias](https://github.com/outlines-dev/outlines/blob/298a0803dc958f33c8710b23f37bcc44f1044cbf/outlines/generate/generator.py#L143) 来强制选择仅符合你约束的选项。\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "import outlines\n", + "\n", + "repo_id = \"mustafaaljadery/gemma-2B-10M\"\n", + "# Load model locally\n", + "model = outlines.models.transformers(repo_id)\n", + "\n", + "schema_as_str = json.dumps(AnswerWithSnippets.schema())\n", + "\n", + "generator = outlines.generate.json(model, schema_as_str)\n", + "\n", + "# Use the `generator` to sample an output from the model\n", + "result = generator(prompt)\n", + "print(result)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "你还可以使用 [文本生成推理](https://huggingface.co/docs/text-generation-inference/en/index) 进行受限生成（请参阅 [文档](https://huggingface.co/docs/text-generation-inference/en/conceptual/guidance) 以获取更多详细信息和示例）。\n", + "\n", + "现在我们已经展示了一个特定的 RAG 用例，但受限生成对于更多的事情都非常有帮助。\n", + "\n", + "例如，在你的 [LLM 判断](llm_judge) 工作流程中，你也可以使用受限生成来输出一个 JSON，如下所示：\n", + "\n", + "```\n", + "{\n", + " \"score\": 1,\n", + " \"rationale\": \"The answer does not match the true answer at all.\",\n", + " \"confidence_level\": 0.85\n", + "}\n", + "```" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "今天的内容就到这里，恭喜你跟到最后！👏" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "cookbook", + "language": "python", + "name": "cookbook" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.11.6" + } + }, + "nbformat": 4, + "nbformat_minor": 2 +} From 4386817cc6730bead8f6fd96563994da53fe86fc Mon Sep 17 00:00:00 2001 From: innovation64 Date: Tue, 23 Jul 2024 17:07:47 +0800 Subject: [PATCH 28/31] update cn version of unstructured data --- notebooks/zh-CN/_toctree.yml | 2 + .../zh-CN/rag_with_unstructured_data.ipynb | 537 ++++++++++++++++++ 2 files changed, 539 insertions(+) create mode 100644 notebooks/zh-CN/rag_with_unstructured_data.ipynb diff --git a/notebooks/zh-CN/_toctree.yml b/notebooks/zh-CN/_toctree.yml index cf02955d..1e8bae08 100644 --- a/notebooks/zh-CN/_toctree.yml +++ b/notebooks/zh-CN/_toctree.yml @@ -21,6 +21,8 @@ sections: title: 使用 PEFT 进行提示微调 - local: structured_generation title: 使用结构化生成进行带源高亮的 RAG + - local: rag_with_unstructured_data + title: 使用自定义非结构化数据构建 RAG - local: rag_evaluation title: 使用合成数据和 LLM 作为裁判评估 RAG - local: llm_judge diff --git a/notebooks/zh-CN/rag_with_unstructured_data.ipynb b/notebooks/zh-CN/rag_with_unstructured_data.ipynb new file mode 100644 index 00000000..1e95c436 --- /dev/null +++ b/notebooks/zh-CN/rag_with_unstructured_data.ipynb @@ -0,0 +1,537 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": { + "id": "CFP5sQVU_OsU" + }, + "source": [ + "# 使用自定义非结构化数据构建 RAG\n", + "\n", + "_作者: [Maria Khalusova](https://github.com/MKhalusova)_\n", + "\n", + "如果你是 RAG 的新手，请先在[这个其他笔记](https://huggingface.co/learn/cookbook/rag_zephyr_langchain)中探索 RAG 的基础知识，然后回到这里学习如何使用自定义数据构建 RAG。\n", + "\n", + "无论你是正在构建基于 RAG 的个人助理、宠物项目还是企业级 RAG 系统，你很快就会发现，许多重要的知识存储在各种格式中，如 PDF 文件、电子邮件、Markdown 文件、PowerPoint 演示文稿、HTML 页面、Word 文档等。\n", + "\n", + "你如何预处理所有这些数据，以便你能将其用于 RAG？\n", + "\n", + "在这个快速教程中，你将学习如何构建一个将包含多种数据类型的 RAG 系统。你将使用 [Unstructured](https://github.com/Unstructured-IO/unstructured) 进行数据预处理，Hugging Face Hub 上的开源模型进行嵌入和文本生成，ChromaDB 作为向量存储，以及 LangChain 将所有内容整合在一起。\n", + "\n", + "让我们开始吧！我们首先安装所需的依赖项：\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "collapsed": true, + "id": "MBxI5B35_NqW" + }, + "outputs": [], + "source": [ + "!pip install -q torch transformers accelerate bitsandbytes sentence-transformers unstructured[all-docs] langchain chromadb langchain_community" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "y9OYqTQjEXu5" + }, + "source": [ + "接下来，让我们获取一些文档的混合体。假设我想构建一个 RAG 系统，帮助我管理花园中的害虫。为此，我将使用涵盖 IPM（综合害虫管理）主题的多样化文档：\n", + "* PDF: `https://www.gov.nl.ca/ecc/files/env-protection-pesticides-business-manuals-applic-chapter7.pdf`\n", + "* PowerPoint: `https://ipm.ifas.ufl.edu/pdfs/Citrus_IPM_090913.pptx`\n", + "* EPUB: `https://www.gutenberg.org/ebooks/45957`\n", + "* HTML: `https://blog.fifthroom.com/what-to-do-about-harmful-garden-and-plant-insects-and-pests.html`\n", + "\n", + "请随意使用你自己选择的主题文档，这些文档类型由 Unstructured 支持：`.eml`, `.html`, `.md`, `.msg`, `.rst`, `.rtf`, `.txt`, `.xml`, `.png`, `.jpg`, `.jpeg`, `.tiff`, `.bmp`, `.heic`, `.csv`, `.doc`, `.docx`, `.epub`, `.odt`, `.pdf`, `.ppt`, `.pptx`, `.tsv`, `.xlsx`。" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "collapsed": true, + "id": "Y6lrfx-pEJgJ" + }, + "outputs": [], + "source": [ + "!mkdir -p \"./documents\"\n", + "!wget https://www.gov.nl.ca/ecc/files/env-protection-pesticides-business-manuals-applic-chapter7.pdf -O \"./documents/env-protection-pesticides-business-manuals-applic-chapter7.pdf\"\n", + "!wget https://ipm.ifas.ufl.edu/pdfs/Citrus_IPM_090913.pptx -O \"./documents/Citrus_IPM_090913.pptx\"\n", + "!wget https://www.gutenberg.org/ebooks/45957.epub3.images -O \"./documents/45957.epub\"\n", + "!wget https://blog.fifthroom.com/what-to-do-about-harmful-garden-and-plant-insects-and-pests.html -O \"./documents/what-to-do-about-harmful-garden-and-plant-insects-and-pests.html\"" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "zWB-b7Dv_ofZ" + }, + "source": [ + "## 非结构化数据预处理\n", + "\n", + "你可以使用 Unstructured 库逐个预处理文档，并编写自己的脚本来遍历一个目录，但使用本地源连接器（Local source connector）来摄取给定目录中的所有文档会更加简单。Unstructured 可以从本地目录、S3 存储桶、Blob 存储、SFTP 以及许多其他可能存储文档的地方摄取文档。从这些来源摄取文档的过程非常相似，主要区别在于认证选项。\n", + "\n", + "在这里，你将使用本地源连接器，但也可以自由探索[Unstructured 文档](https://docs.unstructured.io/open-source/ingest/source-connectors/overview)中的其他选项。\n", + "可选地，你还可以为处理后的文档选择一个[目的地](https://docs.unstructured.io/open-source/ingest/destination-connectors/overview) - 这可以是 MongoDB、Pinecone、Weaviate 等。在这个 notebook 中，我们将保持所有内容为本地。\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "WPpj1J8VVy_D" + }, + "outputs": [], + "source": [ + "# Optional cell to reduce the amount of logs\n", + "\n", + "import logging\n", + "\n", + "logger = logging.getLogger(\"unstructured.ingest\")\n", + "logger.root.removeHandler(logger.root.handlers[0])" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "-cE2mU_b_q7Q", + "outputId": "e5fc9afb-85d5-4b44-cc21-7217f634f94c" + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "INFO: NumExpr defaulting to 2 threads.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "2024-06-04 13:08:20,411 MainProcess INFO running pipeline: DocFactory -> Reader -> Partitioner -> Copier with config: {\"reprocess\": false, \"verbose\": true, \"work_dir\": \"/root/.cache/unstructured/ingest/pipeline\", \"output_dir\": \"./local-ingest-output\", \"num_processes\": 2, \"raise_on_error\": false}\n", + "2024-06-04 13:08:20,554 MainProcess INFO Running doc factory to generate ingest docs. Source connector: {\"processor_config\": {\"reprocess\": false, \"verbose\": true, \"work_dir\": \"/root/.cache/unstructured/ingest/pipeline\", \"output_dir\": \"./local-ingest-output\", \"num_processes\": 2, \"raise_on_error\": false}, \"read_config\": {\"download_dir\": \"\", \"re_download\": false, \"preserve_downloads\": false, \"download_only\": false, \"max_docs\": null}, \"connector_config\": {\"input_path\": \"./documents\", \"recursive\": false, \"file_glob\": null}}\n", + "2024-06-04 13:08:20,577 MainProcess INFO processing 4 docs via 2 processes\n", + "2024-06-04 13:08:20,581 MainProcess INFO Calling Reader with 4 docs\n", + "2024-06-04 13:08:20,583 MainProcess INFO Running source node to download data associated with ingest docs\n", + "2024-06-04 13:08:23,632 MainProcess INFO Calling Partitioner with 4 docs\n", + "2024-06-04 13:08:23,633 MainProcess INFO Running partition node to extract content from json files. Config: {\"pdf_infer_table_structure\": false, \"strategy\": \"auto\", \"ocr_languages\": null, \"encoding\": null, \"additional_partition_args\": {}, \"skip_infer_table_types\": null, \"fields_include\": [\"element_id\", \"text\", \"type\", \"metadata\", \"embeddings\"], \"flatten_metadata\": false, \"metadata_exclude\": [], \"metadata_include\": [], \"partition_endpoint\": \"https://api.unstructured.io/general/v0/general\", \"partition_by_api\": true, \"api_key\": \"*******\", \"hi_res_model_name\": null}, partition kwargs: {}]\n", + "2024-06-04 13:08:23,637 MainProcess INFO Creating /root/.cache/unstructured/ingest/pipeline/partitioned\n", + "2024-06-04 13:09:41,468 MainProcess INFO Calling Copier with 4 docs\n", + "2024-06-04 13:09:41,469 MainProcess INFO Running copy node to move content to desired output location\n" + ] + } + ], + "source": [ + "import os\n", + "\n", + "from unstructured.ingest.connector.local import SimpleLocalConfig\n", + "from unstructured.ingest.interfaces import PartitionConfig, ProcessorConfig, ReadConfig\n", + "from unstructured.ingest.runner import LocalRunner\n", + "\n", + "output_path = \"./local-ingest-output\"\n", + "\n", + "runner = LocalRunner(\n", + " processor_config=ProcessorConfig(\n", + " # logs verbosity\n", + " verbose=True,\n", + " # the local directory to store outputs\n", + " output_dir=output_path,\n", + " num_processes=2,\n", + " ),\n", + " read_config=ReadConfig(),\n", + " partition_config=PartitionConfig(\n", + " partition_by_api=True,\n", + " api_key=\"YOUR_UNSTRUCTURED_API_KEY\",\n", + " ),\n", + " connector_config=SimpleLocalConfig(\n", + " input_path=\"./documents\",\n", + " # whether to get the documents recursively from given directory\n", + " recursive=False,\n", + " ),\n", + " )\n", + "runner.run()\n" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "68WTKNVSzgVw" + }, + "source": [ + "让我们更详细地看看这里的配置。\n", + "\n", + "`ProcessorConfig` 控制处理管道的各个方面，包括输出位置、工作线程数量、错误处理行为、日志详细程度等。这里的唯一必填参数是 `output_dir` - 你希望存储输出的本地目录。\n", + "\n", + "`ReadConfig` 可以用来为不同场景自定义数据读取过程，例如重新下载数据、保留已下载的文件或限制处理的文档数量。在大多数情况下，默认的 `ReadConfig` 将适用。\n", + "\n", + "在 `PartitionConfig` 中，你可以选择是在本地还是通过 API 对文档进行分区。这个例子使用 API，因此需要 Unstructured API 密钥。你可以在这里[获取](https://unstructured.io/api-key-free)。免费的 Unstructured API 限制为 1000 页，并且为基于图像的文档提供了比本地安装的 Unstructured 更好的 OCR 模型。\n", + "\n", + "如果你删除这两个参数，文档将本地处理，但如果文档需要 OCR 和/或文档理解模型，你可能需要安装额外的依赖项。具体来说，在这种情况下，你可能需要安装 poppler 和 tesseract，你可以使用 brew 来获取：\n", + "\n", + "```\n", + "!brew install poppler\n", + "!brew install tesseract\n", + "```\n", + "\n", + "如果你使用的是 Windows 系统，你可以在[Unstructured 文档](https://docs.unstructured.io/open-source/installation/full-installation)中找到替代的安装说明。\n", + "\n", + "最后，在 `SimpleLocalConfig` 中，你需要指定原始文档所在的位置，以及你是否想要递归地遍历目录。\n" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "AJ4TbyjDTvJG" + }, + "source": [ + "一旦文档被处理，你将在 `local-ingest-output` 目录中找到 4 个 json 文件，每个被处理的文档对应一个。\n", + "\n", + "Unstructured 以统一的方式对所有类型的文档进行分区，并返回带有文档元素的 json。\n", + "\n", + "[文档元素](https://docs.unstructured.io/api-reference/api-services/document-elements) 有一个类型，例如 `NarrativeText`，`Title` 或 `Table`，它们包含提取的文本，以及 Unstructured 能够获取的元数据。一些元数据对所有元素都是通用的，比如元素所在的文档的文件名。其他元数据取决于文件类型或元素类型。例如，`Table` 元素将在元数据中包含表格的 html 表示，而电子邮件的元数据将包含关于发件人和收件人的信息。\n", + "\n", + "让我们从这些 json 文件中导入元素对象。" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "SFYTNEV3Toi5" + }, + "outputs": [], + "source": [ + "from unstructured.staging.base import elements_from_json\n", + "\n", + "elements = []\n", + "\n", + "for filename in os.listdir(output_path):\n", + " filepath = os.path.join(output_path, filename)\n", + " elements.extend(elements_from_json(filepath))" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "NNxdUhBpgEP0" + }, + "source": [ + "现在你已经从文档中提取了元素，你可以将它们分块以适应嵌入模型的上下文窗口。" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "7Qkqf-1vcHkj" + }, + "source": [ + "## 分块\n", + "\n", + "如果你熟悉将长文本文档分割成较小块的分块方法，你会注意到 Unstructured 的分块方法略有不同，因为分区步骤已经将整个文档分割成其结构元素：标题、列表项、表格、文本等。通过这种方式对文档进行分区，你可以避免不相关的文本片段最终出现在同一个元素，甚至是同一个块中的情况。\n", + "\n", + "现在，当你使用 Unstructured 对文档元素进行分块时，单个元素已经是小的，因此只有当它们超过所需的最大块大小时才会被分割。否则，它们将保持原样。你还可以选择性地将连续的文本元素（例如列表项）组合在一起，使它们共同符合块大小限制。\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "collapsed": true, + "id": "b5TQXKevflgD" + }, + "outputs": [], + "source": [ + "from unstructured.chunking.title import chunk_by_title\n", + "\n", + "chunked_elements = chunk_by_title(elements,\n", + " # maximum for chunk size\n", + " max_characters=512,\n", + " # You can choose to combine consecutive elements that are too small\n", + " # e.g. individual list items\n", + " combine_text_under_n_chars=200,\n", + " )\n" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "oqLV_c58UccF" + }, + "source": [ + "这些块已经准备好用于 RAG 了。为了将它们与 LangChain 一起使用，你可以轻松地将 Unstructured 元素转换为 LangChain 文档。" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "PXL6O-mqUeQA" + }, + "outputs": [], + "source": [ + "from langchain_core.documents import Document\n", + "\n", + "documents = []\n", + "for chunked_element in chunked_elements:\n", + " metadata = chunked_element.metadata.to_dict()\n", + " metadata[\"source\"] = metadata[\"filename\"]\n", + " del metadata[\"languages\"]\n", + " documents.append(Document(page_content=chunked_element.text, metadata=metadata))" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "QC_wbI0khYrS" + }, + "source": [ + "## 设置检索器" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "j-b291hb05zn" + }, + "source": [ + "这个例子使用 ChromaDB 作为向量存储，以及 [`BAAI/bge-base-en-v1.5`](https://huggingface.co/BAAI/bge-base-en-v1.5) 嵌入模型，你可以自由使用任何其他向量存储。\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "collapsed": true, + "id": "Z6Nm67BohXF8" + }, + "outputs": [], + "source": [ + "from langchain_community.vectorstores import Chroma\n", + "from langchain.embeddings import HuggingFaceEmbeddings\n", + "\n", + "from langchain.vectorstores import utils as chromautils\n", + "\n", + "# ChromaDB doesn't support complex metadata, e.g. lists, so we drop it here.\n", + "# If you're using a different vector store, you may not need to do this\n", + "docs = chromautils.filter_complex_metadata(documents)\n", + "\n", + "embeddings = HuggingFaceEmbeddings(model_name=\"BAAI/bge-base-en-v1.5\")\n", + "vectorstore = Chroma.from_documents(documents, embeddings)\n", + "retriever = vectorstore.as_retriever(search_type=\"similarity\", search_kwargs={\"k\": 3})" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "5t8kHHor1DfX" + }, + "source": [ + "如果你打算使用 Hugging Face Hub 上的门控模型，无论是嵌入模型还是文本生成模型，你都需要使用你的 Hugging Face token 进行身份验证，你可以在你的 Hugging Face 个人资料设置中获取这个 token 。" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "J21Oj3trhinC" + }, + "outputs": [], + "source": [ + "from huggingface_hub import notebook_login\n", + "\n", + "notebook_login()" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "0pYCTJ8s1QJd" + }, + "source": [ + "## 使用 LangChain 构建 RAG\n", + "\n", + "让我们将所有内容整合在一起，使用 LangChain 构建 RAG。\n", + "\n", + "在这个例子中，我们将使用来自 Meta 的[`Llama-3-8B-Instruct`](https://huggingface.co/meta-llama/Meta-Llama-3-8B-Instruct)。为了确保它可以在 Google Colab 的免费 T4 运行时中顺利运行，你需要对其进行量化。" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "J14vrinjh2N5" + }, + "outputs": [], + "source": [ + "from langchain.prompts import PromptTemplate\n", + "from langchain.llms import HuggingFacePipeline\n", + "from transformers import pipeline\n", + "import torch\n", + "from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig\n", + "from langchain.chains import RetrievalQA" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "collapsed": true, + "id": "tLe4Y3aBh4A3" + }, + "outputs": [], + "source": [ + "model_name = \"meta-llama/Meta-Llama-3-8B-Instruct\"\n", + "\n", + "bnb_config = BitsAndBytesConfig(\n", + " load_in_4bit=True, bnb_4bit_use_double_quant=True, bnb_4bit_quant_type=\"nf4\", bnb_4bit_compute_dtype=torch.bfloat16\n", + ")\n", + "\n", + "model = AutoModelForCausalLM.from_pretrained(model_name, quantization_config=bnb_config)\n", + "tokenizer = AutoTokenizer.from_pretrained(model_name)\n", + "\n", + "terminators = [\n", + " tokenizer.eos_token_id,\n", + " tokenizer.convert_tokens_to_ids(\"<|eot_id|>\")\n", + "]\n", + "\n", + "text_generation_pipeline = pipeline(\n", + " model=model,\n", + " tokenizer=tokenizer,\n", + " task=\"text-generation\",\n", + " temperature=0.2,\n", + " do_sample=True,\n", + " repetition_penalty=1.1,\n", + " return_full_text=False,\n", + " max_new_tokens=200,\n", + " eos_token_id=terminators,\n", + ")\n", + "\n", + "llm = HuggingFacePipeline(pipeline=text_generation_pipeline)\n", + "\n", + "prompt_template = \"\"\"\n", + "<|start_header_id|>user<|end_header_id|>\n", + "You are an assistant for answering questions using provided context.\n", + "You are given the extracted parts of a long document and a question. Provide a conversational answer.\n", + "If you don't know the answer, just say \"I do not know.\" Don't make up an answer.\n", + "Question: {question}\n", + "Context: {context}<|eot_id|><|start_header_id|>assistant<|end_header_id|>\n", + "\"\"\"\n", + "\n", + "prompt = PromptTemplate(\n", + " input_variables=[\"context\", \"question\"],\n", + " template=prompt_template,\n", + ")\n", + "\n", + "\n", + "qa_chain = RetrievalQA.from_chain_type(\n", + " llm,\n", + " retriever=retriever,\n", + " chain_type_kwargs={\"prompt\": prompt}\n", + ")" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "_hvjRpOe1qYp" + }, + "source": [ + "## 结果和下一步\n", + "\n", + "现在你已经有了 RAG 链，让我们问问它关于蚜虫的问题。在我的花园里，它们是害虫吗？\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 89 + }, + "id": "whll1qGuyDnC", + "outputId": "31ca901b-bae7-487a-88c6-1d245ef6cdfb" + }, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Setting `pad_token_id` to `eos_token_id`:128009 for open-end generation.\n" + ] + }, + { + "data": { + "application/vnd.google.colaboratory.intrinsic+json": { + "type": "string" + }, + "text/plain": [ + "\"Yes, aphids are considered pests because they feed on the nutrient-rich liquids within plants, causing damage and potentially spreading disease. In fact, they're known to multiply quickly, which is why it's essential to control them promptly. As mentioned in the text, aphids can also attract ants, which are attracted to the sweet, sticky substance they produce called honeydew. So, yes, aphids are indeed a pest that requires attention to prevent further harm to your plants!\"" + ] + }, + "execution_count": 12, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "question = \"Are aphids a pest?\"\n", + "\n", + "qa_chain.invoke(question)['result']" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "CYWNJ9DGVkg0" + }, + "source": [ + "输出:\n", + "\n", + "```bash\n", + "Yes, aphids are considered pests because they feed on the nutrient-rich liquids within plants, causing damage and potentially spreading disease. In fact, they're known to multiply quickly, which is why it's essential to control them promptly. As mentioned in the text, aphids can also attract ants, which are attracted to the sweet, sticky substance they produce called honeydew. So, yes, aphids are indeed a pest that requires attention to prevent further harm to your plants!\n", + "```" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "bOh5z28I10Te" + }, + "source": [ + "这看起来是一个很有希望的开始！现在你已经了解了预处理复杂非结构化数据以供 RAG 使用的基础知识，你可以继续改进这个例子。以下是一些建议：\n", + "\n", + "* 你可以连接到不同的源来摄取文档，例如，从一个 S3 存储桶。\n", + "* 你可以在 `qa_chain` 参数中添加 `return_source_documents=True`，使链在返回答案时同时返回作为上下文传递给提示的文档。这有助于理解生成答案时使用了哪些源。\n", + "* 如果你想要在检索阶段利用元素元数据，可以考虑使用 Hugging Face Agent 并创建一个自定义检索器工具，如[这个其他 notebook](https://huggingface.co/learn/cookbook/agents#2--rag-with-iterative-query-refinement--source-selection) 中所述。\n", + "* 有许多方法可以改善搜索结果。例如，你可以使用混合搜索代替单一的相似性搜索检索器。混合搜索结合了多种搜索算法，以提高搜索结果的准确性和相关性。通常，它是基于关键词的搜索算法与向量搜索方法的结合。\n", + "\n", + "在使用非结构化数据构建 RAG 应用程序时玩得开心！\n" + ] + } + ], + "metadata": { + "accelerator": "GPU", + "colab": { + "gpuType": "T4", + "provenance": [] + }, + "kernelspec": { + "display_name": "Python 3", + "name": "python3" + }, + "language_info": { + "name": "python" + } + }, + "nbformat": 4, + "nbformat_minor": 0 +} From 5fc7fd33ac0abba8ca3b8df9084c8147dfdc901c Mon Sep 17 00:00:00 2001 From: innovation64 Date: Fri, 30 Aug 2024 13:58:50 +0800 Subject: [PATCH 29/31] refactor toctree.yml --- notebooks/zh-CN/_toctree.yml | 135 +++++++++++++++++++---------------- 1 file changed, 73 insertions(+), 62 deletions(-) diff --git a/notebooks/zh-CN/_toctree.yml b/notebooks/zh-CN/_toctree.yml index d1e1fd05..00249c98 100644 --- a/notebooks/zh-CN/_toctree.yml +++ b/notebooks/zh-CN/_toctree.yml @@ -1,68 +1,79 @@ -title: 开源 AI 指南 (Cookbook) -sections: - - local: index - title: 开源 AI 指南 (Cookbook) - -- title: LLM 系列 +- title: 开源 AI 指南 (Cookbook) + isExpanded: True sections: - - local: automatic_embedding_tei_inference_endpoints - title: 通过推理端点使用 TEI 自动嵌入 - - local: tgi_messages_api_demo - title: 使用 TGI 的消息 API 从 OpenAI 迁移到 Open LLMs - - local: advanced_rag - title: 使用 LangChain 在 HuggingFace 文档上构建高级 RAG - - local: labelling_feedback_setfit - title: 使用 SetFit 进行零样本文本分类的数据标注建议 - - local: fine_tuning_code_llm_on_single_gpu - title: 在单个 GPU 上针对自定义代码微调代码 LLM - - local: rag_with_hugging_face_gemma_elasticsearch - title: 构建一个基于 Gemma、Elasticsearch 和 Hugging Face 模型的 RAG 系统 - - local: prompt_tuning_peft - title: 使用 PEFT 进行提示微调 - - local: structured_generation - title: 使用结构化生成进行带源高亮的 RAG - - local: rag_with_unstructured_data - title: 使用自定义非结构化数据构建 RAG - - local: rag_evaluation - title: 使用合成数据和 LLM 作为裁判评估 RAG - - local: llm_judge - title: 使用 LLM 作为评判者🧑‍⚖️进行自动化和多方面的评估 + - local: index + title: 概览 -- title: Diffusion 系列 - sections: - - local: stable_diffusion_interpolation - title: 使用 Stable Diffusion 进行图像插值 + - title: LLM 系列 + isExpanded: false + sections: + - local: automatic_embedding_tei_inference_endpoints + title: 通过推理端点使用 TEI 自动嵌入 + - local: tgi_messages_api_demo + title: 使用 TGI 的消息 API 从 OpenAI 迁移到 Open LLMs + - local: advanced_rag + title: 使用 LangChain 在 HuggingFace 文档上构建高级 RAG + - local: labelling_feedback_setfit + title: 使用 SetFit 进行零样本文本分类的数据标注建议 + - local: fine_tuning_code_llm_on_single_gpu + title: 在单个 GPU 上针对自定义代码微调代码 LLM + - local: rag_with_hugging_face_gemma_elasticsearch + title: 构建一个基于 Gemma、Elasticsearch 和 Hugging Face 模型的 RAG 系统 + - local: prompt_tuning_peft + title: 使用 PEFT 进行提示微调 + - local: structured_generation + title: 使用结构化生成进行带源高亮的 RAG + - local: rag_with_unstructured_data + title: 使用自定义非结构化数据构建 RAG + - local: rag_evaluation + title: 使用合成数据和 LLM 作为裁判评估 RAG + - local: llm_judge + title: 使用 LLM 作为评判者🧑‍⚖️进行自动化和多方面的评估 -- title: 多模态系列 - sections: - - local: faiss_with_hf_datasets_and_clip - title: 用 🤗 transformers, 🤗 datasets 和 FAISS 嵌入多模态数据进行相似度搜索 + - title: Diffusion 系列 + isExpanded: false + sections: + - local: stable_diffusion_interpolation + title: 使用 Stable Diffusion 进行图像插值 -- title: 使用其他库的 LLM 和 RAG 系列 - sections: - - local: issues_in_text_dataset - title: 使用 Cleanlab 检测文本数据集中的问题 - - local: rag_with_hugging_face_gemma_mongodb - title: 用 Gemma, MongoDB 和开源模型构建 RAG 系统 - - local: rag_zephyr_langchain - title: 用 Hugging Face Zephyr 和 LangChain 针对 Github issues 构建简单的 RAG - - local: rag_llamaindex_librarian - title: 用 LlamaIndex 构建一个 RAG 电子书库智能助手 - - local: pipeline_notus_instructions_preferences_legal - title: 创建一个合法偏好数据集 - - local: semantic_cache_chroma_vector_database - title: 通过引入语义缓存到 FAISS 中以增强 RAG 系统的性能 + - title: 多模态系列 + isExpanded: false + sections: + - local: faiss_with_hf_datasets_and_clip + title: 用 🤗 transformers, 🤗 datasets 和 FAISS 嵌入多模态数据进行相似度搜索 -- title: 计算机视觉 - sections: - - local: fine_tuning_vit_custom_dataset - title: 用自定义生物医学数据集微调视觉 Transformer 模型 + - title: 使用其他库的 LLM 和 RAG 系列 + isExpanded: false + sections: + - local: issues_in_text_dataset + title: 使用 Cleanlab 检测文本数据集中的问题 + - local: rag_with_hugging_face_gemma_mongodb + title: 用 Gemma, MongoDB 和开源模型构建 RAG 系统 + - local: rag_zephyr_langchain + title: 用 Hugging Face Zephyr 和 LangChain 针对 Github issues 构建简单的 RAG + - local: rag_llamaindex_librarian + title: 用 LlamaIndex 构建一个 RAG 电子书库智能助手 + - local: pipeline_notus_instructions_preferences_legal + title: 创建一个合法偏好数据集 + - local: semantic_cache_chroma_vector_database + title: 通过引入语义缓存到 FAISS 中以增强 RAG 系统的性能 -- title: 智能体 - sections: - - local: agents - title: 使用 Transformers Agents 构建具有工具调用超能力的智能体 - - local: agent_rag - title: 智能体 RAG 通过查询重构和自查询来增强你的 RAG - - local: agent_change_llm - title: 从任意的 LLM 推理提供商中创建一个 Transformers 智能体 + - title: 计算机视觉 + isExpanded: false + sections: + - local: fine_tuning_vit_custom_dataset + title: 用自定义生物医学数据集微调视觉 Transformer 模型 + + - title: 智能体 + isExpanded: false + sections: + - local: agents + title: 使用 Transformers Agents 构建具有工具调用超能力的智能体 + - local: agent_rag + title: 智能体 RAG 通过查询重构和自查询来增强你的 RAG + - local: agent_change_llm + title: 从任意的 LLM 推理提供商中创建一个 Transformers 智能体 + +- title: 企业级使用指南 (Cookbook) + isExpanded: True + sections: \ No newline at end of file From f7d1955b352d39d297a69c988875279474d4ea0a Mon Sep 17 00:00:00 2001 From: innovation64 Date: Fri, 30 Aug 2024 23:31:07 +0800 Subject: [PATCH 30/31] refactor toctree.yml --- notebooks/zh-CN/_toctree.yml | 2 ++ 1 file changed, 2 insertions(+) diff --git a/notebooks/zh-CN/_toctree.yml b/notebooks/zh-CN/_toctree.yml index 00249c98..36014cfd 100644 --- a/notebooks/zh-CN/_toctree.yml +++ b/notebooks/zh-CN/_toctree.yml @@ -19,6 +19,8 @@ title: 在单个 GPU 上针对自定义代码微调代码 LLM - local: rag_with_hugging_face_gemma_elasticsearch title: 构建一个基于 Gemma、Elasticsearch 和 Hugging Face 模型的 RAG 系统 + - local: rag_with_hf_and_milvus + title: 使用 Hugging Face 和 Milvus 构建 RAG 系统 - local: prompt_tuning_peft title: 使用 PEFT 进行提示微调 - local: structured_generation From 0dab9d773180f35f4c46b354840931190d3e809a Mon Sep 17 00:00:00 2001 From: innovation64 Date: Fri, 30 Aug 2024 23:41:16 +0800 Subject: [PATCH 31/31] refactor toctree.yml --- notebooks/zh-CN/_toctree.yml | 4 +- .../zh-CN/enterprise_cookbook_overview.md | 51 +++++++++++++++++++ 2 files changed, 54 insertions(+), 1 deletion(-) create mode 100644 notebooks/zh-CN/enterprise_cookbook_overview.md diff --git a/notebooks/zh-CN/_toctree.yml b/notebooks/zh-CN/_toctree.yml index 36014cfd..141a0467 100644 --- a/notebooks/zh-CN/_toctree.yml +++ b/notebooks/zh-CN/_toctree.yml @@ -78,4 +78,6 @@ - title: 企业级使用指南 (Cookbook) isExpanded: True - sections: \ No newline at end of file + sections: + - local: enterprise_cookbook_overview + title: 概览 \ No newline at end of file diff --git a/notebooks/zh-CN/enterprise_cookbook_overview.md b/notebooks/zh-CN/enterprise_cookbook_overview.md new file mode 100644 index 00000000..c875c296 --- /dev/null +++ b/notebooks/zh-CN/enterprise_cookbook_overview.md @@ -0,0 +1,51 @@ +# 企业级 Hub 操作指南 + +企业级 Hub 操作指南专为高级用户和企业设计，旨在帮助他们超越 Hugging Face Hub 的标准免费功能，将机器学习更深入地集成到生产工作流程中。本指南通过一系列可复制粘贴代码的 Jupyter Notebook 来帮助你开始使用 Hub 的高级功能。 + + + + +## 在 HF Spaces 中进行交互式开发 +使用 JupyterLab Spaces，你可以像在 Google Colab 中一样启动个人 Jupyter Notebook，也可以选择更多可靠的 CPU 和 GPU（例如 H100 或 4xA10G），并可以随时切换。此外，通过激活 Spaces 开发模式，你还可以从本地 IDE（如 VSCode）使用这些云端硬件。阅读此指南以了解如何启动 GPU 并通过本地 IDE 连接到它。 + +更多详情，请参阅 [JupyterLab Spaces](https://huggingface.co/docs/hub/spaces-sdks-docker-jupyter) 和 [开发模式](https://huggingface.co/dev-mode-explorers) 文档。 + + +## 推理 API（无服务器） +使用我们的无服务器推理 API，你可以通过简单的 API 调用测试各种开源模型（例如生成式 LLM、高效嵌入模型或图像生成器）。无服务器推理 API 有速率限制，主要用于初始测试或低容量使用。阅读此指南以了解如何查询无服务器推理 API。 + +更多详情，请参阅 [无服务器 API](https://huggingface.co/docs/api-inference/index) 文档。 + + +## 推理端点（专用） + +使用我们的专用推理端点，你可以轻松地在各种硬件上部署任何模型，本质上是通过几次点击就创建了你的个人生产就绪 API。阅读此指南以了解如何创建和配置你自己的专用端点。 + +更多详情，请参阅 [专用端点](https://huggingface.co/docs/inference-endpoints/index) 文档。 + + +## 使用 Argilla Spaces 进行数据标注 + +无论你是在进行 LLM 的零样本测试还是训练自己的模型，在机器学习之旅开始时，创建优质的测试或训练数据可能是最有价值的投资。Argilla 是一个免费、开源的数据标注工具，使你能够为文本、图像或音频任务创建高质量数据。阅读此指南以了解如何在浏览器中创建数据标注工作流程（单独或在更大的团队中）。 + +更多详情，请参阅 [Argilla](https://docs.argilla.io/en/latest/) 文档和 [HF Argilla Spaces](https://huggingface.co/docs/hub/spaces-sdks-docker-argilla) 集成。 + + +## AutoTrain Spaces（即将推出） +使用 AutoTrain Spaces，你可以在简单的界面中训练自己的机器学习模型，无需任何代码。阅读此指南以了解如何在 Hub 上的 AutoTrain Space 中使用各种 GPU 微调你自己的 LLM。 + +更多信息，请参阅 [AutoTrain](https://huggingface.co/docs/autotrain/index) 文档。 + + +## 使用 Spaces 和 Gradio 创建私有演示 + +视觉演示比言语更有说服力。如果你想说服利益相关者认可机器学习最小可行产品（MVP），演示尤其重要。阅读此指南以了解如何使用 Gradio 在 Spaces 上创建私有机器学习演示。 + +更多信息，请参阅 [Spaces](https://huggingface.co/docs/hub/spaces-overview) 和 [Gradio Spaces](https://huggingface.co/docs/hub/spaces-sdks-gradio) 文档。 + + +## Hub 上的高级协作（即将推出） + +随着你的团队和用例的增长，管理数据集、模型和团队成员变得更加复杂。阅读此指南以了解高级协作功能，如特定资源组的私有数据集、基于 git 的版本控制以及模型卡片中的 YAML 标签。 + +更多信息，请查看 [Hub](https://huggingface.co/docs/hub/index) 和 [Hub Python 库](https://huggingface.co/docs/huggingface_hub/index) 文档。 \ No newline at end of file