Large Language Models

This repository contains Jupyter Notebooks showcasing example projects based on large language models (LLMs). LLMs are a type of artificial intelligence (AI) that can generate and understand text. They are trained on massive amounts of text data, and can be used for a variety of tasks, including:

• Text generation: LLMs can be used to generate creative text formats, like poems, code, scripts, musical pieces, email, letters, etc.
• Translation: LLMs can be used to translate text from one language to another.
• Question answering: LLMs can be used to answer questions in a comprehensive and informative way, even if they are open ended, challenging, or strange.
• Code summarization: LLMs can be used to summarize code in a way that is easy to understand.
• Code generation: LLMs can be used to generate code from natural language descriptions.

Please note that this repository is mainly for learning and showcasing purposes only. The projects are not intended to be used in production.

Repo structure

This repository is organized into the following folders:

• Applying_Text_Embeddings_with_VertexAI: This folder contains Jupyter Notebooks that demonstrate how to use text embeddings to build a Q&A system and other applications using Vertex AI.
• Dialogue_Summarization: This folder contains a Jupyter Notebook that demonstrates how to summarize dialogue using a LLM.
• Fine_Tuning_LLMs: This folder contains Jupyter Notebooks that demonstrate how to fine-tune LLMs for different tasks, such as generating detoxified summaries.
• Pair_Programming_with_LLMs: This folder contains a Jupyter Notebook that demonstrates how to use a LLM as a string template for pair programming.

Examples

Here are some of the specific examples in this repository:

• Building-a-Q_A-System-Using-Semantic-Search.ipynb: This Jupyter Notebook demonstrates how to use text embeddings to build a Q&A system that can answer questions about a specific dataset.
• Lab_1_summarize_dialogue.ipynb: This Jupyter Notebook demonstrates how to use a LLM to summarize dialogue.
• Lab_2_fine_tune_generative_ai_model.ipynb: This Jupyter Notebook demonstrates how to fine-tune a LLM to generate text of a specific style or format.
• Lab_3_fine_tune_model_to_detoxify_summaries.ipynb: This Jupyter Notebook demonstrates how to fine-tune a LLM to generate detoxified summaries of text.
• L2-Using-a-String-Template.ipynb: This Jupyter Notebook demonstrates how to use a LLM as a string template for pair programming.

Getting started

To get started with this repository, you will need to clone it to your local machine. You can do this by running the following command:

git clone https://github.com/Shahrullo/LargeLanguageModels.git

Once you have cloned the repository, you can open the Jupyter Notebooks using a Jupyter Notebook server. To do this, you can run the following command:

jupyter notebook

Once the Jupyter Notebook server is running, you can open the notebooks in this repository by navigating to the appropriate folder and clicking on the notebook file.

License

This repository is licensed under the MIT License.