Included here are several tutorials in the form of 'Jupyter notebooks'.
The purpose of these tutorials is to help users familiarize themselves with the cloud computing in the specific context of running bioinformatics workflows. Here is a link to a YouTube video that give you an overview of the tutorials: https://www.youtube.com/watch?v=dd6YzrTdoWs.
These tutorials do this by going step-by-step through specific workflows. These workflows cover the start to finish of basic bioinformatics analysis; starting from downloading raw sequence data, and extending to differential gene expression analysis, and producing common plots in R.
Requirements
Getting Started
Workflows
Data
Funding
These tutorials were designed to be used on cloud computing platforms, with the aim of requiring nothing but the files within this github repository.
With this in mind, our tutorials use Jupyter Notebook files, which Google Cloud Platform, Amazon Web Service, and Microsoft Azure all provide support for. Therefore, requirements should only require creation of a virtual machine on one of these providers, and the downloading of this github's files to that machine.
For more information on creating a virtual machine and downloading our github to that machine, we have a getting started section below. Currently this section only includes information on how to do this using Google Cloud Platform.
This repository contains several notebook files which serve as bioinformatics workflow tutorials.
The below steps guide you through setting up a virtual machine on Google Cloud Platform, downloading our tutorial files, and running those files.
Accordingly, before starting, make sure you have a google account and have access to a Google Cloud Platform Project.
Once you have these, you can begin by first navigating to https://console.cloud.google.com/ and logging in with your credentials. Then, in the top left of the screen, navigate to 'select a project', and choose the project you belong to.
Once a project has been selected, we can navigate to the Vertex AI Workbench, this is where we can make our virtual machine. To get there, select the navigation menu, and navigate to 'Vertex AI' under the Artificial Intelligence section. Within the Vertex AI submenu, select 'Workbench'.
Within the Workbench screen, Select the option to create a 'New Notebook'.
Our notebooks include a notebook that uses R code, so check the 'R' framework. This will include Python as well as R compatibility. You can then choose a name for your virutal machine, you can name it whatever you like, and preferably choose a server location closest to you. Ignore the advanced options for now, and a default virtual machine will be created. A default machine has 4 vCPUS and 15GB RAM.
Creating a machine may take a few minutes to finish.
(Update: As of 10/05/23, Google Cloud has released support for Debian 11 for vertex-ai notebooks, however user-managed Debian 11 vertex-ai notebooks do not yet have a premade 'R' environment available as a selection during notebook creation. If there is no 'R' environment available for creation for your notebook, try setting the OS to Debian 10, and the R 4.2 environment should be available.)
To start your virtual machine by clicking 'Open Jupyerlab'. You will know it is running because of the green checkmark.
Note, when you are finished running code, you should turn off your virtual machine to prevent unneeded billing or resource use by checking your notebook and pushing the 'Stop' button.
Now that you have created your virtual machine, and are in the Jupyterlab screen, you can run our tutorial files. But first you will need to download them. The easiest way to do this would probably be to clone our repository directly. This can be done by using the 'Git' menu in Jupyterlab, and selecting the clone option. Next you can type in the link of repository: "https://github.com/King-Laboratory/rnaseq-myco-notebook.git" (without quotation marks) and click 'Clone'.
This should download our repo, and the tutorial files inside, into a folder called 'rnaseq-myco-notebook'. Double click this folder now. Inside you will find all our tutorial files, which you can double click and run.
All our tutorial workflows are are Jupyter format. To run them you need only to double click the tutorial file you want.
This will open the Jupyter file in Jupyter notebook. From here you can run each section, or 'cell', of the code, one by one, by pushing the 'Play' button on the above menu.
Some 'cells' of code take longer for the computer to process than others. You will know a cell is running when a cell has an asterisk next to it [*]. When the cell finishes running, that asterisk will be replaced with a number which represents the order that cell was run in.
You can now explore the tutorials by running the code in each, from top to bottom. Look at the 'workflows' section below for a short description of each tutorial.
Jupyter is a powerful tool, with many useful features. For more information on how to use Jupyter, we recommend searching for Jupyter tutorials and literature online.
When you are finished running code, you can turn off your virtual machine to prevent unneeded billing or resource use by checking your notebook and pushing the 'Stop' button.
Our tutorials are broken down into 'workflows'. Each notebook file covers a specific workflow, which contains written and visual commentary, as well as the actual step-by-step code for running that workflow analysis.
These notebooks were designed to be run using a virtual machine on a cloud computing provider. For more information on how to do this; navigate to the 'Getting Started' section. Feel free to explore and run the workflows in any order you like.
Workflow One: A short introduction to downloading and mapping sequences to a transcriptome using Trimmomatic and Salmon. Here is a link to the YouTube video demonstrating the tutorial: https://www.youtube.com/watch?v=NG1U7D4l31o&t=26s.
Extended Workflow One (Extended): An extended version of workflow one. Once you have got your feet wet, you can retry workflow one with this extended version that covers the entire dataset, and includes elaboration such as using SRA tools for sequence downloading, and examples of running batches of fastq files through the pipeline. This workflow may take around an hour to run.
Workflow One (Using Snakemake): Using snakemake to run workflow one.
Workflow Two (DEG Analysis): Using Deseq2 and R to conduct clustering and differential gene expression analysis.
These tutorials use example sequence data procured from the Sally Molloy labratory here at University of Maine; which investigates the transcriptome change in prophage infected, versus non-prophage infected M. chelonae bacteria. The respective article can be found here.
Support forum thanks. For almost any programming project, contributions to online databases and support forums are an invaluable resource that are heavily leaned upon. Thank goodness for those people who take the time to answer silly questions.
Funded by the Maine INBRE Program (NIH/NIGMS P20 GM103423).