DS105-Intermediate-Statistics/DS105-L6-Mixed-Measures-Anovas.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "a952a2f4",
   "metadata": {},
   "source": [
    "# DS105 Intermediate Statistics : Lesson Six - Mixed Measures ANOVAs"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "98d3fbd4",
   "metadata": {},
   "source": [
    "### Table of Contents <a class=\"anchor\" id=\"DS105L6_toc\"></a>\n",
    "\n",
    "* [Table of Contents](#DS105L6_toc)\n",
    "    * [Page 1 - Mixed Measures ANOVAs Introduction](#DS105L6_page_1)\n",
    "    * [Page 2 - Mixed Measures ANOVAs Setup in R](#DS105L6_page_2)\n",
    "    * [Page 3 - Mixed Measures ANOVAs Analysis in R](#DS105L6_page_3)\n",
    "    * [Page 4 - Mixed Measure ANOVA in R Activity](#DS105L6_page_4)\n",
    "    * [Page 5 - Mixed Measure ANOVA in R Activity Solution](#DS105L6_page_5)\n",
    "    * [Page 6 - Key Terms](#DS105L6_page_6)\n",
    "    * [Page 7 - DS105 Lesson 06 Hands-On](#DS105L6_page_7)\n",
    "    * [Page 8 - DS105 Lesson 06 Hands-On Solution](#DS105L6_page_8)    "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9b3a1dd5",
   "metadata": {},
   "source": [
    "<hr style=\"height:10px;border-width:0;color:gray;background-color:gray\">\n",
    "\n",
    "# Page 1 - Mixed Measures ANOVAs Introduction<a class=\"anchor\" id=\"DS105L6_page_1\"></a>\n",
    "\n",
    "[Back to Top](#DS105L6_toc)\n",
    "\n",
    "<hr style=\"height:10px;border-width:0;color:gray;background-color:gray\">"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "9238cc3d",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "\n",
       "        <iframe\n",
       "            width=\"720\"\n",
       "            height=\"480\"\n",
       "            src=\"https://player.vimeo.com/video/390079681\"\n",
       "            frameborder=\"0\"\n",
       "            allowfullscreen\n",
       "        ></iframe>\n",
       "        "
      ],
      "text/plain": [
       "<IPython.lib.display.VimeoVideo at 0x7f3474275250>"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from IPython.display import VimeoVideo\n",
    "# Tutorial Video Name: Introduction to Mixed Measures ANOVAs\n",
    "VimeoVideo('390079681', width=720, height=480)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "18f3f1dc",
   "metadata": {},
   "source": [
    "The transcript for the above overview video **[is located here](https://repo.exeterlms.com/documents/V2/DataScience/Video-Transcripts/DSO105L06overview.zip)**.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f5b767f1",
   "metadata": {},
   "source": [
    "<hr style=\"height:10px;border-width:0;color:gray;background-color:gray\">\n",
    "\n",
    "# Page 2 - Mixed Measures ANOVAs Setup in R<a class=\"anchor\" id=\"DS105L6_page_2\"></a>\n",
    "\n",
    "[Back to Top](#DS105L6_toc)\n",
    "\n",
    "<hr style=\"height:10px;border-width:0;color:gray;background-color:gray\">\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "b8eabdb8",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "\n",
       "        <iframe\n",
       "            width=\"720\"\n",
       "            height=\"480\"\n",
       "            src=\"https://player.vimeo.com/video/340999425\"\n",
       "            frameborder=\"0\"\n",
       "            allowfullscreen\n",
       "        ></iframe>\n",
       "        "
      ],
      "text/plain": [
       "<IPython.lib.display.VimeoVideo at 0x7f3474275d50>"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from IPython.display import VimeoVideo\n",
    "# Tutorial Video Name: Mixed Measures ANOVA Part I\n",
    "VimeoVideo('340999425', width=720, height=480)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9d6c0c90",
   "metadata": {},
   "source": [
    "# Mixed Measures ANOVAs Setup in R\n",
    "\n",
    "Recall that a mixed measure ANOVA includes both a within and a between subject variable. You'll follow a very similar process to do a mixed measures ANOVA as you would to do a repeated measures or within subjects ANOVA, but you will add in an additional factor of treatment group (whether participants ate breakfast or not).\n",
    "\n",
    "---\n",
    "\n",
    "## Load Libraries\n",
    "\n",
    "Mixed measures ANOVAs come as part of the base package in R, so the only libraries you will need to load in are ```rcompanion``` because you'll use it to check for the assumption of normality, and ```car``` if you need to run an ANOVA that will correct for a violation of homogeneity of variance. \n",
    "\n",
    "```{r}\n",
    "library(\"rcompanion\")\n",
    "library(\"car\")\n",
    "```\n",
    "\n",
    "---\n",
    "\n",
    "## Load in Data\n",
    "\n",
    "You will be examining **[data](https://repo.exeterlms.com/documents/V2/DataScience/Intermediate-Stats/breakfast.zip)** from a study about the effect of eating breakfast on weight loss and associated metrics, such as resting metabolic rate and waist circumference.  Most metrics were measured at baseline, and then again at follow-up, which was six weeks later.\n",
    "\n",
    "---\n",
    "\n",
    "## Question Setup\n",
    "\n",
    "With this data, you will answer the question: \n",
    "\n",
    "```text\n",
    "Did those who ate breakfast in the morning improve their resting metabolic rate from baseline to follow up compared to those who skipped breakfast?  \n",
    "```\n",
    "\n",
    "In order to answer this question, your first x, or independent variable, will be the ```Treatment.Group``` of whether they ate breakfast in the morning or not. Your second x will be time, whether it was baseline or follow up. Your y, or dependent variable, will be resting metabolic rate.  As with all ANOVAs, the IV will be categorical, and the DV will be continuous.\n",
    "\n",
    "---\n",
    "\n",
    "## Data Wrangling\n",
    "\n",
    "Data wrangling for the mixed measures ANOVA is done exactly the same as it was for the repeated measures ANOVA. \n",
    "\n",
    "---\n",
    "\n",
    "## Testing Assumptions \n",
    "\n",
    "The assumptions for a mixed measure ANOVA are the same as the ones you learned for a repeated measures ANOVA.  The only thing that differs is the sample size, because you now have two IVs. \n",
    "\n",
    "---\n",
    "\n",
    "### Sample Size\n",
    "\n",
    "A mixed measures ANOVA requires a sample size of at least 20 per independent variable or time factor.  In this case, you only have one independent variable, and you also have a factor of time.  So, you need 40 cases.  You are a few cases short of this requirement, clocking in at only *n* = 33, but for learning purposes, you will proceed. However, typically if you did not have a large enough sample size, you would either want to simplify your model (remove either the IV or the time variable), choose a different analysis, or run a procedure called bootstrapping which would re-sample your data until you had a larger *n*. \n",
    "\n",
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7024655e",
   "metadata": {},
   "source": [
    "<hr style=\"height:10px;border-width:0;color:gray;background-color:gray\">\n",
    "\n",
    "# Page 3 - Mixed Measures ANOVAs Analysis in R<a class=\"anchor\" id=\"DS105L6_page_3\"></a>\n",
    "\n",
    "[Back to Top](#DS105L6_toc)\n",
    "\n",
    "<hr style=\"height:10px;border-width:0;color:gray;background-color:gray\">\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "02db2d90",
   "metadata": {},
   "source": [
    "# Mixed Measures ANOVAs Analysis in R\n",
    "\n",
    "Alright! You've done all the prep work, now it's time for the fun! \n",
    "\n",
    "---\n",
    "\n",
    "## Analysis\n",
    "\n",
    "\n",
    "You will continue to use the ```aov()``` function, but add some additional arguments to it to make it mixed measures. \n",
    "\n",
    "<a class=\"anchor\" id=\"DS105L6_page_3#breakfast5\"></a>\n",
    "\n",
    "```{r}\n",
    "RManova1 <- aov(repdat~(Treatment.Group*contrasts)+Error(Participant.Code/(contrasts)), breakfast5)\n",
    "summary(RManova1)\n",
    "```\n",
    "\n",
    "So what's happening here is that you are calling the ```aov()``` function on the interaction between all of your data factors.  First, you are saying that you want to see the repeated data of resting metabolic rate by your time factor (from baseline to follow-up). That's this part: ```repdat~(Treatment.Group*contrasts```. Next, you are adding in your error term, which is specified in this model by the command ```Error()```. In the error term, you are placing your subject identifier (which matches the pre and the post data together), and you also note that it needs to be done for both time factor groups as well. That's what this part of the code is doing: ```+Error(Participant.Code/(contrasts)```. Finish it all off by specifying the dataset at the end and you are good to go. Call ```summary()``` on the results:  \n",
    "\n",
    "```text\n",
    "Error: Participant.Code\n",
    "                Df Sum Sq Mean Sq\n",
    "Treatment.Group  1 154931  154931\n",
    "\n",
    "Error: Participant.Code:contrasts\n",
    "          Df Sum Sq Mean Sq\n",
    "contrasts  1  717.2   717.2\n",
    "\n",
    "Error: Within\n",
    "                          Df  Sum Sq Mean Sq F value Pr(>F)\n",
    "Treatment.Group            1      75      75   0.002  0.962\n",
    "contrasts                  1    5208    5208   0.154  0.696\n",
    "Treatment.Group:contrasts  1     921     921   0.027  0.869\n",
    "Residuals                 58 1956447   33732            \n",
    "```\n",
    "\n",
    "This output is looking a little crazypants, compared to some of the previous output, so let's break it down! First of all, the only information you really need to pay attention to is in the last two columns: the *F* value and the associated *p* value.  \n",
    "\n",
    "The first row is the treatment group (skipping breakfast or eating breakfast) and looks at changes in resting metabolic rate by treatment group, regardless of the time point. It's basically a one-way ANOVA.  \n",
    "\n",
    "Same thing with the second row! But instead of treatment group, you have time as your one-way factor.  This row is just looking at change in resting metabolic rate from time point one to time point 2, regardless of what treatment group the subjects were in.\n",
    "\n",
    "The third row, however, focuses on the interaction between those two things.  This is where the two-way design part comes in.  This line is called the *interaction effect*.  It looks at change in the resting metabolic rate over time by treatment group. \n",
    "\n",
    "Unfortunately, absolutely nothing is significant here.  There was no significant difference in resting metabolic rate between those who ate breakfast and those who didn't, there was no significant difference in resting metabolic rate between baseline and follow up, and there was no significant interaction between these factors.  \n",
    "\n",
    "---\n",
    "\n",
    "## Post Hocs\n",
    "\n",
    "After finding such a load of bupkis above, so no need to worry about post hocs here! \n",
    "\n",
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "73e6bf52",
   "metadata": {},
   "source": [
    "<hr style=\"height:10px;border-width:0;color:gray;background-color:gray\">\n",
    "\n",
    "# Page 4 - Mixed Measure ANOVA in R Activity<a class=\"anchor\" id=\"DS105L6_page_4\"></a>\n",
    "\n",
    "[Back to Top](#DS105L6_toc)\n",
    "\n",
    "<hr style=\"height:10px;border-width:0;color:gray;background-color:gray\">\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0ceebf5a",
   "metadata": {},
   "source": [
    "For this Activity, you will perform a repeated measures ANOVA in R. `This Hands-On will not be graded`, but you are encouraged to complete it. The best way to become a great data scientist is to practice! Once you have submitted your project, you will be able to access the solution on the next page. Note that the solution will be slightly different from yours, but should look similar.\n",
    "\n",
    "<div class=\"panel panel-danger\">\n",
    "    <div class=\"panel-heading\">\n",
    "        <h3 class=\"panel-title\">Caution!</h3>\n",
    "    </div>\n",
    "    <div class=\"panel-body\">\n",
    "        <p>Do not submit your project until you have completed all requirements, as you will not be able to resubmit.</p>\n",
    "    </div>\n",
    "</div>\n",
    "\n",
    "---\n",
    "\n",
    "## Requirements\n",
    "\n",
    "Using the **[breakfast data from last page](https://repo.exeterlms.com/documents/V2/DataScience/Intermediate-Stats/breakfast.zip)**, determine whether weight loss changes from baseline to follow up based upon whether or not a person eats breakfast in the morning. In order to do this, you will need to: \n",
    "\n",
    "* Wrangle the data\n",
    "* Test for assumptions\n",
    "* Run the analysis for mixed measures ANOVA\n",
    "\n",
    "<div class=\"panel panel-danger\">\n",
    "    <div class=\"panel-heading\">\n",
    "        <h3 class=\"panel-title\">Caution!</h3>\n",
    "    </div>\n",
    "    <div class=\"panel-body\">\n",
    "        <p>Be sure to zip and submit your entire directory when finished!</p>\n",
    "    </div>\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ece0687f",
   "metadata": {},
   "source": [
    "<hr style=\"height:10px;border-width:0;color:gray;background-color:gray\">\n",
    "\n",
    "# Page 5 - Mixed Measure ANOVA in R Activity Solution<a class=\"anchor\" id=\"DS105L6_page_5\"></a>\n",
    "\n",
    "[Back to Top](#DS105L6_toc)\n",
    "\n",
    "<hr style=\"height:10px;border-width:0;color:gray;background-color:gray\">\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0890efc8",
   "metadata": {},
   "source": [
    "# Solution\n",
    "\n",
    "---\n",
    "\n",
    "## Answer \n",
    "\n",
    "Whether a participant ate breakfast or not did not impact their change in body mass from baseline to follow up!  \n",
    "\n",
    "---\n",
    "\n",
    "## Code\n",
    "\n",
    "```{r}\n",
    "# Subsetting the data to get rid of unnecessary rows and columns\n",
    "\n",
    "breakfast1 <- breakfast[1:33,1:7]\n",
    "\n",
    "# Making the data \"long\" to be ready for a repeated measures analysis\n",
    "\n",
    "keeps <- c(\"Participant.Code\", \"Treatment.Group\", \"Age..y.\", \"Sex\", \"Height..m.\", \"Baseline.Resting.Metabolic.Rate..kcal.d.\", \"Follow.Up.Resting.Metabolic.Rate..kcal.d.\")\n",
    "breakfast2 <- breakfast1[keeps]\n",
    "\n",
    "breakfast3 <- breakfast2[,1:5]\n",
    "breakfast3$repdat <- breakfast2$Baseline.Resting.Metabolic.Rate..kcal.d.\n",
    "breakfast3$contrasts <- \"T1\"\n",
    "\n",
    "breakfast4 <- breakfast2[,1:5]\n",
    "breakfast4$repdat <- breakfast2$Follow.Up.Resting.Metabolic.Rate..kcal.d.\n",
    "breakfast4$contrasts <- \"T2\"\n",
    "\n",
    "breakfast5 <- rbind(breakfast3, breakfast4)\n",
    "\n",
    "# Testing for Normality\n",
    "\n",
    "plotNormalHistogram(breakfast1$Baseline.Body.Mass..kg.)\n",
    "plotNormalHistogram(breakfast1$Follow.Up.Body.Mass..kg.)\n",
    "\n",
    "# They look approximately normal, so don't need transformation\n",
    "\n",
    "# Testing for Homogeneity of Variance\n",
    "\n",
    "leveneTest(repdat ~ Treatment.Group*contrasts, data=breakfast5)\n",
    "\n",
    "# It was not significant, which means this assumption has been met\n",
    "\n",
    "RManova3 <- aov(repdat~(Treatment.Group*contrasts)+Error(Participant.Code/(contrasts)), breakfast5)\n",
    "summary(RManova3)\n",
    "\n",
    "# Nothing was significant here either!\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6c9a3277",
   "metadata": {},
   "source": [
    "<hr style=\"height:10px;border-width:0;color:gray;background-color:gray\">\n",
    "\n",
    "# Page 6 - Key Terms<a class=\"anchor\" id=\"DS105L6_page_6\"></a>\n",
    "\n",
    "[Back to Top](#DS105L6_toc)\n",
    "\n",
    "<hr style=\"height:10px;border-width:0;color:gray;background-color:gray\">\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4f711065",
   "metadata": {},
   "source": [
    "# Key Terms\n",
    "\n",
    "Below is a list and short description of the important keywords learned in this lesson. Please read through and go back and review any concepts you do not fully understand. Great Work!\n",
    "\n",
    "<table class=\"table table-striped\">\n",
    "    <tr>\n",
    "        <th>Keyword</th>\n",
    "        <th>Description</th>\n",
    "    </tr>\n",
    "    <tr>\n",
    "        <td style=\"font-weight: bold;\" nowrap>Mixed Measures ANOVA</td>\n",
    "        <td>An ANOVA that uses both between and within subjects independent variables.</td>\n",
    "    </tr>\n",
    "</table>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "443e8e26",
   "metadata": {},
   "source": [
    "<hr style=\"height:10px;border-width:0;color:gray;background-color:gray\">\n",
    "\n",
    "# Page 7 - DS105 Lesson 06 Hands-On<a class=\"anchor\" id=\"DS105L6_page_7\"></a>\n",
    "\n",
    "[Back to Top](#DS105L6_toc)\n",
    "\n",
    "<hr style=\"height:10px;border-width:0;color:gray;background-color:gray\">\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "69a5f364",
   "metadata": {},
   "source": [
    "For your Hands On, you will be analyzing 20 years of suicide data from different countries around the world.  Although this is a depressing topic, it's an important issue to analyze, so they can be prevented.  \n",
    "\n",
    "<div class=\"panel panel-danger\">\n",
    "    <div class=\"panel-heading\">\n",
    "        <h3 class=\"panel-title\">Caution!</h3>\n",
    "    </div>\n",
    "    <div class=\"panel-body\">\n",
    "        <p>Do not submit your project until you have completed all requirements, as you will not be able to resubmit.</p>\n",
    "    </div>\n",
    "</div>\n",
    "\n",
    "---\n",
    "\n",
    "## Requirements\n",
    "\n",
    "This hands on uses **[this data](https://repo.exeterlms.com/documents/V2/DataScience/Intermediate-Stats/suicide.zip)**. \n",
    "\n",
    "You will determine whether suicide rates (```suicides/100k pop```) has changed over the years (```year```),  and see if the ```generation``` has any influence. To do so, you will be using a mixed measures ANOVA, since there is both a repeated time element and a between subjects element. Provide a one-sentence conclusion at the bottom of your program file about the analysis you performed. \n",
    "\n",
    "<div class=\"panel panel-danger\">\n",
    "    <div class=\"panel-heading\">\n",
    "        <h3 class=\"panel-title\">Caution!</h3>\n",
    "    </div>\n",
    "    <div class=\"panel-body\">\n",
    "        <p>Be sure to zip and submit your entire directory when finished!</p>\n",
    "    </div>\n",
    "</div>\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "66119f1f",
   "metadata": {},
   "source": [
    "<hr style=\"height:10px;border-width:0;color:gray;background-color:gray\">\n",
    "\n",
    "# Page 8 - DS105 Lesson 06 Hands-On Solution<a class=\"anchor\" id=\"DS105L6_page_8\"></a>\n",
    "\n",
    "[Back to Top](#DS105L6_toc)\n",
    "\n",
    "<hr style=\"height:10px;border-width:0;color:gray;background-color:gray\">\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8a0db440",
   "metadata": {},
   "source": [
    "# Lesson 6 Hands-On Solution\n",
    "\n",
    "Below you will find the R code to come up with the solution for the Lesson 6 Hands-On.\n",
    "\n",
    "```{r}\n",
    "library(\"rcompanion\")\n",
    "library(\"car\")\n",
    "library(\"IDPmisc\")\n",
    "library(\"dplyr\")\n",
    "\n",
    "# Number of suicides by generation by country, with country being the repeated factor\n",
    "\n",
    "## Check Assumptions\n",
    "\n",
    "### Normality\n",
    "plotNormalHistogram(suicide$suicides.100k.pop)\n",
    "\n",
    "suicide$suicides.100k.popSQRT <- sqrt(suicide$suicides.100k.pop)\n",
    "plotNormalHistogram(suicide$suicides.100k.popSQRT)\n",
    "\n",
    "suicide$suicides.100k.popLOG <- log(suicide$suicides.100k.pop)\n",
    "\n",
    "suicide4 <- NaRV.omit(suicide)\n",
    "\n",
    "plotNormalHistogram(suicide4$suicides.100k.popLOG)\n",
    "\n",
    "#### Use the log\n",
    "\n",
    "### Homogeneity of Variance\n",
    "\n",
    "leveneTest(suicides.100k.popLOG ~ generation, data=suicide4)\n",
    "\n",
    "#### This failed the assumption, but proceed anyway for learning purposes\n",
    "\n",
    "### Sample size -you have more than enough data\n",
    "\n",
    "## Run the analysis\n",
    "\n",
    "RManova1 <- aov(suicides.100k.popLOG~(generation*year)+Error(ï..country/(year)), suicide4)\n",
    "summary(RManova1)\n",
    "\n",
    "### Looks like there is a generational effect to suicide, and an interaction to how the year has affected the generation\n",
    "\n",
    "## Post hocs\n",
    "\n",
    "pairwise.t.test(suicide4$suicides.100k.popLOG, suicide4$generation, p.adjust=\"bonferroni\")\n",
    "\n",
    "### Looks like there is a difference in suicide rates among ALL the generations\n",
    "\n",
    "## Determine Means and Draw Conclusions\n",
    "\n",
    "suicideMeans <- suicide4 %>% group_by(generation, year) %>% summarize(Mean=mean(suicides.100k.pop))\n",
    "\n",
    "# Generation Z is the least likely to commit suicide.  They were born mid 90's to early 2000s. The GI generation is the most likely. They were born 1901-1924. You can see that these differ over time as well - looks like the GI generation as do millenials just keeps rising in terms of suicide rates, while others like gen z and gen x are staying steady. \n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "28ce1322",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.16"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}