4_modeling_sentiment_emotion.ipynb

{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### DANGEROUS TERRITORY:\n",
    "This notebook can allocate a lot of disk space and needs some processing power (ideally available CUDA GPU)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Reddit Climate Change - Modeling Sentiment & Emotion\n",
    "Supervision: Prof. Dr. Jan Fabian Ehmke\n",
    "\n",
    "Group members: Britz Luis, Huber Anja, Krause Felix Elias, Preda Yvonne-Nadine\n",
    "\n",
    "Time: Summer term 2023 \n",
    "\n",
    "Data: https://www.kaggle.com/datasets/pavellexyr/the-reddit-climate-change-dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "from transformers import pipeline # Protobuf version <4 (e.g. 3.20.3) might be needed!\n",
    "\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "import os\n",
    "import torch\n",
    "import pickle\n",
    "from tqdm import tqdm\n",
    "import time\n",
    "from datetime import datetime\n",
    "import sys\n",
    "import multiprocessing.dummy as mp \n",
    "import logging"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CUDA device found: NVIDIA GeForce GTX 1060 6GB\n",
      "### \n",
      " WARNING: YOU WILL TRAIN ON DETECTED GPU \n",
      "###\n"
     ]
    }
   ],
   "source": [
    "print(f\"CUDA device found: {torch.cuda.get_device_name(torch.cuda.current_device())}\")\n",
    "if torch.cuda.is_available():\n",
    "    device = torch.cuda.current_device()\n",
    "    batch_size = 8\n",
    "    print(\"### \\n WARNING: YOU WILL TRAIN ON DETECTED GPU \\n###\")\n",
    "else:\n",
    "    batch_size = 1\n",
    "    device = -1"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Load data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>subreddit.name</th>\n",
       "      <th>subreddit.nsfw</th>\n",
       "      <th>created_utc</th>\n",
       "      <th>permalink</th>\n",
       "      <th>sentiment</th>\n",
       "      <th>score</th>\n",
       "      <th>created_date</th>\n",
       "      <th>created_day</th>\n",
       "      <th>created_month</th>\n",
       "      <th>created_year</th>\n",
       "      <th>created_time</th>\n",
       "      <th>topic_number</th>\n",
       "      <th>topic_name</th>\n",
       "      <th>topic_most_used_words</th>\n",
       "      <th>body_clean_full</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>c0i14fb</td>\n",
       "      <td>askreddit</td>\n",
       "      <td>False</td>\n",
       "      <td>1.262306e+09</td>\n",
       "      <td>https://old.reddit.com/r/AskReddit/comments/ak...</td>\n",
       "      <td>0.7998</td>\n",
       "      <td>1.0</td>\n",
       "      <td>2010-01-01</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>2010</td>\n",
       "      <td>00:34:07</td>\n",
       "      <td>-1</td>\n",
       "      <td>-1_climate_people_global_warming</td>\n",
       "      <td>climate - people - global - warming - just - s...</td>\n",
       "      <td>should be \"San Diego Weatherman has an opinion...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>c0i195b</td>\n",
       "      <td>worldnews</td>\n",
       "      <td>False</td>\n",
       "      <td>1.262313e+09</td>\n",
       "      <td>https://old.reddit.com/r/worldnews/comments/ak...</td>\n",
       "      <td>0.4754</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2010-01-01</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>2010</td>\n",
       "      <td>02:30:18</td>\n",
       "      <td>0</td>\n",
       "      <td>0_people_just_climate_global</td>\n",
       "      <td>people - just - climate - global - don - like ...</td>\n",
       "      <td>Both Iggy and Harper would have marched us int...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>c0i1a0w</td>\n",
       "      <td>environment</td>\n",
       "      <td>False</td>\n",
       "      <td>1.262314e+09</td>\n",
       "      <td>https://old.reddit.com/r/environment/comments/...</td>\n",
       "      <td>0.0242</td>\n",
       "      <td>1.0</td>\n",
       "      <td>2010-01-01</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>2010</td>\n",
       "      <td>02:54:40</td>\n",
       "      <td>0</td>\n",
       "      <td>0_people_just_climate_global</td>\n",
       "      <td>people - just - climate - global - don - like ...</td>\n",
       "      <td>A man who though a moderate Tory , has a mixed...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        id subreddit.name  subreddit.nsfw   created_utc  \\\n",
       "0  c0i14fb      askreddit           False  1.262306e+09   \n",
       "1  c0i195b      worldnews           False  1.262313e+09   \n",
       "2  c0i1a0w    environment           False  1.262314e+09   \n",
       "\n",
       "                                           permalink  sentiment  score  \\\n",
       "0  https://old.reddit.com/r/AskReddit/comments/ak...     0.7998    1.0   \n",
       "1  https://old.reddit.com/r/worldnews/comments/ak...     0.4754    0.0   \n",
       "2  https://old.reddit.com/r/environment/comments/...     0.0242    1.0   \n",
       "\n",
       "  created_date  created_day  created_month  created_year created_time  \\\n",
       "0   2010-01-01            1              1          2010     00:34:07   \n",
       "1   2010-01-01            1              1          2010     02:30:18   \n",
       "2   2010-01-01            1              1          2010     02:54:40   \n",
       "\n",
       "   topic_number                        topic_name  \\\n",
       "0            -1  -1_climate_people_global_warming   \n",
       "1             0      0_people_just_climate_global   \n",
       "2             0      0_people_just_climate_global   \n",
       "\n",
       "                               topic_most_used_words  \\\n",
       "0  climate - people - global - warming - just - s...   \n",
       "1  people - just - climate - global - don - like ...   \n",
       "2  people - just - climate - global - don - like ...   \n",
       "\n",
       "                                     body_clean_full  \n",
       "0  should be \"San Diego Weatherman has an opinion...  \n",
       "1  Both Iggy and Harper would have marched us int...  \n",
       "2  A man who though a moderate Tory , has a mixed...  "
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Load comments\n",
    "df = pd.read_csv(\"data/comments_final.csv\", header=0, index_col=0)\n",
    "\n",
    "# Sanity check\n",
    "if not df[df.isna().any(axis=1)].empty:\n",
    "    raise Exception(\"Sanity check failed! Empty rows detected!\")\n",
    "\n",
    "df.head(3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(1041570, 16)"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\"Not necessarily.  He could simply be [sympathetic to the environmentalism movement.]( .com/communities/greenhouse/post/ / /james-cameron-sees-avatar-as-environmental-warning/ )  Edit: Imagine a hypothetical scenario, if you will.    An environmentalist friend of Cameron calls him up for a chat:  Environmentalist: Hey James, you know that movie you've been thinking about doing; the one about the evil corporation who's destroying the planet for greed, and how the planet fights back in the end and wins?  James: Yeah?  Environmentalist: Well, I think that if you went ahead and did that, it could have a great effect on people right now.  We're trying to save the earth, and people just aren't really concerned enough for us to REALLY be able to make a difference.  We have the Copenhagen conference coming up at the end of  , and that will mark the beginning of the big push for real, meaningful change to save the environment.  The problem is that we just aren't quite sure that the American public is on board enough to make American assent to a global climate governing body politically feasible.  James: Yeah, I think you might be right...  It's time to do this.  Thanks bud, I'll ttyl.   Environmentalist: Take care, James.    Look at the movie.  Look at the message.  Look at the political climate.  Look at what's going on re: climate change and carbon trading.  Coincidence?  Probably not.    James might have even recognized the impact that his movie might have *on his own*.  Believe it or not, he's a pretty smart guy, so I hear.  \""
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Extract sample text\n",
    "df.body_clean_full.iloc[21]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2010 15986\n",
      "2011 26432\n",
      "2012 40264\n",
      "2013 63797\n",
      "2014 99626\n",
      "2015 99550\n",
      "2016 99589\n",
      "2017 99544\n",
      "2018 99491\n",
      "2019 99419\n",
      "2020 99431\n",
      "2021 99332\n",
      "2022 99109\n"
     ]
    }
   ],
   "source": [
    "# Get samples per year\n",
    "years = df.created_year.unique()\n",
    "\n",
    "for year in years:\n",
    "    print(year, df[df.created_year == year].shape[0])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## HF Transformers Models\n",
    "\n",
    "Be aware: Models rather large, first time to run might take some downloading time (~500MB per model; saved in sth like \"C:\\Users\\Felix\\.cache\\huggingface\\hub\")\n",
    "\n",
    "All HF sentiment models: https://huggingface.co/models?pipeline_tag=text-classification&sort=downloads&search=sentiment\n",
    "\n",
    "All HF emotion detection models: https://huggingface.co/models?pipeline_tag=text-classification&sort=downloads&search=emotion"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Sentiment Model\n",
    "\n",
    "https://huggingface.co/cardiffnlp/twitter-roberta-base-sentiment\n",
    "\n",
    "ENCODING: 0 -> Negative; 1 -> Neutral; 2 -> Positive\n",
    "\n",
    "GPU: ~0.0105 sec/text"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "model = pipeline(\"sentiment-analysis\", model=\"cardiffnlp/twitter-roberta-base-sentiment\", device=device, batch_size=batch_size)\n",
    "model_name = \"sentiment\""
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Climate stance model\n",
    "\n",
    "https://huggingface.co/cardiffnlp/twitter-roberta-base-stance-climate\n",
    "\n",
    "Paper: https://aclanthology.org/S16-1003.pdf\n",
    "\n",
    "\"Climate Change is a Real Concern\" -> favor/against/none\n",
    "\n",
    "GPU: ~0.014 sec/text"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "model = pipeline(\"sentiment-analysis\", model=\"cardiffnlp/twitter-roberta-base-stance-climate\", device=device, batch_size=batch_size)\n",
    "model_name = \"climate_stance\""
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Climate sentiment model\n",
    "\n",
    "https://huggingface.co/climatebert/distilroberta-base-climate-sentiment \n",
    "\n",
    "-> neutral, opportunity, risk"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "model = pipeline(\"sentiment-analysis\", model=\"climatebert/distilroberta-base-climate-sentiment\", device=device, batch_size=batch_size)\n",
    "model_name = \"climate_sentiment\""
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Large emotion model (28 states detected)\n",
    "\n",
    "Model: https://huggingface.co/arpanghoshal/EmoRoBERTa\n",
    "\n",
    "Labels detected: 'remorse', 'disappointment', 'sadness', 'gratitude', 'realization', 'disapproval', 'neutral', 'approval', 'embarrassment', 'caring', 'curiosity', 'confusion', 'annoyance', 'joy', 'optimism', 'relief', 'excitement', 'admiration', 'love', 'disgust', 'grief', 'amusement', 'anger', 'surprise', 'pride', 'nervousness', 'fear', 'desire'\n",
    "\n",
    "GPU: ~0,11 sec/text"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "All model checkpoint layers were used when initializing TFRobertaForSequenceClassification.\n",
      "\n",
      "All the layers of TFRobertaForSequenceClassification were initialized from the model checkpoint at arpanghoshal/EmoRoBERTa.\n",
      "If your task is similar to the task the model of the checkpoint was trained on, you can already use TFRobertaForSequenceClassification for predictions without further training.\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "(486402, 16)"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model = pipeline('sentiment-analysis', model='arpanghoshal/EmoRoBERTa', top_k=1, device=device, batch_size=batch_size) # top_k=None lists all labels\n",
    "model_name = \"emotion_large\"\n",
    "\n",
    "# Only apply for non outlier topics!\n",
    "df = df[df.topic_number != -1]\n",
    "df.shape"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Small emotion model (7 states)\n",
    "\n",
    "Model: https://huggingface.co/j-hartmann/emotion-english-distilroberta-base\n",
    "\n",
    "Labels detected: 'surprise', 'neutral', 'fear', 'anger', 'joy', 'disgust', 'sadness'\n",
    "\n",
    "GPU: ~0,011 sec/text"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "model = pipeline(\"text-classification\", model=\"j-hartmann/emotion-english-distilroberta-base\", top_k=1, device=device, batch_size=batch_size) # top_k=None lists all labels\n",
    "model_name = \"emotion_small\""
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Test model functionality"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "sample_prediction = model(\"Climate change is a big scam! Why is everyone so upset?!?!\")\n",
    "sample_prediction"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Find labels included\n",
    "labels = [i[\"label\"] for i in sample_prediction]\n",
    "print(labels)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Apply model manually"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create backup of df\n",
    "#df_backup = df.copy()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Reduce text to character limit if there is one\n",
    "def reduce_text(text, max_length=np.inf):\n",
    "    if len(text) > max_length:\n",
    "        text = text[0:max_length]\n",
    "    return text"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Comment text reduced to 512 characters!\n"
     ]
    }
   ],
   "source": [
    "# Reduce texts if necessary\n",
    "df[\"body_clean_full\"] = df[\"body_clean_full\"].apply(reduce_text, args=(512, )) # e.g. for small sentiment model\n",
    "print(\"Comment text reduced to 512 characters!\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Only use subset\n",
    "df.sample(frac=1, random_state=42) # shuffle\n",
    "df = df.iloc[:1000, :]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# CAUTION: THIS WILL RUN INFERENCE AND CAN THUS TAKE SOME TIME\n",
    "#df[\"label\"] = model(list(df[\"body_clean_full\"]))\n",
    "results = df.body_clean_full.iloc[:1000].map(model)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Recommended inference on GPU\n",
    "# https://huggingface.co/docs/transformers/pipeline_tutorial#using-pipelines-on-a-dataset\n",
    "def load_iterator():\n",
    "    for i in df.body_clean_full.iloc[:1000]:\n",
    "        # DEBUGGING\n",
    "        # i = i.replace(\"\\\"\", \" \")\n",
    "        # i = i.replace(\"'\", \" \")\n",
    "        # print(i)\n",
    "        # print(\"\")\n",
    "        yield i\n",
    "\n",
    "results = []\n",
    "\n",
    "for out in model(load_iterator()):\n",
    "    results.append(out)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Extract (top) label\n",
    "results = [i[0][\"label\"] for i in results]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Print all top labels found\n",
    "np.unique(results)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Get sample texts for certain label\n",
    "print(df.iloc[:1000,:][pd.Series(results) == \"LABEL_0\"].body_clean_full.iloc[8])"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### DEBUGGING FOLLOWS"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 86,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Filter data for faulty months and years\n",
    "df = df[(df.created_year == 2015) & (df.created_month == 11) | (df.created_year == 2021) & (df.created_month == 4) | (df.created_year == 2021) & (df.created_month == 10)]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import re\n",
    "df.body_clean_full = df.body_clean_full.apply(lambda x: re.sub(r'[^a-zA-Z0-9\\s\\.,?!;:\\'\"()\\[\\]\\{\\}\\-]', \"\", x))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 89,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\"The only people who don't think that climate change is the biggest threat, are the people who don't understand climate change.  Between ISIS and climate change, only one of those issues can potentially destroy the habitability of the entire planet.\""
      ]
     },
     "execution_count": 89,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.body_clean_full.values[5926]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Reproducing error\n",
    "df[(df.created_year == 2015) & (df.created_month == 11)].body_clean_full.iloc[5926:].map(model)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df.body_clean_full.iloc[:1000][0]"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Run prediction per year"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Only use subset\n",
    "# df = df.sample(frac=1, random_state=42) # shuffle\n",
    "# df = df.iloc[:1000, :]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Comment text reduced to 512 characters!\n"
     ]
    }
   ],
   "source": [
    "# Reduce texts if necessary\n",
    "df[\"body_clean_full\"] = df[\"body_clean_full\"].apply(reduce_text, args=(512, )) # e.g. for small sentiment model\n",
    "print(\"Comment text reduced to 512 characters!\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>subreddit.name</th>\n",
       "      <th>subreddit.nsfw</th>\n",
       "      <th>created_utc</th>\n",
       "      <th>permalink</th>\n",
       "      <th>sentiment</th>\n",
       "      <th>score</th>\n",
       "      <th>created_date</th>\n",
       "      <th>created_day</th>\n",
       "      <th>created_month</th>\n",
       "      <th>created_year</th>\n",
       "      <th>created_time</th>\n",
       "      <th>topic_number</th>\n",
       "      <th>topic_name</th>\n",
       "      <th>topic_most_used_words</th>\n",
       "      <th>body_clean_full</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "Empty DataFrame\n",
       "Columns: [id, subreddit.name, subreddit.nsfw, created_utc, permalink, sentiment, score, created_date, created_day, created_month, created_year, created_time, topic_number, topic_name, topic_most_used_words, body_clean_full]\n",
       "Index: []"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Sanity check\n",
    "df[df.body_clean_full.apply(len) > 512]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df.created_year.unique()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Clean data even further! (Necessary for years 2015 and 2021)\n",
    "import re\n",
    "df.body_clean_full = df.body_clean_full.apply(lambda x: re.sub(r'[^a-zA-Z0-9\\s\\.,?!;:\\'\"()\\[\\]\\{\\}\\-]', \"\", x))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Start logging\n",
    "# https://docs.python.org/3/howto/logging.html\n",
    "folder = f\"data/{model_name}_labels/\"\n",
    "if not os.path.exists(folder):\n",
    "    os.mkdir(folder)\n",
    "\n",
    "logging.basicConfig(filename=f\"data/{model_name}_labels/{model_name}_logs.log\", level=logging.DEBUG)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "def save_file(save_df, idx, to_csv=False):\n",
    "    idx = str(idx)\n",
    "    folder = f\"data/{model_name}_labels/\"\n",
    "    path = folder + f\"{model_name}_{idx}\"\n",
    "\n",
    "    if os.path.exists(path + \".pkl\"): \n",
    "        path = path + \"_\" + str(datetime.now())[-5:]\n",
    "        logging.warning(str(datetime.now()) + f\" Warning: path already existed, will save as: {path}\")\n",
    "    if not os.path.exists(folder):\n",
    "        os.mkdir(folder)\n",
    "\n",
    "    try:\n",
    "        with open(path + \".pkl\", \"wb\") as f:\n",
    "            pickle.dump(save_df, f)\n",
    "        logging.info(str(datetime.now()) + f\" Year {idx} saved in {path}\")\n",
    "    except:\n",
    "        # Save to csv if failed\n",
    "        try:\n",
    "            logging.info(str(datetime.now()) + \" Pickling failed, will try to save as csv\")\n",
    "            save_df.to_csv(path + \"_BACKUP.csv\", index=False)\n",
    "        except:\n",
    "            logging.error(str(datetime.now()) + f\" ERROR: Failed to save year {idx}!\")\n",
    "            logging.info(str(sys.exc_info()))\n",
    "            return False\n",
    "\n",
    "    return True"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Sanity check\n",
    "#save_file(df, 2004)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "max_length = 512\n",
    "\n",
    "def executer(df_sample, year):\n",
    "    try:\n",
    "        logging.info(str(datetime.now()) + f\" Running inference for year {year}, with {df_sample.shape[0]} samples\")\n",
    "\n",
    "        # Run inference as recommended for GPU\n",
    "        # https://huggingface.co/docs/transformers/pipeline_tutorial#using-pipelines-on-a-dataset\n",
    "        def load_iterator():\n",
    "            for i in df_sample.body_clean_full: # FIXME DEBUGGING .iloc[83814:]\n",
    "                yield i[0:max_length]\n",
    "\n",
    "        labels = []\n",
    "        idx = 0\n",
    "        for idx, out in enumerate(model(load_iterator())): # Run inference and collect labels\n",
    "            labels.append(out)\n",
    "\n",
    "        #labels = model(list(df_sample[\"body_clean_full\"])) # OLD WAY TO RUN INFERENCE\n",
    "\n",
    "        if model_name in [\"emotion_small\", \"emotion_large\"]:\n",
    "            labels = [i[0][\"label\"] for i in labels]\n",
    "        else: \n",
    "            labels = [i[\"label\"] for i in labels]\n",
    "\n",
    "        df_save = pd.DataFrame({\"id\": df_sample.id, model_name: labels})\n",
    "        save_file(df_save, year)\n",
    "\n",
    "    except:\n",
    "        logging.error(str(datetime.now()) + f\" ERROR with year {year}: \\n\" + str(sys.exc_info()))\n",
    "        logging.info(str(datetime.now()) + f\" Last idx checked: {idx}, with id: {df_sample.iloc[idx,:].id}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Sanity check\n",
    "#executer(2021)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2023-05-15 19:05:51.975160 Running inference for emotion_large \n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████| 2/2 [3:01:40<00:00, 5450.26s/it]  \n"
     ]
    }
   ],
   "source": [
    "# RUN INFERENCE ON ALL DATA\n",
    "\n",
    "years = df.created_year.unique()\n",
    "years = [2015, 2021]\n",
    "#years[years.sort()]\n",
    "\n",
    "logging.info(f\"Running inference for {model_name} \\n\")\n",
    "print(datetime.now(), f\"Running inference for {model_name} \\n\")\n",
    "\n",
    "for year in tqdm(years):\n",
    "    df_sample = df[df.created_year == year]\n",
    "    \n",
    "    executer(df_sample, year) # optimized way through iterator\n",
    "\n",
    "logging.info(f\"Inference finished for {model_name}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df.body_clean_full[df.created_year == 2015].iloc[83815]"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Re-running problematic years monthwise"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2023-05-15 17:09:14.592877 Running inference for emotion_small \n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████| 1/1 [02:05<00:00, 125.75s/it]\n"
     ]
    }
   ],
   "source": [
    "# RUN INFERENCE ON ALL DATA - PER YEAR AND MONTH\n",
    "\n",
    "years = [2021]#, 2015]\n",
    "months = [4, 10]\n",
    "#months = df.created_month.unique()\n",
    "#months[months.sort()]\n",
    "\n",
    "logging.info(f\"\\nSAFETY RUN PER MONTH AND YEAR \\n\")\n",
    "logging.info(f\"Running inference for {model_name} \\n\")\n",
    "print(datetime.now(), f\"Running inference for {model_name} \\n\")\n",
    "\n",
    "for year in tqdm(years):\n",
    "    for month in months:\n",
    "        df_sample = df[(df.created_year == year) & (df.created_month == month)]\n",
    "    \n",
    "        executer(df_sample, str(year) + \"-\" + str(month)) # optimized way through iterator\n",
    "        #executer2(df_sample, year) # old way\n",
    "\n",
    "logging.info(f\"Inference finished for {model_name}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Checking faulty data months"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "problem_1 = df[(df.created_year == 2015) & (df.created_month == 11)].body_clean_full.values[5927]\n",
    "problem_1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "problem_2 = df[(df.created_year == 2021) & (df.created_month == 4)].body_clean_full.values[1583]\n",
    "problem_2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "problem_3 = df[(df.created_year == 2021) & (df.created_month == 10)].body_clean_full.values[1511]\n",
    "problem_3"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "len(problem_1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# TODO try to remove all special characters of month"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Load results"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Load labels file\n",
    "with open(f\"data/{model_name}_labels/{model_name}_2021.pkl\", \"rb\") as f:\n",
    "    labels_df = pickle.load(f)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "labels_df"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Join labels to full results"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [],
   "source": [
    "model_name = \"emotion_large\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [],
   "source": [
    "path = f\"data/{model_name}_labels/\"\n",
    "\n",
    "files = os.listdir(path)\n",
    "files = [i for i in files if i.endswith(\".pkl\")]\n",
    "\n",
    "first = True\n",
    "for f in files:\n",
    "    with open(path + f, \"rb\") as temp_df:\n",
    "        labels_df = pickle.load(temp_df)\n",
    "\n",
    "    if first:\n",
    "        temp = labels_df\n",
    "        first = False\n",
    "    else:\n",
    "        temp = pd.concat([labels_df, temp])\n",
    "\n",
    "temp = temp.drop_duplicates() # removes duplicates from already inferred data, necessary for emotion_small!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>emotion_large</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1071585</th>\n",
       "      <td>hqqvzdi</td>\n",
       "      <td>realization</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1071586</th>\n",
       "      <td>hqqvzz0</td>\n",
       "      <td>approval</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1071587</th>\n",
       "      <td>hqqwdqz</td>\n",
       "      <td>neutral</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1071588</th>\n",
       "      <td>hqqwq4u</td>\n",
       "      <td>annoyance</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1071591</th>\n",
       "      <td>hqqyf7x</td>\n",
       "      <td>disgust</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15979</th>\n",
       "      <td>c1azzhq</td>\n",
       "      <td>neutral</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15980</th>\n",
       "      <td>c1azzu8</td>\n",
       "      <td>realization</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15981</th>\n",
       "      <td>c1b000n</td>\n",
       "      <td>annoyance</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15982</th>\n",
       "      <td>c1b005v</td>\n",
       "      <td>approval</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15984</th>\n",
       "      <td>c1b02v0</td>\n",
       "      <td>realization</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>486402 rows × 2 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "              id emotion_large\n",
       "1071585  hqqvzdi   realization\n",
       "1071586  hqqvzz0      approval\n",
       "1071587  hqqwdqz       neutral\n",
       "1071588  hqqwq4u     annoyance\n",
       "1071591  hqqyf7x       disgust\n",
       "...          ...           ...\n",
       "15979    c1azzhq       neutral\n",
       "15980    c1azzu8   realization\n",
       "15981    c1b000n     annoyance\n",
       "15982    c1b005v      approval\n",
       "15984    c1b02v0   realization\n",
       "\n",
       "[486402 rows x 2 columns]"
      ]
     },
     "execution_count": 53,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "temp"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>subreddit.name</th>\n",
       "      <th>subreddit.nsfw</th>\n",
       "      <th>created_utc</th>\n",
       "      <th>permalink</th>\n",
       "      <th>sentiment</th>\n",
       "      <th>score</th>\n",
       "      <th>created_date</th>\n",
       "      <th>created_day</th>\n",
       "      <th>created_month</th>\n",
       "      <th>created_year</th>\n",
       "      <th>created_time</th>\n",
       "      <th>topic_number</th>\n",
       "      <th>topic_name</th>\n",
       "      <th>topic_most_used_words</th>\n",
       "      <th>body_clean_full</th>\n",
       "      <th>climate_stance</th>\n",
       "      <th>emotion_small</th>\n",
       "      <th>emotion_large</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>942461</th>\n",
       "      <td>hqqvwdx</td>\n",
       "      <td>whitepeopletwitter</td>\n",
       "      <td>False</td>\n",
       "      <td>1.640996e+09</td>\n",
       "      <td>https://old.reddit.com/r/WhitePeopleTwitter/co...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>2022-01-01</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>2022</td>\n",
       "      <td>00:06:49</td>\n",
       "      <td>-1</td>\n",
       "      <td>-1_people_just_like_climate</td>\n",
       "      <td>people - just - like - climate - don - think -...</td>\n",
       "      <td>-clinton/yes-donald-trump-did-call-climate-ch...</td>\n",
       "      <td>favor</td>\n",
       "      <td>surprise</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "             id      subreddit.name  subreddit.nsfw   created_utc  \\\n",
       "942461  hqqvwdx  whitepeopletwitter           False  1.640996e+09   \n",
       "\n",
       "                                                permalink  sentiment  score  \\\n",
       "942461  https://old.reddit.com/r/WhitePeopleTwitter/co...        0.0    1.0   \n",
       "\n",
       "       created_date  created_day  created_month  created_year created_time  \\\n",
       "942461   2022-01-01            1              1          2022     00:06:49   \n",
       "\n",
       "        topic_number                   topic_name  \\\n",
       "942461            -1  -1_people_just_like_climate   \n",
       "\n",
       "                                    topic_most_used_words  \\\n",
       "942461  people - just - like - climate - don - think -...   \n",
       "\n",
       "                                          body_clean_full climate_stance  \\\n",
       "942461   -clinton/yes-donald-trump-did-call-climate-ch...          favor   \n",
       "\n",
       "       emotion_small emotion_large  \n",
       "942461      surprise           NaN  "
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Sanity check\n",
    "df[df.id == \"hqqvwdx\"]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Size of full dataframe:   (1041570, 18)\n",
      "Size of labels dataframe: (486402, 2)\n"
     ]
    }
   ],
   "source": [
    "# Join by index with comments.csv\n",
    "old_shape = df.shape\n",
    "print(\"Size of full dataframe:  \", old_shape)\n",
    "print(\"Size of labels dataframe:\", temp.shape)\n",
    "\n",
    "# Join dataframes\n",
    "df = pd.merge(df, temp, how=\"left\", on=\"id\")\n",
    "\n",
    "# Verify join\n",
    "if old_shape[0] != df.shape[0]:\n",
    "    raise Exception(f\"ERROR: lengths mismatch! Error when joining! \\n {old_shape[0]} -> {df.shape[0]}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>subreddit.name</th>\n",
       "      <th>subreddit.nsfw</th>\n",
       "      <th>created_utc</th>\n",
       "      <th>permalink</th>\n",
       "      <th>sentiment</th>\n",
       "      <th>score</th>\n",
       "      <th>created_date</th>\n",
       "      <th>created_day</th>\n",
       "      <th>created_month</th>\n",
       "      <th>created_year</th>\n",
       "      <th>created_time</th>\n",
       "      <th>topic_number</th>\n",
       "      <th>topic_name</th>\n",
       "      <th>topic_most_used_words</th>\n",
       "      <th>body_clean_full</th>\n",
       "      <th>climate_stance</th>\n",
       "      <th>emotion_small</th>\n",
       "      <th>emotion_large</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>c0i14fb</td>\n",
       "      <td>askreddit</td>\n",
       "      <td>False</td>\n",
       "      <td>1.262306e+09</td>\n",
       "      <td>https://old.reddit.com/r/AskReddit/comments/ak...</td>\n",
       "      <td>0.7998</td>\n",
       "      <td>1.0</td>\n",
       "      <td>2010-01-01</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>2010</td>\n",
       "      <td>00:34:07</td>\n",
       "      <td>-1</td>\n",
       "      <td>-1_climate_people_global_warming</td>\n",
       "      <td>climate - people - global - warming - just - s...</td>\n",
       "      <td>should be \"San Diego Weatherman has an opinion...</td>\n",
       "      <td>favor</td>\n",
       "      <td>surprise</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>c0i195b</td>\n",
       "      <td>worldnews</td>\n",
       "      <td>False</td>\n",
       "      <td>1.262313e+09</td>\n",
       "      <td>https://old.reddit.com/r/worldnews/comments/ak...</td>\n",
       "      <td>0.4754</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2010-01-01</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>2010</td>\n",
       "      <td>02:30:18</td>\n",
       "      <td>0</td>\n",
       "      <td>0_people_just_climate_global</td>\n",
       "      <td>people - just - climate - global - don - like ...</td>\n",
       "      <td>Both Iggy and Harper would have marched us int...</td>\n",
       "      <td>favor</td>\n",
       "      <td>fear</td>\n",
       "      <td>neutral</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>c0i1a0w</td>\n",
       "      <td>environment</td>\n",
       "      <td>False</td>\n",
       "      <td>1.262314e+09</td>\n",
       "      <td>https://old.reddit.com/r/environment/comments/...</td>\n",
       "      <td>0.0242</td>\n",
       "      <td>1.0</td>\n",
       "      <td>2010-01-01</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>2010</td>\n",
       "      <td>02:54:40</td>\n",
       "      <td>0</td>\n",
       "      <td>0_people_just_climate_global</td>\n",
       "      <td>people - just - climate - global - don - like ...</td>\n",
       "      <td>A man who though a moderate Tory , has a mixed...</td>\n",
       "      <td>favor</td>\n",
       "      <td>surprise</td>\n",
       "      <td>approval</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>c0i1hsb</td>\n",
       "      <td>askreddit</td>\n",
       "      <td>False</td>\n",
       "      <td>1.262330e+09</td>\n",
       "      <td>https://old.reddit.com/r/AskReddit/comments/ak...</td>\n",
       "      <td>0.7579</td>\n",
       "      <td>3.0</td>\n",
       "      <td>2010-01-01</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>2010</td>\n",
       "      <td>07:05:41</td>\n",
       "      <td>-1</td>\n",
       "      <td>-1_climate_people_global_warming</td>\n",
       "      <td>climate - people - global - warming - just - s...</td>\n",
       "      <td>Changing the oil *filter* every single time yo...</td>\n",
       "      <td>favor</td>\n",
       "      <td>neutral</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>c0i1pd9</td>\n",
       "      <td>politics</td>\n",
       "      <td>False</td>\n",
       "      <td>1.262349e+09</td>\n",
       "      <td>https://old.reddit.com/r/politics/comments/akc...</td>\n",
       "      <td>-0.9849</td>\n",
       "      <td>32.0</td>\n",
       "      <td>2010-01-01</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>2010</td>\n",
       "      <td>12:37:36</td>\n",
       "      <td>-1</td>\n",
       "      <td>-1_climate_people_global_warming</td>\n",
       "      <td>climate - people - global - warming - just - s...</td>\n",
       "      <td>; We have no history - ours goes back only   y...</td>\n",
       "      <td>none</td>\n",
       "      <td>disgust</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1041565</th>\n",
       "      <td>imlbfv6</td>\n",
       "      <td>terrifyingasfuck</td>\n",
       "      <td>False</td>\n",
       "      <td>1.661990e+09</td>\n",
       "      <td>https://old.reddit.com/r/TerrifyingAsFuck/comm...</td>\n",
       "      <td>0.3182</td>\n",
       "      <td>-6.0</td>\n",
       "      <td>2022-08-31</td>\n",
       "      <td>31</td>\n",
       "      <td>8</td>\n",
       "      <td>2022</td>\n",
       "      <td>23:45:08</td>\n",
       "      <td>0</td>\n",
       "      <td>0_people_like_just_don</td>\n",
       "      <td>people - like - just - don - think - world - w...</td>\n",
       "      <td>I'm sure it's climate change.  Probably has no...</td>\n",
       "      <td>favor</td>\n",
       "      <td>neutral</td>\n",
       "      <td>approval</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1041566</th>\n",
       "      <td>imlbh9l</td>\n",
       "      <td>damnthatsinteresting</td>\n",
       "      <td>False</td>\n",
       "      <td>1.661990e+09</td>\n",
       "      <td>https://old.reddit.com/r/Damnthatsinteresting/...</td>\n",
       "      <td>0.4404</td>\n",
       "      <td>5.0</td>\n",
       "      <td>2022-08-31</td>\n",
       "      <td>31</td>\n",
       "      <td>8</td>\n",
       "      <td>2022</td>\n",
       "      <td>23:45:25</td>\n",
       "      <td>-1</td>\n",
       "      <td>-1_people_just_like_climate</td>\n",
       "      <td>people - just - like - climate - don - think -...</td>\n",
       "      <td>You should check out Paul Nicklen's (the guy i...</td>\n",
       "      <td>favor</td>\n",
       "      <td>neutral</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1041567</th>\n",
       "      <td>imlcpab</td>\n",
       "      <td>askreddit</td>\n",
       "      <td>False</td>\n",
       "      <td>1.661990e+09</td>\n",
       "      <td>https://old.reddit.com/r/AskReddit/comments/x2...</td>\n",
       "      <td>0.4690</td>\n",
       "      <td>2.0</td>\n",
       "      <td>2022-08-31</td>\n",
       "      <td>31</td>\n",
       "      <td>8</td>\n",
       "      <td>2022</td>\n",
       "      <td>23:54:25</td>\n",
       "      <td>-1</td>\n",
       "      <td>-1_people_just_like_climate</td>\n",
       "      <td>people - just - like - climate - don - think -...</td>\n",
       "      <td>They need to change laws so it's more worth se...</td>\n",
       "      <td>favor</td>\n",
       "      <td>neutral</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1041568</th>\n",
       "      <td>imlctc0</td>\n",
       "      <td>pastors</td>\n",
       "      <td>False</td>\n",
       "      <td>1.661990e+09</td>\n",
       "      <td>https://old.reddit.com/r/pastors/comments/x2il...</td>\n",
       "      <td>0.9779</td>\n",
       "      <td>2.0</td>\n",
       "      <td>2022-08-31</td>\n",
       "      <td>31</td>\n",
       "      <td>8</td>\n",
       "      <td>2022</td>\n",
       "      <td>23:55:14</td>\n",
       "      <td>4</td>\n",
       "      <td>4_trans_gender_women_men</td>\n",
       "      <td>trans - gender - women - men - gay - sex - peo...</td>\n",
       "      <td>Can i suggest maybe honing in on LGBTQ?  It's ...</td>\n",
       "      <td>none</td>\n",
       "      <td>neutral</td>\n",
       "      <td>curiosity</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1041569</th>\n",
       "      <td>imldado</td>\n",
       "      <td>newzealand</td>\n",
       "      <td>False</td>\n",
       "      <td>1.661990e+09</td>\n",
       "      <td>https://old.reddit.com/r/newzealand/comments/x...</td>\n",
       "      <td>-0.1143</td>\n",
       "      <td>1.0</td>\n",
       "      <td>2022-08-31</td>\n",
       "      <td>31</td>\n",
       "      <td>8</td>\n",
       "      <td>2022</td>\n",
       "      <td>23:58:47</td>\n",
       "      <td>-1</td>\n",
       "      <td>-1_people_just_like_climate</td>\n",
       "      <td>people - just - like - climate - don - think -...</td>\n",
       "      <td>I'm honestly waiting for climate change and th...</td>\n",
       "      <td>favor</td>\n",
       "      <td>surprise</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>1041570 rows × 19 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "              id        subreddit.name  subreddit.nsfw   created_utc  \\\n",
       "0        c0i14fb             askreddit           False  1.262306e+09   \n",
       "1        c0i195b             worldnews           False  1.262313e+09   \n",
       "2        c0i1a0w           environment           False  1.262314e+09   \n",
       "3        c0i1hsb             askreddit           False  1.262330e+09   \n",
       "4        c0i1pd9              politics           False  1.262349e+09   \n",
       "...          ...                   ...             ...           ...   \n",
       "1041565  imlbfv6      terrifyingasfuck           False  1.661990e+09   \n",
       "1041566  imlbh9l  damnthatsinteresting           False  1.661990e+09   \n",
       "1041567  imlcpab             askreddit           False  1.661990e+09   \n",
       "1041568  imlctc0               pastors           False  1.661990e+09   \n",
       "1041569  imldado            newzealand           False  1.661990e+09   \n",
       "\n",
       "                                                 permalink  sentiment  score  \\\n",
       "0        https://old.reddit.com/r/AskReddit/comments/ak...     0.7998    1.0   \n",
       "1        https://old.reddit.com/r/worldnews/comments/ak...     0.4754    0.0   \n",
       "2        https://old.reddit.com/r/environment/comments/...     0.0242    1.0   \n",
       "3        https://old.reddit.com/r/AskReddit/comments/ak...     0.7579    3.0   \n",
       "4        https://old.reddit.com/r/politics/comments/akc...    -0.9849   32.0   \n",
       "...                                                    ...        ...    ...   \n",
       "1041565  https://old.reddit.com/r/TerrifyingAsFuck/comm...     0.3182   -6.0   \n",
       "1041566  https://old.reddit.com/r/Damnthatsinteresting/...     0.4404    5.0   \n",
       "1041567  https://old.reddit.com/r/AskReddit/comments/x2...     0.4690    2.0   \n",
       "1041568  https://old.reddit.com/r/pastors/comments/x2il...     0.9779    2.0   \n",
       "1041569  https://old.reddit.com/r/newzealand/comments/x...    -0.1143    1.0   \n",
       "\n",
       "        created_date  created_day  created_month  created_year created_time  \\\n",
       "0         2010-01-01            1              1          2010     00:34:07   \n",
       "1         2010-01-01            1              1          2010     02:30:18   \n",
       "2         2010-01-01            1              1          2010     02:54:40   \n",
       "3         2010-01-01            1              1          2010     07:05:41   \n",
       "4         2010-01-01            1              1          2010     12:37:36   \n",
       "...              ...          ...            ...           ...          ...   \n",
       "1041565   2022-08-31           31              8          2022     23:45:08   \n",
       "1041566   2022-08-31           31              8          2022     23:45:25   \n",
       "1041567   2022-08-31           31              8          2022     23:54:25   \n",
       "1041568   2022-08-31           31              8          2022     23:55:14   \n",
       "1041569   2022-08-31           31              8          2022     23:58:47   \n",
       "\n",
       "         topic_number                        topic_name  \\\n",
       "0                  -1  -1_climate_people_global_warming   \n",
       "1                   0      0_people_just_climate_global   \n",
       "2                   0      0_people_just_climate_global   \n",
       "3                  -1  -1_climate_people_global_warming   \n",
       "4                  -1  -1_climate_people_global_warming   \n",
       "...               ...                               ...   \n",
       "1041565             0            0_people_like_just_don   \n",
       "1041566            -1       -1_people_just_like_climate   \n",
       "1041567            -1       -1_people_just_like_climate   \n",
       "1041568             4          4_trans_gender_women_men   \n",
       "1041569            -1       -1_people_just_like_climate   \n",
       "\n",
       "                                     topic_most_used_words  \\\n",
       "0        climate - people - global - warming - just - s...   \n",
       "1        people - just - climate - global - don - like ...   \n",
       "2        people - just - climate - global - don - like ...   \n",
       "3        climate - people - global - warming - just - s...   \n",
       "4        climate - people - global - warming - just - s...   \n",
       "...                                                    ...   \n",
       "1041565  people - like - just - don - think - world - w...   \n",
       "1041566  people - just - like - climate - don - think -...   \n",
       "1041567  people - just - like - climate - don - think -...   \n",
       "1041568  trans - gender - women - men - gay - sex - peo...   \n",
       "1041569  people - just - like - climate - don - think -...   \n",
       "\n",
       "                                           body_clean_full climate_stance  \\\n",
       "0        should be \"San Diego Weatherman has an opinion...          favor   \n",
       "1        Both Iggy and Harper would have marched us int...          favor   \n",
       "2        A man who though a moderate Tory , has a mixed...          favor   \n",
       "3        Changing the oil *filter* every single time yo...          favor   \n",
       "4        ; We have no history - ours goes back only   y...           none   \n",
       "...                                                    ...            ...   \n",
       "1041565  I'm sure it's climate change.  Probably has no...          favor   \n",
       "1041566  You should check out Paul Nicklen's (the guy i...          favor   \n",
       "1041567  They need to change laws so it's more worth se...          favor   \n",
       "1041568  Can i suggest maybe honing in on LGBTQ?  It's ...           none   \n",
       "1041569  I'm honestly waiting for climate change and th...          favor   \n",
       "\n",
       "        emotion_small emotion_large  \n",
       "0            surprise           NaN  \n",
       "1                fear       neutral  \n",
       "2            surprise      approval  \n",
       "3             neutral           NaN  \n",
       "4             disgust           NaN  \n",
       "...               ...           ...  \n",
       "1041565       neutral      approval  \n",
       "1041566       neutral           NaN  \n",
       "1041567       neutral           NaN  \n",
       "1041568       neutral     curiosity  \n",
       "1041569      surprise           NaN  \n",
       "\n",
       "[1041570 rows x 19 columns]"
      ]
     },
     "execution_count": 56,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Save new file\n",
    "save_path = \"data/comments_final_labels.csv\"\n",
    "\n",
    "if not os.path.exists(save_path):\n",
    "    df.to_csv(save_path)\n",
    "else:\n",
    "    print(f\"WARNING: File not saved as {path} already exists!\")"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "base",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.7"
  },
  "orig_nbformat": 4
 },
 "nbformat": 4,
 "nbformat_minor": 2
}