10-Classifiers-v1.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "source": [
    "Creative Commons CC BY 4.0 Lynd Bacon & Associates, Ltd. Not warranted to be suitable for any particular purpose. (You're on your own!)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Classifiers\n",
    "\n",
    "These are algorithms that learn to predict the \"labels\" (values) on categorical \"targets' (dependent variables). \n",
    "\n",
    "Frequently used methods include:\n",
    "\n",
    "* logistic regression (binomial, multinominal)\n",
    "* support vector machine (SVM)\n",
    "* Ridge, Lasso, elasticNet\n",
    "* CART (also for regression)\n",
    "* AdaBoost (a \"boosted\" ensemble classifier, also for regression)\n",
    "* RandomForest (an ensemble method that can also do regression and survival models)\n",
    "* Neural networks of various sorts"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Cost Functions\n",
    "\n",
    "When the target is continuously valued, the _cost function_ that an algorithm minimizes frequently involves a function of squared prediction errors.\n",
    "\n",
    "The cost function minimized by ML classifiers varies.  Some, like logistic regression algorithms, may  use _cross-entropy loss_.   Tree-based methods like RandomForest may use  _Gini impurity_ to measure the node \"purity\" achieved by recursive partitioning.\n",
    "\n",
    "See [Classification Loss Functions on Wikipedia](https://en.wikipedia.org/wiki/Loss_functions_for_classification) for a useful summary."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Binary Classifier: Cross Entropy\n",
    "\n",
    "In the case of a binary classifier, cross entropy H(p,q) can be defined as:\n",
    "\n",
    "\\begin{align}\n",
    "\\large\n",
    "H(p,q) = - \\Sigma p(x)log(q(x))\n",
    "\\end{align}\n",
    "\n"
   ]
  }
 ],
 "metadata": {
  "celltoolbar": "Slideshow",
  "hide_input": false,
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.8"
  },
  "latex_envs": {
   "LaTeX_envs_menu_present": true,
   "autoclose": false,
   "autocomplete": true,
   "bibliofile": "biblio.bib",
   "cite_by": "apalike",
   "current_citInitial": 1,
   "eqLabelWithNumbers": true,
   "eqNumInitial": 1,
   "hotkeys": {
    "equation": "Ctrl-E",
    "itemize": "Ctrl-I"
   },
   "labels_anchors": false,
   "latex_user_defs": false,
   "report_style_numbering": false,
   "user_envs_cfg": false
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": false,
   "sideBar": false,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": false,
   "toc_window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}