Lecture04_sugar.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<style>\n",
       "\n",
       ".CodeMirror {\n",
       "    width: 100vw;\n",
       "}\n",
       "S\n",
       ".container {\n",
       "    width: 99% !important;\n",
       "}\n",
       "\n",
       ".rendered_html {\n",
       "  font-size:0.8em;\n",
       "}\n",
       ".rendered_html table, .rendered_html th, .rendered_html tr, .rendered_html td {\n",
       "     font-size: 100%;\n",
       "}\n",
       "\n",
       "</style>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "%%html\n",
    "<style>\n",
    "\n",
    ".CodeMirror {\n",
    "    width: 100vw;\n",
    "}\n",
    "S\n",
    ".container {\n",
    "    width: 99% !important;\n",
    "}\n",
    "\n",
    ".rendered_html {\n",
    "  font-size:0.8em;\n",
    "}\n",
    ".rendered_html table, .rendered_html th, .rendered_html tr, .rendered_html td {\n",
    "     font-size: 100%;\n",
    "}\n",
    "\n",
    "</style>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [],
   "source": [
    "%%capture\n",
    "%run \"NAND programming language.ipynb\"\n",
    "# from IPython.display import clear_output\n",
    "# clear_output()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Lecture 4: Computing Every function"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "from IPython.core.display import HTML\n",
    "HTML('Laptops etc last 5 rows<br><iframe src=\"http://free.timeanddate.com/countdown/i5vf6j5p/n43/cf11/cm0/cu4/ct1/cs1/ca0/co0/cr0/ss0/cac09f/cpc09f/pct/tcfff/fs100/szw576/szh243/iso2018-09-13T10:30:00\" allowTransparency=\"true\" frameborder=\"0\" width=\"177\" height=\"35\"></iframe>')\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "# Python format strings\n",
    "x= 3\n",
    "print( f\" x = {x} , x^2 = {x*x} \"  )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Syntactic Sugar\n",
    "\n",
    "Define a language _NAND'_ which is NAND + extra commands. Define notion of \"computing a function\" in NAND'\n",
    "\n",
    "__Example:__ NAND' = NAND + function definitions.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "__Theorem:__ For every NAND' program $P'$, there exists a NAND program $P$ s.t. $P$ computes same function as $P'$."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "__Theorem:__ _For every_ NAND' program $P'$ with $n$ inputs, there _exists_ a NAND program $P$ s.t. _for every_ $x\\in \\{0,1\\}^n$, $P(x)=P'(x)$."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "__Proof:__ \n",
    "\n",
    "1. Show a _transformation_ of $P'$ into $P$.\n",
    "\n",
    "2. Prove that transformation works."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [],
   "source": [
    "mystery = IF"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [],
   "source": [
    "print(nandcode(mystery))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "What function does `mystery` compute? [pollev.com/cs121](http://pollev.com/cs121)  \n",
    "__a.__  $MAJ:\\{0,1\\}^3 \\rightarrow \\{0,1\\}$   \n",
    "__b.__  $MAJ:\\{0,1\\}^5 \\rightarrow \\{0,1\\}$   \n",
    "__c.__  $f(a,b,c) = a\\cdot b + (1-a)\\cdot c$   \n",
    "__d.__  $g(a,b,c) = a + b + c \\mod 2$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "%%html\n",
    "<iframe src=\"https://e.ggtimer.com/120?alert=false\" frameborder=\"0\" allowfullscreen  width=\" 80%\" height=\"100\" style=\"box-shadow: 0 0 16px gray; margin: 10px;\" name=\"test\" id=\"test\"></iframe>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "for a in [0,1]:\n",
    "    for b in [0,1]:\n",
    "        for c in [0,1]: print(f\"{a} {b} {c} | {mystery(a,b,c)}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "__EXERCISE:__ Use `IF` and other sugar to write NAND program to compute \n",
    "\n",
    "\n",
    " x   | f(x)\n",
    "-----|--------\n",
    "000  | 1\n",
    "001  | 1\n",
    "010  | 0\n",
    "011  | 0\n",
    "100  | 1\n",
    "101  | 0\n",
    "110  | 0\n",
    "111  | 1\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "%%html\n",
    "<iframe src=\"https://e.ggtimer.com/120?alert=false\" frameborder=\"0\" allowfullscreen  width=\" 80%\" height=\"100\" style=\"box-shadow: 0 0 16px gray; margin: 10px;\" name=\"test\" id=\"test\"></iframe>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [],
   "source": [
    "def f(a,b,c):\n",
    "    return IF(a,\n",
    "             IF(b,\n",
    "               IF(c,1,0),\n",
    "               IF(c,0,1)),\n",
    "             IF(b,\n",
    "               IF(c,0,0),\n",
    "               IF(c,1,1))\n",
    "             )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "for a in [0,1]:\n",
    "    for b in [0,1]:\n",
    "        for c in [0,1]: print(f\"{a} {b} {c} | {f(a,b,c)}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0 0 0 | 1\n",
      "0 0 1 | 1\n",
      "0 1 0 | 0\n",
      "0 1 1 | 0\n",
      "1 0 0 | 1\n",
      "1 0 1 | 0\n",
      "1 1 0 | 0\n",
      "1 1 1 | 1\n"
     ]
    }
   ],
   "source": [
    "def f(a,b,c):\n",
    "    temp = NAND(a,a)\n",
    "    one  = NAND(a,temp)\n",
    "    zero = NAND(one,one)\n",
    "    return IF(a,\n",
    "             IF(b,\n",
    "               IF(c,one,zero),\n",
    "               IF(c,zero,one)),\n",
    "             IF(b,\n",
    "               IF(c,zero,zero),\n",
    "               IF(c,one,one))\n",
    "             )\n",
    "\n",
    "for a in [0,1]:\n",
    "    for b in [0,1]:\n",
    "        for c in [0,1]: print(f\"{a} {b} {c} | {f(a,b,c)}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [],
   "source": [
    "nandcircuit(f)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Temp[0] = NAND(X[0],X[0])\n",
      "Temp[1] = NAND(X[0],Temp[0])\n",
      "Temp[2] = NAND(Temp[1],Temp[1])\n",
      "Temp[3] = NAND(X[2],X[2])\n",
      "Temp[4] = NAND(Temp[2],Temp[3])\n",
      "Temp[5] = NAND(Temp[1],X[2])\n",
      "Temp[6] = NAND(Temp[4],Temp[5])\n",
      "Temp[7] = NAND(X[2],X[2])\n",
      "Temp[8] = NAND(Temp[1],Temp[7])\n",
      "Temp[9] = NAND(Temp[2],X[2])\n",
      "Temp[10] = NAND(Temp[8],Temp[9])\n",
      "Temp[11] = NAND(X[1],X[1])\n",
      "Temp[12] = NAND(Temp[10],Temp[11])\n",
      "Temp[13] = NAND(Temp[6],X[1])\n",
      "Temp[14] = NAND(Temp[12],Temp[13])\n",
      "Temp[15] = NAND(X[2],X[2])\n",
      "Temp[16] = NAND(Temp[2],Temp[15])\n",
      "Temp[17] = NAND(Temp[2],X[2])\n",
      "Temp[18] = NAND(Temp[16],Temp[17])\n",
      "Temp[19] = NAND(X[2],X[2])\n",
      "Temp[20] = NAND(Temp[1],Temp[19])\n",
      "Temp[21] = NAND(Temp[1],X[2])\n",
      "Temp[22] = NAND(Temp[20],Temp[21])\n",
      "Temp[23] = NAND(X[1],X[1])\n",
      "Temp[24] = NAND(Temp[22],Temp[23])\n",
      "Temp[25] = NAND(Temp[18],X[1])\n",
      "Temp[26] = NAND(Temp[24],Temp[25])\n",
      "Temp[27] = NAND(X[0],X[0])\n",
      "Temp[28] = NAND(Temp[26],Temp[27])\n",
      "Temp[29] = NAND(Temp[14],X[0])\n",
      "Y[0] = NAND(Temp[28],Temp[29])\n",
      "\n"
     ]
    }
   ],
   "source": [
    "code= nandcode(f)\n",
    "print(code)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "code.count('\\n')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [],
   "source": [
    "def EVAL(prog,n,m,x):\n",
    "    \"\"\"Evaluate NAND program prog with n inputs and m outputs on input x.\"\"\"\n",
    "    vartable = {} # dictionary for variables\n",
    "    \n",
    "    for i in range(n): vartable[f'X[{i}]']=x[i] # assign x[i] to variable \"X[i]\"\n",
    "    \n",
    "    for line in prog.split('\\n'): # split code into lines\n",
    "        if not(line): continue  # ignore empty lines\n",
    "        a = line.find('=')  \n",
    "        b = line.find('(')\n",
    "        c=line.find(',')\n",
    "        d= line.find(')')\n",
    "        \n",
    "        foo = line[:a].strip()\n",
    "        bar = line[b+1:c].strip()\n",
    "        blah = line[c+1:d].strip()\n",
    "\n",
    "        vartable[foo] =  NAND(vartable[bar],vartable[blah])\n",
    "    \n",
    "    return [vartable[f'Y[{j}]'] for j in range(m)]\n",
    "# Explain this code to each other in groups of 2-4"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "%%html\n",
    "<iframe src=\"https://e.ggtimer.com/240?alert=false\" frameborder=\"0\" allowfullscreen  width=\" 80%\" height=\"100\" style=\"box-shadow: 0 0 16px gray; margin: 10px;\" name=\"test\" id=\"test\"></iframe>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0 0 0 | [1]\n",
      "0 0 1 | [1]\n",
      "0 1 0 | [0]\n",
      "0 1 1 | [0]\n",
      "1 0 0 | [1]\n",
      "1 0 1 | [0]\n",
      "1 1 0 | [0]\n",
      "1 1 1 | [1]\n"
     ]
    }
   ],
   "source": [
    "for a in [0,1]:\n",
    "    for b in [0,1]:\n",
    "        for c in [0,1]: print(f\"{a} {b} {c} | {EVAL(code,3,1,[a,b,c])}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "__Theorem:__ For every $f:\\{0,1\\}^n \\rightarrow \\{0,1\\}$, there is a NAND program of $O(2^n)$ lines that computes $f$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [],
   "source": [
    "def XORn(X): \n",
    "    r = X[0]\n",
    "    for i in range(1,len(X)):\n",
    "        r = XOR(r,X[i])\n",
    "    return r\n",
    "\n",
    "nandcircuit(restrict(XORn,3))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "That is, $\\{ f | f:\\{0,1\\}^n \\rightarrow \\{0,1\\} \\} \\subseteq SIZE(c\\cdot 2^n)$  (powerpoint)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "__Theorem:__ For every $f:\\{0,1\\}^n \\rightarrow \\{0,1\\}$, there is a NAND program of $O(2^n)$ lines that computes $f$."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "__Proof:__ Procedure that takes table of $2^n$ values $f(0^n)$,$f(0^{n-1}1)$,$f(0^{n-2}10)$,$\\ldots$,$f(1^n)$ and converts it to NAND program that computes $f$."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "1. NAND program for $LOOKUP_n:\\{0,1\\}^{2^n+n} \\rightarrow \\{0,1\\}$ where $LOOKUP(T,i)=T_i$ for $i\\in [2^n]$\n",
    "\n",
    "2. \"Hardwire\" the outputs of $f$ inside the table $T$ to obtain a program with $n$ inputs and one output."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0"
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "def LOOKUP(T,I): # T length 2^n, I length n, return T[I]\n",
    "    l = len(I)\n",
    "    if l==1: return IF(I[0],T[1],T[0])\n",
    "    \n",
    "    return IF(I[l-1], # look at most significant bit of I\n",
    "              LOOKUP(T[2**(l-1):],I[:-1]), # recurse on second half of T\n",
    "              LOOKUP(T[:2**(l-1)],I[:-1])) # recurse on first  half of T\n",
    "\n",
    "LOOKUP([0,0,1,1,0,1,1,0],[1,1,1])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "def LOOKUPCODE(n): # return NAND code of LOOKUP - can ignore\n",
    "    def f(X):\n",
    "        return LOOKUP( X[:2**n] ,  X[2**n:2**n+n])\n",
    "    return nandcode(f,numargs=2**n+n)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [],
   "source": [
    "import itertools\n",
    "\n",
    "def int2list(i, n = 0):\n",
    "    \"\"\"Utility function: transform number to list of 0/1 values in binary rep, LSB first\"\"\"\n",
    "    L = [int(c) for c in bin(i)[2:]][::-1]\n",
    "    return L + [0]*max(0,n-len(L))\n",
    "\n",
    "int2list(13,6)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [],
   "source": [
    "def NANDPROG(f,n):\n",
    "    # takes f:{0,1}^n --> {0,1}, returns NAND code to compute f\n",
    "    \n",
    "    lookupcode = LOOKUPCODE(n) # NAND code for lookup function on 2^n + n inputs\n",
    "\n",
    "    for i in range(2**n):\n",
    "        x = int2list(i,n) # binary representation of i, padded to length n\n",
    "        lookupcode = lookupcode.replace(f'X[{i}]',('one' if f(*x) else 'zero'))\n",
    "    \n",
    "    for i in range(2**n,2**n+n):\n",
    "        lookupcode = lookupcode.replace(f'X[{i}]',f'X[{i-2**n}]')\n",
    "    \n",
    "    preamble = '''temp = NAND(X[0],X[0])\n",
    "one  = NAND(temp,X[0])\n",
    "zero = NAND(one,one)\n",
    "'''\n",
    "    return preamble+lookupcode\n",
    "\n",
    "# Explain to code to one another in groups of 2-4"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {
    "scrolled": true,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<iframe src=\"https://e.ggtimer.com/300?alert=false\" frameborder=\"0\" allowfullscreen  width=\" 80%\" height=\"100\" style=\"box-shadow: 0 0 16px gray; margin: 10px;\" name=\"test\" id=\"test\"></iframe>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "%%html\n",
    "<iframe src=\"https://e.ggtimer.com/300?alert=false\" frameborder=\"0\" allowfullscreen  width=\" 80%\" height=\"100\" style=\"box-shadow: 0 0 16px gray; margin: 10px;\" name=\"test\" id=\"test\"></iframe>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [],
   "source": [
    "def f(a,b,c): \n",
    "    if a+b+c > 1: return 1\n",
    "    return 0"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "temp = NAND(X[0],X[0])\n",
      "one  = NAND(temp,X[0])\n",
      "zero = NAND(one,one)\n",
      "Temp[0] = NAND(X[0],X[0])\n",
      "Temp[1] = NAND(one,Temp[0])\n",
      "Temp[2] = NAND(one,X[0])\n",
      "Temp[3] = NAND(Temp[1],Temp[2])\n",
      "Temp[4] = NAND(X[0],X[0])\n",
      "Temp[5] = NAND(zero,Temp[4])\n",
      "Temp[6] = NAND(one,X[0])\n",
      "Temp[7] = NAND(Temp[5],Temp[6])\n",
      "Temp[8] = NAND(X[1],X[1])\n",
      "Temp[9] = NAND(Temp[7],Temp[8])\n",
      "Temp[10] = NAND(Temp[3],X[1])\n",
      "Temp[11] = NAND(Temp[9],Temp[10])\n",
      "Temp[12] = NAND(X[0],X[0])\n",
      "Temp[13] = NAND(zero,Temp[12])\n",
      "Temp[14] = NAND(one,X[0])\n",
      "Temp[15] = NAND(Temp[13],Temp[14])\n",
      "Temp[16] = NAND(X[0],X[0])\n",
      "Temp[17] = NAND(zero,Temp[16])\n",
      "Temp[18] = NAND(zero,X[0])\n",
      "Temp[19] = NAND(Temp[17],Temp[18])\n",
      "Temp[20] = NAND(X[1],X[1])\n",
      "Temp[21] = NAND(Temp[19],Temp[20])\n",
      "Temp[22] = NAND(Temp[15],X[1])\n",
      "Temp[23] = NAND(Temp[21],Temp[22])\n",
      "Temp[24] = NAND(X[2],X[2])\n",
      "Temp[25] = NAND(Temp[23],Temp[24])\n",
      "Temp[26] = NAND(Temp[11],X[2])\n",
      "Y[0] = NAND(Temp[25],Temp[26])\n",
      "\n"
     ]
    }
   ],
   "source": [
    "code = NANDPROG(f,3)\n",
    "print(code)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0 0 0 | [0]\n",
      "0 0 1 | [0]\n",
      "0 1 0 | [0]\n",
      "0 1 1 | [1]\n",
      "1 0 0 | [0]\n",
      "1 0 1 | [1]\n",
      "1 1 0 | [1]\n",
      "1 1 1 | [1]\n"
     ]
    }
   ],
   "source": [
    "for a in [0,1]:\n",
    "    for b in [0,1]:\n",
    "        for c in [0,1]: print(f\"{a} {b} {c} | {EVAL(code,3,1,[a,b,c])}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[1]"
      ]
     },
     "execution_count": 48,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "def DIVTHREE(L): return 0 if sum(L) % 3 else 1\n",
    "def DIVTHREE5(a,b,c,d,e): return DIVTHREE( [a,b,c,d,e] )\n",
    "\n",
    "code = NANDPROG(DIVTHREE5,5)\n",
    "EVAL(code,5,1,[1,0,0,1,1])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[(2, 15), (3, 31), (4, 63), (5, 127), (6, 255), (7, 511), (8, 1023), (9, 2047)]"
      ]
     },
     "execution_count": 49,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "def code(n): return  NANDPROG(restrict(DIVTHREE,n),n)\n",
    "[(n,code(n).count('\\n')) for n in range(2,10)]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[3.75, 3.875, 3.9375, 3.96875, 3.984375, 3.9921875, 3.99609375, 3.998046875]"
      ]
     },
     "execution_count": 50,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "[code(n).count('\\n')/(2**n) for n in range(2,10)]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Is this the best we can do?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "__Theorem:__ $DIVTHREE_n \\in SIZE(O(n))$.   \n",
    "That is, exists $c$ such that $DIVTHREE_n \\in SIZE(c\\cdot n)$ for every sufficiently large $n$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {
    "scrolled": true,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<iframe src=\"https://e.ggtimer.com/420?alert=false\" frameborder=\"0\" allowfullscreen  width=\" 80%\" height=\"100\" style=\"box-shadow: 0 0 16px gray; margin: 10px;\" name=\"test\" id=\"test\"></iframe>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "%%html\n",
    "<iframe src=\"https://e.ggtimer.com/420?alert=false\" frameborder=\"0\" allowfullscreen  width=\" 80%\" height=\"100\" style=\"box-shadow: 0 0 16px gray; margin: 10px;\" name=\"test\" id=\"test\"></iframe>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "__Theorem:__ $DIVTHREE_n \\in SIZE(O(n))$. i.e., exists $c$ such that $DIVTHREE_n \\in SIZE(c\\cdot n)$ for every sufficiently large $n$."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "__Proof outline:__  \n",
    "i) Represent $[3]=\\{0,1,2\\}$ as $\\{0,1\\}^2$.  \n",
    "ii) We'll compute $f:[3]^n \\rightarrow [3]$ where $f(a_0,\\ldots,a_{n-1}) = \\sum_{i=0}^{n-1} a_i \\mod 3$. _enough!_"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "To compute $f(a_0,\\ldots,a_{n-1})$:  \n",
    "i) Set $s=0$  \n",
    "ii) For $i=0,\\ldots,n-1$: set $s = g(s,a_i)$ where $g:[3]^2 \\rightarrow [3]$ is $g(a,b)= a+b \\mod 3$.\n",
    "\n",
    "__Cost:__ $n \\times cost(g) = O(n)$ using main theorem!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Multi-output functions:\n",
    "\n",
    "To compute $f:\\{0,1\\}^n \\rightarrow \\{0,1\\}^2$: define\n",
    "\n",
    "* $g_0:\\{0,1\\}^n \\rightarrow \\{0,1\\}$ as $f_0(x) = f(x)_0$\n",
    "\n",
    "* $g_1:\\{0,1\\}^n \\rightarrow \\{0,1\\}$ as $f_1(x) = f(x)_1$.\n",
    "\n",
    "Using $P_0$ to compute $g_0$ and $P_1$ to compute $g_1$ can obtain $P$ to compute $f$."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Bottom line\n",
    "\n",
    "* Can compute _every_ function $f:\\{0,1\\}^n \\rightarrow \\{0,1\\}^m$ using $O(m \\cdot 2^n)$ (in fact $O(m\\cdot 2^n / n)$) lines.\n",
    "\n",
    "* Sometimes can do _much much much_ better"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Shaving off an $n$ factor\n",
    "\n",
    "Think of $f:\\{0,1\\}^n \\rightarrow \\{0,1\\}$ as $f:\\{0,1\\}^{n-k} \\times \\{0,1\\}^k \\rightarrow \\{0,1\\}$.\n",
    "\n",
    "For every $x\\in \\{0,1\\}^{n-k}$, define $f_x:\\{0,1\\}^k \\rightarrow \\{0,1\\}$ as the function $f_x(y)=f(x,y)$. Can be thought of as a string of length $2^k$.\n",
    "\n",
    "So we can compute the map $x \\mapsto f_x$ using $O(2^k \\cdot 2^{n-k})$ lines.\n",
    "\n",
    "Afte we have $f_x$, we can compute $y \\mapsto f_x(y)=f(x,y)$ with a lookup of $O(2^k)$ lines.\n",
    "\n",
    "If we set $k = \\log(n-2\\log n)$ we get that $2^k \\cdot 2^{n-k} = O(2^n/n)$.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Next up: \"Code = Data\"\n",
    "\n",
    "Think of program $P$ as a _string_ - give it to another program $Q$ as input"
   ]
  }
 ],
 "metadata": {
  "celltoolbar": "Slideshow",
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}