Skip to content

Commit

Permalink
Merge pull request KerasKorea#127 from insurgent92/issue_40
Browse files Browse the repository at this point in the history 
Issue 40
  • Loading branch information
@visionNoob
visionNoob authored Oct 5, 2018
2 parents e4102fa + a74e730 commit 7844ce4
Showing 1 changed file with 129 additions and 37 deletions.
166 changes: 129 additions & 37 deletions 40_Keras와 Jupyter Notebook으로 시작하는 딥러닝/5.4-CNN 시각화해보기.ipynb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"# Paragraph 1\n",
"# Visualizing what convnets learn\n",
"\n",
"This notebook contains the code sample found in Chapter 5, Section 4 of [Deep Learning with Python](https://www.manning.com/books/deep-learning-with-python?a_aid=keras&a_bid=76564dff). Note that the original text features far more content, in particular further explanations and figures: in this notebook, you will only find source code and related comments.\n",
Expand All @@ -44,6 +45,7 @@
"concepts_. Since 2013, a wide array of techniques have been developed for visualizing and interpreting these representations. We won't \n",
"survey all of them, but we will cover three of the most accessible and useful ones:\n",
"\n",
"# Paragraph 2\n",
"* Visualizing intermediate convnet outputs (\"intermediate activations\"). This is useful to understand how successive convnet layers \n",
"transform their input, and to get a first idea of the meaning of individual convnet filters.\n",
"* Visualizing convnets filters. This is useful to understand precisely what visual pattern or concept each filter in a convnet is receptive \n",
Expand All @@ -59,6 +61,23 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"# Paragraph 1\n",
"To do"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Paragraph 2\n",
"To do"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Paragraph 3\n",
"## Visualizing intermediate activations\n",
"\n",
"Visualizing intermediate activations consists in displaying the feature maps that are output by various convolution and pooling layers in a \n",
Expand All @@ -69,46 +88,45 @@
"Let's start by loading the model that we saved in section 5.2:"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Paragraph 3\n",
"To do"
]
},
{
"cell_type": "code",
"execution_count": 2,
"execution_count": 1,
"metadata": {},
"outputs": [
{
"name": "stdout",
"name": "stderr",
"output_type": "stream",
"text": [
"_________________________________________________________________\n",
"Layer (type) Output Shape Param # \n",
"=================================================================\n",
"conv2d_5 (Conv2D) (None, 148, 148, 32) 896 \n",
"_________________________________________________________________\n",
"max_pooling2d_5 (MaxPooling2 (None, 74, 74, 32) 0 \n",
"_________________________________________________________________\n",
"conv2d_6 (Conv2D) (None, 72, 72, 64) 18496 \n",
"_________________________________________________________________\n",
"max_pooling2d_6 (MaxPooling2 (None, 36, 36, 64) 0 \n",
"_________________________________________________________________\n",
"conv2d_7 (Conv2D) (None, 34, 34, 128) 73856 \n",
"_________________________________________________________________\n",
"max_pooling2d_7 (MaxPooling2 (None, 17, 17, 128) 0 \n",
"_________________________________________________________________\n",
"conv2d_8 (Conv2D) (None, 15, 15, 128) 147584 \n",
"_________________________________________________________________\n",
"max_pooling2d_8 (MaxPooling2 (None, 7, 7, 128) 0 \n",
"_________________________________________________________________\n",
"flatten_2 (Flatten) (None, 6272) 0 \n",
"_________________________________________________________________\n",
"dropout_1 (Dropout) (None, 6272) 0 \n",
"_________________________________________________________________\n",
"dense_3 (Dense) (None, 512) 3211776 \n",
"_________________________________________________________________\n",
"dense_4 (Dense) (None, 1) 513 \n",
"=================================================================\n",
"Total params: 3,453,121\n",
"Trainable params: 3,453,121\n",
"Non-trainable params: 0\n",
"_________________________________________________________________\n"
"/Users/homme/anaconda/lib/python3.6/site-packages/h5py/__init__.py:34: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.\n",
" from ._conv import register_converters as _register_converters\n",
"Using TensorFlow backend.\n",
"/Users/homme/anaconda/lib/python3.6/importlib/_bootstrap.py:205: RuntimeWarning: compiletime version 3.5 of module 'tensorflow.python.framework.fast_tensor_util' does not match runtime version 3.6\n",
" return f(*args, **kwds)\n"
]
},
{
"ename": "OSError",
"evalue": "Unable to open file (Unable to open file: name = 'cats_and_dogs_small_2.h5', errno = 2, error message = 'no such file or directory', flags = 0, o_flags = 0)",
"output_type": "error",
"traceback": [
"\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
"\u001b[0;31mOSError\u001b[0m Traceback (most recent call last)",
"\u001b[0;32m<ipython-input-1-996a4fa18819>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[1;32m 1\u001b[0m \u001b[0;32mfrom\u001b[0m \u001b[0mkeras\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mmodels\u001b[0m \u001b[0;32mimport\u001b[0m \u001b[0mload_model\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 2\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 3\u001b[0;31m \u001b[0mmodel\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mload_model\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'cats_and_dogs_small_2.h5'\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 4\u001b[0m \u001b[0mmodel\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0msummary\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;31m# As a reminder.\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
"\u001b[0;32m/Users/homme/anaconda/lib/python3.6/site-packages/keras/engine/saving.py\u001b[0m in \u001b[0;36mload_model\u001b[0;34m(filepath, custom_objects, compile)\u001b[0m\n\u001b[1;32m 247\u001b[0m \u001b[0mopened_new_file\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0misinstance\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mfilepath\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mh5py\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mFile\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 248\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mopened_new_file\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 249\u001b[0;31m \u001b[0mf\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mh5py\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mFile\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mfilepath\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mmode\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m'r'\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 250\u001b[0m \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 251\u001b[0m \u001b[0mf\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mfilepath\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
"\u001b[0;32m/Users/homme/anaconda/lib/python3.6/site-packages/h5py/_hl/files.py\u001b[0m in \u001b[0;36m__init__\u001b[0;34m(self, name, mode, driver, libver, userblock_size, swmr, **kwds)\u001b[0m\n\u001b[1;32m 269\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 270\u001b[0m \u001b[0mfapl\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mmake_fapl\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mdriver\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mlibver\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwds\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 271\u001b[0;31m \u001b[0mfid\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mmake_fid\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mname\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mmode\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0muserblock_size\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mfapl\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mswmr\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mswmr\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 272\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 273\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mswmr_support\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
"\u001b[0;32m/Users/homme/anaconda/lib/python3.6/site-packages/h5py/_hl/files.py\u001b[0m in \u001b[0;36mmake_fid\u001b[0;34m(name, mode, userblock_size, fapl, fcpl, swmr)\u001b[0m\n\u001b[1;32m 99\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mswmr\u001b[0m \u001b[0;32mand\u001b[0m \u001b[0mswmr_support\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 100\u001b[0m \u001b[0mflags\u001b[0m \u001b[0;34m|=\u001b[0m \u001b[0mh5f\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mACC_SWMR_READ\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 101\u001b[0;31m \u001b[0mfid\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mh5f\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mopen\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mname\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mflags\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mfapl\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mfapl\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 102\u001b[0m \u001b[0;32melif\u001b[0m \u001b[0mmode\u001b[0m \u001b[0;34m==\u001b[0m \u001b[0;34m'r+'\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 103\u001b[0m \u001b[0mfid\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mh5f\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mopen\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mname\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mh5f\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mACC_RDWR\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mfapl\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mfapl\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
"\u001b[0;32mh5py/_objects.pyx\u001b[0m in \u001b[0;36mh5py._objects.with_phil.wrapper (/Users/ilan/minonda/conda-bld/h5py_1490026758621/work/h5py/_objects.c:2846)\u001b[0;34m()\u001b[0m\n",
"\u001b[0;32mh5py/_objects.pyx\u001b[0m in \u001b[0;36mh5py._objects.with_phil.wrapper (/Users/ilan/minonda/conda-bld/h5py_1490026758621/work/h5py/_objects.c:2804)\u001b[0;34m()\u001b[0m\n",
"\u001b[0;32mh5py/h5f.pyx\u001b[0m in \u001b[0;36mh5py.h5f.open (/Users/ilan/minonda/conda-bld/h5py_1490026758621/work/h5py/h5f.c:2123)\u001b[0;34m()\u001b[0m\n",
"\u001b[0;31mOSError\u001b[0m: Unable to open file (Unable to open file: name = 'cats_and_dogs_small_2.h5', errno = 2, error message = 'no such file or directory', flags = 0, o_flags = 0)"
]
}
],
Expand Down Expand Up @@ -191,6 +209,8 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"# pragraph 4\n",
"\n",
"In order to extract the feature maps we want to look at, we will create a Keras model that takes batches of images as input, and outputs \n",
"the activations of all convolution and pooling layers. To do this, we will use the Keras class `Model`. A `Model` is instantiated using two \n",
"arguments: an input tensor (or list of input tensors), and an output tensor (or list of output tensors). The resulting class is a Keras \n",
Expand All @@ -199,6 +219,14 @@
"Chapter 7, Section 1."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# pragraph 4\n",
"To do."
]
},
{
"cell_type": "code",
"execution_count": 5,
Expand All @@ -219,11 +247,22 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"When fed an image input, this model returns the values of the layer activations in the original model. This is the first time you encounter \n",
"# pragraph 5\n",
"\n",
"When fed an image input, this model returns the values of the layer activations in the original model. This is the \n",
"first time you encounter \n",
"a multi-output model in this book: until now the models you have seen only had exactly one input and one output. In the general case, a \n",
"model could have any number of inputs and outputs. This one has one input and 8 outputs, one output per layer activation."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# pragraph 5\n",
"To do."
]
},
{
"cell_type": "code",
"execution_count": 6,
Expand Down Expand Up @@ -475,6 +514,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"# pragraph 6\n",
"A few remarkable things to note here:\n",
"\n",
"* The first layer acts as a collection of various edge detectors. At that stage, the activations are still retaining almost all of the \n",
Expand All @@ -491,14 +531,15 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"# pragraph 7\n",
"We have just evidenced a very important universal characteristic of the representations learned by deep neural networks: the features \n",
"extracted by a layer get increasingly abstract with the depth of the layer. The activations of layers higher-up carry less and less \n",
"information about the specific input being seen, and more and more information about the target (in our case, the class of the image: cat \n",
"or dog). A deep neural network effectively acts as an __information distillation pipeline__, with raw data going in (in our case, RBG \n",
"pictures), and getting repeatedly transformed so that irrelevant information gets filtered out (e.g. the specific visual appearance of the \n",
"image) while useful information get magnified and refined (e.g. the class of the image).\n",
"\n",
"# pragraph 8\n",
"This is analogous to the way humans and animals perceive the world: after observing a scene for a few seconds, a human can remember which \n",
"abstract objects were present in it (e.g. bicycle, tree) but could not remember the specific appearance of these objects. In fact, if you \n",
"tried to draw a generic bicycle from mind right now, chances are you could not get it even remotely right, even though you have seen \n",
Expand All @@ -511,6 +552,31 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"# pragraph 6\n",
"To do."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# pragraph 7\n",
"To do."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# pragraph 8\n",
"To do."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Paragraph 9\n",
"## Visualizing convnet filters\n",
"\n",
"\n",
Expand All @@ -524,6 +590,14 @@
"a loss for the activation of filter 0 in the layer \"block3_conv1\" of the VGG16 network, pre-trained on ImageNet:"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Paragraph 9\n",
"To do."
]
},
{
"cell_type": "code",
"execution_count": 12,
Expand All @@ -549,6 +623,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"# paragraph 10\n",
"To implement gradient descent, we will need the gradient of this loss with respect to the model's input. To do this, we will use the \n",
"`gradients` function packaged with the `backend` module of Keras:"
]
Expand Down Expand Up @@ -749,6 +824,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"# paragraph 11\n",
"It seems that filter 0 in layer `block3_conv1` is responsive to a polka dot pattern.\n",
"\n",
"Now the fun part: we can start visualising every single filter in every layer. For simplicity, we will only look at the first 64 filters in \n",
Expand Down Expand Up @@ -832,7 +908,9 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"These filter visualizations tell us a lot about how convnet layers see the world: each layer in a convnet simply learns a collection of \n",
"# paragraph 12\n",
"These filter visualizations tell us a lot about how convnet layers see the world: each layer in a convnet simply \n",
"learns a collection of \n",
"filters such that their inputs can be expressed as a combination of the filters. This is similar to how the Fourier transform decomposes \n",
"signals onto a bank of cosine functions. The filters in these convnet filter banks get increasingly complex and refined as we go higher-up \n",
"in the model:\n",
Expand All @@ -847,6 +925,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"# paragraph 13\n",
"## Visualizing heatmaps of class activation\n",
"\n",
"We will introduce one more visualization technique, one that is useful for understanding which parts of a given image led a convnet to its \n",
Expand All @@ -860,6 +939,8 @@
"indicating how cat-like different parts of the image are, and likewise for the class \"dog\", indicating how dog-like differents parts of the \n",
"image are.\n",
"\n",
"# paragraph 14\n",
"\n",
"The specific implementation we will use is the one described in [Grad-CAM: Why did you say that? Visual Explanations from Deep Networks via \n",
"Gradient-based Localization](https://arxiv.org/abs/1610.02391). It is very simple: it consists in taking the output feature map of a \n",
"convolution layer given an input image, and weighing every channel in that feature map by the gradient of the class with respect to the \n",
Expand Down Expand Up @@ -898,6 +979,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"# paragraph 15\n",
"Let's consider the following image of two African elephants, possible a mother and its cub, strolling in the savanna (under a Creative \n",
"Commons license):\n",
"\n",
Expand Down Expand Up @@ -965,7 +1047,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"# paragraph 16\n",
"The top-3 classes predicted for this image are:\n",
"\n",
"* African elephant (with 92.5% probability)\n",
Expand Down Expand Up @@ -1122,6 +1204,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"# paragraph 17\n",
"This visualisation technique answers two important questions:\n",
"\n",
"* Why did the network think this image contained an African elephant?\n",
Expand All @@ -1130,6 +1213,15 @@
"In particular, it is interesting to note that the ears of the elephant cub are strongly activated: this is probably how the network can \n",
"tell the difference between African and Indian elephants.\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": []
}
],
"metadata": {
Expand Down

0 comments on commit 7844ce4

Please sign in to comment.