Skip to content

Commit

Permalink
fix links and README
Browse files Browse the repository at this point in the history
  • Loading branch information
@jostosh
jostosh committed Oct 11, 2020
1 parent 83abb3d commit 8f351f6
Show file tree
Hide file tree
Showing 2 changed files with 32 additions and 125 deletions.
10 changes: 5 additions & 5 deletions README.md
View file
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
[![Python](https://img.shields.io/pypi/pyversions/libspn-keras.svg?style=plastic)](https://badge.fury.io/py/libspn-keras)
[![tests](https://github.com/pronobis/libspn-keras/workflows/tests/badge.svg)](https://github.com/pronobis/libspn-keras/actions?workflow=tests)
[![tests](https://github.com/pronobis/libspn-keras/workflows/Tests/badge.svg)](https://github.com/pronobis/libspn-keras/actions?workflow=tests)
[![PyPI](https://img.shields.io/pypi/v/libspn-keras.svg)](https://pypi.org/project/libspn-keras/)
[![Read the Docs](https://readthedocs.org/projects/libspn-keras/badge/)](https://libspn-keras.readthedocs.io/)

Expand Down Expand Up @@ -58,10 +58,10 @@ Currently, the repo is in an alpha state. Hence, one can expect some sporadic br
univariate as well as *multivariate* inputs.

## Examples / Tutorials
1. **Image Classification**: A Deep Generalized Convolutional Sum-Product Network (DGC-SPN) with `libspn-keras` in Colab[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/pronobis/libspn-keras/blob/master/examples/notebooks/DGC-SPN%20Image%20Classification.ipynb)
2. **Image Completion**: A Deep Generalized Convolutional Sum-Product Network (DGC-SPN) with `libspn-keras` in Colab.[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/pronobis/libspn-keras/blob/master/examples/notebooks/DGC-SPN%20Image%20Completion.ipynb)
3. **Randomly structured SPNs** for image classification[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/pronobis/libspn-keras/blob/master/examples/notebooks/Randomly%20Structured%20SPNs%20Image%20Classification.ipynb)
4. **Understanding region SPNs**[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/pronobis/libspn-keras/blob/master/examples/notebooks/Understanding%20Region%20SPNs.ipynb)
1. **Benchmark**: `libspn-keras` andd Einsum Networks. [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/pronobis/libspn-keras/blob/master/examples/notebooks/Benchmark%20With%20Einsum%20Networks.ipynb)
2. **Image Classification**: A Deep Generalized Convolutional Sum-Product Network (DGC-SPN). [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/pronobis/libspn-keras/blob/master/examples/notebooks/DGC-SPN%20Image%20Classification.ipynb)
3. **Image Completion**: A Deep Generalized Convolutional Sum-Product Network (DGC-SPN). [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/pronobis/libspn-keras/blob/master/examples/notebooks/DGC-SPN%20Image%20Completion.ipynb)
4. **Understanding region SPNs** [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/pronobis/libspn-keras/blob/master/examples/notebooks/Understanding%20Region%20SPNs.ipynb)
5. More to come, and if you would like to see a tutorial on anything in particular
please raise an issue!

Expand Down
147 changes: 27 additions & 120 deletions ...es/notebooks/Benchmark With Einsums.ipynb → ...ooks/Benchmark With Einsum Networks.ipynb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
},
"source": [
"[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/pronobis/libspn-keras/blob/master/examples/notebooks/Benchmark%20With%20Einsum%20Networks.ipynb)\n",
"# Bench-marking LibSPN-Keras against EinsumNetworks\n",
"# Bench-marking LibSPN-Keras against Einsum Networks\n",
"Recently, [Peharz et al.](https://arxiv.org/abs/2004.06231) proposed _Einsum Networks_, which are essentially SPNs with a specific implementation style. Einsum Networks implement a range of tricks that can be used to easily achieve tensor-based building of SPNs as well as EM learning through _autograd_. \n",
"\n",
"Fortunately, `libspn-keras` already had the same tricks implemented _ever since it was made available_ (before the Einsum paper was published), and so it was straightforward to set up a side-by-side comparison of `libspn-keras` \n",
Expand All @@ -20,29 +20,18 @@
},
{
"cell_type": "code",
"execution_count": 6,
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 85
"height": 241
},
"id": "6bGer1RP3mAF",
"outputId": "9a736743-d95c-424d-8623-6d3eee97f03d"
"outputId": "64b80200-607f-49f3-863d-acbb29d74875"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"fatal: destination path 'EinsumNetworks' already exists and is not an empty directory.\n",
"Processing ./libspn_keras-0.4.0-py3-none-any.whl\n",
"Installing collected packages: libspn-keras\n",
"Successfully installed libspn-keras-0.4.0\n"
]
}
],
"outputs": [],
"source": [
"!git clone https://github.com/cambridge-mlg/EinsumNetworks & pip install libspn_keras --ignore-installed --no-deps"
"!git clone https://github.com/cambridge-mlg/EinsumNetworks & pip install libspn-keras --ignore-installed --no-deps"
]
},
{
Expand All @@ -61,7 +50,7 @@
},
{
"cell_type": "code",
"execution_count": 4,
"execution_count": null,
"metadata": {
"id": "p0yeY0Xe3mAB"
},
Expand Down Expand Up @@ -98,48 +87,16 @@
},
{
"cell_type": "code",
"execution_count": 8,
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 442
},
"id": "8eIlR4Bc3mAL",
"outputId": "c6d5977b-00b7-48f5-a2e2-c86c8598bed6"
"outputId": "d0f3a69d-ed5a-4450-e795-40689eb43c47"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Running on cuda\n",
"EinsumNetwork(\n",
" (einet_layers): ModuleList(\n",
" (0): FactorizedLeafLayer(\n",
" (ef_array): CategoricalArray()\n",
" )\n",
" (1): EinsumLayer()\n",
" (2): EinsumLayer()\n",
" (3): EinsumLayer()\n",
" (4): EinsumLayer()\n",
" (5): EinsumLayer()\n",
" (6): EinsumMixingLayer()\n",
" )\n",
")\n",
"Number of input dist parameters\n",
"227328\n",
"Number of remaining sum parameters\n",
"[16777216, 8388608, 4194304, 2097152, 32768, 32]\n",
"Total trainable parameters\n",
"31717408\n",
"[0] valid LL -76.9047509765625\n",
"[1] valid LL -37.21986155790441\n",
"[2] valid LL -36.9792524988511\n",
"Total time: 43.69 seconds\n",
"-37.363624970523205\n"
]
}
],
"outputs": [],
"source": [
"import sys\n",
"sys.path.append(\"EinsumNetworks/src\")\n",
Expand All @@ -151,6 +108,9 @@
"import csv\n",
"import numpy as np\n",
"\n",
"np.random.seed(1234)\n",
"torch.random.manual_seed(1234)\n",
"\n",
"device = 'cuda' if torch.cuda.is_available() else 'cpu'\n",
"\n",
"print(\"Running on \", device)\n",
Expand Down Expand Up @@ -243,88 +203,35 @@
"source": [
"## Quick Analysis\n",
"We see the following interesting things:\n",
"- A test LLH of around -37.36\n",
"- A total train time of 43.69 seconds\n",
"- A test LLH of around -37.39\n",
"- A total train time of 148.64 seconds (depends of course on the hardware that this notebook runs on)\n",
"- Total number of trainable parameters 31,717,408 \n",
"\n",
"## LibSPN-Keras Implementation\n",
"Instead of hiding all the complexity behind a `Graph.random_binary_trees`, `libspn-keras` offers a layer-based approach that offers the ability to control all network parameters flexibly at every layer. In that sense, we are in no sense forced to use the same number of sums for every layer, nor a fixed number of 2 children per product. We can easily vary all of these parameters. Nevertheless, for the sake of comparison we'll reimplement the above architecture using `libspn-keras`.\n",
"Instead of hiding all the complexity behind a `Graph.random_binary_trees`, `libspn-keras` offers a layer-based approach that gives you control over all network parameters flexibly at every layer. In that sense, we are in no sense forced to use the same number of sums for every layer, nor a fixed number of 2 children per product. We can easily vary all of these parameters. Nevertheless, for the sake of comparison we'll reimplement the above architecture using `libspn-keras`.\n",
"\n",
"Let's see how they compare!"
]
},
{
"cell_type": "code",
"execution_count": 9,
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 833
},
"id": "5HuP4SJT3mAO",
"outputId": "57f50dbf-3397-4839-e4cb-0ad997ce520f"
"outputId": "d2f37e63-d32b-42d8-d183-44ebee5c2503"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Model: \"sequential_sum_product_network_3\"\n",
"_________________________________________________________________\n",
"Layer (type) Output Shape Param # \n",
"=================================================================\n",
"flat_to_regions_3 (FlatToReg (None, 111, 32, 1) 0 \n",
"_________________________________________________________________\n",
"indicator_leaf_3 (IndicatorL (None, 111, 32, 2) 0 \n",
"_________________________________________________________________\n",
"dense_sum_19 (DenseSum) (None, 111, 32, 32) 227328 \n",
"_________________________________________________________________\n",
"permute_and_pad_scopes_rando (None, 128, 32, 32) 4096 \n",
"_________________________________________________________________\n",
"reduce_product_3 (ReduceProd (None, 32, 32, 32) 0 \n",
"_________________________________________________________________\n",
"dense_product_16 (DenseProdu (None, 16, 32, 1024) 0 \n",
"_________________________________________________________________\n",
"dense_sum_20 (DenseSum) (None, 16, 32, 32) 16777216 \n",
"_________________________________________________________________\n",
"dense_product_17 (DenseProdu (None, 8, 32, 1024) 0 \n",
"_________________________________________________________________\n",
"dense_sum_21 (DenseSum) (None, 8, 32, 32) 8388608 \n",
"_________________________________________________________________\n",
"dense_product_18 (DenseProdu (None, 4, 32, 1024) 0 \n",
"_________________________________________________________________\n",
"dense_sum_22 (DenseSum) (None, 4, 32, 32) 4194304 \n",
"_________________________________________________________________\n",
"dense_product_19 (DenseProdu (None, 2, 32, 1024) 0 \n",
"_________________________________________________________________\n",
"dense_sum_23 (DenseSum) (None, 2, 32, 32) 2097152 \n",
"_________________________________________________________________\n",
"dense_product_20 (DenseProdu (None, 1, 32, 1024) 0 \n",
"_________________________________________________________________\n",
"dense_sum_24 (DenseSum) (None, 1, 32, 1) 32768 \n",
"_________________________________________________________________\n",
"root_sum_3 (RootSum) (None, 1) 32 \n",
"=================================================================\n",
"Total params: 31,721,504\n",
"Trainable params: 31,717,408\n",
"Non-trainable params: 4,096\n",
"_________________________________________________________________\n",
"Epoch 1/3\n",
"128/128 [==============================] - 7s 58ms/step - loss: 41.6101 - llh: -41.6101 - val_loss: 37.1047 - val_llh: -37.1047\n",
"Epoch 2/3\n",
"128/128 [==============================] - 7s 55ms/step - loss: 37.1014 - llh: -37.1014 - val_loss: 36.8422 - val_llh: -36.8422\n",
"Epoch 3/3\n",
"128/128 [==============================] - 7s 55ms/step - loss: 36.9422 - llh: -36.9422 - val_loss: 36.7074 - val_llh: -36.7074\n",
"Total time: 23.43 seconds\n",
"1/1 [==============================] - 0s 2ms/step - loss: 37.1249 - llh: -37.1249\n"
]
}
],
"outputs": [],
"source": [
"import libspn_keras as spnk\n",
"import itertools\n",
"import tensorflow as tf\n",
"\n",
"tf.random.set_seed(1234)\n",
"\n",
"def build_ratspn(num_vars, depth):\n",
" # Set global settings\n",
" spnk.set_default_sum_op(spnk.SumOpEMBackprop())\n",
Expand Down Expand Up @@ -356,8 +263,8 @@
" return spnk.models.SequentialSumProductNetwork(sum_product_stack)\n",
"\n",
"train_ds = tf.data.Dataset.from_tensor_slices((train_x_orig,)).shuffle(10_000).batch(batch_size)\n",
"valid_ds = tf.data.Dataset.from_tensor_slices((valid_x_orig,)).batch(len(valid_x_orig))\n",
"test_ds = tf.data.Dataset.from_tensor_slices((test_x_orig,)).batch(len(test_x_orig))\n",
"valid_ds = tf.data.Dataset.from_tensor_slices((valid_x_orig,)).batch(batch_size)\n",
"test_ds = tf.data.Dataset.from_tensor_slices((test_x_orig,)).batch(batch_size)\n",
"\n",
"spn = build_ratspn(train_x_orig.shape[1], depth=depth)\n",
"\n",
Expand All @@ -384,18 +291,18 @@
"source": [
"## Analysis\n",
"We now see the following\n",
"- A test LLH of around -37.12 (not significantly different which makes sense)\n",
"- A total train time of **23.43** seconds\n",
"- A test LLH of around -37.26 (not significantly different which makes sense)\n",
"- A total train time of **63.64** seconds\n",
"- Total number of trainable parameters 31,717,408 (same as above just as a sanity check)\n",
"\n",
"In other words, `libspn-keras` is not just more flexible, but also almost twice as fast here!"
"In other words, `libspn-keras` is not just more flexible, but also more than twice as fast here!"
]
}
],
"metadata": {
"accelerator": "GPU",
"colab": {
"name": "Benchmark With Einsums.ipynb",
"name": "Benchmark With Einsum Networks.ipynb",
"provenance": [],
"toc_visible": true
},
Expand Down

0 comments on commit 8f351f6

Please sign in to comment.