@leriomaggio	+ValerioMaggio
valeriomaggio	vmaggio_at_fbk_dot_eu

git clone https://github.com/leriomaggio/deep-learning-keras-tensorflow.git

• Part I: Introduction

  • Intro to Deep Learning and ANN

    • Perceptron and MLP

  • naive pure-Python implementation

    • fast forward, sgd, backprop

  • Intro to Tensorflow

    • Model + SGD with Tensorflow

  • Introduction to Keras

    • Overview and main features
      • Keras Backend
      • Overview of the core layers
    • Multi-Layer Perceptron and Fully Connected
      • Examples with keras.models.Sequential and Dense
      • HandsOn: FC with keras

• Part II: Supervised Learning and Convolutional Neural Nets

  • Intro: Focus on Image Classification

  • Intro to ConvNets

    • meaning of convolutional filters
      • examples from ImageNet
    • Visualising ConvNets

  • Advanced CNN

    • Dropout
    • MaxPooling
    • Batch Normalisation

  • HandsOn: MNIST Dataset

    • FC and MNIST
    • CNN and MNIST

  • Deep Convolutiona Neural Networks with Keras (ref: keras.applications)

    • VGG16
    • VGG19
    • ResNet50

  • Transfer Learning and FineTuning

  • Hyperparameters Optimisation

• Part III: Unsupervised Learning

  • AutoEncoders and Embeddings
  • AutoEncoders and MNIST
    • word2vec and doc2vec (gensim) with keras.datasets
    • word2vec and CNN

• Part IV: Recurrent Neural Networks

  • Recurrent Neural Network in Keras
    • SimpleRNN, LSTM, GRU

• PartV: Additional Materials:

  • Quick tutorial on theano
  • Perceptron and Adaline (pure-python) implementations
  • MLP and MNIST (pure-python)
  • LSTM for Sentence Generation
  • Custom Layers in Keras
  • Multi modal Network Topologies with Keras

• Wrap up and Conclusions

This tutorial requires the following packages:

• Python version 3.5

  • Python 3.4 should be fine as well
  • likely Python 2.7 would be also fine, but who knows? :P

• numpy version 1.10 or later: http://www.numpy.org/

• scipy version 0.16 or later: http://www.scipy.org/

• matplotlib version 1.4 or later: http://matplotlib.org/

• pandas version 0.16 or later: http://pandas.pydata.org

• scikit-learn version 0.15 or later: http://scikit-learn.org

• keras version 2.0 or later: http://keras.io

• tensorflow version 1.0 or later: https://www.tensorflow.org

• ipython/jupyter version 4.0 or later, with notebook support

(Optional but recommended):

• pyyaml
• hdf5 and h5py (required if you use model saving/loading functions in keras)
• NVIDIA cuDNN if you have NVIDIA GPUs on your machines. https://developer.nvidia.com/rdp/cudnn-download

The easiest way to get (most) these is to use an all-in-one installer such as Anaconda from Continuum. These are available for multiple architectures.

I'm currently running this tutorial with Python 3 on Anaconda

!python --version

Python 3.5.2 

1. Create the keras.json (if it does not exist):

touch $HOME/.keras/keras.json

2. Copy the following content into the file:

{ "epsilon": 1e-07, "backend": "tensorflow", "floatx": "float32", "image_data_format": "channels_last" } 

!cat ~/.keras/keras.json

{	"epsilon": 1e-07,	"backend": "tensorflow",	"floatx": "float32",	"image_data_format": "channels_last" } 

import numpy as np import scipy as sp import pandas as pd import matplotlib.pyplot as plt import sklearn

import keras

Using TensorFlow backend. 

import numpy print('numpy:', numpy.__version__) import scipy print('scipy:', scipy.__version__) import matplotlib print('matplotlib:', matplotlib.__version__) import IPython print('iPython:', IPython.__version__) import sklearn print('scikit-learn:', sklearn.__version__)

numpy: 1.11.1 scipy: 0.18.0 matplotlib: 1.5.2 iPython: 5.1.0 scikit-learn: 0.18 

import keras print('keras: ', keras.__version__) # optional import theano print('Theano: ', theano.__version__) import tensorflow as tf print('Tensorflow: ', tf.__version__)

keras: 2.0.2 Theano: 0.9.0 Tensorflow: 1.0.1