Downloaded 42 times
![What is a single neuron? 20 • 3 inputs [x1,x2,x3] • 3 weights [w1,w2,w3] • Element-wise multiply and sum • Apply activation function f • Often add a bias too (weight of 1) – not](https://image.slidesharecdn.com/dl4cvslides-170915173401/75/Convolutional-Neural-Networks-for-Computer-vision-Applications-20-2048.jpg)
![What is an Activation Function? 21 Sigmoid Tanh ReLU Nonlinearities … “squashing functions” … transform neuron’s output NB: sigmoid output in [0,1]](https://image.slidesharecdn.com/dl4cvslides-170915173401/75/Convolutional-Neural-Networks-for-Computer-vision-Applications-21-2048.jpg)
![What is a Convolutional Neural Network? 27 “like a simple neural network but with special types of layers that work well on images” (math works on numbers) • Pixel = 3 colour channels (R, G, B) • Pixel intensity = number in [0,255] • Image has width w and height h • Therefore image is w x h x 3 numbers](https://image.slidesharecdn.com/dl4cvslides-170915173401/75/Convolutional-Neural-Networks-for-Computer-vision-Applications-27-2048.jpg)
![Softmax • Convert scores ∈ ℝ to probabilities ∈ [0,1] • Then predict the class with highest probability 40](https://image.slidesharecdn.com/dl4cvslides-170915173401/75/Convolutional-Neural-Networks-for-Computer-vision-Applications-40-2048.jpg)
Convolutional Neural Networks for Computer vision Applications
- 1. Convolutional Neural Networks Alex Conway alex@ numberboost.com & JHB Deep Learning Hackathon Info :) For Computer Vision Applications
- 2.
- 3. Check out the DeepLearning Indaba videos & practicals! http://www.deeplearningindaba.com/videos.html http://www.deeplearningindaba.com/practicals.html
- 4. Image Classification 4 http://yann.lecun.com/exdb/mnist/ https://github.com/fchollet/keras/blob/master/examples/mnist_cnn.py (99.25% testaccuracy in 192 seconds and 70 lines of code)
- 5.
- 6.
- 7.
- 8.
- 9. Image Captioning &Visual Attention XXX 9 https://einstein.ai/research/knowing-when-to-look-adaptive- attention-via-a-visual-sentinel-for-image-captioning
- 10.
- 11.
- 12.
- 13.
- 14.
- 15.
- 16.
- 17. 1. What isa neural network? 2. What is a convolutional neural network? 3. How to use a convolutional neural network 4. More advanced methods 5. Practical tips 6. Hackathon Challenge Info 17
- 18. Big Shout Outs JeremyHoward & Rachel Thomas http://course.fast.ai Andrej Karpathy http://cs231n.github.io François Chollet (Keras lead dev) https://keras.io/ 18
- 19. 1.What is aneural network?
- 20. What is asingle neuron? 20 • 3 inputs [x1,x2,x3] • 3 weights [w1,w2,w3] • Element-wise multiply and sum • Apply activation function f • Often add a bias too (weight of 1) – not
- 21. What is anActivation Function? 21 Sigmoid Tanh ReLU Nonlinearities … “squashing functions” … transform neuron’s output NB: sigmoid output in [0,1]
- 22. What is aDeep Neural Network? 22 Inputs outputs hidden layer 1 hidden layer 2 hidden layer 3 Outputs of one layer are inputs into the next layer
- 23. How does aneural network learn? 23 • You need labelled examples “training data” • Initially, the network makes random predictions (weights initialized randomly) • For each training data point, we calculate the error between the network’s predictions and the ground-truth labels (“loss function” e.g. mean squared error) • Using a method called ‘backpropagation’ (really just the chain rule from calculus 1), we use the error to update the weights (using ‘gradient descent’) so that the error is a little bit smaller next time (it learns from past errors)
- 24.
- 25. What is aNeural Network? For much more detail, see: 1. Michael Nielson’s Neural Networks & Deep Learning free online book http://neuralnetworksanddeeplearning.com/chap1.html 2. Anrej Karpathy’s CS231n Notes http://neuralnetworksanddeeplearning.com/chap1.html 25
- 26. 2. What isa convolutional neural network?
- 27. What is aConvolutional Neural Network? 27 “like a simple neural network but with special types of layers that work well on images” (math works on numbers) • Pixel = 3 colour channels (R, G, B) • Pixel intensity = number in [0,255] • Image has width w and height h • Therefore image is w x h x 3 numbers
- 28. 28 This is VGGNet– don’t panic, we’ll break it down piece by piece Example Architecture
- 29.
- 30.
- 31. New Layer Type:Convolutional Layer 31 • 2-d weighted average when multiply kernel over pixel patches • We slide the kernel over all pixels of the image (handle borders) • Kernel starts off with “random” values and network updates (learns) the kernel values (using backpropagation) to try minimize loss • Kernels shared across the whole image (parameter
- 32. Many Kernels =Many “Activation Maps” = Volume 32http://cs231n.github.io/convolutional-networks/
- 33.
- 34.
- 35.
- 36.
- 37.
- 38. New Layer Type:Max Pooling 38
- 39. New Layer Type:Max Pooling • Reduces dimensionality from one layer to next • By replacing NxN sub-area with max value • Makes network “look” at larger areas of the image at a time e.g. Instead of identifying fur, identify cat • Reduces computational load • Reduces overfitting since losing information helps the network generalize 39
- 40. Softmax • Convert scores∈ ℝ to probabilities ∈ [0,1] • Then predict the class with highest probability 40
- 41. Bringing it alltogether xxx 41 Convolution + max pooling + fully connected + softmax
- 42. Bringing it alltogether 42 Convolution + max pooling + fully connected + softmax
- 43. We need labelledtraining data!
- 44.
- 45.
- 46.
- 47.
- 48. 3. How touse a convolutional neural network
- 49. Using a Pre-TrainedImageNet-Winning CNN 49 • We’ve been looking at “VGGNet” • Oxford Visual Geometry Group (VGG) • The runner-up in ILSVRC 2014 • Network is 16 layers (deep!) • Its main contribution was in showing that the depth of the network is a critical component for good performance. • Only 3x3 (stride 1 pad 1) convolutions and 2x2 max pooling • Easy to fine-tune https://blog.keras.io/building-powerful-image-classification- models-using-very-little-data.html
- 50. Example: Classifying ProductImages 50 https://github.com/alexcnwy/CTDL_CNN_TALK_20 170620
- 51.
- 52. Fine-tuning A CNNTo Solve A New Problem • Cut off last layer of pre-trained Imagenet winning CNN • Keep learned network but replace final layer • Can learn to predict different classes • Fine-tuning is re-training new final layer 52
- 53.
- 54.
- 55. Fine-tuning A CNNTo Solve A New Problem • Fix weights in convolutional layers (set trainable=False) • Remove final dense layer that predicts 1000 imagenet • Replace with new dense layer to predict 9 categories 55 Gets 88% accuracy in classifying products into categories
- 56. Visual Similarity 56 • Chopoff last 2 VGG layers • Use dense layer with 4096 activations • Compute nearest neighbours in the space of these activations https://memeburn.com/2017/06/spree-image-search/
- 57.
- 58.
- 59. 4. More advancedtopics
- 60.
- 61. Long, Shelhamer, andDarrell, “Fully Convolutional Networks for Semantic Segmentation” CVPR 2015 Noh et al, “Learning Deconvolution Network for Semantic Segmentation” CVPR 2015 Semantic Segmentation
- 62.
- 63.
- 64.
- 66. http://blog.romanofoti.com/style_transfer/ https://github.com/fchollet/keras/blob/master/examples/neural_style_transfer.py Style Transfer Loss =content_loss + style_loss Content loss = convolutions from pre-trained network Style loss = gram matrix from style image convolutions
- 67.
- 68.
- 69. Practical Tips • usea GPU – AWS p2 instances (use spot!) • when overfitting (validation_accuracy <<< training_accuracy) – Increase dropout – Early stopping • when underfitting (low training_accuracy) 1. Add more data 2. Use data augmentation – Flipping / stretching / rotating – Slightly change hues / brightness 3. Use more complex model – Resnet / Inception / Densenet – Ensamble (average <<< weighted average with learned weights) 70
- 70.
- 71.
- 72.
- 73. Hackathon Details Challenge: Detectpotholes in images Binary classification problem Given +- 5000 training images with labels Given number of potholes + bounding box annotations Given convolutional output for each image (in train & test) Submit predictions on held-out test images MOST ACCURATE MODEL WINS! 74
- 74. Hackathon Details Can formteams of up to 4 people You need to present your approach (2 min max) Prizes for best presentation / most innovative approach (even if not most accurate) Don’t worry if you don’t know any of this stuff – it’s a great opportunity to learn! Hackathons are fun!!!! 75
- 75. Why this isan important problem Potholes cause road deaths: “The Automobile Association (AA) said that if there was proper maintenance of our roads, then there could be an immediate decrease in about 5% of road deaths … South Africa’s has more than 700000 accidents occurring annually.” http://www.roadcover.co.za/potholes-how-they-worsen-our-roads/ 35’000 lives!! 2 levels: • Alert the road authorities where they are to fix quickly • Real time pothole avoidance (much harder!) 76 Big thanks to Dr. Steve Kroon for suggesting and helping with the dataset
- 76. See you tomorrow! Samevenue (THE DIZ 111 Smit Street) 9am – 830pm
- 77. QUESTIONS? Email me :) AlexConway alex @ numberboost.com
Editor's Notes
- #56 Gets 88% accuracy in classifying products into categories