C19 Machine Learning lectures Hilary 2015               Andrew Zisserman

Lecture 1 "Introduction"

Lecture 2 "The SVM Classifier"

Lecture 3 "SVM dual, kernels and regression"

Lecture 4 "Regression continued and multiple classes"

Examples sheet

Recommended books:

• Christopher M. Bishop, "Pattern Recognition and Machine Learning" , Springer (2006), ISBN 0-38-731073-8.
• Hastie, Tibshirani, Friedman, "Elements of Statistical Learning", Second Edition, Springer, 2009. Pdf available online.
• Ian H. Witten and Eibe Frank, "Data Mining: Practical Machine Learning Tools and Techniques" , Second Edition, 2005.
• David MacKay, "Information Theory, Inference, and Learning Algorithms" Which is freely available online!
• Tom Mitchell, "Machine Learning" , McGraw Hill, 1997

Web resources

Recommended Machine Learning Courses on the Web:

• Ben Taskar at the University of Pennsylvania
• Andrew Ng at Stanford     pdfs     video lectures
• Greg Mori at Simon Fraser University
• Geoff Hinton at Toronto - old course   Geoff Hinton at Toronto - new (partial) course
• Chris Williams and Victor Lavrenko at The University of Edinburgh
• Tom Mitchell and Andrew W. Moore at CMU
• links to course on ML and teaching material

K-NN:

• Neighborhood Component Analysis by Jacob Goldberger, Sam Roweis, Geoff Hinton, Ruslan Salakhutdinov. See the video lecture

Support Vector Machines:

• 2D-classification SVM applet
• Chris William's nice introduction to SVMs
• Christoph Lampert's excellent tutorial on SVMs
• Scholkopf and Smola "Learning with kernels" SVM introduction
• Christopher J. C. Burges, "A Tutorial on Support Vector Machines for Pattern Recognition"
• John Shawe-Taylor "Support Vector and Kernel Methods" video lecture
• Bernhard Scholkopf "Introduction to Kernels" video lecture
• David Lindsay's "An introduction to Support Vector Machine implementations in MATLAB"
• Code: Andrea Vedaldi's SVM MATLAB demo

AdaBoost:

• AdaBoost applet

Random forests:

• Antonio Criminisi et al.'s tutorial
• Real-Time Human Pose Recognition (kinect) by Jamie Shotton et al., CVPR 2011

Regression:

• Scholkopf and Smola "A tutorial on Support Vector Regression"
• Code: lasso MATLAB implementation from Koh, Kim and Boyd

PCA:

• A Tutorial on Principal Component Analysis
• "Face Modelling and Recognition Tutorial" by Thomas Vetter and Sami Romdhani
• Dimensionality Reduction applet

Dimensionality Reduction:

• Tutorial on Dimensionality Reduction by Lawrence Saul
• Isomap home page - software and data
• Original Isomap paper by Tenenbaum, de Silva and Langford, Science, 2000
• Original LLE paper by Roweis and Saul, Science, 2000
• Dimensionality Reduction by Carl Henrik Ek. Overview of methods and software.

Software and data:

• Statistical Pattern Recognition Toolbox
• The NETLAB ML Matlab software by Ian Nabney
• Matlab demos for L1 regularization by Schmidt et al.
• Matlab software for Gaussian Processes for Machine Learning by Rasmussen et al.
• The Weka Data Mining Software in Java
• THE MNIST DATABASE of handwritten digits
• UCI Machine Learning Repository (data)