Downloaded 16 times
Machine learning from software developers point of view
- 1. Machine learning from SoftwareDevelopers point of view Let’s dive into the A.I jungle ! @antoined @pierre_paci Antoine Detante Pierre Paci
- 3. Framework Dive in thejungle !
- 4. • Complete ecosystem •Biggest community • 2nd biggest code repository on GitHub • Complete model zoo usable for production • Developed and released by Google Brain • Python, C++, Java, Rust, Haskell • Close relation with Google Cloud ML • Static graph computation • New Dynamic mode since 1.5 : TensorFlow Eager • Hard to escape TensorFlow ecosystem • Raw TensorFlow can be difficult • Describing the TF ecosystem would need an entire presentation
- 7. • Lots ofstate of the art implementation • Facebook publish lots of model • Only one simple API • Learn it once and for all • Part of the ONNX ecosystem • Very quick expansion • Developed and released by Facebook Research • Python • Fork from Lua’s Torch framework • Lots of official paper implementation released in PyTorch • Dynamic graph computation • Deployment • Must have a complete Python pipeline • Must use ONNX and another framework • No direct cloud support • Quite new
- 9. A word onCaffe2 How is Caffe2 different from PyTorch? Caffe2 is built to excel at mobile and at large scale deployments. While it is new in Caffe2 to support multi-GPU, bringing Torch and Caffe2 together with the same level of GPU support, Caffe2 is built to excel at utilizing both multiple GPUs on a single-host and multiple hosts with GPUs. PyTorch is great for research, experimentation and trying out exotic neural networks, while Caffe2 is headed towards supporting more industrial-strength applications with a heavy focus on mobile. This is not to say that PyTorch doesn’t do mobile or doesn’t scale or that you can’t use Caffe2 with some awesome new paradigm of neural network, we’re just highlighting some of the current characteristics and directions for these two projects. We plan to have plenty of interoperability and methods of converting back and forth so you can experience the best of both worlds.
- 10. • Apache project •Low, high level API (Gluon) • ONNX Support • Industry ready • Fit for research and production • Apache Project • Currently, MXNet is supported by Intel, Dato, Baidu, Microsoft, Wolfram Research, and research institutions such as Carnegie Mellon, MIT, the University of Washington, and the Hong Kong University of Science and Technology. • Supported on AWS and Azure • Designed for Big scale • Portable • Nearly all language with binding • C++ binary compilation for all platform (mobile included) • Static and dynamic graph computation • Small model zoo • Small community • But big industry support
- 12. High level API Andcross API
- 13. • Super easyto learn • Can scale to more complex problem • Lots of helpers included • Integrated model zoo • Integration with scikit-learn • Initially a high level interface to Theano and Tensorflow • Now officially part of Tensorflow • Started by François Chollet, from Google • Focus on quick iteration • Behave like a complete framework • No real company behind it • Model zoo is lacking state of the art
- 16. • Apache projet •Low, high level API (Gluon) • ONNX Support • Industry ready • Fit for research and production • Developed by MXNet • Inspired by PyTorch • More adapted to Research or Dynamic graph computation than raw MXNet • Should be supported by CNTK (Microsoft) soon • Small model zoo • Small community • But big industry support
- 18. Common model format Howto switch
- 19. About ONNX ONNX isa community project created by Facebook and Microsoft. We believe there is a need for greater interoperability in the AI tools community. Many people are working on great tools, but developers are often locked in to one framework or ecosystem. ONNX provides a definition of an extensible computation graph model, as well as definitions of built- in operators and standard data types. Operators are implemented externally to the graph, but the set of built-in operators are portable across frameworks. Every framework supporting ONNX will provide implementations of these operators on the applicable data types.
- 20. Global overview Map ofthe ecosystem
- 21.
- 22. Let’s dive into Neuralnetworks API for human beings
- 23. An Open Sourceneural networks library • Written in Python • Running on top of TensorFlow, CNTK & Theano • Can be run on CPU and GPU • Supports CNN and RNN, as well as combinations of the two …built for fast experimentation • User friendliness: designed for human beings, not machines! Consistent and simple API • Modularity: models are sequences or graphs of standalone modules that can be plugged together • Extensibility: new modules are simple to add (as new classes and functions) • Work with Python: models are described in Python code and are compact, easy to debug and easy to extend
- 24.
- 25. Keras has alot of built in layers • Dense layer of neural network • Common activation functions like linear, sigmoid, tanh, ReLU, … • Dropout, L1/L2 regularizers • Convolutional layers (Conv1D, Conv2D, Conv3D, …) • Pooling layers • Recurrent layers (fully connected RNN, LTSM, …) All these layers can be tuned, and you can add custom layers by extending existing ones or writing new Python classes.
- 26.
- 27. Custom optimizer, lossand metrics
- 28. Training the model •Models are trained on Numpy arrays • Input data and labels must be passed to the fit method of the model • The number of epochs is fixed (number of iterations on the dataset) • Validation set can be provided to the fit method (for evaluation of loss and metrics) At the end of the training, fit will return an history of metrics and training loss values at each epochs.
- 29.
- 30.
- 31. Integration with Scikit-Learn Keras provideswrappers which can be used from Scikit-Learn pipelines. It allows to use a Keras Sequential model as a classifier or regressor in Scikit-Learn
- 32. Keras functional API TheKeras functional API is the way to go for defining complex models, such as multi-output models, directed acyclic graphs, or models with shared layers. With the functional API, all models can be called as if it where a layer. It’s easy to reused trained models in a larger pipeline.
- 33. Graphs with multipleinput / output