book

Practical Deep Learning for Cloud, Mobile, and Edge

by Anirudh Koul, Siddha Ganju, Meher Kasam

October 2019

Intermediate to advanced

583 pages

14h 36m

English

O'Reilly Media, Inc.

Read now

Unlock full access

To the Backend/Frontend/Mobile Software DeveloperTo the Data ScientistTo the StudentTo the TeacherTo the Robotics EnthusiastWhat to Expect in Each ChapterConventions Used in This BookUsing Code ExamplesO’Reilly Online LearningHow to Contact UsAcknowledgmentsGroup AcknowledgmentsPersonal Acknowledgments
An ApologyThe Real IntroductionWhat Is AI?Motivating ExamplesA Brief History of AIExciting BeginningsThe Cold and Dark DaysA Glimmer of HopeHow Deep Learning Became a ThingRecipe for the Perfect Deep Learning SolutionDatasetsModel ArchitectureFrameworksHardwareResponsible AIBiasAccountability and ExplainabilityReproducibilityRobustnessPrivacySummaryFrequently Asked Questions
Introducing KerasPredicting an Image’s CategoryInvestigating the ModelImageNet DatasetModel ZoosClass Activation MapsSummary
Adapting Pretrained Models to New TasksA Shallow Dive into Convolutional Neural NetworksTransfer LearningFine TuningHow Much to Fine TuneBuilding a Custom Classifier in Keras with Transfer LearningOrganize the DataBuild the Data PipelineNumber of ClassesBatch SizeData AugmentationModel DefinitionTrain the ModelSet Training ParametersStart TrainingTest the ModelAnalyzing the ResultsFurther ReadingSummary
Image SimilarityFeature ExtractionSimilarity SearchVisualizing Image Clusters with t-SNEImproving the Speed of Similarity SearchLength of Feature VectorsReducing Feature-Length with PCAScaling Similarity Search with Approximate Nearest NeighborsApproximate Nearest-Neighbor BenchmarkWhich Library Should I Use?Creating a Synthetic DatasetBrute ForceAnnoyNGTFaissImproving Accuracy with Fine TuningFine Tuning Without Fully Connected LayersSiamese Networks for One-Shot Face VerificationCase StudiesFlickrPinterestCelebrity DoppelgangersSpotifyImage CaptioningSummary
Tools of the TradeTensorFlow DatasetsTensorBoardWhat-If Tooltf-explainCommon Techniques for Machine Learning ExperimentationData InspectionBreaking the Data: Train, Validation, TestEarly StoppingReproducible ExperimentsEnd-to-End Deep Learning Example PipelineBasic Transfer Learning PipelineBasic Custom Network PipelineHow Hyperparameters Affect AccuracyTransfer Learning Versus Training from ScratchEffect of Number of Layers Fine-Tuned in Transfer LearningEffect of Data Size on Transfer LearningEffect of Learning RateEffect of OptimizersEffect of Batch SizeEffect of ResizingEffect of Change in Aspect Ratio on Transfer LearningTools to Automate Tuning for Maximum AccuracyKeras TunerAutoAugmentAutoKerasSummary
GPU Starvationnvidia-smiTensorFlow Profiler + TensorBoardHow to Use This ChecklistPerformance ChecklistData PreparationData ReadingData AugmentationTrainingInferenceData PreparationStore as TFRecordsReduce Size of Input DataUse TensorFlow DatasetsData ReadingUse tf.dataPrefetch DataParallelize CPU ProcessingParallelize I/O and ProcessingEnable Nondeterministic OrderingCache DataTurn on Experimental OptimizationsAutotune Parameter ValuesData AugmentationUse GPU for AugmentationTrainingUse Automatic Mixed PrecisionUse Larger Batch SizeUse Multiples of EightFind the Optimal Learning RateUse tf.functionOvertrain, and Then GeneralizeInstall an Optimized Stack for the HardwareOptimize the Number of Parallel CPU ThreadsUse Better HardwareDistribute TrainingExamine Industry BenchmarksInferenceUse an Efficient ModelQuantize the ModelPrune the ModelUse Fused OperationsEnable GPU PersistenceSummary
InstallationTrainingModelDataPrivacyEducation and ExplorationOne Last Question
The Landscape of Visual Recognition APIsClarifaiMicrosoft Cognitive ServicesGoogle Cloud VisionAmazon RekognitionIBM Watson Visual RecognitionAlgorithmiaComparing Visual Recognition APIsService OfferingsCostAccuracyBiasGetting Up and Running with Cloud APIsTraining Our Own Custom ClassifierTop Reasons Why Our Classifier Does Not Work SatisfactorilyComparing Custom Classification APIsPerformance Tuning for Cloud APIsEffect of Resizing on Image Labeling APIsEffect of Compression on Image Labeling APIsEffect of Compression on OCR APIsEffect of Resizing on OCR APIsCase StudiesThe New York TimesUberGiphyOmniEarthPhotobucketStaplesInDro RoboticsSummary
Landscape of Serving AI PredictionsFlask: Build Your Own ServerMaking a REST API with FlaskDeploying a Keras Model to FlaskPros of Using FlaskCons of Using FlaskDesirable Qualities in a Production-Level Serving SystemHigh AvailabilityScalabilityLow LatencyGeographic AvailabilityFailure HandlingMonitoringModel VersioningA/B TestingSupport for Multiple Machine Learning LibrariesGoogle Cloud ML Engine: A Managed Cloud AI Serving StackPros of Using Cloud ML EngineCons of Using Cloud ML EngineBuilding a Classification APITensorFlow ServingInstallationKubeFlowPipelinesFairingInstallationPrice Versus Performance ConsiderationsCost Analysis of Inference-as-a-ServiceCost Analysis of Building Your Own StackSummary

JavaScript-Based Machine Learning Libraries: A Brief HistoryConvNetJSKeras.jsONNX.jsTensorFlow.jsTensorFlow.js ArchitectureRunning Pretrained Models Using TensorFlow.jsModel Conversion for the BrowserTraining in the BrowserFeature ExtractionData CollectionTrainingGPU Utilizationml5.jsPoseNetpix2pixBenchmarking and Practical ConsiderationsModel SizeInference TimeCase StudiesSemi-ConductorTensorSpaceMetacarAirbnb’s Photo ClassificationGAN LabSummary
The Development Life Cycle for Artificial Intelligence on MobileA Brief History of Core MLAlternatives to Core MLTensorFlow LiteML KitFritzApple’s Machine Learning ArchitectureDomain-Based FrameworksML FrameworkML Performance PrimitivesBuilding a Real-Time Object Recognition AppConversion to Core MLConversion from KerasConversion from TensorFlowDynamic Model DeploymentOn-Device TrainingFederated LearningPerformance AnalysisBenchmarking Models on iPhonesMeasuring Energy ImpactBenchmarking LoadReducing App SizeAvoid Bundling the ModelUse QuantizationUse Create MLCase StudiesMagic SudokuSeeing AIHomeCourtInstaSaber + YoPuppetSummary
Collecting DataApproach 1: Find or Collect a DatasetApproach 2: Fatkun Chrome Browser ExtensionApproach 3: Web Scraper Using Bing Image Search APITraining Our ModelApproach 1: Use Web UI-based ToolsApproach 2: Use Create MLApproach 3: Fine Tuning Using KerasModel Conversion Using Core ML ToolsBuilding the iOS AppFurther ExplorationSummary
The Life Cycle of a Food Classifier AppAn Overview of TensorFlow LiteTensorFlow Lite ArchitectureModel Conversion to TensorFlow LiteBuilding a Real-Time Object Recognition AppML Kit + FirebaseObject Classification in ML KitCustom Models in ML KitHosted ModelsA/B Testing Hosted ModelsUsing the Experiment in CodeTensorFlow Lite on iOSPerformance OptimizationsQuantizing with TensorFlow Lite ConverterTensorFlow Model Optimization ToolkitFritzA Holistic Look at the Mobile AI App Development CycleHow Do I Collect Initial Data?How Do I Label My Data?How Do I Train My Model?How Do I Convert the Model to a Mobile-Friendly Format?How Do I Make my Model Performant?How Do I Build a Great UX for My Users?How Do I Make the Model Available to My Users?How Do I Measure the Success of My Model?How Do I Improve My Model?How Do I Update the Model on My Users’ Phones?The Self-Evolving ModelCase StudiesLose It!Portrait Mode on Pixel 3 PhonesSpeaker Recognition by AlibabaFace Contours in ML KitReal-Time Video Segmentation in YouTube StoriesSummary
Types of Computer-Vision TasksClassificationLocalizationDetectionSegmentationApproaches to Object DetectionInvoking Prebuilt Cloud-Based Object Detection APIsReusing a Pretrained ModelObtaining the ModelTest Driving Our ModelDeploying to a DeviceBuilding a Custom Detector Without Any CodeThe Evolution of Object DetectionPerformance ConsiderationsKey Terms in Object DetectionIntersection over UnionMean Average PrecisionNon-Maximum SuppressionUsing the TensorFlow Object Detection API to Build Custom ModelsData CollectionLabeling the DataPreprocessing the DataInspecting the ModelTrainingModel ConversionImage SegmentationCase StudiesSmart RefrigeratorCrowd CountingFace Detection in Seeing AIAutonomous CarsSummary
Exploring the Landscape of Embedded AI DevicesRaspberry PiIntel Movidius Neural Compute StickGoogle Coral USB AcceleratorNVIDIA Jetson NanoFPGA + PYNQArduinoA Qualitative Comparison of Embedded AI DevicesHands-On with the Raspberry PiSpeeding Up with the Google Coral USB AcceleratorPort to NVIDIA Jetson NanoComparing the Performance of Edge DevicesCase StudiesJetBotSquatting for Metro TicketsCucumber SorterFurther ExplorationSummary
A Brief Introduction to Reinforcement LearningWhy Learn Reinforcement Learning with an Autonomous Car?Practical Deep Reinforcement Learning with DeepRacerBuilding Our First Reinforcement LearningStep 1: Create ModelStep 2: Configure TrainingStep 3: Model TrainingStep 4: Evaluating the Performance of the ModelReinforcement Learning in ActionHow Does a Reinforcement Learning System Learn?Reinforcement Learning TheoryReinforcement Learning Algorithm in AWS DeepRacerDeep Reinforcement Learning Summary with DeepRacer as an ExampleStep 5: Improving Reinforcement Learning ModelsRacing the AWS DeepRacer CarBuilding the TrackAWS DeepRacer Single-Turn Track TemplateRunning the Model on AWS DeepRacerDriving the AWS DeepRacer Vehicle AutonomouslyFurther ExplorationDeepRacer LeagueAdvanced AWS DeepRacerAI Driving OlympicsDIY RobocarsRoboraceSummary
Machine LearningPerceptronActivation FunctionsNeural NetworksBackpropagationShortcoming of Neural NetworksDesired Properties of an Image ClassifierConvolutionPoolingStructure of a CNNFurther Exploration

Content preview from Practical Deep Learning for Cloud, Mobile, and Edge

Chapter 11. Real-Time Object Classification on iOS with Core ML

So far, we have seen our deep learning models running on the desktop, the cloud, and the browser. Although there are definite upsides to such a setup, it might not be ideal for all scenarios. In this chapter, we explore making predictions using deep learning models on mobile devices.

Bringing the computation closer to the user’s device, rather than a distant remote server, can be advantageous for many reasons:

Latency and interactivity

Sending an image, processing it in the cloud, and returning the result can take several seconds depending on the network quality and quantity of data being transferred. This can make for a poor UX. Decades of UX research, including Jakob Nielsen’s findings in 1993, published in his book Usability Engineering (Elsevier), showed the following:

0.1 second is about the limit for having the user feel that the system is reacting instantaneously.
1 second is about the limit for the user’s flow of thought to stay uninterrupted.
10 seconds is about the limit for keeping the user’s attention focused.

About two decades later, Google published findings that half of all mobile browser users abandon a web page if it takes longer than three seconds to load. Forget three seconds, even a 100 ms increase in latency would result in a 1% decrease in sales for Amazon. That is a lot of lost revenue. Mitigating this by processing on the device instantaneously can make for rich and interactive UXs. ...