book

Practical Deep Learning for Cloud, Mobile, and Edge

by Anirudh Koul, Siddha Ganju, Meher Kasam

October 2019

Intermediate to advanced

583 pages

14h 36m

English

O'Reilly Media, Inc.

Read now

Unlock full access

To the Backend/Frontend/Mobile Software DeveloperTo the Data ScientistTo the StudentTo the TeacherTo the Robotics EnthusiastWhat to Expect in Each ChapterConventions Used in This BookUsing Code ExamplesO’Reilly Online LearningHow to Contact UsAcknowledgmentsGroup AcknowledgmentsPersonal Acknowledgments
An ApologyThe Real IntroductionWhat Is AI?Motivating ExamplesA Brief History of AIExciting BeginningsThe Cold and Dark DaysA Glimmer of HopeHow Deep Learning Became a ThingRecipe for the Perfect Deep Learning SolutionDatasetsModel ArchitectureFrameworksHardwareResponsible AIBiasAccountability and ExplainabilityReproducibilityRobustnessPrivacySummaryFrequently Asked Questions
Introducing KerasPredicting an Image’s CategoryInvestigating the ModelImageNet DatasetModel ZoosClass Activation MapsSummary
Adapting Pretrained Models to New TasksA Shallow Dive into Convolutional Neural NetworksTransfer LearningFine TuningHow Much to Fine TuneBuilding a Custom Classifier in Keras with Transfer LearningOrganize the DataBuild the Data PipelineNumber of ClassesBatch SizeData AugmentationModel DefinitionTrain the ModelSet Training ParametersStart TrainingTest the ModelAnalyzing the ResultsFurther ReadingSummary
Image SimilarityFeature ExtractionSimilarity SearchVisualizing Image Clusters with t-SNEImproving the Speed of Similarity SearchLength of Feature VectorsReducing Feature-Length with PCAScaling Similarity Search with Approximate Nearest NeighborsApproximate Nearest-Neighbor BenchmarkWhich Library Should I Use?Creating a Synthetic DatasetBrute ForceAnnoyNGTFaissImproving Accuracy with Fine TuningFine Tuning Without Fully Connected LayersSiamese Networks for One-Shot Face VerificationCase StudiesFlickrPinterestCelebrity DoppelgangersSpotifyImage CaptioningSummary
Tools of the TradeTensorFlow DatasetsTensorBoardWhat-If Tooltf-explainCommon Techniques for Machine Learning ExperimentationData InspectionBreaking the Data: Train, Validation, TestEarly StoppingReproducible ExperimentsEnd-to-End Deep Learning Example PipelineBasic Transfer Learning PipelineBasic Custom Network PipelineHow Hyperparameters Affect AccuracyTransfer Learning Versus Training from ScratchEffect of Number of Layers Fine-Tuned in Transfer LearningEffect of Data Size on Transfer LearningEffect of Learning RateEffect of OptimizersEffect of Batch SizeEffect of ResizingEffect of Change in Aspect Ratio on Transfer LearningTools to Automate Tuning for Maximum AccuracyKeras TunerAutoAugmentAutoKerasSummary
GPU Starvationnvidia-smiTensorFlow Profiler + TensorBoardHow to Use This ChecklistPerformance ChecklistData PreparationData ReadingData AugmentationTrainingInferenceData PreparationStore as TFRecordsReduce Size of Input DataUse TensorFlow DatasetsData ReadingUse tf.dataPrefetch DataParallelize CPU ProcessingParallelize I/O and ProcessingEnable Nondeterministic OrderingCache DataTurn on Experimental OptimizationsAutotune Parameter ValuesData AugmentationUse GPU for AugmentationTrainingUse Automatic Mixed PrecisionUse Larger Batch SizeUse Multiples of EightFind the Optimal Learning RateUse tf.functionOvertrain, and Then GeneralizeInstall an Optimized Stack for the HardwareOptimize the Number of Parallel CPU ThreadsUse Better HardwareDistribute TrainingExamine Industry BenchmarksInferenceUse an Efficient ModelQuantize the ModelPrune the ModelUse Fused OperationsEnable GPU PersistenceSummary
InstallationTrainingModelDataPrivacyEducation and ExplorationOne Last Question
The Landscape of Visual Recognition APIsClarifaiMicrosoft Cognitive ServicesGoogle Cloud VisionAmazon RekognitionIBM Watson Visual RecognitionAlgorithmiaComparing Visual Recognition APIsService OfferingsCostAccuracyBiasGetting Up and Running with Cloud APIsTraining Our Own Custom ClassifierTop Reasons Why Our Classifier Does Not Work SatisfactorilyComparing Custom Classification APIsPerformance Tuning for Cloud APIsEffect of Resizing on Image Labeling APIsEffect of Compression on Image Labeling APIsEffect of Compression on OCR APIsEffect of Resizing on OCR APIsCase StudiesThe New York TimesUberGiphyOmniEarthPhotobucketStaplesInDro RoboticsSummary
Landscape of Serving AI PredictionsFlask: Build Your Own ServerMaking a REST API with FlaskDeploying a Keras Model to FlaskPros of Using FlaskCons of Using FlaskDesirable Qualities in a Production-Level Serving SystemHigh AvailabilityScalabilityLow LatencyGeographic AvailabilityFailure HandlingMonitoringModel VersioningA/B TestingSupport for Multiple Machine Learning LibrariesGoogle Cloud ML Engine: A Managed Cloud AI Serving StackPros of Using Cloud ML EngineCons of Using Cloud ML EngineBuilding a Classification APITensorFlow ServingInstallationKubeFlowPipelinesFairingInstallationPrice Versus Performance ConsiderationsCost Analysis of Inference-as-a-ServiceCost Analysis of Building Your Own StackSummary

JavaScript-Based Machine Learning Libraries: A Brief HistoryConvNetJSKeras.jsONNX.jsTensorFlow.jsTensorFlow.js ArchitectureRunning Pretrained Models Using TensorFlow.jsModel Conversion for the BrowserTraining in the BrowserFeature ExtractionData CollectionTrainingGPU Utilizationml5.jsPoseNetpix2pixBenchmarking and Practical ConsiderationsModel SizeInference TimeCase StudiesSemi-ConductorTensorSpaceMetacarAirbnb’s Photo ClassificationGAN LabSummary
The Development Life Cycle for Artificial Intelligence on MobileA Brief History of Core MLAlternatives to Core MLTensorFlow LiteML KitFritzApple’s Machine Learning ArchitectureDomain-Based FrameworksML FrameworkML Performance PrimitivesBuilding a Real-Time Object Recognition AppConversion to Core MLConversion from KerasConversion from TensorFlowDynamic Model DeploymentOn-Device TrainingFederated LearningPerformance AnalysisBenchmarking Models on iPhonesMeasuring Energy ImpactBenchmarking LoadReducing App SizeAvoid Bundling the ModelUse QuantizationUse Create MLCase StudiesMagic SudokuSeeing AIHomeCourtInstaSaber + YoPuppetSummary
Collecting DataApproach 1: Find or Collect a DatasetApproach 2: Fatkun Chrome Browser ExtensionApproach 3: Web Scraper Using Bing Image Search APITraining Our ModelApproach 1: Use Web UI-based ToolsApproach 2: Use Create MLApproach 3: Fine Tuning Using KerasModel Conversion Using Core ML ToolsBuilding the iOS AppFurther ExplorationSummary
The Life Cycle of a Food Classifier AppAn Overview of TensorFlow LiteTensorFlow Lite ArchitectureModel Conversion to TensorFlow LiteBuilding a Real-Time Object Recognition AppML Kit + FirebaseObject Classification in ML KitCustom Models in ML KitHosted ModelsA/B Testing Hosted ModelsUsing the Experiment in CodeTensorFlow Lite on iOSPerformance OptimizationsQuantizing with TensorFlow Lite ConverterTensorFlow Model Optimization ToolkitFritzA Holistic Look at the Mobile AI App Development CycleHow Do I Collect Initial Data?How Do I Label My Data?How Do I Train My Model?How Do I Convert the Model to a Mobile-Friendly Format?How Do I Make my Model Performant?How Do I Build a Great UX for My Users?How Do I Make the Model Available to My Users?How Do I Measure the Success of My Model?How Do I Improve My Model?How Do I Update the Model on My Users’ Phones?The Self-Evolving ModelCase StudiesLose It!Portrait Mode on Pixel 3 PhonesSpeaker Recognition by AlibabaFace Contours in ML KitReal-Time Video Segmentation in YouTube StoriesSummary
Types of Computer-Vision TasksClassificationLocalizationDetectionSegmentationApproaches to Object DetectionInvoking Prebuilt Cloud-Based Object Detection APIsReusing a Pretrained ModelObtaining the ModelTest Driving Our ModelDeploying to a DeviceBuilding a Custom Detector Without Any CodeThe Evolution of Object DetectionPerformance ConsiderationsKey Terms in Object DetectionIntersection over UnionMean Average PrecisionNon-Maximum SuppressionUsing the TensorFlow Object Detection API to Build Custom ModelsData CollectionLabeling the DataPreprocessing the DataInspecting the ModelTrainingModel ConversionImage SegmentationCase StudiesSmart RefrigeratorCrowd CountingFace Detection in Seeing AIAutonomous CarsSummary
Exploring the Landscape of Embedded AI DevicesRaspberry PiIntel Movidius Neural Compute StickGoogle Coral USB AcceleratorNVIDIA Jetson NanoFPGA + PYNQArduinoA Qualitative Comparison of Embedded AI DevicesHands-On with the Raspberry PiSpeeding Up with the Google Coral USB AcceleratorPort to NVIDIA Jetson NanoComparing the Performance of Edge DevicesCase StudiesJetBotSquatting for Metro TicketsCucumber SorterFurther ExplorationSummary
A Brief Introduction to Reinforcement LearningWhy Learn Reinforcement Learning with an Autonomous Car?Practical Deep Reinforcement Learning with DeepRacerBuilding Our First Reinforcement LearningStep 1: Create ModelStep 2: Configure TrainingStep 3: Model TrainingStep 4: Evaluating the Performance of the ModelReinforcement Learning in ActionHow Does a Reinforcement Learning System Learn?Reinforcement Learning TheoryReinforcement Learning Algorithm in AWS DeepRacerDeep Reinforcement Learning Summary with DeepRacer as an ExampleStep 5: Improving Reinforcement Learning ModelsRacing the AWS DeepRacer CarBuilding the TrackAWS DeepRacer Single-Turn Track TemplateRunning the Model on AWS DeepRacerDriving the AWS DeepRacer Vehicle AutonomouslyFurther ExplorationDeepRacer LeagueAdvanced AWS DeepRacerAI Driving OlympicsDIY RobocarsRoboraceSummary
Machine LearningPerceptronActivation FunctionsNeural NetworksBackpropagationShortcoming of Neural NetworksDesired Properties of an Image ClassifierConvolutionPoolingStructure of a CNNFurther Exploration

Content preview from Practical Deep Learning for Cloud, Mobile, and Edge

Chapter 8. Cloud APIs for Computer Vision: Up and Running in 15 Minutes

Due to repeated incidents of near meltdown at the nearby nuclear power plant, the library of the city of Springfield (we are not allowed to mention the state) decided that it was too risky to store all their valuable archives in physical form. After hearing that the library from their rival city of Shelbyville started digitizing their records, they wanted to get in on the game as well. After all, their collection of articles such as “Old man yells at cloud” and “Local man thinks wrestling is real” and the hundred-year-old iconic photographs of the Gorge and the statue of the city’s founder Jebediah Springfield are irreplaceable. In addition to making their archives resilient to catastrophes, they would make their archives easily searchable and retrievable. And, of course, the residents of Springfield now would be able to access all of this material from the comfort of their living room couches.

The first step in digitizing documents is, of course, scanning. That’s the easy part. Then starts the real challenge—processing and understanding all of this visual imagery. The team in Springfield had a few different options in front of them.

Perform manual data entry for every single page and every single photograph. Given that the city has more than 200 years of rich history, it would take a really long time, and would be error prone and expensive. It would be quite an ordeal to transcribe all of that material. ...