book

Practical Deep Learning for Cloud, Mobile, and Edge

by Anirudh Koul, Siddha Ganju, Meher Kasam

October 2019

Intermediate to advanced

583 pages

14h 36m

English

O'Reilly Media, Inc.

Read now

Unlock full access

To the Backend/Frontend/Mobile Software DeveloperTo the Data ScientistTo the StudentTo the TeacherTo the Robotics EnthusiastWhat to Expect in Each ChapterConventions Used in This BookUsing Code ExamplesO’Reilly Online LearningHow to Contact UsAcknowledgmentsGroup AcknowledgmentsPersonal Acknowledgments
An ApologyThe Real IntroductionWhat Is AI?Motivating ExamplesA Brief History of AIExciting BeginningsThe Cold and Dark DaysA Glimmer of HopeHow Deep Learning Became a ThingRecipe for the Perfect Deep Learning SolutionDatasetsModel ArchitectureFrameworksHardwareResponsible AIBiasAccountability and ExplainabilityReproducibilityRobustnessPrivacySummaryFrequently Asked Questions
Introducing KerasPredicting an Image’s CategoryInvestigating the ModelImageNet DatasetModel ZoosClass Activation MapsSummary
Adapting Pretrained Models to New TasksA Shallow Dive into Convolutional Neural NetworksTransfer LearningFine TuningHow Much to Fine TuneBuilding a Custom Classifier in Keras with Transfer LearningOrganize the DataBuild the Data PipelineNumber of ClassesBatch SizeData AugmentationModel DefinitionTrain the ModelSet Training ParametersStart TrainingTest the ModelAnalyzing the ResultsFurther ReadingSummary
Image SimilarityFeature ExtractionSimilarity SearchVisualizing Image Clusters with t-SNEImproving the Speed of Similarity SearchLength of Feature VectorsReducing Feature-Length with PCAScaling Similarity Search with Approximate Nearest NeighborsApproximate Nearest-Neighbor BenchmarkWhich Library Should I Use?Creating a Synthetic DatasetBrute ForceAnnoyNGTFaissImproving Accuracy with Fine TuningFine Tuning Without Fully Connected LayersSiamese Networks for One-Shot Face VerificationCase StudiesFlickrPinterestCelebrity DoppelgangersSpotifyImage CaptioningSummary
Tools of the TradeTensorFlow DatasetsTensorBoardWhat-If Tooltf-explainCommon Techniques for Machine Learning ExperimentationData InspectionBreaking the Data: Train, Validation, TestEarly StoppingReproducible ExperimentsEnd-to-End Deep Learning Example PipelineBasic Transfer Learning PipelineBasic Custom Network PipelineHow Hyperparameters Affect AccuracyTransfer Learning Versus Training from ScratchEffect of Number of Layers Fine-Tuned in Transfer LearningEffect of Data Size on Transfer LearningEffect of Learning RateEffect of OptimizersEffect of Batch SizeEffect of ResizingEffect of Change in Aspect Ratio on Transfer LearningTools to Automate Tuning for Maximum AccuracyKeras TunerAutoAugmentAutoKerasSummary
GPU Starvationnvidia-smiTensorFlow Profiler + TensorBoardHow to Use This ChecklistPerformance ChecklistData PreparationData ReadingData AugmentationTrainingInferenceData PreparationStore as TFRecordsReduce Size of Input DataUse TensorFlow DatasetsData ReadingUse tf.dataPrefetch DataParallelize CPU ProcessingParallelize I/O and ProcessingEnable Nondeterministic OrderingCache DataTurn on Experimental OptimizationsAutotune Parameter ValuesData AugmentationUse GPU for AugmentationTrainingUse Automatic Mixed PrecisionUse Larger Batch SizeUse Multiples of EightFind the Optimal Learning RateUse tf.functionOvertrain, and Then GeneralizeInstall an Optimized Stack for the HardwareOptimize the Number of Parallel CPU ThreadsUse Better HardwareDistribute TrainingExamine Industry BenchmarksInferenceUse an Efficient ModelQuantize the ModelPrune the ModelUse Fused OperationsEnable GPU PersistenceSummary
InstallationTrainingModelDataPrivacyEducation and ExplorationOne Last Question
The Landscape of Visual Recognition APIsClarifaiMicrosoft Cognitive ServicesGoogle Cloud VisionAmazon RekognitionIBM Watson Visual RecognitionAlgorithmiaComparing Visual Recognition APIsService OfferingsCostAccuracyBiasGetting Up and Running with Cloud APIsTraining Our Own Custom ClassifierTop Reasons Why Our Classifier Does Not Work SatisfactorilyComparing Custom Classification APIsPerformance Tuning for Cloud APIsEffect of Resizing on Image Labeling APIsEffect of Compression on Image Labeling APIsEffect of Compression on OCR APIsEffect of Resizing on OCR APIsCase StudiesThe New York TimesUberGiphyOmniEarthPhotobucketStaplesInDro RoboticsSummary
Landscape of Serving AI PredictionsFlask: Build Your Own ServerMaking a REST API with FlaskDeploying a Keras Model to FlaskPros of Using FlaskCons of Using FlaskDesirable Qualities in a Production-Level Serving SystemHigh AvailabilityScalabilityLow LatencyGeographic AvailabilityFailure HandlingMonitoringModel VersioningA/B TestingSupport for Multiple Machine Learning LibrariesGoogle Cloud ML Engine: A Managed Cloud AI Serving StackPros of Using Cloud ML EngineCons of Using Cloud ML EngineBuilding a Classification APITensorFlow ServingInstallationKubeFlowPipelinesFairingInstallationPrice Versus Performance ConsiderationsCost Analysis of Inference-as-a-ServiceCost Analysis of Building Your Own StackSummary

JavaScript-Based Machine Learning Libraries: A Brief HistoryConvNetJSKeras.jsONNX.jsTensorFlow.jsTensorFlow.js ArchitectureRunning Pretrained Models Using TensorFlow.jsModel Conversion for the BrowserTraining in the BrowserFeature ExtractionData CollectionTrainingGPU Utilizationml5.jsPoseNetpix2pixBenchmarking and Practical ConsiderationsModel SizeInference TimeCase StudiesSemi-ConductorTensorSpaceMetacarAirbnb’s Photo ClassificationGAN LabSummary
The Development Life Cycle for Artificial Intelligence on MobileA Brief History of Core MLAlternatives to Core MLTensorFlow LiteML KitFritzApple’s Machine Learning ArchitectureDomain-Based FrameworksML FrameworkML Performance PrimitivesBuilding a Real-Time Object Recognition AppConversion to Core MLConversion from KerasConversion from TensorFlowDynamic Model DeploymentOn-Device TrainingFederated LearningPerformance AnalysisBenchmarking Models on iPhonesMeasuring Energy ImpactBenchmarking LoadReducing App SizeAvoid Bundling the ModelUse QuantizationUse Create MLCase StudiesMagic SudokuSeeing AIHomeCourtInstaSaber + YoPuppetSummary
Collecting DataApproach 1: Find or Collect a DatasetApproach 2: Fatkun Chrome Browser ExtensionApproach 3: Web Scraper Using Bing Image Search APITraining Our ModelApproach 1: Use Web UI-based ToolsApproach 2: Use Create MLApproach 3: Fine Tuning Using KerasModel Conversion Using Core ML ToolsBuilding the iOS AppFurther ExplorationSummary
The Life Cycle of a Food Classifier AppAn Overview of TensorFlow LiteTensorFlow Lite ArchitectureModel Conversion to TensorFlow LiteBuilding a Real-Time Object Recognition AppML Kit + FirebaseObject Classification in ML KitCustom Models in ML KitHosted ModelsA/B Testing Hosted ModelsUsing the Experiment in CodeTensorFlow Lite on iOSPerformance OptimizationsQuantizing with TensorFlow Lite ConverterTensorFlow Model Optimization ToolkitFritzA Holistic Look at the Mobile AI App Development CycleHow Do I Collect Initial Data?How Do I Label My Data?How Do I Train My Model?How Do I Convert the Model to a Mobile-Friendly Format?How Do I Make my Model Performant?How Do I Build a Great UX for My Users?How Do I Make the Model Available to My Users?How Do I Measure the Success of My Model?How Do I Improve My Model?How Do I Update the Model on My Users’ Phones?The Self-Evolving ModelCase StudiesLose It!Portrait Mode on Pixel 3 PhonesSpeaker Recognition by AlibabaFace Contours in ML KitReal-Time Video Segmentation in YouTube StoriesSummary
Types of Computer-Vision TasksClassificationLocalizationDetectionSegmentationApproaches to Object DetectionInvoking Prebuilt Cloud-Based Object Detection APIsReusing a Pretrained ModelObtaining the ModelTest Driving Our ModelDeploying to a DeviceBuilding a Custom Detector Without Any CodeThe Evolution of Object DetectionPerformance ConsiderationsKey Terms in Object DetectionIntersection over UnionMean Average PrecisionNon-Maximum SuppressionUsing the TensorFlow Object Detection API to Build Custom ModelsData CollectionLabeling the DataPreprocessing the DataInspecting the ModelTrainingModel ConversionImage SegmentationCase StudiesSmart RefrigeratorCrowd CountingFace Detection in Seeing AIAutonomous CarsSummary
Exploring the Landscape of Embedded AI DevicesRaspberry PiIntel Movidius Neural Compute StickGoogle Coral USB AcceleratorNVIDIA Jetson NanoFPGA + PYNQArduinoA Qualitative Comparison of Embedded AI DevicesHands-On with the Raspberry PiSpeeding Up with the Google Coral USB AcceleratorPort to NVIDIA Jetson NanoComparing the Performance of Edge DevicesCase StudiesJetBotSquatting for Metro TicketsCucumber SorterFurther ExplorationSummary
A Brief Introduction to Reinforcement LearningWhy Learn Reinforcement Learning with an Autonomous Car?Practical Deep Reinforcement Learning with DeepRacerBuilding Our First Reinforcement LearningStep 1: Create ModelStep 2: Configure TrainingStep 3: Model TrainingStep 4: Evaluating the Performance of the ModelReinforcement Learning in ActionHow Does a Reinforcement Learning System Learn?Reinforcement Learning TheoryReinforcement Learning Algorithm in AWS DeepRacerDeep Reinforcement Learning Summary with DeepRacer as an ExampleStep 5: Improving Reinforcement Learning ModelsRacing the AWS DeepRacer CarBuilding the TrackAWS DeepRacer Single-Turn Track TemplateRunning the Model on AWS DeepRacerDriving the AWS DeepRacer Vehicle AutonomouslyFurther ExplorationDeepRacer LeagueAdvanced AWS DeepRacerAI Driving OlympicsDIY RobocarsRoboraceSummary
Machine LearningPerceptronActivation FunctionsNeural NetworksBackpropagationShortcoming of Neural NetworksDesired Properties of an Image ClassifierConvolutionPoolingStructure of a CNNFurther Exploration

Content preview from Practical Deep Learning for Cloud, Mobile, and Edge

Chapter 10. AI in the Browser with TensorFlow.js and ml5.js

Written in collaboration with guest author: Zaid Alyafeai

You’re a developer who dreams big. You have a kickass AI model that you would like a lot of people to try. How many is a lot? Ten thousand? A million? No, silly. You like to dream big. How about 100 million people? That’s a nice round number. Now convincing 100 million people to download and install an app and make space for it on their phones is not an easy sell. But what if we told you that they all have an app already installed, just for you. No downloads. No installs. No app stores. What kind of black magic is this!? Of course, it’s the web browser. And as a bonus, it also runs on your PC.

This is what Google did with its home page when it decided to launch its first-ever AI doodle to their billions of users (Figure 10-1). And what better theme to pick for it than the music of J.S. Bach. (Bach’s parents wanted to call him J.S. Bach, 310 years before JavaScript was even created. They had quite the foresight!)

To explain briefly, the doodle allowed anyone to write one line (voice) of random notes for two measures using mouse clicks. When the user clicked a button labeled Harmonize, the input would then be processed against hundreds of musical pieces written by Bach that contain between two and four lines (voices) of music. The system would figure out which notes would sound best along with the user’s input to create a much richer Bach-like sounding musical piece. ...