Flutter realtime object detection with Tensorflow Lite
An app made with Flutter and TensorFlow Lite for realtime object detection using model YOLO, SSD, MobileNet, PoseNet.
-
Realtime object detection on the live camera
-
Using Model: YOLOv2-Tiny, SSDMobileNet, MobileNet, PoseNet
-
Save image has been detected
-
MVVM architecture
- Install Packages
camera: get the streaming image buffers https://pub.dev/packages/camera tflite: run model TensorFlow Lite https://pub.dev/packages/tflite provider: state management https://pub.dev/packages/provider 2. Configure Project
- Android
android/app/build.gradle android { ... aaptOptions { noCompress 'tflite' noCompress 'lite' } ... } minSdkVersion 21 3. Load model
loadModel() async { Tflite.close(); await Tflite.loadModel( model: "assets/models/yolov2_tiny.tflite", //ssd_mobilenet.tflite, mobilenet_v1.tflite, posenet_mv1_checkpoints.tflite labels: "assets/models/yolov2_tiny.txt", //ssd_mobilenet.txt, mobilenet_v1.txt //numThreads: 1, // defaults to 1 //isAsset: true, // defaults: true, set to false to load resources outside assets //useGpuDelegate: false // defaults: false, use GPU delegate ); } 4. Run model
For Realtime Camera
//YOLOv2-Tiny Future<List<dynamic>?> runModelOnFrame(CameraImage image) async { var recognitions = await Tflite.detectObjectOnFrame( bytesList: image.planes.map((plane) { return plane.bytes; }).toList(), model: "YOLO", imageHeight: image.height, imageWidth: image.width, imageMean: 0, // defaults to 127.5 imageStd: 255.0, // defaults to 127.5 threshold: 0.2, // defaults to 0.1 numResultsPerClass: 1, ); return recognitions; } //SSDMobileNet Future<List<dynamic>?> runModelOnFrame(CameraImage image) async { var recognitions = await Tflite.detectObjectOnFrame( bytesList: image.planes.map((plane) { return plane.bytes; }).toList(), model: "SSDMobileNet", imageHeight: image.height, imageWidth: image.width, imageMean: 127.5, imageStd: 127.5, threshold: 0.4, numResultsPerClass: 1, ); return recognitions; } //MobileNet Future<List<dynamic>?> runModelOnFrame(CameraImage image) async { var recognitions = await Tflite.runModelOnFrame( bytesList: image.planes.map((plane) { return plane.bytes; }).toList(), imageHeight: image.height, imageWidth: image.width, numResults: 5 ); return recognitions; } //PoseNet Future<List<dynamic>?> runModelOnFrame(CameraImage image) async { var recognitions = await Tflite.runPoseNetOnFrame( bytesList: image.planes.map((plane) { return plane.bytes; }).toList(), imageHeight: image.height, imageWidth: image.width, numResults: 5 ); return recognitions; } For Image
Future<List<dynamic>?> runModelOnImage(File image) async { var recognitions = await Tflite.detectObjectOnImage( path: image.path, model: "YOLO", threshold: 0.3, imageMean: 0.0, imageStd: 127.5, numResultsPerClass: 1 ); return recognitions; } Output format: YOLO,SSDMobileNet [{ detectedClass: "dog", confidenceInClass: 0.989, rect: { x: 0.0, y: 0.0, w: 100.0, h: 100.0 } },...] MobileNet [{ index: 0, label: "WithMask", confidence: 0.989 },...] PoseNet [{ score: 0.5, keypoints: { 0: { x: 0.2, y: 0.12, part: nose, score: 0.803 }, 1: { x: 0.2, y: 0.1, part: leftEye, score: 0.8666 }, ... } },...] 5. Issue
* IOS Downgrading TensorFlowLiteC to 2.2.0 Downgrade your TensorFlowLiteC in /ios/Podfile.lock to 2.2.0 run pod install in your /ios folder 6. Source code
please checkout repo github https://github.com/hiennguyen92/flutter_realtime_object_detection - Demo Illustration: https://www.youtube.com/watch?v=__i7PRmz5kY&ab_channel=HienNguyen
- Image
 |  |
 |  |
 |  |