- Notifications
You must be signed in to change notification settings - Fork 82
Description
Hi!
I'm trying to use the AVAssetImageGenerator.generateCGImagesAsynchronouslyForTimes_completionHandler via ffigen to extract video frames (e.g. the thumbnail). For some reason, when the completion handler is called, in some situations (not always!) it seems like the CGImageRef is already released when the listener is called. Here's a shortened piece of code that shows the usage from Dart:
Future<List<ui.Image>> _extractFramesAt( VideoSource source, { required List<Duration> timestamps, ui.Size? preferredSize, }) async { return using<Future<List<ui.Image>>>((alloc) async { // Prepare the asset image generator final avAsset = videoSourceToAVAsset(source); final generator = AVAssetImageGenerator.assetImageGeneratorWithAsset_(avAsset); generator.appliesPreferredTrackTransform = true; // Omitted... // This closure is called every time an image is generated final closure = ObjCBlock_ffiVoid_CMTime_CGImage_CMTime_AVAssetImageGeneratorResult_NSError.listener( (requestedTime, cgImagePointer, actualTime, result, error) { if (error != null) { imageStream.addError(error); return; } if (cgImagePointer == nullptr) { imageStream.addError(Exception('No image generated')); return; } // Paint the image onto a CGBitmap final width = darwinLib.CGImageGetWidth(cgImagePointer); final height = darwinLib.CGImageGetHeight(cgImagePointer); final colorSpace = darwinLib.CGColorSpaceCreateDeviceRGB(); if (width == 0 || height == 0) { // <- This gets called often with both width and height equal to 0. imageStream.addError(Exception('Invalid image size: $width x $height')); return; } // Omitted... }, ); // Run the generator. The closure will be called for each timestamp generator.generateCGImagesAsynchronouslyForTimes_completionHandler_( cmTimeArray, closure, ); // Omitted... }); }From what I can understand, somehow the CGImageRef is getting released too early, before the listener is called. I was able to consistently replicate this on my phone. Also, I was able to fix this in the generated code by adding an explicit CGImageRetain call to the listener block:
// Before typedef void (^ListenerBlock51)(CMTime , struct CGImage * , CMTime , AVAssetImageGeneratorResult , NSError* ); ListenerBlock51 wrapListenerBlock_ObjCBlock_ffiVoid_CMTime_CGImage_CMTime_AVAssetImageGeneratorResult_NSError(ListenerBlock51 block) NS_RETURNS_RETAINED { return ^void(CMTime arg0, struct CGImage * arg1, CMTime arg2, AVAssetImageGeneratorResult arg3, NSError* arg4) { block(arg0, arg1, arg2, arg3, objc_retain(arg4)); }; } // After typedef void (^ListenerBlock51)(CMTime , struct CGImage * , CMTime , AVAssetImageGeneratorResult , NSError* ); ListenerBlock51 wrapListenerBlock_ObjCBlock_ffiVoid_CMTime_CGImage_CMTime_AVAssetImageGeneratorResult_NSError(ListenerBlock51 block) NS_RETURNS_RETAINED { return ^void(CMTime arg0, struct CGImage * arg1, CMTime arg2, AVAssetImageGeneratorResult arg3, NSError* arg4) { CGImageRetain(arg1); // <- Added this block(arg0, arg1, arg2, arg3, objc_retain(arg4)); }; } and then releasing the CGImage on the Dart side. This seems to fix the issue, but I'm not sure whether it should be that way.
If needed, I can provide an example project which should be able to reproduce this issue. I'm using ffigen/objc from #1463
Metadata
Metadata
Assignees
Labels
Type
Projects
Status