Extracting bitmap from AVPlayer is very uncertain - ios

Question

I want to extract a UIIMage from a video player at a certain time point and do some processing on that image.
I have this code:
pausePlayer()
let time = player!.currentTime()
imageFromVideo(url: fileURL(for: movieName!), at: time.seconds) { image in
let result = self.detectLines(image: image!
if let result = result {
// Display results by handing off to the InferenceViewController.
DispatchQueue.main.async {
self.drawNewResults(result: result)
}
}
and
public func imageFromVideo(url: URL, at time: TimeInterval, completion: #escaping (UIImage?) -> Void) {
DispatchQueue.global(qos: .background).async {
let asset = AVURLAsset(url: url)
let assetIG = AVAssetImageGenerator(asset: asset)
assetIG.appliesPreferredTrackTransform = true
assetIG.apertureMode = AVAssetImageGenerator.ApertureMode.encodedPixels
let cmTime = CMTime(seconds: time, preferredTimescale: CMTimeScale(30.0))
var newTime = CMTime(seconds: time, preferredTimescale: CMTimeScale(30.0))
let thumbnailImageRef: CGImage
do {
thumbnailImageRef = try assetIG.copyCGImage(at: cmTime, actualTime: &newTime)
} catch let error {
print("Error: \(error)")
return completion(nil)
}
print("Time on click: %f", cmTime.seconds)
print("Actual time: %f", newTime.seconds)
DispatchQueue.main.async {
completion(UIImage(cgImage: thumbnailImageRef))
}
}
}
The problem is that there is a huge gap between the time I want the image and the image I get. Typically it is something like 0.2s and 0.4s, roughly an offset between 7 14 frames. So the processing I do and show is just wrong since I am not working on the image that is observed.
Example:
Time on click: %f 0.8333333333333334
Actual time: %f 1.0666666666666667
Time on click: %f 1.6333333333333333
Actual time: %f 2.1666666666666665

Answer 1

As suggested here: swift: How to take screenshot of AVPlayerLayer()
assetIG.requestedTimeToleranceAfter = CMTime.zero
assetIG.requestedTimeToleranceBefore = CMTime.zero
did the trick.