I followed a solution in this answer to write a video file into a CMSampleBuffer. I want to do trajectory recognition on a given video file. Therefore VNDetectTrajectoriesRequest requires CMSampleBuffer to be timestamped. But I have no clue how to achieve that.
The code I use looks like this:
guard let path = Bundle.main.path(forResource: "IMG_9900", ofType: "MOV") else {
fatalError("no movie found")
}
let url = URL(fileURLWithPath: path)
let asset :AVAsset = AVAsset(url: url)
let reader = AVAssetReader(asset: asset)
guard let track = asset.tracks(withMediaType: .video).last else {
return
}
let trackOutput = AVAssetReaderTrackOutput(track: track, outputSettings: nil)
reader.add(trackOutput)
reader.startReading()
// Get first sample buffer
var sample = trackOutput.copyNextSampleBuffer()
while sample != nil {
print("1",sample)
sample = trackOutput.copyNextSampleBuffer()
let requestHandler = VNImageRequestHandler(cmSampleBuffer: sample!)
try requestHandler.perform([request])
print("2",sample)
}
Related
I want to try read the Data from a encrypted "video" file, decrypt the data and make it run on AVPlayer.
My app can download a video, the video comes encrypted from my API. All I have to do is: decrypt the file with a cypher and then it becomes a valid mp4 file.
Theses steps are covered. I did the process and did write the decrypted data in a file, played it on my machine and passed it with file URL to AVPlayer and it played as well.
However, I do not want have to save the decrypted file. I did a bit of research and came up with AVAssetResourceLoaderDelegate. Tried to implement it but without success. The examples I saw was hitting a online URL so I'm not entirely sure if its possible to do it local as I tried.
Any one can help me?
guard var components = URLComponents.init(url: video.url, resolvingAgainstBaseURL: false) else { return }
components.scheme = "encryptedVideo"
guard let url = components.url else { return }
let asset = AVURLAsset(url: fileDecryptedURL)
asset.resourceLoader.setDelegate(self, queue: DispatchQueue.global(qos: .background))
let playerItem = AVPlayerItem(asset: asset)
let player = AVPlayer(playerItem: playerItem)
player.actionAtItemEnd = .none
self.avPlayerViewController?.player = player
self.avPlayerViewController?.player?.play()
func resourceLoader(_ resourceLoader: AVAssetResourceLoader, shouldWaitForLoadingOfRequestedResource loadingRequest: AVAssetResourceLoadingRequest) -> Bool {
if let dataRequest = loadingRequest.dataRequest,
let url = loadingRequest.request.url,
let contentRequest = loadingRequest.contentInformationRequest {
guard var components = URLComponents.init(url: url, resolvingAgainstBaseURL: false) else { return false }
components.scheme = "file"
guard let localUrl = components.url else { return false }
let storageProvider = StorageVideo()
let dataMaybe = storageProvider.videoData(url: localUrl)
guard let encryptedData = dataMaybe,
let decryptedData = try? RNCryptor.decrypt(data: encryptedData,
withPassword: "omitted") else {
return false
}
contentRequest.contentType = AVFileType.mp4.rawValue
contentRequest.contentLength = Int64(decryptedData.count)
// contentRequest.isByteRangeAccessSupported = true
dataRequest.respond(with: decryptedData)
loadingRequest.finishLoading()
// Did this to save the file to see if it was ok. I could play the file in my machine
// If I pass this file url to a asset (above step) it load as well
// if let documentDirectory = FileManager.default.urls(for: .documentDirectory, in: .userDomainMask).first {
// let fileDecryptedURL = documentDirectory.appendingPathComponent("test").appendingPathExtension("mp4")
// try? decryptedData.write(to: fileDecryptedURL)
// }
}
return true
}
I have an app that saves a recorded video to the documents directory, as well as a Post object, and populates a collection view from the Post object. However upon restarting the app, the collection view is empty, so the videos being saved to the docs directory is not persisting (at least I think that's the problem).
This is the function that saves the video:
func imagePickerController(_ picker: UIImagePickerController, didFinishPickingMediaWithInfo info: [String : Any]) {
let mediaType = info[UIImagePickerControllerMediaType] as! NSString
dismiss(animated: true, completion: nil)
if mediaType == kUTTypeMovie {
var uniqueVideoID = ""
var videoURL:NSURL? = NSURL()
var uniqueID = ""
uniqueID = NSUUID().uuidString
// Get the path as URL and store the data in myVideoVarData
videoURL = info[UIImagePickerControllerMediaURL] as? URL as NSURL?
let myVideoVarData = try! Data(contentsOf: videoURL! as URL)
// Write data to temp diroctory
let tempPath = NSSearchPathForDirectoriesInDomains(FileManager.SearchPathDirectory.documentDirectory, FileManager.SearchPathDomainMask.userDomainMask, true)
let tempDocumentsDirectory: AnyObject = tempPath[0] as AnyObject
uniqueVideoID = uniqueID + "TEMPVIDEO.MOV"
let tempDataPath = tempDocumentsDirectory.appendingPathComponent(uniqueVideoID) as String
try? myVideoVarData.write(to: URL(fileURLWithPath: tempDataPath), options: [])
// Get the time value of the video
let fileURL = URL(fileURLWithPath: tempDataPath)
let asset = AVAsset(url: fileURL)
let duration : CMTime = asset.duration
// Remove the data from the temp Document Diroctory.
do{
let fileManager = FileManager.default
try fileManager.removeItem(atPath: tempDataPath)
} catch {
//Do nothing
}
// Check to see if video is under the 18500 (:30 seconds)
if duration.value <= 18500 {
// Write the data to the Document Directory
let docPaths = NSSearchPathForDirectoriesInDomains(FileManager.SearchPathDirectory.documentDirectory, FileManager.SearchPathDomainMask.userDomainMask, true)
let documentsDirectory: AnyObject = docPaths[0] as AnyObject
uniqueVideoID = uniqueID + "VIDEO.MOV"
let docDataPath = documentsDirectory.appendingPathComponent(uniqueVideoID) as String
try? myVideoVarData.write(to: URL(fileURLWithPath: docDataPath), options: [])
print("docDataPath under picker ",docDataPath)
print("Video saved to documents directory")
//Create a thumbnail image from the video
let assetImageGenerate = AVAssetImageGenerator(asset: asset)
assetImageGenerate.appliesPreferredTrackTransform = true
let time = CMTimeMake(asset.duration.value / 3, asset.duration.timescale)
if let videoImage = try? assetImageGenerate.copyCGImage(at: time, actualTime: nil) {
//Add thumbnail & video path to Post object
let video = Post(pathToVideo: URL(fileURLWithPath: docDataPath), thumbnail: UIImage(cgImage: videoImage))
posts.append(video)
print("Video saved to Post object")
}
}else{
print("Video not saved")
}
}
}
Specifically, this is where the video path and thumbnail are added to my object:
//Add thumbnail & video path to Post object
if let videoImage = try? assetImageGenerate.copyCGImage(at: time, actualTime: nil) {
let video = Post(pathToVideo: URL(fileURLWithPath: docDataPath), thumbnail: UIImage(cgImage: videoImage))
posts.append(video)
So I do give it the path to the video in the documents directory; how can I ensure that the data persists there?
EDIT:
To verify if the videos are being saved on the device, connect the device to Xcode and navigate to Window->Devices in Xcode. Then select your device on the left and find your app in the Installed Apps list. Select your app and click on the gear icon at the bottom of the list and press 'Show Containter'. Wait for a few seconds and you should see all the folders in your app.
Secondly, not sure why you are writing the video and deleting it and writing it back again and also why use 'try?' instead of actually catching any exceptions thrown during the file write?
I'm trying to use AVAssetExportSession on video selected in share extension and getting
Error Domain=NSURLErrorDomain Code=-3000 "Cannot create file"
UserInfo={NSLocalizedDescription=Cannot create file,
NSUnderlyingError=0x14811fdb0 {Error Domain=NSOSStatusErrorDomain
Code=-12124 "(null)"}}
But I can create file manually at the same NSURL without an error. Here is a function I'm using
func reencodeVideo() {
let videoAsset = AVURLAsset(URL: video.url)
let videoTrack = videoAsset.tracksWithMediaType(AVMediaTypeVideo)[0] as AVAssetTrack
print(videoTrack.estimatedDataRate)
let exportSession = AVAssetExportSession(asset: videoAsset, presetName: AVAssetExportPreset1920x1080)
guard let outputURL = uploadableFileURL else {
return
}
let fileManager = NSFileManager.defaultManager()
// let created = fileManager.createFileAtPath(outputURL.path!, contents: nil, attributes: nil)
if let path = outputURL.path where fileManager.fileExistsAtPath(path) {
print("file exists")
}
do {
try fileManager.removeItemAtURL(outputURL)
print("deleted")
} catch {
print(error)
}
exportSession?.outputURL = outputURL
exportSession?.outputFileType = AVFileTypeQuickTimeMovie
exportSession?.exportAsynchronouslyWithCompletionHandler{
print(exportSession?.status)
}
}
private var uploadableFileURL: NSURL? {
guard let tempFileName = video.url.lastPathComponent else {
return nil
}
let fileManager = NSFileManager.defaultManager()
guard let containerURL = fileManager.containerURLForSecurityApplicationGroupIdentifier(Constants.appGroupIdentifier) else {
return nil
}
return containerURL.URLByAppendingPathComponent("videoFile.mov")
}
I've successfully created file in the same directory, but AVAssetExportSession returns an error there.
Any ideas what I'm doing wrong?
I've tried using AVAssetReader and AVAssetWriter, and AVAssetWriter returns same error when trying to start. Encode process completes successfully if I'm using Documents directory and fails only when using shared app group container.
You're issue might be related to the use of the document folder and icloud sync.
See https://forums.developer.apple.com/message/77495#77495
If you do something like :
guard let containerURL = fileManager.containerURLForSecurityApplicationGroupIdentifier(Constants.appGroupIdentifier) else {
return nil
}
let libraryURL = containerURL.URLByAppendingPathComponent("Library", isDirectory: true)
let cachesURL = libraryURL.URLByAppendingPathComponent("Caches", isDirectory: true)
return cachesURL.URLByAppendingPathComponent("videoFile.mov")
I can't figure out from your code where your uploadableFileURL comes from, but the following works for me:
if let video = videoAsset {
let fm = NSFileManager.defaultManager().URLsForDirectory(.CachesDirectory, inDomains: .UserDomainMask)
let url = fm[0].URLByAppendingPathComponent("\(NSDate())").URLByAppendingPathExtension("mp4")
let exporter = AVAssetExportSession(asset: video, presetName: AVAssetExportPreset3840x2160)
exporter?.outputURL = url
exporter?.outputFileType = AVFileTypeMPEG4
exporter?.exportAsynchronouslyWithCompletionHandler() { _ in
let data = NSData(contentsOfURL: url)
}
}
I want to assing thumbnails of videos to uiimageviews by reading the videos names and create a thumbnail based on these names.
Here is my code:
let fileManager = NSFileManager.defaultManager()
// We need just to get the documents folder url
let documentsUrl = fileManager.URLsForDirectory(.DocumentDirectory, inDomains: .UserDomainMask)[0] as NSURL
do { let directoryUrls = try NSFileManager.defaultManager().contentsOfDirectoryAtURL(documentsUrl, includingPropertiesForKeys: nil, options: NSDirectoryEnumerationOptions.SkipsSubdirectoryDescendants)
} catch {
let directoryUrls = documentsUrl
}
let video = AVAsset(URL: directoryUrls)
let AVThumbnailAsset = AVAssetImageGenerator(asset: video)
let time = CMTimeMakeWithSeconds(1.0, 1)
var actualTime : CMTime = CMTimeMake(0, 0)
let myImage: CGImage
let convertedImage: UIImage
do{
myImage = try AVThumbnailAsset.copyCGImageAtTime(time, actualTime: &actualTime)
convertedImage = UIImage.init(CGImage: myImage)
} catch {
print("This is the Error")
print(error)
convertedImage = UIImage(named: "2.35:1")!
}
videoThumb.image = convertedImage
What am I doing wrong?
If I have a MPMoviePlayerController in Swift:
MPMoviePlayerController mp = MPMoviePlayerController(contentURL: url)
Is there a way I can get the number of frames within the video located at url? If not, is there some other way to determine the frame count?
I don't think MPMoviePlayerController can help you.
Use an AVAssetReader and count the number of CMSampleBuffers it returns to you. You can configure it to not even decode the frames, effectively parsing the file, so it should be fast and memory efficient.
Something like
var asset = AVURLAsset(URL: url, options: nil)
var reader = AVAssetReader(asset: asset, error: nil)
var videoTrack = asset.tracksWithMediaType(AVMediaTypeVideo)[0] as! AVAssetTrack
var readerOutput = AVAssetReaderTrackOutput(track: videoTrack, outputSettings: nil) // NB: nil, should give you raw frames
reader.addOutput(readerOutput)
reader.startReading()
var nFrames = 0
while true {
var sampleBuffer = readerOutput.copyNextSampleBuffer()
if sampleBuffer == nil {
break
}
nFrames++
}
println("Num frames: \(nFrames)")
Sorry if that's not idiomatic, I don't know swift.
Swift 5
func getNumberOfFrames(url: URL) -> Int {
let asset = AVURLAsset(url: url, options: nil)
do {
let reader = try AVAssetReader(asset: asset)
//AVAssetReader(asset: asset, error: nil)
let videoTrack = asset.tracks(withMediaType: AVMediaType.video)[0]
let readerOutput = AVAssetReaderTrackOutput(track: videoTrack, outputSettings: nil) // NB: nil, should give you raw frames
reader.add(readerOutput)
reader.startReading()
var nFrames = 0
while true {
let sampleBuffer = readerOutput.copyNextSampleBuffer()
if sampleBuffer == nil {
break
}
nFrames = nFrames+1
}
print("Num frames: \(nFrames)")
return nFrames
}catch {
print("Error: \(error)")
}
return 0
}
You could also use frames per second to calculate total frames.
var player: AVPlayer?
var playerController = AVPlayerViewController()
var videoFPS: Int = 0
var totalFrames: Int?
guard let videoURL = "" else { return }
player = AVPlayer(url: videoURL)
playerController.player = player
guard player?.currentItem?.asset != nil else {
return
}
let asset = self.player?.currentItem?.asset
let tracks = asset!.tracks(withMediaType: .video)
let fps = tracks.first?.nominalFrameRate
let duration = self.player?.currentItem?.duration
self.videoFPS = lround(Double(fps!))
self.totalFrames = lround(Double(self!.videoFPS) * durationSeconds)