I am building an action extension for GarageBand on iOS which transforms and uploads audio but no matter what I try, I just could not get to the exported file.
Let’s consider the following code — it should:
find and load shared audio from extensionContext
initialise audio player
play the sound
It works if I run the extension in Voice Memos.app — the url to the file looks like this: file:///tmp/com.apple.VoiceMemos/New%20Recording%202.m4a
Now, If I run the code in GarageBand.app, the url points to (what I presume) is GarageBand’s app container, as the url looks something like /var/…/Containers/…/Project.band/audio/Project.m4a, and the audio will not be loaded and cannot therefore be manipulated in any way.
// edit: If I try to load contents of the audio file, it looks like the data only contains aac header (?) but the rest of the file is empty
What is interesting is this: The extension renders a react-native view and if I pass the view the fileUrl (/var/…Project.band/audio/Project.m4a) and then pass it down to XMLHTTPRequest, the file gets uploaded. So it looks like the file can be accessed in some way?
I’m new to Swift/iOS development so this is kind of frustrating for me, I feel like I tried just about everything.
The code:
override func viewDidLoad() {
super.viewDidLoad()
var audioFound :Bool = false
for inputItem: Any in self.extensionContext!.inputItems {
let extensionItem = inputItem as! NSExtensionItem
for attachment: Any in extensionItem.attachments! {
print("attachment = \(attachment)")
let itemProvider = attachment as! NSItemProvider
if itemProvider.hasItemConformingToTypeIdentifier(kUTTypeAudio as String) {
itemProvider.loadItem(forTypeIdentifier: kUTTypeAudio as String,
options: nil, completionHandler: { (audioURL, error) in
OperationQueue.main.addOperation {
if let audioURL = audioURL as? NSURL {
print("audioUrl = \(audioURL)")
// in our sample code we just present and play the audio in our app extension
let theAVPlayer :AVPlayer = AVPlayer(url: audioURL as URL)
let theAVPlayerViewController :AVPlayerViewController = AVPlayerViewController()
theAVPlayerViewController.player = theAVPlayer
self.present(theAVPlayerViewController, animated: true) {
theAVPlayerViewController.player!.play()
}
}
}
})
audioFound = true
break
}
}
if (audioFound) {
break
}
}
}
Related
I am trying to use AVPlayer to play/cache a remote asset using two tools on Github: CachingPlayerItem with Cache. I found the solution elsewhere(scroll down), which nearly gets me there, My issue now is that I have to tap twice on the remote audio asset (a hyperlink in Firebase) to get it to stream. For some mysterious reason, AVPlayer will not play the remote asset unless it is cached in my case. I am aware that I can directly stream the url using AVPlayerItem(url:) but that is not the solution I am seeking; the sample code for CachingPlayerItem say that should not be necessary.
In my tinkering, I think something is happening with the async operations that are performed when I call playerItem.delegate = self. Maybe I am misunderstanding how this asynchronous delegate operation is working... Any clarity and pointers would be appreciated.
import AVKit
import Cache
class AudioPlayer: AVPlayer, ObservableObject, AVAudioPlayerDelegate {
let diskConfig = DiskConfig(name: "DiskCache")
let memoryConfig = MemoryConfig(expiry: .never, countLimit: 10, totalCostLimit: 10)
lazy var storage: Cache.Storage<String, Data>? = {
return try? Cache.Storage(diskConfig: diskConfig, memoryConfig: memoryConfig, transformer: TransformerFactory.forData())
}()
/// Plays audio either from the network if it's not cached or from the cache.
func startPlayback(with url: URL) {
let playerItem: CachingPlayerItem
do {
let result = try storage!.entry(forKey: url.absoluteString)
// The video is cached.
playerItem = CachingPlayerItem(data: result.object, mimeType: "audio/mp4", fileExtension: "m4a")
} catch {
// The video is not cached.
playerItem = CachingPlayerItem(url: url)
}
playerItem.delegate = self // Seems to be the problematic line if the result is not cached.
self.replaceCurrentItem(with: playerItem) // This line is different from what you do. The behaviour doesnt change whether I have AVPlayer as private var.
self.automaticallyWaitsToMinimizeStalling = false
self.play()
}
}
extension AudioPlayer: CachingPlayerItemDelegate {
func playerItem(_ playerItem: CachingPlayerItem, didFinishDownloadingData data: Data) {
// Video is downloaded. Saving it to the cache asynchronously.
storage?.async.setObject(data, forKey: playerItem.url.absoluteString, completion: { _ in })
print("Caching done!")
}
}
I have a tableview which sets a UIImage to hold either an image url from AWS or a thumbnail generated from a video URL also from AWS. The video url refuses to display in my tableview and it throws this error in the debugger.
2017-12-29 12:20:37.053337-0800 VideoFit[3541:1366125] CredStore - performQuery - Error copying matching creds. Error=-25300, query={
class = inet;
"m_Limit" = "m_LimitAll";
"r_Attributes" = 1;
sync = syna;
}
When I click the cell to display either the image or video url, it segues to the video player correctly and then I get the error again and the triangular start button has a line through it signifying that there is no video to be played.
But when I print the url it has successfully passed so that is not the issue, the issue is AVPlayer can't handle my AWS video url for some reason. It is an https link so that means it must be secure but I already set my arbitrary loads to true
<key>NSAppTransportSecurity</key>
<dict>
<key>NSAllowsArbitraryLoads</key>
<true/>
</dict>
Here is some code for displaying my videos in the videoPlayer VC and also the thumbnail generator function, perhaps there is some issue lying with these?
import UIKit
import AVKit
import MediaPlayer
class VideoPlayerVC: AVPlayerViewController {
var urlToPlay: String?
override func viewDidLoad() {
super.viewDidLoad()
print("Here is the url ---> \(String(describing: urlToPlay))")
playVideo()
}
private func playVideo() {
guard let urlFromString = urlToPlay else { print("No url to play") ;return }
let url = URL(string: urlFromString)
print("Here is the url to play ---> \(String(describing: url))")
let asset: AVURLAsset = AVURLAsset(url: url!)
let item: AVPlayerItem = AVPlayerItem(asset: asset)
let player: AVPlayer = AVPlayer(playerItem: item)
self.player = player
self.showsPlaybackControls = true
self.player?.play()
}
}
This is how I make the thumbnail, when my cellforRow atIndexPath method runs it throws this error for every video object in the tableview.
override func tableView(_ tableView: UITableView, cellForRowAt indexPath: IndexPath) -> UITableViewCell {
if let cell = tableView.dequeueReusableCell(withIdentifier: "sortedExerciseCell") as? SortedExerciseCell! {
// Do if check for videoURI and imageURI
if selectedExerciseArray[indexPath.row].imageURI != nil {
if let imageURI = URL(string: selectedExerciseArray[indexPath.row].imageURI!) {
print("It's a photo!")
// Using DispatchQueue.global(qos: .background).async loads cells in background
DispatchQueue.global(qos: .background).async {
let data = try? Data(contentsOf: imageURI)
DispatchQueue.main.async {
cell.exerciseImg.image = UIImage(data: data!)
}
}
}
} else {
if let videoURI = URL(string: selectedExerciseArray[indexPath.row].videoURI!) {
print("It's a video!")
print(videoURI)
DispatchQueue.global(qos: .background).async {
DispatchQueue.main.async {
cell.exerciseImg.image = self.thumbnailForVideoAtURL(url: videoURI)
// for every video object the above error is thrown!!!
}
}
}
}
cell.exerciseName.text = selectedExerciseArray[indexPath.row].name
return cell
} else {
return UITableViewCell()
}
}
// Used to just display the video thumbnail in cell, when clicked on will display video as needed
private func thumbnailForVideoAtURL(url: URL) -> UIImage? {
let asset = AVAsset(url: url)
let assetImageGenerator = AVAssetImageGenerator(asset: asset)
do {
print("Doing the video thumbnail from backend")
let imageRef = try assetImageGenerator.copyCGImage(at: CMTimeMake(1, 60) , actualTime: nil)
return UIImage(cgImage: imageRef)
} catch {
print("This is failing for some reason")
print("error")
return nil
}
}
I've looked at similar questions on stack overflow about this but none seem to give a complete answer on how to solve this problem, most chalking it up to an iOS 11 bug that can't be fixed or transport security (Already have arbitrary loads on so this can't be the issue..) anyone else have any approaches that might work?
Important note - My backend developer can only view the video url's from a webpage, in other words he must make a basic website to display the video after downloading it from the web. I'm not sure if this is standard procedure for handling video url's from AWS but I decided to try loading the url with "loadHTMLString" in a custom UIViewController conforming to WKUIDelegate, I see the video but the same situation happens where the triangular start button is crossed out signifying no video can be played. I'm not really sure what else I can try at this moment, any help is appreciated.
Here is one of the links I've pulled from my backend.
https://videofitapptestbucket.s3.us-west-2.amazonaws.com/100001427750
It seems that your problem is in file extension. DMS is not recognized by OS, it throws when creating image with assetgenerator.
Error Domain=AVFoundationErrorDomain Code=-11828 "Cannot Open" UserInfo={NSLocalizedFailureReason=This media format is not supported., NSLocalizedDescription=Cannot Open, NSUnderlyingError=0x6040000534d0 {Error Domain=NSOSStatusErrorDomain Code=-12847 "(null)"}}
Rename your files on server. The mp4 extension seems to work just fine with your file.
Also, subclassing AVPlayerViewController is generally not that great idea as Apple says that it will result in undefined behavior. (read: random mess). I would suggest to use class that have AVPlayer inside.
Try:
if let path = urlToPlay {
let url = URL(string: path)!
let videoPlayer = AVPlayer(url: url)
self.player = videoPlayer
DispatchQueue.main.async {
self.player?.play()
}
}
Call it in viewDidAppear() method
I am trying to make an app that allows users to download videos to play offline, like downloading some videos for offline viewing in Netflix.
I am using Firebase as a server for the videos. Here comes the action for a download button:
#IBAction func btnDL(_ sender: Any) {
self.dlBtnOutlet.isEnabled = false
//Firebase
let storage = Storage.storage()
let videoRef = storage.reference(forURL: "gs://videowcoredata.appspot.com/Some video file.mp4")
//Local file system
let documentsURL = FileManager.default.urls(for: .documentDirectory, in: .userDomainMask).first!
let localURL = documentsURL.appendingPathComponent("movie.mp4")
// Download to the local filesystem
let downloadTask = videoRef.write(toFile: localURL) { (URL, error) -> Void in
if (error != nil) {
print("Uh-oh, an error occurred!")
print(error)
} else {
print("Local file URL is returned")
self.genericURL = String(localURL.path)
print(localURL.path)
//Save the link to CoreData
let context = (UIApplication.shared.delegate as! AppDelegate).persistentContainer.viewContext
let myUrl = MyFiles(context: context)
myUrl.fileURL = self.genericURL
(UIApplication.shared.delegate as! AppDelegate).saveContext()
I am using CoreData just for the persistence of the downloaded file link. Saving the links to an array, and returning the last member of the array as the downloaded link. I am trying to play the file in a new AVPlayerViewController, with the help of prepare for segue.
override func prepare(for segue: UIStoryboardSegue, sender: Any?) {
// Fetch CoreData
let destination = segue.destination as! AVPlayerViewController
let context = (UIApplication.shared.delegate as! AppDelegate).persistentContainer.viewContext
do {
urlList = try context.fetch(MyFiles.fetchRequest())
}
catch {
print("Oops we have an error!)")
}
self.genericURL = (urlList.last?.fileURL)!
destination.player = AVPlayer(url: URL(fileURLWithPath: self.genericURL))
}
I define two variables in the beginning of the code to access later:
var urlList : [MyFiles] = []
var genericURL = ""
So, my questions are:
I can manage to download the file, locate the downloaded file and save its path to CoreData for persistence. I can play the local video from the local file with the code above, but when I restart the simulator I no longer have access to the file. I need to download again, which is the thing I am trying to avoid. What can be the reason for this?
I wish to do this without using CoreData, since it is about File handling. But I don't have adequate resources to learn Filemanager. I simply want to protect my videos (I don't want that user syncs them) and I want my users to have offline access if they wish, but I don't want to include the videos in the Bundle. What would be the best way for this?
Thanks in advance for help!
I'm looking for a possibility to import a PDF in order to do some further tasks with it, just like described in this Question: Importing PDF files to app
After two days of looking around in the inter webs I found that an action extension might be the solution, this is how far I got:
override func viewDidLoad() {
super.viewDidLoad()
let fileItem = self.extensionContext!.inputItems.first as! NSExtensionItem
let textItemProvider = fileItem.attachments!.first as! NSItemProvider
let identifier = kUTTypePDF as String
if textItemProvider.hasItemConformingToTypeIdentifier(identifier) {
textItemProvider.loadItemForTypeIdentifier(identifier, options: nil, completionHandler: handleCompletion)
}
}
func handleCompletion(pdfFile: NSSecureCoding?, error: NSError!) {
print("PDF loaded - What to do now?")
}
The completion handler is called properly so I assume the PDF is loaded - but then I don't now how to proceed. If the action extension only handles images or text it could easily be downcasted, but the only way to work with files I know is with path names - which I do not have and don't know how to obtain. Plus, I'm pretty sure Sandboxing is also part of the party.
I guess I only need a push in the right direction which Class or Protocol could be suitable for my need - any suggestions highly appreciated.
For anyone else looking for an answer - I found out by myself, and it's embarrassingly easy:
func handleCompletion(fileURL: NSSecureCoding?, error: NSError!) {
if let fileURL = fileURL as? NSURL {
let newFileURL = NSURL(fileURLWithPath: NSTemporaryDirectory().stringByAppendingString("test.pdf"))
let fileManager = NSFileManager.defaultManager()
do {
try fileManager.copyItemAtURL(fileURL, toURL: newFileURL)
// Do further stuff
}
catch {
print(error)
}
}
}
I have made a class which handles audio that is played in my app:
import AVFoundation
class GSAudio{
var soundFileNameURL: NSURL = NSURL()
var soundFileName = ""
var soundPlay = AVAudioPlayer()
func playSound (soundFile: String){
soundFileName = soundFile
soundFileNameURL = NSURL(fileURLWithPath: NSBundle.mainBundle().pathForResource(soundFileName, ofType: "aif", inDirectory:"Sounds")!)
do{
try soundPlay = AVAudioPlayer(contentsOfURL: soundFileNameURL)
} catch {
print("Could not play sound file!")
}
soundPlay.prepareToPlay()
soundPlay.play()
}
}
But the problem is when I call this code by using something like GSAudio().playSound("Start") from a different class, the sound does not play. Does anyone know why this is? Any help is appreciated.
Many thanks.
I have method AVAudioPlayer and its working fine when the button click action then perform this below method play well, but You want to call class and method (ex 'GSAudio().playSound("Start")') from different class, you create the delegate method to call, its work well.
#IBAction func btnPlayAction(sender: AnyObject) {
let fileURL: NSURL = NSURL(string: previewUrl)!
let soundData = NSData(contentsOfURL: fileURL)
do {
playerVal = try AVAudioPlayer(data: soundData!)
}
catch {
print("Something bad happened. Try catching specific errors to narrow things down",error)
}
playerVal.delegate = self
playerVal.prepareToPlay()
playerVal.play()
}
hope its helpful
Check Your Sound File (URL) Path..(print That) and Checkout file:// Is There Or Not That's Main Problem In Your Sound ULR Ok Add it File://
//Or Use this One
NSURL *SoundURL = [NSURL fileURLWithPath:YourFilePath];
Try This One And Let me Know is i'm Correct or not
I have a helper on GitHub that does what you want. It's properly better to make this class a Singleton so you can control your music better and don't create multiple instances of your helper.
https://github.com/crashoverride777/Swift-Music-Helper