AVAssetDownloadTask iOS13 - ios

Tried iOS13.0 and iOS13.1 and still not working, I tried both AVAggregateAssetDownloadTask and AVAssetDownloadURLSession but none of them working. Not any delegate was called to tell me error of finish, and I found downloaded cache was only 25Kb what was not the right size.
The error is:
Error Domain=AVFoundationErrorDomain Code=-11800 "The operation could not be completed" UserInfo={NSLocalizedDescription=The operation could not be completed, _NSURLErrorFailingURLSessionTaskErrorKey=BackgroundAVAssetDownloadTask <AFDCA3CC-FA49-488B-AB16-C74425345EE4>.<1>, _NSURLErrorRelatedURLSessionTaskErrorKey=(
"BackgroundAVAssetDownloadTask <AFDCA3CC-FA49-488B-AB16-C74425345EE4>.<1>"
), NSLocalizedFailureReason=An unknown error occurred (-16654)}

Found out AVAssetDownloadURLSession can only download HLS with master playlist structure which contains codec attribute into EXT-X-STREAM-INF m3u8 meta on iOS 13+.
I have no idea if this is a bug or function restriction.
(m3u8 meta have no CODECS attribute can be played with AVFoundation, but can't be downloaded with AVAssetDownloadURLSession)
Anyway, the solution is:
If you have HLS master playlist:
add CODECS attribute into your #EXT-X-STREAM-INF in m3u8 meta.
e.g.
#EXTM3U
#EXT-X-VERSION:3
#EXT-X-STREAM-INF:BANDWIDTH=63701,CODECS="mp4a.40.34"
playlist.m3u8
If you haven't HLS master playlist yet:
You have to make a master playlist even if you're not supporting adaptive streaming.
The master playlist is the only m3u8 which can contain #EXT-X-STREAM-INF hence CODECS attribute.

So, I found out that the 'AVAssetDownloadTask' had some error in calling delegates in iOS 13 (13.1,13.2.13.3). Finally, in iOS 13.4.1, Apple has fixed this error and now delegates have called after setting delegate and starting the task. Below is what I used to start downloading the m3u8 file from the server and saving it as an Asset to play later offline.
func downloadVideo(_ url: URL) {
let configuration = URLSessionConfiguration.background(withIdentifier: currentFileName)
let downloadSession = AVAssetDownloadURLSession(configuration: configuration,
assetDownloadDelegate: self,
delegateQueue: OperationQueue.main)
// HLS Asset URL
let asset = AVURLAsset(url: url)
// Create new AVAssetDownloadTask for the desired asset
let downloadTask = downloadSession.makeAssetDownloadTask(asset: asset,
assetTitle: currentFileName,
assetArtworkData: nil,
options: nil)
// Start task and begin download
downloadTask?.resume()
}
I tried this on iOS 12 and iOS 13.4.1 and it is working as expected. Also, it was already on the Apple Developer Forums here. Hope this helps someone.

Related

Unable to play audio / speech from iOS Swift Safari Extension on device (OSStatus error 2003329396)

I'm currently trying to use AVSpeechSynthesizer to speak text from within an iOS Safari extension:
let synthesizer = AVSpeechSynthesizer()
...
let utterance = AVSpeechUtterance(string: self.text)
utterance.rate = 0.55;
self.synthesizer.speak(utterance)
On a simulator this works fine. However, on a physical device, I get the following error (even when the device is unmuted/volume-up):
NSURLConnection finished with error - code -1002
NSURLConnection finished with error - code -1002
NSURLConnection finished with error - code -1002
[AXTTSCommon] Failure starting audio queue alp!
[AXTTSCommon] Run loop timed out waiting for free audio buffer
[AXTTSCommon] _BeginSpeaking: speech cancelled error: Error Domain=TTSErrorDomain Code=-4001 "(null)"
[AXTTSCommon] _BeginSpeaking: couldn't begin playback
I have looked through quite a few SO and Apple Dev Forums threads and have tried many of the proposed solutions with no luck. Here are the things I've tried:
Linking AVFAudio.framework and AVFoundation.framework to the extension.
Starting an AVAudioSession prior to playing the utterance:
do {
let session = AVAudioSession.sharedInstance()
try session.setCategory(.playback, mode: .default, options: [.mixWithOthers, .allowAirPlay])
try session.setActive(true, options: .notifyOthersOnDeactivation)
} catch let error {
print("Error starting audio: \(error.localizedDescription)")
}
This actually results in another error being thrown right before the same errors above:
Error starting audio: The operation couldn’t be completed. (OSStatus error 2003329396.)
Playing a plain mp3 audio file:
guard let url = Bundle.main.url(forResource: "sample", withExtension: "mp3") else {
print("Couldn't find file")
return
}
do {
self.player = try AVAudioPlayer(contentsOf: url)
self.player.play()
print("**playing sound")
} catch let error as NSError {
print("Error playing sound: \(error.localizedDescription)")
}
This prints the following:
**playing sound
[aqsrv] AQServer.cpp:72 Exception caught in AudioQueueInternalNotifyRunning - error -66671
Enabling Audio, AirPlay, and Picture in Picture in Background Modes for the main target app (not available for the extension).
Any help would be appreciated.
EDIT:
The solution below gets rejected due to a validation error when submitting to App Store Connect.
I filed a Technical Support Incident with Apple, and this was their response:
Safari extensions are very short-lived, hence not fit for audio playback or speech synthesis. Not being able to validate an app extension in Xcode with a manually-added plist entry for background audio is the designed behavior. The general recommendation is to synthesize speech using JavaScript in conjunction with the Web Speech API.
TLDR: Use the Web Speech API for text-to-speech in Safari extensions, not AVSpeechSynthesizer.
Original answer:
Adding the following to the extension's Info.plist allowed the audio to play as expected:
<dict>
...
<key>UIBackgroundModes</key>
<array>
<string>audio</string>
</array>
...
</dict>
Interestingly, it actually shows the same errors in the console as before, but it does play the audio.

Why can't I play certain .m3u8 items with AVPlayer?

I'm working on a project where I have an instance of AVPlayer capable of playing different audio content that I retrieve from a backend, from podcast to music and streamings. Every content has two types of urls: one with mp3 and another with a m3u8 file. All the mp3 files work good. However some m3u8 files work fine and others don't. In particular, those who don't work cause the AVPlayer to crash with the error:
Error Domain=AVFoundationErrorDomain Code=-11819 "Cannot Complete Action"
UserInfo={NSLocalizedRecoverySuggestion=Try again later.,
NSLocalizedDescription=Cannot Complete Action.}
I don't understand what the problem is. According to this answer it is a wrong Manifest file, which in my case is - for example - the following:
#EXTM3U
#EXT-X-MEDIA:TYPE=AUDIO,URI="_64/index.m3u8",GROUP-ID="2#48000-64000",NAME="AAC 64",DEFAULT=NO,AUTOSELECT=NO
#EXT-X-MEDIA:TYPE=AUDIO,URI="_80/index.m3u8",GROUP-ID="2#48000-80000",NAME="AAC 80",DEFAULT=NO,AUTOSELECT=NO
#EXT-X-MEDIA:TYPE=AUDIO,URI="_96/index.m3u8",GROUP-ID="2#48000-96000",NAME="AAC 96",DEFAULT=NO,AUTOSELECT=NO
#EXT-X-STREAM-INF:BANDWIDTH=133336,CODECS="mp4a.40.2",AUDIO="2#48000-96000"
_96/index.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=100641,CODECS="mp4a.40.2",AUDIO="2#48000-64000"
_64/index.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=116989,CODECS="mp4a.40.2",AUDIO="2#48000-80000"
_80/index.m3u8
On the Apple forum, I found this answer which says iOS 14+ is on fault. Unfortunately I cannot test with an iOS 13 physical device.
Do you have any suggestion?
Tested on Xcode 13.1 with iPhone 7plus with iOS 15.0.2.
Finally I found a solution for this issue. What worked for me was this. I believe the problem was that my manifest files were structured like the following:
#EXT-X-MEDIA:TYPE=AUDIO,URI="_64/index.m3u8", GROUP-ID="1#48000-64000",NAME="Audio 64",DEFAULT=NO,AUTOSELECT=NO
In particular they had DEFAULT=NO,AUTOSELECT=NO. Therefore before calling replaceCurrentItem I now do the following:
let asset = AVAsset(url: url)
let playerItem = AVPlayerItem(asset: asset)
for characteristic in asset.availableMediaCharacteristicsWithMediaSelectionOptions {
if let group = asset.mediaSelectionGroup(forMediaCharacteristic: AVMediaCharacteristic.audible) {
if let option = group.options.first {
playerItem.select(option, in: group)
}
}
}
This makes all my HLS audio playable by the AVPlayer.
I dont see version in your .m3u8. Try adding #EXT-X-VERSION:03 into your playlist. AVPlayer does need to have version included in playlist (Android EXO player does not need it). Here is example of playlist that might work:
#EXTM3U
#EXT-X-VERSION:03
#EXT-X-MEDIA:TYPE=AUDIO,URI="_64/index.m3u8",GROUP-ID="2#48000-64000",NAME="AAC 64",DEFAULT=NO,AUTOSELECT=NO
#EXT-X-MEDIA:TYPE=AUDIO,URI="_80/index.m3u8",GROUP-ID="2#48000-80000",NAME="AAC 80",DEFAULT=NO,AUTOSELECT=NO
#EXT-X-MEDIA:TYPE=AUDIO,URI="_96/index.m3u8",GROUP-ID="2#48000-96000",NAME="AAC 96",DEFAULT=NO,AUTOSELECT=NO
#EXT-X-STREAM-INF:BANDWIDTH=133336,CODECS="mp4a.40.2",AUDIO="2#48000-96000"
_96/index.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=100641,CODECS="mp4a.40.2",AUDIO="2#48000-64000"
_64/index.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=116989,CODECS="mp4a.40.2",AUDIO="2#48000-80000"
_80/index.m3u8

HLS m3u8 Download takes a lot of time

I am trying to download a HLS stream onto my iPhone using apple's documentation mentioned here - https://developer.apple.com/library/archive/documentation/AudioVideo/Conceptual/MediaPlaybackGuide/Contents/Resources/en.lproj/HTTPLiveStreaming/HTTPLiveStreaming.html
Even small 2 minutes videos are taking about 3-4 mins to download, i am having a fast internet connection so that is not an issue.
What i want to know is there any specific stream i need to download or specify while downloading? The streams in my m3u8 are 320p, 720p, 1080p. Is it downloading all of them so it's taking that much of time?
This is how i am setting up code to download hls -
let urlConfiguration = URLSessionConfiguration.background(withIdentifier: downloadIdentifier)
// Create a new AVAssetDownloadURLSession with background configuration, delegate, and queue
let assetDownloadSession = AVAssetDownloadURLSession(configuration: urlConfiguration,
assetDownloadDelegate: self,
delegateQueue: OperationQueue.main)
let asset = AVURLAsset(url: URL(string: assetUrl)!)
// Create new AVAssetDownloadTask for the desired asset
let avAssetDownloadTask = assetDownloadSession?.makeAssetDownloadTask(asset: asset,
assetTitle: title,
assetArtworkData: nil,
options: nil)
// Start task and begin download
avAssetDownloadTask?.resume()
You might consider using a tool like Charles Proxy or Wireshark to determine the throughput of underlying network requests. You may have a slow server.

How can you obtain a current manifest from the playhead in AV player?

I am currently trying to capture the currently playing HLS/DASH manifest in the AV player.
Would anyone have any code snippets/documentation that could help with it?
To access the current url that the avplayer is streaming from:
if let currAsset = player.currentItem.asset as? AVURLAsset {
var url = currAsset.url
... use the url ...
}
I didn't find any official documentation about this either. I suppose then at the time of writing that AVPlayer does not provide the manifest.
Failing that, consider downloading the playlist and parse the manifest by your own.
I found M3U8Kit useful to parse m3u8 manifest.

How to add url in avaudioplayer in swift

i just want to ask on how to play audio using avAudion using url link..
and im getting error bad request.. can someone help me?
i already tried using avplayer but it is not suitable for me.. though it is working but.. i kinda prefer to use avaudioplayer
let mp3URL = NSURL(fileURLWithPath:"https://s3.amazonaws.com/kargopolov/kukushka.mp3")
do {
// 2
audioPlayer = try AVAudioPlayer(contentsOf: mp3URL as URL)
audioPlayer.play()
// 3
Timer.scheduledTimer(timeInterval: 1.0, target: self, selector: #selector(updateAudioProgressView), userInfo: nil, repeats: true)
progressView.setProgress(Float(audioPlayer.currentTime/audioPlayer.duration), animated: false)
}
catch {
print("An error occurred while trying to extract audio file")
}
According to the AVAudioPlayer documentation, if you want to play audio over the internet (that's not already downloaded to disk or memory) you should use AVPlayer.
The docs describe AVAudioPlayer as
An audio player that provides playback of audio data from a file or memory.
A little further down, in the overview section, emphasis added:
Use this class for audio playback unless you are playing audio captured from a network stream or require very low I/O latency.
Notes
You can get some more information from the runtime by accessing the error object that is thrown when the connection fails. Change catch { to catch let error { and then you can log out the error as part of your message, like so:
catch let error{
print("An error occurred while trying to extract audio file: \(error)")
}
When I run your sample code in Xcode playgrounds with the change noted above, I see the following:
An error occurred while trying to extract audio file: Error Domain=NSOSStatusErrorDomain Code=2003334207 "(null)"
Notice the error's Domain and Code. Pasting that error into Google yields some results that indicate that the URL might not be resolving correctly. (Indeed, clicking on that mp3 link shows a bad access error message.)
The first result is another StackOverflow post with a similar issues. The second one is an Apple Developer Forum post which has some more information.
Let's try to change the URL to a publicly accessible sample mp3 file. (I found this one by searching the web for "test mp3 file" on Google.)
You'll want to change NSURL to URL, and instead of fileURLWithPath, you're going to want to use another initializer. (Say, string:.)
Whenever you see contentsOf...: in a media or file API, there's a good chance it expects data or a file, to the exclusion of a network stream. Similarly, when you see an initializer or method that takes a fileURL..., the system expects to be pointing to a local resource, not a network URL.
if let mp3URL = URL(string: "https://s3.amazonaws.com/kargopolov/kukushka.mp3"){
// do s.t
}
Anyway, you should know the basics of ios

Resources