I have an audio messaging app where I'd like a user to play an audio message and have it transcribed as it is playing.
Audio messages are recorded and stored via Firebase storage as m4a file types.
The error I am receiving is...
There was an error: Error Domain=AVFoundationErrorDomain Code=-11838 "Cannot initialize an instance of AVAssetReader with an asset at non-local URL
The following is my code...
func transcribeAudio(url: URL) {
let recognizer = SFSpeechRecognizer()
let request = SFSpeechURLRecognitionRequest(url: url)
recognizer?.recognitionTask(with: request) { [unowned self] (result, error) in
guard let result = result else {
print("There was an error: \(error!)")
DispatchQueue.main.async {
self.transcriptionLabel.text = "We're not able to transcribe this audio message."
}
return
}
if result.isFinal {
DispatchQueue.main.async {
self.transcriptionLabel.text = result.bestTranscription.formattedString
}
}
}
}
Any guidance would be appreciated. Have not been able to find any existing solution..
UPDATE:
I was able to get the transcription to work, but to make that possible...I had to download the URL I had fetched from Firebase storage via URLSession.shared.downloadTask(with: url), save it locally, and insert that local url into the SFSpeechURLRecognitionRequest method.
While this technically works, I'm really hoping there's a way I can just do it with the original non local url..it's also extremely slow.
Related
I am trying to save a large video locally to the photo library using PHPhotoLibrary but i notice that it takes a very long time is there any way to get progress or even better to make the process faster
my code:
func saveToLibrary(videoURL: URL, complition: #escaping () -> Void) {
PHPhotoLibrary.requestAuthorization { status in
guard status == .authorized else { return }
PHPhotoLibrary.shared().performChanges({
PHAssetChangeRequest.creationRequestForAssetFromVideo(atFileURL: videoURL)
}) { success, error in
if !success {
print("Could not save video to photo library: \( error as Any)")
} else {
complition()
}
}
}
}
Do this on a background thread so that your UI doesn't get locked up.
You can first download the video in a temporary local file using NSURLSessionDownloadTask, and then pass the the URL from the that local file to PHPhotoLibrary. This way you can monitor the download progress.
Something like this would work:
// Download the remote URL to a file
let task = URLSession.shared.downloadTask(with: url) {
(tempURL, response, error) in
// Early exit on error
guard let tempURL = tempURL else {
return
}
// The file is downloaded in the tempURL we can save it in the library
saveToLibrary(videoURL: tempURL, complition: {})
}
// Start the download
task.resume()
// Monitor the progress
let observation = task.progress.observe(\.fractionCompleted) { progress, _ in
print(progress.fractionCompleted)
}
I am trying to build an audio app for apple watch. But the problem is whenever I keep my hands down , audio will stop playing.
I have turned background mode on as well.
Can anyone please help me with this? I am stuck at this part.
Here is the Code I have used for playing audio.
func play(url : URL) {
do {
if #available(watchOSApplicationExtension 4.0, *) {
WKExtension.shared().isFrontmostTimeoutExtended = true
} else {
// Fallback on earlier versions
}
self.player = try AVAudioPlayer(contentsOf: url)
player!.prepareToPlay()
player?.delegate = self
player?.play()
print("-----------------")
print("Playing Audio")
print("*****************\nCurrent Time \(String(describing: self.player?.currentTime))")
} catch let error as NSError {
self.player = nil
print(error.localizedDescription)
} catch {
print("*************************")
print("AVAudioPlayer init failed")
}
}
Make sure you are trying to play with Audio Data, not Audio URL and have added policy: .longFormAudio in your category setup. As per Apple documentation, these two settings have to be set for audio to play in background mode.
// Set up the session.
let session = AVAudioSession.sharedInstance()
do {
try session.setCategory(
.playback,
mode: .default,
policy: .longFormAudio
)
} catch let error {
fatalError("*** Unable to set up the audio session: \(error.localizedDescription) ***")
}
// Set up the player.
let player: AVAudioPlayer
do {
player = try AVAudioPlayer(data: audioData)
} catch let error {
print("*** Unable to set up the audio player: \(error.localizedDescription) ***")
// Handle the error here.
return
}
// Activate and request the route.
session.activate(options: []) { (success, error) in
guard error == nil else {
print("*** An error occurred: \(error!.localizedDescription) ***")
// Handle the error here.
return
}
// Play the audio file.
player.play()
}
I have tested this code and its working with only Bluetooth connectivity in Watch application not in watch speaker.
Simply turning on background mode is not enough. You also need to activate the AVAudioSession.
It's all well documented by Apple here: Playing Background Audio.
Configure and Activate the Audio Session
Before you can play audio, you need to set up and activate the audio session.
session.setCategory(AVAudioSession.Category.playback,
mode: .default,
policy: .longForm,
options: [])
Next, activate the session, by calling the activate(options:completionHandler:) method.
session.activate(options: []) { (success, error) in
// Check for an error and play audio.
}
Ref: https://developer.apple.com/documentation/watchkit/playing_background_audio
Example:
var player: AVAudioPlayer?
let session: AVAudioSession = .sharedInstance()
func prepareSession() {
do {
try session.setCategory(AVAudioSession.Category.playback,
mode: .default,
policy: .longForm,
options: [])
}
catch {
print(error)
}
}
func play(url: URL) {
do {
player = try AVAudioPlayer(contentsOf: url)
}
catch {
print(error)
return
}
session.activate(options: []) { (success, error) in
guard error == nil else {
print(error!)
return
}
// Play the audio file
self.player?.play()
}
}
Simple Test:
prepareSession()
if let url = Bundle.main.url(forResource: "test", withExtension: "mp3") {
play(url: url)
}
else {
print("test.mp3 not found in project: put any mp3 file in and name it so")
}
I would like to improve the integration of my App FM-Pod with Siri intents shortcuts. I've done this App to listen the radio on the HomePod and, as of now, I've been able to start the playback, change the stations, etc. but I'm facing a strange issue which causes the audio playback to stop alone after about 1 minute...
Does anyone knows the reason? What's wrong?
Here is the code in Swift for starting the playback, leveraging on AVAudioPlayer:
open func handle(intent: StartFMPodIntent, completion: #escaping (StartFMPodIntentResponse) -> Void) {
DataManager.getStationDataWithSuccess(filename: "favorites") { (data) in
if debug { print("Stations JSON Found") }
guard let data = data, let jsonDictionary = try? JSONDecoder().decode([String: [RadioStation]].self, from: data), let stationsArray = jsonDictionary["station"]
else {
if debug { print("JSON Station Loading Error") }
return
}
HPRIntentHandler.stations = stationsArray
if !FRadioPlayer.shared.isPlaying {
FRadioPlayer.shared.radioURL = URL(string: HPRIntentHandler.stations![0].streamURL0!)
let response = StartFMPodIntentResponse(code: .success, userActivity: nil)
response.stationName = stationsArray[0].name
completion(response)
}
}
}
According to https://developer.apple.com/library/archive/documentation/General/Conceptual/ExtensibilityPG/ExtensionOverview.html#//apple_ref/doc/uid/TP40014214-CH2-SW2
Extension is killed after code is finished running.
Since the getStationDataWithSuccess run in Asynchronously, its code reaches the end before the asynchronous function finishes running.
You have to find different place to handle this.
I have a tableview which sets a UIImage to hold either an image url from AWS or a thumbnail generated from a video URL also from AWS. The video url refuses to display in my tableview and it throws this error in the debugger.
2017-12-29 12:20:37.053337-0800 VideoFit[3541:1366125] CredStore - performQuery - Error copying matching creds. Error=-25300, query={
class = inet;
"m_Limit" = "m_LimitAll";
"r_Attributes" = 1;
sync = syna;
}
When I click the cell to display either the image or video url, it segues to the video player correctly and then I get the error again and the triangular start button has a line through it signifying that there is no video to be played.
But when I print the url it has successfully passed so that is not the issue, the issue is AVPlayer can't handle my AWS video url for some reason. It is an https link so that means it must be secure but I already set my arbitrary loads to true
<key>NSAppTransportSecurity</key>
<dict>
<key>NSAllowsArbitraryLoads</key>
<true/>
</dict>
Here is some code for displaying my videos in the videoPlayer VC and also the thumbnail generator function, perhaps there is some issue lying with these?
import UIKit
import AVKit
import MediaPlayer
class VideoPlayerVC: AVPlayerViewController {
var urlToPlay: String?
override func viewDidLoad() {
super.viewDidLoad()
print("Here is the url ---> \(String(describing: urlToPlay))")
playVideo()
}
private func playVideo() {
guard let urlFromString = urlToPlay else { print("No url to play") ;return }
let url = URL(string: urlFromString)
print("Here is the url to play ---> \(String(describing: url))")
let asset: AVURLAsset = AVURLAsset(url: url!)
let item: AVPlayerItem = AVPlayerItem(asset: asset)
let player: AVPlayer = AVPlayer(playerItem: item)
self.player = player
self.showsPlaybackControls = true
self.player?.play()
}
}
This is how I make the thumbnail, when my cellforRow atIndexPath method runs it throws this error for every video object in the tableview.
override func tableView(_ tableView: UITableView, cellForRowAt indexPath: IndexPath) -> UITableViewCell {
if let cell = tableView.dequeueReusableCell(withIdentifier: "sortedExerciseCell") as? SortedExerciseCell! {
// Do if check for videoURI and imageURI
if selectedExerciseArray[indexPath.row].imageURI != nil {
if let imageURI = URL(string: selectedExerciseArray[indexPath.row].imageURI!) {
print("It's a photo!")
// Using DispatchQueue.global(qos: .background).async loads cells in background
DispatchQueue.global(qos: .background).async {
let data = try? Data(contentsOf: imageURI)
DispatchQueue.main.async {
cell.exerciseImg.image = UIImage(data: data!)
}
}
}
} else {
if let videoURI = URL(string: selectedExerciseArray[indexPath.row].videoURI!) {
print("It's a video!")
print(videoURI)
DispatchQueue.global(qos: .background).async {
DispatchQueue.main.async {
cell.exerciseImg.image = self.thumbnailForVideoAtURL(url: videoURI)
// for every video object the above error is thrown!!!
}
}
}
}
cell.exerciseName.text = selectedExerciseArray[indexPath.row].name
return cell
} else {
return UITableViewCell()
}
}
// Used to just display the video thumbnail in cell, when clicked on will display video as needed
private func thumbnailForVideoAtURL(url: URL) -> UIImage? {
let asset = AVAsset(url: url)
let assetImageGenerator = AVAssetImageGenerator(asset: asset)
do {
print("Doing the video thumbnail from backend")
let imageRef = try assetImageGenerator.copyCGImage(at: CMTimeMake(1, 60) , actualTime: nil)
return UIImage(cgImage: imageRef)
} catch {
print("This is failing for some reason")
print("error")
return nil
}
}
I've looked at similar questions on stack overflow about this but none seem to give a complete answer on how to solve this problem, most chalking it up to an iOS 11 bug that can't be fixed or transport security (Already have arbitrary loads on so this can't be the issue..) anyone else have any approaches that might work?
Important note - My backend developer can only view the video url's from a webpage, in other words he must make a basic website to display the video after downloading it from the web. I'm not sure if this is standard procedure for handling video url's from AWS but I decided to try loading the url with "loadHTMLString" in a custom UIViewController conforming to WKUIDelegate, I see the video but the same situation happens where the triangular start button is crossed out signifying no video can be played. I'm not really sure what else I can try at this moment, any help is appreciated.
Here is one of the links I've pulled from my backend.
https://videofitapptestbucket.s3.us-west-2.amazonaws.com/100001427750
It seems that your problem is in file extension. DMS is not recognized by OS, it throws when creating image with assetgenerator.
Error Domain=AVFoundationErrorDomain Code=-11828 "Cannot Open" UserInfo={NSLocalizedFailureReason=This media format is not supported., NSLocalizedDescription=Cannot Open, NSUnderlyingError=0x6040000534d0 {Error Domain=NSOSStatusErrorDomain Code=-12847 "(null)"}}
Rename your files on server. The mp4 extension seems to work just fine with your file.
Also, subclassing AVPlayerViewController is generally not that great idea as Apple says that it will result in undefined behavior. (read: random mess). I would suggest to use class that have AVPlayer inside.
Try:
if let path = urlToPlay {
let url = URL(string: path)!
let videoPlayer = AVPlayer(url: url)
self.player = videoPlayer
DispatchQueue.main.async {
self.player?.play()
}
}
Call it in viewDidAppear() method
I'm using the SKTAudio library given by Ray Wenderlich on his Github. Here's the code I've modified for playing a sound effect that I call everytime I play one:
public func playSoundEffect(_ filename: String) {
let url = Bundle.main.url(forResource: filename, withExtension: nil)
if (url == nil) {
print("Could not find file: \(filename)")
return
}
if !UserDefaults.standard.bool(forKey: "soundOff"){
var error: NSError? = nil
do {
soundEffectPlayer = try AVAudioPlayer(contentsOf: url!)
} catch let error1 as NSError {
error = error1
soundEffectPlayer = nil
}
if let player = soundEffectPlayer {
DispatchQueue.global(qos: .background).async {
player.numberOfLoops = 0
player.prepareToPlay()
player.play()
}
} else {
print("Could not create audio player: \(error!)")
}
}
}
I'm using this because this way I can mute the sound effects easily, since this is using a singleton class method. The thing is, this produces minor lag everytime I play a sound effect. If I mute the game, no lag. I tried making the sound play in a background thread, but the result is the same.
Is there any easy change to this code to make it have NO LAG?
EDIT: This is not a duplicate because none of the other answers on other questions solved my problem.