I work on speech recognition. I solve the text-to-speech and speech-to-text with IOS frameworks. But now i want to convert saved audio file conversations to text. How can i solve this ? Thank you for all replies.
I have worked on same things which are working for me.
I have audio file in my project bundle which. So I have written following code to convert audio to text.
let audioURL = Bundle.main.url(forResource: "Song", withExtension: "mov")
let recognizer = SFSpeechRecognizer(locale: Locale(identifier: "en-US"))
let request = SFSpeechURLRecognitionRequest(url: audioURL!)
request.shouldReportPartialResults = true
if (recognizer?.isAvailable)! {
recognizer?.recognitionTask(with: request) { result, error in
guard error == nil else { print("Error: \(error!)"); return }
guard let result = result else { print("No result!"); return }
print(result.bestTranscription.formattedString)
}
} else {
print("Device doesn't support speech recognition")
}
First get audio url from where you have store audio file.
Then create instance of SFSpeechRecognizer with locale that you have want.
Create instance of SFSpeechURLRecognitionRequest which are used to requesting recognitionTask.
recognitionTask will give you result and error. Where result contains bestTranscription.formattedString. formmatedString is your test result of audio file.
If set request.shouldReportPartialResults = true, this will give your partial result of every line speak in audio.
I hope this will help you.
Related
I want to programmatically navigate chapters of a mp4 video.
The chapters work in QuickTime, so I assume the video format isn't the issue.
The code from this page should return an array of the chapters but only returns an empty one instead:
https://developer.apple.com/documentation/avfoundation/media_playback/presenting_chapter_markers
let asset = AVAsset(url: <# Asset URL #>)
let chapterLocalesKey = "availableChapterLocales"
asset.loadValuesAsynchronously(forKeys: [chapterLocalesKey]) {
var error: NSError?
let status = asset.statusOfValue(forKey: chapterLocalesKey, error: &error)
if status == .loaded {
let languages = Locale.preferredLanguages
let chapterMetadata = asset.chapterMetadataGroups(bestMatchingPreferredLanguages: languages)
// Process chapter metadata.
}
else {
// Handle other status cases.
}
}
Has anyone an idea how to do it?
I have a video sharing app, and when you save a video to firebase storage it works perfectly for videos that are roughly 1 minute or shorter.
The problem that I am having, is when I try to post a longer video (1 min or greater) it never saves to firebase.
The only thing that I can think of is this error that I am getting, and this error only shows up about 30 seconds after I click the save button:
[BackgroundTask] Background Task 101 ("GTMSessionFetcher-firebasestorage.googleapis.com"), was created over 30 seconds ago. In applications running in the background, this creates a risk of termination. Remember to call UIApplication.endBackgroundTask(_:) for your task in a timely manner to avoid this.
Here is my code to save the video to firebase.
func saveMovie(path: String, file: String, url: URL) {
var backgroundTaskID: UIBackgroundTaskIdentifier?
// Perform the task on a background queue.
DispatchQueue.global().async {
// Request the task asseration and save the ID
backgroundTaskID = UIApplication.shared.beginBackgroundTask(withName: "Finish doing this task", expirationHandler: {
// End the task if time expires
UIApplication.shared.endBackgroundTask(backgroundTaskID!)
backgroundTaskID = UIBackgroundTaskIdentifier.invalid
})
// Send the data synchronously
do {
let movieData = try Data(contentsOf: url)
self.storage.child(path).child("\(file).m4v").putData(movieData)
} catch let error {
fatalError("Error saving movie in saveMovie func. \(error.localizedDescription)")
}
//End the task assertion
UIApplication.shared.endBackgroundTask(backgroundTaskID!)
backgroundTaskID = UIBackgroundTaskIdentifier.invalid
}
}
Any suggestions on how I can allow my video time to upload?
Finally figured this out after a long time...
All you have to do is use .putFile("FileURL") instead of .putdata("Data"). Firebase documentation says you should use putFile() instead of putData() when uploading large files.
But the hard part is for some reason you can't directly upload the movie URL that you get from the didFinishPickingMediaWithInfo function and firebase will just give you an error. So what I did instead was get the data of the movie, save the movie data to a path in the file manager, and use the file manager path URL to upload directly to firebase which worked for me.
//Save movie to Firestore
do {
// Convert movie to Data.
let movieData = try Data(contentsOf: movie)
// Get path so we can save movieData into fileManager and upload to firebase because movie URL does not work, but fileManager url does work.
guard let path = FileManager.default.urls(for: .documentDirectory, in: .userDomainMask).first?.appendingPathComponent(postId!) else { print("Error saving to file manager in addPost func"); return }
do {
try movieData.write(to: path)
// Save the file manager url file to firebase storage
Storage.storage().reference().child("Videos").child("\(postId!).m4v").putFile(from: path, metadata: nil) { metadata, error in
if let error = error {
print("There was an error \(error.localizedDescription)")
} else {
print("Video successfully uploaded.")
}
// Delete video from filemanager because it would take up too much space to save all videos to file manager.
do {
try FileManager.default.removeItem(atPath: path.path)
} catch let error {
print("Error deleting from file manager in addPost func \(error.localizedDescription)")
}
}
} catch let error {
print("Error writing movieData to firebase \(error.localizedDescription)")
}
} catch let error {
print("There was an error adding video in addPost func \(error.localizedDescription)")
}
I wanna streaming my app to twitch, youtube or such a streaming service without any other application likes mob crush.
According to Apple, by using Broadcast Extension I can stream my application screen.
Broadcast Extension gave video data as a type of CMSampleBuffer. Then I should send that data to rtmp sever like youtube, twitch or etc.
I think if I can get video data, I can stream the other things without using Broadcast Extension in my app. So I try to send RPScreenRecorder data to rtmp server, but I doesn't work.
Here is a code I wrote.
I use HaishinKit open source framework to rtmp communication.
(https://github.com/shogo4405/HaishinKit.swift/tree/master/Examples/iOS/Screencast)
let rpScreenRecorder : RPScreenRecorder = RPScreenRecorder.shared()
private var broadcaster: RTMPBroadcaster = RTMPBroadcaster()
rpScreenRecorder.startCapture(handler: { (cmSampleBuffer, rpSampleBufferType, error) in
if (error != nil) {
print("Error is occured \(error.debugDescription)")
} else {
if let description: CMVideoFormatDescription = CMSampleBufferGetFormatDescription(cmSampleBuffer) {
let dimensions: CMVideoDimensions = CMVideoFormatDescriptionGetDimensions(description)
self.broadcaster.stream.videoSettings = [
"width": dimensions.width,
"height": dimensions.height ,
"profileLevel": kVTProfileLevel_H264_Baseline_AutoLevel
]
}
self.broadcaster.appendSampleBuffer(cmSampleBuffer, withType: .video)
}
}) { (error) in
if ( error != nil) {
print ( "Error occured \(error.debugDescription)")
} else {
print ("Success")
}
}
}
If you have any solution, please answer me :)
I've tried a similar setup and it is possible to achieve what you'd like, just need to adjust it a little:
I don't see it in your example, but make sure that the broadcaster's endpoint is correctly set up. For example:
let endpointURL: String = "rtmps://live-api-s.facebook.com:443/rtmp/"
let streamName: String = "..."
self.broadcaster.streamName = streamName
self.broadcaster.connect(endpointURL, arguments: nil)
Then in the startCapture's handler block you need to filter by the buffer type to send the correct data to the stream. In this case you're only sending the video so we can ignore audio. (You can also find some examples with HaishinKit to send audio too.) For example:
RPScreenRecorder.shared().startCapture(handler: { (sampleBuffer, type, error) in
if type == .video, broadcaster.connected {
if let description: CMVideoFormatDescription = CMSampleBufferGetFormatDescription(sampleBuffer) {
let dimensions: CMVideoDimensions = CMVideoFormatDescriptionGetDimensions(description)
broadcaster.stream.videoSettings = [
.width: dimensions.width,
.height: dimensions.height ,
.profileLevel: kVTProfileLevel_H264_Baseline_AutoLevel
]
}
broadcaster.appendSampleBuffer(sampleBuffer, withType: .video)
}
}) { (error) in }
Also make sure that the screen is updated during streaming. I've noticed that if you're recording a static window with RPScreenRecorder, then it will only update the handler when there's actually new video data to send. For testing I've added a simple UISlider which will update the feed when you move it around.
I've tested it with Facebook Live and I think it should work with other RTMP services too.
The duplicate answer does not works at all
import Cocoa
import AVFoundation
var error: NSError?
println("Hello, Audio!")
var url = NSURL(fileURLWithPath: "/Users/somebody/myfile.mid") // Change to a local midi file
var midi = AVMIDIPlayer(contentsOfURL: url, soundBankURL: nil, error: &error)
if midi == nil {
if let e = error {
println("AVMIDIPlayer failed: " + e.localizedDescription)
}
}
midi.play(nil)
while midi.playing {
// Spin (yeah, that's bad!)
}
I've made a couple of changes to your code but this seems to "work" (we'll get to that)
First off, import the MP3 file to your playground as described in this answer
Then you can use your file like so:
import UIKit
import AVFoundation
print("Hello, Audio!")
if let url = Bundle.main.url(forResource: "drum01", withExtension: "mp3") {
do {
let midi = try AVMIDIPlayer(contentsOf: url, soundBankURL: nil)
midi.play(nil)
while midi.isPlaying {
// Spin (yeah, that's bad!)
}
} catch (let error) {
print("AVMIDIPlayer failed: " + error.localizedDescription)
}
}
Notice:
printinstead of println
In Swift 3 a lot of things was renamed and some of the "old" methods that took an &error parameter was changed to use do try catch instead. Therefore the error has gone from your call and has been replaced with a try.
The above will fail! You will see error code -10870 which can be found in the AUComponent.h header file and which translates to:
kAudioUnitErr_UnknownFileType
If an audio unit uses external files as a data source, this error is returned
if a file is invalid (Apple's DLS synth returns this error)
So...this leads me to thinking you need to do one of two things, either:
find a .midi file and use that with the AVMidiPlayer
find something else to play your file, for instance AVFilePlayer or AVAudioEngine
(you can read more about error handling in Swift here).
Hope that helps you.
The mp3 file must be in the Resources folder.
You play an mp3 with code like this (not the MIDI player):
if let url = Bundle.main.url(forResource: "drum01", withExtension: "mp3") {
let player = try? AVAudioPlayer(contentsOf: url)
player?.prepareToPlay()
player?.play()
}
Scenario:
I wish to reduce the size of individual videos from my iTouch photo library.
1. Collect videoAssets from library.
2. Get a thumbnail of the PHAsset - works.
3. Get the actual video from the library.
4. Request the AVAssetForVideo from the library.
5. Convert the video via ExportSessions... loading assorted parameters.
6. Attempt to run the export into a tmp directory for use.
* FAILS *
Here's the debug output:
Here's the error message:
func getVideoFromPhotoLibrary() {
let videoAssets = PHAsset.fetchAssetsWithMediaType(.Video, options:nil)
videoAssets.enumerateObjectsUsingBlock {
(obj:AnyObject!, index:Int, stop:UnsafeMutablePointer<ObjCBool>) in
let mySize = CGSizeMake(120,120)
let myAsset = obj as! PHAsset
let imageManager = PHImageManager.defaultManager()
var myVideo:BlissMedium?
// Request the poster frame or the image of the video
imageManager.requestImageForAsset(myAsset, targetSize:mySize, contentMode: .AspectFit, options: nil) {
(imageResult, info) in
let thumbnail = UIImage(named:"videoRed")
myVideo = BlissMedium(blissImage: imageResult, creationDate:myAsset.creationDate)
myVideo!.mediumType = .video
}
// Actual Video:
imageManager.requestAVAssetForVideo(myAsset, options: nil, resultHandler: {result, audio, info in
let asset = result as! AVURLAsset
let mediaURL = asset.URL
let session = AVAssetExportSession(asset: asset, presetName: AVAssetExportPresetMediumQuality)
let filename = "composition.mp4"
session.outputURL = NSURL(string: NSTemporaryDirectory());
session.outputFileType = AVFileTypeQuickTimeMovie;
session.exportAsynchronouslyWithCompletionHandler({ () -> Void in
dispatch_async(dispatch_get_main_queue(), {
if session.status == AVAssetExportSessionStatus.Completed {
println("Success")
}
else {
println(session.error?.localizedDescription)
//The requested URL was not found on this server.
}
})
})
})
if nil != myVideo {
self.gBlissVideoMedia.append(myVideo!)
}
}
}
I checked to be sure the target path/file exist; then I added the 'AVFileTypeMPEG4' output type to match the intended .mp4:
let targetDir = createTempDirectory("bliss/composition.mp4") as String?
if NSFileManager.defaultManager().fileExistsAtPath(targetDir!) {
println("*** file exists! ***")
} else {
return
}
session.outputURL = NSURL(string: targetDir!);
session.outputFileType = AVFileTypeMPEG4
I'm still having problems:
* file exists! *
Optional("The operation could not be completed")
What am I doing wrong; what's missing?
Update:
I'm able to successfully run the export to my NSHomeDirectory() vs NSTemporaryDictory() in Objective-C.
However... the same code written in Swift fails.
I notice a change in absolute path to the target output in Swift, not found in Objective-C:
Perhaps it's a Swift 1.2 bug???
I am not sure if you can save in the root of the temp directory, I normally use this function to create a new temp directory that I can use:
func createTempDirectory(myDir: String) -> String? {
let tempDirectoryTemplate = NSTemporaryDirectory().stringByAppendingPathComponent(myDir)
let fileManager = NSFileManager.defaultManager()
var err: NSErrorPointer = nil
if fileManager.createDirectoryAtPath(tempDirectoryTemplate, withIntermediateDirectories: true, attributes: nil, error: err) {
return tempDirectoryTemplate
} else {
return nil
}
}
Try to make your conversion in the directory returned by this function.
I hope that helps you!
I didn't quite understand what that last part of code did, where you find out if a file exists or not. Which file is it you are locating?
Since I didn't understand that then this might be irrelevant, but in your topmost code I notice that you set the filename to composition.mp4, but let the outputURL be NSURL(string: NSTemporaryDirectory()). With my lack of Swiftness I might be missing something, but it seems to me as if you're not using the filename at all, and are trying to write the file as a folder. I believe setting a proper URL might fix the problem but I'm not sure. An Objective-c-example of this could be:
NSURL * outputURL = [[NSURL alloc]
initFileURLWithPath:[NSString pathWithComponents:
#[NSTemporaryDirectory(), #"composition.mp4"]]];
The outputURL is supposed to point to the actual file, not the folder it lies in. I think..
Anyway, if that doesn't work I do have a few other thoughts as well.
Have you tried it on an actual device? There may be a problem with the simulator.
Also, sadly, I have gotten the error -12780 countless times with different root-problems, so that doesn't help very much.
And, I see you check if session.status == AVAssetExportSessionStatus.Completed, have you checked what the actual status is? Is it .Failed, or perhaps .Unknown? There are several statuses.
This might be a long shot, but in one of my apps I am using the camera to capture video/audio, then encode/convert it using AVAssetExportSession. There were strange errors when starting to record, as well as after recording(exporting). I found out that I could change the AVAudioSession, which apparently has something to do with how the device handles media.
I have no idea how to Swift, but here's my code (in viewDidAppear of the relevant view)
NSError *error;
AVAudioSession *aSession = [AVAudioSession sharedInstance];
[aSession setCategory:AVAudioSessionCategoryPlayAndRecord error:&error];
[aSession setMode:AVAudioSessionModeVideoRecording error:&error];
[aSession setActive: YES error: &error];
The category PlayAndRecord allowed me to start the camera much faster, as well as getting rid of the occasional hanging AVAssetExportSessionStatus.Unknown and the occasional crash .Failed (which also threw the -12780-error).