Programmatically convert flv to mp4 on iOS - ios

In our iOS app, we will receive flv**(Container)** file having video and audio streaming something like this
Input #0, flv, from 'test.flv':
Metadata:
streamName : flvLiveStream
encoder : Lavf55.12.100
Duration: 00:00:48.00, start: 66064.401000, bitrate: 632 kb/s
Stream #0:0, 41, 1/1000: **Video: h264 (Baseline)**, 1 reference frame, yuv420p(progressive, left), 1280x720, 0/1, 15 fps, 1k tbr, 1k tbn
Stream #0:1, 14, 1/1000: **Audio: pcm_alaw,** 8000 Hz, mono, s16, 64 kb/s
and this needs to be converted to mp4 container and format as well, I am trying using ffmpeg, I believe thats only the way , using transcoding.c file but failed at this stage
Impossible to convert between the formats supported by the filter 'in' and the filter 'auto_scaler_0'
I am trying to learn from OSX command like ffmpeg -I test.flv test.mp4 ,
is that feasible to port to iOS
will it work in all different scenarios,
In Summary
--- What is the best possible way to convert flv to mp4 on the iOS device
where video will be in h264 and audio will be in swf codec ?

NSArray *paths = NSSearchPathForDirectoriesInDomains(NSDocumentDirectory, NSUserDomainMask, YES);
NSString *documentsDirectory = [paths objectAtIndex:0];
NSString *filePath = [documentsDirectory stringByAppendingPathComponent:#"MyVideo.mp4"];
NSURL *outputURL = [NSURL fileURLWithPath:filePath];
[self convertVideoToLowQuailtyWithInputURL:localUrl outputURL:outputURL handler:^(AVAssetExportSession *exportSession)
{
if (exportSession.status == AVAssetExportSessionStatusCompleted) {
// Video conversation completed
}
}];
- (void)convertVideoToLowQuailtyWithInputURL:(NSURL*)inputURL outputURL:(NSURL*)outputURL handler:(void (^)(AVAssetExportSession*))handler {
[[NSFileManager defaultManager] removeItemAtURL:outputURL error:nil];
AVURLAsset *asset = [AVURLAsset URLAssetWithURL:inputURL options:nil];
AVAssetExportSession *exportSession = [[AVAssetExportSession alloc] initWithAsset:asset presetName:AVAssetExportPresetPassthrough];
exportSession.outputURL = outputURL;
exportSession.outputFileType = AVFileTypeMPEG4;
[exportSession exportAsynchronouslyWithCompletionHandler:^(void) {
handler(exportSession);
}];
}

Related

iOS Convert AVI to MP4

(Please note I've already looked at this other SO post.)
The Problem
I'm trying to convert an avi video to an mp4 so that I can play it natively on an iOS app using Objective-C
What I've Tried
I'm trying the following code to do that conversion:
- (void)convertVideoToLowQuailtyWithInputURL:(NSURL*)inputURL outputURL:(NSURL*)outputURL handler:(void (^)(AVAssetExportSession*))handler {
[[NSFileManager defaultManager] removeItemAtURL:outputURL error:nil];
AVURLAsset *asset = [AVURLAsset URLAssetWithURL:inputURL options:nil];
AVAssetExportSession *exportSession = [[AVAssetExportSession alloc] initWithAsset:asset presetName:AVAssetExportPresetHighestQuality];
exportSession.outputURL = outputURL;
exportSession.outputFileType = AVFileTypeMPEG4;
[exportSession exportAsynchronouslyWithCompletionHandler:^(void) {
handler(exportSession);
}];
}
The error that is returned from the exportSession is Cannot Open
Extra Information
When I run the video that I'm trying to convert through Mediainfo I get the following for the video:
7 332kb/s, 1920*1080 (16:9), at 25.000 FPS, AVC (Baseline#L3.1) (CABAC / 1 Ref Frames)
And this for the audio:
128 kb/s, 8 000 Hz, 16 bits, 1 channel, PCM (Little / Signed)
I also used the exportPresetsCompatibleWithAsset: method on AVAssetExportSession and got the following results:
AVAssetExportPreset1920x1080,
AVAssetExportPresetLowQuality,
AVAssetExportPresetAppleM4A,
AVAssetExportPresetHEVCHighestQuality,
AVAssetExportPreset640x480,
AVAssetExportPreset3840x2160,
AVAssetExportPresetHEVC3840x2160,
AVAssetExportPresetHighestQuality,
AVAssetExportPreset1280x720,
AVAssetExportPresetMediumQuality,
AVAssetExportPreset960x540,
AVAssetExportPresetHEVC1920x1080
Another thing to note is that when playing with the preset and the output I managed to get an audio only file that was basically white noise. This was using the preset AVAssetExportPresetAppleM4A.
I hope that I've jotted down enough information.
Update
Using the comment by Ashley, i've created a function to return the export settings compatible with the asset.
- (void)determineCompatibleExportForAsset:(AVURLAsset *)asset completion:(void(^)(NSArray<ExportSettings *> *exports))handler {
NSArray<NSString *> *presets = #[
AVAssetExportPresetLowQuality,
AVAssetExportPresetMediumQuality,
AVAssetExportPresetHighestQuality,
AVAssetExportPresetHEVCHighestQuality,
AVAssetExportPreset640x480,
AVAssetExportPreset960x540,
AVAssetExportPreset1280x720,
AVAssetExportPreset1920x1080,
AVAssetExportPreset3840x2160,
AVAssetExportPresetHEVC1920x1080,
AVAssetExportPresetHEVC3840x2160,
AVAssetExportPresetAppleM4A,
AVAssetExportPresetPassthrough
];
NSArray<NSString *> *outputs = #[
AVFileTypeQuickTimeMovie,
AVFileTypeMPEG4,
AVFileTypeAppleM4V,
AVFileTypeAppleM4A,
AVFileType3GPP,
AVFileType3GPP2,
AVFileTypeCoreAudioFormat,
AVFileTypeWAVE,
AVFileTypeAIFF,
AVFileTypeAIFC,
AVFileTypeAMR,
AVFileTypeMPEGLayer3,
AVFileTypeSunAU,
AVFileTypeAC3,
AVFileTypeEnhancedAC3,
AVFileTypeJPEG,
AVFileTypeDNG,
AVFileTypeHEIC,
AVFileTypeAVCI,
AVFileTypeHEIF,
AVFileTypeTIFF
];
__block int counter = 0;
int totalCount = (int)presets.count * (int)outputs.count;
NSMutableArray<ExportSettings *> *exportSettingsArray = [#[] mutableCopy];
for (NSString *preset in presets) {
for (NSString *output in outputs) {
[AVAssetExportSession determineCompatibilityOfExportPreset:preset withAsset:asset outputFileType:output completionHandler:^(BOOL compatible) {
if (compatible) {
ExportSettings *exportSettings = [[ExportSettings alloc] initWithPreset:preset outputType:output];
[exportSettingsArray addObject:exportSettings];
}
counter++;
if (counter == totalCount) {
if (handler) {
handler([exportSettingsArray copy]);
}
}
}];
}
}
}
The results of this are as follows:
"Preset: AVAssetExportPresetAppleM4A Output: com.apple.m4a-audio",
"Preset: AVAssetExportPresetPassthrough Output: com.microsoft.waveform-audio",
"Preset: AVAssetExportPresetPassthrough Output: public.aifc-audio",
"Preset: AVAssetExportPresetPassthrough Output: public.aiff-audio",
"Preset: AVAssetExportPresetPassthrough Output: com.apple.coreaudio-format",
"Preset: AVAssetExportPresetPassthrough Output: com.apple.quicktime-movie"
From this I deduced that using the preset AVAssetExportPresetPassthrough and output type AVFileTypeQuickTimeMovie would be compatible.
However when running the following code: (i've tried .mp4, .mov and .qt for the file type)
NSArray *paths = NSSearchPathForDirectoriesInDomains(NSDocumentDirectory, NSUserDomainMask, YES);
NSString *documentsDirectory = [paths objectAtIndex:0];
NSString *filePath = [documentsDirectory stringByAppendingPathComponent:#"MyVideo.mov"];
NSURL *outputURL = [NSURL fileURLWithPath:filePath];
NSURL *localURL = [NSBundle URLForResource:#"20180626_145233-v" withExtension:#"avi" subdirectory:nil inBundleWithURL:[NSBundle mainBundle].bundleURL];
[self convertVideoToLowQuailtyWithInputURL:localURL outputURL:outputURL handler:^(AVAssetExportSession *exportSession) {
switch ([exportSession status]) {
case AVAssetExportSessionStatusFailed:
NSLog(#"Export failed: %#", [exportSession error]);
break;
case AVAssetExportSessionStatusCancelled:
NSLog(#"Export canceled");
break;
case AVAssetExportSessionStatusCompleted:
NSLog(#"Successfully");
NSLog(#"OutputURL: %#", outputURL.absoluteString);
break;
default:
break;
}
}];
Which calls:
- (void)convertVideoToLowQuailtyWithInputURL:(NSURL*)inputURL outputURL:(NSURL*)outputURL handler:(void (^)(AVAssetExportSession*))handler {
[[NSFileManager defaultManager] removeItemAtURL:outputURL error:nil];
AVURLAsset *asset = [AVURLAsset URLAssetWithURL:inputURL options:nil];
AVAssetExportSession *exportSession = [[AVAssetExportSession alloc] initWithAsset:asset presetName:AVAssetExportPresetPassthrough];
exportSession.outputURL = outputURL;
exportSession.outputFileType = AVFileTypeQuickTimeMovie;
[exportSession exportAsynchronouslyWithCompletionHandler:^(void) {
handler(exportSession);
}];
}
I get this error:
Export failed: Error Domain=AVFoundationErrorDomain Code=-11800 "The operation could not be completed" UserInfo={NSLocalizedFailureReason=An unknown error occurred (-12842), NSLocalizedDescription=The operation could not be completed, NSUnderlyingError=0x60400024def0 {Error Domain=NSOSStatusErrorDomain Code=-12842 "(null)"}}
The reason you cannot open the file is because iOS does not support AVI files.
Here is the link to the Apple documentation on supported file types within AVFoundation.
Or this image for posterity should the values change:
Finally, you can just inspect the definition of AVFileType from within XCode to see this list for yourself.
For what it is worth, a cursory inspection on AVI via Google indicates that there are limitations with the AVI container that have been rectified in newer formats that are supported by iOS, so there is likely little incentive to add support at a later date.
While also a potential issue being that Microsoft created AVI, I cannot locate any restrictive licensing that would prohibit someone from using it, but IANAL.
You should use FFmpeg library for that kind of purposes.
Here is an open-source video-player based on FFmpeg: https://github.com/kolyvan/kxmovie
I don't know if this player still works for the latest iOS but in any case in source codes, you can discover many interesting things that can help you with your problem (such as how to use FFmpeg library).
Very easy to do with MobileFFMpeg
https://stackoverflow.com/a/59325680/1466453
Once you have MobileFFMpeg then make a call as follows:
[MobileFFMpeg execute:#"-i infile.avi youroutput.mp4"];

AVAssetWriter convert aac 5.1 audio track from AVAsset fail on appendSampleBuffer

I am trying to extract audio track from .mp4 video file and convert to .m4a audio file with this outputSettings using AVAssetWriter class
AudioChannelLayout channelLayout;
memset(&channelLayout, 0, sizeof(AudioChannelLayout));
channelLayout.mChannelLayoutTag = kAudioChannelLayoutTag_Stereo;
NSDictionary *outputSettings = [NSDictionary dictionaryWithObjectsAndKeys:
[NSNumber numberWithInt: kAudioFormatMPEG4AAC], AVFormatIDKey,
[NSNumber numberWithFloat:44100.0], AVSampleRateKey,
[NSNumber numberWithInt:2], AVNumberOfChannelsKey,
[NSNumber numberWithInt:128000], AVEncoderBitRateKey, // 128 kbps
[NSData dataWithBytes:&channelLayout length:sizeof(AudioChannelLayout)], AVChannelLayoutKey,
nil];
Case 1: .mp4 video file with audio track params (print with CMFormatDescriptionRef):
mediaType:'soun'
mediaSubType:'aac '
mSampleRate: 44100.000000
mFormatID: 'aac '
mChannelsPerFrame: 2
ACL: {Stereo (L R)}
result: successfully create .m4a output file with defined output params
Case 2: .mp4 video file with audio track params(print with CMFormatDescriptionRef):
mediaType:'soun'
mediaSubType:'aac '
mSampleRate: 48000.000000
mFormatID: 'aac '
mChannelsPerFrame: 6
ACL: {5.1 (C L R Ls Rs LFE)}
result: convert fails, when adding sample buffer [AVAssetWriter appendSampleBuffer: ...] with unknown error:
Error Domain: NSOSStatusErrorDomain
code: -12780
description: The operation could not be completed
To convert video to audio I use the same algorithm as described there: https://github.com/rs/SDAVAssetExportSession/blob/master/SDAVAssetExportSession.m
Also I tried to setup channelLayout.mChannelLayoutTag with kAudioChannelLayoutTag_MPEG_5_1_D, and update AVNumberOfChannelsKey with 6 value, but it doesn't work for me.
Can anyone help me to understand what I am doing wrong? May be there are no solution to perform this task using only iOS AVFoundation framework? Should I use different outputParams for audio track with 5.1 aac 6 channels?
I haven't tried this with 5.1 audio, but when extracting audio from video I like to use a passthrough AVAssetExportSession on an audio-only AVMutableComposition as this avoids transcoding which would be slow and throw away audio quality. Something like:
AVMutableComposition* newAudioAsset = [AVMutableComposition composition];
AVMutableCompositionTrack* dstCompositionTrack;
dstCompositionTrack = [newAudioAsset addMutableTrackWithMediaType:AVMediaTypeAudio preferredTrackID:kCMPersistentTrackID_Invalid];
AVAsset* srcAsset = [AVURLAsset URLAssetWithURL:srcURL options:nil];
AVAssetTrack* srcTrack = [[srcAsset tracksWithMediaType:AVMediaTypeAudio] objectAtIndex:0];
CMTimeRange timeRange = srcTrack.timeRange;//CMTimeRangeMake(kCMTimeZero, srcAsset.duration);
NSError* error;
if(NO == [dstCompositionTrack insertTimeRange:timeRange ofTrack:srcTrack atTime:kCMTimeZero error:&error]) {
NSLog(#"track insert failed: %#\n", error);
return 1;
}
__block AVAssetExportSession* exportSesh = [[AVAssetExportSession alloc] initWithAsset:newAudioAsset presetName:AVAssetExportPresetPassthrough];
exportSesh.outputFileType = AVFileTypeAppleM4A;
exportSesh.outputURL = dstURL;
[exportSesh exportAsynchronouslyWithCompletionHandler:^{
AVAssetExportSessionStatus status = exportSesh.status;
NSLog(#"exportAsynchronouslyWithCompletionHandler: %i\n", status);
if(AVAssetExportSessionStatusFailed == status) {
NSLog(#"FAILURE: %#\n", exportSesh.error);
} else if(AVAssetExportSessionStatusCompleted == status) {
NSLog(#"SUCCESS!\n");
}
}];
I don't have any 5.1 files on hand, so if that doesn't work you may need to look more closely at the line
AVAssetTrack* srcTrack = [[srcAsset tracksWithMediaType:AVMediaTypeAudio] objectAtIndex:0];
p.s. this code from 2012 "just worked", which was nice.

Not able to save .mp4 video to camera roll

I have am iOS DVR application where i record and save video in .mp4 format in the document directory.
I am trying to move that video to camera roll but i am not able to move it.
I think it might be because of video format. Below is format of the video
H264 - MPEG-4 AVC
Resolution: 352 x 258
Frame rate: 15
I tried UISaveVideoAtPathToSavedPhotosAlbum but no luck.
I also tried to convert the video to another .mp4 using asset library using following code but status is always Failed.
[self convertVideoToLowQuailtyWithInputURL:localUrl outputURL:outputURL handler:^(AVAssetExportSession *exportSession)
{
if (exportSession.status == AVAssetExportSessionStatusCompleted) {
// Video conversation completed
}
}];
- (void)convertVideoToLowQuailtyWithInputURL:(NSURL*)inputURL outputURL:(NSURL*)outputURL handler:(void (^)(AVAssetExportSession*))handler {
[[NSFileManager defaultManager] removeItemAtURL:outputURL error:nil];
AVURLAsset *asset = [AVURLAsset URLAssetWithURL:inputURL options:nil];
AVAssetExportSession *exportSession = [[AVAssetExportSession alloc] initWithAsset:asset presetName:AVAssetExportPresetPassthrough];
exportSession.outputURL = outputURL;
exportSession.outputFileType = AVFileTypeMPEG4;
[exportSession exportAsynchronouslyWithCompletionHandler:^(void) {
handler(exportSession);
}];
}
How can I save recorded video file to camera roll ?

Export 2 audio tracks (running at the same time) to one file m4a)

I'm developing an ios audio app on xcode and I'm trying to use 2 audio files I have recorded - which are playing at the same time and export it to one audio file.
All I have managed to do is merge 2 audio files to one, but the 2 audios are playing one after another and not in sync at the same time.
Does anyone have a clue how I can sort it out?
Thanks
You should take a look at this for AAC conversion (http://atastypixel.com/blog/easy-aac-compressed-audio-conversion-on-ios/). It's super useful.
Another thing you might want to consider... combining two audio signals is as easy as adding the samples together. So what you could do is:
Open both recordings and get an array for each of the recordings that holds the audio samples.
Make a for() loop that adds each sample and puts it in an output array
for(int i = 0; i<numberOfSamples; i++) {
exportBuffer[i] = firstTrack[i] + secondTrack[i];
}
and then write the exportBuffer to an m4a file.
This code will only work if the two files are the same exact length, so adjust it to your needs. You'll need to add a conditional that fires if you've reached the end of one of the arrays. In that case, just add 0's.
Try Apple's MixerHost sample app.
/* Implement this method if you have already saved your recorded audio file */
-(void)mixAudio{
AVMutableComposition *composition = [[AVMutableComposition alloc] init];
AVMutableCompositionTrack *compositionAudioTrack = [composition addMutableTrackWithMediaType:AVMediaTypeAudio preferredTrackID:kCMPersistentTrackID_Invalid];
[compositionAudioTrack setPreferredVolume:0.8];
NSString *soundOne =[[NSBundle mainBundle]pathForResource:#"RecordAudio1" ofType:#"wav"];
NSURL *url = [NSURL fileURLWithPath:soundOne];
AVAsset *avAsset = [AVURLAsset URLAssetWithURL:url options:nil];
NSArray *tracks = [avAsset tracksWithMediaType:AVMediaTypeAudio];
AVAssetTrack *clipAudioTrack = [tracks objectAtIndex:0];
[compositionAudioTrack insertTimeRange:CMTimeRangeMake(kCMTimeZero, avAsset.duration) ofTrack:clipAudioTrack atTime:kCMTimeZero error:nil];
AVMutableCompositionTrack *compositionAudioTrack1 = [composition addMutableTrackWithMediaType:AVMediaTypeAudio preferredTrackID:kCMPersistentTrackID_Invalid];
[compositionAudioTrack setPreferredVolume:0.8];
NSString *soundOne1 =[[NSBundle mainBundle]pathForResource:#"RecordAudio2" ofType:#"wav"];
NSURL *url1 = [NSURL fileURLWithPath:soundOne1];
AVAsset *avAsset1 = [AVURLAsset URLAssetWithURL:url1 options:nil];
NSArray *tracks1 = [avAsset1 tracksWithMediaType:AVMediaTypeAudio];
AVAssetTrack *clipAudioTrack1 = [tracks1 objectAtIndex:0];
[compositionAudioTrack1 insertTimeRange:CMTimeRangeMake(kCMTimeZero, avAsset1.duration) ofTrack:clipAudioTrack1 atTime: kCMTimeZero error:nil];
AVAssetExportSession *exportSession = [AVAssetExportSession
exportSessionWithAsset:composition
presetName:AVAssetExportPresetAppleM4A];
if (nil == exportSession) return NO;
NSString *soundOneNew = [documentsDirectory stringByAppendingPathComponent:#"combined10.m4a"];
//NSLog(#"Output file path - %#",soundOneNew);
// configure export session output with all our parameters
exportSession.outputURL = [NSURL fileURLWithPath:soundOneNew]; // output path
exportSession.outputFileType = AVFileTypeAppleM4A; // output file type
// perform the export
[exportSession exportAsynchronouslyWithCompletionHandler:^{
if (AVAssetExportSessionStatusCompleted == exportSession.status) {
NSLog(#"AVAssetExportSessionStatusCompleted");
} else if (AVAssetExportSessionStatusFailed == exportSession.status) {
// a failure may happen because of an event out of your control
// for example, an interruption like a phone call comming in
// make sure and handle this case appropriately
NSLog(#"AVAssetExportSessionStatusFailed");
} else {
NSLog(#"Export Session Status: %d", exportSession.status);
}
}];
}

iOS Extracting Audio from .mov file

I've been trying to extract audio from a .mov file for a while now and I just can't seem to get it working. Specifically, I need to extract the audio and save it as an .aif or .aiff file .
I've tried using an AVMutableComposition, and loading the mov file as a AVAsset. Adding only the audio track to the AVMutableComposition before finally using an AVAssetExportSession (setting the output file type to AVFileTypeAIFF, which is the format I need it in), to write the file to an aif.
I get an error saying that this output file type is invalid, I'm unsure why:
* Terminating app due to uncaught exception 'NSInvalidArgumentException', reason: 'Invalid output file type'
AVAssetExportSession *exporter;
exporter = [[AVAssetExportSession alloc] initWithAsset:composition presetName:AVAssetExportPresetHighestQuality] ;
exporter.audioMix = audioMix;
exporter.outputURL=[NSURL fileURLWithPath:filePath];
exporter.outputFileType=AVFileTypeAIFF; //Error occurs on this line
I'm not sure if the above approach would work, but im open to the possibility that I'm just doing something wrong. However if anyone knows another way to accomplish what I'm trying to achieve, than any help would be greatly appreciated.
I can post more detailed code if it is needed, but at the moment I'm trying a few other approaches so its a bit messy right now.
Thanks for the help!
-(void)getAudioFromVideo {
float startTime = 0;
float endTime = 10;
[super viewDidLoad];
NSArray *paths = NSSearchPathForDirectoriesInDomains(NSDocumentDirectory,NSUserDomainMask, YES);
NSString *documentsDirectory = [paths objectAtIndex:0];
NSString *audioPath = [documentsDirectory stringByAppendingPathComponent:#"soundOneNew.m4a"];
AVAsset *myasset = [AVAsset assetWithURL:[[NSBundle mainBundle] URLForResource:#"VideoName" withExtension:#"mp4"]];
AVAssetExportSession *exportSession=[AVAssetExportSession exportSessionWithAsset:myasset presetName:AVAssetExportPresetAppleM4A];
exportSession.outputURL=[NSURL fileURLWithPath:audioPath];
exportSession.outputFileType=AVFileTypeAppleM4A;
CMTime vocalStartMarker = CMTimeMake((int)(floor(startTime * 100)), 100);
CMTime vocalEndMarker = CMTimeMake((int)(ceil(endTime * 100)), 100);
CMTimeRange exportTimeRange = CMTimeRangeFromTimeToTime(vocalStartMarker, vocalEndMarker);
exportSession.timeRange= exportTimeRange;
if ([[NSFileManager defaultManager] fileExistsAtPath:audioPath]) {
[[NSFileManager defaultManager] removeItemAtPath:audioPath error:nil];
}
[exportSession exportAsynchronouslyWithCompletionHandler:^{
if (exportSession.status==AVAssetExportSessionStatusFailed) {
NSLog(#"failed");
}
else {
NSLog(#"AudioLocation : %#",audioPath);
}
}];
}
Because the outputFileType is wrong. For .mov file, it often is #"com.apple.quicktime-movie". And the audio extracted is .mov format. To make sure, you can use this method to get the supported output type:
NSArray *supportedTypeArray=exportSession.supportedFileTypes;
for (NSString *str in supportedTypeArray)
NSLog(#"%#",str);
And you can export audio in audio format (.m4a, can be played by AVAudioPlayer) by using methods like this:
AVAssetExportSession *exportSession=[AVAssetExportSession exportSessionWithAsset:self.mAsset presetName:AVAssetExportPresetAppleM4A];
exportSession.outputURL=myUrl;
exportSession.outputFileType=AVFileTypeAppleM4A;
exportSession.timeRange=timeRange;
[exportSession exportAsynchronouslyWithCompletionHandler:^{
if (exportSession.status==AVAssetExportSessionStatusFailed) {
NSLog(#"failed");
}
}];
Just small info for others (I did not know that..)
If you have video file(in my case it was mp4),You can play only the audio with AVAudioPlayer.
You don't need to extract(or convert) the audio from the video.

Resources