On an ios device, can AVAssetReaderOutput be told to only use software-based decoders (i.e. kAppleSoftwareAudioCodecManufacturer rather than kAppleHardwareAudioCodecManufacturer)?
I see that this is possible using Audio Format Services in AudioToolbox, but I don't see how to carry this over to AVFoundation.
The reason for this is that I'd like to decode compressed audio from the itunes library while iPodMusicPlayer is playing - since hardware-assisted decoding does not support simultaneous decoding of multiple songs, my app will need to use software decoding (right?)
I'd rather not do the software decoding as a 2-step process (i.e. export compressed file to app sandbox, then open that using AudioToolbox).
Well, although I haven't found a way to specify the software decoder in AVFoundation, I ended up working around this by reading each track of the compressed song file with an AVAssetReaderTrackOutput, then passing the compressed buffers to an AudioConverterRef.
Related
I working on a project which involves decoding AAC track from a mp4 file into PCM format. So far, the only way I found that does this is by using AVAssetReader. However, this approach has 2 problems for me:
1) According to the guide, the AVAssetReader is not recommended for real-time processing. However, My project requires live decoding and playback, where the decoded PCMs are post-processed. Will this be a problem? If yes, what will be the alternative?
2) AVAssetReader seems to decode the track sequentially. It does not seem to allow jumping to a random point and decode from there, which is something required by my project. What will be solution?
Answer 1: If you have to deal with track in iPod library, AVAssetReader is the only way.If not, you can choose another decoder like FFmpeg.
Answer 2: AVAssetReader supports random access.It has a timeRange property, see https://stackoverflow.com/a/6719873/1060971.
I've been exploring options on iOS to achieve hardware accelerated decoding of raw H.264 stream and so far I only found that the only option is to write the H.264 stream into an MP4 file and then pass the file to an instance of AVAssetReader. Although this method works, it's not particulary suitable for realtime applications. AVFoundation reference indicates the existence of a CALayer that can display compressed video frames (AVSampleBufferDisplayLayer) and I believe this would be a valid alternative to the method mentioned above. Unfortunately this layer is only available on OSX. I would like to file an enchament radar but before I do so I would like to know from someone that has experience with this layer if indeed could be use to display H.264 raw data if was available on iOS. Currently in my app, the decompressed YUV frames are rendered via openGLES. Using this layer means that I will not need to use openGLES anymore?
In iOS 8 the AVSampleBufferDisplayLayer class is available now.
Take a Look and have Fun
I'm writing an RTSP/H.264 client. Live555 for parsing the RTSP is great, but using ffmpeg for software decoding is just too slow. I'd like to use AVFoundation to hardware decode the samples. I'm not sure how to do this. My question is, is there any way to get AVFoundation (AVAssetReader?) to decode these samples as they come in and display the feed on-screen?
From now the media sample encoded with H264 comes from memory can't use hardware decode, because iOS doesn't open these interfaces, you can only decode local file or by HTTP Live Streaming. However, there is a possible solution that write every sample into a separate mp4 file, then read it with AVAssetReader, but I didn't try that, maybe speed is a limit.
This may at least get you started
https://github.com/mooncatventures-group/FFPlayer-tests
I wish to export a sound recording on iOS from an app into some kind of format that's suitable to be sent over email. Since this leaves a compressed format this leaves out uncompressed wav, and leaves a choice of mp3, ogg, m4a..
What readily available libraries (or even APIs) are available in iOS to do this task?
AVFoundation will do the job for you. In particular, you should look at AVExportSession, which is explicity written for encoding pcm into mp3 or m4a. Sorry, no ogg here. This stuff was available since iOS4.
I really stucked with that problem, because I haven't seen enough information in the internet regarding video encoding in iOS, however we can observe plenty of apps that deal with the problem of video streaming successfully (skype, qik, justin.tv, etc.)
I'm going to develop an application, that should send video frames obtained from camera and encoded in h.263 (h.264 or MPEG-4 it is under decision) to a web-server. For this, I need some video encoding library. Obviously, ffmpeg can deal with that task, but it is under LGPL license, which could probably lead to some problems in submitting the app in the AppStore. On the other hand, there are some applications, which are seemed to use ffmpeg library, but only Timelapser clearly states this fact in app description. Does this mean, that other apps are not using ffmpeg or just hiding this information?
Please, share your thoughts and experience in this topic. I'm open for dicsussion.
After googling and making some research in this area, I found this one library http://www.foxitsolutions.com/iphone_h264_sdk.html. They really use hardware encoding. I've examined demo example with instruments, and they showed me that while encoding, ~12% cpu is used and syscall read() constantly called. From that I can conclude, that their library uses standard AVFoundation's AVAssetWriter to write into the temporary file, and (most probably) concurrent thread is used to read this temp file for retrieving encoded frames.
Also, take a look at http://www.videolan.org/developers/x264.html. It is under GPL, but still can be useful.