I'm trying to stream RTSP/RTP iPhone camera capture to a Wowza server.
Apple's API does not allow direct access to H264 encoded frames, but only allow you to write it into a container '.mov' file.
Either way, I cannot get access to that file content until AVAssetWriter has finished writing, which doesn't allow me to stream live camera capture.
I've tried accessing it using named pipe in order to get access to the file's content on real-time but no success there - AVAssetWriter will not write to an existing file.
Does anyone know how to do it?
Thanks!
Edit: Starting on iOS 8, encoder & decoder has APIs
You can use a AVCaptureVideoDataOutput to process/stream each frame while the camera is running and AVAssetWriter to write the video file at the same time (appending each frame of the video data output queue).
See also
Simultaneous AVCaptureVideoDataOutput and AVCaptureMovieFileOutput
and Can use AVCaptureVideoDataOutput and AVCaptureMovieFileOutput at the same time?
Only solution i've found working so far,
is capturing without sound, then the file is written to the location you've defined.
Otherwise it's probably written to a temp location you can't reach.
Here is Apple's example for capturing video: AVCam
You'll need to remove sound channels.
If anyone has a better way, please publish it here.
Related
So I have put together a sample project https://github.com/liuxuan30/TestH264.git that uses VideoToolBox to have a H264 sample decoder to display a stream file, captured from a camera.
The H264 decoder using VideoToolBox is copied from internet, I didn't write it, when I tried to play my h264 stream file, it plays too fast, comparing to ffmpeg or ffplay, which both played back at a normal speed.
I wanted to ask, how to fix this behaviour? Thanks.
This happens because of this constant kCMSampleAttachmentKey_DisplayImmediately:
If this key is present, the sample should be displayed as soon as possible rather than
according to its presentation timestamp. Use this attachment at run time to request this
behavior from a display pipeline such as the AVSampleBufferDisplayLayer class.
This attachment is not written to media files.
from Apple documation
So you have two options of displaying:
Display immediately - which is probably good for real-time stream, when you need to display frame as soon as possible
Display frames at specific timestamp
*comparing to ffmpeg or ffplay, which both played back at a normal speed.
ffplay and ffmpeg probably use timestamp at this point.
I have same result as you from your test H.264 file, but it's happens because you get all decoded frame at once so decoder is displaying it immediately.
You can watch this video for more information about VideoToolbox framework:
Direct Access to Video Encoding and Decoding
I am trying to write a program for live streaming the audio to the server. Does AVAudioRecorder has the streaming functionality or should I use any other frameworks? Preferably I am trying to use apple builtin frameworks.
I have used AVCaptureSession for streaming audio coupled with AVCaptureDevice as Audio as input device and output device as AVCaptureAudioDataOutput which in turn calls AVCaptureAudioDataOutputSampleBufferDelegate and gives data as a buffered stream.
AVFoundation Cameras and Media Capture
According to this document, you have to initialize an AVAudioRecorder with a file path, which means: if you want to do the live streaming, you have to either wait for the current recording to finish, or initialize a new AVAudioRecorder with another path.
I would recommend you to create multiple AVAudioRecorder instances and run each instance based on the size of the audio chunk. (You can also divide them based on the time, but make sure your buffer is large to keep them all)
And, just upload previous chunks and then start a new instance to keep the recording going on.
This is a two part question:
Using AVAudioRecorder is it possible to have a waveform respond to the incoming audio in real time similar to what happens when you activate siri on the iphone. Perhaps using averagePowerForChannel?
Also, is there a way to gather the audio samples of a recording to render a waveform?
I know novocaine exists, but I was hoping not to use a framework.
Does not seem possible using AVAudioRecorder by itself.
An alternative would be to use AVCaptureSession with an AVCaptureAudioDataOutput which provides access to the raw audio buffer, from which the wave form can be read.
Most of the processing would be done in the delegate:
func captureOutput(AVCaptureOutput!, didOutputSampleBuffer: CMSampleBuffer!, from: AVCaptureConnection!)
You would probably need to implement some sort of throttling to only process every Nth sample so that your visualiser code doesn't interfere with the audio.
AVCaptureSession is far more rudimentary compared to AVAudioRecorder - it does not provide any recording facilities by itself for example, and so if you wanted to also record the audio you would need to use an AVAssetWriter to save the samples.
This SO question shows how to access the sample buffers. It uses AVAssetReader to load a file, but the delegate is exactly the same as would be used for realtime processing:
Reading audio samples via AVAssetReader
Is this possible to access the raw audio PCM data that is being played when using XAudio2 to play file?
I've been searching for several ways to access a decoded version of audio files being played in SL4/Windows Phone, without success.
According to this post someone had success writing a custom XAPO that just grabs samples and is enabled on a Submix Voice. http://social.msdn.microsoft.com/Forums/windowsapps/en-US/05593fad-dfd8-4c77-983b-8c84cd4a324b/xaudio2-saving-output-custom-xapos-slow-down-audio-play-backwards
Please note that if you just want to do this for audio processing this approach is not optimal because you are limited to the speed of audio playback.
Long story short, I am trying to implement a naive solution for streaming video from the iOS camera/microphone to a server.
I am using AVCaptureSession with audio and video AVCaptureOutputs, and then using AVAssetWriter/AVAssetWriterInput to capture video and audio in the captureOutput:didOutputSampleBuffer:fromConnection method and write the resulting video to a file.
To make this a stream, I am using an NSTimer to break the video files into 1 second chunks (by hot-swapping in a different AVAssetWriter that has a different outputURL) and upload these to a server over HTTP.
This is working, but the issue I'm running into is this: the beginning of the .mp4 files appear to always be missing audio in the first frame, so when the video files are concatenated on the server (running ffmpeg) there is a noticeable audio skip at the intersections of these files. The video is just fine - no skipping.
I tried many ways of making sure there were no CMSampleBuffers dropped and checked their timestamps to make sure they were going to the right AVAssetWriter, but to no avail.
Checking the AVCam example with AVCaptureMovieFileOutput and AVCaptureLocation example with AVAssetWriter and it appears the files they generate do the same thing.
Maybe there is something fundamental I am misunderstanding here about the nature of audio/video files, as I'm new to video/audio capture - but thought I'd check before I tried to workaround this by learning to use ffmpeg as some seem to do to fragment the stream (if you have any tips on this, too, let me know!). Thanks in advance!
I had the same problem and solved it by recording audio with a different API, Audio Queue. This seems to solve it, just need to take care of timing in order to avoid sound delay.