I have a streaming app which is able to receive a video stream via raw h264 or h265 (HEVC). I render the stream via AVSampleBufferDisplayLayer. I uploaded my app via TestFlight and some users mentioned that the h265 freezes sometimes. I started to investigate the issue and it turned out the h265 stream sometimes freezes but without any errors. Even the AVSampleBufferDisplayLayer status is returning ".rendering" and "isReadyForMoreMediaData" is returning true. The stream only continues after I receive an I-Frame/ Key-Frame.
It is a real time streaming app and the problem occurs when I lose a frame during the streaming session (which can happen regularly). I connect to a 3rd Party streaming server so I can't change I-Frame interval. On h264 when I lose a frame then the stream will get pixelated until I receive the next I-Frame (which is expected) but for some reason the stream freezes when I lose a frame via h265. Are there any settings which I can apply to the "CMSampleBuffer" via the "CFDictionarySetValue" to avoid this?
Here is the code I use currently to render a frame
https://gist.github.com/grill2010/6299f90c5f021f8856474b573ae1fc41
Edit: I built a small sample app to reproduce the problem. It is available here
https://bitbucket.org/grill2010/hevcdecodertest/src/master/
The project will render a HEVC stream (stream length is only a few seconds in total). The sample app includes two buttons "Start decoding" and "Start Decoding with lost frame". If you click "Start decoding" the stream will be rendered correctly but if you click "Start Decoding with lost frame" you see that it will freeze immediately the app simulates a frame loss.
Related
I have to add live video streaming in one of my app; I am using Wowza Streaming Cloud for the same. I am using REST Api to make it as my requirement.
The flow I had used is:
Create a stream
Start stream
Check status unless it is "started"
if the status is started, I start broadcasting video.
It goes well sometimes, but sometimes when I try to broadcast even after starting the stream, it says:
Status is idle, An error occurred when trying to connect to host: (code = 15)
Also I see a green screen on player side, and the video on player side is not continuous, it keeps on fluttering.
For Player I used the code provided in sample app.
For Broadcasting I had used GoCoder SDK wherein I set all of the properties like host address, port, app name, stream name, etc.
Do I have to set bitrate or anything?
Where am I wrong?
That error occurs when the entrypoint itself is open for more than 20 minutes without a connection. Once you get an IP returned from the API for the stream, you can connect to it right away. The errors you're getting are showing idle due to lack of connection and it sounds like the timing between starting the event, checking the event, and then finally connecting are hitting this restriction.
is there any way to send Microphone audio stream to service side in real time?
I am using WCF service at the middle layer where I am converting audio to text using system.speech. It is working fine if I am sending wav file as memory stream but how it possible in a live scenario using the microphone?
I have a CoreAudio based player that streams remote mp3s.
It uses NSURLConnection to retrieve the mp3 data -> uses AudioConverter to convert the stream into PCM -> and feeds the stream into an AUGraph to play audio.
The player works completely fine in my demo app(it only contains a play button), but when i add the player to another project, but when coupled with a project that already makes networking calls, and updates UI, the player fails to play audio past a few seconds.
Am possibly experiencing a threading issue? What are some preventative approaches that i can take or look into that can prevent this from happening?
You do not mention anything in your software architecture about buffering your data between receiving it via NSURLConnection and when you send it to your player.
Data will arrive in chunks with inconsistent arrival rates.
Please see these answers I posted regarding buffering and network jitter.
Network jitter
and
Network jitter and buffering queue
In a nutshell, you can receive data and immediately send it to your player because the next data may not arrive in time.
You don't mention the rate that the mp3 file is delivered. If it is delivered very quickly over a fast connection... are you buffering all of the data received or is it getting lost somewhere in your app? There is a chance that your problem is that you are receiving way too much data too fast and not properly buffering up the data received.
Im using MPMoviePlayerController to stream audio from a server, but after playing the audio for more than two minutes, the audio starts to stop and resume alot, im streaming more than one file one after one, so because of the interruption, some of the audio files are being skipped with those two console messages:
Took background task assertion (38) for playback stall
Ending background task assertion (38) for playback stall
I'm losing a lot of tracks because of this error.
for the first while, i thought that was a memory issue, but the console shows that each time a loose a track, it print those messages,
Check your network connectivity and the stream encoding.
This console output pretty much says exactly what your problem is; the stream dries out of content and could not keep up playing without interruption.
Either your network connection is unstable or the content is encoded in bandwidths that are far too high for your network connection.
For clarification; even if your local internet peering is offering high bandwidths, you should still check the bandwidths of the entire route. For example, you could try to download the streamed files via your browser for testing the throughput.
Are you trying it on a simulator or a device? It may be a simulator issue.
Also, on device, try streaming through multiple networks, e.g., LTE, wifi, etc., see if there is any difference
I have created an iPhone app with recording with AudioUnit, Conversion, Audio Editing and Merging parts. I done everything except Conversion. This app will work only in iOS 4 or higher.
I tried to convert .caf to .m4a file. But I am getting kAudioConverterErr_HardwareInUse error. Then I tried to convert .caf file to .wav file. Then .wav file to .m4a file. But I am getting the same issue.
I am not clear with this issue. In the Apple documentation, they mentioned like ;
"Returned from the AudioConverterFillComplexBuffer function if the underlying hardware codec has become unavailable, probably due to an audio interruption.
On receiving this error, your application must stop calling AudioConverterFillComplexBuffer. You can check the value of the kAudioConverterPropertyCanResumeFromInterruption property to determine if the converter you are using can resume processing after an interruption. If so, then wait for an interruption-ended call from Audio Session Services, reactivate the audio session, and finally resume using the codec.
If the converter cannot resume processing after an interruption, then on interruption you must abandon the conversion, re-instantiate the converter, and perform the conversion again."
Please help me to resolve it.
I just resolved such a problem.
In my case, I have MPMoviePlayerController, audio queue player, audio recorder in the application.
the movie player needs manually calling "stop" method when content ends.
Otherwise the play state is lock at MPMoviePlaybackStatePlaying. Then I can no more play MP3 and get "hwiu" when I try it. But PCM still work.
Maybe it's because the compressed audio (MP3, AAC, ...) is handled by a unique hardware device. If you are using different techniques (MPMoviePlayerController and audio queue service) to playback compressed audio, you need to release the device once after you finish playing since they are all share the same device.