I have a video source which gives me a raw h264 stream. I need to re-stream this live input in a way it is cross-compatible and playable without any plugin. I've tried using ffmpeg+ffserver to produce a fragmented mp4, but unfortunately my iPhone isn't playing it.
Is there a way to make it (raw h264 in mp4 container) playable in iOS's Safari, or maybe another cross-platform container?
Ps: i'm using a raspberry pi 3 to host ffmpeg processes, so i'm avoiding re encoding tasks; instead i'm just trying to fit my raw h264 in a "ios-compatible" container and make it accessible through a media server.
For live streams you must use HTTP Live Streaming (HLS) with either the traditional MPEG-TS or fMP4 for newer iOS versions (see Apple HLS examples).
With FFmpeg you can use the hls muxer. The hls_segment_type option is used to choose between mpegts and fmp4.
Related
We are hosting mp3 files on AWS s3. We have built a web app (in React) that will play back the mp3s. However, it sometimes becomes distorted when played in Safari on iOS. The strange thing is that this does not happen all the time.
Here is the original file (sometimes distorted): https://sayyit-prod-static-assets.s3.amazonaws.com/static/audio/Darrin+M.+McMahon.original.mp3
Here is the file sounds when distorted: https://sayyit-prod-static-assets.s3.amazonaws.com/static/audio/WhatsApp+Video+2019-09-26+at+11.06.49+AM.mp4
Now, this distortion only happens when playing it through our app. When we provide a direct link to s3 (like I did above), it works. The distortion also happens when linking directly to s3 in our app.
Here are some ideas:
The mp3 file is broken
When going directly to the S3 link, it downloads entirely, which seems to allow the mp3 file to play perfectly
Any help would be greatly appreciated.
The sample rate on this MP3 file is 16 kHz. That's very low (not abnormal for voice), but also uncharacteristically low for a 128k MP3. I suspect that there's a bug with the resampler (as the iPhone hardware is locked to 48 kHz anyway), or that you're hitting an edge case bug with the decoder.
I'd recommend that you stop using MP3 and solve a few things at once. While MP3 is of acceptable quality, it's quality for a given bitrate isn't as good as alternatives. These days, you should consider using Opus. It's supported on iOS if muxed into a CAF file, and is extremely efficient. You could drop the bitrate down to 48k for voice and still have excellent quality. And, you'll bypass whatever resampling or decoding issue you're having now all in one go.
Is there any good solution of iOS AVPlayer to let user choose specified HLS video resolution / bandwidth?
So the question will be separated in to two
Get the resolution / bandwidth list in m3u8:
Specify the stream resolution and bandwidth
For 1. A workaround solution is to get indicatedBitrate of
AVPlayerItemAccessLogEvent
(Get bandwidth of stream from m3u stream)
The other possible solution is to download and parse m3u8, apart from the AVPlayer interface.
For 2. A workaround solution to change the default adaptive behavior of AVPlayer is to use preferredPeakBitRate or preferredMaximumResolution. But video quality might still get lower if network gets slower. (Change HLS bandwidth manually?)
Thank you.
I am developing a Live Streaming application where mobile application (iOS/Android) records the video and Audio and then encodes raw pixels to h.264 and AAC using VideoToolBox and AudioToolBox, these encoded pixels are converted to PES (Packetized Elementary Stream) separately (Video & Audio). Now we are stuck at what to transfer to the server either PES or MPEG-TS, which one gives the minimum latency and packet loss like Periscope, Meerkat, Qik, UStream and other live streaming applications.
For transmitting which networking protocol is best suitable TCP/UDP.
And what is required at server to receive these packets. I know FFMPEG will trans-code and generates the segmented files(.ts) and .m3u8 file for HLS streaming, but do we need any pipe before FFMPEG layer?
Please give me some ideas in terms of which is best and what are pros and cons of each.
Thanks
Shiva.
I need some advice from people experienced with streaming video.
I have a task to put together a system that allows video coming from RS-170 (composite) video cameras and have them displayed on an iPad. The catch is that no wireless (no Wi-Fi, no bluetooth) is allowed. Only a wired interface.
The physical I/O options on an iPad are apparently extremely limited, but I did manage to come across a company named Redpark that makes an RS232-to-Lightning cable. So my proposed solution is to have the video feeds go into a box with software that digitizes and encodes the video, and then sends it over RS232 to the iPad using that cable. The catch here is that the maximum bandwidth on that cable is 115kbps.
My preliminary testing of this setup on a prototype system have been less than stellar so far. I set up two PCs, each with serial ports, and hooked them together with a null modem. I then set the baud rates of the ports to 115kpbs and then attempted to stream a web cam video feed over the serial connection in real-time using ffmpeg. The results weren't very encouraging, but I at least did manage to get some sort of image to show up.
I guess I need to play around with the ffmpeg encoding options some more. But I need to ask: am I wasting my time with this idea, or should what I am asking here be possible?
For SDA LQ standard ("low quality") we encode H.264 mp4 (using x264) with a 128 kbps video track. The hardware decoding on the iPad can play it. It is maximum 320x240 30 fps video. The quality depends heavily on the material. For mostly nonmoving material, it is watchable. If there is a lot of movement or lighting changes, you may not be able to make out much. You can check out some examples at the link. Video game video, but some may be comparable to your application.
Without knowing more about your requirements (resolution, framerate, type of material), it is difficult to say more. However, given the right material, it is definitely possible to do it and have it be watchable (for some definitions of watchable).
I am writing a client-server application which does real time video transmission from an android based phone to a server. The captured video from the phone camera is encoded using the android provided h264 encoder and transmitted via UDP socket. The frames are not RTP encapsulated. I need it to reduce the overhead and hence the delay.
On the receiver, I need to decode the incoming encoded frame. The data being sent on the UDP socket not only contains the encoded frame but some other information related to the frame as a part of its header. Each frame is encoded as an nal unit.
I am able to retrieve the frames from the received packet as a byte array. I can save this byte array as raw h264 file and playback using vlc and everything works fine.
However, I need to do some processing on this frame and hence need to use it with opencv.
Can anyone help me with decoding a raw h264 byte array in opencv?
Can ffmpeg be used for this?
Short answer: ffmpeg and ffplay will work directly. I remember Opencv can be built on top of those 2. so shouldn`t be difficult to use the FFMEPG/FFSHOW plug in to convert to cv::Mat. Follow the docuemnts
OpenCV can use the FFmpeg library (http://ffmpeg.org/) as backend to
record, convert and stream audio and video. FFMpeg is a complete,
cross-reference solution. If you enable FFmpeg while configuring
OpenCV than CMake will download and install the binaries in
OPENCV_SOURCE_CODE/3rdparty/ffmpeg/. To use FFMpeg at runtime, you
must deploy the FFMepg binaries with your application.
https://docs.opencv.org/3.4/d0/da7/videoio_overview.html
Last time, I have to play with DJI PSDK. And they only allow stream at UDP port udp://192.168.5.293:23003 with H.264
So I wrote a simple ffmpeg interface to stream to the PSDK. But I have to debug it beforehand. So I use ffplay to show this network stream to proof it is working. This is the script to show the stream. So you have to work on top of this to work as opencv plugin
ffplay -f h264 -i udp://192.168.1.45:23003