Imagine i have a server which will have hardware to play games on it and it will stream the game itself to the client. Client also will send controls(keyboard & mouse) to the server. (Like Nvidia Grid).
How can i calculate the bandwith required(approximate) on client?
(Game video should be uncompressed)
For example;
-1080p #120Fps
-2K #60Fps
I just want to know how can i calculate(Math behind it) the bandwith requirement(Video+Controls). I know uncompressed video will require huge bandwith.
Uncompressed...
1920*1080*60*3 (24bit color) per frame = 373248000 bytes/sec. For video.
Over 2Gb/sec. Assuming a raw datagram.
Related
I am trying to create a RTSP client which live broadcast Audio and Video. I modified the iOS code at link http://www.gdcl.co.uk/downloads.htm and able to broadcast the Video to server properly. But now i am facing issues in broadcasting the audio part. In the link example the code is written in such a way that it writes the Video data to file and than reads the data from the file and upload the NALU's video packets to RTSP server.
For Audio part i am not sure how to proceed on it. Right now what i have tried is that get the audio buffer from mic and than broadcast it to the server directly by adding RTP headers and ALU.. but This approach is not properly working as Audio starts lagging behind and lag increases with time. Can someone let me know if there is some better approach to achieve this and with lip sycn audio/video.
Are you losing any packets on the client? If so, you need to leave "space." If you receive packet 1,2,3,4,6,7, You need to leave space for the missing packet (5).
The other possibility is a what is known as a clock drift problem. The clock (crystal) on your client and server are not perfectly in sync with each other.
This can be caused by environment, temperature changes, etc.
Let's say in a perfect world your server is producing audio samples 20ms audio samples at 48000 hz. Your client is playing them back using a sample rate of 48000 hz. Realistically your client and server are not exactly 48000hz. Your server might be 48000.001 and your client might be 47999.9998. So your server might be delivering faster than your client or vise versa. You would either consume packets too fast and under run the buffer or lag too far behind and overflow the client buffer. In your case, it sounds like the client is playing back too slow and slowly lagging behind the server. You might only lag a couple milliseconds per minute but the issue will keep continuing and it will look like a 1970s lip synced Kung Fu movie.
In other devices, there is often a common clock line to keep things in sync. For example, Video camera clocks, midi clocks. multitrack recorder clocks.
When you deliver data over IP, there is no common clock shared between a client and server. So your issue concerns syncing clocks between disparate devices with no. I have successfully solved this problem using this general approach:
A) Let the client count the rate of packets that come in over a period of time.
B) Let the client count the rate that the packets are consumed (played back).
C) Adjust the sample rate of the client based on A and B.
So your client requires that you adjust the sample rate of the playback. So yes you play it faster or slower. Note that the playback rate change will be very very subtle. You might set the sample rate to be 48000.0001 hz instead of 48000 hz. The difference in pitch would be undetectable by humans as it would only cause a fraction a cent difference in pitch. I gave an explanation of a very simplified approach. There many other nuances and edge cases that must be considered when developing such a control system. You don't just set it and forget it. You need a control system to manage the playback.
An interesting test to demonstrate this is to take two devices with the exact same file. A long recording (say 3 hours) is best. Start them at the same time. After 3 hours of playback, you will notice that one is ahead of the other.
This post explains that it is NOT a trivial task to stream audio and video.
We have an FFMPEG stream being streamed to mobile devices. We're using the HTML5 <video src="..." webkit-playsinline> tag to display the video inline (inside a real-time streaming app). We've managed to reduce the delay at the FFMPEG end down to the minimum but there's still a lag at the iOS end, where the player presumably buffers for a couple of seconds.
Is there any way to reduce the client-side delay?
We need as close to real-time as possible and skipping is acceptable.
If you are using an HTML5 video tag then the iOS device will use Quicktime to playback the video. Apple offers no control over internal mechanism like buffer settings for its Quicktime player. For a project on Apple TV I even work with a guy in Cupertino at Apple and they just won't allow any access to the information you would require on their device.
Typically if you use HLS:
Is this a real-time delivery system?
No. It has inherent latency corresponding to the size and duration of the media files containing stream segments. At least one segment must fully download before it can be viewed by the client, and two may be required to ensure seamless transitions between segments. In addition, the encoder and segmenter must create a file from the input; the duration of this file is the minimum latency before media is available for download. Typical latency with recommended settings is in the neighborhood of 30 seconds.
What is the latency?
Approximately 30 seconds, with recommended settings. See question #15.
For live streaming scenario on iOS you better off tuning the streaming chain before the actual player:
capture -> transcoding -> upload -> streaming server -> delivery -> playback
Using ffmpeg you can tune for zero lantency streaming at transcoding level which I understand you have done. After that using a well established streaming server like Wowza and CDN delivery will help you get there (of course at a certain cost - and assuming you need a streaming server which you may not).
If you go all native for your iOS app you may look at MPMoviePlayerController. I have no experience with native app code in iOS so I let you decide if it is worth the time (still I doubt it will be possible because of the underlying Quicktime/HLS layer).
I also came across this which sounds interesting but I have not tested it and even with such an approach you will face limitations.
Even if it may not be the answer you were looking for I hope this helps.
I have an HD video that I am streaming to an iOS app. I want to allow the user the ability to cap the max stream quality (low, medium, high) considering the video is several GBs when streaming at the max bit rate. Along the same lines, I would like to automatically choose a setting based on cellular vs wifi connection, for the obvious data-cap reasons.
I have no problem getting the current bit rate by accessing the AVPlayerItemAccessLogEvent, but am lost when it comes to forcing a lower quality stream.
Is this even possible with HLS? Thanks!
If you are using AVPlayer, the right way should be
preferredPeakBitRate
From Apple doc here, The desired limit, in bits per second, of network bandwidth consumption for this item.
It's not exactly dynamic, but I did solve this problem by creating four different m3u8 playlists. I labeled each playlist to represent a stream quality (low, medium, high, extreme). The user would select one based on the desired max quality. The extreme playlist includes the URLs of all qualities. The high playlist has less URLs than the extreme, the medium less URLs than the high, and the low less URLs than the medium. Whenever the user selects a different quality, I would just switch the base stream playlist to the respective quality playlist URL.
Here is a simple example of the four different playlists.
HLS_Movie_Extreme.m3u8
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=64000
stream-0-64000/prog_index.m3u8
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=350000
stream-1-350000/prog_index.m3u8
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=800000
stream-2-800000/prog_index.m3u8
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=1200000
stream-3-1200000/index prog_index.m3u8 m3u8
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=1800000
stream-4-1800000/prog_index.m3u8
HLS_Movie_High.m3u8
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=64000
stream-0-64000/prog_index.m3u8
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=350000
stream-1-350000/prog_index.m3u8
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=800000
stream-2-800000/prog_index.m3u8
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=1200000
stream-3-1200000/index prog_index.m3u8 m3u8
HLS_Movie_Medium.m3u8
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=64000
stream-0-64000/prog_index.m3u8
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=350000
stream-1-350000/prog_index.m3u8
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=800000
stream-2-800000/prog_index.m3u8
HLS_Movie_Low.m3u8
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=64000
stream-0-64000/prog_index.m3u8
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=350000
stream-1-350000/prog_index.m3u8
Like I said, it's not dynamic, but you could use various techniques to get the users network connection and point to the desired quality playlist if needed. For me, it was sufficient to get the user's preference, and adjust the stream accordingly.
How to serve videos like Youtube does ? Even if the video is long (almost 2 hours long) and is viewed in HD, it would almost instantly play and seeking to not yet loaded parts are very fast.
I'm using a dedicated server from Rackspace with 100Mb up/down for this test, my ping time is below 50ms to the server. My local internet connection is 10Mb, I could maximize my internet connection when I download something from the server so connection to the server is not the issue here.
I'm trying to emulate this and I've tried Real time streaming using Wowza and Pseudostreaming using the H264 Streaming Module. Neither could compare to how fast Youtube delivers video.
Video test file is MP4 (h.264), 300MB, 2 hours long, total bitrate is set to 500kbps, and JWPlayer as the video player
Wowza Streaming (RTMP) - Loading then playing the video is fast, but not as fast as youtube. Seeking is not as fast as well it takes
around 5 - 7 seconds to move to the new position and continue playing the video.
Pseudostreaming H264 Streaming Module (HTTP) - Loading the video takes a long time since its downloading the video header first before
playing it. A 2 hours video has around 2.5MB of MOOV ATOM (video
header file) that it needs to download first before it could play.
Once it starts playing seeking to not downloaded parts is on par with
Wowza but not as fast as Youtube.
What do I need to serve videos with the speed of Youtube? I also need it to buffer/download the video when paused just like Youtube so Real Streaming like Wowza is out.
Pseudostreaming using the H264 Streaming module would have been nice since it does buffer when paused, its just that the initial loading time is very long! Anyway I could remove that initial load time?
What are my other options? I'm open to any other option that I could use in my server.
The way YouTube works is different and they keep on changing the way it works. Doing the reverse engineering on that by capturing the YouTube feeds over wire-shark over last 4 years told me that the pattern is very dynamic. The segmentation is one key, the dual buffer, multiple caching servers and techniques, using the client machine as the buffer render and the functionalities of the player matters a lot. There are many many factors which make YouTube video fast and sleek.
You can emulate the same to some extent but building exactly the same needs loads of efforts and infrastructure.
I am receiving a MJPEG Stream from my camera. When I look at the video data with an hex editor it seems that it doesn't contain any streaming information. I just see one raw JPEG after another, but no information about the framerate etc. .
Is the lack of any meta information normal for MJPEG or is it just related to the camera I am using? If there a no information about the stream, how can a player know how fast to play the video?
The lack of metadata is normal. IP Cameras typically send MJPEG as just that, one JPEG image after another as a stream. This is the most basic valid MJPEG file. If you were to take a bunch of jpegs, cat them together into a large, giant file, and feed it to ffmpeg, it would see it as a valid mjpeg format file. Some cameras will add an additional header to contain audio data, but it is not needed to be considered valid motion jpeg.
Many cameras will include a header like X-Framerate, in the HTTP header when the stream is initially sent, or you can set it as part of the camera configuration. However, when a camera sends only jpegs, there is no way to tell from the stream itself what the framerate is.
Is the lack of any meta information normal for MJPEG or is it just related to the camera I am using? If there a no information about the stream, how can a player know how fast to play the video?
To add to already answered, IP camera is a live video source and frames are typically presented as soon as they arrive from camera. Rare IP camera attaches extra per frame information other than fame size (some don't do even this! they send data and separators only). Still some do attach time stamps and extra data like motion detection state.
Most of the IP cameras don't do constant frame rate. That is, frame rate might vary, esp. lower down in low light conditions. It is the responsibility of the receiving side to attach per frame time stamps when multiplexing the data into container format. Time stamp might be recovered from metadata (which rarely exists) or - more frequently - receiver stamps a frame with local receive time.
This is the way for the player to play back video sequence in proper rate. Live feed is typically presented on "show received frame as soon as possible" basis.
Normally MJPEG data is sent within a streaming media wrapper such as AVI or MOV (quicktime). The wrapper format will contain the framerate and information about the optional audio data.