Can anyone shed some light on the live streaming stack Youtube Gaming uses to achieve that crazy framerate? (60fps)
Looks to me like HLS, but better. Receives chunks of binary data every ~3 seconds, one chunk is ~200kb.
What do you think the source type is?
PS: I have implemented live streaming through HTTP using https://github.com/phoboslab/jsmpeg - sending binary data through websockets to the browser. While it works cross-vendor and achieves ~20fps it's nowhere near and doesn't have sound.
YouTube provides multiple formats, depending on the client device/browser.
The streams that look like HLS are using DASH (see Dynamic Adaptive Streaming over HTTP and Adaptive Bitrate Streaming).
Look for network requests for the manifest files which have the video/vnd.mpeg.dash.mpd or application/dash+xml MIME types.
They use separate streams for audio and video synchronized on the player.
For 60fps it seems to support both MPEG-DASH (H.264/AVC in MP4 container with AAC audio in M4A) and WebM DASH (VP9 with Opus in WebM). WebM is royalty-free, MPEG is paid.
Here's the output from youtube-dl --list-formats for a radom stream:
format code extension resolution note
249 webm audio only DASH audio , opus # 50k
250 webm audio only DASH audio , opus # 70k
251 webm audio only DASH audio , opus #160k
171 webm audio only DASH audio 113k , audio#128k (44100Hz), 2.27MiB
140 m4a audio only DASH audio 129k , m4a_dash container, aac #128k (44100Hz), 2.81MiB
141 m4a audio only DASH audio 255k , m4a_dash container, aac #256k (44100Hz), 5.59MiB
247 webm 720p DASH video , video only
136 mp4 720p DASH video , video only
248 webm 1080p DASH video , video only
137 mp4 1080p DASH video , video only
160 mp4 256x144 DASH video 110k , 15fps, video only, 2.28MiB
278 webm 256x144 DASH video 190k , webm container, VP9, 15fps, video only, 2.11MiB
242 webm 426x240 DASH video 227k , 30fps, video only, 3.58MiB
133 mp4 426x240 DASH video 250k , 30fps, video only, 5.18MiB
243 webm 640x360 DASH video 403k , 30fps, video only, 6.69MiB
134 mp4 640x360 DASH video 617k , 30fps, video only, 8.97MiB
244 webm 854x480 DASH video 864k , 30fps, video only, 12.89MiB
135 mp4 854x480 DASH video 1115k , 30fps, video only, 17.25MiB
302 webm 1280x720 DASH video 2933k , VP9, 60fps, video only, 43.81MiB
298 mp4 1280x720 DASH video 3373k , h264, 60fps, video only, 54.42MiB
303 webm 1920x1080 DASH video 5055k , VP9, 60fps, video only, 80.96MiB
299 mp4 1920x1080 DASH video 5584k , h264, 60fps, video only, 96.97MiB
13 3gp unknown
17 3gp 176x144
36 3gp 320x240
5 flv 400x240
43 webm 640x360
18 mp4 640x360
22 mp4 1280x720 (best)
Related
Objective
I am trying to record a video (duration ≈ 1min to 4hours) from a webcam plugged into a PC. Then play just after the end of the recording the video on an Ipad connected to the same network as the PC.
To do this I'm trying to record the video using ffmpeg (the detailed command I use is below). Then I serve the video that's on the PC to the Ipad via a Java server (The server works well with other videos so I don't think the problem is here).
Problem
When I play the video on the IPad native player, for example in Safari, the sound is going faster than the video and therefore it's desynchronizing from the video. If I request a certain time the video and audio re-sync then the audio is desynchronizing again while playing.
The video when played on VLC on the PC or on Firefox by requesting the server is well synced.
Another video (I downloaded this one to test) played on the Ipad from the server stay synced.
It is only with the combination of a video record by myself using FFmpeg played in the iOS native player that I can observe the desynchronization.
FFmpeg command
I use FFmpeg to record the video from the webcam and save it into a file format readable on iOS.
ffmpeg -y -f avfoundation -framerate 25 -pixel_format nv12 -video_size 1280x720 -i "0:0" -r 25 out.mp4
The resulting recorded file info according to ffmpeg is below:
> ffmpeg -i out.mp4
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'out.mp4':
Metadata:
major_brand : isom
minor_version : 512
compatible_brands: isomiso2avc1mp41
encoder : Lavf58.45.100
Duration: 00:00:33.76, start: 0.000000, bitrate: 1510 kb/s
Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 1280x720, 1440 kb/s, 25 fps, 25 tbr, 12800 tbn, 50 tbc (default)
Metadata:
handler_name : VideoHandler
Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, mono, fltp, 61 kb/s (default)
Metadata:
handler_name : SoundHandler
Compatible video formats from TV and Video iPad Pro specs
Video formats supported: H.264 video up to 4K, 30 frames per second, High Profile level 4.2 with AAC‑LC audio up to 160 Kbps, 48kHz, stereo audio or Dolby Audio up to 1008 Kbps, 48kHz, stereo or multichannel audio, in .m4v, .mp4, and .mov file formats; MPEG‑4 video up to 2.5 Mbps, 640 by 480 pixels, 30 frames per second, Simple Profile with AAC‑LC audio up to 160 Kbps per channel, 48kHz, stereo audio or Dolby Audio up to 1008 Kbps, 48kHz, stereo or multichannel audio, in .m4v, .mp4, and .mov file formats; Motion JPEG (M‑JPEG) up to 35 Mbps, 1280 by 720 pixels, 30 frames per second, audio in ulaw, PCM stereo audio in .avi file format
I don't see what codecs or profiles I don't respect?
ps: I write PC but I'm currently trying this on a Mac. Eventually, I want to be able to do it also on Linux or even Windows. It looks like I just have to change the input devices to do that, which is avfoundation for MacOS.
I have created player in iOS. It works well for URLs such as mp4 and m3u8.
But I have one webm extension URL. AVPlayer is not working for this.
Sample url:
http://images.all-free-download.com/footage_preview/webm/dead_trees_146.webm
Is there any reason? Or can we make something possible to play this url?
From the apple docs:
iOS supports many industry-standard video formats and compression standards, including the following:
H.264 video, up to 1.5 Mbps, 640 by 480 pixels, 30 frames per second, Low-Complexity version of the H.264 Baseline Profile with AAC-LC audio up to 160 Kbps, 48 kHz, stereo audio in .m4v, .mp4, and .mov file formats
H.264 video, up to 768 Kbps, 320 by 240 pixels, 30 frames per second, Baseline Profile up to Level 1.3 with AAC-LC audio up to 160 Kbps, 48 kHz, stereo audio in .m4v, .mp4, and .mov file formats
MPEG-4 video, up to 2.5 Mbps, 640 by 480 pixels, 30 frames per second, Simple Profile with AAC-LC audio up to 160 Kbps, 48 kHz, stereo audio in .m4v, .mp4, and .mov file formats
Numerous audio formats, including the ones listed in Audio Technologies.
It doesn't support WebM. You may try using a library like OGVKit to play WebM videos.
I have a .h264 file in my project bundle and i have to play it. I searched for long time and everybody suggested FFMPEG but there is no proper guidance. In another way at least suggest me how to convert this .h264 file in to .mp4 file programmatically by that way at least i can depend on MPMoviePlayer. Please suggest me.
Thank you.
Use the native AVFoundation framework
https://developer.apple.com/av-foundation/
iOS supports many industry-standard video formats and compression standards, including the following:
H.264 video, up to 1.5 Mbps, 640 by 480 pixels, 30 frames per second, Low-Complexity version of the H.264 Baseline Profile with AAC-LC audio up to 160 Kbps, 48 kHz, stereo audio in .m4v, .mp4, and .mov file formats
H.264 video, up to 768 Kbps, 320 by 240 pixels, 30 frames per second, Baseline Profile up to Level 1.3 with AAC-LC audio up to 160 Kbps, 48 kHz, stereo audio in .m4v, .mp4, and .mov file formats
MPEG-4 video, up to 2.5 Mbps, 640 by 480 pixels, 30 frames per second, Simple Profile with AAC-LC audio up to 160 Kbps, 48 kHz, stereo audio in .m4v, .mp4, and .mov file formats
Numerous audio formats, including the ones listed in Audio Technologies
In order to get > 10 minute videos on my app accepted, I need to implement a html5 video in a webview. The video is linking as I can hear the audio but the video is a black screen. The videos are all mp4 and I am using the following html
<html><body style="margin: 0; padding: 0;"><video width="480" height="300" controls="controls"><source src="'+video+'" type="video/mp4"></video></body></html>
can anyone help me get my imagery viewing?
I am going to guess that it is the video format that is the issue.
Here is what is suported:
H.264 video up to 720p, 30 frames per second, Main Profile level 3.1
Audio: AAC-LC up to 160 Kbps, 48kHz, stereo
File types: .m4v, .mp4, and .mov
MPEG-4 video, up to 2.5 Mbps, 640 by 480 pixels, 30 frames per second, Simple Profile
Audio: AAC-LC up to 160 Kbps per channel, 48kHz, stereo
File types: .m4v, .mp4, and .mov
Miro might help you out, since it has a "for iphone" option that sets up the video just right.
We currently have a system working on chrome,ie7 and mozilla. We have jwplayer that plays .flv videos on these browsers. We have now to support the play of videos in ipad. Will it be possible if we upgrade the jwplayer to version 5 which supports html5 video tags. Will html5 video tags in ipad be able to support play of .flv files?
The jwplayer works fine on the iPad but you will need another video type for ipad support. The iPad will not play FLV file types, as shown on the iPad Video specs page:
Video formats supported: H.264 video up to 1080p, 30 frames per second, High Profile level 4.1 with AAC-LC audio up to 160 Kbps, 48kHz, stereo audio in .m4v, .mp4, and .mov file formats; MPEG-4 video up to 2.5 Mbps, 640 by 480 pixels, 30 frames per second, Simple Profile with AAC-LC audio up to 160 Kbps per channel, 48kHz, stereo audio in .m4v, .mp4, and .mov file formats; Motion JPEG (M-JPEG) up to 35 Mbps, 1280 by 720 pixels, 30 frames per second, audio in ulaw, PCM stereo audio in .avi file format