How to play audio from an http data stream in swift - ios

I have an audio data stream coming in from a http response. I receive packets of bytes using the URLSessionDataDelegate method:
urlSession(_ session: URLSession, dataTask: URLSessionDataTask, didReceive data: Data)
I have successfully played the audio after appending all the data packets into a single Data object, using an AVAudioPlayer object and it's initWithData: initializer method.
What I really want to do is start audio playback while data is still coming in - streaming audio effectively. I haven't seen any answers that seem elegant for this use-case.
Options I've seen are:
Using the AudioToolbox: Audio File Stream Services & Audio Queues
Using the NSStream API, writing to a file and playing audio from that file concurrently
How would I achieve audio streaming playback from the Data packets coming in?

The easiest way is to use AVPlayer of AVFoundation framework. Instantiate the playerItem with your URL and pass it the player. Following code will do for you.
let urlString = "your url string"
guard let url = URL.init(string: urlString)
else {
return
}
let playerItem = AVPlayerItem.init(url: url)
player = AVPlayer.init(playerItem: playerItem)
player.play()

Consider AVPlayer for your requirement, something like this :
import AVKit
var player: AVPlayer?
func audioPlayer() {
do {
try AVAudioSession.sharedInstance().setCategory(AVAudioSessionCategoryPlayback)
player = AVPlayer(url: URL.init(string: "your url")!)
//This is for a player screen, if you don't want to show a player screen you comment this part
let controller = AVPlayerViewController()
controller.player = player
controller.showsPlaybackControls = false
self.addChildViewController(controller)
let screenSize = UIScreen.main.bounds.size
let videoFrame = CGRect(x: 0, y: 130, width: screenSize.width, height: (screenSize.height - 130) / 2)
controller.view.frame = videoFrame
self.view.addSubview(controller.view)
// till here
player?.play()
} catch {
}
}
For more please read this : https://developer.apple.com/documentation/avfoundation/avplayer

Related

AVPlayer cannot service a synchronized playback request via setRate:time:atHostTime: until its status is AVPlayerStatusReadyToPlay

I am trying to write a rather simple iOS app synchronizing the playback of a video with AVPlayer (from AVFoundation) to audio playback from multiple AKPlayers (Audiokit 4.0), using the .setRate function of AVPlayer together with CMClockGetHostTimeClock() and mach_absolute_time().
For some reason all my attempts end in the error message
AVPlayer cannot service a synchronized playback request via setRate:time:atHostTime: until its status is AVPlayerStatusReadyToPlay.
Obviously I keep on missing something. The movie plays just fine if I replace everything starting from "let time..." with a simple
videoPlayer.play()
This is a minimal "ViewController.swift" to reproduce the error:
import UIKit
import AudioKit
import AVFoundation
class ViewController: UIViewController {
var videoPlayer = AVPlayer()
override func viewDidLoad() {
super.viewDidLoad()
guard let videopath = Bundle.main.url(forResource: "test", withExtension: "mov") else {
debugPrint("test.mov not found")
return
}
videoPlayer = AVPlayer(url: videopath)
videoPlayer.automaticallyWaitsToMinimizeStalling = false
let playerLayer = AVPlayerLayer(player: videoPlayer)
playerLayer.frame = self.view!.bounds
self.view!.layer.addSublayer(playerLayer)
let time: TimeInterval = 1 // 1 second in the future
videoPlayer.masterClock = CMClockGetHostTimeClock()
let hostTime = mach_absolute_time()
let cmHostTime = CMClockMakeHostTimeFromSystemUnits(hostTime)
let cmVTime = CMTimeMakeWithSeconds(time, preferredTimescale: videoPlayer.currentTime().timescale)
let futureTime = CMTimeAdd(cmHostTime, cmVTime)
videoPlayer.setRate(1, time: CMTime.invalid, atHostTime: futureTime)
}
}

AVPlayer Fairplay HLS won't stop audio playback when video is paused

I am using Fairplay implementation as per Apple's Fairplay Streaming sample code at https://developer.apple.com/streaming/fps/, although I tried to choose only parts that are related to Online Fairplay Streaming, not the persistence/offline playback. In the below code a video without Fairplay plays/pauses/seeks normally, but when I play a Fairplay protected video, only the video track behaves correctly.
Pausing playback won't stop the audio playback, changing audio track won't stop the previous audio track, so both plays together and perhaps the seek also does not work.
Besides this helper class below, I have AssetLoaderDelegate and AssetPlaybackManager from Apple's client sample code of FairPlay Streaming Server SDK https://developer.apple.com/streaming/fps/ and I have updated the code to handle SPC/CKC for our DRM keys provider.
Did I miss to implement some important part of the code to handle audio for FPS Streaming? Can you please point me into right direction? Many thanks.
class PlayHelper {
static let shared = PlayHelper()
fileprivate var playerViewController: PlayerViewController?
init() {
AssetPlaybackManager.sharedManager.delegate = self
}
// Play video without DRM
func playVideo(from urlString: String, at context: UIViewController) {
guard let videoURL = URL(string: urlString) else {
Log.error("Video URL can't be created from string: \(urlString)")
return }
let player = AVPlayer(url: videoURL)
let playerViewController = PlayerViewController()
playerViewController.player = player
context.present(playerViewController, animated: true) {
playerViewController.player?.play()
}
}
// Play FPS video
func playFpsVideo(with asset: AVURLAsset, at context: UIViewController) {
// Cleanup, should be done when playerViewController is actually dismissed
if self.playerViewController != nil {
// The view reappeared as a results of dismissing an AVPlayerViewController.
// Perform cleanup.
AssetPlaybackManager.sharedManager.setAssetForPlayback(nil)
self.playerViewController?.player = nil
self.playerViewController = nil
}
let item = AVPlayerItem(asset: asset)
let player = AVPlayer(playerItem: item)
// Customize player
player.appliesMediaSelectionCriteriaAutomatically = true
let playerViewController = PlayerViewController()
playerViewController.player = player
self.playerViewController = playerViewController
context.present(playerViewController, animated: true) {
playerViewController.player?.play()
}
}
// Stop video
func stop() {
// Cleanup, should be done when playerViewController is dismissed
if self.playerViewController != nil {
// Results of dismissing an AVPlayerViewController, perform cleanup
AssetPlaybackManager.sharedManager.setAssetForPlayback(nil)
self.playerViewController?.player = nil
self.playerViewController = nil
}
}
}
// MARK: - Extend `PlayHelper` to conform to the `AssetPlaybackDelegate` protocol
extension PlayHelper: AssetPlaybackDelegate {
func streamPlaybackManager(_ streamPlaybackManager: AssetPlaybackManager, playerReadyToPlay player: AVPlayer) {
player.play()
}
func streamPlaybackManager(_ streamPlaybackManager: AssetPlaybackManager, playerCurrentItemDidChange player: AVPlayer) {
guard let playerViewController = playerViewController, player.currentItem != nil else { return }
playerViewController.player = player
}
}
I can also provide the code in AssetLoaderDelegate and AssetPlaybackManager if needed.
My bad. I called play() twice in the code above... Grrr.. Once when the presentation of the PlayerViewController finished and second time in the callback from AssetPlaybackDelegate that is triggered by KVO in AssetPlaybackManager. This way the player controls stopped playing the video, but most probably a second (audio) stream was still playing there. I removed the play() in playerReadyToPlay callback and now all the controls in the Player works as expected. I can pause, resume, seek, change audio tracks.

How to play Vimeo videos in iOS Swift?

I create an iOS app that make use of Vimeo for playing videos. I was wondering what the best method is to show Vimeo videos within iOS Swift.
I have fixed that, when an image is clicked, the placeholder image will be hidden, but the video won't play directly. Beside that all Vimeo Interface elements are visible. I know you should have to give credits to Vimeo, so it goes without saying that I display an Vimeo logo at the bottom of the video. Is there a possibility to hide all Vimeo elements of the web player?
Sources where I can find more information would be nice.
If there are any questions left, let me know!
Thanks in advance.
You can use this Swift library HCVimeoVideoExtractor to extract the mp4 video URL then use AVPlayer to play it. Simply pass the Vimeo video link or video id.
let url = URL(string: "https://vimeo.com/254597739")!
HCVimeoVideoExtractor.fetchVideoURLFrom(url: url, completion: { ( video:HCVimeoVideo?, error:Error?) -> Void in
if let err = error {
print("Error = \(err.localizedDescription)")
return
}
guard let vid = video else {
print("Invalid video object")
return
}
print("Title = \(vid.title), url = \(vid.videoURL), thumbnail = \(vid.thumbnailURL)")
if let videoURL = vid.videoURL[.Quality540p] {
let player = AVPlayer(url: videoURL)
let playerController = AVPlayerViewController()
playerController.player = player
self.present(playerController, animated: true) {
player.play()
}
}
})
The best way that I've been able to figure out is to use a WKWebView or UIWebView. Steps:
Add a WKWebView of desired size inside the View Controller.
Get the video embed code from the original Vimeo video (Click on the "Share" button below the video to find the embed code.)
Click on "More Options" on the Vimeo Share sheet that pops-up to configure the look of the embed video.
Use the following code sample to embed the video:
let webView = WKWebView(frame: CGRect(x: 0, y: 0, width: self.view.frame.size.width, height: self.view.frame.size.height))
self.view.addSubview(webView)
let embedHTML="<html><head><style type=\"text/css\">body {background-color: transparent;color: black;}</style><meta name=\"viewport\" content=\"width=device-width, initial-scale=1.0, maximum-scale=1.0, user-scalable=yes\"/></head><body style=\"margin:0\"><div><iframe src=\"//player.vimeo.com/video/139785390?autoplay=1&title=1&byline=1&portrait=0\" width=\"640\" height=\"360\" frameborder=\"0\" webkitallowfullscreen mozallowfullscreen allowfullscreen></iframe></div></body></html>"
let url = URL(string: "https://")!
webView.loadHTMLString(embedHTML as String, baseURL:url )
webView.contentMode = UIViewContentMode.scaleAspectFit
The way that I've managed to implement this is using AVKit and AVFoundation:
let url: URL! = URL(string: "https://01-lvl3-pdl.vimeocdn.com/01/3355/3/91775232/243724947.mp4?expires=1498547278&token=073f7b03877a8ed3c8029")
let player: AVPlayer = AVPlayer(url: url)
let controller: AVPlayerViewController = AVPlayerViewController()
controller.view.translatesAutoresizingMaskIntoConstraints = false
controller.player = player
// Add `controller` to your view somehow
player.play()
The trick being that you cannot use the link where it displays the website (in my case https://vimeo.com/91775232).
I had to inspect the source and find the actual URL for the video (i.e.: https://01-lvl3-pdl.vimeocdn.com/01/3355/3/91775232/243724947.mp4?expires=1498547278&token=073f7b03877a8ed3c8029).
Once I used that, everything worked fine.
For Swift 4, I have used WKWebView in storyboard and implemented following code:
import WebKit
#IBOutlet weak var webKitView: WKWebView!
To play video in WebView :
if yourVimeoLink.lowercased().contains("vimeo.com") {
let url: NSURL = NSURL(string: yourVimeoLink)
webKitView.contentMode = UIViewContentMode.scaleAspectFit
webKitView.load(URLRequest(url: url as URL))
}
Hope will help! :)
import AVFoundation
import AVKit
let videoURL = "https://vimeo.com/blablabla/whatevertheURLis"
func playExternalVideo() {
let videoURL = NSURL(string: self.videoURL)
let player = AVPlayer(url: videoURL as! URL)
let playerViewController = AVPlayerView()
playerViewController.player = player
}
Make sure to add the AVKit Player View and give it a class of AVPlayerView. As for the Vimeo deal, it should play without Vimeo's UI Controls. For more help watch this video. https://www.youtube.com/watch?v=fhD7hXrpExE

How to stream a video with AVURLAsset and save to disk the cached data

Some days ago I was asked to check how difficult is to play a video while downloading it from Internet. I know it's an easy task because someone told me a while ago. So, I checked and it was super easy.
The problem was that I wanted to save to disk the video to do not force the user to download it again and again.
The problem was to access the buffer and store it to disk.
Many answers in Stackoverflow says it is nor possible. Specially with videos.
My original code to play the video:
import AVFoundation
....
//MARK: - Accessors
lazy var player: AVPlayer = {
var player: AVPlayer = AVPlayer(playerItem: self.playerItem)
player.actionAtItemEnd = AVPlayerActionAtItemEnd.None
return player
}()
lazy var playerItem: AVPlayerItem = {
var playerItem: AVPlayerItem = AVPlayerItem(asset: self.asset)
return playerItem
}()
lazy var asset: AVURLAsset = {
var asset: AVURLAsset = AVURLAsset(URL: self.url)
return asset
}()
lazy var playerLayer: AVPlayerLayer = {
var playerLayer: AVPlayerLayer = AVPlayerLayer(player: self.player)
playerLayer.frame = UIScreen.mainScreen().bounds
playerLayer.backgroundColor = UIColor.clearColor().CGColor
return playerLayer
}()
var url: NSURL = {
var url = NSURL(string: "https://clips.vorwaerts-gmbh.de/big_buck_bunny.mp4")
return url!
}()
//MARK: - ViewLifeCycle
override func viewDidLoad() {
super.viewDidLoad()
view.layer.addSublayer(playerLayer)
player.play()
}
The solution for this problem is to use AVAssetExportSession and AVAssetResourceLoaderDelegate:
First step is to add a notification to know when the video finish. Then we can start saving it to disk.
override func viewDidLoad() {
super.viewDidLoad()
NSNotificationCenter.defaultCenter().addObserver(self, selector: #selector(playerItemDidReachEnd(_:)), name: AVPlayerItemDidPlayToEndTimeNotification, object: nil)
...
}
deinit {
NSNotificationCenter.defaultCenter().removeObserver(self)
}
The implementation of our function:
func playerItemDidReachEnd(notification: NSNotification) {
if notification.object as? AVPlayerItem == player.currentItem {
let exporter = AVAssetExportSession(asset: asset, presetName: AVAssetExportPresetHighestQuality)
let filename = "filename.mp4"
let documentsDirectory = NSFileManager.defaultManager().URLsForDirectory(NSSearchPathDirectory.DocumentDirectory, inDomains: NSSearchPathDomainMask.UserDomainMask).last!
let outputURL = documentsDirectory.URLByAppendingPathComponent(filename)
exporter?.outputURL = outputURL
exporter?.outputFileType = AVFileTypeMPEG4
exporter?.exportAsynchronouslyWithCompletionHandler({
print(exporter?.status.rawValue)
print(exporter?.error)
})
}
}
Finally we need to make our AVURLAsset delegate of AVAssetResourceLoaderDelegate:
lazy var asset: AVURLAsset = {
var asset: AVURLAsset = AVURLAsset(URL: self.url)
asset.resourceLoader.setDelegate(self, queue: dispatch_get_main_queue())
return asset
}()
And:
extension ViewController : AVAssetResourceLoaderDelegate {
}
I created a small demo with this code in GitHub.
The team at Calm has open-sourced our implementation to this. It's available as a CocoaPod. It's called PersistentStreamPlayer.
Features include:
streaming of audio file, starting playback as soon as first data is available
also saves streamed data to a file URL as soon as the buffer completes
exposes timeBuffered, helpful for displaying buffer progress bars in the UI
handles re-starting the audio file after the buffer stream stalls (e.g. slow network)
simple play, pause and destroy methods (destroy clears all memory resources)
does not keep audio file data in memory, so that it supports large files that don't fit in RAM
You can find it here: https://github.com/calmcom/PersistentStreamPlayer

Why does Swift's AVPlayer loads the playerItem for twice on one play?

I'm using AVFoundation's AVPlayer for streaming external mp3 files. I have a counter on the back-end that counts how many times a file loaded. The only client for this service is only me and whenever I trigger to play the AVPlayer, the counter increases two which means AVPlayer makes the request twice. Is there a reason for this, or how can I prevent that from happening? Here is my code:
#IBAction func listen(sender: UIButton) {
let urlstring = "http://api.server.com/endpoint-to-mp3"
let url = NSURL(string: urlstring)
let playerItem = AVPlayerItem(URL: url!)
let player = AVPlayer(playerItem: playerItem)
let playerLayer = AVPlayerLayer(player: player)
playerLayer.frame = CGRectMake(0, 0, 300, 50)
self.view.layer.addSublayer(playerLayer)
player.volume = 1.0
player.play()
}
AVPlayer is making a network request to the URL whenever you initialize the player with AVPlayerItem. This call only fetches the file information and file size. (At this point I am able to observe 2 requests sometime, which could increase your count to 3)
Later when you are attaching the player to any view, another call is happening to fetch the complete file. (You can use Charles to observe your network traffic, fyi)
This behaviour is same when you init the player with init(url:) So I don't see any way that could prevent this from happening at the moment.

Resources