How would I setup an AirPlay video & audio receiver for iOS (and then save the stream as a video file)?
I know that this goes against Apple's guidelines, this is not intended for AppStore distribution. I am fine using private APIs.
Note: I am using Pythonista (with objc_util), so, if possible, answers written in Python will be very helpful, although Swift/Objective-C is still greatly appreciated.
I assume you have this idea after the recent incredible (but short-lived) Vidyo app went on the App Store. I managed to snag a copy before it was taken down, but recreating this effect in Pythonista is certainly desirable.
You could start with the unofficial AirPlay specification, which describes how the AirPlay protocol works. Specifically, you want the section on Screen Mirroring. From this, you may be able to put together an AirPlay interface.
I don't think objc_util will be necessary for this, Python provides some pretty low-level networking modules.
From reading the spec, you'll need to set up a server. Flask probably can't handle it, Flask is likely too high-level. It looks like the AirPlay streaming stuff doesn't even stay within the realm of valid HTTP requests.
I suspect you'll have a lot of trouble with this. The AirPlay spec (especially screen mirroring) is pretty complicated. You'll need to let your server receive a H.264-encoded video livestream (this is the same format Apple uses to livestream its events), and you'll also need to set up a system for synchronizing your video based on data sent through a separate stream. On top of all this, you'll need to provide some endpoints that return data about your server.
I suppose it's entirely possible that Vidyo found some private APIs that make this easier. I don't see any clear reason why iOS would implement an AirPlay server somewhere, but it's not outside the realm of possibility. If this exists, I'm not aware of it. You'll have to do more research.
Good luck ;)
Related
I am attempting to make my app, this is an app that will integrate live streaming that could be accessed by anyone that is signed up. I have been searching and the information is very vague.
What factors do I have to take into consideration to make an IOS app with live streaming?
I would have no clue how to implement any of this and I haven't seen much documentation about how to implement this. What third party software would I have to use, would I need to create something in a different programming language?
This app is going to be only for IOS for now.
Thanks,
The question is, how many users? How much do you want to spend? How critical is performance/quality/downtime, etc.? Is this a business app?
Due to having multiple users connect to one stream, I would suggest that the streamer streams it to a server, whereby the server deals with the bandwidth requirements to clients, allowing for it to more easily scale. If it is a one-on-one or up to maybe one on three, you could still think of p2p for a social stream (e.g. streaming to family) but not e.g. a business conference (unless you want them to have e2e encryption, avoiding server, etc. which is usually complete overkill, though do implement encryption if possible, quite easy using libraries).
Now if you're asking coding wise, look for some tutorials, e.g. https://www.raywenderlich.com/5191-video-streaming-tutorial-for-ios-getting-started is quite nice, it's quite straight forward and will probably do what you want for the client accessing it. Use the AVFoundation Playback https://developer.apple.com/av-foundation/ it has links to HTTP live streaming and how to implement it as well.
I'm trying to find a way to get the average power level for a channel, that comes out from the audio played in the embedded video. I'm using YouTube's iOS helper library for embedding the video https://developers.google.com/youtube/v3/guides/ios_youtube_helper
A lot of the answers I've found in StackOverflow refer to AVAudioPlayer, but that's not my case. I also looked in the docs of AudioKit framework to find something that can give the output level of the current audio, but I couldn't find anything related, maybe I missed something over there. I also looked in EZAudio framework even tough it's deprecated, and I also couldn't find something that relates to my case.
My direction of thinking was to find a way to get the actual level that's coming out from the device, but I found one answer in SO that's saying this is not allowed in iOS, although he didn't mention any source for this statement.
https://stackoverflow.com/a/12664340/4711172
So, any help would be much appreciated.
The iOS security sandbox blocks apps from seeing the device's digital audio output stream, or any other app's internal audio output (unless explicitly shared, e.g. inter-app audio, etc.) (when using Apple App store permitted public APIs.)
(Just a guess, but this was possibly and originally implemented in iOS to prevent apps from capturing samples of DRM'd music and/or recording phone call conversations.)
Might be a bit off/weird, but just in case -
Have you considered closing a loop? Meaning - record the incoming audio using 'AVAudioRecorder' and get the audio levels from there?.
See Apple's documentation for AVAudioRecorder (in the overview they're specifying: "Obtain input audio-level data that you can use to provide level metering")
AVAudioRecorder documentation
I'm trying to put together an open source library that allows iOS devices to play files with unsupported containers, as long as the track formats/codecs are supported. e.g.: a Matroska video (MKV) file with an H264 video track and an AAC audio track. I'm making an app that surely could use that functionality and I bet there are many more out there that would benefit from it. Any help you can give (by commenting here or—even better— collaborating with me) is much appreciated. This is where I'm at so far:
I did a bit of research trying to find out how players like AVPlayerHD or Infuse can play non-standard containers and still have hardware acceleration. It seems like they transcode small chunks of the whole video file and play those in sequence instead.
It's a good solution. But if you want to throw that video to an Apple TV, things don't work as planned since the video is actually a bunch of smaller chunks being played as a playlist. This site has way more info, but at its core streaming to Apple TV is essentially a progressive download of the MP4/MPV file being played.
I'm thinking a sort of streaming proxy is the way to go. For the playing side of things, I've been investigating AVSampleBufferDisplayLayer (more info here) as a way of playing the video track. I haven't gotten to audio yet. Things get interesting when you think about the AirPlay side of things: by having a "container proxy", we can make any file look like it has the right container without the file size implications of transcoding.
It seems like GStreamer might be a good starting point for the proxy. I need to read up on it; I've never used it before. Does this approach sound like a good one for a library that could be used for App Store apps?
Thanks!
Finally got some extra time to go over GStreamer. Especially this article about how it is already updated to use the hardware decoding provided by iOS 8. So no need to develop this; GStreamer seems to be the answer.
Thanks!
The 'chucked' solution is no longer necessary in iOS 8. You should simply set up a video decode session and pass in NALUs.
https://developer.apple.com/videos/wwdc/2014/#513
I am trying to implement Progressive Downloading of a video in my iOS application that can be played through AVPlayer. I have already implemented a downloader module that can download the files to the iPad. However, I have discovered I cannot play a file that is still being written to
So, as far as I can tell, my only solution would be through downloading a list of file 'chunks' and then keep playing through every file as they are ready (ie: downloaded), probably using HLS
Searching I have come across this question which implements the progressive download through hls but other than that, I can find no other way
However, I keep coming across search results that say how to configure web servers to leverage the iOS support for HTTP Progressive Downloading, but with no mention of how to do it from the iOS side
So, any one have any ideas and/or experience about this?
EDIT: I have also found there could be a way of doing it other way around (ie: streaming, then writing streamed data to disk) which was suggested by this question but still cannot get it to work as it seems it does not work with non-local assets!
From what you say, you might want to change approach and attempt to stream the file. Downloading and playing at the same time, I would say is the definition of Streaming. I hate when people post links to the Apple documentation but in this instance reading a tiny bit of this documentation will help you more than I ever can. It should all make sense if you are lready working with connections and video, you just need to change your approach.
The link: https://developer.apple.com/library/ios/documentation/networkinginternet/conceptual/streamingmediaguide/Introduction/Introduction.html
So, I wrote a quick little app for the iphone that takes in an http URL, and plays the .mp4 video located at that URL. It does more than that, of course, but that's the meat of it. Naturally, I wanted to have it on more than just a single mobile platform, so I decided to target BlackBerry next.
However, I'm running into a lot of problems with the BlackBerry Environment. First of all, I learn that I can only download 256k files! I learn how to set that variable in my MDS simulator and I learn that this is NOT a production solution, because any end users will have to have their BES or MDS admin change the setting there. Then, I find a video less than 2 MB I can practice with. Going to the browser prompts me to save the video rather than it playing in the browser like I expected. After saving the video, it refuses to play, saying it's the wrong format.
So. I can't find a reference to whether BlackBerry can stream with HTTP. I've heard it can use RTSP, though, and heard some rumors that it can't use HTTP, which would really suck. I also can't find a reference to what format BlackBerry uses, although I can find a million programs that will convert one file to the 'BlackBerry' format.
Surely SOMEONE must have tried to stream video with the BlackBerry before. How did they go about doing so? Is it just a hopeless pipedream? Will I have to go with RTSP?
Sorry for the lack of a concrete question. I'm just really lost, and I hate how so many tutorials or forum posts seem to assume I know the capabilities of the Blackberry.
Edit: I finally found out that the .3gp format, which I'd never heard of, is what BlackBerry uses. Still have no idea how to stream videos off the web, though. I found "How To - Play video within a BlackBerry smartphone application" That seemed useful, but the code doesn't work if you give it a URL, even though it claims it does.
While you are correct that the tutorial claims the code will load any valid URL, the API documentation for javax.microedition.media.Manager.createPlayer specifies "A locator string in URI syntax that describes the media content" which may not, in fact be the same as any valid URL. Luckily createPlayer will also take an InputStream and a String specifying the content type. So you should be able to open the URL as documented in the API for HttpConnection, grab the content type string, and open the input stream to create the player.
I will admit that I haven't done that, but it would be my next step.
BTW remember to run your HttpConnection fetch on a thread separate from the application event thread.