I have a live video chat application and I use a TURN server which supports STUN/TURN and both UPD/TCP transmission.
Sometimes users can be connected to the network which blocks that much ports and protocols that WebRTC connection just cannot happen (usually those are corporate networks). I would like to check if a WebRTC connection is possible before users try to connect to each other (actually, perform a technical check).
How can I do it? Ideas I have in my head:
Try to download a hosted chunk of data (audio file, for example) via WebRTC - is it possible and would this be enough to make sure both inbound and outbound connections are open?
Use a TURN server as a host to make a connection to and see if it fails (have no idea if I can do it or not)
Use Flash to try to download/upload a chunk of data over specific ports and protocols. May be even using Cirrus. However, I am not sure this test will be accurate from WebRTC prospective.
Any other ideas?
Additional requirement: the checking technique must support Chrome, Opera and Firefox. Preferably also IE/Safari via Temasys plugin.
Edition 1 - gathering ICE candidates is a good idea, however, it is not 100% reliable. Once I checked logs in my application and it actually gathered relay ICE candidates, but video/audio transmission failed. Tested on Apprtc as well and got same results.
The best way to check is to connect with just a data channel first. Your users won't notice. If that works then audio and video are almost guaranteed to work. As a bonus, you can use the data channel for signaling for super-fast connecting when your users are ready.
the typical WebRTC approach to this is to create a peerconnection with STUN and TURN servers, call createOffer and setLocalDescription and watch the candidates gathered. See e.g. http://webrtc.github.io/samples/src/content/peerconnection/trickle-ice/
If you get srflx candidates, your stun server works (i.e. UDP is not blocked). More interesting is whether you get relay candidates. If you do, using TURN as a fallback will work. Quality might suffer if TURN/TCP is used. If you don't get relay candidates... calls are very unlikely to work.
Related
Hey Stackoverflow community :)
Im looking into making a camera stream video from a an RC device into a computer using wifi.
After considering all of the options I had Im left with two:
use UDP to transfer video in packets
use packet injection and packet sniffing on the receiving device.
I was wondering what are the pros and cons of each method (for that specific purpose of video transmission)?
after looking around I found many implementations for both ways but nowhere have they specified why one is better than the other.
few things that I have not mentioned:
I know UDP does not have error correction which can make the video weird- I dont care about the quality of the video as long as it will be recognizeable.
I dont want to use connection based protocol (TPC, etc)- I dont want to wait for handshake when I get disconnected.
thanks :)
I'm trying to do a similar thing. My take on this is basically when you use the wifi cards in monitor mode (i.e. using packet sniffing/injection) you don't actually need to be connected to that network. Typically, you still need to be connected to an Access point as a client then you can communicate using UDP through that connection. But, in this case, the UDP messages are routed to the Wifi cards and the packets are injected out without being associated with any client. Then, any 'client' just has to sniff or listen on that same channel to get the transmission. So the benefit is not only does UDP not check for lost frames/etc, but also in this case you don't need to be connected to the network to get the packets.
In my case, this is preferable, since basically you will need to connect to the AP in the former case and that would require more capable hardware on the receiver side typically (more range is needed for the association part since you need to send messages back over TCP essentially to get it connected).
FYI here are the links/repos I am using and it also is a reference to what I am talking about
https://docs.px4.io/master/en/tutorials/video_streaming_wifi_broadcast.html
https://github.com/svpcom/wifibroadcast
I am using an off the shelf 'solution' in the short term, the Accsoon Cineye Air, which basically transmits HDMI 300ft line of sight over WiFi. You need an android phone to receive it, and basically I'm using the Vysor application (paid version is $40) to mirror the screen to my desktop. It works, but the latency is still more than I want : 60ms at least from the cineeye, so you can drive it around but its not as quick as DJI which is around 30-40ms ), which is my goal.
I want to capture and send some packets to the access point for testing purposes. I have the packet captures made via wireshark but I do not know how to proceed to be able to send these packets from my macbook.
I tried things like scapy, colasoft etc, but they do not seem to work as they require an external wifi adapter to be able to relay these packets outwards.
Two main questions:
Is it possible to send custom packets from macbook to a required access point. (Without using external network adapter).
If yes, what are some tooling/Scripting options that I can look at ? Any recommendations?
I am networking novice so please pardon me if the question is trivial. Thank you!
Your builtin Wifi adapter likely does not support Wifi injection.
You can check this by googling the Wifi chip that is within your computer (there are various methods to get that info depending on your OS) whether it supports Wifi injection or not.
So yeah, you'll likely need an external card (check the specs before buying it)
I'm searching for 4 days, but can't get it. I built all libraries and integrated it in my custom project, but I don't know what steps should I do to make it work. The only thing that i found with code example\explanation is tech.appear.in/2015/05/25/Getting-started-with-WebRTC-on-iOS , but it is poor and unclear for me, AppRTCDemo source code too. I read about WebRTC for browsers but still can't reproduce it on iOS.
Can anybody explain or provide links to explanation on how to completely build iOS native app using WebRTC API for example p2p ios chat?
Besides the fact that I do not understand code logic provided in demo, I can't understand:
1) What is ICE servers for my iOS app? Should I take care of it? Is it something server side? Should I code and run it myself, or I can use existing Parse background?
2) What is signaling mechanism in iOS app? Is it client side only, or it must be implemented on server side too?
3) And maybe someone can explain step-by-step guide, maybe with some code, how to implement simple iOS p2p chat using WebRTC? For example:
"You have to:
Create ICE/STUN/TURN server on parse core using this =source= and this tutorial =tutorial=.
Create RTCPeerConnection using created ICEServer:
RTCPeerConnectionFactory *pcFactory = [[RTCPeerConnectionFactory alloc] init];
RTCPeerConnection *peerConnection = [pcFactory peerConnectionWithICEServers:kICEServerURL constraints:nil delegate:self];
Create DataChannel using ...
Send signal using ... explained here =link=
Set local and remote descriptions ...
Send Data ... using ...
... " or something similar.
I'm sorry for asking this, but I'm losing my mind trying to figure it out. Thank you!
I am not an expert in webrtc but i will try to explain some of your questions.
1.ICE servers-- NATs and firewalls impose significant problem in setting up IP endpoints. so IETF standards STUN, TURN and ICE were developed to address the NAT traversal problem.
STUN helps connect IP end-points:
discover whether they are behind a NAT/firewall, and if so,
to determine the public IP address and type of the firewall. STUN then uses this information to assist in establishing peer-to-peer IP connectivity.
TURN, which stands for Traversal Using Relay NAT, provides a fallback NAT traversal technique using a media relay server to facilitate media transport between end-points.
ICE is a framework that leverages both STUN and TURN to provide reliable IP set-up and media transport, through a SIP offer/answer model for end-points to exchange multiple candidate IP addresses and ports (such as private addresses and TURN server addresses).
2.Signaling is the process of coordinating communication. This signalling part needs to be implemented by you according to your needs(for ex. if you have sip structure in place then you will have to implement sip signalling). In order for a WebRTC application to set up a 'call', its clients need to exchange information:
Session control messages used to open or close communication.
Error messages.
Media metadata such as codecs and codec settings, bandwidth and media types.
Key data, used to establish secure connections.
Network data, such as a host's IP address and port as seen by the outside world.
Steps
for offerer:
first create the peer connection and pass the ice candidates into it
as parameters.
set event handlers for three events:
onicecandidate-- onicecandidate returns locally generated ICE candidates so you can pass them over other peer(s) i.e. list of ice candidates that are returned by STUN/TURN servers; these ice candidates contains your public ipv4/ipv6 addresses as well as UDP random addresses
onaddstream--onaddstream returns remote stream (microphone and camera of your friend!).
addStream` attaches your local microphone and camera for other peer.
Now create SDP offer by calling setLocalDescription function and set remote SDP by calling setRemoteDescription.
For Answerer:
setRemoteDescription
createAnswer
setLocalDescription
oniceCandidate--On getting locally generated ICE
addiceCandidate--On getting ICE sent by other peer
onaddstream--for remote stream to add
I hope this will make some of your doubts clear.
I came through the process of implementing it few month ago. What I've found was the library was not stable - sometimes it was working sometimes not.
Additionally my iPhone was always becoming hot when I was using it.
I would not suggest using this library and overall WebRTC technology for commercial projects.
This is my implementation, which was working few months ago:
https://github.com/aolszak/WebRTC-iOS
Good luck!
I want to implement a peer-to-peer video chat feature for a web application I am currently developing. After doing my research, I've decided that using webRTC's Javascript APIs is the way to go. The application uses AngularJS in the front end and Ruby on Rails in the back end. The main issue I'm encountering while conceptualizing this application is linking the front end with the backend, and creating and maintaining the connection between user streams.
For the signaling aspect of the network, I want to utilize ActionController::Live and the Ruby gem em-event source to push live messages from the server to users and indicate which of their connections are online. Then, when they are ready to make a connection, they will create a custom room and the URL will be sent to the user that they wish to connect with, creating their offer. Once the user clicks on the link sent to them, they send back their answer. When the user responds, the ICE candidate process will begin for each of the users. Do you think that this is a sufficient signaling channel to set up the PeerConnection? What other major players am I missing?
From the research that I have done about WebRTC's RTCPeerConnection, once the initial connection is set up, and both users have public IP addresses corresponding to their stream, the connection is sustained through RTCPeerConnection, more specifically getPeerConnection(). Am I wrong? Are there other factors that I am not considering?
WebRTC makes the process of creating MediaStreams very simple with their getUserMedia method. Once these streams are created they can be added to the RTCPeerConnection that was established. Both as local and remote streams.
If you have any other suggestions for me, please let me know. I want to create this feature using webRTC, it seems like so much fun
There are certainly many ways to handle the call signaling so I'm not going to comment specifically on your approach. I will say that if you plan on supporting ICE trickling the ICE candidates will start flowing very early in the process so you really need an open signalling channel between your peers almost immediately when trying to connect to a peer.
We developed our solution for WebSphere on top of MQTT which is an open, and very simple pub/sub protocol. You can use any open MQTT broker with the protocol and there are a number of open source components available to make WebRTC development extremely easy including an AngularJS WebRTC module (angular-rtcomm), a core pure JavaScript module and much more. We also released a simple JSON based protocol as part of this open source solution. You can take a look at the signaling protocol. You can also read more details about the overall solution here (www.wasdev.net/webrtc). Here you'll find the base JavaScript libraries as well as a number of open source sample solutions. All of these can be forked on github.
In general you want to build your signaling on a protocol that will allow you to grow over time. It should work well for the web and mobile apps. From our experience it took a lot of time to get all this to work well and our goal was to not only support peer-to-peer calls but to support using media resources like Dialogic's XMS PowerMedia server on the backend for multiway support, record/playback and more. We also needed to support federation via SIP trunking so we wanted to make sure the protocol could be easily translated to SIP signaling while also supporting transcoding between media protocols like VP8 and H.264.
Note that if you're looking to only support peer-to-peer calling between WebRTC clients you can do that with these rtcomm open source components only, including an open MQTT broker and save yourself a ton of time. You can literally get something up and running in a matter of hours. The developer version of the WebSphere Liberty beta with the new rtcomm-1.0 service enabled also includes a built in MQTT broker and supports the open WebRTC signaling protocol linked above. You can use WebSphere for development and deploy a single server of this in production for free. You can also use Ruby on Rails with Liberty as well if you'd like.
Even if you decide not to use Liberty you can use all the open source components along with something like Mosquito (which is an open source MQTT broker) to get a solution off the ground quickly. There are also a number of MQTT clients available for many different programming languages including JavaScript, Java, etc. Check out https://eclipse.org/paho/. If you decide to build you're own signaling protocol you might still find these open source components helpful to see how we approached integration with the WebRTC PeerConnection.
I am designing an iOS app for a customer who wants to allow real-time (with minimum lag, max 50ms) conversations between users (a sort of Teamspeak). The lag must be low because the audio can also be live music, played with instruments, so all the users need to synchronize. I need a server, which will request audio recordings to every client and send to others (and make them hear the same sound at the same time).
HTTP is easy to manage/implement and easy to scale, but very low-performing because an average HTTP request takes > 50ms... (with a mid-level hardware), so I was thinking of TCP/UDP connections kept open between clients and server.
But I have some questions:
If I develop the server in Python (using TwistedMatrix, for example), how are its performance ?
I can't develop the server in C++ because it is hard to manage (scalable) and to develop.
Anyone used Nodejs (which is easy to scale) to manage TCP/UDP connections?
If I use HTTP, will it be fast enough with Keep-Alive? Becuase usually the time required for an HTTP Request to be performed is > 50ms (because opening-closing connection is hard), and I want the total procedure to be less than that time.
The server will be running on a Linux machine.
And finally: which type of compression can you suggest me? I thought Ogg Vorbis would be nice, but if there's anything better (and can be used in iOS), I am open to changes.
Thank you,
Umar.
First off, you are not going to get sub 50 ms latency. Others have tried this. See for example http://ejamming.com/ a service that attempts to do what you are doing, but has a musically noticeable delay over the line and is therefore, in the ears of many, completely unusable. They use special routing techniques to get the latency as low as possible and last I heard their service doesn't work with some router configurations.
Secondly, what language you use on server probably doesn't make much difference, as the delay from client to server will be worse than any delay caused by your service, but if I understand your service correctly, you are going to need a lot of servers (or server threads) just relaying audio data between clients or doing some sort of minimal mixing. This is a small amount of work per connection, but a lot of connections, so you need something that can handle that. I would lean towards something like Java, Scala, or maybe Go. I could be wrong, but I don't think this is a good use-case for node, which, as I understand it, does not do multithreading well at this time. Also, don't poo-poo C++, scalable services have been built C++. You could also build the relay part of the service in C++ and the rest in whatever.
Third, when choosing a compression format, you'll have to choose one that can survive packet loss if you plan to use UDP, and I think UDP is the only way to go for this. I don't think vorbis is up to this task, but I could be wrong. Off the top of my head, I'm not sure of anything that works on the iPhone and is UDP friendly, but I'm sure there are lots of things. Speex is an example and is open-source. Not sure if the latency and quality meet your needs.
Finally, to be blunt, I think there are som other things you should research a bit more. eg. DNS is usually cached locally and not checked every http call (though it may depend on the system/library. At least most systems cache dns locally). Also, there is no such protocol as TCP/UDP. There is TCP/IP (sometimes just called TCP) and UDP/IP (sometimes just called UDP). You seem to refer to the two as if they are one. The difference is very important for what you are doing. For example, HTTP runs on top of TCP, not UDP, and UDP is considered "unreliable", but has less overhead, so it's good for streaming.
Edit: speex
What concerns the server, the request itself is not a bottleneck. I guess you have sufficient time to set up the connection, as it happens only in the beginning of the session. Therefore the protocol is not of much relevance.
But consider that HTTP is a stateless protocol and not suitable for audio streaming. There are a couple of real time streaming protocols you can choose from. All of them will work over TCP or UDP (e.g. use raw sockets), and there are plenty of implementations.
In your case, the bottleneck with latency is not the server but the network itself. The connection between an iOS device and a wireless access point (AP) eats up about 40ms if the AP is not misconfigured and connection is good. (ping your iPhone.) In total, you'd have a minimum of 80ms for the path iOS -> AP -> Server -> AP -> iOS. But it is difficult to keep that latency stable. (Typical latency of AirPlay on my local network is about 300ms.)
I think live music over iOS devices is not practicable today. Try skype between two iOS devices and look how close you can get to 50ms. I'd bet no one can do it significantly better, what concerns latency.
Update: New research result!
I have to revise my claims regarding the latency of wifi connections of the iDevice. Apparently when you first ping your device, latency will be bad. But if I ping again no later than 200ms after that, I see an average latency 2ms-3ms between AP and iDevice.
My interpretation is that if there is no communication between AP and iDevice for more than 200ms, the network adapter of the iDevice will go to a less responsive sleep mode, probably to save battery power.
So it seems, live music is within reach again... :-)
Update 2
The ping-interval required for keep alive of low latency apparently differs from device to device. The reported 200ms is for an 3rd gen. iPad. For my iPhone 4 it's more like 50ms.
While streaming audio you probably don't need to bother with this, as data is exchanged on a more frequent basis. In my own context, I have sparse communication between an iDevice and a server, but low latency is crucial. A keep alive therefore is the way to go.
Best, Peter