I'm creating a flutter plugin to make WebRTC calls using the Twilio API. On the iOS side I use CXProvider and CallKit to make/receive calls. My problem is, the native call screen UI is always launched in the background and my Flutter app stay on the front.
Here a demo video :
I really don't understand this behavior.
This is how I display the incoming notification
func reportIncomingCall(from: String, uuid: UUID) {
let callHandle = CXHandle(type: .generic, value: from)
let callUpdate = CXCallUpdate()
callUpdate.remoteHandle = callHandle
callUpdate.localizedCallerName = from
callUpdate.supportsDTMF = true
callUpdate.supportsHolding = true
callUpdate.supportsGrouping = false
callUpdate.supportsUngrouping = false
callUpdate.hasVideo = false
// this display the callInvite UI
self.callKitProvider.reportNewIncomingCall(with: uuid, update: callUpdate) { error in
if let error = error {
print("error", error as Any)
}
}
}
This is how I answer a call from the native side
func performAnswerVoiceCall(uuid: UUID, completionHandler: #escaping (Bool) -> Swift.Void) {
if let ci = self.twilioVoiceDelegate!.callInvite {
let acceptOptions: AcceptOptions = AcceptOptions(callInvite: ci) { (builder) in
builder.uuid = ci.uuid
}
let theCall = ci.accept(options: acceptOptions, delegate: self.twilioVoiceDelegate!)
self.twilioVoiceDelegate!.call = theCall
self.twilioVoiceDelegate!.callCompletionCallback = completionHandler
self.twilioVoiceDelegate!.callInvite = nil
}
}
If anyone has a suggestion, It will be a pleasure
That's is how CallKIt works. Try to receive a call using WhatsApp on iOS. You get the same behavior
Related
Since iOS 14 released startVPNTunnel() stopped working and the app doesn't create VPN connection.
My configuration is the following:
private func _save(_ account: VPNServer, completion: VPNConfigureCompletion?) {
let keychain = Keychain(service: Bundle.main.bundleIdentifier!)
keychain["sharedSecret"] = account.key
let ikev2Protocol = NEVPNProtocolIKEv2()
ikev2Protocol.useExtendedAuthentication = false
ikev2Protocol.authenticationMethod = .sharedSecret
ikev2Protocol.disconnectOnSleep = false
ikev2Protocol.serverAddress = account.ip
ikev2Protocol.sharedSecretReference = keychain[attributes: "sharedSecret"]?.persistentRef
ikev2Protocol.remoteIdentifier = account.ip
if #available(iOS 13.0, *) {
ikev2Protocol.enableFallback = true
}
vpnManager.protocolConfiguration = ikev2Protocol
vpnManager.isEnabled = true
vpnManager.saveToPreferences { error in
if let err = error {
print("Failed to save profile: \(err.localizedDescription)")
self.delegate?.setConnectionStatus(status: .disconnected)
NotificationCenter.default.post(name: NSNotification.Name.NEVPNStatusDidChange, object: "Failed to save profile")
} else {
completion?()
}
}
}
This code saves configuration, but when you try to connect, it throws (even manually in iPhone settings). It worked correctly on iOS 13. We have tried to make server to IPv6 compatible, but without success. I also tried different to add VPN.IKEv2.ChildSecurityAssociationParameters without success.
Anyone has solution?
I'm migrating iOS app to the Twilio Voice 5.x API (iOS 13 & Xcode 11) and have some questions:
My app uses incoming calls. Twilio requires: 3. Report the call to CallKit. Refer to this example for how to report the call to CallKit.
My question: Does it mean I can't use more custom incoming call view controller in foreground mode?
Right after getting call invite I send request to get info about caller and show it. I do it in background. It worked in previous versions but
doesn't work now because interruption and handler must work at the same thread and immediately. What do you advise?
You can update value on the CallKit inbound call screen by using reportCall like this:
extension TwilioInteractor: TVONotificationDelegate {
func callInviteReceived(_ callInvite: TVOCallInvite) {
let from = callInvite.from ?? "?"
let uuid = callInvite.uuid
displayIncomingCall(from: from, uuid: uuid)
fetcher.fetchCaller(for: from, callInvite: callInvite) { caller in
self.updateIncomingCall(from: caller, uuid: uuid)
}
}
private func displayIncomingCall(from: String, uuid: UUID) {
let callHandle = CXHandle(type: .generic, value: from)
let callUpdate = CXCallUpdate()
callUpdate.remoteHandle = callHandle
callUpdate.hasVideo = false
callUpdate.supportsDTMF = true
callUpdate.supportsHolding = false
callUpdate.supportsGrouping = false
callUpdate.supportsUngrouping = false
self.callKitProvider.reportNewIncomingCall(with: uuid, update: callUpdate) { (error) in
if let error = error {
Log.e("Failed to report incoming call successfully: \(error.localizedDescription).")
} else {
Log.twilio("Incoming call successfully reported.")
}
}
}
private func updateIncomingCall(from: String, uuid: UUID) {
let callHandle = CXHandle(type: .generic, value: from)
let callUpdate = CXCallUpdate()
callUpdate.remoteHandle = callHandle
self.callKitProvider.reportCall(with: uuid, updated: callUpdate)
}
}
When using Voice Over in iOS, calling UIAccessibility.post(notification:argument:) to announce a field error doesn't actually announce the error.
I have a submit button and, when focusing the button, voice over reads the button title as you would expect. When pressing the button, voice over reads the title again. When the submit button is pressed, I am doing some validation and, when there is a field error, I am trying to announce it by calling:
if UIAccessibility.isVoiceOverRunning {
UIAccessibility.post(notification: .announcement, argument: "my field error")
}
Interestingly enough, if I stop on a breakpoint in the debugger the announcement happens. When I don't stop on a breakpoint, the announcement doesn't happen.
The notification is posting on the main thread and, if is like NotificationCenter.default, I assume that it is handled on the same thread it was posted on. I have tried to dispatch the call to the main queue, even though it is already on the main thread, and that doesn't seem to work either.
The only thing that I can think is that the notification is posted and observed before voice over is finished reading the submit button title and the announcement notification won't interrupt the current voice over.
I would really appreciate any help on this.
This is an admittedly hacky solution, but I was able to prevent the system announcement from pre-empting my own by dispatching to the main thread with a slight delay:
DispatchQueue.main.asyncAfter(deadline: .now() + 0.1) {
UIAccessibility.post(notification: .announcement, argument: "<Your text>")
}
Another work around is to use .screenChanged instead and pass the error label, as:
UIAccessibility.post(notification: .screenChanged, argument: errorLabel)
Your problem may happen because the system needs to take over during the field error appears and, in this case, any customed VoiceOver notification is cancelled.🤯
I wrote an answer about problems with queueing multiple VoiceOver notifications that may help you to understand your current situation.🤓
Your notification works with a breakpoint because you're delaying it and the system works during this time : there's no overlap between your notification and the system work.
A simple solution may be to implement a short delay before sending your notification but the delay depends on the speech rate that's why this is only a temporary workaround. 🙄
Your retry mechanism is smart and could be improved inside a loop of few retries in case of many system takeovers. 👍
I was facing the same issue, so I picked up #brandenesmith idea of the notification queue and wrote a little helper class.
class AccessibilityAnnouncementQueue {
static let shard = AccessibilityAnnouncementQueue()
private var queue: [String] = []
private init() {
NotificationCenter.default.addObserver(self,
selector: #selector(announcementFinished(_:)),
name: UIAccessibility.announcementDidFinishNotification,
object: nil)
}
func post(announcement: String) {
guard UIAccessibility.isVoiceOverRunning else { return }
queue.append(announcement)
postNotification(announcement)
}
private func postNotification(_ message: String) {
let attrMessage: NSAttributedString = NSAttributedString(string: message, attributes: [.accessibilitySpeechQueueAnnouncement: true])
UIAccessibility.post(notification: .announcement, argument: attrMessage)
}
#objc private func announcementFinished(_ sender: Notification) {
guard
let userInfo = sender.userInfo,
let firstQueueItem = queue.first,
let announcement = userInfo[UIAccessibility.announcementStringValueUserInfoKey] as? String,
let success = userInfo[UIAccessibility.announcementWasSuccessfulUserInfoKey] as? Bool,
firstQueueItem == announcement
else { return }
if success {
queue.removeFirst()
} else {
postNotification(firstQueueItem)
}
}
}
I am able to get this to work using a retry mechanism where I register as an observer of the UIAccessibility.announcementDidFinishNotification and then pull the announcement and success status out of the userInfo dictionary.
If the success status is false and the announcement is the same as the one I just sent, I post the notification again. This happens on repeat until the announcement was successful.
There are obviously multiple problems with this approach including having to de-register, what happens if another object manages to post the same announcement (this shouldn't ever happen in practice but in theory it could), having to keep track of the last announcement sent, etc.
The code would look like:
private var _errors: [String] = []
private var _lastAnnouncement: String = ""
init() {
NotificationCenter.default.addObserver(
self,
selector: #selector(announcementFinished(_:)),
name: UIAccessibility.announcementDidFinishNotification,
object: nil
)
}
func showErrors() {
if !_errors.isEmpty {
view.errorLabel.text = _errors.first!
view.errorLabel.isHidden = false
if UIAccessibility.isVoiceOverRunning {
_lastAnnouncement = _errors.first!
UIAccessibility.post(notification: .announcement, argument: _errors.first!)
}
} else {
view.errorLabel.text = ""
view.errorLabel.isHidden = true
}
}
#objc func announcementFinished(_ sender: Notification) {
guard let announcement = sender.userInfo![UIAccessibility.announcementStringValueUserInfoKey] as? String else { return }
guard let success = sender.userInfo![UIAccessibility.announcementWasSuccessfulUserInfoKey] as? Bool else { return }
if !success && announcement == _lastAnnouncement {
_lastAnnouncement = _errors.first!
UIAccessibility.post(notification: .announcement, argument: _errors.first!)
}
}
The problem is that this retry mechanism will always be used because the first call to UIAccessibility.post(notification: .announcement, argument: _errors.first!) always (unless I am stopped on a breakpoint). I still don't know why the first post always fails.
If somebody uses RxSwift, probably following solution will be more suitable:
extension UIAccessibility {
static func announce(_ message: String) -> Completable {
guard !message.isEmpty else { return .empty() }
return Completable.create { subscriber in
let postAnnouncement = {
DispatchQueue.main.async {
UIAccessibility.post(notification: .announcement, argument: message)
}
}
postAnnouncement()
let observable = NotificationCenter.default.rx.notification(UIAccessibility.announcementDidFinishNotification)
return observable.subscribe(onNext: { notification in
guard let userInfo = notification.userInfo,
let announcement = userInfo[UIAccessibility.announcementStringValueUserInfoKey] as? String,
announcement == message,
let success = userInfo[UIAccessibility.announcementWasSuccessfulUserInfoKey] as? Bool else { return }
success ? subscriber(.completed) : postAnnouncement()
})
}
}
}
I'm having a little confusion in regards to showing my UI when opening the app from an incoming background video call. I am successfully causing iOS to summon the default "incoming video call" interface when the app is in the background, but after the call is answered, my app isn't being woken up properly?
When I receive the call push, I setup a CXProvider and notify CallKit of an incoming call:
func handleIncomingCallFromBackground() {
//These properties are parsed from the push payload before this method is triggered
guard let callingUser = callingUser, roomName != nil else {
print("Unexpected nil caller and roomname (background)")
return
}
let callHandleTitle = "\(callingUser.first_name) \(callingUser.surname)"
let configuration = CXProviderConfiguration.default
let callKitProvider = CXProvider(configuration: configuration)
callKitProvider.setDelegate(self, queue: nil)
let callHandle = CXHandle(type: .generic, value: callHandleTitle)
self.callHandle = callHandle
let callUpdate = CXCallUpdate.default
callUpdate.remoteHandle = callHandle
let callUUID = UUID()
self.callUUID = callUUID
callKitProvider.reportNewIncomingCall(with: callUUID, update: callUpdate) { error in
if error != nil {
self.resetTwilioObjects()
}
}
}
I respond to the answer call delegate method for the CXProvider, in which I get an access token for the video call from the server, send a response to the server to alert the caller that we've accepted the call, and perform a segue to our own video call controller (that's all jobsVC.showVideoCallVC() does) which handles connecting the call through a Twilio room etc. Code below.
func provider(_ provider: CXProvider, perform action: CXAnswerCallAction) {
guard let customer = callingUser, let callHandle = self.callHandle, let uuid = callUUID, let roomName = roomName else {
resetTwilioObjects()
return
}
self.twilioAPI = TwilioAPI()
self.twilioAPI!.accessToken(room: roomName) { (success, accessToken) in
switch success{
case true:
guard let token = accessToken else {
return
}
let customerID = customer.userID
DispatchQueue.global().asyncAfter(deadline: .now() , execute: {
self.twilioAPI!.postResponse(customerID: customerID, status: 1) { (success) in
if let success = success, !success {
self.resetTwilioObjects()
}
}
})
guard let jobsVC = P_ChildHomeJobsVC.instance else {
print("Jobs VC unexpectedly nil")
return
}
//All this method does is perform a segue, the parameters are stored for later.
jobsVC.showVideoCallVC(callingUser: customer, callHandle: callHandle, callKitProvider: provider, callUUID: uuid, twilioAccessToken: token, roomName: roomName)
action.fulfill()
default:
action.fail()
self.resetTwilioObjects()
}
}
}
Depending on whether or not the device is locked, I get differing behaviour:
If the device is locked, upon hitting my app icon to open the app, I get the latest app screenshot show up with a green bar at the top instead of the UI.
If the device is unlocked, upon hitting my app icon to open the app nothing happens at all - it stays on the iOS interface.
According to my logs, the segue is actually being performed correctly and the inner workings of the video call controller are even being fired, but shortly after I get the applicationWillResignActive: delegate call and everything stops.
What's odd (or maybe not) is that if the device is locked whilst the app is still in the foreground, everything works as expected: the app is correctly woken up and the updated UI is shown. I noticed that I still get the applicationWillResignActive: call, but immediately get applicationDidBecomeActive: after that.
Does anyone have any suggestions or hints as to what I might be doing wrong?
I'm trying to enable video calling on my swift app using Linphone.
I was able to enable audio calling, but I can't make it working with video. The app is always crashing if I enable this line:
linphone_call_params_enable_video(linCallParams, 1)
I want to only receive video and audio here.
#objc func startVideoCall() {
linphone_core_enable_video_display(theLinphone.lc, 1)
linphone_core_enable_video_capture(theLinphone.lc, 1)
let linCallParams = linphone_core_create_call_params(theLinphone.lc, nil)
linphone_call_params_enable_video(linCallParams, 1)
linphone_call_params_set_video_direction(linCallParams, LinphoneMediaDirectionSendRecv)
linphone_call_params_set_audio_direction(linCallParams, LinphoneMediaDirectionSendRecv)
let call = linphone_core_invite_with_params(theLinphone.lc, calleeAccount, linCallParams)
linphone_core_set_native_video_window_id(theLinphone.lc, &videoStreamView)
linphone_core_set_native_preview_window_id(theLinphone.lc, &videoStreamPreview)
do {
try audioSession.setActive(true)
} catch {
print("Audio error: \(error.localizedDescription)")
}
linphone_call_params_unref(linCallParams)
}
This code combo fixed my issue
private func bridge<T: AnyObject>(obj : T) -> UnsafeRawPointer {
let pointer = Unmanaged.passUnretained(obj).toOpaque()
return UnsafeRawPointer(pointer)
}
let viewPointer = UnsafeMutableRawPointer(mutating: bridge(obj: view))
linphone_core_set_native_video_window_id(theLinphone.lc, viewPointer)
let previewPointer = UnsafeMutableRawPointer(mutating: bridge(obj: previewStream))
linphone_core_set_native_preview_window_id(theLinphone.lc, previewPointer)