I am trying to implement voice interaction on iOS with SwiftUI and Siri. I am just trying to get a shortcut to work with a single string parameter. I am very new to the framework and have been stuck. Below is what I tried:
Started on the Scrumdinger project (from an Apple SwiftUI tutorial, which I have in a GitHub repo. I stopped at commit 90731 before any Siri updates)
Created a button to add a Siri shortcut based on NSUserActivity to create a new scrum, but without any parameters; this worked (d64133)
Afterwards, I tried a bunch of things that didn't work:
Created an intent extension target with a group to share
Created an intent with a parameter, naming it "ScrumIntent"
Created a shortcut for the intent
Requested authorization for Siri
Currently, my set up has all the above and I can create the shortcut, which appears on the shortcuts app. However, it gets stuck and I don't even know how to debug it. If I try voice, Siri complains something went wrong. If I try to click on the shortcut, it is also stuck. Below you can see an image of how it gets stuck, with a stop button almost mid complete.
I am trying to simply set a string in user defaults when the callback gets handled. No special checking on the handler.
Some points about the implementation:
The handler is only in the extension target, but I am not sure if that's correct. In SwiftUI we can use the modifier onContinueUserActivity for NSUserActivities, but I couldn't find a similar one for intents (I understood we need intents for parameters).
The "Add Siri shortcut" button simply defines the Intent and sets suggestedInvocationPhrase.
Is there any other detail or essential piece of code I should share here? I was not sure so I just shared the github links for now.
Related
We are allowed to build custom intent for Siri, by extending SiriKit.
Moreover, Apples present how to manage basic integrations and customizations to Siri Shortcuts and Shortcuts app
Link for shortcuts app
Nonetheless, some apps have their own shortcuts UI with a little customization; for instance with the button to allow access for API, how can we do that?
In the documentation presented by Apple, there is no such instruction for how is it possible to customize the UI inside the shortcuts app regarding the custom intent created by my app.
I tried to make custom failure for the Intent response but didn't find the option for API access. Like ↴
However nothing quite similar to the UI presented by Shortcuts App, while attempt to first run trello ↴
These actions are provided by the Shortcuts app itself rather than by Siri intents exposed by another app. Indeed, these actions don’t even need the Trello or Wunderlist apps installed.
This means that Shortcuts can present a different UI.
The UI that it will show for your app is defined by the parameters in your intents file.
You can’t create the same experience for your app, however the user can perform any required authorisation in your app itself and it is reasonable to expect that they have run your app before trying to set up shortcuts that use it.
I'm trying to create an INIntent to be set up with the new iOS Shortcuts app in iOS 12. I have read the documentation and watched the apple video about it.
I have managed to set everything up and the shortcuts work well, however I can’t seem to figure out how to make customizable options that the user can edit when setting up the shortcut. An example of this would be how the Calendar app’s shortcuts are set up:
Notice the options for Get, Add Filter, Sort by, etc..
I know I can add parameters to the intents in the Intents.intentdefeniton file, but those parameters needs to be set from the app itself before the interaction is donated to the iOS system via SiriKit.
Regardless of what I do the Intent just shows up empty without options.
Any help is appreciated!
I am sorry to bring you bad news, but there is no way to do that. The apps that appear there (like Trello, Overcast, Pocket, etc...) were already supported by Workflow.app before Shortcuts.app was released.
You can try to work around it by calling URL Schemes directly but it might not be the solution you are after.
I think you need to create an Intents UI Extension for this.
https://developer.apple.com/documentation/sirikit/creating_an_intents_ui_extension
My app has a custom intent and it suggests a shortcut to the user like this:
let intent = ReadIntent()
if let shortcut = INShortcut(intent: intent) {
INVoiceShortcutCenter.shared.setShortcutSuggestions([shortcut])
}
My app always launches to handle the shortcut, so I just handle it in application(_:continue:restorationHandler:). There's no Intent Extension.
This is very convenient and works like a charm… however, I can't figure out how to get Siri to respond when the user triggers a shortcut. All the information I find online explains how to do it when using an Intent Extension.
In the Soup Chef example, for instance, it says:
Soup Chef marks each shortcut types with Supports background
execution; this way, the system doesn’t need to open the app to handle
the intent. When marked this way, the system uses Soup Chef’s intent
app extension to handle the order.
And then it goes on to explain how to add a custom response in the app extension. But my app needs to launch anyway, so I don't mark that option and the intent isn't handled by the extension. Is there no way to trigger the response without it? It sounds like something really simple, and the response is already in the intent definition file, I just can't get Siri to say it. :(
As far as I can tell, you can only get Siri to respond if the intent is executed in the background. And that's only possible by marking the intent as 'Supports background execution' and using an intent extension.
I'm scratching my head how I'm supposed to test my branch.io integration on simulator.
For link generation, I'm using the Javascript/web SDK instead of the iOS SDK. When you click a button to 'view content in app' on my landing page, it will generate the link and follow it.
All this works just great, but when I open the jump page in the simulator, it never actually attempts to open the local app on the phone which has the same bundle identifier.
I would guess this might be because the current app store URL box is blank (because it doesn't exist yet)... but I am not sure how I'm supposed to test if it works if I can't get the deeplink to trigger it locally.
Thanks!
Not exactly ideal, but if you're just trying to pass in static params to your app through a Branch link and test your app's implementation of them in the simulator, you can just hardcode them in the Branch initSessionWithLaunchOptions:andRegisterDeepLinkHandler: block. This block is called every time the application becomes active again (i.e., if it's backgrounded) and on application start, even if it's used in application:didFinishLaunchingWithOptions:.
The Branch handler block passes in a params dictionary, which you can either override (they'll be nil or just contain a few keys, if the block wasn't called from a tapped link) or just create your own to pass into your app.
This obviously doesn't work for testing desktop web site redirects or App Store downloads (or for many other use cases for Branch), but if you just wanted to test redirecting into specific views for your app from a clicked link (or in my case, how a dynamically generated view would look on a 4S when I didn't have a 4S available to test on), it gets the job done.
#Tallboy, unfortunately a simulator is not fit for testing deep links for two reasons:
there is no App Store on the simulator, so you cannot see true redirect behavior
the simulator does not support Universal Links
You're absolutely right -- you can click the link then manually open the app. In this case, we use our "deferred deep linking" mechanisms to determine that link was previously clicked. That method is discussed in our documentation here.
If you have questions about any of this, please ask. You can also reach the Branch team via support#branch.io.
I did this by using the Reminders app, create a new list, add that url as new item in that list, then click/tap that link in that item. It should open your app. :)
I found a work around for this problem. Just use any online note taking app from Safari. I've used googlekeep to create a new and pasted Branch url, then click/tap that link from there.
Universal Links in iOS9 use the Handoff mechanism to look at a link in an app instead of the browser: Whenever a Universal Link is clicked, iOS activates the corresponding app and calls -application:continueUserActivity:restorationHandler with a user activity containing the Universal Link's URL (see Apple guide on Universal Links).
I'd like to trigger this mechanism manually, i.e. creating an NSUserActivity instance with a webpageURL property and dispatching it to be handled by the current app's -application:continueUserActivity:restorationHandler. Is there a way to achieve this?
At the moment I'm just calling -application:continueUserActivity:restorationHandler directly, but this feels wrong and I'd like to implement it in a more proper way.