I tried using the official Apple example:
SpeekToMe
I edited the example in the following way to get the confidence levels:
if let result = result {
for t in result.transcriptions
{
for s in t.segments
{
print("POSSIBLE TRANSCRIPTION: \(s.substring) confidence: \(s.confidence)")
}
}
self.textView.text = result.bestTranscription.formattedString
isFinal = result.isFinal
}
Problem is the confidence levels are always = 0.
I found similar questions but setting the defaultTaskHint to dictation (or anything else) didn't help.
Does anyone have any suggestions on how to get the proper confidence values?
Related
I've been trying for a few years to find out how to get the USD / USDZ metersPerUnit value in an iOS / macCatalyst app but so far have not discovered any solution. The issue has become more important as our users utilize more of their own USDZ models to create multi-model 3D and AR scenes in our app.
From Apple's SceneKit documentation I would expect that SCNSceneSource.getProperty(forKey: SCNSceneSourceAssetUnitKey) would provide the value, but I have not found the getProperty API to ever return any value other than nil for any type of model file including: .obj, .scn, .dae, .usdc, .usdz.
According to Pixar's USD spec, the default value for metersPerUnit if unspecified is 0.01 (i.e., centimeters) and some USDZ sources like Sketchfab seem to nearly always set it to 0.01. But Apple's apps like RealityConverter and usdconvert enable the user to set the value directly so we're seeing lots of models with other values.
I'd like to do something like the sample code below, but I cannot get it to work.Since SCNSource seems to use ModelIO I would have thought ModelIO would have an API for this, but I have not discovered it. Is there some API anyone can suggest to get the metersPerUnit value?
do {
var options: [SCNSceneSource.LoadingOption : Any] = [
.animationImportPolicy: SCNSceneSource.AnimationImportPolicy.doNotPlay
]
if let modelSource = SCNSceneSource(url: url) {
if let units = modelSource.property(forKey: SCNSceneSourceAssetUnitKey) as? [String : Any] {
if let metersPerUnit = units[SCNSceneSourceAssetUnitMeterKey] as? Float {
options[.convertUnitsToMeters] = NSNumber(value: metersPerUnit)
}
}
let scene = try modelSource.scene(options: options)
}
} catch {
throw NSError(domain: "OurApp", code: 0, userInfo: [NSLocalizedDescriptionKey: "Model \(url.lastPathComponent) cannot be loaded"])
}
Im looking for a fast way to compare two frames of video, and decide if a lot has changed between them. This will be used to decide if I should send a request to image recognition service over REST, so I don't want to keep sending them, until there might be some different results. Something similar is doing Vuforia SDK. Im starting with a Framebuffer from ARKit, and I have it scaled to 640:480 and converted to RGB888 vBuffer_image. It could compare just few points, but it needs to find out if difference is significant nicely.
I started by calculating difference between few points using vDSP functions, but this has a disadvantage - if I move camera even very slightly to left/right, then the same points have different portions of image, and the calculated difference is high, even if nothing really changed much.
I was thinking about using histograms, but I didn't test this approach yet.
What would be the best solution for this? It needs to be fast, it can compare just smaller version of image, etc.
I have tested another approach using VNFeaturePointObservation from Vision. This works a lot better, but Im afraid it might be more CPU demanding. I need to test this on some older devices. Anyway, this is a part of code that works nicely. If someone could suggest some better approach to test, please let know:
private var lastScanningImageFingerprint: VNFeaturePrintObservation?
// Returns true if these are different enough
private func compareScanningImages(current: VNFeaturePrintObservation, last: VNFeaturePrintObservation?) -> Bool {
guard let last = last else { return true }
var distance = Float(0)
try! last.computeDistance(&distance, to: current)
print(distance)
return distance > 10
}
// After scanning is done, subclass should prepare suggestedTargets array.
private func performScanningIfNeeded(_ sender: Timer) {
guard !scanningInProgress else { return } // Wait for previous scanning to finish
guard let vImageBuffer = deletate?.currentFrameScalledImage else { return }
guard let image = CGImage.create(from: vImageBuffer) else { return }
func featureprintObservationForImage(image: CGImage) -> VNFeaturePrintObservation? {
let requestHandler = VNImageRequestHandler(cgImage: image, options: [:])
let request = VNGenerateImageFeaturePrintRequest()
do {
try requestHandler.perform([request])
return request.results?.first as? VNFeaturePrintObservation
} catch {
print("Vision error: \(error)")
return nil
}
}
guard let imageFingerprint = featureprintObservationForImage(image: image) else { return }
guard compareScanningImages(current: imageFingerprint, last: lastScanningImageFingerprint) else { return }
print("SCANN \(Date())")
lastScanningImageFingerprint = featureprintObservationForImage(image: image)
executeScanning(on: image) { [weak self] in
self?.scanningInProgress = false
}
}
Tested on older iPhone - as expected this causes some frame drops on camera preview. So I need a faster algorithm
I need to identify if the device is rebooted.
Currently saving time in the database, and periodically check time interval from last boot using the following code as suggested in Apple forums:
func bootTime() -> Date? {
var tv = timeval()
var tvSize = MemoryLayout<timeval>.size
let err = sysctlbyname("kern.boottime", &tv, &tvSize, nil, 0);
guard err == 0, tvSize == MemoryLayout<timeval>.size else {
return nil
}
return Date(timeIntervalSince1970: Double(tv.tv_sec) + Double(tv.tv_usec) / 1_000_000.0)
}
But the problem with this is even without a reboot, tv.tv_sec value differs around 30 (it varies from 0 seconds to 30 secs).
Anybody have any idea about this variation? or any other better way to identify device reboot without using sysctl or other reliable sysctl.
https://developer.apple.com/forums/thread/101874?answerId=309633022#309633022
Any pointer is highly appreciated.
I searched over SO, all the answer points to the solution mentioned here. Which have the issue as I mentioned. Please don't mark duplicate.
When using the Speech framework, I am consistently noticing zero confidence values for certain locales (e.g. "vi-VN", "pt-PT", ...), while non-zero, accurate confidence values are returned for other locales (e.g. "ko-KR", "ja-JP", ...).
Looking at the documentation, the confidence would be zero if there was no recognition. However, when the zero confidence occurs, the formattedString of the bestTranscription is populated and accurate (same for each segment substring text).
I have tried instantiating the locales in various ways (language code only, language and region code, -/_ formatting, grabbing an instance directly off of the SFSpeechRecognizer.supportedLocales() array). I have also tried setting the defaultTaskHint of SFSpeechRecognizer and taskHint of SFSpeechRecognitionRequest to dictation.
I am stuck at this point. Any help would be appreciated. Thanks in advance :)
guard let locale = Locale(identifier: "vi-VN"),
let recognizer = SFSpeechRecognizer(locale: locale),
recognizer.isAvailable else {
return
}
recognizer.defaultTaskHint = .dictation
let request = SFSpeechURLRecognitionRequest(url: ...)
request.contextualStrings = ...
request.shouldReportPartialResults = true
request.taskHint = .dictation
recognizer.recognitionTask(with: request) { (result, error) in
...
if (result.isFinal) {
let transcription = result.bestTranscription
/// transcription.formattedString is correct
/// all segments confidence values are 0, but with the properly recognized substring text.
}
...
}
Is there a way to calculate the ETA for a Route and not just for a direction? The problem with the directions is that it only shows one result for the fastest possible route.
Right now i can only find this:
directions.calculateETA { (etaResponse, error) -> Void in
if let error = error {
print("Error while requesting ETA : \(error.localizedDescription)")
//travelTime = "Not Available"
}else{
print("No error requesting ETA")
print("\(Int((etaResponse?.expectedTravelTime)!/60)) min")
}
}
At the moment, MapKit has no method for calculating an ETA just for a single route. It wouldn't really make sense anyways to call an ETA for a single MKRoute, since you can only get an MKRoute instance as the result of an MKDirections.calculate(completionHandler:) call, at which point you already have the travel times for the individual routes. The separate MKDirections.calculateETA function exists in order to give a quick method for just getting the ETA, since as the documentation states, this method yields results way faster than the calculate function.
You can access the travel time for all MKRoutes like this
directions.calculate { response, error in
guard error == nil, let response = response else {return}
for route in response.routes {
let eta = route.expectedTravelTime
}
}
Bear in mind that MKETAResponse.expectedTravelTime incorporates traffic conditions into its calculations, while MKRoute.expectedTravelTime uses ideal conditions, but at the moment, MapKit doesn't have a method for calculating ETA for a single MKRoute object with traffic.
MKDirections has a calculate(completionHandler:) function which returns multiple routes in the MKDirectionsResponse object that is returned:
directions.calculate { response, error in
var routes = response?.routes
let selectedRoute = routes[0]
let distance = selectedRoute.distance
let eta = selectedRoute.expectedTravelTime
// …
}
routes is an array of MKRoute objects which you can then pick out the ones you want.